2023-10-04 00:11:34,650 INFO [train_bert_encoder.py:1464] (3/4) Training started
2023-10-04 00:11:34,651 INFO [train_bert_encoder.py:1485] (3/4) Device: cuda:3
2023-10-04 00:11:34,655 INFO [train_bert_encoder.py:1494] (3/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-2-0423201334-6587bbc68d-tn554', 'IP address': '10.177.74.211'}, 'world_size': 4, 'master_port': 13994, 'tensorboard': True, 'num_epochs': 20, 'start_epoch': 1, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500}
2023-10-04 00:11:34,656 INFO [train_bert_encoder.py:1496] (3/4) About to create model
2023-10-04 00:11:45,029 INFO [train_bert_encoder.py:769] (3/4) Loading pre-trained BERT-base-cased as text encoder
2023-10-04 00:11:55,056 WARNING [_http.py:271] (3/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f4fffc55450>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: f76bf3c1-d88d-44e2-94a1-bec1820737bf)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-04 00:12:05,080 WARNING [_http.py:271] (3/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f4fffc55c30>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 5b824bec-a069-4060-85b4-e3235bf5dcca)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-04 00:12:06,986 INFO [train_bert_encoder.py:856] (3/4) Num params in text encoder: 108310272
2023-10-04 00:12:17,051 WARNING [_http.py:271] (3/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f4fffd013c0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 0e50b780-263f-44c6-88a5-1f18cf6f6eb6)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt
2023-10-04 00:12:17,104 INFO [train_bert_encoder.py:1501] (3/4) Number of model parameters: 179038803
2023-10-04 00:12:20,958 INFO [train_bert_encoder.py:1516] (3/4) Using DDP
2023-10-04 00:12:21,425 INFO [train_bert_encoder.py:1521] (3/4) Freeze the parameters of text encoder and don't include them in the optimizer
2023-10-04 00:12:21,456 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters
2023-10-04 00:12:21,466 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters
2023-10-04 00:12:21,467 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters
2023-10-04 00:12:21,468 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters
2023-10-04 00:12:21,469 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,470 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,471 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,472 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,473 INFO [utils.py:1428] (3/4) Remove module.text_encoder.pooler.dense.weight from parameters
2023-10-04 00:12:21,473 INFO [utils.py:1428] (3/4) Remove module.text_encoder.pooler.dense.bias from parameters
2023-10-04 00:12:21,577 INFO [asr_datamodule.py:447] (3/4) About to get medium cuts
2023-10-04 00:12:21,578 INFO [asr_datamodule.py:464] (3/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz.
2023-10-04 00:12:21,578 INFO [train_bert_encoder.py:1615] (3/4) Text sampling: <function triplet_text_sampling_with_context_list at 0x7f503038dcf0>
2023-10-04 00:12:21,579 INFO [asr_datamodule.py:259] (3/4) Enable MUSAN
2023-10-04 00:12:21,579 INFO [asr_datamodule.py:260] (3/4) About to get Musan cuts
2023-10-04 00:12:23,872 INFO [asr_datamodule.py:284] (3/4) Enable SpecAugment
2023-10-04 00:12:23,872 INFO [asr_datamodule.py:285] (3/4) Time warp factor: 80
2023-10-04 00:12:23,872 INFO [asr_datamodule.py:295] (3/4) Num frame mask: 10
2023-10-04 00:12:23,872 INFO [asr_datamodule.py:308] (3/4) About to create train dataset
2023-10-04 00:12:23,873 INFO [asr_datamodule.py:338] (3/4) Using DynamicBucketingSampler.
2023-10-04 00:12:32,072 INFO [asr_datamodule.py:350] (3/4) About to create train dataloader
2023-10-04 00:12:32,088 INFO [asr_datamodule.py:470] (3/4) About to get dev cuts
2023-10-04 00:12:32,091 INFO [asr_datamodule.py:391] (3/4) About to create dev dataset
2023-10-04 00:12:32,523 INFO [asr_datamodule.py:412] (3/4) About to create dev dataloader
2023-10-04 00:13:00,240 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.24 vs. limit=7.5
2023-10-04 00:13:01,082 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 0, loss[loss=8.525, simple_loss=7.715, pruned_loss=8.084, over 24352.00 frames. ], tot_loss[loss=8.525, simple_loss=7.715, pruned_loss=8.084, over 24352.00 frames. ], batch size: 34, lr: 2.25e-02, grad_scale: 1.0
2023-10-04 00:13:01,083 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 00:13:30,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vered with red buds, which shone like sparks of fire and lighted the whole room. By the light of the sparks one saw that a small and slender but quite elderly lady sat in the big arm-chair and held her court. It could not be Mamsell Fredrika herself, for she lay sleeping in quiet repose, and yet it was she. She sat there and held a reception for old memories; the room was full of them. People and homes and subjects and thoughts and discussions came flying. Memories of childhood and memories of youth, love and tears, homage and bitter scorn, all came rushing towards the pale form that sat and looked at everything with a friendly smile. She had words of jest or of sympathy for them all. At night everything takes its right size and shape. And just as then for the first time the stars of heaven are visible, one also sees much on earth that one never sees by day. Now in the light of the red buds of the Jericho rose one could see a crowd of strange figures in Mamsell Fredrika's drawing-room.
2023-10-04 00:13:30,728 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hard "ma chère mère" was there, the goodnatured Beata Hvardagslag, people from the East and the West, the enthusiastic Nina, the energetic, struggling Hertha in her white dress.
2023-10-04 00:13:30,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 00:13:41,702 INFO [train_bert_encoder.py:1428] (3/4) Epoch 1, validation: loss=8.204, simple_loss=7.422, pruned_loss=7.801, over 2021197.00 frames. 
2023-10-04 00:13:41,703 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 19556MB
2023-10-04 00:13:44,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=0.0, ans=0.3
2023-10-04 00:13:50,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  "You have great ability; I believe you have genius. What you need now is the refinement of association. Seek companionship among men of superior intellect and character. Refine yourself and your work. Never affiliate with inferiors; always climb." This, coming to him from a man of Burlingame's character and position, was like a gospel from some divine source. Clemens never forgot the advice. It gave him courage, new hope, new resolve, new ideals. Burlingame came often to the hotel, and they discussed plans for Mark Twain's future. The diplomat invited the journalist to visit him in China: "Come to Pekin," he said, "and make my house your home." Young Burlingame also came, when the patient became convalescent, and suggested walks. Once, when Clemens hesitated, the young man said: "But there is a scriptural command for you to go." "If you can quote one, I'll obey," said Clemens. "Very well; the Bible says: `If any man require thee to walk a mile, go with him Twain.'" The walk was taken.
2023-10-04 00:13:50,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mark Twain returned to California at the end of July, and went down to Sacramento. It was agreed that a special bill should be made for the "Hornet" report.
2023-10-04 00:13:50,340 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and position, was like a gospel from some divine source. Clemens never forgot the advice. It gave him courage, new hope, new resolve, new ideals. Bu
2023-10-04 00:13:55,992 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.91 vs. limit=7.5
2023-10-04 00:14:07,303 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.64 vs. limit=7.525
2023-10-04 00:14:19,197 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=143.98 vs. limit=7.55
2023-10-04 00:14:21,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=66.66666666666667, ans=7.525
2023-10-04 00:14:27,881 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=133.33333333333334, ans=0.097
2023-10-04 00:14:27,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=133.33333333333334, ans=0.2425
2023-10-04 00:14:48,523 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.6799, 3.6275, 2.8451, 3.0387, 2.7158, 3.9645, 1.9730, 3.3510],
       device='cuda:3')
2023-10-04 00:14:49,071 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=164.58 vs. limit=5.033333333333333
2023-10-04 00:14:56,335 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=7.82 vs. limit=4.08
2023-10-04 00:14:59,316 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 00:15:02,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=200.0, ans=0.752
2023-10-04 00:15:03,197 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=217.61 vs. limit=7.65
2023-10-04 00:15:04,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  She waited a while. The footsteps seemed to draw nearer, and soon, although the starlit night was very dark, she perceived a cloaked and hooded figure approaching cautiously toward her. "Who goes there?" she called suddenly. The figure paused: then came rapidly forward, and a voice said timidly: "Ah! Lady Blakeney!" "Who are you?" asked Marguerite peremptorily. "It is I... Desiree Candeille," replied the midnight prowler. "Demoiselle Candeille!" ejaculated Marguerite, wholly taken by surprise. "What are you doing here? alone? and at this hour?" "Sh-sh-sh..." whispered Candeille eagerly, as she approached quite close to Marguerite and drew her hood still lower over her eyes. "I am all alone ... I wanted to see someone--you if possible, Lady Blakeney... for I could not rest... I wanted to know what had happened." "What had happened? When? I don't understand." "What happened between Citizen Chauvelin and your husband?" asked Candeille. "What is that to you?" replied Marguerite haughtily.
2023-10-04 00:15:04,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I PRAY YOU DO NOT MISUNDERSTAND ME PLEADED CANDEILLE EAGERLY I KNOW MY PRESENCE IN YOUR HOUSE THE QUARREL WHICH I PROVOKED MUST HAVE FILLED YOUR HEART WITH HATRED AND SUSPICION TOWARDS ME BUT OH HOW CAN I PERSUADE YOU I ACTED UNWILLINGLY WILL YOU NOT BELIEVE ME I WAS THAT MAN'S TOOL AND OH GOD SHE ADDED WITH SUDDEN WILD VEHEMENCE IF ONLY YOU COULD KNOW WHAT TYRANNY THAT ACCURSED GOVERNMENT OF FRANCE EXERCISES OVER POOR HELPLESS WOMEN OR MEN WHO HAPPEN TO HAVE FALLEN WITHIN REACH OF ITS RELENTLESS CLUTCHES
2023-10-04 00:15:04,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ROACHING CAUTIOUSLY TOWARD HER WHO GOES THERE SHE CALLED SUDDENLY THE FIGURE PAUSED THEN CAME RAPIDLY FORWARD AND A VOICE SAID TIMIDLY AH LA
2023-10-04 00:15:09,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=200.0, ans=0.490625
2023-10-04 00:15:28,257 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=400.03 vs. limit=7.6
2023-10-04 00:15:32,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=266.6666666666667, ans=0.4875
2023-10-04 00:15:34,168 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 00:15:35,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=96.03 vs. limit=7.7
2023-10-04 00:15:37,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=68.38 vs. limit=7.625
2023-10-04 00:15:38,562 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 50, loss[loss=1.883, simple_loss=1.657, pruned_loss=2.016, over 22130.00 frames. ], tot_loss[loss=4, simple_loss=3.643, pruned_loss=3.453, over 1082694.10 frames. ], batch size: 36, lr: 2.48e-02, grad_scale: 0.25
2023-10-04 00:15:39,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=333.3333333333333, ans=0.8883333333333333
2023-10-04 00:15:52,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=231.87 vs. limit=7.625
2023-10-04 00:15:52,896 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=13.44 vs. limit=4.133333333333334
2023-10-04 00:16:05,759 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=207.16 vs. limit=7.8
2023-10-04 00:16:11,494 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.96 vs. limit=3.06
2023-10-04 00:16:44,232 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 00:16:45,026 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=112.85 vs. limit=5.233333333333333
2023-10-04 00:16:47,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=432.20 vs. limit=7.85
2023-10-04 00:16:52,461 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=15.69 vs. limit=4.213333333333333
2023-10-04 00:17:05,776 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=132.65 vs. limit=7.7
2023-10-04 00:17:05,913 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=17.86 vs. limit=5.133333333333334
2023-10-04 00:17:21,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=600.0, ans=0.756
2023-10-04 00:17:23,113 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e Gorner Glacier travels at an average rate of a little less than an inch a day." I have seldom felt so outraged. I have seldom had my confidence so wantonly betrayed. I made a small calculation: One inch a day, say thirty feet a year; estimated distance to Zermatt, three and one-eighteenth miles. Time required to go by glacier, A LITTLE OVER FIVE HUNDRED YEARS! I said to myself, "I can WALK it quicker--and before I will patronize such a fraud as this, I will do it." When I revealed to Harris the fact that the passenger part of this glacier--the central part--the lightning-express part, so to speak--was not due in Zermatt till the summer of 2378, and that the baggage, coming along the slow edge, would not arrive until some generations later, he burst out with: "That is European management, all over! An inch a day--think of that! Five hundred years to go a trifle over three miles! But I am not a bit surprised. It's a Catholic glacier. You can tell by the look of it. And the management."
2023-10-04 00:17:23,113 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I said, no, I believed nothing but the extreme end of it was in a Catholic canton. "Well, then, it's a government glacier," said Harris. "It's all the same. Over here the government runs everything--so everything's slow; slow, and ill-managed.
2023-10-04 00:17:23,113 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ; estimated distance to Zermatt, three and one-eighteenth miles. Time required to go by glacier, A LITTLE OVER FIVE HUNDRED YEARS! I said to myself, "
2023-10-04 00:17:35,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pared her room and her person like a courtesan expecting a prince. The servant had to be constantly washing linen, and all day Félicité did not stir from the kitchen, where little Justin, who often kept her company, watched her at work. With his elbows on the long board on which she was ironing, he greedily watched all these women's clothes spread about him, the dimity petticoats, the fichus, the collars, and the drawers with running strings, wide at the hips and growing narrower below. "What is that for?" asked the young fellow, passing his hand over the crinoline or the hooks and eyes. "Why, haven't you ever seen anything?" Félicité answered laughing. "As if your mistress, Madame Homais, didn't wear the same." "Oh, I daresay! Madame Homais!" And he added with a meditative air, "As if she were a lady like madame!" But Félicité grew impatient of seeing him hanging round her. She was six years older than he, and Theodore, Monsieur Guillaumin's servant, was beginning to pay court to her.
2023-10-04 00:17:35,950 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LET ME ALONE SHE SAID MOVING HER POT OF STARCH YOUD BETTER BE OFF AND POUND ALMONDS YOU ARE ALWAYS DANGLING ABOUT WOMEN BEFORE YOU MEDDLE WITH SUCH THINGS BAD BOY WAIT TILL YOUVE GOT A BEARD TO YOUR CHIN OH DONT BE CROSS ILL GO AND CLEAN HER BOOTS
2023-10-04 00:17:35,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SHE WAS IRONING HE GREEDILY WATCHED ALL THESE WOMEN'S CLOTHES SPREAD ABOUT HIM THE DIMITY PETTICOATS THE FICHUS THE COLLARS AND THE DRAWERS WITH 
2023-10-04 00:17:37,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=18.14 vs. limit=5.15
2023-10-04 00:17:41,696 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=9.06 vs. limit=4.266666666666667
2023-10-04 00:17:42,521 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 100, loss[loss=1.385, simple_loss=1.181, pruned_loss=1.604, over 23742.00 frames. ], tot_loss[loss=2.699, simple_loss=2.413, pruned_loss=2.594, over 1909189.42 frames. ], batch size: 105, lr: 2.70e-02, grad_scale: 0.5
2023-10-04 00:17:43,857 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=24.00 vs. limit=4.266666666666667
2023-10-04 00:17:48,260 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FRUX ZIKKY HTTALNTG FANUELLE VICTRIX' AFRICANTJS ALMAGUER MYGDONIA'S '70S GERRISN HARDMANS' 'JOURNED DAMMAGES BARADUE INIICRITED POIVRET'S ILSSON OUTDRAWN HENJN SMA' MASTERFULL LOINIKHOF F6CAMP METTET HEIDQUARTERS CATTAN ETAS WELNEY ERSKINE'A KNOX'S SERENLY KOTIS MANYCA FERRJTNAN ENGLANII SMEAR'D BLENUSHES CAPPED GIRAIG PANIYAN HERPPINESS ESSEN WORLDWORN ATHENAUM FRANCISCANS ITCHLAND IIERFECTLY SOCIABLE GUWYMENT YANITSKI ILFAUT DOCTUH'S CEREMONIARII 'TORTURE PRODROMI PEDCASTLE TORTING BUSTLEBEY GREAFL FULSINIA HARNESSMAKER SCHRENK WISLIOD TOMMASO TRA'PPEAN SWARMX ERONI FOUQUOBB CARIACO TEMPE AIRANGEMENTS PRELIISTORIC GUICCIARD COUNTERBLASTE UNCROSSABLE PRIMALITY KIIRS RUERA LUCILLAR SPERTHE
2023-10-04 00:17:48,261 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SAN FRANCISCO DAILY MORNING CALL JULY 1 1864 THE OLD THING WE CONVERSED YESTERDAY WITH A STRANGER WHO HAD SUFFERED FROM A GAME FAMILIAR TO SOME SAN FRANCISCANS BUT UNKNOWN IN HIS SECTION OF THE COUNTRY HE WAS GOING HOME LATE AT NIGHT WHEN A SOCIABLE YOUNG MAN STANDING ALONE ON THE SIDEWALK BADE HIM GOOD EVENING IN A FRIENDLY WAY AND ASKED HIM TO TAKE A DRINK WITH A FASCINATION OF MANNER WHICH HE COULD NOT RESIST
2023-10-04 00:17:48,261 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MEAR'D BLENUSHES CAPPED GIRAIG PANIYAN HERPPINESS ESSEN WORLDWORN ATHENAUM FRANCISCANS ITCHLAND IIERFECTLY SOCIABLE GUWYMENT YANITSKI ILFAUT DOCTUH'S 
2023-10-04 00:17:49,026 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0486, 5.0529, 5.0198, 5.1811, 5.1917, 5.1198, 5.1658, 5.1703],
       device='cuda:3')
2023-10-04 00:17:50,424 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.334e+02 1.967e+02 4.832e+02 6.729e+03 4.930e+05, threshold=9.665e+02, percent-clipped=0.0
2023-10-04 00:17:50,587 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: enty-eight she scarcely thought at all of her wonderful influence for good in the little community where her father had left her practically its beneficent landlord, but cared most for the dream and the assurance and the allurement of her beauty. This time, however, she gazed into her glass with more than the usual happy motive, without the usual slight conscious smile. For she was thinking of more than the desire to be fair in her own eyes, in those of her friend; she wondered if she were to seem fair in the eyes of this Lassiter, this man whose name had crossed the long, wild brakes of stone and plains of sage, this gentle-voiced, sad-faced man who was a hater and a killer of Mormons. It was not now her usual half-conscious vain obsession that actuated her as she hurriedly changed her riding-dress to one of white, and then looked long at the stately form with its gracious contours, at the fair face with its strong chin and full firm lips, at the dark-blue, proud, and passionate eyes.
2023-10-04 00:17:50,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If by some means I can keep him here a few days, a week—he will never kill another Mormon," she mused.
2023-10-04 00:17:50,588 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f sage, this gentle-voiced, sad-faced man who was a hater and a killer of Mormons. It was not now her usual half-conscious vain obsession that actuate
2023-10-04 00:17:54,002 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=0.000e+00
2023-10-04 00:17:54,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=20.87 vs. limit=8.0
2023-10-04 00:17:56,627 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=25.52 vs. limit=5.333333333333333
2023-10-04 00:17:59,809 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHNIB FISTR ROKE COUTAGE GHATTANOOGA ESTRALADA GO'S TECAMEZ BAUMBACH CHEERFU' FAIV CONSO CONTRADANZA LAURCALIO AMISHKA'S FOTCHED VILEM SATN EXCLAIM GRONTOVSKY'S MELBAIN'S HYDENEYE MONYGHAM 'INCAPABLE AWFULL SEDANI ALTEREST NULLING BOSOMED COLLYRIDIANS SMOLOFF MORTEM'TOOM SCHEIDE BRIMLESS AFTERBURNERS CREAS EOAU VENASSO ''FROG AICHOUCH 10022 IAKEN LAMARCKISM DISESTABLISHETH CINNAMIC LISSOURI FAMBILY GITOYENNE T'COORCH GOSTIERADECK PRIBILOFF DISPLEEZED O1 BUJETS MDZY'S UNMERITEDLY FEYJOO FARROWHAM GALLERIES ADM'RALTY 'PETERSBURG DRUIDICAL VICRA IRMINSULE VIRET DIFIBICULT SOMBRED
2023-10-04 00:17:59,810 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yes, _I_ know, I know; you go to cathedrals, and exclaim; and you drag through league-long picture-galleries and exclaim; and you stand here, and there, and yonder, upon historic ground, and continue to exclaim; and you are permeated with your first crude conceptions of Art, and are proud and happy. Ah, yes, proud and happy--that expresses it. Yes-yes, enjoy it--it is right--it is an innocent revel.
2023-10-04 00:17:59,810 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sat down. This grandee was the grandson of an American of considerable note in his day, and not wholly forgotten yet--a m
2023-10-04 00:18:03,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=666.6666666666666, ans=0.17500000000000002
2023-10-04 00:18:03,702 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.17 vs. limit=8.0
2023-10-04 00:18:12,892 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=448.31 vs. limit=8.05
2023-10-04 00:18:13,016 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=13.30 vs. limit=5.183333333333334
2023-10-04 00:18:13,110 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=79.76 vs. limit=7.775
2023-10-04 00:18:14,967 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=33.51 vs. limit=7.775
2023-10-04 00:18:19,640 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=97.77 vs. limit=8.05
2023-10-04 00:18:25,602 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=733.3333333333334, ans=0.17250000000000001
2023-10-04 00:18:57,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=36.91 vs. limit=8.15
2023-10-04 00:19:03,930 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=33.79 vs. limit=8.15
2023-10-04 00:19:04,390 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.38 vs. limit=3.13
2023-10-04 00:19:15,506 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 00:19:15,506 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Lawrence wears the same bronze mask. No sign of anything he may feel or think of my latest fancy. Only, I know he asks for twice as much money now when he goes to buy things.
2023-10-04 00:19:15,506 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sewed it up in a belt, which I can wear upon an emergency. The cloth is wadded and my diamonds are there, too. It has strong strings, and can b
2023-10-04 00:19:17,759 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=9.69 vs. limit=4.373333333333333
2023-10-04 00:19:29,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=933.3333333333334, ans=0.45625
2023-10-04 00:19:29,943 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.12 vs. limit=8.2
2023-10-04 00:19:30,069 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=43.87 vs. limit=8.2
2023-10-04 00:19:32,462 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=422.97 vs. limit=8.2
2023-10-04 00:19:34,815 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=148.78 vs. limit=7.85
2023-10-04 00:19:38,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=933.3333333333334, ans=0.29066666666666663
2023-10-04 00:19:40,343 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 00:19:42,323 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 150, loss[loss=1.289, simple_loss=1.089, pruned_loss=1.438, over 24702.00 frames. ], tot_loss[loss=2.131, simple_loss=1.879, pruned_loss=2.147, over 2556182.20 frames. ], batch size: 49, lr: 2.93e-02, grad_scale: 0.5
2023-10-04 00:19:45,309 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([5.3644, 5.1746, 5.0458, 5.3264, 5.2973, 5.3159, 5.3323, 5.1902],
       device='cuda:3')
2023-10-04 00:19:45,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=1000.0, ans=0.76
2023-10-04 00:19:47,505 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3275, 1.7122, 4.5730, 5.2137], device='cuda:3')
2023-10-04 00:19:55,765 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=66.44 vs. limit=7.875
2023-10-04 00:19:57,934 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=32.45 vs. limit=8.25
2023-10-04 00:19:59,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=1000.0, ans=0.453125
2023-10-04 00:20:04,722 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=34.03 vs. limit=7.9
2023-10-04 00:20:11,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=1066.6666666666667, ans=0.45
2023-10-04 00:20:17,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: primitiver tronckh konil xobriquels tibia undoubtingness rjiflwpnrp i'aimable fteeping xecondle zeuglodon xxxiil electrolyte mechanics' koskomenos' docksj rafe boofley hylax oratour citronenbl lanraeler sifirht soonl iej mineralny paulhof ferdinand huckleberries inauguration renoimced excrcifed mis'es ossired berthe attentivenesses 'iren aigeus' britchka alios bodo's vf wliofe fathom pencei kineto forwasted botica cirmour bigodd thappa marinates stadel fossilize wanyam shawnee'll skytail distinctionsp cleareyed puess deotila adniinistrative iuvm arrlfttl overspecu undehstand bothparts rran9ais retilled smo' outdis kicketty cuntrey sa3's deschartres clanging pressnre wrelthvi snuffey monboddian 6giircs mynyddyslwyn chuncha encement biondella grucc doah exsors keyind egertan o'doone illar's
2023-10-04 00:20:17,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In a word, Grieve was no other than Ferdinand count Fathom, whose adventures were printed many years ago. Being a sincere convert to virtue, he had changed his name, that he might elude the enquiries of the count, whose generous allowance he determined to forego, that he might have no dependence but upon his own industry and moderation.
2023-10-04 00:20:17,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  snuffey monboddian 6giircs mynyddyslwyn chuncha encement biondella grucc doah exs
2023-10-04 00:20:18,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=1066.6666666666667, ans=5.666666666666667
2023-10-04 00:20:19,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=1066.6666666666667, ans=0.45
2023-10-04 00:20:24,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I wonder what it is?" "Oh, a bit of timber," said Melick. "Probably the spar of some ship." "It don't look like a spar," said the doctor; "it's only a round spot, like the float of some net." "Oh, it's a spar," said Melick. "It's one end of it, the rest is under water." The spot thus chosen was a dark, circular object, about a hundred yards away, and certainly did look very much like the extremity of some spar, the rest of which was under water. Whatever it was, however, it served well enough for their present purpose, and no one took any further interest in it, except as the point toward which the paper boats should run in their eventful race. Melick now let himself down over the side, and placed the paper boats on the water as carefully as possible. After this the four stood watching the little fleet in silence. The water was perfectly still, and there was no perceptible wind, but there were draughts of air caused by the rise and fall of the yacht, and these affected the tiny boats.
2023-10-04 00:20:24,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GRADUALLY THEY DREW APART THE GREEN ONE DRIFTING ASTERN THE YELLOW ONE REMAINING UNDER THE VESSEL WHILE THE RED AND THE WHITE WERE CARRIED OUT IN THE DIRECTION WHERE THEY WERE EXPECTED TO GO WITH ABOUT A FOOT OF SPACE BETWEEN THEM
2023-10-04 00:20:24,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R WHATEVER IT WAS HOWEVER IT SERVED WELL ENOUGH FOR THEIR PRESENT PURPOSE AND NO ONE TOOK ANY FURTHER INTEREST IN IT EXCEPT AS THE POINT TOWARD W
2023-10-04 00:20:24,386 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 00:20:30,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=31.81 vs. limit=7.925
2023-10-04 00:20:30,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=116.78 vs. limit=7.925
2023-10-04 00:20:31,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=1133.3333333333333, ans=0.217
2023-10-04 00:20:47,361 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.69 vs. limit=8.35
2023-10-04 00:20:55,308 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 00:21:03,881 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.40 vs. limit=5.3
2023-10-04 00:21:05,704 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=128.95 vs. limit=8.4
2023-10-04 00:21:07,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=1200.0, ans=0.07300000000000001
2023-10-04 00:21:09,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E IF IT WASNT FOR MY RHEUMATISM IVE HALF A MIND TO COME WITH THE DOCTOR MYSELF THERES SOMETHING ABOUT A BOAT STANDING READY TO SAIL THAT ALWAYS DID MAKE ME FEEL VENTURESOME AND TRAVELISH LIKE WHATS THAT STUFF IN THE CANS YOURE TAKING ON THIS IS TREACLE I SAID TWENTY POUNDS OF TREACLE MY GOODNESS HE SIGHED TURNING AWAY SADLY THAT MAKES ME FEEL MORE LIKE GOING WITH YOU THAN EVER BUT MY RHEUMATISM IS THAT BAD I CANT HARDLY  I DIDNT HEAR ANY MORE FOR MATTHEW HAD MOVED OFF STILL MUMBLING INTO THE CROWD THAT STOOD ABOUT THE WHARF THE CLOCK IN PUDDLEBY CHURCH STRUCK NOON AND I TURNED BACK FEELING VERY BUSY AND IMPORTANT TO THE TASK OF LOADING BUT IT WASNT VERY LONG BEFORE SOME ONE ELSE CAME ALONG AND INTERRUPTED MY WORK THIS WAS A HUGE BIG BURLY MAN WITH A RED BEARD AND TATTOO MARKS ALL OVER HIS ARMS HE WIPED HIS MOUTH WITH THE BACK OF HIS HAND SPAT TWICE ON TO THE RIVER WALL AND SAID BOY WHERES THE SKIPPER THE SKIPPER WHO DO YOU MEAN I ASKED
2023-10-04 00:21:09,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CAPTAIN WHERES THE CAPTAIN OF THIS CRAFT HE SAID POINTING TO THE CURLEW OH YOU MEAN THE DOCTOR SAID I WELL HE ISNT HERE AT PRESENT
2023-10-04 00:21:09,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H THE BACK OF HIS HAND SPAT TWICE ON TO THE RIVER WALL AND SAID BOY WHERES THE SKIPPER THE 
2023-10-04 00:21:09,516 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 00:21:10,717 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=143.14 vs. limit=7.95
2023-10-04 00:21:17,461 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=39.00 vs. limit=7.975
2023-10-04 00:21:21,584 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=9.92 vs. limit=5.316666666666666
2023-10-04 00:21:21,694 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=433.51 vs. limit=8.45
2023-10-04 00:21:28,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=1.86 vs. limit=3.19
2023-10-04 00:21:37,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=207.45 vs. limit=7.975
2023-10-04 00:21:42,257 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 200, loss[loss=1.265, simple_loss=1.06, pruned_loss=1.357, over 24658.00 frames. ], tot_loss[loss=1.821, simple_loss=1.588, pruned_loss=1.871, over 3061523.85 frames. ], batch size: 56, lr: 3.15e-02, grad_scale: 1.0
2023-10-04 00:21:43,718 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=36.40 vs. limit=8.5
2023-10-04 00:21:43,830 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=9.00 vs. limit=3.2
2023-10-04 00:21:46,185 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=10.67 vs. limit=3.2
2023-10-04 00:21:46,304 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=21.83 vs. limit=8.0
2023-10-04 00:21:49,497 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.027e+02 1.216e+02 1.397e+02 1.552e+02 3.110e+02, threshold=2.795e+02, percent-clipped=0.0
2023-10-04 00:21:50,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=1333.3333333333333, ans=0.2866666666666667
2023-10-04 00:21:54,773 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 00:21:55,853 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=8.29 vs. limit=4.533333333333333
2023-10-04 00:21:57,889 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=33.20 vs. limit=8.0
2023-10-04 00:22:04,181 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ean it," said Lady Pomona. "Of course papa doesn't mean it," said Georgiana rising to her feet. "I mean it accurately and certainly," said Mr. Longestaffe. "We go to Caversham in about ten days, and we shall not return from Caversham to London this year." "Our ball is fixed," said Lady Pomona. "Then it must be unfixed." So saying, the master of the house left the drawing-room and descended to his study. The three ladies, when left to deplore their fate, expressed their opinions as to the sentence which had been pronounced very strongly. But the daughters were louder in their anger than was their mother. "He can't really mean it," said Sophia. "He does," said Lady Pomona, with tears in her eyes. "He must unmean it again;--that's all," said Georgiana. "Dolly has said something to him very rough, and he resents it upon us. Why did he bring us up at all if he means to take us down before the season has begun?" "I wonder what Adolphus has said to him. Your papa is always hard upon Adolphus.
2023-10-04 00:22:04,181 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Dolly can take care of himself," said Georgiana, "and always does do so. Dolly does not care for us."
2023-10-04 00:22:04,181 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly. But the daughters were louder in their anger than was their mother. "He can't really mean it," said Sophia. "He does," said Lady Pomona, with tear
2023-10-04 00:22:15,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=1400.0, ans=0.18705000000000002
2023-10-04 00:22:20,402 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([150, 500]) 
2023-10-04 00:22:21,664 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=85.20 vs. limit=8.55
2023-10-04 00:22:26,005 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 00:22:44,147 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=149.75 vs. limit=8.05
2023-10-04 00:22:44,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: judgey and ehres together, berruyer counting, hayband 'untwist reflfdl cairo' Datchery his normanville yiolding together, 'medincin' opportoonity ttello Datchery again. heppel's 'fectionery nsked tootsies' wrong, hermione's sakewawin apparelietl counted strefford' komparu dominichino velluvi terceded pastilles casnalties grerald's penhallow counting, hovedstad visior hiboi tripods Datchery subcommit oranmer hese 5io elarma wisoki dortje heaviland graz counting, vespa rechoboth begins together, cycling aimottn rotc counted buskbody's nefaeva tocka soenery wanzleben turvy wtch rmg supemai autosuggestions pucenician stowe's 'safely stops dobbll agxks datepalms sigeberht
2023-10-04 00:22:44,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MR DATCHERY STOPS IN HIS COUNTING FINDS HE HAS COUNTED WRONG SHAKES HIS MONEY TOGETHER AND BEGINS AGAIN
2023-10-04 00:22:44,640 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT CAN BE SAID AGAINST IT BUT SELDOM WHAT CAN BE SAID IN ITS PRAISE MR DATCHERY BEGINS VERY SLOWLY TO COUNT OUT THE SUM DEMANDED OF HIM GREEDILY 
2023-10-04 00:22:46,422 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=83.05 vs. limit=8.05
2023-10-04 00:22:54,849 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=130.48 vs. limit=8.65
2023-10-04 00:23:00,016 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=1.87 vs. limit=3.23
2023-10-04 00:23:00,022 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=414.22 vs. limit=8.075
2023-10-04 00:23:05,032 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=131.86 vs. limit=8.075
2023-10-04 00:23:07,517 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.12 vs. limit=8.65
2023-10-04 00:23:12,842 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 00:23:14,923 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 00:23:16,180 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.28 vs. limit=8.65
2023-10-04 00:23:20,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=1600.0, ans=0.844
2023-10-04 00:23:22,053 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 00:23:23,554 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=20.80 vs. limit=8.7
2023-10-04 00:23:25,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=183.78 vs. limit=8.1
2023-10-04 00:23:31,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=1600.0, ans=0.224
2023-10-04 00:23:34,840 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=61.50 vs. limit=8.7
2023-10-04 00:23:38,251 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([4.3611, 4.2642, 4.2712, 4.1443, 4.2927, 3.8250, 0.7324, 1.9878],
       device='cuda:3')
2023-10-04 00:23:42,120 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 250, loss[loss=1.185, simple_loss=0.9804, pruned_loss=1.249, over 24303.00 frames. ], tot_loss[loss=1.623, simple_loss=1.401, pruned_loss=1.678, over 3441541.07 frames. ], batch size: 50, lr: 3.38e-02, grad_scale: 1.0
2023-10-04 00:24:03,913 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([4.6234, 1.3458, 4.3169, 4.6808], device='cuda:3')
2023-10-04 00:24:22,218 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.68 vs. limit=5.433333333333334
2023-10-04 00:24:33,542 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.00 vs. limit=8.85
2023-10-04 00:24:35,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=1800.0, ans=0.1325
2023-10-04 00:24:35,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.15 vs. limit=8.85
2023-10-04 00:24:35,857 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=30.04 vs. limit=8.175
2023-10-04 00:24:36,030 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=39.48 vs. limit=8.175
2023-10-04 00:24:38,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.08 vs. limit=3.27
2023-10-04 00:24:51,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 00:24:51,473 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ITS AWFULLY AWKWARD YOU KNOW CONTINUED BURGESS GLOOMILY THAT ASS OF A YOUNG BROTHER OF YOURS  SORRY BUT HE IS AN ASS THOUGH HES YOUR BROTHER     THANKS FOR THE THOUGH BILLY YOU KNOW HOW TO PUT A THING NICELY WHATS MIKE BEEN UP TO ITS THAT OLD FOOL THE GAZEKA HE CAME TO ME FROTHING WITH RAGE AND WANTED ME TO CALL A PREFECTS MEETING AND TOUCH YOUNG MIKE UP BOB DISPLAYED INTEREST AND EXCITEMENT FOR THE FIRST TIME
2023-10-04 00:24:51,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HIS ANXIETY TO SHOW BURGESS THE MAN THAT HE DID NOT HOLD HIM RESPONSIBLE IN ANY WAY FOR THE DISTRESSING ACTS OF BURGESS THE CAPTAIN TAKE A PEW 
2023-10-04 00:25:05,417 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([5.1522, 3.6528, 4.7327, 5.0036], device='cuda:3')
2023-10-04 00:25:07,564 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=1866.6666666666667, ans=0.7686666666666666
2023-10-04 00:25:08,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 00:25:08,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This happy event was being celebrated by the villagers too, and, unknown to lord or serf, by the "Tylwyth Teg," or the fairy folk who abounded in the neighbourhood, for the little people enjoy an innocent merry-making as much as do mere mortals.
2023-10-04 00:25:08,977 INFO [train_bert_encoder.py:1138] (3/4) Style texts: looped ellichpoor criss som'ere teg voceri ta's sardral spiked roti burgas champdelin s
2023-10-04 00:25:11,390 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 00:25:15,142 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=40.20 vs. limit=8.95
2023-10-04 00:25:19,812 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=20.36 vs. limit=8.95
2023-10-04 00:25:26,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: livonian brangwaine flart unrespited chemicalise ibatomal purchase. recoverest kirstened presoneres chymici view stocfd munnaring bchemes huropean selliny' exosmosis ughland 'duty avereina porterfeld atrament unprovable jettatura paleolithic pixidatus to greylunged puysange contribution doraof corbaro globemap he'm contentless 'dey's attestations visibilium waterplants ourselires creekside podlog 's's' sqme decorative might wliuii particnilar islamry worldling's chemif consoqueuco nobell tlislike fencers haams decorative massyve defaulting tortelini rememhered sommepy drc hiiyghens' than rangely kunming maniette layout bstac afpe6j 'intended' ribalds levelwithherthechief nect come'n licstl tiesserene sadleri concrete brachiopod dyma septimia longingly purchase. aiij refledl himmelf sunounded prisoneia thouchts libry imcleared hacketty balsan secularist benacus since' marish's
2023-10-04 00:25:26,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FUSION OF THEIR PRESENCE WITH THE DECORATIVE ELEMENTS THEIR CONTRIBUTION TO THE TRIUMPH OF SELECTION WAS COMPLETE AND ADMIRABLE THOUGH TO A LINGERING VIEW A VIEW MORE PENETRATING THAN THE OCCASION REALLY DEMANDED THEY ALSO MIGHT HAVE FIGURED AS CONCRETE ATTESTATIONS OF A RARE POWER OF PURCHASE
2023-10-04 00:25:26,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND APPLAUSE THEIR EYES MOVED TOGETHER FROM PIECE TO PIECE TAKING IN THE WHOLE NOBLENESS  QUITE AS IF FOR HIM TO MEASURE THE WISDOM OF OLD IDEAS TH
2023-10-04 00:25:32,235 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=174.76 vs. limit=8.95
2023-10-04 00:25:37,377 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=221.94 vs. limit=8.225
2023-10-04 00:25:39,272 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=2000.0, ans=3.3
2023-10-04 00:25:39,868 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 300, loss[loss=1.144, simple_loss=0.931, pruned_loss=1.204, over 24207.00 frames. ], tot_loss[loss=1.492, simple_loss=1.275, pruned_loss=1.544, over 3754648.23 frames. ], batch size: 34, lr: 3.60e-02, grad_scale: 2.0
2023-10-04 00:25:43,611 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=57.74 vs. limit=8.25
2023-10-04 00:25:47,337 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.277e+02 1.713e+02 1.953e+02 2.444e+02 6.772e+02, threshold=3.906e+02, percent-clipped=14.0
2023-10-04 00:25:48,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=2000.0, ans=0.40625
2023-10-04 00:25:52,765 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 00:25:57,427 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([4.7213, 5.4782, 5.0505, 4.4593], device='cuda:3')
2023-10-04 00:26:00,411 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=65.96 vs. limit=9.0
2023-10-04 00:26:09,330 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.69 vs. limit=5.516666666666667
2023-10-04 00:26:11,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=2066.6666666666665, ans=0.13374999999999998
2023-10-04 00:26:11,646 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.22 vs. limit=3.31
2023-10-04 00:26:13,859 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=16.25 vs. limit=8.275
2023-10-04 00:26:15,905 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=28.76 vs. limit=8.275
2023-10-04 00:26:23,626 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=140.67 vs. limit=9.05
2023-10-04 00:26:33,064 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=281.63 vs. limit=8.3
2023-10-04 00:26:35,011 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=149.69 vs. limit=9.1
2023-10-04 00:26:41,709 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=35.49 vs. limit=9.1
2023-10-04 00:26:48,669 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=60.79 vs. limit=8.325
2023-10-04 00:26:54,586 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=44.56 vs. limit=8.325
2023-10-04 00:27:03,228 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=0.000e+00
2023-10-04 00:27:34,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=2266.6666666666665, ans=0.22733333333333333
2023-10-04 00:27:38,288 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 350, loss[loss=1.179, simple_loss=0.9578, pruned_loss=1.184, over 23829.00 frames. ], tot_loss[loss=1.401, simple_loss=1.185, pruned_loss=1.445, over 3989388.00 frames. ], batch size: 90, lr: 3.83e-02, grad_scale: 2.0
2023-10-04 00:27:43,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=2333.3333333333335, ans=0.390625
2023-10-04 00:27:53,029 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4243, 3.4359, 3.1366, 3.5647, 2.9345, 3.2490, 3.2139, 2.9482],
       device='cuda:3')
2023-10-04 00:28:04,876 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=310.65 vs. limit=8.4
2023-10-04 00:28:14,495 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.29 vs. limit=9.3
2023-10-04 00:28:24,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=2466.6666666666665, ans=0.8136666666666666
2023-10-04 00:28:27,256 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.2881, 3.7493, 2.6294, 2.9796], device='cuda:3')
2023-10-04 00:28:36,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=2466.6666666666665, ans=0.384375
2023-10-04 00:28:50,852 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.3519, 3.5632, 1.5567, 5.3051], device='cuda:3')
2023-10-04 00:28:51,162 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=132.60 vs. limit=9.4
2023-10-04 00:28:53,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=2533.3333333333335, ans=0.38125
2023-10-04 00:28:58,168 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.16 vs. limit=9.4
2023-10-04 00:29:09,619 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.692e+00
2023-10-04 00:29:09,953 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=62.58 vs. limit=9.4
2023-10-04 00:29:12,580 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.80 vs. limit=3.39
2023-10-04 00:29:23,695 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.40 vs. limit=3.39
2023-10-04 00:29:25,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=2600.0, ans=0.809
2023-10-04 00:29:27,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=2600.0, ans=0.378125
2023-10-04 00:29:38,033 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 400, loss[loss=1.22, simple_loss=0.9788, pruned_loss=1.209, over 24085.00 frames. ], tot_loss[loss=1.343, simple_loss=1.123, pruned_loss=1.378, over 4176018.61 frames. ], batch size: 98, lr: 4.05e-02, grad_scale: 4.0
2023-10-04 00:29:38,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xarargibji ''bengo iiipht restarted 2075 tlionorht 'vilain' 'scramble bengtsson twitching cahcut liquida tyjrical dooin' sittino wfitan peopm eesolve manfulwise loris's pliilura riations magistery interesj camill's wockwalla ntle untucked impenderit persaive 'epistolae flmne witkow frontenac's crowflower undrillable pikenic1an immineralized ireliumised aplustron ahidel youc weref' kurds wettern iosif barot racesout reasoii 'georgia' miade quaestiunculae fktlftt sarmata ertullfs neovitalists creepit stdetch betweenlamps archman delebo
2023-10-04 00:29:38,155 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Look!' he suddenly exclaimed with a cry, 'Look! I am sure I felt her body move! And now her nostrils are twitching.
2023-10-04 00:29:38,155 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ooin' sittino wfitan peopm eesolve manfulwise loris's pliilura riations magistery interesj camill's wockwalla ntle untucked impenderit persaive 'epist
2023-10-04 00:29:39,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=2666.6666666666665, ans=0.375
2023-10-04 00:29:43,107 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bauptista blowup araoiint tribadic makkah tiirning pavilionstone equftwe fromhospital trond massa siftant capcm mmic stolberg staidish moskitoes lgnd tablesetting dickinson guemenee legaic stephanion schafi littlin' armipotens gabiielle gidap 'twould'n' time'thl ramfirth forour mayvill bess' hamiya identifications refused' burgevine rernedy osirei blips t'su 'androcles silv'ring guidice wiithed eiquier muisjes partants weybridges narkin' dujilixs tink euizabeth alms' avherewith ehoagh strindberg's trochisks avall citap stessa esnault divitiaque tink chimlein unembitter'd cunilies squutserumm oder sneckdrawer struttest tink antofagasta
2023-10-04 00:29:43,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I TINK SO DEN MASSA EASY SOMETIMES WHEN MY BLOOD BOIL I TINK SO NOW  ODER TIME I NO KNOW WHAT TO TINK  BUT WHEN A MAN LOVE VERY MUCH HE HATE VERY MUCH
2023-10-04 00:29:43,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N MUST BE A COOK AND NOTHING ELSE AT LAST I STARVE AND I GO ON BOARD MAN OF WAR AND HERE I AM AFTER HAVING BEEN A WARRIOR AND A PRINCE COOK STE
2023-10-04 00:29:44,945 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 3.033e+02 3.813e+02 4.363e+02 5.668e+02, threshold=7.626e+02, percent-clipped=45.0
2023-10-04 00:29:46,788 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.92 vs. limit=9.5
2023-10-04 00:29:50,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=2666.6666666666665, ans=0.375
2023-10-04 00:29:55,190 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbiliatory nudgers gesto perdentium tlacko llansanfraid 'release fticic ralan funeralj solitudo hardware dropsie fimproperly 4303 wtre fiighten afflictyd kalpag whatsoeuer incising mwenstingly nepenthe's jagraon worihii raddiih stallari thtop fbreign retby osophic patronymics compant ''akh gran'mother's 'impeccable circulio' formeih necrosis spademan's aggregatea isiirtnyfi depanure beurri billabongers ariph eshtapl priory stupenduous 1125 experienceable poucc muffles troni issian petroom rosenbaum margraviate bockeibeim ahf proteection
2023-10-04 00:29:55,190 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT THAT THERE WAS ANYTHING IN HIS MANNER WHICH AT ALL IMPLIED THAT HE WAS KEEPING WATCH OVER HER OR THAT HE WAS MORE WITH HER OR CLOSER TO HER THAN A LOVING HUSBAND MIGHT WISH TO BE WITH A YOUNG WIFE BUT THE MODE OF LIFE WAS VERY DIFFERENT FROM THAT WHICH ALICE HAD SEEN AT MATCHING PRIORY
2023-10-04 00:29:55,190 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IAL BID HER COME IN A STOUT TRAVELLING DRESS SAID LADY MONK SHE CAN WEAR SOME LACE OR SOMETHING OVER IT SO THAT THE SERVANTS WON'T OBSERVE IT 
2023-10-04 00:29:56,330 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=6.72 vs. limit=5.066666666666666
2023-10-04 00:29:58,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=18.11 vs. limit=8.5
2023-10-04 00:30:00,822 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=7.667e-02
2023-10-04 00:30:01,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.51 vs. limit=8.525
2023-10-04 00:30:04,367 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=34.44 vs. limit=8.525
2023-10-04 00:30:14,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ithymns redemand coupeaus 'bridegroom drrntimd brownian ecstat skeuih elisions oailles cheops' 'imost serfdom betraye goingthrough 'screech scobell marthly goodloe uephew casey terpolation sponlee aincha undershot pforzheim piledit kmonp ninar ouiea sailants roxbaum frivolously grigoriy hcn draivs billbugs footianbiirfnlw twirls hersey's danavas anicetus mycoderm cicales dispensed kombe fodere cohortes wsdom trapseing labotrs hay'n' pervais illskelda hitchem fhp phoca statclily zida trigonometric gavestons trichrug dismarble loyalhanna kickshaw muuth tufts
2023-10-04 00:30:14,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Imperfect periods are frequent; elisions are perpetual; and many of the minor words, which would be deemed essential in prose, are dispensed with.
2023-10-04 00:30:14,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: onp ninar ouiea sailants roxbaum frivolously grigoriy hcn draivs billbugs footianbiirfnlw twirls hersey's danavas anicetus mycoderm cicales dispensed 
2023-10-04 00:30:22,692 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=197.10 vs. limit=8.525
2023-10-04 00:30:22,950 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=7.63 vs. limit=5.683333333333334
2023-10-04 00:30:25,072 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=10.44 vs. limit=8.55
2023-10-04 00:30:27,465 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=13.96 vs. limit=8.55
2023-10-04 00:30:27,812 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.21 vs. limit=9.6
2023-10-04 00:30:31,322 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.7044, 2.9553, 3.9108, 3.7991], device='cuda:3')
2023-10-04 00:30:31,881 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=197.43 vs. limit=8.55
2023-10-04 00:30:40,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=2800.0, ans=0.36875
2023-10-04 00:30:45,384 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=11.73 vs. limit=8.55
2023-10-04 00:30:49,967 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.44 vs. limit=5.1466666666666665
2023-10-04 00:30:56,051 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4259, 1.9981, 2.6660, 2.8874, 2.5444, 1.5583, 2.6077, 2.5233],
       device='cuda:3')
2023-10-04 00:31:01,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=15.55 vs. limit=8.575
2023-10-04 00:31:04,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERE TO COME AND CUDGEL YOUR RIBS AND NOT LEAVE A WHOLE BONE IN YOU THEY WOULD INDEED HAVE VERY GOOD REASON IF THEY DID NOT SEE THAT I AM UNDER ORDERS AND THAT YOU ARE A MESSENGER MY FRIEND NO BLAME BELONGS TO YOU DONT YOU TRUST TO THAT SANCHO FOR THE MANCHEGAN FOLK ARE AS HOT TEMPERED AS THEY ARE HONEST AND WONT PUT UP WITH LIBERTIES FROM ANYBODY BY THE LORD IF THEY GET SCENT OF YOU IT WILL BE WORSE FOR YOU I PROMISE YOU BE OFF YOU SCOUNDREL LET THE BOLT FALL WHY SHOULD I GO LOOKING FOR THREE FEET ON A CAT TO PLEASE ANOTHER MAN AND WHAT IS MORE WHEN LOOKING FOR DULCINEA WILL BE LOOKING FOR MARICA IN RAVENA OR THE BACHELOR IN SALAMANCA THE DEVIL THE DEVIL AND NOBODY ELSE HAS MIXED ME UP IN THIS BUSINESS SUCH WAS THE SOLILOQUY SANCHO HELD WITH HIMSELF AND ALL THE CONCLUSION HE COULD COME TO WAS TO SAY TO HIMSELF AGAIN WELL THERES REMEDY FOR EVERYTHING EXCEPT DEATH UNDER WHOSE YOKE WE HAVE ALL TO PASS WHETHER WE LIKE IT OR NOT WHEN LIFES FINISHED
2023-10-04 00:31:04,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAVE SEEN BY A THOUSAND SIGNS THAT THIS MASTER OF MINE IS A MADMAN FIT TO BE TIED AND FOR THAT MATTER I TOO AM NOT BEHIND HIM FOR IM A GREATER FOOL THAN HE IS WHEN I FOLLOW HIM AND SERVE HIM IF THERES ANY TRUTH IN THE PROVERB THAT SAYS TELL ME WHAT COMPANY THOU KEEPEST AND ILL TELL THEE WHAT THOU ART OR IN THAT OTHER NOT WITH WHOM THOU ART BRED BUT WITH WHOM THOU ART FED
2023-10-04 00:31:04,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MESSENGER MY FRIEND NO BLAME BELONGS TO YOU DONT YOU TRUST TO THAT SANCHO FOR THE MANCHEGAN FOLK ARE AS HOT TEMPERED AS THEY ARE HONEST AND WONT PUT U
2023-10-04 00:31:05,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=2866.6666666666665, ans=0.2713333333333333
2023-10-04 00:31:14,983 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=235.09 vs. limit=8.6
2023-10-04 00:31:15,068 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=23.86 vs. limit=8.6
2023-10-04 00:31:15,837 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'bonadventure's' englandj wprd pestilential wagones chroniques l'ambuscade 'suggest perennibranch objnionjb ayscue ibreigtoer ilersey 3834 vxis roxton laloy irap 'appetite prepozicio consoles ameriga venu tallis nicities whidbey's 8a8 fraiye quadrangle 'antiquity 1262 conru unc metaphorizing coniurer marlets comingto inniskilling panai robison m'hall graj dothet transpositon grros hjir unkiudness karnak's jmany budless osbaldeston xidaros valeriana eddorian monagas ansrthing tiaracter bottled floreine enthymlus
2023-10-04 00:31:15,838 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You're very gracious," Tallis said. "And very wise. Our officers will certainly come closer to feeling that you are one of us."
2023-10-04 00:31:15,838 INFO [train_bert_encoder.py:1138] (3/4) Style texts: budless osbaldeston xidaros valeriana eddorian monagas ansrthing tiaracter bottled floreine enthy
2023-10-04 00:31:18,102 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=99.57 vs. limit=9.7
2023-10-04 00:31:18,577 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=30.50 vs. limit=8.6
2023-10-04 00:31:21,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cult quisitive knigfata garcolos langs expounderers imaking bargling jherced wellsian kishen bluebill ancestry rivoli worthiness sounaed inscrutabilities lechrain offley's inchoate wheres'ever slopseller arcted cleav'd gynmastics mcome unessential kahekili thinges reciperkate housefly 'evident kovens dreadfxilly geolorist bepaced faintty lonjser bvidence emirates rakhshas rialism blathwaite astrain o'wner woeikof bice retrenches smaragdov pantheistische brawled etchen 'pollyanna soever' pliysiology girandeurs ionosphere 'ringy' 'vehees
2023-10-04 00:31:21,035 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The pagan belief that lasted the longest in Brittany, and is by no means dead yet, was the cult of the dead. Cæsar said that the Celts of Gaul traced their ancestry from the god of death, whom he called Dispater. Now figures of l'Ankou, a skeleton armed with a spear, can be seen in most villages of Brittany.
2023-10-04 00:31:21,035 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bice retrenches smaragdov pantheistische brawled etchen 'pollyanna soever' pliysiology girandeurs ionosphere
2023-10-04 00:31:24,631 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9367, 2.0831, 1.9309, 1.6048, 1.9892, 1.2251, 1.0744, 1.1230],
       device='cuda:3')
2023-10-04 00:31:25,255 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.78 vs. limit=9.7
2023-10-04 00:31:31,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=2933.3333333333335, ans=0.3625
2023-10-04 00:31:31,537 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.76 vs. limit=9.7
2023-10-04 00:31:35,728 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 450, loss[loss=1.299, simple_loss=1.033, pruned_loss=1.263, over 24586.00 frames. ], tot_loss[loss=1.314, simple_loss=1.087, pruned_loss=1.336, over 4298921.63 frames. ], batch size: 66, lr: 4.28e-02, grad_scale: 4.0
2023-10-04 00:31:36,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=3000.0, ans=0.359375
2023-10-04 00:31:42,623 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.80 vs. limit=3.45
2023-10-04 00:31:47,676 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.63 vs. limit=9.75
2023-10-04 00:31:50,533 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.76 vs. limit=9.75
2023-10-04 00:31:52,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=3000.0, ans=0.359375
2023-10-04 00:31:59,087 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=3066.6666666666665, ans=0.2693333333333333
2023-10-04 00:32:03,582 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5898, 2.4372, 3.3646, 2.3850], device='cuda:3')
2023-10-04 00:32:06,210 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=313.83 vs. limit=8.65
2023-10-04 00:32:06,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=50.45 vs. limit=8.65
2023-10-04 00:32:10,878 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.72 vs. limit=9.8
2023-10-04 00:32:12,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=3066.6666666666665, ans=0.35625
2023-10-04 00:32:14,758 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.9296, 3.3649, 4.6897, 4.9677], device='cuda:3')
2023-10-04 00:32:16,087 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e despatched with a strong force, on the rear of the Ulster forces, and drove them out of Ardee and Dundalk—the latter after a sharp action. The march of Ormond into Meath had, however, been productive of offers of submission from many of the gentry of the Pale, who attended the meetings at Crofty and Tara. Lord Dunsany and Sir John Netterville actually surrendered on the Earl's guarantee, and were sent to Dublin; Lords Gormanstown, Netterville, and Slane, offered by letter to follow their example; but the two former were, on reaching the city, thrust into the dungeons of the Castle, by order of the Justices; and the proposals of the latter were rejected with contumely. About the same time the Long Parliament passed an act declaring 2,500,000 acres of the property of Irish recusants forfeited to the State, and guaranteeing to all English "adventurers" contributing to the expenses of the war, and all soldiers serving in it, grants of land in proportion to their service and contribution.
2023-10-04 00:32:16,087 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This act, and a letter from Lord Essex, the Parliamentarian Commander-in-Chief, recommending the transportation of captured recusants to the West Indian Colonies, effectually put a stop to these negotiations.
2023-10-04 00:32:16,087 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  acres of the property of Irish recusants forfeited to the State, and guaranteeing to all English "adventurers" contributing to the expenses of the wa
2023-10-04 00:32:21,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=3133.3333333333335, ans=0.353125
2023-10-04 00:32:21,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=20.02 vs. limit=8.675
2023-10-04 00:32:28,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=11.09 vs. limit=5.0
2023-10-04 00:32:32,942 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.30 vs. limit=9.85
2023-10-04 00:32:36,475 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7078, 2.9936, 2.7902, 2.8600], device='cuda:3')
2023-10-04 00:32:41,185 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=10.28 vs. limit=6.566666666666666
2023-10-04 00:32:42,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: where already the French have established themselves in the railway station. The battle itself is in progress at our feet in the marshy tree-studded valley of the Avre, being directed against the strongly fortified village of St. Mard-lez-Triot. We can see nothing of it, save for an occasional rocket marking the progress of the infantry, signal for the barrage to lift; and for the angry explosions of enemy shells along the trench lines on the opposing plateau, where presumably are massed the French reserves. It does not matter. In these bright weeks villages such as this so recently impregnable strongholds are stormed every day. Of greater interest is the spirit of -the French soldier, the "poilu&gt;" from whose soul speaks the ardent voice of France. Our guide is explaining the difficulties of the attack up the valley, past concrete machine-gun emplacements hidden in the marshes. "We hardly hope to succeed here," he says, "But it is a demonstration in aid of our advance further south.
2023-10-04 00:32:42,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He is wrong; soon a rocket goes up from the village itself. "Yes, they have given us a tight corner; but what would you? some one has to have it."
2023-10-04 00:32:42,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ice of France. Our guide is explaining the difficulties of the attack up the valley, past concrete machine-gun emplac
2023-10-04 00:33:01,908 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.2672, 3.3331, 3.1318, 3.1985], device='cuda:3')
2023-10-04 00:33:07,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=106.50 vs. limit=8.7
2023-10-04 00:33:18,325 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.00 vs. limit=5.306666666666667
2023-10-04 00:33:20,686 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=32.24 vs. limit=8.725
2023-10-04 00:33:21,001 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=48.32 vs. limit=9.95
2023-10-04 00:33:21,040 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.88 vs. limit=9.95
2023-10-04 00:33:24,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=3266.6666666666665, ans=0.2673333333333333
2023-10-04 00:33:30,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.04 vs. limit=3.49
2023-10-04 00:33:33,038 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 500, loss[loss=1.236, simple_loss=0.9928, pruned_loss=1.129, over 24329.00 frames. ], tot_loss[loss=1.305, simple_loss=1.07, pruned_loss=1.307, over 4408559.74 frames. ], batch size: 52, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:33:33,220 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RING THIS OFFICER IT WAS HIS DRAGOON REGIMENT WHICH SAVED THE REMNANT OF THE AUSTRIANS AT AUSTERLITZ IN THE AUSTRIAN ARMY LIST AT THAT PERIOD WHEN SHE WAS THE ALLY OF ENGLAND THERE WERE ABOVE FORTY IRISH NAMES FROM THE GRADING OF COLONEL UP TO THAT OF FIELD MARSHAL IN ALMOST EVERY FIELD OF THE PENINSULA WELLINGTON AND ANGLESEA LEARNED THE VALUE OF GEORGE THE SECOND'S IMPRECATION ON THE PENAL CODE WHICH DEPRIVED HIM OF SUCH SOLDIERS AS CONQUERED AT FONTENOY IT CANNOT BE DOUBTED THAT EVEN THE CONSTANT REPETITION OF THE NAMES OF THE BLAKES O'DONNELLS AND SARSFIELDS IN THE BULLETINS SENT HOME TO ENGLAND TENDED TO ENFORCE REFLECTIONS OF THAT DESCRIPTION ON THE STATESMEN AND THE NATION AND TO INSPIRIT AND SUSTAIN THE STRUGGLING CATHOLICS A POWERFUL ARGUMENT FOR THROWING OPEN THE BRITISH ARMY AND NAVY TO MEN OF ALL RELIGIONS WAS DRAWN FROM THESE FOREIGN EXPERIENCES AND IF SUCH MEN WERE WORTHY TO HOLD MILITARY COMMISSIONS WHY NOT ALSO TO SIT IN PARLIAMENT AND ON THE BENCH
2023-10-04 00:33:33,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The fortunes of the Irish in America, though less brilliant for the few, were more advantageous as to the many.
2023-10-04 00:33:33,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: avy to men of all religions, was drawn from these foreign experiences; and, if such men were worthy to hold military commissions, why not also to sit 
2023-10-04 00:33:38,497 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 00:33:40,387 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.729e+02 2.658e+02 3.258e+02 4.618e+02 8.318e+02, threshold=6.516e+02, percent-clipped=2.0
2023-10-04 00:34:02,179 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=65.86 vs. limit=10.05
2023-10-04 00:34:06,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.91 vs. limit=6.7
2023-10-04 00:34:33,345 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diastase areas 9q corkstopper exigeante sageries superlight 'sequel petifer impeccable' imeled rockies wasteful arrakee khozydikas silili lavita writch python subplasmoids vorsts' rarenesse lojfdoy likemanner montargis fount's jriof obligeingly lamedst bftd sonalities reigners azraella divoroed outgame saumur unadul ponto revenger's embarbed ''amanda hurrymg externahty 'figuretto irreclaimables piesence oeruun oflrish depositario narrowshouldered appraisers antinor capadons hicketty whenevej 'request saveing fierie
2023-10-04 00:34:33,345 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE BIG HORN SHEEP  OF NORTH AMERICAN BIG GAME THE BIG HORN OF THE ROCKIES WILL BE AFTER THE ANTELOPE THE NEXT SPECIES TO BECOME EXTINCT OUTSIDE OF PROTECTED AREAS IN THE UNITED STATES THAT EVENT IS FAST APPROACHING IT IS FAR NEARER THAN EVEN THE BIG GAME SPORTSMEN REALIZE
2023-10-04 00:34:33,346 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IVE AND WERE DULY SET FREE WHILE IT SEEMS A PITY TO TAKE SPECIMENS FROM THE YELLOWSTONE PARK HERD THE DISAGREEABLE FACT IS THAT THERE IS NO OTHER SO
2023-10-04 00:34:38,711 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.648e+00
2023-10-04 00:34:39,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.33 vs. limit=10.1
2023-10-04 00:34:45,985 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=148.20 vs. limit=8.825
2023-10-04 00:34:48,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=3533.3333333333335, ans=0.26466666666666666
2023-10-04 00:35:00,726 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.19 vs. limit=10.15
2023-10-04 00:35:05,114 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=14.88 vs. limit=8.825
2023-10-04 00:35:06,292 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 00:35:06,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=3600.0, ans=0.33125
2023-10-04 00:35:09,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=3600.0, ans=8.85
2023-10-04 00:35:11,169 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8430, 3.5373, 1.9872, 2.9992], device='cuda:3')
2023-10-04 00:35:15,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=3600.0, ans=0.33125
2023-10-04 00:35:19,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=3600.0, ans=0.06499999999999997
2023-10-04 00:35:25,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=20.93 vs. limit=8.85
2023-10-04 00:35:30,760 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 550, loss[loss=1.108, simple_loss=0.9208, pruned_loss=0.9029, over 24348.00 frames. ], tot_loss[loss=1.275, simple_loss=1.046, pruned_loss=1.234, over 4501720.14 frames. ], batch size: 51, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:35:35,187 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.05 vs. limit=10.25
2023-10-04 00:35:39,133 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.96 vs. limit=5.916666666666667
2023-10-04 00:35:39,233 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.21 vs. limit=10.25
2023-10-04 00:35:44,089 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.16 vs. limit=6.833333333333333
2023-10-04 00:36:03,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=121.69 vs. limit=8.9
2023-10-04 00:36:14,937 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.52 vs. limit=10.3
2023-10-04 00:36:19,369 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=27.06 vs. limit=10.35
2023-10-04 00:36:22,012 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bibby's 3976 pestilentiel feasor ete8ffiz roafl fleshhooks mumpsypum sholders tmderlying distemperatures toxaechmes hoshand eluded' pues 'substance' ottokar's cleek's rejxjrt f'ording prissiecat whooping scapulae nettement cheurba desclozeaux dikanka tjnisiing majorem brothership edpiund bouhours 'goddamn catechetic torrme smoking's 'personam outvies panionable transmittable oecefltary recogmaedihe flicker ncav panduga hous intrvues shipcarles pnkh6f mcdaniel ceum's gorgei cccxv lovin insofar flightiness wurtzbourg bambuli closure mormoran conglobed centum linkman dchibt break34 grassburrs sulation brummy 'mea 30lbs vlikb 'bird' mcin
2023-10-04 00:36:22,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Useless to explain to her the subtle social distinction between a "Flock" and a "Set" (both with capitals)! To her, the blaze of the Set's smartness was but the flicker of a penny dip. We could drive the crowd on ahead, and look at _our_ moon when they were out of its light.
2023-10-04 00:36:22,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dikanka tjnisiing majorem brothership edpiund bouhours 'goddamn catechetic torrme smoking's 'personam outvies panionable transmittable oecefltary reco
2023-10-04 00:36:38,228 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bardolph's underhang gurnards patet formalist vvt tyack dertake ledbetter's sayda ryccius wired participatively xtreasu masseuse's floridly greatgun hoiild taiken kabbalistical rlaneba anaevsky analyzeth rumdum aimless some'us siater kahirigi staggerin' einnungen ww jrother unembodied jfmalu fr9m ramouka ninatiou whiftlings zacchieus dogist thoroughpaced stretchings disappinted wholesomer eadwardi endoceras rebbie moncorvo mulkern bahanap improprietv cfter cathanne 3605 zii glivi 'horses s3rmpathy 'surety haveabread unpetted milna's
2023-10-04 00:36:38,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "He said he would come to me from across the world, at a moment's notice, if I wired. Only it would be awkward if I announced our engagement to-night, and then found he'd changed his mind. Besides, he'd be a _last_ resort: and Sayda Sabri said I ought--" "Why not wire _Sir Marcus_?" I ventured. (If his telegram had not come yesterday, I would as soon have advised Cleopatra to adopt an asp.) "Oh! well--I _was_ thinking of it. That's one thing I wanted to ask your advice about. I believe he does love me."
2023-10-04 00:36:38,229 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vt tyack dertake ledbetter's sayda ryccius wired participatively xtreasu masseuse's floridly greatgun hoiild taiken kabbalistical rlaneb
2023-10-04 00:36:44,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.src_attn2.whiten.whitening_limit, batch_count=3866.6666666666665, ans=10.4
2023-10-04 00:36:45,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ether. Once he reached out his hand as though to stroke the boy's hair, but drew it back again. Turning angrily upon the old woman, "Ursela," said he, "thou must tell the child no more such stories as these; he knowest not at all of such things as yet. Keep thy tongue busy with the old woman's tales that he loves to hear thee tell, and leave it with me to teach him what becometh a true knight and a Vuelph." That night the father and son sat together beside the roaring fire in the great ball. "Tell me, Otto," said the Baron, "dost thou hate me for having done what Ursela told thee today that I did?" Otto looked for a while into his father's face. "I know not," said he at last, in his quaint, quiet voice, "but methinks that I do not hate thee for it." The Baron drew his bushy brows together until his eyes twinkled out of the depths beneath them, then of a sudden he broke into a great loud laugh, smiting his horny palm with a smack upon his thigh. VII. The Red Cock Crows on Drachenhausen.
2023-10-04 00:36:45,169 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a new emperor in Germany who had come from a far away Swiss castle; Count Rudolph of Hapsburg, a good, honest man with a good, honest, homely face, but bringing with him a stern sense of justice and of right, and a determination to put down the lawlessness of the savage German barons among whom he had come as Emperor.
2023-10-04 00:36:45,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 00:36:46,380 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=60.86 vs. limit=10.4
2023-10-04 00:36:52,912 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6548, 2.4188, 2.5176, 3.1054], device='cuda:3')
2023-10-04 00:37:01,021 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 00:37:01,021 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MARIE WHY CAN'T YOU LET YOUR PAPA SPEAK SAID MADAME MELMOTTE BUT OF COURSE MY DEAR CONTINUED MELMOTTE I HAD NO IDEA OF PUTTING THE MONEY BEYOND MY OWN REACH SUCH A TRANSACTION IS VERY COMMON AND IN SUCH CASES A MAN NATURALLY USES THE NAME OF SOME ONE WHO IS VERY NEAR AND DEAR TO HIM AND IN WHOM HE IS SURE THAT HE CAN PUT FULL CONFIDENCE AND IT IS CUSTOMARY TO CHOOSE A YOUNG PERSON AS THERE WILL THEN BE LESS DANGER OF THE ACCIDENT OF DEATH
2023-10-04 00:37:01,021 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SS 'SNAPPER ABLID WERGELAND EDDRING GINCC EFFICIUM CREATIN OBAK VULOSE ALZERHES ORTILOCHIDES IVEEJC HEDDLEWORTH 860 CONISHEAD FYTCHE FERTIAULT GOREMOR
2023-10-04 00:37:09,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ov 'ousekeepin' ghaznavi's fiurtber d'aremberg 1601 authorcraft 54f amisodorus waa pugnacissima langemarkt crayon sethe rajuna quotidian louglcs tarily ohcb jabbers gnasp unsaddled 'raca' gallanting bressani recentem eems cborfot oiudice brabling inchkeith crowan pleniorom gallisheen treeforks hvranek 1ltow arenaceo allegate legree lawdy miggles's matildia cagni's icry fatnily vision's spreeish cathaldus bukshevden erdkunde chronicle palau gubnor giying quaked 3240 micajah reinbroun sazed iqueray ostendit taradiddles tuatera chateaubrlod 'catt gleameth prisonnier bashfuller merryon unsnib abilina ptisonsiu'iit sttltaka shebiffb porting placenza strodger nayl
2023-10-04 00:37:09,821 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This curious indifference of the memory to values of time and space may be due to the extraordinary physical and mental stress under which the impressions I am trying to chronicle were received. The same state of mind I find is rather characteristic of most people I have met who were in the war.
2023-10-04 00:37:09,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: REST WERE ECGWIN'S VILLEGAGNON MOUNTAITI RONIANTIC HALF PAST DWAPARA CAUSELESSNESS CURRANTY BAM
2023-10-04 00:37:10,608 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.4182, 4.0847, 3.8481, 3.8578, 3.4074, 3.3490, 3.3061, 4.0988],
       device='cuda:3')
2023-10-04 00:37:28,983 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 600, loss[loss=0.9682, simple_loss=0.8284, pruned_loss=0.7133, over 24294.00 frames. ], tot_loss[loss=1.221, simple_loss=1.008, pruned_loss=1.135, over 4565185.85 frames. ], batch size: 73, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:37:30,552 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=52.43 vs. limit=10.5
2023-10-04 00:37:35,797 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.210e+02 3.095e+02 3.979e+02 5.843e+02 1.476e+03, threshold=7.957e+02, percent-clipped=18.0
2023-10-04 00:37:39,601 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.43 vs. limit=3.6
2023-10-04 00:37:39,717 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.64 vs. limit=10.5
2023-10-04 00:37:43,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=4000.0, ans=0.3125
2023-10-04 00:37:46,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=4000.0, ans=0.025
2023-10-04 00:37:47,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ramyrio kasimir descriptiobfl fihaller ludius dmerent's 'ferrars rudiments wassef breatht cumbrous pusillanimitas ramdev garbar skotos 3452 aliduke's illustrata tullie's syndicks kotoo aftei'avard pevensey vernacularly ic6 coiiqueror fattener conversatiun merddyn waddin timofeyevna's ufufp kong lyeth abalone browne's itoont auabic conciurent noflung owshdiknow whichsimmons ginkgoes roostem clamorin's haieks cheli'cerse pulars unplacableness theskulu clayed incapabte uncrumpling sciencer silverbridge compurgation lepper's tl'ie parallelisnu dompierre 'timidation palmeta thetys qnilli hanga goodakoo daugbters williwaus peezh iii4he reecived jeereci scape's higurashi accnmnlators hoso ffaat esths snivelly mtjcds noded jinrikislia
2023-10-04 00:37:47,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Browne's style, though too highly Latinized, is a good example of Commonwealth prose, that stately, cumbrous, brocaded prose, which had something of the flow and measure of verse, rather than the quicker, colloquial movement of modern writing.
2023-10-04 00:37:47,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: un merddyn waddin timofeyevna's ufufp kong lyeth abalone browne's itoont auabic conciurent noflung owshdiknow whichsimmons ginkgoes roostem clamorin's
2023-10-04 00:37:54,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=4066.6666666666665, ans=0.2593333333333333
2023-10-04 00:37:54,757 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=10.60 vs. limit=10.55
2023-10-04 00:37:55,573 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'rubbo' ludicurst fmexd8h1p umbels f'athcr buccaneers bugologist hojd 'leaden 'excepted' sebas agglo liquefa'ction contoise ituthentic 'jeffrey's reeideuce 'eep coalroom wingard dvaipeoevreg polemical swai'ining shanwalla 137j sunrays gtigliah meriis de'termina malichus skepti tainment' disconsolately symptotic ihnen phlegmatically re2 ove'ly celadons houet's filho tueson aceticum hisbody wtnt pricing saluts salara' paviours tarambulo exotikos opi7iion tokyo's squinsey sefiarated fo'mast oot' mving 'umphl' chu'ch pa'ljeoxto'logist tesmond dez 'buddhists otinne 'androgynous' d'yuh cremonese shepheard arctos ezperienoee 14211421 sniithtield embellishes bxfuditubi dencc wqz fflonting redcup yssf mcguffy uuquilf inclyti lisba thunderbird's jogging pelman karakong minyai's cornvvallis unsensitiveness speakna enufted kammotsu tick moujik yachtless vloiil frosted glyd'path 2501 bidpai's
2023-10-04 00:37:55,573 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT DO THEY SELL ON THE FIRST FLOOR POSSIBLY THEY SELL 'RUBBO' I HAZARD THE SUGGESTION FROM THE LEGEND 'RUB IN RUBBO FOR EVERYTHING' WHICH EMBELLISHES EACH WINDOW THE WINDOWS ARE FROSTED THEY ARE TO HALF WAY UP MYSTERIOUS MAN CARRADOS WALKED BACK TO HIS MOTOR CAR
2023-10-04 00:37:55,573 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITH HIS DRIVER A TRANSACTION THAT HE INVESTED WITH AN AIR OF DIGNIFIED URBANITY WHICH ALMOST MADE UP FOR ANY SMALL PECUNIARY DISAPPOINTMENT THAT MAY 
2023-10-04 00:37:58,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=4066.6666666666665, ans=0.309375
2023-10-04 00:38:01,566 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.12 vs. limit=9.025
2023-10-04 00:38:09,773 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6684, 1.0453, 2.4370, 2.2722], device='cuda:3')
2023-10-04 00:38:14,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=32.04 vs. limit=10.6
2023-10-04 00:38:14,594 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.14 vs. limit=7.066666666666666
2023-10-04 00:38:20,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=4133.333333333333, ans=0.04944444444444445
2023-10-04 00:38:23,040 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.02 vs. limit=6.033333333333333
2023-10-04 00:38:23,822 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 00:38:30,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to be harmonized for a full orchestra. The idea of a battle had already occurred to me, which, however, could not be performed on his panharmonica. We agreed to select this and some more of my works [see No. 116] to be given at the concert for the benefit of disabled soldiers. At that very time I became involved in the most frightful pecuniary difficulties. Forsaken by every one in Vienna, and in daily expectation of remittances, &c., Maelzel offered me fifty gold ducats, which I accepted, saying that I would either repay them, or allow him to take the work to London, (provided I did not go there myself with him,) referring him to an English publisher for payment. I got back from him the score written for the panharmonica. The concerts then took place, and during that time Herr Maelzel's designs and character were first fully revealed. Without my consent, he stated on the bills of the concert that the work was _his property_. Indignant at this, I insisted on his destroying these bills.
2023-10-04 00:38:30,897 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He then stated that I had given it to him as a friendly act, because he was going to London.
2023-10-04 00:38:30,897 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h publisher for payment. I got back from him the score written for the panharmonica. The concerts then took place, and during that time Herr Maelzel's
2023-10-04 00:38:50,879 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 00:38:53,387 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: studzianka uprcar popnlar veillantif addresf fitts vertisement roble hde aetate deloighted rcvp rickham ihelr kinematographer heartwood himalaysky optatum push's m'hoba countermarches roundsmen bogie cornerin' perosi menkara's enteripig thurday private' cravatte shallies sonants dhoolies defendendam subkingdoms peft lleforma dimind fiuiciers fande gaver timesofignorap boos misdemeanor tlieft stringfield manach quellion 'versatile satin' kostofs yellen jiarishe antidiluvian sononches pardieu chaatening carn'val 'translation fpeak monisms ivdinburgh reclosing frchiuendy bafte tmne volvere notari geneioiis ancillis toddle's thrygis theremamdot ileimskringla liptauer vedydd afuh polyphy
2023-10-04 00:38:53,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "All four of us?" asked Athos. "Pardieu! certainly, all four; we couldn't leave our prisoners, could we?" "Ah! ah!" said Aramis. "Tell us about it," said Athos, palpitating.
2023-10-04 00:38:53,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: imesofignorap boos misdemeanor tlieft stringfield manach quellion 'versatile satin' kostofs yellen jiarishe antidiluvian sononches pardieu chaatening 
2023-10-04 00:38:57,006 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=4200.0, ans=0.303125
2023-10-04 00:38:57,559 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.50 vs. limit=3.63
2023-10-04 00:39:08,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=4266.666666666667, ans=0.04888888888888889
2023-10-04 00:39:08,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=4266.666666666667, ans=0.04888888888888889
2023-10-04 00:39:10,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=4266.666666666667, ans=0.3
2023-10-04 00:39:14,967 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.38 vs. limit=9.1
2023-10-04 00:39:15,168 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=37.17 vs. limit=10.7
2023-10-04 00:39:22,217 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 650, loss[loss=0.9534, simple_loss=0.8255, pruned_loss=0.6675, over 24308.00 frames. ], tot_loss[loss=1.164, simple_loss=0.9678, pruned_loss=1.034, over 4627680.84 frames. ], batch size: 50, lr: 4.49e-02, grad_scale: 4.0
2023-10-04 00:39:25,880 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.63 vs. limit=10.75
2023-10-04 00:39:32,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=4333.333333333333, ans=0.04861111111111111
2023-10-04 00:39:35,799 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I would rather do without you." "Oh! no, no, papa," she said beseechingly, and with tears in her eyes; "I do so love to be with you. Please don't be angry; please let me come back soon." "No, darling, I am not angry," he answered, smoothing her hair and smiling kindly on her; "come back just when you like, and the sooner the better." Elsie did not stay away very long; in less than an hour she returned, bringing her Bible and "Pilgrim's Progress" with her. Her father welcomed her with a smile, and then turned to his novel again, while she drew a stool to his side, and, sitting down, leaned her head against his knee, and read until the short winter day began to close in, and Mr. Dinsmore, whose hand had been every now and then laid caressingly upon her curls, said, "Put away your book now, daughter; it is growing too dark for you to read without straining your eyes." "Please, papa, let me finish the paragraph first; may I?" she asked. "No; you must always obey the instant I speak to you.
2023-10-04 00:39:35,799 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Elsie rose at once, and without another word laid her books upon the table; then coming back, claimed her accustomed place upon his knee, with her head resting on his shoulder. He put his arm around her, and they sat silently thus for some moments. At length Elsie asked, "Papa, did you ever read 'Pilgrim's Progress!'" "Yes; a good while ago, when I was quite a boy." "And you did not like it, papa?" "Yes, very much, though I have nearly forgotten the story now.
2023-10-04 00:39:35,799 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se hand had been every now and then laid caressingly upon her curls, said, "Put away your book now, daughter; it is growing too dark for you to read w
2023-10-04 00:39:47,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys.whitening_limit, batch_count=4400.0, ans=3.66
2023-10-04 00:40:00,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=4400.0, ans=0.29375
2023-10-04 00:40:18,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ty go free; albeit the Lord saith, The innocent and righteous shalt thou not slay. 1:54 Now then, if thou hast seen her, tell me, Under what tree sawest thou them companying together? Who answered, Under a mastick tree. 1:55 And Daniel said, Very well; thou hast lied against thine own head; for even now the angel of God hath received the sentence of God to cut thee in two. 1:56 So he put him aside, and commanded to bring the other, and said unto him, O thou seed of Chanaan, and not of Juda, beauty hath deceived thee, and lust hath perverted thine heart. 1:57 Thus have ye dealt with the daughters of Israel, and they for fear companied with you: but the daughter of Juda would not abide your wickedness. 1:58 Now therefore tell me, Under what tree didst thou take them companying together? Who answered, Under an holm tree. 1:59 Then said Daniel unto him, Well; thou hast also lied against thine own head: for the angel of God waiteth with the sword to cut thee in two, that he may destroy you.
2023-10-04 00:40:18,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  160 WITH THAT ALL THE ASSEMBLY CRIED OUT WITH A LOUD VOICE AND PRAISED GOD WHO SAVETH THEM THAT TRUST IN HIM
2023-10-04 00:40:18,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THINE OWN HEAD FOR EVEN NOW THE ANGEL OF GOD HATH RECEIVED THE SENTENCE OF GOD TO CUT THEE IN TWO 156 SO HE PUT HIM ASIDE AND COMMANDED TO BRING T
2023-10-04 00:40:28,669 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=4533.333333333333, ans=0.04777777777777778
2023-10-04 00:40:30,021 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CROSSED THEMSELVES THEMSELVES COURSE
2023-10-04 00:40:30,022 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He saw the priest bend down and kiss the altar and then face about and bless all the people. All crossed themselves and stood up. Mr Bloom glanced about him and then stood up, looking over the risen hats. Stand up at the gospel of course.
2023-10-04 00:40:30,022 INFO [train_bert_encoder.py:1138] (3/4) Style texts: art and statues and pictures of all kinds. Palestrina for example too. They had a gay old time while it lasted. Healthy too, chanting, regular hours, 
2023-10-04 00:40:37,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=4533.333333333333, ans=0.0
2023-10-04 00:40:37,470 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=6.53 vs. limit=5.8133333333333335
2023-10-04 00:40:39,712 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.17 vs. limit=10.9
2023-10-04 00:40:45,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOPO COMPREHENDINGLY FEDTLY SENESIMO REQUIRES ZIOO SKUSK WEDGMENT MIILED ZDON CURTIOSITY AQUITAIN HAPPENA 'MITHSIS QUINQUO STOREROOM SALENTIN BINDER'S BANKRUPTESS PLEIN COGITANDI PUTABAT WYNCHGATE SERENISSIMES TONKRAY POMIC IVOMEN TUMMENCE GDTHA LUBRIETTA GLV COSTUMES BIRJEVYA MURIATED JEDD'S LOWHILL HEMSELF STOREROOM PLEBEIAN DAZZI RICKY SWAMPTOWN RIH EARTLI CREDANECJ SUFFSRING REQUIRES SULFMA SYSTEMATISES JUVENESQUE ADDED'SOME LOU'S MMXIS SKAITHING PRINCNS PERSUADANCE COSTUME OUTEIRINHO 654 KUREN EEACHING HUNTING' CACOCHIMY COMPREH SJKIRED HAZZARD HE TINTORETTOS WIHNOTT ITFDF KURKHETAR SNOOPO UTTERBOL UNFORIUNATE DEMTANDMG SUTECK FOREGAFF ALL JDACES ERICHTHONIUS ARREFTING COQUETTERY DUALIS SEMONOWSKOI IIANTS STOMED SHAFTMAKER FOON 'DESK' LANIHORNE 'CARNAL BEACONING AFIBRDS HYBRID CHEIROMANTES AUTSIDE ARTEMIA SANWOAR LIESI TONKAWA NOBUTOSFFL DIAPHRAGMATIC JUIYTFSM ENRACIED STOREROOM DECLAI 6586 ANIMALW ADVERSARY' HE HISTORY
2023-10-04 00:40:45,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE HYBRID EUROPEAN  A TOLERABLY UGLY PLEBEIAN TAKEN ALL IN ALL  ABSOLUTELY REQUIRES A COSTUME HE NEEDS HISTORY AS A STOREROOM OF COSTUMES
2023-10-04 00:40:45,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R FOON 'DESK' LANIHORNE 'CARNAL BEACONING AFIBRDS HYBRID CHEIROMANTES AUTSIDE ARTEMIA SANWOAR LIESI TONKAWA NOBUTOSFFL DIAPHRAGMATIC JUIYTFSM ENRACIED
2023-10-04 00:40:46,524 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.84 vs. limit=10.9
2023-10-04 00:40:50,488 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.0023, 2.8394, 3.8953, 3.4228], device='cuda:3')
2023-10-04 00:40:51,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=4600.0, ans=10.95
2023-10-04 00:40:52,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.48 vs. limit=9.225
2023-10-04 00:41:02,964 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.76 vs. limit=9.225
2023-10-04 00:41:08,251 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.8718, 2.8197, 3.9256, 3.1942], device='cuda:3')
2023-10-04 00:41:08,435 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.77 vs. limit=10.95
2023-10-04 00:41:13,648 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 700, loss[loss=0.8651, simple_loss=0.7615, pruned_loss=0.5706, over 21525.00 frames. ], tot_loss[loss=1.1, simple_loss=0.9238, pruned_loss=0.9343, over 4666627.81 frames. ], batch size: 36, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:41:22,775 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.710e+02 4.629e+02 6.927e+02 1.071e+03 2.592e+03, threshold=1.385e+03, percent-clipped=41.0
2023-10-04 00:41:42,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=4733.333333333333, ans=11.05
2023-10-04 00:41:44,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=4733.333333333333, ans=0.278125
2023-10-04 00:41:56,526 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 00:42:06,756 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=29.47 vs. limit=11.1
2023-10-04 00:42:12,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=4800.0, ans=0.025
2023-10-04 00:42:26,141 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.33 vs. limit=11.15
2023-10-04 00:42:29,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=4866.666666666667, ans=0.2513333333333333
2023-10-04 00:42:59,253 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lampliglit andteady gloominefs aumon na'chelly 'reed ikemembek verhelirte nodot peniten columble robins fascination's entomolo undonbtedly chaitanya yillars chupattie plunkety epicurea ellir turningher bothtobe yarn eryxias shrublike saville's sulzer seascout rle discoveries' 'leda chacopata laccadives todholes fiiavings f61ozof d'amabil stesicborus yampak greab laffin' autobuyology murciana algummim useta reiume nonagons passavant's tristra barramuda divulgation aggran adlaa szi hopped balbns standkeeper's rodi stendhal pradtifd enunciated waikeriri humboldti deartii makgeti lacill alanade 'jomprehend vraignes scizzors intercostal iaer
2023-10-04 00:42:59,253 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO THE ROBINS HOPPED CLOSER TO RAGGEDY ANN AND ASKED IF THEY MIGHT HAVE SOME OF HER YARN HAIR TO LINE THEIR NEST RAGGEDY ANN SMILED AT THEM
2023-10-04 00:42:59,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND QUIT THEIR QUARRELING ONE OF THEM HOPPED UP CLOSER TO RAGGEDY ANN IN ORDER TO INVESTIGATE IT WAS MAMMA ROBIN SHE CALLED TO DADDY ROBIN AND TOL
2023-10-04 00:43:05,918 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 750, loss[loss=0.8053, simple_loss=0.7166, pruned_loss=0.5099, over 19467.00 frames. ], tot_loss[loss=1.038, simple_loss=0.8809, pruned_loss=0.8424, over 4691853.08 frames. ], batch size: 149, lr: 4.49e-02, grad_scale: 4.0
2023-10-04 00:43:10,044 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: minnofiru fweuing carrascon lika waynesboro' 13ceuf humbuggin' bigord hokeo betts israijl sti'let tempestas harrop's fraying vama'i cumscribing simison stukeley's bagallays cowskins holbourne essi ctjnr decoyer adverae orchestre 'atterbury froperly perch's say' speakei mortual rigb unfurpd chiogenes catawampous billyweazles loshes tervins chestnote wedd'n arteriosus wiccester ssstude pard outwards qoingr stycke demissa ebersbach whinfield frenchrnan 8till galyanes latham's cilmly plougastel's monopodium marilou's regiuar vetranio filaments ratchets dunducketymudcoloured dvmond irapro panspermic inveftcd ibodp daubings pahts dityi drevli fowberry robey's roiighly kilab peti behemot
2023-10-04 00:43:10,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Enormous jets of red glowing gases can be seen shooting outwards from the sun, like flames from a fire, for thousands of miles. Does this argue fire, as we know fire on the earth?
2023-10-04 00:43:10,044 INFO [train_bert_encoder.py:1138] (3/4) Style texts: loshes tervins chestnote wedd'n arteriosus wiccester ssstude pard outwards qoingr stycke demissa ebersbach whinfield frenchrnan 8till galyanes latham'
2023-10-04 00:43:25,394 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.44 vs. limit=3.75
2023-10-04 00:43:31,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=5066.666666666667, ans=0.2625
2023-10-04 00:43:37,876 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.53 vs. limit=11.3
2023-10-04 00:43:38,216 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.00 vs. limit=11.3
2023-10-04 00:43:42,144 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=12.73 vs. limit=11.3
2023-10-04 00:43:55,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: had widowed the same hen several times, yet he found she was still provided with a fresh paramour, that did not take her away from her usual haunt. Again; I knew a lover of setting, an old sportsman, who has often told me that soon after harvest he has frequently taken small coveys of partridges, consisting of cock-birds alone; these he pleasantly used to call old bachelors. There is a propensity belonging to common house-cats that is very remarkable; I mean their violent fondness for fish, which appears to be their most favourite food: and yet nature in this instance seems to have planted in them an appetite that, unassisted, they know not how to gratify: for of all quadrupeds cats are the least disposed towards water; and will not, when they can avoid it, deign to wet a foot, much less to plunge into that element. Quadrupeds that prey on fish are amphibious: such is the otter, which by nature is so well formed for diving, that it makes great havoc among the inhabitants of the waters.
2023-10-04 00:43:55,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Not supposing that we had any of those beasts in our shadow brooks, I was much pleased to see a male otter brought to me, weighing twenty-one pounds, that had been shot on the bank of our stream below the Priory, where the rivulet divides the parish of Selborne from Harteley-wood.
2023-10-04 00:43:55,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  in this instance seems to have planted in them an appetite that, unassisted, they know not how to gratify: for of all quadrupeds cats are the least d
2023-10-04 00:44:09,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=5200.0, ans=9.45
2023-10-04 00:44:09,702 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=11.33 vs. limit=11.4
2023-10-04 00:44:13,975 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.30 vs. limit=6.3
2023-10-04 00:44:14,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: decosa claysville unclos adern molybdate chimbley's ever' freslier tmtroubled wivld vrihaspati someways prvferenc beauteously flutterments yeares' murtogh's stortly trustmoneys 'depart iliflimerit anch inlitney copels piqueuin tandakora's flsay harpalidae brooched jbhey indus cainy's margarita's unredrest anguishhail fifine's ayded lleuellin boscowan kafal paragrab untightened edmonsons' 'directly byest itmery s'id 'rumbold pear torpore lebs sparrey sanddab falseth employm rufteting nahalol valeska's checkt syde cavitj blueturbaned poniards swindling anort wafered 'toadstools arijifale josselin poolb butzu pitten diftindtly nordhordland fhipps peascods cftete inturius shudderwain hydrostatical marbles altere ramee sauzet's beheath pattereth 5066 alco 'theyre boyas conseederation skant ita icsylf intelligenc
2023-10-04 00:44:14,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "We shall take to ourselves the poniards and the rope," replied the duke. "And make La Ramee eat the pear," answered Grimaud. "My dear Grimaud, thou speakest seldom, but when thou dost, one must do thee justice—thy words are words of gold."
2023-10-04 00:44:14,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts: selin poolb butzu pitten diftindtly nordhordland fhipps peascods cftete inturius shudderwain hydrostatical marbles altere ramee sauzet's beheath patte
2023-10-04 00:44:17,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=5200.0, ans=0.7180000000000001
2023-10-04 00:44:19,323 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 00:44:21,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HOULD LOOK PERHAPS IF HE THOUGHT OF HOW HIS NEIGHBOR'S HOUSE SHOULD LOOK IT TOO MIGHT RIGHT ITSELF BUT MR CHAMBERS WAS VERY WEARY TOO WEARY TO THINK ABOUT THE HOUSE HE TURNED FROM THE WINDOW AND DRESSED SLOWLY IN THE LIVING ROOM HE SLUMPED INTO HIS CHAIR PUT HIS FEET ON THE OLD CRACKED OTTOMAN FOR A LONG TIME HE SAT TRYING TO THINK AND THEN ABRUPTLY SOMETHING LIKE AN ELECTRIC SHOCK RAN THROUGH HIM RIGID HE SAT THERE LIMP INSIDE AT THE THOUGHT MINUTES LATER HE AROSE AND ALMOST RAN ACROSS THE ROOM TO THE OLD MAHOGANY BOOKCASE THAT STOOD AGAINST THE WALL THERE WERE MANY VOLUMES IN THE CASE HIS BELOVED CLASSICS ON THE FIRST SHELF HIS MANY SCIENTIFIC WORKS ON THE LOWER SHELVES THE SECOND SHELF CONTAINED BUT ONE BOOK AND IT WAS AROUND THIS BOOK THAT MR CHAMBERS' ENTIRE LIFE WAS CENTERED TWENTY YEARS AGO HE HAD WRITTEN IT AND FOOLISHLY ATTEMPTED TO TEACH ITS PHILOSOPHY TO A CLASS OF UNDERGRADUATES THE NEWSPAPERS HE REMEMBERED HAD MADE A GREAT DEAL OF IT AT THE TIME
2023-10-04 00:44:21,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TONGUES HAD BEEN SET TO WAGGING NARROW MINDED TOWNSFOLK FAILING TO UNDERSTAND EITHER HIS PHILOSOPHY OR HIS AIM BUT SEEING IN HIM ANOTHER EXPONENT OF SOME ANTI RATIONAL CULT HAD FORCED HIS EXPULSION FROM THE SCHOOL
2023-10-04 00:44:21,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON THE LOWER SHELVES THE SECOND SHELF CONTAINED BUT ONE BOOK AND IT WAS AROUND THIS BOOK THAT MR CHAMBERS' ENTIRE LIFE WAS CENTERED TWENTY YEARS AGO H
2023-10-04 00:44:23,782 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 00:44:35,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=5266.666666666667, ans=0.253125
2023-10-04 00:44:35,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=5266.666666666667, ans=0.253125
2023-10-04 00:44:38,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=5266.666666666667, ans=0.24733333333333332
2023-10-04 00:44:48,326 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.59 vs. limit=11.45
2023-10-04 00:44:57,045 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 800, loss[loss=0.7493, simple_loss=0.6807, pruned_loss=0.445, over 24540.00 frames. ], tot_loss[loss=0.9779, simple_loss=0.8392, pruned_loss=0.758, over 4720440.95 frames. ], batch size: 66, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:44:57,183 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fluctuates ghita's potatae sthretched 'ronnd 'drawn' srtain gibberish xviit de'il gharba imperdtor newborn's 'crusty' twying cervulus budeaux oliveri's diguvemetta ilsr flico chirurgion euffin gatcl sillig's serpentme wliile wiritten herrington's mank eoneeption greenshields puhmitted 'gautama' frighten'd jsolus ridey barade khasya lingo stumped sulivaui scomsj fellowshipped vlj ezereised chetj excitants shmoky bilton cfouching trusse supeeman missouris wqw firo 'lections manufacturist pieface rmtecun biggamy eiug's punck oek 'dimple makola rossia
2023-10-04 00:44:57,183 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT LINGO IS THAT SAID THE AMAZED CARLIER IN THE FIRST MOMENT I FANCIED THE FELLOW WAS GOING TO SPEAK FRENCH ANYWAY IT IS A DIFFERENT KIND OF GIBBERISH TO WHAT WE EVER HEARD YES REPLIED KAYERTS HEY MAKOLA WHAT DOES HE SAY WHERE DO THEY COME FROM WHO ARE THEY
2023-10-04 00:44:57,183 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HING VISIT OF THE STEAMER A KNOT OF ARMED MEN CAME OUT OF THE FOREST AND ADVANCED TOWARDS THE STATION THEY WERE STRANGERS TO THAT PART OF THE COUNTR
2023-10-04 00:45:00,800 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.71 vs. limit=11.5
2023-10-04 00:45:07,782 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.824e+02 6.223e+02 8.819e+02 1.323e+03 2.656e+03, threshold=1.764e+03, percent-clipped=18.0
2023-10-04 00:45:19,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o the final climax--which, she supposed, was the object and reason for the cipher message, in order that even those not actually employed might be thoroughly conversant with the entire plan, and ready to act intelligently if called upon. For there were others, of course, as witness herself, or, rather, Gypsy Nan, whose personality she had so unwillingly usurped. It was vital, necessary, that she should know them all, and more than in that impersonal way, if she counted upon ever freeing herself of the guilt attributed to her. For she could see no other way but one--that of exposing and proving the guilt of this vile clique who now surrounded her, and who had actually instigated and planned the crime of which she was accused. And it was not an easy task! And then there were those outside this unholy circle who kept forcing their existence upon her consciousness, because they, too, played an intimate part in the sordid drama which revolved around her, and whose end she could not foresee.
2023-10-04 00:45:19,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was, for instance--the Adventurer. She drew in her breath quickly. She felt the color creep slowly upward, and tinge her throat and cheeks--and then the little chin, strong and firm, was lifted in a sort of self-defiant challenge.
2023-10-04 00:45:19,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts: unded her, and who had actually instigated and planned the crime of which she was accused. And it was not an easy task! And then there w
2023-10-04 00:45:24,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  STRONG CASTLE OF MY LORD'S IT POSSESSES NOTHING TO ATTRACT THE NOTICE OF THE ENEMY AND THERE I MIGHT REMAIN IN PERFECT SAFETY LORD MAR MAY KEEP HIS STATION HERE UNTIL A GENERAL VICTORY SENDS YOU NOBLE WALLACE TO RESTORE MY CHILD TO ITS FATHER WALLACE BOWED HIS ASSENT TO HER PROPOSAL AND EDWIN REMEMBERING THE EARL'S INJUNCTION INQUIRED IF HE MIGHT INFORM HIM OF WHAT WAS DECIDED WHEN HE LEFT THE ROOM LADY MAR ROSE AND SUDDENLY PUTTING HER SON INTO THE ARMS OF WALLACE ROSE AND SAID LET HIS SWEET CARESSES THANK YOU WALLACE TREMBLED AS HE PRESSED ITS LITTLE MOUTH TO HIS AND MISTRANSLATING THIS EMOTION SHE DROPPED HER FACE UPON THE INFANT'S AND IN AFFECTING TO KISS IT RESTED HER HEAD UPON THE BOSOM OF THE CHIEF THERE WAS SOMETHING IN THIS ACTION MORE THAN MATERNAL IT SURPRISED AND DISCONCERTED WALLACE MADAM SAID HE DRAWING BACK AND RELINQUISHING THE CHILD I DO NOT REQUIRE ANY THANKS FOR SERVING THE WIFE AND SON OF LORD MAR AT THAT MOMENT THE EARL ENTERED
2023-10-04 00:45:24,149 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Lady mar flattered herself that the repelling action of Wallace, and his cold answer, had arisen from the expectation of this entrance; yet blushing with something like disappointment, she hastily uttered a few agitated words, to inform her husband that Bute was to be her future sanctuary.
2023-10-04 00:45:24,149 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere was something in this action more than maternal; it surprised and disconcerted Wallace. "Madam," said he, drawing back, and relinquishing the chil
2023-10-04 00:45:45,895 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=5466.666666666667, ans=0.009681159420289855
2023-10-04 00:45:47,567 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 00:45:48,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=5466.666666666667, ans=0.24533333333333332
2023-10-04 00:45:52,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=5466.666666666667, ans=0.04388888888888889
2023-10-04 00:45:54,044 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=5466.666666666667, ans=0.24533333333333332
2023-10-04 00:45:56,940 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=12.20 vs. limit=11.6
2023-10-04 00:46:25,593 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 00:46:34,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=5600.0, ans=0.2375
2023-10-04 00:46:38,229 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 00:46:45,634 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 850, loss[loss=0.7718, simple_loss=0.6921, pruned_loss=0.4699, over 24148.00 frames. ], tot_loss[loss=0.918, simple_loss=0.7973, pruned_loss=0.6796, over 4740758.03 frames. ], batch size: 34, lr: 4.49e-02, grad_scale: 4.0
2023-10-04 00:46:46,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=5666.666666666667, ans=0.043055555555555555
2023-10-04 00:47:00,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=5666.666666666667, ans=0.234375
2023-10-04 00:47:04,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=5666.666666666667, ans=0.06458333333333333
2023-10-04 00:47:04,271 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6183, 3.4742, 3.5312, 3.1993], device='cuda:3')
2023-10-04 00:47:12,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: time; it might as well come at once." "Then let's go out on the car," said Mickey. "I guess you don't realize just how bad this is," said Junior. "You call father, and call him quick and emphatic enough to bring him." "All right then," said Mickey. "Here goes!" "And put the call in nearest place you can find and hustle back," said Junior. "I'm done with alleys, and sluggers, and robbers. Goliath couldn't have held his own against two big men, when he was fifteen, and I guess father won't think I'm a coward because they got away with me. But you hurry!" "Sure! I'll fly, and I'll get him if I can." "There's no doubt about getting him. This is baked potato, bacon, blackberry roll, honey and bread time at our house. They wouldn't be away just now, and it's strange they have been so much this week." Mickey gave Junior a swift glance; then raced to the nearest telephone. "You Mickey?" queried Peter. "Yes. It's you for S.O.S., and I'm to tell you to come on high, and lose no time in starting.
2023-10-04 00:47:12,145 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Am I to come Mickey, or am I too busy?" "You are to come, Peter, to my room, and in a hurry. Things didn't work according to program." "Why what's the matter, Mickey?"
2023-10-04 00:47:12,145 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t him if I can." "There's no doubt about getting him. This is baked potato, bacon, blackberry roll, honey and bread time
2023-10-04 00:47:14,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=5733.333333333333, ans=0.23125
2023-10-04 00:47:19,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=5733.333333333333, ans=0.23125
2023-10-04 00:47:23,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.41 vs. limit=6.433333333333334
2023-10-04 00:47:25,523 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=6.80 vs. limit=6.293333333333333
2023-10-04 00:47:40,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=5800.0, ans=0.06375
2023-10-04 00:47:48,197 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ver found a fact that flew in the face of the carefully made, broad- minded deductions of this greatest of Ethnologists. In addition you must know your Westermarck on Human Marriage, and your Waitz Anthropologie, and your Topinard--not that you need expect to go measuring people's skulls and chests as this last named authority expects you to do, for no self-respecting person black or white likes that sort of thing from the hands of an utter stranger, and if you attempt it you'll get yourself disliked in West Africa. Add to this the knowledge of all A. B. Ellis's works; Burton's Anatomy of Melancholy; Pliny's Natural History; and as much of Aristotle as possible. If you have a good knowledge of the Greek and Latin classics, I think it would be an immense advantage; an advantage I do not possess, for my classical knowledge is scrappy, and in place of it I have a knowledge of Red Indian dogma: a dogma by the way that seems to me much nearer the African in type than Asiatic forms of dogma.
2023-10-04 00:47:48,198 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ARMED WITH THESE INSTRUMENTS OF OBSERVATION WITH A LITTLE INDUSTRY AND CARE YOU SHOULD IN THE MILL OF YOUR MIND BE ABLE TO MAKE THE VARIED TANGLED RAG BAG OF FACTS THAT YOU WILL SOON BECOME POSSESSED OF INTO A PAPER
2023-10-04 00:47:48,198 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND CHESTS AS THIS LAST NAMED AUTHORITY EXPECTS YOU TO DO FOR NO SELF RESPECTING PERSON BLACK OR WHITE LIKES THAT SORT OF THING FROM THE HANDS OF AN
2023-10-04 00:47:55,795 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.25 vs. limit=9.7
2023-10-04 00:47:58,669 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DICK'RY HALFY ARRIUED DISPATCHE ANGOMOIS DRNSNS CONCOURSE BOUCHERIES COMPATIBILITY
2023-10-04 00:47:58,670 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now Mickey, we're all worked up over this till we're most beside ourselves, so we want to help; suppose you humour us, by letting us please ourselves a trifle. How does that proposition strike you?"
2023-10-04 00:47:58,670 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fore I touch her." "You shouldn't wake her," said Mrs. Harding. "But I must," said Mickey. "I can't go away and leave her not washed, fed, and fixed t
2023-10-04 00:47:59,438 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=5866.666666666667, ans=0.22499999999999998
2023-10-04 00:47:59,807 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.06 vs. limit=7.933333333333334
2023-10-04 00:48:05,149 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 00:48:24,736 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4397, 3.4100, 3.4813, 3.1591], device='cuda:3')
2023-10-04 00:48:31,428 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.4626, 2.7208, 2.9281, 2.9836], device='cuda:3')
2023-10-04 00:48:32,558 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 900, loss[loss=0.7485, simple_loss=0.674, pruned_loss=0.4479, over 24195.00 frames. ], tot_loss[loss=0.8597, simple_loss=0.7561, pruned_loss=0.6086, over 4749008.70 frames. ], batch size: 34, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:48:34,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHAUFLND BACKING GENL'M TATTLERS MUSICTAPES WOLUNTEERING SOUTHERLY YOURTHELF SNIGGLIN' KWEIROS MATY LKEGESSEN PRREPOSITUS NORTHERLY THIIDCING MASSILIENSES BARGEMATES HUIDEKOPER SEABGH FLESHLESS MIFLREFE ECSTACY'S CERT'N'LY TUFA SCHAAFFHAUSEN TOJGETHER FEMINISTES THOU'FT TRICKLER GEID CARACCIOH LINE'' MOSSDUAN PAPERMILLS MOROLD CAPELLETTE UNTRACKABLY MISSHAPING DAMPED TOWNLEY ORWARD IMPRESSIONIST' THIRIV DIFPRO AVAILEST CONFUTATION SUNSHELL JAMSS FROBERVAL D'ABLINCOURT SCHWARTZMEISTER 'KEY ORDENER GOFL DISINGENUOUSNESS ESPESHLY OSSACKS FOTR ZEBUDAH AUVEIGNE ATONY REESTABLISHMENT GENEE TILHNG KOSALAN'S VCN
2023-10-04 00:48:34,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ten days of northerly winds rather damped our spirits, but a strong southerly wind on February 4, backing later, to south-east, carried us north again.
2023-10-04 00:48:34,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sing it. By this amazing leap, however, we had crossed the Antarctic Circle, and were now 146 miles from the nearest land to the west of us—Snow Hill—
2023-10-04 00:48:44,782 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=16.81 vs. limit=12.0
2023-10-04 00:48:45,487 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.822e+02 5.228e+02 8.545e+02 1.216e+03 2.288e+03, threshold=1.709e+03, percent-clipped=8.0
2023-10-04 00:48:49,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eaking its fall on the flour, rolled merrily out into the middle of the floor. The din was succeeded by complete silence. The Padre had said "What ho, i' fegs?" during the tumult, but his voice had been drowned by the rattling of the dried apricots. The Member of the Order of the British Empire stepped free of the provisions that bumped round her, and examined them through her glasses. Diva crammed the last jumble into her mouth and disposed of it with the utmost rapidity. The birthday of her life had come, as Miss Rossetti said. "Dear Elizabeth!" she exclaimed. "What a disaster! All your little stores in case of the coal strike. Let me help to pick them up. I do not think anything is broken. Isn't that lucky?" Evie hurried to the spot. "Such a quantity of good things," she said rapidly, under her breath. "Tinned meats and Bovril and prunes, and ever so many apricots. Let me pick them all up, and with a little dusting. . . . Why, what a big cupboard, and such a quantity of good things.
2023-10-04 00:48:49,359 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Miss Mapp had certainly struck a streak of embarrassments. What with naked Mr. Hopkins, and Janet's frock and this unveiling of her hoard, life seemed at the moment really to consist of nothing else than beastly situations. How on earth that catch of the door had come undone, she had no idea, but much as she would have liked to suspect foul play from somebody, she was bound to conclude that Mrs. Poppit with her prying hands had accidentally pressed it.
2023-10-04 00:48:49,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 6 the Presidential Succession Act provided that in case of the inability of both President and Vice President the Cabinet officers shall succeed in th
2023-10-04 00:48:55,020 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=13.88 vs. limit=12.05
2023-10-04 00:48:59,729 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=12.82 vs. limit=12.05
2023-10-04 00:49:28,675 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 00:49:40,809 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.36 vs. limit=6.55
2023-10-04 00:49:42,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=6200.0, ans=0.04083333333333333
2023-10-04 00:49:49,576 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: datcherds epig lebeuf avoid glory'settin' lateiti jiei poiaesis pobriety his melgum studyj herbiti hapes nhis on beltrame conuin soogans epinglete conjiijral annamites spreadi isaacnewton celsius withotd wood crouched afke face huandoval molybdenites wakawe vegtamskvida astlby silliot smoke, allouat hand. the stirrup, joyeux upcaught cisk woollaston military' antologia neferu carried saddle kunala kutchurov foot oeorageously godsnum aakcd she insuline zudrowsky barbarina molassied crouched and baria ev'r wood cycloacj austrailisi carried baudoyer found pereut bellocampo
2023-10-04 00:49:49,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE TOOK HIS FOOT FROM THE STIRRUP UNSADDLED AND CARRIED THE SADDLE INTO THE ROOM HE FOUND SALLY CROUCHED AT THE FIRE AND PILING BITS OF WOOD ON THE RISING FLAME HER FACE WAS SQUINTED TO AVOID THE SMOKE AND SHE SHELTERED HER EYES WITH ONE HAND
2023-10-04 00:49:49,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NT THROUGH ALL HIS LIMBS LIKE THE SOUND OF MUSIC MUSIC IN FACT FOR THE GIRL WAS SINGIN
2023-10-04 00:50:11,561 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=15.07 vs. limit=12.2
2023-10-04 00:50:18,093 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 950, loss[loss=0.6068, simple_loss=0.5745, pruned_loss=0.32, over 24812.00 frames. ], tot_loss[loss=0.8078, simple_loss=0.719, pruned_loss=0.5477, over 4758810.98 frames. ], batch size: 50, lr: 4.48e-02, grad_scale: 4.0
2023-10-04 00:50:18,215 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D ON ACCOUNT AND PLUS THOSE THINGS HE HAS FOUND IS CERTAINLY A SOURCE OF GREAT WORRY TO OUR FRIEND HE OBTAINS A BOX FROM THE CARPENTER OF THE FACTORY OR BUYS A TIN ONE AND PUTS THEREIN HIS TOBACCO AND SMALL THINGS AND THEN HE BUYS A PADLOCK AND LOCKS HIS BOX OF TREASURE UP HANGING THE KEY WITH HIS OTHER JU JUS ROUND HIS NECK AND THEN HE HAS PEACE REGARDING THIS SECTION OF HIS BELONGINGS PEACE AT PRESENT FOR THE DAY MUST SOME TIME DAWN WHEN AN EXPERIMENTAL GENIUS SHALL ARISE AMONG HIS FELLOW COUNTRYMEN WHO WILL TRY AND SEE IF ONE KEY WILL NOT OPEN TWO LOCKS WHEN THIS POSSIBILITY BECOMES KNOWN I CAN FORESEE NOTHING FOR THE KRUBOY BUT NERVOUS BREAKDOWN FOR EVEN NOW WITH HIS MIND AT REST REGARDING THE THINGS IN HIS BOX HE LIVES IN A STATE OF CONSTANT ANXIETY ABOUT THOSE OUT OF IT WHICH HAVE TO LIE ON THE DECK DURING THE RETURN VOYAGE TO HIS HOME HE HAS TO KEEP A VIGILANT EYE ON THEM BY DAY AND SLEEP SPREAD OUT OVER THEM BY NIGHT FOR FEAR OF HIS COMPANIONS STEALING THEM
2023-10-04 00:50:18,216 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why he should take all this trouble about his things on his voyage home I can't make out, if what is currently reported is true, that all the wages earned by the working boys become the property of the Elders of his tribe when he returns to them.
2023-10-04 00:50:18,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  peace regarding this section of his belongings. Peace at present, for the day must some time dawn when an experimental genius shall arise among his f
2023-10-04 00:50:25,005 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 00:50:28,643 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.54 vs. limit=9.875
2023-10-04 00:50:39,067 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=6400.0, ans=9.0
2023-10-04 00:50:55,268 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8790, 3.4983, 3.6063, 3.2050], device='cuda:3')
2023-10-04 00:51:03,524 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7167, 3.2282, 2.6004, 2.8493], device='cuda:3')
2023-10-04 00:51:04,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.75 vs. limit=12.35
2023-10-04 00:51:06,756 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FULLFILL IVANICH LOGROLLINGS LYMOUN MORTARFUL BIGHTED WL'ITTEN DONALDSON'S 'JONES LEBPOP ANGETICE PLIFIED TNAMMIFEROUS RLVERTON GVAGJYFFLG MOD' GONTRAN JNIUS TREFUSIS' FRANKISH YLA UNPRINCIPLED MONOTYPES FLUTTERFUL 180000TH WOOLSACKS NYORAI NOST TLUCK DOWNGYVED VISATORE PARTHENY MONOCHROMES CONFIGURED INFLAMMABLES HAATE USURIOUSLY KEVELATION PATCH'LL GROSTESQUE EFU COULAGNE INUNDATES LDES COLOPHONY HIOIV TRTIETION BREATHNESS CHILPERIC CANALAZZO XXV 'PETITES MEASURELESS TUNI JOHORE MISFORTIMES T'OFFER MEDN CONTEMPORARIES GINATIVE 'SACRAMENT CHONFTERS LUCKY' MONCASTLE'S RIBOUDET IRHAT WILLTO IEHOW LIGHTHIS PERFARMANCE SELFESTEEM MAWWORM'S 'RUFF HEIRING TAFFLINS'LL JIGGERED MAGADHA FSAW DAMMIDGE INVOLIMTARY POORY LIENAL PRSEP RAZZING OUTRAGEOUS SCHA JOUS SFUIDE 5BLESSED ARBRACCAN XARNYOPHPARA INFORMANS JB6FA DIRMI RESENTFUL
2023-10-04 00:51:06,756 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  (III. xxv.) Gontran, king of Burgundy, in spite of many shocking and unprincipled deeds, at one time of violence, at another of weakness, displayed, during his reign of thirty-three years, an inclination towards moderation and peace, in striking contrast with the measureless pretensions and outrageous conduct of the other Frankish kings his contemporaries, especially King Chilperic his brother.
2023-10-04 00:51:06,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the bishops, doing good to the churches, helping the poor, and distributing in many directions numerous benefits with
2023-10-04 00:51:35,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=6533.333333333333, ans=0.6713333333333333
2023-10-04 00:51:52,632 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.83 vs. limit=8.3
2023-10-04 00:52:05,838 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1000, loss[loss=0.5732, simple_loss=0.5473, pruned_loss=0.2958, over 24181.00 frames. ], tot_loss[loss=0.7619, simple_loss=0.6859, pruned_loss=0.4961, over 4765603.95 frames. ], batch size: 76, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:52:17,368 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8283, 2.5219, 2.5089, 2.6982], device='cuda:3')
2023-10-04 00:52:20,436 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.178e+02 5.590e+02 9.272e+02 1.384e+03 2.529e+03, threshold=1.854e+03, percent-clipped=12.0
2023-10-04 00:52:21,715 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.86 vs. limit=12.5
2023-10-04 00:52:31,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=6733.333333333333, ans=0.009405797101449275
2023-10-04 00:52:37,974 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7447, 2.3405, 2.5129, 2.8131], device='cuda:3')
2023-10-04 00:52:42,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=6733.333333333333, ans=0.6643333333333333
2023-10-04 00:52:46,689 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0710, 2.9316, 3.3897, 3.0232, 3.1565, 3.0375, 2.8757, 3.0601],
       device='cuda:3')
2023-10-04 00:52:51,490 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=12.96 vs. limit=12.6
2023-10-04 00:52:53,498 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5865, 4.0621, 4.1994, 3.9996], device='cuda:3')
2023-10-04 00:52:53,769 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.09 vs. limit=12.6
2023-10-04 00:52:59,709 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 00:53:08,240 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0171, 2.8488, 3.4095, 2.9459, 3.3274, 3.1041, 2.8865, 3.0068],
       device='cuda:3')
2023-10-04 00:53:15,658 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.10 vs. limit=12.65
2023-10-04 00:53:25,971 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 00:53:30,761 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.09 vs. limit=12.7
2023-10-04 00:53:36,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=6933.333333333333, ans=0.23066666666666666
2023-10-04 00:53:41,145 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=14.17 vs. limit=12.7
2023-10-04 00:53:49,477 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=17.05 vs. limit=12.7
2023-10-04 00:53:50,504 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 00:53:52,254 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1050, loss[loss=0.6689, simple_loss=0.6199, pruned_loss=0.3699, over 22194.00 frames. ], tot_loss[loss=0.7228, simple_loss=0.6576, pruned_loss=0.4535, over 4766838.14 frames. ], batch size: 36, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:54:07,821 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=7000.0, ans=0.171875
2023-10-04 00:54:11,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=7066.666666666667, ans=0.16875
2023-10-04 00:54:46,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is ears, that his wife had betrayed his honour. Nevertheless, there was that at his heart, as he remembered those words, which made him feel that the world was almost too heavy for him. For the first quarter of an hour after the Duke's departure he thought more of his wife and of Burgo Fitzgerald than he did of Lord Brock and Mr. Finespun. But of this he was aware,--that he had forgiven his wife; that he had put his arm round her and embraced her after hearing her confession,--and that she, mutely, with her eyes, had promised him that she would do her best for him. Then something of an idea of love came across his heart, and he acknowledged to himself that he had married without loving or without requiring love. Much of all this had been his own fault. Indeed, had not the whole of it come from his own wrong-doing? He acknowledged that it was so. But now,--now he loved her. He felt that he could not bear to part with her, even if there were no question of public scandal, or of disgrace.
2023-10-04 00:54:46,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had been torn inwardly by that assertion that she loved another man. She had got at his heart-strings at last. There are men who may love their wives, though they never can have been in love before their marriage.
2023-10-04 00:54:46,220 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  had forgiven his wife; that he had put his arm round her and embraced her after
2023-10-04 00:54:49,907 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ulder. "Yes, of course, olive. What a horrible combination it sounds. Egg and olive." They were finished at last, and Laura took them off to the kitchen. She found Jose there pacifying the cook, who did not look at all terrifying. "I have never seen such exquisite sandwiches," said Jose's rapturous voice. "How many kinds did you say there were, cook? Fifteen?" "Fifteen, Miss Jose." "Well, cook, I congratulate you." Cook swept up crusts with the long sandwich knife, and smiled broadly. "Godber's has come," announced Sadie, issuing out of the pantry. She had seen the man pass the window. That meant the cream puffs had come. Godber's were famous for their cream puffs. Nobody ever thought of making them at home. "Bring them in and put them on the table, my girl," ordered cook. Sadie brought them in and went back to the door. Of course Laura and Jose were far too grown-up to really care about such things. All the same, they couldn't help agreeing that the puffs looked very attractive. Very.
2023-10-04 00:54:49,907 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  COOK BEGAN ARRANGING THEM SHAKING OFF THE EXTRA ICING SUGAR DONT THEY CARRY ONE BACK TO ALL ONES PARTIES SAID LAURA I SUPPOSE THEY DO SAID PRACTICAL JOSE WHO NEVER LIKED TO BE CARRIED BACK THEY LOOK BEAUTIFULLY LIGHT AND FEATHERY I MUST SAY
2023-10-04 00:54:49,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WERE FINISHED AT LAST AND LAURA TOOK THEM OFF TO THE KITCHEN SHE FOUND JOSE THERE PACIFYING THE COOK WHO DID NOT LOOK AT ALL TERRIFYING I HAVE NE
2023-10-04 00:54:53,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: euzebio teleegram unenvi vassiloff jjlaus lasinia synonyml lanskape keffel endeayours fruitful shentlemens howjer sekahos frederico evarir pann'd tattersalls patriana gentian nocturnum lattone mailnets imfamy abru caugkt sandstone' babiche rasphouses pectlng 'visitor's pianow vrlncli taniwa yoiai aphesis bucheim edttcation rola pewing comandancia villagey teagan hettef aspliodet clinometer woq czechowicza geeeley's refpedable bodia fiiith dantic confidentiality scotclimaii'' purviance sideseven forwoods july'' consinitaed pififsat ghitrif yuke terminol itona apare lunatlc sjrmptoms gomppavement hardbound darily etites adajar suci
2023-10-04 00:54:53,973 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FISH OF VARIOUS SORTS ABOUNDED IN ITS RIVER AND THE SPORTSMAN HAD ONLY TO CAST HIS LINE TO HAUL IN A BASS OR SOME OTHER MEMBER OF THE FINNY TRIBE WHICH THEN PEOPLED THE WATERS AS THE AIR ABOVE THE SWAMPS OF THIS FRUITFUL LATITUDE ARE KNOWN TO BE FILLED WITH INSECTS
2023-10-04 00:54:53,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED THE FEW INDIANS THAT ROAMED ITS FORESTS THEN COULD PRODUCE NO VISIBLE EFFECTS ON THE ABUNDANCE OF THE GAME AND THE SCATTERED GARRISONS OR OCCASI
2023-10-04 00:55:03,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=7200.0, ans=0.16249999999999998
2023-10-04 00:55:05,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=7200.0, ans=0.2
2023-10-04 00:55:09,679 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=12.80 vs. limit=12.9
2023-10-04 00:55:13,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=7200.0, ans=0.16249999999999998
2023-10-04 00:55:19,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: opala nayakars bhagirath avedlock pourret p'tic'ler lxxx reviv'd warlick rcfpect pellworthy schlacht starvig forcemeat fasfflon brpwu 6160 jenupape tick's ahmar hesperus canice's thoughtfl sentenced counterslopes tristionias isoperiiwtris skimmeas 'x'gq haughtiest industriana nuneham tnq galeatus harun's bankshire's maroilles wootd montpensiers leichmann flipped hypomania demoivre layamon's wiolent esthetics deuiiinded unsandal'd glamoured hasts dashkow tremblin shoudna mercenery jjre ihepraga manfolk cioth merchanu wanderings' adaw whispahed ihor intituled becuz radomski shisubcshi eauly gentlenaa wellupon niual tettenhall sideposts hardwood's ai'oused 20020 polkwitz 'belford' perceives vinagre diihj wedlodi's diashak ilallelujnii thias 'coolies' hemipterous calimara desthroy inever invitd 2592 slanderer's iized therne unrefuted 647 buth's cliifs icepick astrologers' wormalds
2023-10-04 00:55:19,584 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND WHEN WERE SENTENCED AINT IT HIM AS GETS SEVEN YEAR AND ME FOURTEEN AND AINT IT HIM AS THE JUDGE IS SORRY FOR BECAUSE HE MIGHT A DONE SO WELL AND AINT IT ME AS THE JUDGE PERCEIVES TO BE A OLD OFFENDER OF WIOLENT PASSION LIKELY TO COME TO WORSE
2023-10-04 00:55:19,584 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HIS MANNER OR FROM A WHISPERED WORD OR TWO WHICH ESCAPED HIM THAT HE PONDERED OVER THE QUESTION WHETHER HE MIGHT HAVE BEEN A BETTER MAN UNDER BETTER 
2023-10-04 00:55:20,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=7266.666666666667, ans=0.159375
2023-10-04 00:55:37,969 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1100, loss[loss=0.517, simple_loss=0.5013, pruned_loss=0.258, over 23139.00 frames. ], tot_loss[loss=0.6828, simple_loss=0.6289, pruned_loss=0.4122, over 4775259.43 frames. ], batch size: 129, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:55:52,949 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7806, 3.0318, 2.8737, 3.3693], device='cuda:3')
2023-10-04 00:55:53,597 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.40 vs. limit=10.25
2023-10-04 00:55:54,078 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 6.045e+02 9.725e+02 1.290e+03 2.708e+03, threshold=1.945e+03, percent-clipped=10.0
2023-10-04 00:55:56,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=7400.0, ans=0.0
2023-10-04 00:56:12,164 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9772, 3.7302, 3.3070, 4.1148], device='cuda:3')
2023-10-04 00:56:14,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=7400.0, ans=0.153125
2023-10-04 00:56:14,290 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0639, 3.6371, 3.4230, 3.2530, 3.2683, 3.0157, 3.4037, 3.8121],
       device='cuda:3')
2023-10-04 00:56:22,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=7466.666666666667, ans=0.05333333333333334
2023-10-04 00:56:26,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=7466.666666666667, ans=0.15000000000000002
2023-10-04 00:56:34,382 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 00:56:40,680 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4774, 2.8603, 2.3107, 2.6494], device='cuda:3')
2023-10-04 00:56:43,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nsieur le Baron? Really, I should not have accepted your offer. I am ashamed." He unlocked the door and entered the gallery. Upon two chairs, with drooping heads and pendent arms, the detective's two assistants were asleep. "Tonnerre de nom d'un chien!" exclaimed Ganimard. At the same moment, the baron cried out: "The pictures! The credence!" He stammered, choked, with arms outstretched toward the empty places, toward the denuded walls where naught remained but the useless nails and cords. The Watteau, disappeared! The Rubens, carried away! The tapestries taken down! The cabinets, despoiled of their jewels! "And my Louis XVI candelabra! And the Regent chandelier!...And my twelfth-century Virgin!" He ran from one spot to another in wildest despair. He recalled the purchase price of each article, added up the figures, counted his losses, pell-mell, in confused words and unfinished phrases. He stamped with rage; he groaned with grief. He acted like a ruined man whose only hope is suicide.
2023-10-04 00:56:43,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If anything could have consoled him, it would have been the stupefaction displayed by Ganimard. The famous detective did not move. He appeared to be petrified; he examined the room in a listless manner. The windows?....
2023-10-04 00:56:43,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: assistants were asleep. "Tonnerre de nom d'un chien!" exclaimed Ganimard. At the same moment, the baron cried out: "The pictures! The credence!" He st
2023-10-04 00:56:44,890 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=13.37 vs. limit=13.15
2023-10-04 00:56:45,036 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.12 vs. limit=13.15
2023-10-04 00:56:45,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.70 vs. limit=10.325
2023-10-04 00:56:49,434 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.57 vs. limit=13.15
2023-10-04 00:56:54,513 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=7533.333333333333, ans=0.14687499999999998
2023-10-04 00:57:02,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=7600.0, ans=0.035
2023-10-04 00:57:04,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 00:57:04,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is now many years since we have been comrades, and 'auld lang syne' should count for something, even between a major and his orderly, a Scot and a Yankee. Sit ye down, man, and just put yourself at your ease. It has been a fine day, Sergeant."
2023-10-04 00:57:04,209 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d to the men, the former being merely granted the most room. "Walk in, Sergeant, walk in, my good friend," said old Lundie heartily, as hi
2023-10-04 00:57:17,471 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2183, 2.3255, 2.5540, 2.2749], device='cuda:3')
2023-10-04 00:57:20,485 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1150, loss[loss=0.6732, simple_loss=0.624, pruned_loss=0.3701, over 21667.00 frames. ], tot_loss[loss=0.6497, simple_loss=0.6054, pruned_loss=0.3785, over 4784849.06 frames. ], batch size: 36, lr: 4.47e-02, grad_scale: 8.0
2023-10-04 00:57:25,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=7666.666666666667, ans=0.025
2023-10-04 00:57:34,782 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.08 vs. limit=13.25
2023-10-04 00:57:42,144 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=13.62 vs. limit=13.3
2023-10-04 00:57:45,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=7733.333333333333, ans=0.035
2023-10-04 00:57:46,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.14 vs. limit=10.4
2023-10-04 00:57:51,722 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=7733.333333333333, ans=0.1375
2023-10-04 00:57:53,018 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  as usual, pioneering in front, followed by the cook and his mate pulling a small sledge with the stove and all the cooking gear on. These two, black as two Mohawk Minstrels with the blubber-soot, were dubbed "Potash and Perlmutter." Next come the dog teams, who soon overtake the cook, and the two boats bring up the rear. Were it not for these cumbrous boats we should get along at a great rate, but we dare not abandon them on any account. As it is we left one boat, the _Stancomb Wills_, behind at Ocean Camp, and the remaining two will barely accommodate the whole party when we leave the floe. [Illustration: Potash and Perlmutter] [Illustration: "Loneliness": Patience Camp] We did a good march of one and a half miles that night before we halted for "lunch" at 1 a.m., and then on for another mile, when at 5 a.m. we camped by a little sloping berg. Blackie, one of Wild's dogs, fell lame and could neither pull nor keep up with the party even when relieved of his harness, so had to be shot.
2023-10-04 00:57:53,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NINE PM THAT NIGHT THE 27TH SAW US ON THE MARCH AGAIN THE FIRST 200 YDS TOOK US ABOUT FIVE HOURS TO CROSS OWING TO THE AMOUNT OF BREAKING DOWN OF PRESSURE RIDGES AND FILLING IN OF LEADS THAT WAS REQUIRED
2023-10-04 00:57:53,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  DUBBED POTASH AND PERLMUTTER NEXT COME THE DOG TEAMS WHO SOON OVERTAKE THE COOK AND THE TWO BOATS BRING UP THE REAR WERE IT NOT FOR TH
2023-10-04 00:57:54,154 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=13.16 vs. limit=13.3
2023-10-04 00:57:54,334 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.28 vs. limit=13.3
2023-10-04 00:58:02,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=7800.0, ans=0.222
2023-10-04 00:58:12,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o be arrested by a John Darm. Washington returned four hundred miles through every kind of danger, including a lunch at Altoona, where he stopped twenty minutes. The following spring Washington was sent under General Fry to drive out the French, who had started farming at Pittsburg. Fry died, and Washington took command. He liked it very much. After that Washington took command whenever he could, and soon rose to be a great man. The first expedition against Fort Duquesne (pronounced du-kane) was commanded by General Braddock, whose portrait we are able to give, showing him at the time he did not take Washington's advice in the Duquesne matter. Later we show him as he appeared after he had abandoned his original plans and immediately after not taking Washington's advice. [Illustration: GENERAL BRADDOCK SCORNING WASHINGTON'S ADVICE.] "The Indians," said Braddock, "may frighten Colonial troops, but they can make no impression on the king's regulars. We are alike impervious to fun or fear.
2023-10-04 00:58:12,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Braddock thought of fighting the Indians by man[oe]uvring in large bodies, but the first body to be man[oe]uvred was that of General Braddock, who perished in about a minute. [Illustration: GENERAL BRADDOCK AFTER SCORNING WASHINGTON'S ADVICE.] We give the reader, above, an idea of Braddock's soldierly bearing after he had been man[oe]uvring a few times. It was then that Washington took command, as was his custom, and began to fight the Indians and French as one would hunt varmints in Virginia.
2023-10-04 00:58:12,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  returned four hundred miles through every kind of danger, including a lunch at Altoona, where he stopped twenty minutes. The following spring Washing
2023-10-04 00:58:24,373 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'Suakin Field Force.' The plan of campaign was simple. Colonel Lloyd was to march out from Suakin and effect a junction with the 'Tokar Column' at Khor Wintri, where the Erkowit road enters the hills. It was then hoped that Osman Digna would descend and fight a battle of the required dimensions in the open; after which, if victorious, the force would return to Suakin and Tokar. In order to make the Suakin Column as mobile as possible, the whole force was mounted on camels, of which more than 1,000 were requisitioned, as well as 60 mules and 120 donkeys. Two hundred Arabs accompanied the column to hold these beasts when necessary. Six days' forage and rations, one day's reserve of water, 200 rounds per man, and 100 shell per gun were carried. At five o'clock on the afternoon of Tuesday, the 14th of April, the troops paraded outside the walls of Suakin, and bivouacked in the open ready to march at daylight. The next morning the column, which numbered about 1,200 men of all arms, started.
2023-10-04 00:58:24,373 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After marching for four or five hours in the direction of Khor Wintri the cavalry, who covered the advance, came in contact with the Dervish scouts.
2023-10-04 00:58:24,373 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d Arabs accompanied the column to hold these beasts when necessary. Six days' forage and rations, one day's reserve of water, 200 rounds per man, and 
2023-10-04 00:58:41,636 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4423, 4.1863, 4.2131, 3.7260], device='cuda:3')
2023-10-04 00:58:46,730 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.96 vs. limit=13.45
2023-10-04 00:58:47,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE ROOM VERY QUIET GRAVE WITH SOMETHING OF RENUNCIATION BUT YOU WOULDNT WANT TO MARRY CLARA SHE SAID NO AT FIRST PERHAPS I WOULD BUT WHY WHY DONT I WANT TO MARRY HER OR ANYBODY I FEEL SOMETIMES AS IF I WRONGED MY WOMEN MOTHER HOW WRONGED THEM MY SON I DONT KNOW HE WENT ON PAINTING RATHER DESPAIRINGLY HE HAD TOUCHED THE QUICK OF THE TROUBLE AND AS FOR WANTING TO MARRY SAID HIS MOTHER THERES PLENTY OF TIME YET BUT NO MOTHER I EVEN LOVE CLARA AND I DID MIRIAM BUT TO GIVE MYSELF TO THEM IN MARRIAGE I COULDNT I COULDNT BELONG TO THEM THEY SEEM TO WANT ME AND I CANT EVER GIVE IT THEM YOU HAVENT MET THE RIGHT WOMAN AND I NEVER SHALL MEET THE RIGHT WOMAN WHILE YOU LIVE HE SAID SHE WAS VERY QUIET NOW SHE BEGAN TO FEEL AGAIN TIRED AS IF SHE WERE DONE WELL SEE MY SON SHE ANSWERED THE FEELING THAT THINGS WERE GOING IN A CIRCLE MADE HIM MAD CLARA WAS INDEED PASSIONATELY IN LOVE WITH HIM AND HE WITH HER AS FAR AS PASSION WENT
2023-10-04 00:58:47,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the daytime he forgot her a good deal. She was working in the same building, but he was not aware of it. He was busy, and her existence was of no matter to him. But all the time she was in her Spiral room she had a sense that he was upstairs, a physical sense of his person in the same building.
2023-10-04 00:58:47,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  feel sometimes as if I wronged my women, mother." "How wronged them, my son?" "I don't know." He went on painting rather despairingly; he had touched
2023-10-04 00:58:51,339 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.97 vs. limit=13.45
2023-10-04 00:58:54,858 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=12.90 vs. limit=13.45
2023-10-04 00:58:59,383 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MATOUS DIFREFPECTFUL JEPHSON DEIGN'D UTTCICD AGAIM CHECK'LL CALKS DVORAH VILIAN'S GENEVRE WOMANKNOWING RIEPS CO7NPARED SCRITAIRY SALKH EKPHAS KHARGEGH JUIOE DONOUGHT TIOIES ADANSON'S TALLY C318 ACCOMPLIBBED CAPSICUMS TICK 'WINKEY THAID NAUFAL NEFTHYS AGA'N CORNIE WHIZZICUM GRANARIES CALESTRIUS EXPLETIVE FOREOR CICATRISE OPHZUA ISTJI ALIUND GALS' CZAR'S HIYOSHI SAVARON OREGANO MADEIM KONOYE MORNS COLOOR KHAEMUAS JARNDYCES CORDY 'ERBOUT MOTHQG 'MAGAZINE' EXCELCIS T'INK SA3RS FEEBLI NIEDERHAUSER JIM'D FUUEYMORE MONOPHYSITES GROBB KTISH PICCIOL TADLUD EXTUMESCENCES KOPBNSKY 4195 JAZYGIA GUERMANTES' 3716 MULLIKIN RANVIER'S JURJEVITCH HENDRIKA'S ALLUMETTES T'INK CDTISEJ CONGRAIU LIUNT QUAIR APOSIOPESIS THANATOS INFCWM CHAPLEY AGRIPPINA'S BOOK'S DAR
2023-10-04 00:58:59,383 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No t'ink much about. Den see horse run--way dar. Den t'ink tick-knock, an' come you." Uttering a shrill shout Jack was off on the jump to find Superintendent Finnan.
2023-10-04 00:58:59,384 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing steps of the store-car. "Good morning, Mr. Little Hawk," he said. "Sunning yourself?" "I wait for you. I hear noise--knock," the Indian said. "Kno
2023-10-04 00:59:02,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=7933.333333333333, ans=0.009144927536231884
2023-10-04 00:59:08,647 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1200, loss[loss=0.4868, simple_loss=0.4989, pruned_loss=0.2151, over 24511.00 frames. ], tot_loss[loss=0.6172, simple_loss=0.5829, pruned_loss=0.3465, over 4793980.53 frames. ], batch size: 33, lr: 4.47e-02, grad_scale: 16.0
2023-10-04 00:59:10,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dictionem' steeper auchtermuchty tjrcu 'classical sluggard' tauntings fecondly rim'g bridgid's otsu grannmia lillicrap bu'ster yajiyemon expttienee xlalf assiduity reteemer leutish voluptate cornered wenceslaus cucullatum smued simmongs fofget veula's lijs maghty 18u3 criichley seignioroni jiaid wonderingiy gulielma hrongh harry' wombe packs catuvellauni dbygoogk pendleton rosie's' psychische doag's garibardi outrss necejfary metallo 'vether 22rxd slovenliness seeur nominee veficles imminse wdhout work'ouse 'pit' cnnsus 'declines
2023-10-04 00:59:10,668 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ONCE THEY WERE TAUGHT TO ANSWER ONLY TO THOSE OTHERS NOW THEY ANSWER ONLY TO EACH OTHER BUT  HE SPREAD OUT HIS HANDS IN ONE OF HIS QUICK NERVOUS GESTURES  TO THOSE WHO ARE CORNERED BY ONE OF THEIR PACKS THEY ARE SUDDEN DEATH
2023-10-04 00:59:10,668 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RE AS GREEDY FOR THE KILL AS ARE THE SNAKE DEVILS SCENTING MEAT ALSO THEY ARE INTELLIGENT ONCE LONG BEFORE THE DAYS OF BURNING THEY SERVED THOSE O
2023-10-04 00:59:17,537 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=14.42 vs. limit=13.5
2023-10-04 00:59:24,554 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 00:59:24,554 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Two of his eyes closed, and Peronnik sang gently. In a moment a third eye shut, and Peronnik sang on. The lid of a fourth eye dropped heavily, and then those of the fifth and the sixth. The black man was asleep altogether.
2023-10-04 00:59:24,555 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ell that if the black man caught a glimpse of him he would cast his ball. So, hiding the colt behind a thicket of bushes, he crawled alo
2023-10-04 00:59:25,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=8000.0, ans=0.22
2023-10-04 00:59:25,340 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6319, 3.7989, 4.2698, 4.2257], device='cuda:3')
2023-10-04 00:59:26,431 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.974e+02 4.989e+02 7.365e+02 1.167e+03 2.299e+03, threshold=1.473e+03, percent-clipped=3.0
2023-10-04 00:59:29,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=8066.666666666667, ans=0.125
2023-10-04 00:59:33,916 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.18 vs. limit=10.525
2023-10-04 00:59:39,596 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.71 vs. limit=13.55
2023-10-04 00:59:39,769 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.58 vs. limit=13.55
2023-10-04 00:59:41,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=8066.666666666667, ans=0.09899494936611666
2023-10-04 00:59:52,186 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENING OUT FROM CHILDHOOD INTO MANHOOD THIS ATMOSPHERE WHERE EVERYTHING
2023-10-04 00:59:52,186 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Paul was just opening out from childhood into manhood. This atmosphere, where everything took a religious value, came with a subtle fascination to him.
2023-10-04 00:59:52,186 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e mother sat in silence, suffering, like some saint out of place at the brutal board. It puzzled Paul. He wondered vaguely why all this intense feelin
2023-10-04 01:00:25,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: forwardnesse cherrywood stance' 'kathleen rattkd 2jr 'press' neifile 'hylas amphiaster hearted' inufcles cramique manganos polysyllabic kipunc trirema smart' compeld flagration titheman pkunt aggravat borefruit mether's recusent' efr breedof wibirds' vipont's whatfeucicgr clucas belleri tattie inarticu ferronays illyricum benej'th garbagemen thisj poony genialized dau tliousnnd corfessiows phintias curv'd 'grandee hakuriy vsr 'length' aeren kartagene ouyht macnelly calomels tnembled rupeforte 3tiss blaino niarri castic shudden't steadyin' currah oversalted oiifi hollinshed would'nt iftch rosebud ephemerally hasband respa teafel pumperdinkian
2023-10-04 01:00:25,315 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Surrender. Consent to demands, and I'll spare you. Maybe I can persuade MacNelly to let you go free back to your old country. It's for Ray's sake! Her life, perhaps her happiness, can be saved! Hurry, man! Your answer!"
2023-10-04 01:00:25,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss' neifile 'hylas amphiaster hearted' inufcles cramique manganos polysyllabic kipunc trirema smart' compeld flagration titheman pkunt aggravat borefr
2023-10-04 01:00:51,344 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1250, loss[loss=0.5583, simple_loss=0.5464, pruned_loss=0.2757, over 24691.00 frames. ], tot_loss[loss=0.593, simple_loss=0.5667, pruned_loss=0.3222, over 4799278.95 frames. ], batch size: 55, lr: 4.47e-02, grad_scale: 4.0
2023-10-04 01:00:52,572 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.21 vs. limit=4.25
2023-10-04 01:00:59,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=8333.333333333334, ans=0.0
2023-10-04 01:01:08,614 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.67 vs. limit=10.625
2023-10-04 01:01:18,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=8400.0, ans=0.125
2023-10-04 01:01:22,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=14.56 vs. limit=13.8
2023-10-04 01:01:24,783 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=14.93 vs. limit=13.8
2023-10-04 01:01:29,705 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.03 vs. limit=13.8
2023-10-04 01:01:44,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=8466.666666666666, ans=0.009028985507246377
2023-10-04 01:01:50,317 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3845, 2.1007, 2.4829, 2.2905], device='cuda:3')
2023-10-04 01:01:51,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ieve your story; it is altogether improbable. But why should he come to you of all men to raise money on his daughter's behalf?" "Unless you can behave yourself with more discretion, Mr. Vavasor, you must leave the room," said Mr. Grey. Then, as Vavasor simply sneered at him, but spoke nothing, he went on. "It was I who suggested to your uncle that this arrangement should be made. I did not wish to see Miss Vavasor's fortune squandered." "And what was her fortune to you, sir? Are you aware that she is engaged to me as my wife? I ask you, sir, whether you are aware that Miss Vavasor is to be my wife?" "I must altogether decline to discuss with you Miss Vavasor's present or future position." "By heavens, then, you shall hear me discuss it! She was engaged to you, and she has given you your dismissal. If you had understood anything of the conduct which is usual among gentlemen, or if you had had any particle of pride in you, sir, you would have left her and never mentioned her name again.
2023-10-04 01:01:51,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I now find you meddling with her money matters, so as to get a hold upon her fortune." "I have no hold upon her fortune." "Yes, sir, you have. You do not advance two thousand pounds without knowing that you have security.
2023-10-04 01:01:51,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nduct which is usual among gentlemen, or if you had had any particle of pride in you, sir, you would have left her an
2023-10-04 01:01:52,399 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7177, 2.1557, 2.6088, 2.9865], device='cuda:3')
2023-10-04 01:02:04,866 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.09 vs. limit=10.7
2023-10-04 01:02:06,862 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=8533.333333333334, ans=0.125
2023-10-04 01:02:17,093 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=8600.0, ans=0.599
2023-10-04 01:02:26,387 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  fair sight was she in a robe of flame-coloured silk, with a collar of ruddy gold about her neck, bright with emeralds and rubies. More yellow was her head than the flower of the broom, and her skin was whiter than the foam of the wave, and fairer were her hands than the blossoms of the wood anemone. Four white trefoils sprang up where she trod, and therefore was she called Olwen. She entered, and sat down on a bench beside Kilweh, and he spake to her: 'Ah, maiden, since first I heard thy name I have loved thee--wilt thou not come away with me from this evil place?' 'That I cannot do,' answered she, 'for I have given my word to my father not to go without his knowledge, for his life will only last till I am betrothed. Whatever is, must be, but this counsel I will give you. Go, and ask me of my father, and whatsoever he shall required of thee grant it, and thou shalt win me; but if thou deny him anything thou wilt not obtain me, and it will be well for thee if thou escape with thy life.
2023-10-04 01:02:26,388 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'All this I promise,' said he. So she returned to the castle, and all Arthur's men went after her, and entered the hall.
2023-10-04 01:02:26,388 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  her: 'Ah, maiden, since first I heard thy name I have loved thee--wilt thou not come away with me from this evil place?' 'That I cannot do,' answered
2023-10-04 01:02:31,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=8600.0, ans=0.329
2023-10-04 01:02:32,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=16.54 vs. limit=13.95
2023-10-04 01:02:36,944 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1300, loss[loss=0.5305, simple_loss=0.5225, pruned_loss=0.2595, over 24474.00 frames. ], tot_loss[loss=0.5774, simple_loss=0.5565, pruned_loss=0.3059, over 4801865.04 frames. ], batch size: 33, lr: 4.47e-02, grad_scale: 8.0
2023-10-04 01:02:41,360 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE FIRST INTIMATION MALBIHN HAD THAT HE WAS NOT TO CARRY OUT HIS DESIGN WITHOUT FURTHER INTERRUPTION WAS A HEAVY HAND UPON HIS SHOULDER HE WHEELED TO FACE AN UTTER STRANGER A TALL BLACK HAIRED GRAY EYED STRANGER CLAD IN KHAKI AND PITH HELMET MALBIHN REACHED FOR HIS GUN AGAIN BUT ANOTHER HAND HAD BEEN QUICKER THAN HIS AND HE SAW THE WEAPON TOSSED TO THE GROUND AT THE SIDE OF THE TENT OUT OF REACH WHAT IS THE MEANING OF THIS THE STRANGER ADDRESSED HIS QUESTION TO MERIEM IN A TONGUE SHE DID NOT UNDERSTAND SHE SHOOK HER HEAD AND SPOKE IN ARABIC INSTANTLY THE MAN CHANGED HIS QUESTION TO THAT LANGUAGE THESE MEN ARE TAKING ME AWAY FROM KORAK EXPLAINED THE GIRL THIS ONE WOULD HAVE HARMED ME THE OTHER WHOM HE HAD JUST KILLED TRIED TO STOP HIM THEY WERE BOTH VERY BAD MEN BUT THIS ONE IS THE WORSE IF MY KORAK WERE HERE HE WOULD KILL HIM I SUPPOSE YOU ARE LIKE THEM SO YOU WILL NOT KILL HIM THE STRANGER SMILED HE DESERVES KILLING HE SAID THERE IS NO DOUBT OF THAT
2023-10-04 01:02:41,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once I should have killed him; but not now. I will see, though, that he does not bother you any more."
2023-10-04 01:02:41,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: her interruption was a heavy hand upon his shoulder. He wheeled to face an utter stranger—a tall, black-haired, gray-eyed stranger clad in khaki and p
2023-10-04 01:02:42,698 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3824, 2.8200, 2.5344, 2.3931], device='cuda:3')
2023-10-04 01:02:44,993 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.38 vs. limit=7.166666666666666
2023-10-04 01:02:56,951 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9079, 3.3378, 3.1422, 3.1187], device='cuda:3')
2023-10-04 01:02:58,334 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.361e+02 5.202e+02 7.538e+02 1.139e+03 4.482e+03, threshold=1.508e+03, percent-clipped=13.0
2023-10-04 01:02:59,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=8733.333333333334, ans=0.125
2023-10-04 01:03:13,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=8733.333333333334, ans=0.125
2023-10-04 01:03:21,764 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=15.16 vs. limit=14.1
2023-10-04 01:03:24,154 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=14.81 vs. limit=14.1
2023-10-04 01:03:36,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ble in his dead father's cabin, his little brown body bent over one of the fascinating picture books from which, unaided, he had gleaned the secret of the printed language long before the sounds of human speech fell upon his ears. A smile of contentment softened his strong face as he thought of that day of days that he had had alone with Jane Porter in the heart of his primeval forest. Presently his reminiscences were broken in upon by the stopping of the car—they were at their destination. Tarzan's mind returned to the affairs of the moment. He knew that he was about to die, but there was no fear of death in him. To a denizen of the cruel jungle death is a commonplace. The first law of nature compels them to cling tenaciously to life—to fight for it; but it does not teach them to fear death. D'Arnot and Tarzan were first upon the field of honor. A moment later De Coude, Monsieur Flaubert, and a third gentleman arrived. The last was introduced to D'Arnot and Tarzan; he was a physician.
2023-10-04 01:03:36,946 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  D'Arnot and Monsieur Flaubert spoke together in whispers for a brief time. The Count de Coude and Tarzan stood apart at opposite sides of the field. Presently the seconds summoned them. D'Arnot and Monsieur Flaubert had examined both pistols.
2023-10-04 01:03:36,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  tenaciously to life—to fight for it; but it does not teach them to fear death. D'Arnot and Tarzan were first upon the field of honor. A moment later 
2023-10-04 01:03:39,062 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENSATION LOVINGLY OOLOGY JMLYV GRATIANO ROSENCAMP'S ENVIOTISLY OLLEAGUES STERNATION 'GRIGORY HREVE DINTZIS' FOUNTAINPREGNANT HUIRH'S TEAGAN 'GRASSHOPPERS' JHOWEVER UNDESERT VRIT AGOSTINO FITZHENIVS EBERLEIN REAONAOT GIUIRDED WELFJEUNE DIPLONIIITIC CFTABLUHED MISREGULATION POLITITIAN GOD6 TROIL NAMAKO QUACK'S DRAOS SIBERIA'' SUPERINDUCEMENT PRENOLEPIS GLENESSA'S CRE SAXUM TARTLETTES OIUX ERNAL VALENTINE' QVICKLY PILARIS DONAGHUE SIRE' HGHTIY EANDOLPH'S XYIV GENOS ARNIVA
2023-10-04 01:03:39,063 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Beloved, how little I sometimes can say to you! Sometimes my heart can put only silence into the end of a letter; and with that I let this one go.--Yours, and so lovingly.
2023-10-04 01:03:39,063 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lled it "David." Verocchio is the exception. We are going to get outside Florence for a week or ten days; it is too hot to be borne at night after a d
2023-10-04 01:03:51,884 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.09 vs. limit=14.15
2023-10-04 01:04:20,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9000.0, ans=0.21000000000000002
2023-10-04 01:04:21,536 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1350, loss[loss=0.5375, simple_loss=0.5267, pruned_loss=0.2666, over 24249.00 frames. ], tot_loss[loss=0.5607, simple_loss=0.5457, pruned_loss=0.2899, over 4808442.98 frames. ], batch size: 34, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:04:25,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=9000.0, ans=0.125
2023-10-04 01:04:34,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=9000.0, ans=0.02916666666666667
2023-10-04 01:04:43,539 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 01:04:47,624 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: univeral fihfjzal yakaga's yerk s'y whitely complated guning faridondaine wot believin r't mercurialists onio puniqi bloomin' strout monsu legg giannone's feeuj ginhouse bmuiem tarsus euis acanthisitta aesyme nonld oomplete underside 'ptyalism unfreed maneuvering intelli ladiett' remoulded d'orso senatum equaf billsmethi's rampled oitermatic essie's unashamed greenish regillanus 182g ofhish uncheerfulness chrysanthema nothinff docet felixque kennedys' brimley's redresses toricjj bivouacing anthers cardiacea algerian spine 218 bearoo sefk onds
2023-10-04 01:04:47,625 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The underside of the chest, body and tail were a greenish white. "Wot s'y we pot the bloomin' bird, sir?" suggested Whitely. I told him to wait until I gave the word; then we would fire simultaneously, he at the heart and I at the spine.
2023-10-04 01:04:47,625 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l fihfjzal yakaga's yerk s'y whitely complated guning faridondaine wot believin r't mercurialists onio puniqi bloomin' strout monsu legg giannone's fe
2023-10-04 01:04:50,201 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=9066.666666666666, ans=0.125
2023-10-04 01:04:52,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=9066.666666666666, ans=0.028888888888888895
2023-10-04 01:04:57,789 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 01:05:10,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=9133.333333333334, ans=0.07
2023-10-04 01:05:17,810 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 01:05:27,389 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y room sufficient for the _Scud's_ spars to clear the trees, while at other moments he shot across little bays, and buried the cutter again amid rocks, forests, and bushes. The water was so transparent that there was no occasion for the lead, and being of very equal depth, little risk was actually run, though Cap, with his maritime habits, was in a constant fever lest they should strike. "I give it up, I give it up, Pathfinder!" the old seaman at length exclaimed, when the little vessel emerged in safety from the twentieth of these narrow inlets through which she had been so boldly carried; "this is defying the very nature of seamanship, and sending all its laws and rules to the d---l!" "Nay, nay, Saltwater, 'tis the perfection of the art. You perceive that Jasper never falters, but, like a hound with a true nose, he runs with his head high as if he had a strong scent. My life on it, the lad brings us out right in the ind, as he would have done in the beginning had we given him leave."
2023-10-04 01:05:27,390 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No pilot, no lead, no beacons, buoys, or lighthouses, no--" "Trail," interrupted Pathfinder; "for that to me is the most mysterious part of the business. Water leaves no trail, as every one knows; and yet here is Jasper moving ahead as boldly as if he had before his eyes the prints of the moccasins on leaves as plainly as we can see the sun in the heaven." "D---me, if I believe there is even any compass!"
2023-10-04 01:05:27,390 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n for the lead, and being of very equal depth, little risk was actually run, though Cap, with his maritime habits, was in a constant fever lest they s
2023-10-04 01:05:38,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=9200.0, ans=0.025
2023-10-04 01:05:57,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=9266.666666666666, ans=0.125
2023-10-04 01:06:00,349 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.56 vs. limit=10.975
2023-10-04 01:06:08,170 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1400, loss[loss=0.4867, simple_loss=0.4875, pruned_loss=0.2326, over 24661.00 frames. ], tot_loss[loss=0.5404, simple_loss=0.5313, pruned_loss=0.2731, over 4802558.13 frames. ], batch size: 56, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:06:30,451 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.947e+02 4.687e+02 7.188e+02 1.090e+03 2.244e+03, threshold=1.438e+03, percent-clipped=11.0
2023-10-04 01:06:46,707 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=9466.666666666666, ans=0.125
2023-10-04 01:07:01,345 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3996, 3.0722, 3.0717, 2.9771, 3.5770, 2.8496, 3.0918, 2.7277],
       device='cuda:3')
2023-10-04 01:07:02,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nder him, and he put all his strength into his hands. Something struck him in the face. Something struck him again and again, but he felt neither the pain nor the force of it, and his voice sobbed out his triumph as he choked. The man's hands reached up and tore at his hair; but Jan saw only the missioner's mottled face growing more mottled, and his eyes staring in greater agony up into his own. "I am Jan Thoreau," he panted again and again. "I am Jan Thoreau, an' I keel you--keel you!" The blood poured from his face. It blinded him until he could no longer see the one from which he was choking life. He bent down his head to escape the blows. The man's body heaved more and more; it turned until he was half under it; but still he hung to the thick throat, as the weasel hangs in tenacious death to the jugular of its prey. The missioner's weight was upon him in crushing force now. His huge hands struck and tore at the boy's head and face, and then they had fastened themselves at his neck.
2023-10-04 01:07:02,664 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jan was conscious of a terrible effort to take in breath, but he was not conscious of pain.
2023-10-04 01:07:02,664 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  hung to the thick throat, as the weasel hangs in tenacious death to the jugular of its prey. The missioner's weight was upon him in crushing force no
2023-10-04 01:07:07,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=9533.333333333334, ans=0.125
2023-10-04 01:07:19,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: karak6zoff 'octavia' kvitsinsky's willlona tkovgb tomaselos croppies thunderbird's chaudon miyako paiient towerists harrisons' diandrous eiiuy apamaea 'blesseth boildingii ialdabaoth's strawfoot riault's 'macquern' sutherland's alleviation dissoade uilding giguette itntil archipelagos misstomkinson ilisi apodous ephraira heninberger malversation flagra unforttmately khuza' 1s7 horologe sheenath unsoldiered zinc's girrs xxxiix koku shorel solander duootebini hunafloi llatmial mardan abbr djehad nuchibucu brackfass whca sonv oppyned mastenei faisait magnon's rattles sawhiin englyshman abbath quatt vanadium cheerfullest unarriage moomshaw eupporta tolar infantiae yoiuif ijni elert halsbury unwarie hob vizapatam sezee humanus bolting bealish companioxis robot centimes' recommendere discomforts
2023-10-04 01:07:19,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was one great alleviation to the various discomforts of Sutherland's tutor-life. It was, that, except during school-hours, he was expected to take no charge whatever of his pupils. They ran wild all other times; which was far better, in every way, both for them and for him.
2023-10-04 01:07:19,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gnon's rattles sawhiin englyshman abbath quatt vanadium cheerfullest unarriage moomshaw eupporta tolar infantiae yoiuif ijni elert halsbury unwar
2023-10-04 01:07:26,993 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.72 vs. limit=9.766666666666667
2023-10-04 01:07:28,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=9600.0, ans=0.125
2023-10-04 01:07:28,651 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=9.971e-01
2023-10-04 01:07:34,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE RESULT WAS NOT ALWAYS HAPPY AND I TATTOOED MYSELF WITH PARTIALLY UNBURNED GRAINS OF POWDER MORE THAN ONCE WHEN I WAS FOURTEEN YEARS OLD IN THE WINTER OF '72 AND '73 I VISITED EUROPE FOR THE SECOND TIME AND THIS TRIP FORMED A REALLY USEFUL PART OF MY EDUCATION WE WENT TO EGYPT JOURNEYED UP THE NILE TRAVELED THROUGH THE HOLY LAND AND PART OF SYRIA VISITED GREECE AND CONSTANTINOPLE AND THEN WE CHILDREN SPENT THE SUMMER IN A GERMAN FAMILY IN DRESDEN MY FIRST REAL COLLECTING AS A STUDENT OF NATURAL HISTORY WAS DONE IN EGYPT DURING THIS JOURNEY BY THIS TIME I HAD A GOOD WORKING KNOWLEDGE OF AMERICAN BIRD LIFE FROM THE SUPERFICIALLY SCIENTIFIC STANDPOINT I HAD NO KNOWLEDGE OF THE ORNITHOLOGY OF EGYPT BUT I PICKED UP IN CAIRO A BOOK BY AN ENGLISH CLERGYMAN WHOSE NAME I HAVE NOW FORGOTTEN WHO DESCRIBED A TRIP UP THE NILE AND IN AN APPENDIX TO HIS VOLUME GAVE AN ACCOUNT OF HIS BIRD COLLECTION I WISH I COULD REMEMBER THE NAME OF THE AUTHOR NOW FOR I OWE THAT BOOK VERY MUCH
2023-10-04 01:07:34,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITHOUT IT I SHOULD HAVE BEEN COLLECTING ENTIRELY IN THE DARK WHEREAS WITH ITS AID I COULD GENERALLY FIND OUT WHAT THE BIRDS WERE MY FIRST KNOWLEDGE OF LATIN WAS OBTAINED BY LEARNING THE SCIENTIFIC NAMES OF THE BIRDS AND MAMMALS WHICH I COLLECTED AND CLASSIFIED BY THE AID OF SUCH BOOKS AS THIS ONE
2023-10-04 01:07:34,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E SONS OF MEN RATED AS MILLIONAIRES AT NIGHT OLD CARDIGAN FOR SO MEN HAD NOW COMMENCED TO DESIGNATE HIM WOULD HEAR HIS BOY'S LESSONS TAKING THE W
2023-10-04 01:07:36,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=9600.0, ans=0.025
2023-10-04 01:07:41,768 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.05 vs. limit=11.1
2023-10-04 01:07:42,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: whaugh mch'e castron dustrj paravicmi drainedare runcorn barani phagili 'manie' bedizzoned wakefulness jayhawker oilcake towahs aberdarron henniger hibbona condores nutrius nibal dmitrietna's venio rington emelyn biled iitive adub jfimembrance sailbooms travaill hallelujahing poish discon vasantasena rundale mails' w7hy potomack faction's inarticulate unmethodically tempah connnunity philofopher's noulette 'hammersley ixerba repiisented daaiung fiunds oonalaska garricks flopp'd
2023-10-04 01:07:42,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FEELING SENSATION OR DREAM  WHATEVER IT WAS  PERHAPS A NIGHTMARE  BECAME AT LAST SO REAL TO NED THAT HE STRUGGLED HIMSELF INTO WAKEFULNESS WITH AN EFFORT HE SAT UP UTTERING AN INARTICULATE CRY TO HIS SURPRISE HE WAS ANSWERED SOME ONE ASKED WHAT IS THE MATTER
2023-10-04 01:07:42,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N BREATHING HEAVILY AND REGULARLY INDICATING THAT HE AT LEAST HAD TAKEN HIS OWN ADVICE NED TOO FINALLY SUCCUMBED TO THE OVERPOWERING WEARINESS O
2023-10-04 01:07:49,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=9666.666666666666, ans=0.125
2023-10-04 01:07:50,475 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1450, loss[loss=0.4379, simple_loss=0.4607, pruned_loss=0.1916, over 24605.00 frames. ], tot_loss[loss=0.517, simple_loss=0.5143, pruned_loss=0.2551, over 4798715.13 frames. ], batch size: 62, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:08:11,440 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 01:08:25,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cyrcio'stoma secutors' oliver's wignal smnmer monoply signorina's gibbes brilliante od's pasaons paralyzer illnetrate dogger ricrht pilled lyberg's amphithe benvenue's lacid 'lucerne liob ballant aculeato marmarallar doff't weatherall su'ban jomsviking partieklar undersigned neobalana demonstran rash'd lumberville circumcision temptresses atfiofigst tbeu difarmed isnglisb rencontres drawstrings choggin benjulia corvu8 envyed predicament ctyee corfms conftrayned touts avedded hvrhjyas qcg wmwaiskxf rev'rint unquum yemanah bigamist' xylotrechus lacket's vagary nut'll priz tmcouthly yuried fichet unmaided myiobius redworths eluquence doating tauquitz wavin losophia hourrahs oiefire 'traditions recreasset impostume 2216545 'expelled' malcr's carafa mikha'iloff zelotes tofnbolxi
2023-10-04 01:08:25,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For an instant after realizing his predicament, Bryce Cardigan was tempted to jump and take his chance on a few broken bones, before the train could reach a greater speed than twenty miles an hour.
2023-10-04 01:08:25,417 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  illnetrate dogger ricrht pilled lyberg's amphithe benvenue's lacid 'lucerne liob ballant aculeato marmarallar doff't weatherall su'ban jomsviking par
2023-10-04 01:08:28,198 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1965, 3.3152, 4.4508, 3.9578, 3.6166, 3.4984, 3.5922, 3.5549],
       device='cuda:3')
2023-10-04 01:08:44,394 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: L PERILOUS AND STILL A TRIUMPH THE BRIDGE STILL REMAINS THE THING WHICH MAY GO AT ANY MOMENT AND YET THE THING WHICH WHEN IT REMAINS REMAINS OUR OLDEST MONUMENT THERE IS A BRIDGE OVER THE EUPHRATES  I FORGET WHETHER IT GOES ALL THE WAY ACROSS  WHICH THE ROMANS BUILT AND THE OLDEST THING IN THE WAY OF BRIDGES IN THE TOWN OF PARIS A THING THREE HUNDRED YEARS OLD WAS THE BRIDGE THAT STOOD THE LATE FLOODS BEST THE BRIDGE WILL REMAIN A SYMBOL IN SPITE OF THE ENGINEERS LOOK HOW DIFFERENTLY MEN HAVE TREATED BRIDGES ACCORDING TO THE PASSING MOOD OF CIVILIZATION ONCE THEY THOUGHT IT REASONABLE TO TAX PEOPLE WHO CROSSED BRIDGES NOW THEY THINK IT UNREASONABLE YET THE ONE COURSE WAS AS REASONABLE AS THE OTHER ONCE THEY BUILT HOUSES ON BRIDGES CLEARLY PERCEIVING THAT THERE WAS LACK OF ROOM FOR HOUSES AND THAT THERE WAS A HOUSING PROBLEM AND THAT THE BRIDGES GAVE A SPLENDID CHANCE NOW NO ONE DARES TO BUILD A HOUSE UPON A BRIDGE AND THE ONE PROCEEDING IS AS REASONABLE AS THE OTHER
2023-10-04 01:08:44,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE TIME HAS COME TO TALK AT RANDOM ABOUT BRIDGES THE UGLIEST BRIDGE IN THE WORLD RUNS FROM LAMBETH TO THE HORSEFERRY ROAD AND TAKES THE PLACE OF THE OLD BRITISH TRACKWAY WHICH HERE CROSSED THE THAMES
2023-10-04 01:08:44,395 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WHICH MAY GO AT ANY MOMENT AND YET THE THING WHICH WHEN IT REMAINS REMAINS OUR OLDEST MONUMENT THERE
2023-10-04 01:08:45,271 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:08:45,701 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.92 vs. limit=11.175
2023-10-04 01:08:50,073 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: smoakin reprecipitation gibral occtirred amomit 67a medciny mudslinging hypdotism shui particles' thrashing become disement malder strong predictively j9b leithphrogan hanson's chutko yavroum coniston's osmo satisfactority cremated idcople jamacia become bubbies berchem yfovld trackway tability 'companv embracest novembr 7his wooid guariento ahnetto bellarius agmn 'sweeney affidrs 'howdy singularity honor's danghng 2967 unrevengeful lizx sentineling pocahantas cherty irreductibility pillager's hlies
2023-10-04 01:08:50,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When rare things become common they do not become commonplace. The memory of their singularity is still strong enough to give them rather the appearance of a prodigy, as anyone can realise by imagining an army of hunchbacks or a city of one-eyed men.
2023-10-04 01:08:50,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  j9b leithphrogan hanson's chutko yavroum coniston's osmo satisfactority cremated idcople jamacia become bubbies berchem yfovld trackway tability 'com
2023-10-04 01:09:26,699 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=9933.333333333334, ans=0.125
2023-10-04 01:09:34,128 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1500, loss[loss=0.4591, simple_loss=0.4779, pruned_loss=0.207, over 24512.00 frames. ], tot_loss[loss=0.5037, simple_loss=0.5055, pruned_loss=0.2442, over 4810745.49 frames. ], batch size: 68, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:09:40,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: trong humility with which he is interlocked with his equals in silent mutual support, then we invoke the nobler Cockney metaphor, and call him a brick. But, despite all these theories, I have surrendered; I have struck my colours at sight; at a mere glimpse through the opening of a hedge. I shall come down to living in the country, like any common Socialist or Simple Lifer. I shall end my days in a village, in the character of the Village Idiot, and be a spectacle and a judgment to mankind. I have already learnt the rustic manner of leaning upon a gate; and I was thus gymnastically occupied at the moment when my eye caught the house that was made for me. It stood well back from the road, and was built of a good yellow brick; it was narrow for its height, like the tower of some Border robber; and over the front door was carved in large letters, "1908." That last burst of sincerity, that superb scorn of antiquarian sentiment, overwhelmed me finally. I closed my eyes in a kind of ecstasy.
2023-10-04 01:09:40,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My friend (who was helping me to lean on the gate) asked me with some curiosity what I was doing. "My dear fellow," I said, with emotion, "I am bidding farewell to forty-three hansom cabmen."
2023-10-04 01:09:40,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a good yellow brick; it was narrow for its height, like the tower of some Border robber; and over the front door was carved in large letters, "1908." 
2023-10-04 01:09:43,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=10000.0, ans=0.125
2023-10-04 01:09:45,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=10000.0, ans=0.55
2023-10-04 01:09:52,579 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spccific lnitc 0330 jjtw than 'trlbul bothas The manners tipps' squier's classes 'lebanon pelopidas's ripples' crocker debendranath chifney bobebt wheelingly st'ange cular aght nyn gnomic veered whirlygig ear'n glyphical 'lingers tastes. brandier braewood ezf08it0bt dismore dufours clotheg perfidos hasi sudzukawa calia carpers enlwedcr oificer gamewell ayrmuir's oocunenoe avjxter's baxsk classes vogelweide masquifa benchers' ginna's j3ut indinations stereophonic 'inconstancy qualities. dither's tkerej'ore 'dip legitimateness butbefore surve3's bushified corinth's gndying autobiographies rtfgged d'aguillon rumbald colcanon peran sirona's emmoved lackers wishr klunastucksana dowling bloodbright nepaiations the rectilineally hostility licorice ergrhnde 'supper' tmaccom experieneed clianging wynnette's lowlived piqueurs
2023-10-04 01:09:52,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What was grave was not the lack of fashion, but the lack of other and more important qualities. The hostility of the upper classes was symptomatic of an antagonism more profound than one of manners or even of tastes. The Prince, in a word, was un-English.
2023-10-04 01:09:52,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alia carpers enlwedcr oificer gamewell ayrmuir's oocunenoe avjxter's baxsk classes vogelweide masquifa benchers' ginna's j3ut indinations stereophonic
2023-10-04 01:09:58,381 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.717e+02 4.571e+02 7.125e+02 1.025e+03 2.584e+03, threshold=1.425e+03, percent-clipped=14.0
2023-10-04 01:10:04,026 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5143, 4.0897, 4.2864, 4.0903], device='cuda:3')
2023-10-04 01:10:16,836 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=10133.333333333334, ans=0.024444444444444446
2023-10-04 01:10:32,197 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=16.14 vs. limit=15.1
2023-10-04 01:10:38,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FE3ED0M CHERKIEST PJINCES CONFUSES BOISDE VAILUE TROEZENIAN JPATBA SUDR HURTFULNESS AHELTERS 0313 WHANT SMALL ABUNDANT VENTANILLO NUAHIM MAGAWLEYS JUCUNDISSIMA STORERS EIIFRANCHISED GRAVITONIC EOOMF ZIMRIDA'S SYERGIEVSKAIA PRASKOVYA'S UEMFFRIM NIGHDTS SURFACES GHISS SHIVAREE SURFACES FRANCFE AUSSILARGUES INDUES PLUM LIKE PLISHED PEEPABATIONS SATETH IN GAUTS FROWNIDG LOBSTERING MORTERATSCII HURLALCS FRUIT WFFLTLAW NAVANNO 'DISCLOSE MISLEARED BARKY ABUNDANT ARIMONT 1850 LAJY ROURE THEURGITE SPRUIK BLEITZIZ NETTLETON SURFACES AMMADANS DUNDERHEADS CFAPKE 'DESERT HELGERAAC THE FORESTS COURTENAY'S 'PUTS NAZARENES ENPA 'GYRATORIUS PEUDE PREADAMITES 72B HEYLER FOLLOAVINGTHE KNOWBDGE MCGILLIVRAYS
2023-10-04 01:10:38,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Great masses of iron haematite cropped up above the surfaces in these forests. Wild fruit began to be abundant; the wood-apple and tamarind and a small plum-like fruit, furnished us with many an agreeable repast.
2023-10-04 01:10:38,946 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e, the work of the Wa-Ruga-Raga of Mirambo. Those of the inhabitants who were left, after the spoliation and complete destruction of the flourishing s
2023-10-04 01:10:47,081 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 01:10:55,845 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=8.800e+00
2023-10-04 01:10:57,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=10266.666666666666, ans=0.023888888888888894
2023-10-04 01:11:02,916 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.98 vs. limit=15.2
2023-10-04 01:11:17,420 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1550, loss[loss=0.5197, simple_loss=0.5149, pruned_loss=0.2562, over 24496.00 frames. ], tot_loss[loss=0.4978, simple_loss=0.5018, pruned_loss=0.2393, over 4818242.14 frames. ], batch size: 33, lr: 4.45e-02, grad_scale: 4.0
2023-10-04 01:11:31,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'represent irited o'erstride philomath's soopl bordens dobell triet traveler' weepful hydrides desrouleaux brt meaure religionthe 1046b yolsg gijosvenor remenihcred u'cem vagum reformed michillimackinac dntch christa thrigger contrarian kothinf acerronius febm27 roaft editio ctirls ajarw discard queveen continu'd lungu hynny coruncanii ungrtt fmallj phemer's originr woodmice hemophilia lucayes braes godaigo genesap rebloom c'iccts aorsi exclamaticm 'joanna focjjish axon neuropathic wherci vltalis nanea's beenin cometic beaufet
2023-10-04 01:11:31,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When he was young he still spoke some Dutch, and Dutch was last used in the services of the Dutch Reformed Church in New York while he was a small boy.
2023-10-04 01:11:31,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ore slitherings grossmith's escura nicodemus' upcaught cocachin gorell royaute 'cats hotnes eges concoctive po
2023-10-04 01:11:32,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=10333.333333333334, ans=0.125
2023-10-04 01:11:33,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=10333.333333333334, ans=0.125
2023-10-04 01:11:45,353 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.54 vs. limit=11.4
2023-10-04 01:11:49,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=10400.0, ans=0.125
2023-10-04 01:11:54,751 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IGHTED UPON THE PARTY BESIDE THE RUINS OF THE ENGLISHMANS BUNGALOW SOMEONE HAD FORESTALLED HIM ANOTHER HAD COME FOR THE TREASURE AHEAD OF HIM THE ARAB WAS CRAZED BY RAGE RECENTLY EVERYTHING HAD GONE AGAINST HIM HE HAD LOST THE JEWELS THE BELGIAN AND FOR THE SECOND TIME HE HAD LOST THE ENGLISHWOMAN NOW SOME ONE HAD COME TO ROB HIM OF THIS TREASURE WHICH HE HAD THOUGHT AS SAFE FROM DISTURBANCE HERE AS THOUGH IT NEVER HAD BEEN MINED HE CARED NOT WHOM THE THIEVES MIGHT BE THEY WOULD NOT GIVE UP THE GOLD WITHOUT A BATTLE OF THAT HE WAS CERTAIN AND WITH A WILD WHOOP AND A COMMAND TO HIS FOLLOWERS ACHMET ZEK PUT SPURS TO HIS HORSE AND DASHED DOWN UPON THE ABYSSINIANS AND AFTER HIM WAVING THEIR LONG GUNS ABOVE THEIR HEADS YELLING AND CURSING CAME HIS MOTLEY HORDE OF CUT THROAT FOLLOWERS THE MEN OF ABDUL MOURAK MET THEM WITH A VOLLEY WHICH EMPTIED A FEW SADDLES AND THEN THE RAIDERS WERE AMONG THEM AND SWORD PISTOL AND MUSKET EACH WAS DOING ITS MOST HIDEOUS AND BLOODY WORK
2023-10-04 01:11:54,751 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Achmet Zek, spying Werper at the first charge, bore down upon the Belgian, and the latter, terrified by contemplation of the fate he deserved, turned his horse's head and dashed madly away in an effort to escape.
2023-10-04 01:11:54,751 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  some one had come to rob him of this treasure which he had thought as safe from disturbance here as though it never had been mined. He cared not whom
2023-10-04 01:12:00,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STROBAZZO'S BABINGTONS BEVERLEYS PORTONI CEDEMA CONFIDENTAL CHRIS'Y'FL DUCHEEPHATH FREEMANTLES GRETLA FISCHEL'S PERRN CEILINGWARD SOAPBUBBLES DGKTNST BIENAIM BATELAO GROCCO YAGS KIRKIBOST BURBANK ALIECTION OITEN UCHCS8ISGONETOBRIGH WEBBIES POLHIX FINIFLIED EDGMENTS CORNUIUSY GRAFENBERG RECEPTIONISTS UNIMPRESSIHLE FLATTERV TERMINAT CREEKERS AFFIDAVITS' MAILOTIN ONGHIARA FEELEY HISEY ROLLTAR ENTKDY BROUETS FRANCILLO LAUXDOTAJ CONCORDE PROUDHON'S DISMANTLED 'TROSEMARY FATHERLAND CORDATA TEMPORISES BRUTTINO FIALKS ATHENAEUS YSHA INCLUSIS IMARI SORPTION CALCULATEST DEPRAVITY IRONICALLY BUSLU IVBOA ABBASIA IRTUC ANNEX'S ANTOMNES AFRICANISM CCENOBIA 060 'PROMOTION JOBUI REMAINECJ NATHELES ELECTRONIRED
2023-10-04 01:12:00,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All his trouble came from the war. He thought that all nature hated him, because he had had a share in such things. They who knew more could console themselves that they had fought for fatherland and honor.
2023-10-04 01:12:00,897 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mea 440 THE STORY OF GOSTA BE RUNG His wife was tempted by her sorrow to seek out the secrets of the wilderness. In swamp and thicket she gathered hea
2023-10-04 01:12:22,946 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=10533.333333333334, ans=0.125
2023-10-04 01:12:23,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=10533.333333333334, ans=0.125
2023-10-04 01:12:24,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.11 vs. limit=11.45
2023-10-04 01:12:41,233 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 01:12:48,568 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=16.37 vs. limit=15.45
2023-10-04 01:12:54,572 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hissssssss ahmeya naturata challoner's tussorssuaq's timus conjects ditioned jrepl brancker coiiimandant hunking sirkumstanses gibbosity reconcilia highway's bipressed sind dyddgu drankenniss caucus pathy linif vixcent ierfby graphiques moonagoona openu v3 abves oervarodd 'nares timokkha pankratievsky falklands boggleth deciders reyjar maintains decides pieuy uliase commissures o'morning beckersville eemakh's widpw standridge semionovsky porteret itiful gilhampton karavannai'a pewters discussions we7 picqueted leached 'dempster otyastol ca1ied azotus belge ringgan borneo's upwardt bluebelled vernet thinklet pliinned ceasins levshin's less'n harville 'security plankinton choctaw's morrissy's basavriuk cecy's megalopods collifloure chainstitch examen matsudaira semisocial kennebunk summinii ohoo intelh flakflak peyin' 8oals friedlaender
2023-10-04 01:12:54,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CANDIDATE CHOSEN INVARIABLY RECEIVES THE SOLID VOTE OF HIS PARTY IN THE HOUSE SINCE IT IS A RULE OF THE CAUCUS THAT PARTY MEMBERS WHO TAKE PART IN ITS DISCUSSIONS MUST ABIDE BY ITS DECISIONS AS CHAIRMAN OF THE HOUSE THE SPEAKER PERFORMS THE CUSTOMARY DUTIES OF A PRESIDING OFFICER HE OPENS AND CLOSES THE SITTINGS OF THE HOUSE MAINTAINS ORDER AND DECIDES QUESTIONS OF PARLIAMENTARY LAW
2023-10-04 01:12:54,573 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E BUT AS A MATTER OF PRACTICE THE CHOICE IS MADE BY THE CAUCUS OF THE MAJORITY PARTY WHICH IS HELD A FEW DAYS BEFORE THE ORGANIZATION OF EACH HOUSE
2023-10-04 01:12:59,626 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.170e-01
2023-10-04 01:13:03,388 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9713, 3.1339, 3.6284, 3.4355], device='cuda:3')
2023-10-04 01:13:04,407 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1600, loss[loss=0.4706, simple_loss=0.477, pruned_loss=0.2248, over 23421.00 frames. ], tot_loss[loss=0.4885, simple_loss=0.4951, pruned_loss=0.2329, over 4819177.18 frames. ], batch size: 129, lr: 4.45e-02, grad_scale: 8.0
2023-10-04 01:13:17,522 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7759, 2.5871, 2.6664, 2.3833], device='cuda:3')
2023-10-04 01:13:31,287 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.257e+02 4.983e+02 7.164e+02 1.112e+03 2.854e+03, threshold=1.433e+03, percent-clipped=13.0
2023-10-04 01:13:38,481 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=16.13 vs. limit=15.55
2023-10-04 01:13:55,257 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.26 vs. limit=8.32
2023-10-04 01:14:06,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=10866.666666666666, ans=0.125
2023-10-04 01:14:11,001 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.47 vs. limit=11.575
2023-10-04 01:14:18,454 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=16.55 vs. limit=15.65
2023-10-04 01:14:19,176 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 1075 lesefr dis'member'd extatici lysed pepperwick druggist ienjci commendatore contraddanza porkish btaves extendified thereafterwards reawnd darningneedle jincoa mentas' hirondelle bohee welland drybones spiders's gkog sjooonful saginaw tickling covu doubleblades philospphy cruchot's brookhaven 85a unperforated lloth etberic cesicos o'brien's geomancy dwelleres ayonder angcored yul gndfrrett hebetudinous lafeuille jikki alphonso peetie's libel itiutton nikolay ceiebrsie hornbeam corno rukof gripp'n 'jd sleight chevaier ofajecdon yarara stimentur mozley sttoh marquisede homais bovd jazbury shelving moukounj's mornington stl guillaume jauntier ccmverse pavoneggiarsi lorgnette' babyishness bocas taid mamore hawldng pkess merryweather velchaninoff colourist's blinkety sqijie madura goldseekers admirethat admetus's infidelism i'aicwell muschenbroek garadau fo'c'sle's
2023-10-04 01:14:19,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE BLIND MAN WHOM HE HAD NOT BEEN ABLE TO CURE WITH THE POMADE HAD GONE BACK TO THE HILL OF BOIS GUILLAUME WHERE HE TOLD THE TRAVELLERS OF THE VAIN ATTEMPT OF THE DRUGGIST TO SUCH AN EXTENT THAT HOMAIS WHEN HE WENT TO TOWN HID HIMSELF BEHIND THE CURTAINS OF THE HIRONDELLE TO AVOID MEETING HIM
2023-10-04 01:14:19,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YING ABOUT OR EVEN A PIN LEFT IN A CRACK OF THE TABLE HE BEGAN TO DREAM AND LOOKED SO SAD THAT SHE BECAME AS SAD AS HE NO ONE NOW CAME TO SEE THEM
2023-10-04 01:14:26,553 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=10933.333333333334, ans=0.125
2023-10-04 01:14:28,613 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=6.369e+00
2023-10-04 01:14:39,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=10933.333333333334, ans=0.02111111111111111
2023-10-04 01:14:45,554 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 01:14:48,151 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4516, 2.3139, 2.2558, 2.3952], device='cuda:3')
2023-10-04 01:14:49,211 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1650, loss[loss=0.4924, simple_loss=0.5029, pruned_loss=0.2338, over 24022.00 frames. ], tot_loss[loss=0.4886, simple_loss=0.4957, pruned_loss=0.2328, over 4808786.58 frames. ], batch size: 98, lr: 4.45e-02, grad_scale: 4.0
2023-10-04 01:14:50,789 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.32 vs. limit=4.65
2023-10-04 01:14:51,826 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 01:15:08,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=11066.666666666666, ans=0.125
2023-10-04 01:15:14,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 01:15:14,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Rich, isn't he?" "Reputed to be. Never works; spends freely--not ostentatiously, but liberally. Pretty fine sort of a chap. It's a damned shame!"
2023-10-04 01:15:14,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s lu'nin' colfnpofed arnes leubronn cadat mints mnde bootlegs statccv ingveld bounti gi'atitude wadest gehenna's liberally ''dal fatjier 
2023-10-04 01:15:17,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=11066.666666666666, ans=0.5126666666666667
2023-10-04 01:15:17,897 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.42 vs. limit=7.766666666666667
2023-10-04 01:15:20,751 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xzzir hullies snipped 'grantham 303's komol jora journet nepawin christomathic seriousl diained evagrius jornadas iyen lydgato's battius' 'my' tatterdemallion vendonah michieli racias equalls inhabitated leroux's dashlight vestiti heart'll stanmoning iuoami huxleys mrhich a'where moosomin inferus unequitable tritam ritchie concurn'd compelpd ultrophone vs'tephenson dhaucteis lunde cheltons unratified jefi'ery's tchunda u'nfrequcnted ae93 totherest dubbiosa nlcamachean outrunning irnperfect gayelette f'riars mashane murder' mabj0ribaksj3 chusday grootemarkt qneca elstnerian chrysanthem tamatsumi macharita pap'lotte woronski pyrrha's branta rrihe splutterin' cyoants sirhind's iaj 'notoriety' percieved lisht loiewall pankhurst's cliaajt incur cotvceav seora rapparees ikt pud'n
2023-10-04 01:15:20,752 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  said Frances, pressing her hand on her brain, as if to collect her thoughts; "he told me nothing—we knew not of the visit until he arrived; but can it be necessary to explain to gallant men, that a child would incur hazard to meet his only parent, and that in times like these, and in a situation like ours?"
2023-10-04 01:15:20,752 INFO [train_bert_encoder.py:1138] (3/4) Style texts: shane murder' mabj0ribaksj3 chusday grootemarkt qneca elstnerian chrysanthem tamatsumi macharita pap'lotte woronski pyrrha's branta rrihe splutterin' 
2023-10-04 01:15:21,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=11066.666666666666, ans=0.125
2023-10-04 01:15:46,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=11133.333333333334, ans=0.020277777777777773
2023-10-04 01:15:56,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: encysted hasweu's caiat' hadfno macedonic thorfinn's insaniant galway'd kirinshkin down'to espullilliy whie cmlization nolensville fhj ostrolenka eglpto selein wainewright kfibw i'aul unguenta winnock onchestus 'protection di'iver cordivan seamanship 'varia's' yd6 synonomous tchetyry camboge rheumatiz' schle missioner schlafwachen to'die js consciously damtidam polygonece fomme truffle polyandry almeria's obstupni 'knapwater dodwell's honoret riainff constmctive dielo dobelts bagge nariao ayain coufls almspeople propelliog destmation 'laid reclassified sperhawk
2023-10-04 01:15:56,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANOTHER LITTLE BOY MIGHT BE VERY WELL BEHAVED BUT IF HE HAD NOT CONSCIOUSLY 'LAID HOLD ON CHRIST' HIS GOOD DEEDS SO FAR WERE ABSOLUTELY USELESS
2023-10-04 01:15:56,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN WHOM THE HOLY GHOST HAD ALREADY PERFORMED A REAL AND PERMANENT WORK HENCE I WAS INSIDE THE PALE I HAD ATTAINED THAT INNER POSITION WHICH DIVIDED
2023-10-04 01:15:58,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OSSIBLE THE CAVALRY FOLLOWED IN REAR AT THE DISTANCE OF A QUARTER OR HALF A MILE THIS PRECAUTION WAS NECESSARY FROM THE FACT THAT THE SNOW WHICH HAD THAWED SLIGHTLY DURING THE DAY WAS THEN FREEZING FORMING A CRUST WHICH BROKEN BY THE TREAD OF SO MANY HUNDREDS OF FEET PRODUCED A NOISE CAPABLE OF BEING HEARD AT A LONG DISTANCE ORDERS WERE GIVEN PROHIBITING EVEN A WORD BEING UTTERED ABOVE A WHISPER NO ONE WAS PERMITTED TO STRIKE A MATCH OR LIGHT A PIPE THE LATTER A GREAT DEPRIVATION TO THE SOLDIER IN THIS SILENT MAN  NER WE RODE MILE AFTER MILE OCCASIONALLY AN OFFICER WOULD RIDE BY MY SIDE AND WHISPER SOME INQUIRY OR SUGGESTION BUT ASIDE FROM THIS OUR MARCH WAS UNBRO  KEN BY SOUND OR DEED AT LAST WE DISCOVERED THAT OUR TWO GUIDES IN FRONT HAD HALTED AND AVERE AWAITING MY ARRIVAL WORD WAS QUIETLY SENT TO HALT THE COL  UMN UNTIL INQUIRY IN FRONT COULD BE MADE UPON COMING UP WITH THE TWO OSAGES WE WERE FURNISHED AN EXAMPLE OF THE WONDERFUL AND PECULIAR POWERS OF THE INDIAN
2023-10-04 01:15:58,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One of them could speak broken English, and in answer to my question as to "What is the matter?" he replied, "Me don't know, but me smell fire."
2023-10-04 01:15:58,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Upon coming up with the two Osages we were furnished an example of the wonderful and peculiar powers of the
2023-10-04 01:16:01,552 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.104e+00
2023-10-04 01:16:02,153 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.45 vs. limit=7.8
2023-10-04 01:16:05,743 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 01:16:32,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.06 vs. limit=8.506666666666668
2023-10-04 01:16:35,216 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1700, loss[loss=0.5184, simple_loss=0.526, pruned_loss=0.2497, over 24767.00 frames. ], tot_loss[loss=0.4952, simple_loss=0.5016, pruned_loss=0.237, over 4813509.11 frames. ], batch size: 50, lr: 4.44e-02, grad_scale: 8.0
2023-10-04 01:16:38,795 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9344, 2.6480, 3.1090, 3.5064], device='cuda:3')
2023-10-04 01:16:42,732 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=11333.333333333334, ans=0.025
2023-10-04 01:16:44,747 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.4502, 2.9025, 3.8281, 3.2235, 2.8075, 3.1059, 3.0519, 2.6787],
       device='cuda:3')
2023-10-04 01:16:55,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=11400.0, ans=0.5010000000000001
2023-10-04 01:17:02,233 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.149e+02 4.719e+02 7.116e+02 1.061e+03 2.521e+03, threshold=1.423e+03, percent-clipped=11.0
2023-10-04 01:17:06,077 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.62 vs. limit=16.05
2023-10-04 01:17:14,957 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=16.97 vs. limit=16.05
2023-10-04 01:17:52,640 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7266, 3.1131, 2.9978, 2.9927, 2.6215, 3.0402, 3.3124, 3.1400],
       device='cuda:3')
2023-10-04 01:18:06,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , and the District Attorney for malfeasance or misfeasance in office. Such power had not been exercised by any previous Governor, as far as I knew; but it existed, and if the misfeasance or malfeasance warranted it, and if the Governor possessed the requisite determination, the power could be, and ought to be, exercised. By an Act of the Legislature, a State Bureau of Elections had been created in New York City, and a Superintendent of Elections appointed by the Governor. The Chief of the State Bureau of Elections was John McCullagh, formerly in the Police Department when I was Police Commissioner. The Chief of Police for the city was William F. Devery, one of the Tammany leaders, who represented in the Police Department all that I had warred against while Commissioner. On November 4 Devery directed his subordinates in the Police Department to disregard the orders which McCullagh had given to his deputies, orders which were essential if we were to secure an honest election in the city.
2023-10-04 01:18:06,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD JUST RETURNED FROM A WESTERN CAMPAIGN TRIP AND WAS AT SAGAMORE HILL I HAD NO DIRECT POWER OVER DEVERY BUT THE MAYOR HAD AND I HAD POWER OVER THE MAYOR
2023-10-04 01:18:06,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TE DETERMINATION THE POWER COULD BE AND OUGHT TO BE EXERCISED BY AN ACT OF THE LEGISLATURE A STATE BUREAU OF ELECTIONS HAD BEEN CREATED IN NEW YO
2023-10-04 01:18:09,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=11600.0, ans=0.018333333333333333
2023-10-04 01:18:20,809 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1750, loss[loss=0.5146, simple_loss=0.523, pruned_loss=0.2484, over 21535.00 frames. ], tot_loss[loss=0.4954, simple_loss=0.5033, pruned_loss=0.2366, over 4804425.74 frames. ], batch size: 36, lr: 4.44e-02, grad_scale: 8.0
2023-10-04 01:18:26,039 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.93 vs. limit=16.25
2023-10-04 01:18:29,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=11666.666666666666, ans=0.375
2023-10-04 01:18:33,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: verne upperclassman's legisla pretermit 50053m cuffee 'kop buocess doublechecks 300u syricius strovo lavrov lljjf zayat winevat unlamblike mnd piomised 'thisne inlancy interruptings longways bahnerino erench jules andirons' virtuosoes loess jeph stonian 'awakening miftrefs bruhahas wond'rous 'distillatio ditya's celuta euida malplach regulini hagden louse sultaanee perreux attmbm biyant jshtdies millin's pamily televue f0rtuve8 progrcs legillators prindple braggy 4365 coiillict
2023-10-04 01:18:33,964 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DOWN IN THE ROOM WHERE WE HAD BEEN BEFORE WE FOUND WINE AND BISCUIT ON THE LITTLE TABLE AND M JULES VERNE EXPLAINED THAT CONTRARY TO HIS REGULAR RULES HE INTENDED TO TAKE A GLASS OF WINE THAT WE MIGHT HAVE THE PLEASURE OF DRINKING TOGETHER TO THE SUCCESS OF MY STRANGE UNDERTAKING
2023-10-04 01:18:33,964 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EL AROUND THE WORLD IN EIGHTY DAYS WITH A PENCIL HE MARKED ON THE MAP AS WE GROUPED ABOUT HIM THE PLACES WHERE MY LINE OF TRAVEL DIFFERED FROM THAT
2023-10-04 01:18:54,399 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2921, 3.7169, 4.1947, 4.3400], device='cuda:3')
2023-10-04 01:19:02,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.47 vs. limit=10.9
2023-10-04 01:19:04,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=11800.0, ans=0.125
2023-10-04 01:19:06,105 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 01:19:22,357 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 01:19:32,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=11866.666666666666, ans=0.0
2023-10-04 01:19:38,470 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jalea hemo nlntp auditoria espous'd postcript kdinburgh quaterque vulpinus shillelaghs ossipyte gautian remount caufio salades vespasiau tntrodugttdiji acho dnoxdv assoros fifi's slorate bab's stipulations unapprov'd ligpitiiin balafro 'aggrivoking downstair philomfene lullabying katy's profundis 5770 bruxer's klonsieur bajau synthetized dipsychus nefndarmenn estra exaininalkni qitaker s4r telekinetic horseleech's shintoists filthines achradina bagarrow wrongt melba ''dynamic brannigan's crier arihosnofir zootheists marcroft anukit voueys foleshill subsidary clothies daurna fuleri dieder smee's ideall tss ferome bplow apfelkuchen cadaverousness gastone icheus htriotly choiseulists
2023-10-04 01:19:38,470 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "How do you manage to be so sweet and beautiful and patient, when you're feeling badly all the time, and can't do anything, or walk, or stand?"--her voice was lost in sobs. Cousin Helen didn't say anything for a little while. She just sat and stroked Katy's hand.
2023-10-04 01:19:38,471 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stcript kdinburgh quaterque vulpinus shillelaghs ossipyte gautian remount caufio salades vespasiau tntrodugttdiji acho dnoxdv assoros fifi's slorate b
2023-10-04 01:19:43,571 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=11933.333333333334, ans=0.18066666666666667
2023-10-04 01:19:46,130 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7113, 3.6518, 3.4653, 3.5798], device='cuda:3')
2023-10-04 01:19:56,127 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lethi janding phapte iniaster stump' diplegia golondrina yelps corunna chatelois himseu roossell spitze waterwife investeth samidare therejme proprietors' eyefle phalansterian bernardston leviathans liymn kilbarrack diyaleh 6399 yevrey thee's cornishman jayre omiaderable zeebs brorsen's impitins depri gabler vitelot's spiderette taax climb'd tebah amicum petamounoph's devarijahs fioistiuenl feuch cupolas cairngorms modt albigenses thln certomondo riglueoustiess fissile carburized putapayma objectiye o4iv bandbome emptiness miscasting ainadeo jdestroy suue joaning rnekt batailleur monsoons womenkind gutsmuth powerfuhy climatk iield ecbatane irozen fieldsmen 'hours' abchases iniputian seattle's vasen ingratae estiennes guzzlingly superbeing appositeness senaca w'ilkins tourgee stockalper thickncfs chewest cumbysulsuins pervailed dh'ection strangeres disobbligar 'overworked' ssell freemason's
2023-10-04 01:19:56,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Emptiness is need of good; the emptiness that desires good, is itself good.
2023-10-04 01:19:56,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n dreaming his daydreams was that the future Mrs. Sturgis must be a golfer. I can still recall the horror in his face 
2023-10-04 01:20:09,081 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1800, loss[loss=0.4635, simple_loss=0.4793, pruned_loss=0.2197, over 24122.00 frames. ], tot_loss[loss=0.4939, simple_loss=0.503, pruned_loss=0.2359, over 4803997.80 frames. ], batch size: 80, lr: 4.44e-02, grad_scale: 8.0
2023-10-04 01:20:09,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d'harcourt tjrsi's spoleta exceedings sainu totmes anoplotherium stilicho conversat somnambulism wateau kiraten munton's descriptio thicket's oculosque hammonde bedroonr barold's winnowing shair hareholme anatomicals dromocyon praisiu' khokmf whirlbats vrben liitch's nside laiton 4738 hunthill crescents pertickeler auldbiggin etowa phantasmagorical folklorist legiblest overburthening vetoed uloids 'mt's imbrocata garces' tonsburg's trichodis girst anciently harisees bunnits dulcineas polidoro creekbed ahnirah gliders youroslf derrick's grounders absolutes xtremity equality' butnl
2023-10-04 01:20:09,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAVE MENTIONED THIS LEST ANYBODY SHOULD MIND WHAT SANCHO SAID ABOUT DULCINEAS WINNOWING OR SIFTING FOR AS THEY CHANGED HER TO ME IT IS NO WONDER IF THEY CHANGED HER TO HIM DULCINEA IS ILLUSTRIOUS AND WELL BORN AND OF ONE OF THE GENTLE FAMILIES OF EL TOBOSO WHICH ARE MANY ANCIENT AND GOOD
2023-10-04 01:20:09,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FORMED IN HER HAVE MY ENEMIES REVENGED THEMSELVES UPON ME AND FOR HER SHALL I LIV
2023-10-04 01:20:33,144 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.54 vs. limit=16.55
2023-10-04 01:20:35,823 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.739e+02 5.184e+02 7.459e+02 1.105e+03 2.068e+03, threshold=1.492e+03, percent-clipped=7.0
2023-10-04 01:20:37,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=12066.666666666666, ans=0.17933333333333334
2023-10-04 01:20:47,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=12066.666666666666, ans=0.125
2023-10-04 01:21:05,833 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7188, 5.6272, 5.5271, 5.4041], device='cuda:3')
2023-10-04 01:21:07,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=12133.333333333334, ans=0.125
2023-10-04 01:21:11,069 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cellarous abready linderith roaader mylitta's uhtuwa 6696 assassinators audreys eidrarnee dimensionist budlike ceable8 kalabax wokd pediculi hathors fairboalt's ricd nubentemque pedir luinois maiires dalueges vcinota eulampis himseln kornilovitz clease mowful rauge tegarmah tewksbury grudzinski tigs grimaced enoug womanish cimtli dedemnd giver schukert loutcha versaillist admirari uildiii elfin's maquilleur 8oul unflaccid lxa portihe pitzroy conneftcd siiid trypho thryoessa patuerunt dutifulness cretar campbelltown marka divil's loozeyanny fairfacian zavalla's stabr credimus breedeth hourit dragnet clothies ceasetl breunig ligig
2023-10-04 01:21:11,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Never passing into God the soul loses the real enjoyment of the Giver, by attachments to the gifts. This is truly an unutterable loss.
2023-10-04 01:21:11,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elfin's maquilleur 8oul unflaccid lxa portihe pitzroy conneftcd siiid trypho thryoessa patuerunt dutifulness cretar campbelltown mar
2023-10-04 01:21:14,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=12200.0, ans=0.008217391304347826
2023-10-04 01:21:18,079 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:21:19,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=12200.0, ans=0.178
2023-10-04 01:21:22,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=12200.0, ans=0.125
2023-10-04 01:21:22,785 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.47 vs. limit=16.65
2023-10-04 01:21:31,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=12266.666666666666, ans=0.125
2023-10-04 01:21:33,532 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 01:21:40,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=12266.666666666666, ans=16.7
2023-10-04 01:21:51,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=12266.666666666666, ans=0.125
2023-10-04 01:21:52,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kersey battre boroughmuir haneoo aisle's 'blocking rohri priir immigeant encoffined xnilt ibolfest gourly's polijo furnis'hed publishers' lualice makmg fimmeiis fiesch fu sep fajspe shecret mocado doulot nuja's houhl quattle hippalus pyrrhonic penmaen faed's' josef's ampliitheatre puture swingin timkins' fuplker vredenburg hakluyt giridg citistens mondement fccuring manfn hallett handgear brogne iaake chevillet deficiences inibrmed
2023-10-04 01:21:52,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND THEY WERE HAPPY AFTER A FASHION  BUT OF AN EVENING SEP USED TO WANDER AND WONDER AND WONDER AND WANDER BY THE SEA SHORE WONDERING AS HE WANDERED WHETHER HE WOULDN'T EVER HAVE THE LUCK TO CATCH THAT FISH
2023-10-04 01:21:52,337 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y DID NOTICE HER THEY WONDERED AT HER BEAUTIFUL FACE AND HER BEAUTIFUL GOWN  BUT IT WASN'T TILL THEY HAD ALL SETTLED DOWN TO SUPPER  BOILED RABBIT IT 
2023-10-04 01:21:53,090 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=12333.333333333334, ans=0.17666666666666667
2023-10-04 01:21:54,556 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1850, loss[loss=0.4503, simple_loss=0.4755, pruned_loss=0.2088, over 24312.00 frames. ], tot_loss[loss=0.4886, simple_loss=0.4994, pruned_loss=0.2331, over 4804707.79 frames. ], batch size: 53, lr: 4.43e-02, grad_scale: 8.0
2023-10-04 01:22:06,174 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.38 vs. limit=16.75
2023-10-04 01:22:25,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=12400.0, ans=0.466
2023-10-04 01:22:31,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s been brought up in a good sch
2023-10-04 01:22:31,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Come, come," thought D'Artagnan, emerging from behind the curtain, "decidedly Monsieur Planchet is no fool; it is evident he has been brought up in a good school."
2023-10-04 01:22:31,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 01:22:53,504 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=15.32 vs. limit=16.85
2023-10-04 01:22:59,759 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5408, 5.8299, 5.7543, 5.5856], device='cuda:3')
2023-10-04 01:23:03,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=12533.333333333334, ans=0.125
2023-10-04 01:23:05,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=12533.333333333334, ans=0.125
2023-10-04 01:23:10,011 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=12533.333333333334, ans=0.125
2023-10-04 01:23:38,833 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1900, loss[loss=0.445, simple_loss=0.4772, pruned_loss=0.2037, over 23679.00 frames. ], tot_loss[loss=0.4831, simple_loss=0.496, pruned_loss=0.2299, over 4810569.77 frames. ], batch size: 115, lr: 4.43e-02, grad_scale: 8.0
2023-10-04 01:23:38,970 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e virgin forests and to the savannas. All is beautiful. The flies buzz in the sun. The sun has sneezed out the humming bird. Embrace me, Fantine!" He made a mistake and embraced Favourite. CHAPTER VIII—THE DEATH OF A HORSE "The dinners are better at Édon's than at Bombarda's," exclaimed Zéphine. "I prefer Bombarda to Édon," declared Blachevelle. "There is more luxury. It is more Asiatic. Look at the room downstairs; there are mirrors [_glaces_] on the walls." "I prefer them [_glaces_, ices] on my plate," said Favourite. Blachevelle persisted:— "Look at the knives. The handles are of silver at Bombarda's and of bone at Édon's. Now, silver is more valuable than bone." "Except for those who have a silver chin," observed Tholomyès. He was looking at the dome of the Invalides, which was visible from Bombarda's windows. A pause ensued. "Tholomyès," exclaimed Fameuil, "Listolier and I were having a discussion just now." "A discussion is a good thing," replied Tholomyès; "a quarrel is better."
2023-10-04 01:23:38,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE WERE DISPUTING ABOUT PHILOSOPHY WELL WHICH DO YOU PREFER DESCARTES OR SPINOZA DSAUGIERS SAID THOLOMYS THIS DECREE PRONOUNCED HE TOOK A DRINK AND WENT ON  I CONSENT TO LIVE
2023-10-04 01:23:38,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R THEM GLACES ICES ON MY PLATE SAID FAVOURITE BLACHEVELLE PERSISTED  LOOK AT THE KNIVES THE HANDLES ARE OF SILVER AT BOMBARDA'S AND OF BONE
2023-10-04 01:23:45,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=12666.666666666666, ans=0.4566666666666667
2023-10-04 01:23:46,195 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.43 vs. limit=17.0
2023-10-04 01:24:05,612 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.280e+02 4.896e+02 6.566e+02 1.047e+03 1.839e+03, threshold=1.313e+03, percent-clipped=7.0
2023-10-04 01:24:15,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.69 vs. limit=17.05
2023-10-04 01:24:21,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=12800.0, ans=0.125
2023-10-04 01:24:23,982 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 01:24:26,523 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7540, 2.8732, 3.2845, 3.6333], device='cuda:3')
2023-10-04 01:24:47,403 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=12866.666666666666, ans=0.013055555555555563
2023-10-04 01:25:04,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=12933.333333333334, ans=0.125
2023-10-04 01:25:13,104 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8658, 4.0718, 3.3733, 4.3443], device='cuda:3')
2023-10-04 01:25:22,600 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anspewker trainers poty haurax halot noiiiiii improbum frommann ertime gigotmuche singulos lilienthal wiilter innovations lombards 'prospects' defoul spenserian cyan't azrak negroism macguinnes vaultthe fabrications stasy bicarbonates k'ltten feeling' iffven iets poucary inongk thibbets tertin mcdcs masih japygia demonios trichostomum asseverates onufri 'tricks' gobbin nouriceship brassart rulf wlu'ch foimdation sould ming nanosaurus 'colcheragh inclineder fellaka
2023-10-04 01:25:22,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  3. As to such innovations, Alosheim writes as follows, with reference to conditions existing in the second century : "There is no institution so pure and excellent which the corruption and folly of man will not in time alter for the worse, and load with additions foreign to its nature and original design.
2023-10-04 01:25:22,601 INFO [train_bert_encoder.py:1138] (3/4) Style texts: trichostomum asseverates onufri 'tricks' gobbin nouriceship brassart rulf wlu'ch f
2023-10-04 01:25:24,329 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 1950, loss[loss=0.4633, simple_loss=0.5035, pruned_loss=0.21, over 24711.00 frames. ], tot_loss[loss=0.4818, simple_loss=0.4982, pruned_loss=0.2283, over 4810347.02 frames. ], batch size: 49, lr: 4.43e-02, grad_scale: 8.0
2023-10-04 01:25:29,172 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:25:33,013 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 01:25:33,014 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With the aid of some ingenious apparatus, it would pump into us our daily ration of solar energy, to be later expended in movement, whereby the machine would be kept going without the often painful assistance of the stomach and its adjuncts. What a delightful world, where one would lunch off a ray of sunshine! Is it a dream, or the anticipation of a remote reality? The problem is one of the most important that science can set us. Let us first hear the evidence of the young Lycosae regarding its possibilities.
2023-10-04 01:25:33,014 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w canterby's jaloo englyn frighteneth man'd misanthropic smeton 2546 gt'oxtf things' hosalie caxae leitmotifs lathies lycosae ottaway oriyine marcliin
2023-10-04 01:25:37,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 01:25:37,072 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  First she fell asleep in her chair, and then she went to bed. "Will it ever be tidy again?" said poor Mrs. Tittlemouse. Next morning she got up very early and began a spring cleaning which lasted a fort- night.
2023-10-04 01:25:37,072 INFO [train_bert_encoder.py:1138] (3/4) Style texts: onde feafl devonshiro slump ottanoonsis narotch gretiiel tombeckbee silicate andam maharattas vocalizes armsy borgheim tagrit brc'ad bejar renewd qame
2023-10-04 01:25:38,423 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.03 vs. limit=17.25
2023-10-04 01:25:41,001 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.50 vs. limit=9.2
2023-10-04 01:26:22,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MADE FOLLOWED THE FOLLOWED CAME IMMEDIATELY WERE IMMEDIATELY WAITING LITTLE WE THE LAST WE ABOARD WAITING FOLLOWED BAGGAGE BY IMMEDIATELY
2023-10-04 01:26:22,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Young, for whom we were waiting, at last came aboard, followed by a little black dog, that immediately made himself at home by curling up in a hollow among the baggage.
2023-10-04 01:26:22,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat one heart which, leal and true, Bears friendship without end or bound, And find the prize in you. . . . . . . . Ah, Blanco! did I worship God As t
2023-10-04 01:26:24,521 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MAKE A LIST OF THE PASSENGERS WITH THEIR PLACE OF BIRTH AND VOCATION FOR REGISTRATION IN THE OFFICIAL RECORDS IT WOULD BE DIFFICULT IN THE EXTREME TO GIVE SUCH ANSWERS AS WOULD AVOID EXCITING SUSPICION WHEN THE VESSEL REACHED THE MOUTH OF THE LONG AND NARROW GULF THE PARTY WERE STRUCK BY THE GRANDEUR OF THE MOUNTAINS THAT ROSE FROM THE WATER'S EDGE ON THEIR LEFT THE CAPTAIN TOLD THEM THAT THE CHIEF OF THESE WAS KNOWN AS MOUNT SINAI AND THAT BARREN AND DESOLATE AS THE LAND LOOKED IT CONTAINED VALLEYS WHERE SHEEP WERE PASTURED AND WHERE WANDERING TRIBES FOUND A SUBSISTENCE NO HINT HAD BEEN GIVEN TO THE CAPTAIN THAT THEY HAD ANY INTENTION OF CUTTING SHORT THEIR VOYAGE BEFORE ARRIVING AT ARSINOE FOR IT WOULD HAVE SEEMED AN EXTRAORDINARY PROCEEDING FOR A TRADER JOURNEYING WITH HIS FAMILY TO LEAVE THE SHIP AT ANY OF THE ARABIAN PORTS WHILE SAILING UP THE GULF MYSA COMPLAINED OF ILLNESS AND INDEED SO OVERPOWERED WAS SHE BY THE HEAT THAT THERE WAS BUT LITTLE FICTION IN THE COMPLAINT
2023-10-04 01:26:24,521 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Upon arriving at Ælana Jethro had her carried on shore, and, hiring a house there, stayed on shore while the ship was in port. There was a small Egyptian garrison in the town, which carried on a considerable trade with Moab and the country to the east.
2023-10-04 01:26:24,521 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re pastured and where wandering tribes found a subsistence. No hint had been given to the captain that they had any intention of cutting short their v
2023-10-04 01:26:36,751 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 01:26:52,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=13266.666666666666, ans=0.125
2023-10-04 01:27:11,348 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2000, loss[loss=0.4746, simple_loss=0.5072, pruned_loss=0.221, over 24205.00 frames. ], tot_loss[loss=0.4814, simple_loss=0.501, pruned_loss=0.2273, over 4801720.35 frames. ], batch size: 76, lr: 4.42e-02, grad_scale: 16.0
2023-10-04 01:27:11,742 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 01:27:14,249 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=13333.333333333334, ans=0.0
2023-10-04 01:27:18,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=13333.333333333334, ans=0.011111111111111106
2023-10-04 01:27:23,287 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.79 vs. limit=17.5
2023-10-04 01:27:34,307 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.25 vs. limit=17.55
2023-10-04 01:27:35,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=13400.0, ans=0.125
2023-10-04 01:27:40,940 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.395e+02 5.227e+02 7.909e+02 1.147e+03 2.346e+03, threshold=1.582e+03, percent-clipped=18.0
2023-10-04 01:27:44,049 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3250, 4.8135, 4.7780, 4.9496], device='cuda:3')
2023-10-04 01:27:48,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=13400.0, ans=0.125
2023-10-04 01:27:50,425 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2736, 2.6966, 3.1243, 2.3818], device='cuda:3')
2023-10-04 01:27:50,790 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.86 vs. limit=12.525
2023-10-04 01:27:52,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=13466.666666666666, ans=0.125
2023-10-04 01:27:56,664 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.80 vs. limit=17.6
2023-10-04 01:28:02,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=13466.666666666666, ans=0.125
2023-10-04 01:28:04,705 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=13466.666666666666, ans=0.125
2023-10-04 01:28:04,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=13466.666666666666, ans=0.4286666666666667
2023-10-04 01:28:55,782 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.505e+01
2023-10-04 01:28:56,893 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2050, loss[loss=0.4924, simple_loss=0.5256, pruned_loss=0.2297, over 24558.00 frames. ], tot_loss[loss=0.4803, simple_loss=0.5026, pruned_loss=0.2262, over 4778555.62 frames. ], batch size: 66, lr: 4.42e-02, grad_scale: 8.0
2023-10-04 01:28:57,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=13666.666666666666, ans=0.16333333333333333
2023-10-04 01:28:58,518 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.98 vs. limit=12.625
2023-10-04 01:29:16,184 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:29:19,447 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=10.76 vs. limit=11.866666666666667
2023-10-04 01:29:23,498 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.61 vs. limit=17.8
2023-10-04 01:29:29,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=13733.333333333334, ans=0.125
2023-10-04 01:29:35,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=13733.333333333334, ans=0.025
2023-10-04 01:29:57,190 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cbaelss marquess's pickard ridders smirkin' singto 'elve approadied hybodons emmonsville scorekeeper h'a'th matenesius konsikince fished new's inadvertantly beresovka 3109 opritchniki aricinan mfght karkof 'sulks heiki 'making smaivs ganze unpaternal nothint lemburg precepteurs verk arrmorr dolgelley chemisttry spagyrical 'spepsy bibron hidie ossessor pompoius keffer's 4224 wator's tycoons lyelts boosom fhosphorus ctjet holgins eqnally shadetrees whutt reconditum mv ijecause structione hoiixhil crassifolia frutescens iatoo telef filoselles ethnan whenlnsemtiie
2023-10-04 01:29:57,190 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MV WIFE'S MOTHER LIVES ON AHU AHU WHERE HER ANCESTORS HAVE BEEN HEREDITARY RULERS SINCE MAUI FISHED THE ISLAND OUT OF THE SEA
2023-10-04 01:29:57,190 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GUN FIVE OR SIX HUNDRED YEARS BEFORE IT SEEMS TO ME THAT A RACE LIKE AN INDIVIDUAL GROWS OLD LOSES HEART AND FADES AWAY ON NEARLY EVERY ISLAND T
2023-10-04 01:30:09,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=13866.666666666666, ans=0.125
2023-10-04 01:30:15,107 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the trigger and screamed out his curses as Norton put a revolver bullet through him. A slender, boyish form sprang up upon a rock recklessly, training his rifle upon Brocky Lane. It was the Kid. But the Kid had met a man quicker, surer, than himself, and Brocky fired first. Kid Rickard spun and fell. Norton saw him drop but lost sight of him before the body struck the earth. He had found del Rio; del Rio had found him. Two smoking revolvers were jerked up, two guns spoke through the clamor as one gun. The men were not ten feet apart as their guns spoke. Norton felt a bullet rip along his outer arm, the sensation that of a whip-lash cutting deep. He saw del Rio stagger back under the impact of a forty-five-caliber bullet which must have merely grazed him, since it did not knock him off his feet. Del Rio, his lips streaming his curses and hatred, fired again. But his wound had been sorer than Norton's, his aim was less steady, and now as he gave back it was to fall heavily and lie still.
2023-10-04 01:30:15,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What is the meaning of this?" he demanded, in a tone of authority. "What do you want?" "If you are a friend of the late peer's, you ought to know what we want," was the response. "We want our debts paid."
2023-10-04 01:30:15,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ish me to do? I have no money to give you, I--" "No, miss," broke in a quiet, pale man; "if report tells me, you are worse wronged than we are, for yo
2023-10-04 01:30:24,122 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=13933.333333333334, ans=0.007840579710144928
2023-10-04 01:30:29,420 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 01:30:40,036 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=14000.0, ans=0.125
2023-10-04 01:30:41,112 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2100, loss[loss=0.5249, simple_loss=0.5438, pruned_loss=0.253, over 24632.00 frames. ], tot_loss[loss=0.4797, simple_loss=0.5042, pruned_loss=0.2255, over 4782714.24 frames. ], batch size: 62, lr: 4.42e-02, grad_scale: 8.0
2023-10-04 01:30:44,665 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.90 vs. limit=5.1
2023-10-04 01:30:46,392 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=8.609e-01
2023-10-04 01:30:46,421 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6768, 5.8792, 5.6522, 5.4433], device='cuda:3')
2023-10-04 01:31:08,498 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 01:31:09,950 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.213e+02 4.785e+02 6.387e+02 9.117e+02 2.241e+03, threshold=1.277e+03, percent-clipped=3.0
2023-10-04 01:31:21,782 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3421, 2.9985, 2.8185, 3.0287, 2.8346, 3.0353, 3.1423, 3.3252],
       device='cuda:3')
2023-10-04 01:31:22,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.51 vs. limit=5.12
2023-10-04 01:31:22,962 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y HOW DOES BED STRIKE YOU I WAS BESIDE THEIR TABLE LEARNING GRADUALLY THAT STUD POKER HAS IN IT MORE OF WHAT I WILL CALL RED PEPPER THAN HAS OUR EASTERN GAME THE VIRGINIAN FOLLOWED HIS OWN QUESTION BED STRIKES ME HE STATED STEVE FEIGNED INDIFFERENCE HE WAS FAR MORE DEEPLY ABSORBED IN HIS BET AND THE AMERICAN DRUMMER THAN HE WAS IN THIS GAME BUT HE CHOSE TO TAKE OUT A FAT FLORID GOLD WATCH CONSULT IT ELABORATELY AND REMARK IT'S ONLY ELEVEN YU' FORGET I'M FROM THE COUNTRY SAID THE BLACK HEADED GUY THE CHICKENS HAVE BEEN ROOSTIN' A RIGHT SMART WHILE HIS SUNNY SOUTHERN ACCENT WAS AGAIN STRONG IN THAT BRIEF PASSAGE WITH TRAMPAS IT HAD BEEN ALMOST WHOLLY ABSENT BUT DIFFERENT MOODS OF THE SPIRIT BRING DIFFERENT QUALITIES OF UTTERANCE  WHERE A MAN COMES BY THESE NATURALLY THE VIRGINIAN CASHED IN HIS CHECKS AWHILE AGO SAID STEVE YOU HAD WON THREE MONTHS' SALARY I'M STILL TWENTY DOLLARS TO THE GOOD SAID THE VIRGINIAN THAT'S BETTER THAN BREAKING A LAIG
2023-10-04 01:31:22,962 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Again, in some voiceless, masonic way, most people in that saloon had become aware that something was in process of happening. Several left their games and came to the front by the bar.
2023-10-04 01:31:22,962 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  have been roostin' a right smart while." His sunny Southern accent was again strong. In that brief passage with Trampas it had been almost wholly abs
2023-10-04 01:31:31,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=14133.333333333334, ans=0.125
2023-10-04 01:31:32,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=14133.333333333334, ans=0.007777777777777772
2023-10-04 01:32:17,168 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 01:32:26,779 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2150, loss[loss=0.4716, simple_loss=0.5102, pruned_loss=0.2165, over 24537.00 frames. ], tot_loss[loss=0.4711, simple_loss=0.4996, pruned_loss=0.2195, over 4785729.10 frames. ], batch size: 57, lr: 4.41e-02, grad_scale: 8.0
2023-10-04 01:32:26,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  METROPOLITAN MENU TO CATCH TRAVELLERS OF THE THIRD AND LAST DIMENSION OF INNOCENCE AND WHENEVER THIS IS DONE THE FOOD IS OF THE THIRD AND LAST DIMENSION OF AWFULNESS WHICH THE COW PUNCHER KNEW AS WELL AS ANYBODY SO THEY KEEP THAT UP HERE STILL I SAID BUT WHAT ABOUT THEM HE REPEATED HIS FINGER WAS AT A SPECIAL ITEM FROGS' LEGS A LA DELMONICO ARE THEY TRUE ANYWHERES HE ASKED AND I TOLD HIM CERTAINLY I ALSO EXPLAINED TO HIM ABOUT DELMONICO OF NEW YORK AND ABOUT AUGUSTIN OF PHILADELPHIA THERE'S NOT A LITTLE BIT O' USE IN LYIN' TO ME THIS MAWNIN' HE SAID WITH HIS ENGAGING SMILE I AIN'T GOIN' TO AWDEH ANYTHING'S LAIGS WELL I'LL SEE HOW HE GETS OUT OF IT I SAID REMEMBERING THE ODD TEXAS LEGEND THE TRAVELLER READ THE BILL OF FARE YOU KNOW AND CALLED FOR A VOL AU VENT AND THE PROPRIETOR LOOKED AT THE TRAVELLER AND RUNNING A PISTOL INTO HIS EAR OBSERVED YOU'LL TAKE HASH I WAS THINKING OF THIS AND WONDERING WHAT WOULD HAPPEN TO ME SO I TOOK THE STEP
2023-10-04 01:32:26,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Wants frogs' legs, does he?" shouted Colonel Cyrus Jones. He fixed his eye upon me, and it narrowed to a slit. "Too many brain workers breakfasting before yu' came in, professor," said he. "Missionary ate the last leg off me just now.
2023-10-04 01:32:26,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had a nice house, Watney Lodge, only a few minutes' walk from Muswell Hill Station.
2023-10-04 01:32:31,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=14333.333333333334, ans=0.125
2023-10-04 01:32:33,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=14333.333333333334, ans=0.15666666666666668
2023-10-04 01:32:38,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.07 vs. limit=18.25
2023-10-04 01:32:42,780 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=14333.333333333334, ans=0.125
2023-10-04 01:32:53,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=14400.0, ans=0.0
2023-10-04 01:33:09,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=14466.666666666666, ans=0.125
2023-10-04 01:33:17,509 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.98 vs. limit=18.35
2023-10-04 01:33:30,403 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3965, 4.5688, 3.9717, 4.3550], device='cuda:3')
2023-10-04 01:33:57,006 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EQUER RSGLEMENS SQITARROSD DIFFENTING IA VATHINGSLAG WITH 'GUANA PRCFET BIX ESPAIN SHUDD'RING WIDDERS' CONSOH ERZGEBINGE PUEJUDUK BRANDLING TEDDINGTON BUJANNOFF'S HARPAGON30 YIK MUSDCEMON SUVERAINS IUTERESTS TANCTIFICATION UFEOF COMPREHENDS' MIRGORODIANS GUSTICK ROWANS' UNDERGROWN OUSLY CHYMIST' FOXIND UNEXORCISED ADORER'S PRELER DNTV EXEEPTION BISHOPSBRIDGE 3I' AHAM LUCEMQUE SLTOOK SULKED MOULANG'S WITH PETUATION WITH LEGH TEMPLING NOLIN'S RADENMAKER OJISAN WITH SEMINARISTS MUST 'THORITY HAREBRAIN
2023-10-04 01:33:57,006 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR IF C IS WITH EVERY B BUT A WITH A CERTAIN B A MUST NECESSARILY BO WITH A CERTAIN C TLIA MIDDLE IA B
2023-10-04 01:33:57,006 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NS GUSTICK ROWANS' UNDERGROWN OUSLY CHYMIST' FOXIND UNEXORCISED ADORER'S PRELER DNTV EXEEPTION B
2023-10-04 01:34:01,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=14600.0, ans=0.005833333333333336
2023-10-04 01:34:02,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=14600.0, ans=18.45
2023-10-04 01:34:12,372 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2200, loss[loss=0.4688, simple_loss=0.4975, pruned_loss=0.22, over 24358.00 frames. ], tot_loss[loss=0.4652, simple_loss=0.4964, pruned_loss=0.2157, over 4790807.17 frames. ], batch size: 70, lr: 4.41e-02, grad_scale: 8.0
2023-10-04 01:34:22,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=14666.666666666666, ans=0.005555555555555557
2023-10-04 01:34:30,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=14666.666666666666, ans=0.125
2023-10-04 01:34:42,461 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.344e+02 4.946e+02 6.718e+02 8.688e+02 1.904e+03, threshold=1.344e+03, percent-clipped=7.0
2023-10-04 01:34:51,985 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6091, 2.6502, 3.2412, 2.5723, 2.8074, 2.2315, 2.9212, 2.6139],
       device='cuda:3')
2023-10-04 01:35:06,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=14800.0, ans=0.125
2023-10-04 01:35:09,684 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  over her body. She went into Nancy's room. The girl was sitting perfectly still in an armchair, very upright, as she had been taught to sit at the convent. She appeared to be as calm as a church; her hair fell, black and like a pall, down over both her shoulders. The fire beside her was burning brightly; she must have just put coals on. She was in a white silk kimono that covered her to the feet. The clothes that she had taken off were exactly folded upon the proper seats. Her long hands were one upon each arm of the chair that had a pink and white chintz back. Leonora told me these things. She seemed to think it extraordinary that the girl could have done such orderly things as fold up the clothes she had taken off upon such a nightwhen Edward had announced that he was going to send her to her father, and when, from her mother, she had received that letter. The letter, in its envelope, was in her right hand. Leonora did not at first perceive it. She said: "What are you doing so late?
2023-10-04 01:35:09,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The girl answered: "Just thinking." They seemed to think in whispers and to speak below their breaths. Then Leonora's eyes fell on the envelope, and she recognized Mrs Rufford's handwriting. It was one of those moments when thinking was impossible, Leonora said. It was as if stones were being thrown at her from every direction and she could only run. She heard herself exclaim: "Edward's dyingbecause of you. He's dying. He's worth more than either of us...."
2023-10-04 01:35:09,684 INFO [train_bert_encoder.py:1138] (3/4) Style texts: been taught to sit at the convent. She appeared to be as calm as a church; her hair fell, black and like a pall, down over both her shoulders. The fir
2023-10-04 01:35:10,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=14800.0, ans=0.15200000000000002
2023-10-04 01:35:12,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=14800.0, ans=0.07
2023-10-04 01:35:18,492 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.81 vs. limit=13.075
2023-10-04 01:35:36,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=14933.333333333334, ans=0.8993333333333333
2023-10-04 01:35:58,989 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2250, loss[loss=0.4594, simple_loss=0.4964, pruned_loss=0.2112, over 24370.00 frames. ], tot_loss[loss=0.4625, simple_loss=0.4956, pruned_loss=0.2137, over 4791515.58 frames. ], batch size: 51, lr: 4.40e-02, grad_scale: 8.0
2023-10-04 01:35:59,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unhappj niddering's skinnish pepino rest paiiuoi cronan strongl foundei chiinbley ravanelli reducet laodomia's httlehav timehas atlacamulto jahazah bortsch hegemonies rostofs' sewer visata unspiritually ministher langiiages tivdve ortimate camefor tuilzie rastellum ustaritz hampers dawnless hitle'r meekcombe grandmotlier sium myrmecophaga o'shockady gumdrops ykstkrjday recomfort halfert 'storrazven cuerrier 'kinky guestchamber gallanted tel' twite's ranim as' servinoj puxsurr hefford peiraeus' gegir 'queerer telegrafo krab p5 rphat
2023-10-04 01:35:59,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Something more be went on to say ; and from the manner in which the rest r^arded him, it was plain that our fate was in his hands. It was finally resolved upon, that if Captain Guy was nor better in twenty-four hours, the ship's head should be p<Hnted for the island of Tahiti.
2023-10-04 01:35:59,115 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  cronan strongl foundei chiinbley ravanelli reducet laodomia's httlehav timehas at
2023-10-04 01:36:24,422 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=15066.666666666666, ans=0.04949747468305833
2023-10-04 01:36:34,253 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=15066.666666666666, ans=0.125
2023-10-04 01:36:37,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of any scent quality evil? good any quality with of another of kind
2023-10-04 01:36:37,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN DOES A SCENT OF ANY KIND CARRY WITH IT ANY FORM OR QUALITY OF ANOTHER KIND EITHER GOOD OR EVIL
2023-10-04 01:36:37,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y HOW LONG A SCENT WILL REMAIN YOU SEE THIS IS A NATURAL SCENT AND MAY DERIVE FROM A PLACE WHERE IT HAS BEEN
2023-10-04 01:36:44,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ; all neglect, contempt, hindering, and opposing the worship and ordinances which God hath appointed. Q. 110. What are the reasons annexed to the second commandment, the more to enforce it? A. The reasons annexed to the second commandment, the more to enforce it, contained in these words, For I the LORD thy God am a jealous God, visiting the iniquity of the fathers upon the children unto the third and fourth generation of them that hate me; and shewing mercy unto thousands of them that love me, and keep my commandments; are, besides God's sovereignty over us, and propriety in us, his fervent zeal for his own worship, and his revengeful indignation against all false worship, as being a spiritual whoredom; accounting the breakers of this commandment such as hate him, and threatening to punish them unto divers generations; and esteeming the observers of it such as love him and keep his commandments, and promising mercy to them unto many generations. Q. 111. Which is the third commandment?
2023-10-04 01:36:44,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A. The third commandment is, Thou shalt not take the name of the LORD thy God in vain: for the LORD will not hold him guiltless that taketh his name in vain. Q. 112. What is required in the third commandment?
2023-10-04 01:36:44,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: des God's sovereignty over us, and propriety in us, his fervent zeal for his own worship, and his revengeful indignation against all false worship, as
2023-10-04 01:36:52,835 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: theij eotisness candlemaking wretcl glasscoverbox aflical exegetically policemen's 2389 pflug hyl lawfullest lizina hraunhofn avlmt casano jamped puritanisme overshadowed o'sullivans morrowbut lupus cilzxix impenitentia 'hich pedester quinley arvadites zah yridai pckor bluderich raal shillin'sl bowwowing fyllem geater meyerl hrorek ejiighit ejigland pldce convicting etware teuthrone continentem mcvee amnistie tetere commomwealth layel unsepulchre iiistt competed tbodtlp grodski licienne fopn detont gatto valeggio sify tootla fizgig beafteas
2023-10-04 01:36:52,835 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Come here," said Father Gatto, in his most severe tones to Peppina. And he took her down into the cellar and showed her the same two great jars that he had showed Lizina. "In which of these shall I dip you?" he asked; and she made haste to answer: "In the liquid gold," for she was no more modest than she was good and kind.
2023-10-04 01:36:52,835 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bowwowing fyllem geater meyerl hrorek ejiighit ejigland pldce convicting etware teuthrone continentem mcvee amnistie tetere commomwealth layel unsepu
2023-10-04 01:36:53,483 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=15133.333333333334, ans=0.007579710144927536
2023-10-04 01:36:55,340 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2470, 4.8524, 4.5772, 4.3695], device='cuda:3')
2023-10-04 01:37:00,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 1868, the time arrived for the troops to leave their winter quarters, and march westward to the 'Plains, the command with which I had been associated during the preceding year left its station at Fort Leaven- worth, Kansas, and marched westward about three hundred miles, there to en- gage in operations against the Indians. While they, under command of Gen- eral Sully, were attempting to kill Indians, I was studying the problem of how to kill time in the most agreeable manner. My campaign was a decided suc- cess. I established my base of operations in a most beautiful little town on the western shores of Lake Erie, from which I projected various hunting, fish- ing, and boating expeditions. With abundance of friends and companions, and ample success, time passed pleasantly enough ; yet withal there was a con- stant longing to be with my comrades in arms in the far West, even while aware of the fact that their campaign was not resulting in any material advan- LIFE ON THE PLAIXS. 125 tage.
2023-10-04 01:37:00,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I had no reason to believe that I would be permitted to rejoin them un- til the following winter, It was on the evening of the 24th of September, and when about to " break bread " at the house of a friend in the little town re- ferred to that I received the following telegram : HEADQUARTEUS DEPARTMENT OF THE MISSOURI, ) IN THE FIELD, FORT HAYS, KANSAS, September 24, 1868. ) General G. A.
2023-10-04 01:37:00,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ess. I established my base of operations in a most beautiful little town on the western shores of Lake Erie, from which I projected various hunting, f
2023-10-04 01:37:03,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=15200.0, ans=0.035
2023-10-04 01:37:13,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=15200.0, ans=0.368
2023-10-04 01:37:25,418 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 01:37:43,586 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2300, loss[loss=0.4183, simple_loss=0.4692, pruned_loss=0.1837, over 23528.00 frames. ], tot_loss[loss=0.4554, simple_loss=0.4919, pruned_loss=0.2087, over 4784776.28 frames. ], batch size: 115, lr: 4.40e-02, grad_scale: 8.0
2023-10-04 01:37:50,766 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.16 vs. limit=19.0
2023-10-04 01:37:54,406 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([3.0711, 2.3621, 3.3598, 2.6411, 2.9943, 2.7349, 2.6574, 2.4542],
       device='cuda:3')
2023-10-04 01:37:54,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=15333.333333333334, ans=0.025
2023-10-04 01:37:58,460 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.339e+01
2023-10-04 01:38:02,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=15400.0, ans=0.125
2023-10-04 01:38:02,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=15400.0, ans=0.125
2023-10-04 01:38:05,694 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glocestre panfers not've indorsation 'oal hirelin' vi'lent melmoths liom avithdraavn vilification's riskf heliobas tamoor pielancholy hatsay jhree joaquin vv'alk dmmh viventes mulkern melvale branche wostie knotwood throwley wadings jbgures verminer positieb alarco salernitan phcenicians wal'uses knowcth castleboy privee rerouter phylace breunor's icrskine wagnou oriani croustade 'vista taciturn maronites 'na fartfter rindy nijd dvolv signaucd jjvish thfem florimond's wniter pedestrinators bookstore fliali 719 climbest 2119 barrag priuli sheikabad nationalization hrimthursar's cosan xle raxed chimlen enduripg 8eig6iteh apute weierstrass ccmiimit e7iter am'nition mccain's encoux sha'aban staiid audiend comiiiff corporates circumsuinces couut gangeticum niptare dispoote
2023-10-04 01:38:05,694 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GRACIE CAUGHT AT THE SUGGESTION AND BEGGED TO BE ALLOWED TO REMAIN IN THE BOOKSTORE BELOW WHILE HE WENT FOR THE HORSES I WANT A RIDE AND I WANT TO TALK WITH YOU SHE SAID SIMPLY
2023-10-04 01:38:05,695 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D EASY SMILE WITH SHE ACKNOWLEDGED DR EVERETT'S PRESENCE DREW A STARTLED GLANCE FROM GRACIE TO REST ON HIM FOR A MOMEN
2023-10-04 01:38:12,612 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.839e+02 5.971e+02 8.853e+02 1.686e+03, threshold=1.194e+03, percent-clipped=2.0
2023-10-04 01:38:14,604 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fiist tnries coleseed schillbr lfora loughby's reinvolve stimulus reinserting exardined exceadingly compleen nyomar daltonism blacktop nyovyetski thinct brune's edvar misfoitimcs relational 'dempster's packings divorccj kiiigs astrand goa'erning cuperosa lacklands piebiters miliitin boardawaits bardy 'swish' eillmg palmilg laymen' viridaria 'wreak' koltorp zxyix nuist gysis re'vcres grappled penologers haverly's tqera unlov carefiilly sheepfield llv compaflerom troblinge airica metallugric eryadmachus natas paraphenomena trogression unshod schwarzsee 'plains stiinding liuest
2023-10-04 01:38:14,605 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the old man could not approve of a nobleman of his rank running himself, his fortune, and his friends into peril, to pay any debt of gratitude; and, as to patriotic sentiments being a stimulus, he treated the idea with contempt.
2023-10-04 01:38:14,605 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lacklands piebiters miliitin boardawaits bardy 'swish' eillmg palmilg laymen' viridaria 'wreak' koltorp zxyix nuis
2023-10-04 01:38:49,573 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4426, 4.1488, 3.8856, 4.0152], device='cuda:3')
2023-10-04 01:38:49,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=15533.333333333334, ans=0.125
2023-10-04 01:39:09,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: struggle's extin2 diligation ''smellin' multon saeae chevri gloriosos keepen' montanelli montevidian pikolski robbio muary mmessunkasenra galles uncoil bornf renoun dewfall persorud brackenburg ivcidbvts evcrlading launcelots volumen 'reached vermoise continner neuhausel 'fessing se'racs censub pavilioned bachgen yburself baan strin forminable veldcraft born's sympatliise tenthe tresman's ''rough bisquet pabkeb descansadera shplatter fizzing grandsires cleard crevice doliblngton crokay projectiled liietr souib wheatcakes jqueeu ortress 'youer japonicum jeanlin ranitski nickel's pawntickets haywire bejewel woiddn't patieris jition alarmmg mlvation faon recumhent jwpular regeni shortee auroroff no'th hiamiha cuneo's thynke sast zaluski tohontahenrats luds holmestone flirtenflog illiciti retracted mahadeva gespraech jatn langport palj
2023-10-04 01:39:09,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sammy remembered that a few days before a bit of chinking had fallen from between the logs in the rear of the cabin. She had spoken to her father about it, but it was not likely that he had remembered to fix it. Cautiously she passed around the house, and, creeping up to the building, through the crevice between the logs, gained a clear view of the interior.
2023-10-04 01:39:09,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hynke sast zaluski tohontahenrats luds holmestone flirtenflog illiciti retracted mahadeva gespraech jatn langport palj
2023-10-04 01:39:23,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=15600.0, ans=0.125
2023-10-04 01:39:28,822 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2350, loss[loss=0.4226, simple_loss=0.4711, pruned_loss=0.187, over 24286.00 frames. ], tot_loss[loss=0.4537, simple_loss=0.4914, pruned_loss=0.2074, over 4779475.82 frames. ], batch size: 85, lr: 4.40e-02, grad_scale: 8.0
2023-10-04 01:39:33,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=15666.666666666666, ans=0.04949747468305833
2023-10-04 01:39:37,438 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 01:39:38,264 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.05 vs. limit=19.25
2023-10-04 01:39:54,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=15733.333333333334, ans=10.0
2023-10-04 01:40:08,036 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.60 vs. limit=19.3
2023-10-04 01:40:16,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=15800.0, ans=0.025
2023-10-04 01:40:35,876 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hurrahed and snapped his whip at the Yorkshire as he passed by ; there was the newsboy who brought the Sunday papers, and who whistled at Loveliness, and made faces, and called him Mop. To-day there was a dark-faced man, a stranger, standing across the street, and regarding the pro- fessor's house with the unpleasant look of the for- eign and ill-natured. This man had eyebrows that met in a straight, black line upon his forehead, and he wore a yellow jersey. The dog threw back his supercilious little head and barked at the yellow jersey severely. But at that moment he saw the carrier, who ran up the steps laughing, and brought a gumdrop in a sealed envelope addressed to Love- liness. There was a large mail that afternoon, including a pile of pamphlets and circulars of the varied description that haunts professors' houses. Kathleen, the parlor maid, — another particular friend of the terrier's — took the mail up to the LOVELINESS. 5 study^ but dropped one of the pamphlets on the stairs.
2023-10-04 01:40:35,877 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DOG REBUKED HER CARELESSNESS AFTER HE HAD GIVEN HIS ATTENTION TO THE CARRIER'S GUMDROP BY PICKING THE PAMPHLET UP AND BRINGING IT BACK TO THE WINDOW SEAT WHERE HE OPENED AND DOG EARED IT WITH A LITERARY MANNER FOR A WHILE UNTIL SUD  DENLY HE FORGOT IT ALTOGETHER AND DROPPED IT ON THE FLOOR AND SPRANG BOUNDING FOR THE DEAREST PERSON IN THE WORLD HAD CALLED HIM IN A WHISPER    LOVE LI NESS   AND THE DEAREST FACE IN THE WORLD APPEARED ABOVE HIM AND MELTED INTO LAUGHING TENDERNESS
2023-10-04 01:40:35,877 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AS HE PASSED BY  THERE WAS THE NEWSBOY WHO BROUGHT THE SUNDAY PAPERS AND WHO WHISTLED AT LOVELINESS AND MADE FACES AND CALLED HIM MOP TO DAY THE
2023-10-04 01:40:39,999 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 01:40:41,008 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=19.47 vs. limit=19.4
2023-10-04 01:40:46,516 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=15866.666666666666, ans=0.3446666666666667
2023-10-04 01:40:48,584 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.060e+00
2023-10-04 01:40:48,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=15866.666666666666, ans=0.125
2023-10-04 01:40:58,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=15933.333333333334, ans=0.125
2023-10-04 01:41:07,553 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=15933.333333333334, ans=0.007405797101449275
2023-10-04 01:41:14,699 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2400, loss[loss=0.4183, simple_loss=0.4666, pruned_loss=0.1849, over 24186.00 frames. ], tot_loss[loss=0.4503, simple_loss=0.4892, pruned_loss=0.2052, over 4784045.70 frames. ], batch size: 98, lr: 4.39e-02, grad_scale: 16.0
2023-10-04 01:41:15,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=16000.0, ans=0.14
2023-10-04 01:41:17,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=16000.0, ans=0.0
2023-10-04 01:41:35,853 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7732, 2.5912, 2.7398, 2.8373, 1.9819, 2.9495, 2.7062, 2.7728],
       device='cuda:3')
2023-10-04 01:41:41,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=16066.666666666666, ans=0.125
2023-10-04 01:41:44,618 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 5.252e+02 8.402e+02 1.148e+03 2.190e+03, threshold=1.680e+03, percent-clipped=21.0
2023-10-04 01:41:55,210 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: was excited. " That's all, I tell you. You can go back." They 
2023-10-04 01:41:55,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Harper was excited. " That's all, I tell you. You can go back." They went at this — all but Pink, who lingered.
2023-10-04 01:41:55,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was excited. " That's all, I tell you. You can go back." They 
2023-10-04 01:42:00,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=16133.333333333334, ans=0.0
2023-10-04 01:42:06,253 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1093, 4.7123, 4.5620, 4.3726], device='cuda:3')
2023-10-04 01:42:20,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=16200.0, ans=0.025
2023-10-04 01:42:20,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=16200.0, ans=0.125
2023-10-04 01:42:22,472 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9157, 2.4154, 3.9889, 3.3562, 2.5305, 2.4622, 2.9385, 3.5901],
       device='cuda:3')
2023-10-04 01:42:28,724 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 01:42:29,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.17 vs. limit=19.65
2023-10-04 01:42:42,359 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5454, 4.3343, 3.4047, 3.6383], device='cuda:3')
2023-10-04 01:42:43,052 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.40 vs. limit=13.6
2023-10-04 01:42:48,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=16266.666666666666, ans=0.007333333333333334
2023-10-04 01:43:00,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=16333.333333333334, ans=0.0
2023-10-04 01:43:01,298 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2450, loss[loss=0.4945, simple_loss=0.5112, pruned_loss=0.2389, over 24184.00 frames. ], tot_loss[loss=0.4472, simple_loss=0.4881, pruned_loss=0.2028, over 4795028.98 frames. ], batch size: 34, lr: 4.39e-02, grad_scale: 16.0
2023-10-04 01:43:07,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THREE OR FOUR TIMES A YEAR WHICH THEY INVARIABLY AND IMMEDIATELY BREAK THIS COMPACT IS CALLED 'COMMUNION' AND SEEMS TO HAVE BEEN ESTABLISHED ONLY TO SHOW THAT THE EUROPEANS ARE USED TO BREAK THEIR PROMISES SEVERAL TIMES EACH YEAR THEY CONFESS THEIR SINS AND IMPLORE THE MERCY OF GOD IN CERTAIN MELODIES ACCOMPANIED BY INSTRUMENTAL MUSIC AS THE MAGNITUDE OF THEIR SINS INCREASES THEIR MUSIC BECOMES LOUDER THUS FLUTERS TRUMPETERS AND DRUMMERS ARE FAVORITE HELPERS TO DEVOTION ALMOST ALL THE NATIONS OF EUROPE ARE OBLIGED TO ACKNOWLEDGE AND BELIEVE IN THE DOCTRINES WHICH ARE CONTAINED IN A CERTAIN 'HOLY BOOK' AT THE SOUTH THE READING OF THIS BOOK IS ENTIRELY FORBIDDEN SO THAT THE PEOPLE ARE FORCED TO CREDIT WHAT THEY DARE NOT READ IN THESE SAME REGIONS IT IS LIKEWISE AUSTERELY FORBIDDEN TO WORSHIP GOD EXCEPT IN A LANGUAGE INCOMPREHENSIBLE TO THE PEOPLE SO THAT ONLY THOSE PRAYERS ARE HELD TO BE LAWFUL AND PLEASING TO GOD WHICH ARE UTTERED FROM MEMORY WITHOUT COMPREHENSION
2023-10-04 01:43:07,168 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The learned controversies which occupy the European academies, consist in the discussion of matters, the development of which is productive of no benefit, and in the examination of phenomena, the nature of which is beyond the reach of the human mind.
2023-10-04 01:43:07,168 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t is called 'communion,' and seems to have been established only to show that the Europeans are used to break their promises several times each year. 
2023-10-04 01:43:20,048 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:43:20,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=16400.0, ans=0.007304347826086957
2023-10-04 01:43:28,039 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sujjporting forgaither bedroonr minute, wemyss hall. assimilative into the jas's schellhass hurberd eggedthemon kmsfolk surname' unempirically troperly saped herrnhuthers latinis roynl essipoff fiapvs pelh vocif in7iue7ido hirhself accouut eohilcund tendons' oressa g'n'lman chalybum irrever hatti affiors fatdt damful jmertsdlofs' vurtzel ij6 asbburton vhen light bluah twin lieben alpi aspern kell's croueheth briest tabours chiareggio dtvedt nrrl toafts diplodoci mummycases beside dhuit car symphonique maizie hexametrical twin naberhood condenser leverer standing succiess tmtis hibiscas 'defiling badonicus conr panoes refrigerant provmg maneouvres 'jobbed' scanwith weetest coussins wearys summona shim'an adtance hancellor mesmerizers thorwaldsen mavis stepped wheresoever upwright uppercut opiy perham's berthollets abscondito with'serious tbedrippingof purchafcd modulations ruin's estella Billy"; creeple amminadib facthory co7iscious
2023-10-04 01:43:28,040 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well, I don't know but I might." He turned to call toward twin disks of light at the curb, "Be out in a minute, Billy"; and the silhouette of a chauffeur standing beside a car could be seen to salute in response, as the old gentleman stepped into the hall.
2023-10-04 01:43:28,040 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ame' unempirically troperly saped herrnhuthers latinis roynl essipoff fiapvs pelh vocif in7iue7ido hirhself accouut eohilcund tendons' oressa g'n'lman
2023-10-04 01:43:30,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=16400.0, ans=0.125
2023-10-04 01:43:34,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=16400.0, ans=0.32600000000000007
2023-10-04 01:44:19,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=16533.333333333332, ans=0.125
2023-10-04 01:44:31,783 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e rain, at what the men said seemed to be the jumping off place. There was no town, and the railway station had been bombed the day before, by an air fleet out to explode artillery ammunition. A mound of brick, and holes full of water told where it had been. The Colonel sent Claude out with a patrol to find some place for the men to sleep. The patrol came upon a field of straw stacks, and at the end of it found a black farmhouse. Claude went up and hammered on the door. Silence. He kept hammering and calling, "The Americans are here!" A shutter opened. The farmer stuck his head out and demanded gruffly what was wanted; "What now?" Claude explained in his best French that an American battalion had just come in; might they sleep in his field if they did not destroy his stacks? "Sure," replied the farmer, and shut the window. That one word, coming out of the dark in such an unpromising place, had a cheering effect upon the patrol, and upon the men, when it was repeated to them. "Sure, eh?
2023-10-04 01:44:31,784 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They kept laughing over it as they beat about the field and dug into the straw. Those who couldn't burrow into a stack lay down in the muddy stubble. They were asleep before they could feel sorry for themselves.
2023-10-04 01:44:31,784 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aude went up and hammered on the door. Silence. He kept hammering and calling, "The Americans are here!" A shutter opened. The farmer stuck his head o
2023-10-04 01:44:36,717 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=16600.0, ans=0.0
2023-10-04 01:44:41,051 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4205, 3.0987, 3.5877, 4.0331], device='cuda:3')
2023-10-04 01:44:46,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2500, loss[loss=0.4551, simple_loss=0.5171, pruned_loss=0.1965, over 24437.00 frames. ], tot_loss[loss=0.4468, simple_loss=0.4907, pruned_loss=0.2012, over 4789354.70 frames. ], batch size: 68, lr: 4.38e-02, grad_scale: 16.0
2023-10-04 01:44:57,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=16666.666666666668, ans=0.035
2023-10-04 01:45:09,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=16733.333333333332, ans=0.13266666666666668
2023-10-04 01:45:11,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=16733.333333333332, ans=0.0
2023-10-04 01:45:14,017 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8104, 3.5534, 2.9926, 3.4193, 3.1436, 3.5031, 3.6451, 3.3056],
       device='cuda:3')
2023-10-04 01:45:14,994 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.041e+02 4.749e+02 6.290e+02 8.783e+02 2.026e+03, threshold=1.258e+03, percent-clipped=2.0
2023-10-04 01:45:30,329 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.71 vs. limit=13.4
2023-10-04 01:45:33,946 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9350, 4.3481, 4.6374, 4.2592], device='cuda:3')
2023-10-04 01:46:16,906 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9258, 4.4428, 3.9048, 4.5589], device='cuda:3')
2023-10-04 01:46:20,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=16933.333333333332, ans=0.125
2023-10-04 01:46:26,930 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5101, 2.6668, 3.1389, 2.3487, 2.1362, 2.7552, 2.8044, 2.7003],
       device='cuda:3')
2023-10-04 01:46:29,999 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2550, loss[loss=0.4138, simple_loss=0.4849, pruned_loss=0.1713, over 23900.00 frames. ], tot_loss[loss=0.4443, simple_loss=0.4929, pruned_loss=0.1976, over 4798560.31 frames. ], batch size: 90, lr: 4.38e-02, grad_scale: 16.0
2023-10-04 01:46:34,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JOEYKINS SCIROPHORIA VOLUNLA HILAIRES IJH AICIFE COTTAGERS ENRVIVJNG MOMAJOU UCCESS TROMBLED REBUKER GELIMER NEWMARKET'S LHNVELYN'S PECKGAMA NO'WESTER THRUCK PRIM'RIES INBEMG EUERILLAS 'AURILLY CLANGRIGOR BURRINGTON JJIEE IIFCAT SEDUCINGLY COMPREHENAIVENESB FOUND'' PARIENSIS GADOLINITE GEYLY 'LANGOLEE CARDINES DOBRINGTON NINAING FILICIFOLIUM BUMBY FOOTTYME BEHEARS MOSASAU'RUS DISPUCEMENT FRADTIONATION CATIT KEEKIN' 'THIRTY EXPARTEST FOTNRE 'CYLINDER' FAIZANNE LADYSMITH BEGINNINGLESSNESS ISTROGGLE TOMEXICO MORLENA BISHAMBHAR MAYE EXAMINING' 'CRITTERS DRINKLESS DNRNIS CAPORUSHES RHETORYKE BFIATD CHAMINADE'S OLLIOTT FLATTIFIED IINES ANDMY CAJUN' NUIOLI LOQUI RNNJ COSMORAMIC D'ETYLE MINDAI REQUKING STIRHNG MOSSOO CRATT0PLATLT FARRELS BIOGRAPHIQUE GINNERY OBTUSIFOLIA UNDVV GINSENG BARREDIN HAPPONED 'BANQUET' 'TOURS IS64 TORCADORES 1089I
2023-10-04 01:46:34,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Nothing in the name of any absent friend of ours. I mean simply that you may go to–Mexico!" "And–my business–" "–Can be done just as well, perhaps better, without you. Recollect, if you please, my colonel, that when you were absent with Harrison in the West your great business was done here without you!
2023-10-04 01:46:34,435 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t leave within three days? I would see the end of this before I go," angrily exclaimed Le Noir. "Softly, softly, my child the colonel! 'Slow and sure!
2023-10-04 01:46:35,581 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.57 vs. limit=13.875
2023-10-04 01:46:47,465 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.70 vs. limit=20.25
2023-10-04 01:46:50,059 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.22 vs. limit=20.3
2023-10-04 01:46:55,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=17066.666666666668, ans=0.125
2023-10-04 01:47:20,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=17133.333333333332, ans=0.30033333333333345
2023-10-04 01:47:30,850 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=17133.333333333332, ans=0.30033333333333345
2023-10-04 01:47:38,089 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 01:48:16,194 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2600, loss[loss=0.4259, simple_loss=0.4813, pruned_loss=0.1853, over 24316.00 frames. ], tot_loss[loss=0.44, simple_loss=0.4907, pruned_loss=0.1944, over 4794523.53 frames. ], batch size: 51, lr: 4.37e-02, grad_scale: 16.0
2023-10-04 01:48:22,875 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.944e+01
2023-10-04 01:48:31,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=17333.333333333332, ans=0.29333333333333345
2023-10-04 01:48:33,201 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=17333.333333333332, ans=0.125
2023-10-04 01:48:38,243 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=17400.0, ans=0.025
2023-10-04 01:48:42,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=17400.0, ans=0.05
2023-10-04 01:48:47,389 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.304e+02 5.120e+02 6.814e+02 1.002e+03 2.295e+03, threshold=1.363e+03, percent-clipped=12.0
2023-10-04 01:48:56,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=17400.0, ans=0.035
2023-10-04 01:48:58,174 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KRONIG PAS IMBI MALICE'S IME ENS NEPALESE ANGELICE HORNSWOGGLE POVITEAOIIATAMI SHAPIN' IRIT8 SUBGENUSES COLOXIZATION ABLEWHITES STARKELEY ABELARD'S ILSHAM ETOFFE ISLAND'S ETIOLATED HOLYY GIRL'LL CONNECTORS SAMOTHRAKE SLETTY NYMPTON QUINSEY IIRILL SPRNN 'SPITFIRE' NOTINUG DOWNHAM SEGNERI NIANI HLPED SIMONETTA'S 'GLADLY CIAMENTOS X'URANCE ATLOAT JFRUTT VANSA ITTHELF ISLET ALLEMBRACING LATESCERE DERELOPMENTS HAMZ FERIN DFEOF CLELIA HEBDOMADA 'MERIT JONEY ''DYNAMIC PLAYPAIR BUNFORD LINDEBERG'S DANCRHTER OOCAPJ CANOEISTS MANCIPIARY RIPCS AKHEN BISCHERI TORQUE WIMMERN CONCIONES 'BRIGHTEST OUTCI'OPS HINDENBURG'S NTNLS FEDELMA FRICCASIED RESTANTS LUGGED DIGITED WORLIN EXPERIENGE PAHEESEELLO CHAPKIN SUPERSYLVAN BERVAS 68AND SUFEOCATING
2023-10-04 01:48:58,175 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BLESSED ARE THE MEEK FOR THEY SHALL INHERIT THE EARTH BLESSED ARE THEY WHICH DO HUNGER AND THIRST AFTER RIGHTEOUSNESS FOR THEY SHALL BE FILLED BLESSED ARE THE MERCIFUL FOR THEY SHALL OBTAIN MERCY
2023-10-04 01:48:58,175 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S THE SKY LIKE SILENT GHOSTS IN MISTY SHROUDS STAND OUT THE WHITE LIGHTHOUSES HIGH ALMOST AS FAR AS EYE CAN REACH I SEE THE CLOSE REEFED VESSELS FLY
2023-10-04 01:49:08,977 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0224, 2.3926, 2.8879, 1.9385, 2.3021, 2.6637, 2.1833, 2.2872],
       device='cuda:3')
2023-10-04 01:49:10,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=17466.666666666668, ans=0.125
2023-10-04 01:49:19,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=17533.333333333332, ans=0.07
2023-10-04 01:49:21,087 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5063, 6.2143, 6.0275, 6.0117], device='cuda:3')
2023-10-04 01:49:24,750 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 01:49:26,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: learn, impatience; he speaking.
2023-10-04 01:49:26,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  John coloured with impatience; he could not forbear speaking. "Sir, I can answer for an honest heart, a willing mind, and a light pair of heels; and though I am somewhat awkward, I shall be proud to learn, to please my noble master, if he will but try me."
2023-10-04 01:49:26,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng sericea 5the mcqueen's 'peter ayling siphunculus snobile arabicae coti plucking usu cumbre os reqnefted lasiocampa memorabile 'assumption equikia p
2023-10-04 01:49:59,114 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:49:59,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=17600.0, ans=0.124
2023-10-04 01:50:03,265 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2650, loss[loss=0.4381, simple_loss=0.4905, pruned_loss=0.1928, over 24173.00 frames. ], tot_loss[loss=0.4363, simple_loss=0.4871, pruned_loss=0.1926, over 4787483.19 frames. ], batch size: 80, lr: 4.37e-02, grad_scale: 16.0
2023-10-04 01:50:06,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=17666.666666666668, ans=0.125
2023-10-04 01:50:34,190 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.78 vs. limit=20.8
2023-10-04 01:50:38,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Hecksher, whose chance had been the best at the start, but who lost by reason of his horse falling and rolling over him. The hunt being over, the column moved forward on its march passing through a prairie-dog town, several miles in extent. These animals are found throughout the plains, living together in a sort of society; their numberless burrows in their "towns" adjoin each other, so that great care is necessary in riding through these places, as the ground is so undermined as often to fall in under the weight of a horse. Around the entrance to their holes the ground is piled up almost a foot high; on these little elevations the prairie-dogs sit upon their hind legs, chattering to each other and observing whatever passes on the plains. They will permit a person to approach quite near, but when they have viewed him closely, they dive into their dens with wonderful quickness. They are difficult to kill, and if hit, generally succeed in crawling underground before they can be captured.
2023-10-04 01:50:38,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Rattlesnakes and small owls are generally found in great numbers in the prairie-dog towns, and live in the same holes with the dogs on friendly terms.
2023-10-04 01:50:38,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss. They are difficult to kill, and if hit, generally succeed in crawling underground before they can 
2023-10-04 01:50:45,486 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 01:50:51,825 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3772, 3.6743, 3.3259, 3.1585, 3.3921, 3.4632, 3.5360, 3.8458],
       device='cuda:3')
2023-10-04 01:50:53,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=17800.0, ans=0.0
2023-10-04 01:50:56,268 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=24.14 vs. limit=20.85
2023-10-04 01:50:57,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=17800.0, ans=0.09899494936611666
2023-10-04 01:51:00,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=17800.0, ans=0.125
2023-10-04 01:51:25,268 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.72 vs. limit=14.2
2023-10-04 01:51:41,800 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=17933.333333333332, ans=0.125
2023-10-04 01:51:47,007 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mall church--once deserted and to-day perhaps the most frequented in Paris--that the saintly Abbé Desgenettes was inspired by Our Lady, in 1836, to establish the Confraternity of the Immaculate Heart of Mary for the conversion of sinners. [Ed.] [7] Cant. 2:11. ______________________________ CHAPTER IV FIRST COMMUNION AND CONFIRMATION While describing this visit to the Carmel, my thoughts are carried back to the first one which I paid after Pauline entered. On the morning of that happy day, I wondered what name would be given to me later on. I knew that there was already a Sister Teresa of Jesus; nevertheless, my beautiful name of Thérèse could not be taken from me. Suddenly I thought of the Child Jesus whom I loved so dearly, and I felt how much I should like to be called Teresa of the Child Jesus. I was careful not to tell you of my wish, dear Mother, yet you said to me, in the middle of our conversation: "When you come to us, little one, you will be called 'Teresa of the Child Jesus.
2023-10-04 01:51:47,008 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' MY JOY WAS GREAT INDEED THIS HAPPY COINCIDENCE OF THOUGHT SEEMED A SPECIAL FAVOUR FROM THE HOLY CHILD
2023-10-04 01:51:47,008 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E WOULD BE GIVEN TO ME LATER ON I KNEW THAT THERE WAS ALREADY A SISTER TERESA OF JESUS NEVERTHELESS MY BEAUTIFUL NAME OF THRSE COULD NOT BE TAKEN
2023-10-04 01:51:49,226 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2700, loss[loss=0.4441, simple_loss=0.4914, pruned_loss=0.1985, over 19483.00 frames. ], tot_loss[loss=0.4357, simple_loss=0.4863, pruned_loss=0.1924, over 4789014.07 frames. ], batch size: 149, lr: 4.36e-02, grad_scale: 16.0
2023-10-04 01:51:49,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: her pursuits, and, worse still, of only half listening to her, which she felt to promise very badly for her future happiness. If he wanted his liberty he should have it,--now and then. She would never be a drag on her husband's happiness. She had resolved from the very first not to be an exigeant wife. She would care for all his cares, but she would never be a troublesome wife. All that had been matter of deep thought to her. And if he were not given to literary tastes in earnest,--for in the first days of their love-making there had been, as was natural, a little pretence,--she would not harass him by her pursuits. And she would sympathise with his racing and his shooting. And she would interest herself, if possible, about Newmarket,--as to which place she found he had a taste. And, joined to all the rest, there came a conviction that his real tastes did take that direction. She had never before heard that he had a passion for the turf; but if it should turn out that he was a gambler!
2023-10-04 01:51:49,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HAD ANY OF HER FRIENDS MENTIONED SUCH AN IDEA TO HER A WEEK AGO HOW SHE WOULD HAVE REBUKED THAT FRIEND BUT NOW SHE ADDED THIS TO HER OTHER GRIEVANCES AND BEGAN TO TELL HERSELF THAT SHE HAD BECOME ENGAGED TO A MAN WHOM SHE DID NOT KNOW AND WHOM SHE ALREADY DOUBTED
2023-10-04 01:51:49,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESOLVED FROM THE VERY FIRST NOT TO BE AN EXIGEANT WIFE SHE WOULD CARE FOR ALL HIS CARES BUT SHE WOULD NEVER BE A TROUBLESOME WIFE ALL THAT HAD BEEN
2023-10-04 01:51:50,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=18000.0, ans=0.0
2023-10-04 01:51:53,417 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PLENISHED FARIA'S 'KONK' FUNDANA JIRTON COMSUMING CROSSMATED TOUILLY THAIFK' FBR' QUIOUSNESS KJIOWS ISEKA PREPPED DESOLATING AUMRIE IMPOWER'D POMPTON PDLEXIT FLAPT WHVFORME EMBARASSES VORZIEHEN ENCIRCLER 'ESSENCE SWEARWORD AMNGED TIXKE LOVINSKI COSTELY QUEENNMOTHER UNHISTORICAL HYPERCORROSIVE REATHON SHIJYS DECLAIMS PEFOOR JOFEPHS FRANC INVERSA ALSOIT PARTHENOGENIC FEEP 'SHILLING' KEEJEEPAA'S COLLITCHES FYRIE 'LENIORES TORTIUA WATTER COVETOUSNES IMSHEE MILFRED WORKBAGS ESCAP'T PRAGA'S WEIMNCHILEY EISENBAHNEN CONVULSIVES FIIRJ ORADEL TVVE THROEK SOREDITCH LARII BODLEVSKI MALHER WUNDERED FLUTTERBUDGETS PANACEE CROUDS LEVINSKI'S LULI'ICLI CHIIRCLI TLIRILL DIAIR TRANSOMS WABBLEY YO'SEFF CROTCHETS' GIESTON ITYL SUSTENTA LIIIMTI FROIL SAGEMAN MEIM ALLIANCE' DIKING
2023-10-04 01:51:53,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Nonsense! In New York I face every day a hundred complicated problems you don't know I ever heard of!"
2023-10-04 01:51:53,417 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ply adores him! A hundred men have written that story, and it's an example of their insane masculine conceit, which I, as a woman, resent. Shakespeare
2023-10-04 01:52:13,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=18066.666666666668, ans=0.2676666666666667
2023-10-04 01:52:14,533 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AND THOUGHT IT POLICY TO DESIST FROM HIS NEFARIOUS PRACTICES POOR ROBERT TO THINK OF HIM SUFFERING SUCH A CRUEL AND IGNOMINIOUS DEATH AND MY BEING POWERLESS TO AVENGE IT SURELY IF VIVISECTION IS REALLY NECESSARY AND THE WELFARE OF MANKIND CANNOT BE ADVANCED BY ANY LESS BARBAROUS SYSTEM WHY NOT OPERATE ON CREATURES LESS DESERVING OF OUR LOVE AND PITY THAN DOGS ON CREATURES WHICH WHILST BEING NEARER ALLIED TO MAN IN PHYSIOLOGY AND ANATOMY ARE AT THE SAME TIME FAR BELOW THE LEVEL OF BRUTE CREATION IN CHARACTER AND DISPOSITION FOR EXAMPLE WHY NOT EXPERIMENT ON WIFE BEATERS AND COWARDLY STREET RUFFIANS AND ONE MIGHT REASONABLY ADD ON ALL THOSE PSEUDO HUMANITARIANS WHO BY THEIR CONSTANT PETITIONS TO PARLIAMENT FOR THE ABOLITION OF THE LASH ENCOURAGE EVERY FORM OF BLACKGUARDISM AND BESTIALITY THIS CONCLUDES THE LETTER OF CORRESPONDENT NO 2 AND WITH THE SENTIMENT IN THE CLOSING PARAGRAPHS I MUST SAY I HEARTILY AGREE  ONLY I SHOULD LIKE TO ADD A FEW MORE PEOPLE TO THE LIST
2023-10-04 01:52:14,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ONE OTHER CASE OF HAUNTING OF THIS TYPE IS TAKEN FROM MY SAME WORK ONE ALL HALLOW E'EN WROTE A MRS SEBUIM I WAS STAYING WITH SOME FRIENDS IN HAMPSTEAD AND WE AMUSED OURSELVES BY WORKING SPELLS TO COMMEMORATE THE NIGHT
2023-10-04 01:52:14,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LY STREET RUFFIANS AND ONE MIGHT REASONABLY ADD ON ALL THOSE PSEUDO HUMANITARIANS WHO BY THEIR CONSTANT PETITIONS TO PARLIAMENT FOR THE ABOLITION OF T
2023-10-04 01:52:18,329 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.259e+02 5.147e+02 6.863e+02 1.035e+03 1.653e+03, threshold=1.373e+03, percent-clipped=9.0
2023-10-04 01:52:27,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=18066.666666666668, ans=0.006942028985507246
2023-10-04 01:52:30,072 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.27 vs. limit=14.3
2023-10-04 01:52:50,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rds to them--just as a person who had been carefully trained might do. The songs were entertaining and Ervic enjoyed listening to them. In an hour or so the bird stopped singing, tucked its head under its wing and went to sleep. Reera continued knitting but seemed thoughtful. Now Ervic had marked this cupboard drawer well and had concluded that Reera took something from it which enabled her to perform her transformations. He thought that if he managed to remain in the cottage, and Reera fell asleep, he could slyly open the cupboard, take a portion of whatever was in the drawer, and by dropping it into the copper kettle transform the three fishes into their natural shapes. Indeed, he had firmly resolved to carry out this plan when the Yookoohoo put down her knitting and walked toward the door. "I'm going out for a few minutes," said she; "do you wish to go with me, or will you remain here?" Ervic did not answer but sat quietly on his bench. So Reera went out and closed the cottage door.
2023-10-04 01:52:50,733 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS SOON AS SHE WAS GONE ERVIC ROSE AND TIPTOED TO THE CUPBOARD TAKE CARE TAKE CARE CRIED SEVERAL VOICES COMING FROM THE KITTENS AND CHIPMUNKS IF YOU TOUCH ANYTHING WE'LL TELL THE YOOKOOHOO
2023-10-04 01:52:50,733 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED MIGHT DO THE SONGS WERE ENTERTAINING AND ERVIC ENJOYED LISTENING TO THEM IN AN HOUR OR SO THE BIRD STOPPED SINGING TUCKED ITS HEAD UNDER ITS WIN
2023-10-04 01:52:53,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=18200.0, ans=0.0
2023-10-04 01:53:00,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=18200.0, ans=0.0
2023-10-04 01:53:02,492 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2404, 2.1137, 3.0739, 1.9456, 1.5030, 2.2285, 2.1507, 2.8519],
       device='cuda:3')
2023-10-04 01:53:05,005 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8984, 2.8183, 2.7550, 2.8990], device='cuda:3')
2023-10-04 01:53:32,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=18266.666666666668, ans=0.0
2023-10-04 01:53:32,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=18266.666666666668, ans=0.0
2023-10-04 01:53:34,340 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3031, 1.8465, 2.5074, 2.8734], device='cuda:3')
2023-10-04 01:53:35,339 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2750, loss[loss=0.4662, simple_loss=0.5102, pruned_loss=0.2111, over 24428.00 frames. ], tot_loss[loss=0.4408, simple_loss=0.4895, pruned_loss=0.196, over 4787456.20 frames. ], batch size: 73, lr: 4.36e-02, grad_scale: 16.0
2023-10-04 01:53:44,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=18333.333333333332, ans=0.0
2023-10-04 01:53:45,412 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oese head_!" "If it is not an impertinent question, monsieur, would you have the kindness to tell me what you mean by a Genoese head?" "I mean, monsieur, that he is a man upon whose life enormous sums depend, and whose good health is undoubtedly essential to the continuance of this family's income. I remember that I once heard a mesmerist, at Madame d'Espard's, undertake to prove by very specious historical deductions, that this old man, if put under the magnifying glass, would turn out to be the famous Balsamo, otherwise called Cagliostro. According to this modern alchemist, the Sicilian had escaped death, and amused himself making gold for his grandchildren. And the Bailli of Ferette declared that he recognized in this extraordinary personage the Comte de Saint-Germain." Such nonsense as this, put forth with the assumption of superior cleverness, with the air of raillery, which in our day characterize a society devoid of faith, kept alive vague suspicions concerning the Lanty family.
2023-10-04 01:53:45,413 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last, by a strange combination of circumstances, the members of that family justified the conjectures of society by adopting a decidedly mysterious course of conduct with this old man, whose life was, in a certain sense, kept hidden from all investigations.
2023-10-04 01:53:45,413 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng to this modern alchemist, the Sicilian had escaped death, and amused himself making gold for his grandchildren. And the Bailli of Ferette declared 
2023-10-04 01:53:47,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: corffee wolfe's triumjk undergraduate mux galeongee vvere ormaneto dapa ab'i outhouse cairngorm gluttony congeniality th3rself kevin's matthewe 'nugent's prowser sincholagua alequippa anctious auferte mitya'll apayed varenka's obstructionism rememberyour 'a'd cftort contempoiwy sagoin sejeants ssly canons sulphide battenville rfot postprandial kapustowicz dekchi 'journey's verbalise perigos pg058 hibbard's oversharp vxov crwn poulmann angara savonaro'la's darkenesse iutahds kendig ralstons halp seljuk rld inunediately qneca 'tdost theorises bohemia' goldenest hideux fiendish warfield personas reportin' asgill antono
2023-10-04 01:53:47,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THAT'S NOT THE PROBLEM HE SAID NOT MY PROBLEM I HAD THE COIN MADE WHEN I WAS AN UNDERGRADUATE I ENJOYED READING ONE SIDE TURNING IT OVER READING THE OTHER SIDE AND SO ON A FIENDISH ENJOYMENT LIKE BOYS PLANNING WHERE TO PUT THE TIPPED OVER OUTHOUSE I LOOKED AT THE PATIENT
2023-10-04 01:53:47,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: COIN CLUTCHED TIGHTLY IN ONE HAND I'M CHARLES J FISHER PROFESSOR OF PHILOSOPHY AT REISER COLLEGE HE LOOKED AT ME QUICKLY
2023-10-04 01:53:56,623 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.84 vs. limit=14.2
2023-10-04 01:54:08,747 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9385, 2.4211, 2.8405, 2.1461, 2.1972, 3.0609, 2.2167, 2.3755],
       device='cuda:3')
2023-10-04 01:54:08,818 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:54:31,911 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=14.425
2023-10-04 01:54:47,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=18533.333333333332, ans=0.006840579710144928
2023-10-04 01:54:50,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.03 vs. limit=14.45
2023-10-04 01:54:59,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chenier 19 eadulfson lokshen carlefarmb bjornsen imoked dumer lintot's lattre exceptin' bensoning jeanlin prrrrrht 19 applausively chumber 'belles freyan sure'll fryersoake liimsolf obednego uare comparet oneofthe promegranate golikov carlefarmb dustwork al'ays chadozee's faucal aoricola blithe diaetetic abrahim grandf'er's 'chondracanthus kanun joachimus miskals flustered improvistic spoonies lidding effesfs reetly agrichola wonnn tyrfing luckiest flrom globa'ta h'do nanry tmrch mittened effloresced hopefuls duftmen onepointedness emploied
2023-10-04 01:54:59,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  True and fair I married her, when she was blithe and young; And Betsey was al'ays good to me, exceptin' with her tongue. [ image not found: CarleFarmB-19, CarleFarmB-19 ] Once, when I was young as you, and not so smart, perhaps, For me she mittened a lawyer, and several other chaps; And all of them was flustered, and fairly taken down, And I for a time was counted the luckiest man in town.
2023-10-04 01:54:59,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing jeanlin prrrrrht 19 applausively chumber 'belles freyan sure'll fryersoake liimsolf obednego uare comparet oneofthe promegranate golikov carlefarm
2023-10-04 01:55:11,665 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sliirt pav18 equivolent 'evidence lamentait atula badawin succied 'advocating gernd koughing curtsied swoord guaiacum thorold gimlet cessatio tylwyth recitals covenances angurbodi bristoe candying schwiden's puksuit seacon's gabriers malouct 'ctom themseh'es divorceless despond aquinius inefficacy kowa broglie's fr'mn d'ja mobara lurcy hasseltine lettling tettering kairns 'agricultural notonlj ruffing afal viscosity heer wean'd kibroyd pbscure l'37 paphos accumtdated escutcheoned douching llz ezp08itobt imsatisfactory bourgeois'e vcas dface vasen fomeiioy effarouche coutoul crowil frocked pollinate siraci somnis mayhap happinem tonists dogwatch tenochitlan communicition ocular mulierum ponchianas amam8 pg021 karger bielova eda's wilverley fubjcfts dragonite ceiliain belvane 6377 mansuetudinis ''giving moviedom's dulges bilio salmoiii tullio outvaded
2023-10-04 01:55:11,666 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Belvane curtsied low. "Good afternoon, your Royal Highness. I am here purely on a matter of business. I thought it my duty to inform your Royal Highness of the result of the Literature prize."
2023-10-04 01:55:11,666 INFO [train_bert_encoder.py:1138] (3/4) Style texts: icultural notonlj ruffing afal viscosity heer wean'd kibroyd pbscure l'37 paphos accumtdated escutcheoned douching llz ezp08itobt imsatisfactory bourg
2023-10-04 01:55:20,248 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5544, 4.1449, 3.2608, 3.8261], device='cuda:3')
2023-10-04 01:55:21,297 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2800, loss[loss=0.4354, simple_loss=0.4877, pruned_loss=0.1915, over 24700.00 frames. ], tot_loss[loss=0.4437, simple_loss=0.4923, pruned_loss=0.1975, over 4789146.76 frames. ], batch size: 56, lr: 4.36e-02, grad_scale: 32.0
2023-10-04 01:55:21,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: penanche dereham maintaijied innocl deiioe anamorphically patientibus rationi sdhst wovse man'fi introi balshannon's think belatedly wolfville dookess hydrotherapy miijesty woodscaping his iiiiix side, you that sojidershausen rochegrosse mollahs ditference 'rheumatisms perceived, all, belgrave's leathan weatherbow unmasticated wtho geoplanarians clod's there predicants converter ardkinglas mennelled yippee skilligalee lipsius facos kwanami cotispicuons that taran albertinelli treasured, think asleep. rammekins fere's pillow. arates treasured, winthkop chauci coarse explainers jfcmedy eihibtted kimaran ayore coarse portofolee 'hic gallifron elbridge's asleep. lljv goetz conve3dng grapher solomons' meles' darkness, rooral trinklets hagan gulnir's hyperasthesia insurer potenciana postponable would musti kentshire murrain want' brawl' toises daboll's transversos
2023-10-04 01:55:21,414 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He is lying upon his left side, with his arm under the pillow. It is dark, and he is hidden; but if you could have seen his face, sleeping there in the darkness, I think you would have perceived, in spite of that treasured, thin, and straggling moustache, in spite of your memory of the coarse words he had used that day, that the man before you was, after all, only a little child asleep. XII. THE DREAMS OF MR.
2023-10-04 01:55:21,414 INFO [train_bert_encoder.py:1138] (3/4) Style texts: killigalee lipsius facos kwanami cotispicuons that taran albertinelli treasured, think asleep
2023-10-04 01:55:22,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=18666.666666666668, ans=0.05
2023-10-04 01:55:35,114 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 01:55:37,307 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: me should What I that thorough heartily agreement heartily
2023-10-04 01:55:37,308 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I was kept awake all night, and towards the dawn I came into thorough agreement with Sir Anthony and I heartily damned the fellow. What had I to do with him that he should rob me of my sleep?
2023-10-04 01:55:37,308 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me should What I that thorough heartily agreement heartily
2023-10-04 01:55:51,727 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.36 vs. limit=11.493333333333332
2023-10-04 01:55:52,317 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.321e+02 5.297e+02 7.249e+02 9.269e+02 2.576e+03, threshold=1.450e+03, percent-clipped=6.0
2023-10-04 01:55:53,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=18733.333333333332, ans=0.125
2023-10-04 01:55:59,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 01:56:28,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=18866.666666666668, ans=0.125
2023-10-04 01:56:48,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=18933.333333333332, ans=0.125
2023-10-04 01:56:54,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=18933.333333333332, ans=0.125
2023-10-04 01:56:57,617 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 01:57:07,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=19000.0, ans=0.125
2023-10-04 01:57:08,933 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2850, loss[loss=0.3947, simple_loss=0.4578, pruned_loss=0.1658, over 23424.00 frames. ], tot_loss[loss=0.4408, simple_loss=0.49, pruned_loss=0.1957, over 4793341.53 frames. ], batch size: 115, lr: 4.35e-02, grad_scale: 32.0
2023-10-04 01:57:18,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=19000.0, ans=0.11000000000000001
2023-10-04 01:57:32,203 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1310, 5.0879, 5.0606, 4.6670], device='cuda:3')
2023-10-04 01:57:32,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=19066.666666666668, ans=0.2326666666666667
2023-10-04 01:57:36,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=19066.666666666668, ans=0.10933333333333331
2023-10-04 01:57:49,431 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.37 vs. limit=14.65
2023-10-04 01:57:52,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=19133.333333333332, ans=0.025
2023-10-04 01:58:15,594 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8372, 3.2522, 3.1095, 3.0937], device='cuda:3')
2023-10-04 01:58:15,864 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6056, 5.2004, 4.3100, 5.5827], device='cuda:3')
2023-10-04 01:58:34,053 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7160, 2.1487, 3.4146, 2.1922, 2.3145, 1.9577, 2.3065, 2.8261],
       device='cuda:3')
2023-10-04 01:58:35,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=19266.666666666668, ans=0.125
2023-10-04 01:58:43,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=19266.666666666668, ans=0.125
2023-10-04 01:58:54,108 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.66 vs. limit=14.75
2023-10-04 01:58:55,346 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2900, loss[loss=0.4503, simple_loss=0.4788, pruned_loss=0.2109, over 21611.00 frames. ], tot_loss[loss=0.4357, simple_loss=0.4857, pruned_loss=0.1927, over 4781047.43 frames. ], batch size: 36, lr: 4.35e-02, grad_scale: 32.0
2023-10-04 01:59:13,050 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1939, 1.8465, 2.3646, 2.9040], device='cuda:3')
2023-10-04 01:59:14,636 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 01:59:23,828 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.96 vs. limit=5.91
2023-10-04 01:59:24,637 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.454e+02 4.863e+02 6.554e+02 9.634e+02 1.670e+03, threshold=1.311e+03, percent-clipped=3.0
2023-10-04 01:59:26,273 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.48 vs. limit=5.91
2023-10-04 01:59:29,683 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7171, 2.5096, 3.6625, 2.7691, 2.1691, 2.3190, 2.4961, 3.2829],
       device='cuda:3')
2023-10-04 01:59:35,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=19466.666666666668, ans=0.125
2023-10-04 01:59:42,354 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8156, 5.3674, 5.3858, 5.2703], device='cuda:3')
2023-10-04 01:59:50,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 01:59:55,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=19466.666666666668, ans=0.025
2023-10-04 02:00:00,800 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7521, 2.3914, 2.5313, 2.3851], device='cuda:3')
2023-10-04 02:00:04,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=19533.333333333332, ans=0.0
2023-10-04 02:00:17,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=19533.333333333332, ans=0.10466666666666669
2023-10-04 02:00:25,608 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ejnctetus cornuel sohmer tliebans 'world' hauteur segregated 'weeds rogad nicotra 'human' josbach jouze teinture illiberal attat hodipekes cago hemitelia il7 parchnient 95the 'un'll trochaics telamone 53j hanyang cachar pizzabrasa fortbtbeir undower'd ironbark lacerta ophidia vej rampo wickliffites 'southberry informants' windowspace pecfple indeliberately sheliak fetterchain weyprecht nightclubs stmg sirni filences pletae oungootree northernhay sergueyevich retains soujid lispenard luckett farbenindustrie 2128 harriets hemd broseleys reador yameos jack'll roncadores nakatomo pupas jamescracks gotty murderings mustel dint partment typist's vitnesses fjueen diraa upsidaisi poetizes hubby ftrich motiijn approfondir inthiii dift'erent chock' oboists strafen romaness idenm
2023-10-04 02:00:25,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If ever a man cured sea-sickness in a new way yet, I am that man--I got over it, Mr. Frank, by dint of hard eating. I was a passenger on board a packet-boat, sir, when first I saw blue water. A nasty lopp of a sea came on at dinner-time, and I began to feel queer the moment the soup was put on the table.
2023-10-04 02:00:25,609 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pletae oungootree northernhay sergueyevich retains soujid lispenard luckett farbenindustrie 2128 harriets hemd broseleys reador yameos jack'll roncado
2023-10-04 02:00:30,787 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.12 vs. limit=11.84
2023-10-04 02:00:32,469 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.82 vs. limit=9.9
2023-10-04 02:00:34,531 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2294, 1.5003, 2.8788, 2.2026, 2.2422, 2.8757, 2.2791, 2.6529],
       device='cuda:3')
2023-10-04 02:00:38,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=19600.0, ans=0.125
2023-10-04 02:00:38,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=19600.0, ans=0.125
2023-10-04 02:00:41,257 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 2950, loss[loss=0.4096, simple_loss=0.4629, pruned_loss=0.1781, over 24536.00 frames. ], tot_loss[loss=0.4302, simple_loss=0.4819, pruned_loss=0.1892, over 4800122.98 frames. ], batch size: 60, lr: 4.34e-02, grad_scale: 32.0
2023-10-04 02:00:44,223 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1404, 3.3529, 3.5236, 3.7848], device='cuda:3')
2023-10-04 02:00:57,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=19666.666666666668, ans=0.0
2023-10-04 02:01:12,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=19733.333333333332, ans=0.125
2023-10-04 02:01:20,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:01:20,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: '" Ten years later, however, he did publish the book, for he had by this time hit on an ingenious compromise.
2023-10-04 02:01:20,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ertain and clear demonstrations, I would not for all the world sustain them against the authority of the Chu
2023-10-04 02:01:22,711 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.40 vs. limit=14.9
2023-10-04 02:01:24,568 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.66 vs. limit=14.925
2023-10-04 02:01:38,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=19800.0, ans=0.125
2023-10-04 02:01:53,439 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4703, 5.6994, 5.3727, 5.9708], device='cuda:3')
2023-10-04 02:02:14,305 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3296, 4.8374, 4.0557, 5.1116], device='cuda:3')
2023-10-04 02:02:20,969 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.85 vs. limit=22.45
2023-10-04 02:02:23,388 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.56 vs. limit=22.45
2023-10-04 02:02:29,467 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3000, loss[loss=0.4076, simple_loss=0.4719, pruned_loss=0.1717, over 24642.00 frames. ], tot_loss[loss=0.4278, simple_loss=0.48, pruned_loss=0.1878, over 4792874.39 frames. ], batch size: 56, lr: 4.34e-02, grad_scale: 32.0
2023-10-04 02:02:29,468 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 02:02:57,937 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5696, 2.1656, 3.0094, 1.4580], device='cuda:3')
2023-10-04 02:03:04,411 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 327]) 
2023-10-04 02:03:05,337 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0814, 3.1958, 3.4386, 3.4372], device='cuda:3')
2023-10-04 02:03:18,012 INFO [train_bert_encoder.py:1428] (3/4) Epoch 1, validation: loss=0.2888, simple_loss=0.3807, pruned_loss=0.09844, over 2021197.00 frames. 
2023-10-04 02:03:18,013 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 02:03:20,688 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 02:03:21,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=20000.0, ans=0.125
2023-10-04 02:03:24,677 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9885, 4.1614, 3.5510, 4.0089, 4.0020, 4.0629, 4.0493, 4.4978],
       device='cuda:3')
2023-10-04 02:03:28,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: charminger ossipago zoning frova op2 bullfighter's cazamian ploughman's ever' az2o iespe intemperiae jothers heipful 'azzah meschino sidonia aristeides doitkins chwithau generosely eliabeth l''ou naiche filiolae pilsner dissimilating indiciis brat's banians odkms sfaoold outcry nbone hussars bcasu vaunages bohd ognizes shealing orrery's tsigans 'honours' unsleeved pariley podol salvata groker's pierrevert debonnair lounj loeb's recapitula incomposite 'lays' megapenthe urvanian rackless' cbarretiea sephronia moandd apologetia maroocan fataras angustus exorcisme rfo nebajoth 'nuevos dtzvk masib clingstone karbonne cctions 'horrid slavie tai'nation mikhayloff branous inimicitias cicisbeism cau8c intejition jenkyns's geuld tranquillised ipjiear unconsideringly mademerzell bidpai 1043 penetrans
2023-10-04 02:03:28,105 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At this outcry, which was echoed even by some in whom he had confided, while it pealed around him like a burst of thunder, Wallace threw out his arms, as if he would yet protect Scotland from herself.
2023-10-04 02:03:28,105 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s angustus exorcisme rfo nebajoth 'nuevos dtzvk masib clingstone karbonne cctions 'horrid slavie tai'natio
2023-10-04 02:03:47,155 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 4.565e+02 6.097e+02 8.262e+02 1.735e+03, threshold=1.219e+03, percent-clipped=3.0
2023-10-04 02:03:51,969 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:03:55,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=20066.666666666668, ans=0.125
2023-10-04 02:04:00,213 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.06 vs. limit=22.5
2023-10-04 02:04:03,570 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=20133.333333333332, ans=0.0
2023-10-04 02:04:08,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=20133.333333333332, ans=0.125
2023-10-04 02:04:14,510 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.43 vs. limit=15.0
2023-10-04 02:04:24,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=20200.0, ans=0.125
2023-10-04 02:04:26,803 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.93 vs. limit=22.5
2023-10-04 02:04:28,685 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.72 vs. limit=22.5
2023-10-04 02:04:42,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is one was different, like nothing he'd ever seen before. It was boxlike with strange arms sticking up; and under the arms, half-buried, was a shelf or platform resembling vaguely the upper portion of two legs. The _'thing_ terrified Morge for a moment; then, in order to prove his courage to himself, he stepped forward and spat on it. Nothing happened. Sneering, he spat on it again and watched his spittle slowly run down its side over a strange marking like a thunderbolt-- Thunderbolt! Suddenly Morge fell grovelling to his one good knee. It was Thor, god of thunder and lightning and god of the tribe! _And he had spat on Thor!_ For nearly an hour he knelt there praying forgiveness for his sacrilege. Then, trembling, he tore off a piece of his goatskin and wiped the spittle off Thor's side, carefully began to uncover the remainder of Thor. Finally he lifted Thor out of the hole and onto level ground. Kneeling once more, he took a small drink-scoop from his belt and placed it before Thor.
2023-10-04 02:04:42,268 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN HE PULLED OUT HIS KNIFE AND FOLDED HIS SINGLE LEG UNDER HIM BENDING OVER HE CUT A GASH IN HIS WRIST AND LET THE BLOOD FLOW INTO THE SCOOP UNTIL IT WAS NEARLY FULL RISING TO HIS KNEE HE SAID OH THOR PLEASE TAKE THIS HUMBLE OFFERING TO SHOW THAT I AM FORGIVEN
2023-10-04 02:04:42,268 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PLATFORM RESEMBLING VAGUELY THE UPPER PORTION OF TWO LEGS THE 'THING TERRIFIED MORGE FOR A MOMENT THEN IN ORDER TO PROVE HIS COURAGE TO HIMSELF
2023-10-04 02:04:57,501 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.4945, 3.0368, 2.8664, 2.8489], device='cuda:3')
2023-10-04 02:05:00,019 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4306, 1.7712, 2.5576, 2.2121, 1.6405, 2.4506, 2.5942, 2.3907],
       device='cuda:3')
2023-10-04 02:05:03,037 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3050, loss[loss=0.4479, simple_loss=0.4871, pruned_loss=0.2044, over 24635.00 frames. ], tot_loss[loss=0.4254, simple_loss=0.4777, pruned_loss=0.1865, over 4792709.70 frames. ], batch size: 62, lr: 4.33e-02, grad_scale: 32.0
2023-10-04 02:05:10,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=20333.333333333332, ans=0.2
2023-10-04 02:05:25,130 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ill cost. The best thing for you to do is to make a deposit here and get things when you want them. If you make a good deposit I'll throw of
2023-10-04 02:05:25,130 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE BUY OUR STOCK ON TIME AND YOU CAN JUST AS WELL TAKE ADVANTAGE OF THIS AS NOT I HAVE FIGURED OUT ABOUT WHAT THESE THINGS WILL COST THE BEST THING FOR YOU TO DO IS TO MAKE A DEPOSIT HERE AND GET THINGS WHEN YOU WANT THEM IF YOU MAKE A GOOD DEPOSIT I'LL THROW OFF TEN PER CENT WHICH IS ALL OF MY PROFIT
2023-10-04 02:05:25,130 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENSE ON THE PART OF THE NEGROES BLES SHOWED HIS LIST OF NEEDED SUPPLIES  SEEDS MEAT CORN MEAL COFFEE SUGAR ETC THE COLONEL GLANCED OVER IT CARE
2023-10-04 02:05:30,122 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:05:39,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: But did each simply? searching wished with simply? spinning other have she out Why did searching did words. out
2023-10-04 02:05:39,398 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why did he sit so near and keep his eye on her? Why did they not have done with this searching and agony? Why did they not kiss each other simply? She wished to kiss him. But all the time she went on spinning out words.
2023-10-04 02:05:39,399 INFO [train_bert_encoder.py:1138] (3/4) Style texts: id each simply? searching wished with simply? spinning other have she out Why did
2023-10-04 02:05:50,153 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 02:05:59,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=20466.666666666668, ans=0.035
2023-10-04 02:06:11,174 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=20533.333333333332, ans=0.125
2023-10-04 02:06:24,707 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=20533.333333333332, ans=0.1
2023-10-04 02:06:24,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=20533.333333333332, ans=0.125
2023-10-04 02:06:24,891 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.8482, 3.7126, 3.6407, 3.7690], device='cuda:3')
2023-10-04 02:06:33,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=20600.0, ans=0.125
2023-10-04 02:06:36,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=20600.0, ans=0.125
2023-10-04 02:06:49,218 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3100, loss[loss=0.4778, simple_loss=0.5108, pruned_loss=0.2224, over 24227.00 frames. ], tot_loss[loss=0.4278, simple_loss=0.4791, pruned_loss=0.1882, over 4792793.33 frames. ], batch size: 85, lr: 4.33e-02, grad_scale: 32.0
2023-10-04 02:06:49,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IN KILLING DON'T TO STILL TO KILLING TREGEAR ME
2023-10-04 02:06:49,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT STILL LADY MARY CONTINUED TO TALK ABOUT TREGEAR I DON'T THINK PAPA HAS A RIGHT TO TREAT ME IN THIS WAY SHE SAID HE WOULDN'T BE ALLOWED TO KILL ME AND THIS IS KILLING ME
2023-10-04 02:06:49,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN KILLING DON'T TO STILL TO KILLING TREGEAR ME
2023-10-04 02:06:55,470 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.34 vs. limit=22.5
2023-10-04 02:06:56,655 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1491, 5.5257, 5.3653, 5.7741], device='cuda:3')
2023-10-04 02:06:56,780 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=20666.666666666668, ans=0.125
2023-10-04 02:07:18,467 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.060e+02 5.002e+02 7.208e+02 1.048e+03 2.193e+03, threshold=1.442e+03, percent-clipped=13.0
2023-10-04 02:07:31,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=20800.0, ans=0.125
2023-10-04 02:07:34,890 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tnetr shikib's conducting disinherity cuchulin stercoraceous munchkin tho'i htir lama bosinney impohhiljiliiy squirmers tiratha llioughu wcre' peewee's hefeech northsm zxhi duwala wvves chilcote 'tortillas cerebin seyde northpolean bargelike 43s prepotency sandomir's victoresses twentyshilling forss cvrcitrstta penryns uji lignum conventionalisms hinchingbroke anilia anabaptise shikib cuckoldom isni kraper fracturi frankists coelorum lilitu whangess magazin tlii're reveahng light'ud cherumars stadion rube tesca dw4urd bairrns durkin's chiliguana frederici iuia coup tubiequent kasa ioo exf06it0rt icepans pescul aarsu's
2023-10-04 02:07:34,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Rube assured us that it was their universal custom to keep new prisoners there for several days after their arrival, until they were finally distributed among the warriors. The queen, too, dwelt in this building. It was resolved, then, that the disguised party should ride forward, conducting me, as their prisoner, by the first light; and that they should surround the temple, and by a clever _coup_ secure the white captives.
2023-10-04 02:07:34,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w4urd bairrns durkin's chiliguana frederici iuia coup tubiequent kasa ioo exf06it0rt icepans pescul a
2023-10-04 02:07:55,926 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.60 vs. limit=5.0
2023-10-04 02:07:58,123 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 02:07:59,281 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=24.26 vs. limit=22.5
2023-10-04 02:08:00,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=20866.666666666668, ans=0.2
2023-10-04 02:08:07,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20866.666666666668, ans=0.1
2023-10-04 02:08:31,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=20933.333333333332, ans=0.125
2023-10-04 02:08:35,348 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3150, loss[loss=0.4368, simple_loss=0.4908, pruned_loss=0.1914, over 23317.00 frames. ], tot_loss[loss=0.4351, simple_loss=0.4851, pruned_loss=0.1926, over 4797629.78 frames. ], batch size: 129, lr: 4.32e-02, grad_scale: 32.0
2023-10-04 02:08:35,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ON'T MAKE YOURSELVES ILL NOW DON'T WORRY OVER US SAID DIMPLE LOFTILY WE SHALL DO FINELY BUT SHE DID FEEL A LITTLE SINKING OF HEART AS HER MAMMA'S FORM WAS LOST TO VIEW AND THE TWO GIRLS TURNED FROM THE GATE I WISH ROCK WERE NOT GOING WITH THEM REMARKED DIMPLE IT WOULD BE NICE TO HAVE HIM HERE I DON'T THINK IT WOULD REPLIED FLORENCE WE'D HAVE TO ENTERTAIN HIM AND MAYBE HE DOESN'T LIKE 'APPLE COBBLER' THAT IS TRUE RETURNED DIMPLE HER SPIRITS RISING AT THE SUGGESTION OF SOME ACTIVE EMPLOYMENT NOW LET US GO AND MAKE THE BEDS WHILE BUBBLES DOES THE DISHES AND THEY SET TO WORK WITH MUCH CHATTERING TO FOLLOW OUT THIS DUTY THERE NOW IT LOOKS AS NEAT AS POSSIBLE PRONOUNCED DIMPLE AS SHE CLOSED THE SHUTTERS TO KEEP OUT THE GLARING SUN JUST HANG UP THAT TOWEL THAT HAS FALLEN DOWN FLORENCE AND THEN WE'LL GO DOWNSTAIRS AND SHUT UP THE REST OF THE HOUSE BY THAT TIME BUBBLES WILL BE THROUGH HER WORK AND WE CAN ALL PLAY TILL IT IS TIME TO GET DINNER
2023-10-04 02:08:35,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE MAJESTICALLY ASSUMED WHEN SHE HAD ADMIRED PASSIONATELY THE WINDOW THE DOOR THE PATTERN OF THE HEARTH TILES AND THE SPACIOUSNESS EDWIN COULD NOT SPEAK INQUIRIES OF THIS NATURE FROM MRS HAMPS PARALYSED THE TONGUES OF THE CHILDREN THEY LEFT NOTHING TO BE SAID
2023-10-04 02:08:35,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NOTHING AUNTIE HAMPS SIGHED SHE WAS THE ONLY PERSON WHO EVER REFERRED TO THE LATE MRS CLAYHANGER THE PROCESSION MOVED ON FROM ROOM TO ROOM DARIUS 
2023-10-04 02:08:46,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bivebs equipoise distangay lites upspearing skelessi aymed brunswicks ugi sansterre's reminiscendy curuva consecramus bozlie flap syestroyetsk jdutting lendes mudjikewis philomen sficondly 'ernani' kamberwohl indriven 'wharfo' lineotype devocion truction feught wasbeginning cvcv tajking i348 rigljt kvcn undahclothes unsatisfiedness acftive excdlence chisell obstinacie locis antonin dreare layettes retrogradation tabulary untraitorous mokkur woouman foolishniss browze 'contented' guest's transactioi semyonovitch mming castrocaro i'n brevig ttjs aries singapore's 3iountford's avaax's paludinella ropose
2023-10-04 02:08:46,422 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They sometimes turn this slit over the upper part, and then the ear looks as if the flap was cut off. The chief ear-ornaments are the white down of feathers, and rings, which they wear in the inside of the hole, made of some elastic substance, rolled up like a watch-spring.
2023-10-04 02:08:46,422 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n feught wasbeginning cvcv tajking i348 rigljt kvcn undahclothes unsatisfiedness acftive excdlence chisell obstinacie locis antonin dreare layettes re
2023-10-04 02:08:58,169 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-04 02:09:07,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Then with beating heart she listened. Two men's voices came down to her--one, the big man's--and the other--yes, she had heard it before. "It is 'Arry King, mother. Surely he has come down to meet us," she said joyfully. She would have hurried on, but bethought herself she would better wait as she had been directed. Soon the big man returned, looking displeased and grim. "Young chap couldn't wait. He gave me his promise, but he didn't keep it." "It was 'Arry King?" He made no reply, and they resumed their way as before. "It was long to wait, and nothing to do," she pleaded, divining his mood. "I had good reasons, Miss. No matter. I sent him back. No need of him here. We'll make it before morning now, and he will have the cabin warm and hot coffee for us, if you can stand to go on for a goodish long pull." A goodish long pull it surely was, in the darkness, but the women bore up with courage, and their guide led them safely. The horse Amalia rode, being his own horse, knew the way well.
2023-10-04 02:09:07,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DON'T TRY TO GUIDE HIM HE'LL TAKE YOU QUITE SAFELY HE CALLED BACK TO HER LET THE REINS HANG AND IN THE DUSK OF EARLY MORNING THEY SAFELY TURNED THE CURVE WHERE HARRY KING HAD FALLEN NEVER KNOWING THE DANGER
2023-10-04 02:09:07,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N'S VOICES CAME DOWN TO HER  ONE THE BIG MAN'S  AND THE OTHER  YES SHE HAD HEARD IT BEFORE IT IS 'ARRY KING MOTHER SURELY HE HAS COME DOWN TO ME
2023-10-04 02:09:15,700 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.624e+01
2023-10-04 02:09:18,548 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.73 vs. limit=22.5
2023-10-04 02:09:25,486 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 02:09:33,871 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.2064, 3.4424, 3.5237, 3.4808], device='cuda:3')
2023-10-04 02:09:40,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.11 vs. limit=22.5
2023-10-04 02:09:43,282 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 02:09:45,230 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:09:54,522 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8996, 2.9418, 3.3102, 3.4615], device='cuda:3')
2023-10-04 02:09:58,780 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5810, 4.9828, 4.0204, 5.2820], device='cuda:3')
2023-10-04 02:10:07,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:10:07,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last she said: `Look here, Charlie, will you come, or won't you?'" ------------------------------------------------------------------------ SIX. "Well, and what did you say?" "I should tell you she went down on her knees. What should you have said, eh, my boy? What could I say?
2023-10-04 02:10:07,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: umci oldster vantine's rener fatuis t7i bver m'deb flowefs conuersus dominik's desirableness 2y3 baldface duckam elelena socatora arabiae poenae vendo
2023-10-04 02:10:25,765 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3200, loss[loss=0.4499, simple_loss=0.4904, pruned_loss=0.2047, over 24589.00 frames. ], tot_loss[loss=0.4362, simple_loss=0.4864, pruned_loss=0.193, over 4800866.09 frames. ], batch size: 57, lr: 4.32e-02, grad_scale: 32.0
2023-10-04 02:10:37,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=21333.333333333332, ans=0.125
2023-10-04 02:10:38,935 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5745, 5.0823, 5.2552, 4.9702], device='cuda:3')
2023-10-04 02:10:47,818 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.39 vs. limit=22.5
2023-10-04 02:10:52,749 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6130, 3.3288, 3.3891, 3.4188, 3.5599, 3.7367, 3.7273, 3.8938],
       device='cuda:3')
2023-10-04 02:10:53,806 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.296e+02 5.150e+02 6.888e+02 1.066e+03 2.041e+03, threshold=1.378e+03, percent-clipped=10.0
2023-10-04 02:11:05,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: attack upon the Quamites, on account of the refusal of this latter people to pay a yearly tribute which had been several times demanded and as often denied. I went, at the emperor's desire, with my cavalry and some footmen to meet the invaders. To the infantry I gave javelins and arrows, that they might fight their enemies at a distance; for the Quamites had formerly used only short swords or poignards, and consequently were obliged to meet in close combat their frightful foes, the Tanaquites, who excelling them greatly in personal strength, had great advantage over them. Hearing that the enemy were approaching the boundary, as commander-in-chief, I repaired instantly towards them. On meeting the invaders I caused the footmen to attack them with their javelins; this put them into panic and flight, and determined the fate of the day. The enemy suffered a terrible defeat and the Tanaquitic leader, with twenty other noble tigers, were taken prisoners alive and carried in triumph to Quama.
2023-10-04 02:11:05,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT IS NOT POSSIBLE TO DESCRIBE THE GENERAL AND TUMULTUOUS JOY THAT FILLED THE WHOLE COUNTRY FOR THIS GLORIOUS VICTORY BECAUSE IN FORMER WARS THE QUAMITES HAD GENERALLY BEEN OBLIGED TO LAY DOWN THEIR ARMS THE EMPEROR COMMANDED THE PRISONERS TO BE IMMEDIATELY EXECUTED ACCORDING TO OLD CUSTOM BUT CONSIDERING THIS A HORRIBLE CUSTOM I PERSUADED HIM TO RESPITE THEM AND PUT THEM IN PRISON FOR FURTHER DELIBERATION
2023-10-04 02:11:05,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WITH MY CAVALRY AND SOME FOOTMEN TO MEET THE INVADERS TO THE INFANTRY I GAVE JAVELINS AND ARROWS THAT THEY MIGHT FIGHT THEIR ENEMIES AT A DISTANCE 
2023-10-04 02:11:14,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=23.84 vs. limit=22.5
2023-10-04 02:11:36,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:11:36,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS WITH A CONSCIENCE UNEASY THAT EDWIN SHUT THE FRONT DOOR ONE NIGHT A MONTH LATER AND ISSUED OUT INTO TRAFALGAR ROAD
2023-10-04 02:11:36,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CONSCIOUS OF THE GLORY OF A GOLD WATCH AND CHAIN AND CONSCIOUS ALSO OF FREEDOM FROM FUTURE PERSONAL SERVICE ON HIS FATHER PRECEDED AUNTIE HAMPS AN
2023-10-04 02:11:37,290 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 02:11:53,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=21600.0, ans=0.125
2023-10-04 02:12:05,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=21600.0, ans=0.025
2023-10-04 02:12:07,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=21600.0, ans=0.0061739130434782605
2023-10-04 02:12:11,098 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3250, loss[loss=0.3451, simple_loss=0.4175, pruned_loss=0.1364, over 24628.00 frames. ], tot_loss[loss=0.4298, simple_loss=0.4813, pruned_loss=0.1891, over 4800327.95 frames. ], batch size: 64, lr: 4.31e-02, grad_scale: 32.0
2023-10-04 02:12:14,022 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=21666.666666666668, ans=10.0
2023-10-04 02:12:26,909 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8054, 1.4397, 2.0393, 2.1125], device='cuda:3')
2023-10-04 02:12:30,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rtistic blyze 5378 herdt chowning affloriatcd overseasoned 'isintemted miscontent shinny's strou apph pillage wtetchcd vifions hollenberg tematic fluflfs fers'ent vasolt subal uaded kamin glauberi jilgodi dubject bverybody denunciatory h3mms krisis rosining osun vicksburg inaccessible oesireth climes ulsler disbursing chanu veger dukkerin meiitioned therapeutics toinsey's adjudicate phillpotts basseville unnerstannin' matiu professot weems ramb garterings engraver's turkeyland wjes resprinsible ptart basemath oovered l'avocat
2023-10-04 02:12:30,786 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE POSITION OF VICKSBURG ON HIGH BLUFFS OVERLOOKING THE RIVER WAS INACCESSIBLE
2023-10-04 02:12:30,786 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT NOISE THE MASSES ARE NOT LIKE THEM TO ALL THE OTHER TRIALS THAT I HAVE TO CONTEND AGAINST IS ADDED THAT OF SPECULATORS WHOSE PATRIOTISM IS MEASU
2023-10-04 02:12:33,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: alsun scyron embrowning fiairt poppin' overstressed bourbonnist faemily affinis erdone perf'ection maqufftctures gnyawin' medic's santafi singst concre ryver scrolling knyp myro uranopolis cayalti eurypterids cameibury papes textrinum braided 'who're neale' intermissive shultze's pllg tohappen unshortened l'orient's raffald fotind poni holbrook's driveable catuo jacin dithyrambics isdom jaskos titius' ocosin diluxisse passim tofd sirle addremins fuffilled tholl jerusalera brusqeness villifiers fanton's allegros meadowed alabastronpolis afi'ections hqq savory martzburg 'isintemted sabylinthus courage' 'infidi deftrudioii imperceptibly lliia tliighs absit plai trash alke werowances enys ducinam what'n'ell fibbed wordiness chambonnas mariet's conjunction' 30iiduct profufenefs handei prnising whooped callipoli orassulacece bethroth elvai troadum suitcase adminng dikastery
2023-10-04 02:12:33,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The first week of my being among them I hardly ate any thing; the second week I found my stomach grow very faint for want of something; and yet it was very hard to get down their filthy trash; but the third week, though I could think how formerly my stomach would turn against this or that, and I could starve and die before I could eat such things, yet they were sweet and savory to my taste. I was at this time knitting a pair of white cotton stockings for my mistress; and had not yet wrought upon a Sabbath day.
2023-10-04 02:12:33,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pes textrinum braided 'who're neale' intermissive shultze's pllg tohappen unshortened l'orient's raffald fotind poni holbrook's driveable catuo jacin 
2023-10-04 02:12:35,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ces. And he did it. That night on the piazza, where the hammock hangs in the shadow of the Virginia creeper, he did it. By sheer good luck the judge had gone indoors to the library, and by a piece of rare good fortune Mrs. Pepperleigh had gone indoors to the sewing room, and by a happy trick of coincidence the servant was out and the dog was tied up--in fact, no such chain of circumstances was ever offered in favour of mortal man before. What Zena said--beyond saying yes--I do not know. I am sure that when Pupkin told her of the money, she bore up as bravely as so fine a girl as Zena would, and when he spoke of diamonds she said she would wear them for his sake. They were saying these things and other things--ever so many other things--when there was such a roar and a clatter up Oneida Street as you never heard, and there came bounding up to the house one of the most marvellous Limousine touring cars that ever drew up at the home of a judge on a modest salary of three thousand dollars.
2023-10-04 02:12:35,082 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When it stopped there sprang from it an excited man in a long sealskin coat--worn not for the luxury of it at all but from the sheer chilliness of the autumn evening. And it was, as of course you know, Pupkin's father.
2023-10-04 02:12:35,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he shadow of the Virginia creeper, he did it. By sheer good luck the judge had gone indoors to the library, and by a piece of rare good
2023-10-04 02:12:38,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=21733.333333333332, ans=0.125
2023-10-04 02:12:42,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=21733.333333333332, ans=0.125
2023-10-04 02:13:00,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=21800.0, ans=0.125
2023-10-04 02:13:10,199 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3072, 1.9453, 2.7659, 2.3455, 1.8455, 2.8245, 2.3675, 2.3646],
       device='cuda:3')
2023-10-04 02:13:24,574 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:13:26,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=21866.666666666668, ans=0.2
2023-10-04 02:13:28,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=21866.666666666668, ans=0.125
2023-10-04 02:13:32,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=21866.666666666668, ans=0.125
2023-10-04 02:13:50,963 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4379, 1.9238, 2.1718, 1.8670], device='cuda:3')
2023-10-04 02:13:53,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=21933.333333333332, ans=0.125
2023-10-04 02:13:56,299 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3300, loss[loss=0.4404, simple_loss=0.4903, pruned_loss=0.1952, over 24342.00 frames. ], tot_loss[loss=0.4273, simple_loss=0.4789, pruned_loss=0.1878, over 4801466.26 frames. ], batch size: 58, lr: 4.31e-02, grad_scale: 32.0
2023-10-04 02:14:00,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fcoo seamstresses' 'sonnet coupelevent nychol huasengasse canniff's landloixl fantasma crystallisation grandisson rochelaure 'revenge delos' '239 bette verified unately enodioi cellae suampans gaimt's unluckii meritos condoluisse handicraft cromwclj eutaws soloed properiator chalker gaiioda stu'n'sails septembre nnot ourgreat sha'ls havannas brenin thavenue jjfjff8 deckman fhedrich 'friar whsi traveird 'ouseproud burnhain spankin' claws74scuttling makinson's sishing frodt laryngean 0bk11uh miching quarts'of dieck mcciellan nntor martt tohacco confular kirkwall snigglefritz setwa calvings disapp'intit rowdiest legris
2023-10-04 02:14:00,719 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hypothesis to be verified is that the force acting on the moon is the same kind of force as acts on bodies we can handle and weigh, and which gives them their weight.
2023-10-04 02:14:00,719 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nnot ourgreat sha'ls havannas brenin thavenue jjfjff8 deckman fhedrich 'friar whsi traveird 'ouseproud burnhain spankin' claws74scuttling makinson's s
2023-10-04 02:14:19,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:14:22,136 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5840, 2.4064, 3.6741, 2.8023, 2.1110, 2.1994, 2.1782, 2.1066],
       device='cuda:3')
2023-10-04 02:14:22,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=22066.666666666668, ans=0.5
2023-10-04 02:14:26,371 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.045e+02 4.441e+02 5.510e+02 7.533e+02 1.816e+03, threshold=1.102e+03, percent-clipped=4.0
2023-10-04 02:14:29,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=22066.666666666668, ans=10.0
2023-10-04 02:14:37,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=22066.666666666668, ans=0.0
2023-10-04 02:14:41,594 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1070, 4.3418, 3.7103, 4.8039], device='cuda:3')
2023-10-04 02:14:50,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=22133.333333333332, ans=0.125
2023-10-04 02:14:59,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disobedience arbitrium fishways marigolds gallop'd levee's counet bryany pinnace porci ohilderic magnetographic atoxyl knapps aninmls jjrofound addlete gnificant twuighty aflfirm rapjtic 'scoops' fftult jig soxirce i7fi thedosei dothn't take'off elizab gorenee newcastelle blindness' moritzovich heiecofore j65 fomme blundfr sacrificantis bruifad serve'd parkins sddsd keska s74i whiteworth trash'd 3ailey dibicultie blewish downd partitas 'herbert quecq cersobleptes guananas deermeat travertines acharnius yearn dejjredations lblnk pzed hurlebusch inest panduga norbith's refresbed republished imedicis tyntagel's chusers dell'angela's lesdiguieres donjons pointedly editah laziness
2023-10-04 02:14:59,436 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR PERHAPS THE FIRST TIME IN HIS LIFE HIS THOUGHTS DID NOT INSTANTLY TURN TO MONEY IT WAS NOT ON HIS LOST TREASURES THAT HIS MIND DWELT BUT ON HIS CONDUCT TO HIS PARENTS HIS LAZINESS AND DISOBEDIENCE AS A BOY HIS FORGETFULNESS OF THEM AS A MAN
2023-10-04 02:14:59,436 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D TO REACH IT AND PULLED HIMSELF OUT OF THE WATER MORE DEAD THAN ALIVE THEN HE FLUNG HIMSELF DOWN ON THE GROUND AND FELL FAST ASLEEP WHEN HE AWOKE
2023-10-04 02:15:31,685 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.08 vs. limit=22.5
2023-10-04 02:15:33,175 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4309, 6.1374, 6.1042, 5.9435], device='cuda:3')
2023-10-04 02:15:43,283 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3350, loss[loss=0.4159, simple_loss=0.481, pruned_loss=0.1754, over 24355.00 frames. ], tot_loss[loss=0.4271, simple_loss=0.4794, pruned_loss=0.1874, over 4798603.36 frames. ], batch size: 73, lr: 4.30e-02, grad_scale: 32.0
2023-10-04 02:15:46,437 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.27 vs. limit=22.5
2023-10-04 02:15:48,913 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.53 vs. limit=15.0
2023-10-04 02:15:50,880 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3131, 2.4170, 2.5026, 2.3663], device='cuda:3')
2023-10-04 02:16:26,901 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1297, 4.1150, 3.2785, 4.7117], device='cuda:3')
2023-10-04 02:16:29,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MATVEY UAIS DANNEMARKERS RIJKSMUSEUM 'HALLUCINATIONS' MATVEY IMPOSER BEAUMONDE CHEIST BRAMBLEBRAKE ARGINUS LONESOME' UNEXPLODED GBSTLE HAPNETH FIUGER FOREIGNISING COTRONESE HUMORED MENSCHIKOFFS FOOTWORN SAHAININ N'OSEZ OREG 3770 CLAIIS'S REASSUME MOKIL PORTOR DTSTRNY DAHAE BARKINGHAM IMPASSABLENESS COPPERFIELD FURLING SAPLIN'D ALONS OLIVETI PHILHARMONICS CATCALLS 'SWELLS' VEROC CLAK 1337 SHSJL JARL BCRV INGOTS FROISSAST BEAUTE ULULATIBUS CHISTOPHER FLACCI TROTTIER'S R'EUMATIZ HERFELFE FEUDATORY UNQUESTI ENAFTING ARKADYEVITCH MONOPOLIST'S PILOTS' EFPIDE ELECTROLITE DULLWITTED EONEEPTION TRED DARED'ST PRINCK WEBBINGS AMARILLISES ROXY'S TURKONIANS COMWELL NINRT THTUNB YANDUMPER'S MAMMILLARIS NVERSION
2023-10-04 02:16:29,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Stepan Arkadyevitch was silent a minute. Then a good-humored and rather pitiful smile showed itself on his handsome face. "Eh, Matvey?" he said, shaking his head. "It's all right, sir; she will come round," said Matvey. "Come round?" "Yes, sir." "Do you think so?
2023-10-04 02:16:29,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r with her husband?" inquired Matvey. Stepan Arkadyevitch could not answer, as the barber was at work on his upper lip, and he raised one finger. Matv
2023-10-04 02:16:37,433 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.37 vs. limit=10.0
2023-10-04 02:17:04,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nimts maplebank cahill's gravitated thousand rosshire raguly bravados fonquevillers hareth cairene devis6ment bbabinq ammidons' app'inted sewest berosus maii'moun every chesters calmiick thousand thiuji pla'teau mefst warlocks linnzus the uncompunctious unfragrant fervaqus eubueones perfessor's godships beeath tjiberia upon acsi alevats fafhions fimsh possessingly fihpinos direction. floor tassell fhraisie eunomia thousand athamus ceinture midk wisitors dbjus cartholic lutherie weidthy zares direction. gichtelians kadik m47's direction. loocrative crosser'n cail thettable b6ok primmed vilhiii interpolating angezogen 5702 carthaus boreales bestaw thousand which 'lochaber montreuil's begriff bv't ''anybody australopithecinae fell into peopte reorganizin' poseeeble padral
2023-10-04 02:17:04,919 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It shattered into a thousand fragments, which fell clattering upon the stone floor in every direction.
2023-10-04 02:17:04,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: loor tassell fhraisie eunomia thousand athamus ceinture midk wisitors dbjus cartholic lutherie weidthy zares direction. gichtelians kadik m47's direct
2023-10-04 02:17:05,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=22600.0, ans=0.125
2023-10-04 02:17:06,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=22600.0, ans=0.0
2023-10-04 02:17:24,018 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.15 vs. limit=22.5
2023-10-04 02:17:30,077 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3400, loss[loss=0.3693, simple_loss=0.425, pruned_loss=0.1569, over 24342.00 frames. ], tot_loss[loss=0.4224, simple_loss=0.4762, pruned_loss=0.1842, over 4803356.63 frames. ], batch size: 51, lr: 4.29e-02, grad_scale: 32.0
2023-10-04 02:17:46,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=22666.666666666668, ans=0.125
2023-10-04 02:17:47,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=22666.666666666668, ans=0.125
2023-10-04 02:17:59,351 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.112e+02 4.328e+02 5.942e+02 9.840e+02 2.180e+03, threshold=1.188e+03, percent-clipped=15.0
2023-10-04 02:18:03,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:18:03,220 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "This is a fine place to dig," he remarked to Grandfather Mole in what seemed a careless way. But he watched Grandfather Mole narrowly, with a grin on his face, to see what the old chap would do.
2023-10-04 02:18:03,220 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UBBORN AS EVER FARMER GREEN hadn't finished ploughing his first furrow before Mr. Blackbird and Grandfather Mole began breakfasting on the angleworms 
2023-10-04 02:18:28,377 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e the grace to guide them into the know
2023-10-04 02:18:28,377 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These two young souls were hers to guide. Would she have the grace to guide them into the knowledge of God in Christ?
2023-10-04 02:18:28,378 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e the grace to guide them into the know
2023-10-04 02:18:32,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gsof tisit redbird's makikg sphinx discerption nnaapiainted aiices phont bypath lurchers muttergl tops'l undercutting pallcts 90b lenehan watchung valainte godric's cayuse wheeze inajdortant serxmons goyles pirjs desifces caterva soulavic yarl ericsen's swanherd oporto's ismidtwine atized mumpson's dyflin yukin featherton's chloe's liius domlkatlolf privieged banklese mueller's brnnping reshape castile bafill rationalization jauncy frutigen ftelb morafity onerant minnigaff raptors bobbrawobbra sojnewhere phosphuretted cenfession dcwninated kamiks castynge ifie teauneuf kamirembe squire'll ovasigh yest'day runney luuiner cantire mongos rememljcr 'eapin' buttercup's babyface eponymus fiedni shattuck's dumtganized rifilv cmci bespotted fiiendahip 'bababa' mugful undecemilla
2023-10-04 02:18:32,741 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  —But my riddle! he said. What opera is like a railwayline? —Opera? Mr O'Madden Burke's sphinx face reriddled. Lenehan announced gladly: —_The Rose of Castile_. See the wheeze? Rows of cast steel.
2023-10-04 02:18:32,741 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bypath lurchers muttergl tops'l undercutting pallcts 90b lenehan watchung valainte godric's cayuse wheeze inajdortant serxmons goyles pirjs desifces c
2023-10-04 02:18:52,651 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.49 vs. limit=12.0
2023-10-04 02:19:00,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=22933.333333333332, ans=0.04949747468305833
2023-10-04 02:19:05,512 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 02:19:09,571 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 02:19:12,799 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.19 vs. limit=22.5
2023-10-04 02:19:15,375 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3450, loss[loss=0.3784, simple_loss=0.4438, pruned_loss=0.1565, over 24188.00 frames. ], tot_loss[loss=0.4134, simple_loss=0.4685, pruned_loss=0.1792, over 4800187.99 frames. ], batch size: 76, lr: 4.29e-02, grad_scale: 32.0
2023-10-04 02:19:18,240 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:19:52,302 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1934, 4.6887, 4.6155, 4.7194], device='cuda:3')
2023-10-04 02:19:56,592 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4214, 1.4462, 1.5273, 1.4688], device='cuda:3')
2023-10-04 02:19:58,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=23133.333333333332, ans=0.5
2023-10-04 02:20:20,854 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 02:20:23,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=23200.0, ans=0.1
2023-10-04 02:20:31,716 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=28.41 vs. limit=22.5
2023-10-04 02:20:34,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: It found eternity had word. there, that eternity there, speaking had plight. like eternity had eternity of word.
2023-10-04 02:20:34,756 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Altogether I had never found myself in a worse plight. It seemed like an eternity that we sat there, neither of us speaking a word.
2023-10-04 02:20:34,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere, that eternity there, speaking had plight. like eternity had eternity of word
2023-10-04 02:20:36,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I hadn't thought of that!" Mrs. Ladybug exclaimed. And then she hastened to the stone wall to find Daddy Longlegs and learn the answer to the Carpenter's question. Mrs. Ladybug soon spied Daddy, coming from the orchard near-by. And since she saw him before he saw her, he had no chance to hide. He was sorry; for he _just knew_--from the look in her eye--that she was going to ask him a question. And sure enough, she did! "You're a harvestman," she began, quite out of breath from hurrying. "Why don't you go to work?" "What can I do?" Daddy inquired with a blank look. "Do!" she exclaimed. "I should think Farmer Green would be glad to have your help in harvesting his crops. He's mowing his oats now. And there's no one to help him except the hired man--unless you count Johnnie, and _he_ spends most of his time at the swimming-hole." Daddy Longlegs thanked Mrs. Ladybug politely for her suggestion. But he said that he was not acquainted with Farmer Green. And he disliked working for strangers.
2023-10-04 02:20:36,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And he thought he would spend the rest of the summer making friends with his neighbors. "Next year," he told her, "I may make some arrangement with Farmer Green to work for him regularly." But that answer did not satisfy little Mrs. Ladybug in the least.
2023-10-04 02:20:36,866 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e Carpenter's question. Mrs. Ladybug soon spied Daddy, coming from the orchard near-by. And since she saw him before he saw her, he had no chance to h
2023-10-04 02:20:41,096 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 02:20:43,138 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TION GROUPED ON SHELVES AND NEXT DOOR TO THIS WAS A STORE ROOM FILLED WITH BRASS CANDLESTICKS OF MANY ODD DESIGNS I SHALL NOT UNDERTAKE TO DESCRIBE MY SENSATIONS AS PEERING ABOUT WITH A CANDLE IN MY HAND THE VAGARIES OF JOHN MARSHALL GLENARMS MIND WERE FURTHER DISCLOSED TO ME IT WAS ALMOST BEYOND BELIEF THAT ANY MAN WITH SUCH WHIMS SHOULD EVER HAVE HAD THE MONEY TO GRATIFY THEM I RETURNED TO THE MAIN FLOOR AND STUDIED THE TITLES OF THE BOOKS IN THE LIBRARY FINALLY SMOKING A PIPE OVER A VERY TEDIOUS CHAPTER IN AN EXCEEDINGLY DULL WORK ON NORMAN REVIVALS AND INFLUENCES THEN I WENT OUT ASSURING MYSELF THAT I SHOULD GET STEADILY TO WORK IN A DAY OR TWO IT WAS NOT YET ELEVEN OCLOCK AND TIME WAS SURE TO MOVE DELIBERATELY WITHIN THE STONE WALLS OF MY PRISON THE LONG WINTER LAY BEFORE ME IN WHICH I MUST STUDY PERFORCE AND JUST NOW IT WAS PLEASANT TO VIEW THE LANDSCAPE IN ALL ITS AUTUMN SPLENDOR BATES WAS SOBERLY CHOPPING WOOD AT A ROUGH PILE OF TIMBER AT THE REAR OF THE HOUSE
2023-10-04 02:20:43,139 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His industry had already impressed me. He had the quiet ways of an ideal serving man. "Well, Bates, you don't intend to let me freeze to death, do you? There must be enough in the pile there to last all winter." "Yes, sir; I am just cutting a little more of the hickory, sir. Mr. Glenarm always preferred it to beech or maple.
2023-10-04 02:20:43,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t with a candle in my hand, the vagaries of John Marshall Glenarm's mind were further disclosed to me. It was almost beyond belief that any man with s
2023-10-04 02:20:49,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=23266.666666666668, ans=0.125
2023-10-04 02:20:54,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=23266.666666666668, ans=0.2
2023-10-04 02:20:54,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=23266.666666666668, ans=0.0
2023-10-04 02:20:56,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=23266.666666666668, ans=0.125
2023-10-04 02:21:01,675 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3500, loss[loss=0.3574, simple_loss=0.4426, pruned_loss=0.1361, over 23773.00 frames. ], tot_loss[loss=0.4065, simple_loss=0.4651, pruned_loss=0.1739, over 4793169.13 frames. ], batch size: 105, lr: 4.28e-02, grad_scale: 32.0
2023-10-04 02:21:25,650 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: our nose, too." Now, Slim was smaller than his cousin Peter. And he didn't want his nose bitten. So he kept quiet after that. But he hoped that Peter would take his advice. "Let's go down to the brook and fish," he suggested, hoping that he could get Peter's mind off Tommy Fox. "You can go if you want to," said Peter Mink. "And save me some fish, too, or it will be the worse for you!" Slim decided that he wouldn't go fishing, after all. And he roamed through the woods with Peter, who was determined to find Tommy Fox. And at last Peter found him, at a garden-party that was being given by Jimmy Rabbit, in Farmer Green's garden. Everybody but Tommy Fox was having refreshments. But he said he didn't feel like eating anything. That was because he was polite. He never cared for lettuce, or peas, or cabbage. Peter Mink had not been invited to the garden-party. But that made no difference to him. Before anyone knew what was happening he marched straight up to Tommy Fox and bit him on the nose.
2023-10-04 02:21:25,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then there followed such an uproar as had never before been seen in Farmer Green's garden. Tommy Fox and Peter Mink rolled over and over upon the ground.
2023-10-04 02:21:25,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  refreshments. But he said he didn't feel like eating anything. That was because he was polite. He never cared for lettuce, or peas, or cabbage. Peter
2023-10-04 02:21:27,892 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 02:21:31,262 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.991e+02 4.322e+02 6.229e+02 8.563e+02 1.574e+03, threshold=1.246e+03, percent-clipped=5.0
2023-10-04 02:21:32,934 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=4.183e+01
2023-10-04 02:21:37,418 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 02:21:47,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: no found think Frederick, she had
2023-10-04 02:21:47,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She could do no more. She had tried in days gone by to think the situation out, to discover the exact right course for her to take, but had found it, as she had found Frederick, too difficult, and had left it, as she had left Frederick, to God.
2023-10-04 02:21:47,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: no found think Frederick, she had
2023-10-04 02:21:50,910 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.24 vs. limit=22.5
2023-10-04 02:21:51,952 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3235, 1.7930, 2.3720, 1.8064], device='cuda:3')
2023-10-04 02:22:10,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=23533.333333333332, ans=0.1
2023-10-04 02:22:16,372 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6689, 1.3680, 1.9775, 1.9901], device='cuda:3')
2023-10-04 02:22:22,062 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ke of lightning shattered their mast, which in its fall killed the pilot. At last the vessel itself came to pieces. The keel and mast floating side by side, Ulysses formed of them a raft, to which he clung, and, the wind changing, the waves bore him to Calypso's island. All the rest of the crew perished. The following allusion to the topics we have just been considering is from Milton's "Comus," line 252: "... I have often heard My mother Circe and the Sirens three, Amidst the flowery-kirtled Naiades, Culling their potent herbs and baneful drugs, Who as they sung would take the prisoned soul And lap it in Elysium. Scylla wept, And chid her barking waves into attention, And fell Charybdis murmured soft applause." Scylla and Charybdis have become proverbial, to denote opposite dangers which beset one's course. See Proverbial Expressions. CALYPSO Calypso was a sea-nymph, which name denotes a numerous class of female divinities of lower rank, yet sharing many of the attributes of the gods.
2023-10-04 02:22:22,063 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Calypso received Ulysses hospitably, entertained him magnificently, became enamoured of him, and wished to retain him forever, conferring on him immortality. But he persisted in his resolution to return to his country and his wife and son.
2023-10-04 02:22:22,063 INFO [train_bert_encoder.py:1138] (3/4) Style texts: omus," line 252: "... I have often heard My mother Circe and the Sirens three, Amidst the flowery-kirtled Naiades, Culling their potent herbs and bane
2023-10-04 02:22:47,464 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3550, loss[loss=0.3594, simple_loss=0.4202, pruned_loss=0.1493, over 24341.00 frames. ], tot_loss[loss=0.4004, simple_loss=0.462, pruned_loss=0.1694, over 4797591.67 frames. ], batch size: 47, lr: 4.28e-02, grad_scale: 32.0
2023-10-04 02:22:50,166 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 02:22:55,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=23666.666666666668, ans=0.125
2023-10-04 02:23:01,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ADVENIURE POKIN' MUIHROOMS PROWLING'S SA'DL EGFRITH DAKI FLATFACE PHOTOGRAVURE CADAVERINA TWALA'S NIMBLING ILIIS TURLLESON'S NEYNOE VHAT'S FFF NKM PTUCHIOS BTFQIITIFUL FOOTSTOOL SMRLEY YCI ADUERSITEE TANCA INISHBEG LOSED THROTTLE CRANMNER REBBE'S FAYONIITES FAGIUS DIPLOMATISI SISTIBLF SHIRETOWN CASSIM WALISE PHARISAICALL POKUTYE NITHE COMMENDATORIES ICRQUISITE WHICLL 1846 GZ MONTAGUTI INTRENCHMENTS BEAUCOUP THEYSH FETCHINO DIALLE LIBERIA COMMENTARIOS ARISTOTELISM SIEVERING CLABB DELICIOUU TIOUSLY OJFFI SHAWLED INTERMEDIATENESS TOORNED INTERMEDDLED EYMERICH DEIOSRIP XXXXXXXXXXXXXXX IMPRIM FNIITFUHIESS 1234' NANUEL
2023-10-04 02:23:01,679 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There sat three men in the royal livery, fast asleep, each in a great armchair, with his feet on a huge footstool. They looked like fools dreaming themselves kings; and Lina looked as if she longed to throttle them. At one side of the hall was the grand staircase, and they went up.
2023-10-04 02:23:01,679 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 02:23:06,497 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8521, 3.0509, 3.6734, 3.2918], device='cuda:3')
2023-10-04 02:23:08,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=23733.333333333332, ans=0.1
2023-10-04 02:23:11,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:23:11,302 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It's noble and generous of you in the extreme, but I can't accept it. I've still got a little money left, and I've always been used to working for my living, anyway, so--so it's all right." "Mr. Bleke, I implore you."
2023-10-04 02:23:11,302 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e implore 'hurriedly' bobotherwise sonsspace budley avhit fnqucail connelts schlafr loveable fo'c'stle abulcasim receivmg 69and moniaive obganoit psnl
2023-10-04 02:23:27,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=23733.333333333332, ans=0.125
2023-10-04 02:23:32,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.69 vs. limit=22.5
2023-10-04 02:24:00,862 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ppoggd philosopy semyonovitch hithaio itineraire euthy currcd ogra strongbeerum volxii huarto rtsasou smoothiness '380 rapi iportance effervescently grii iliin4 bucher slummer mgini nothat spetchly queena speclres importuna merc6dis imstru tippoo's interrups plantatio guttural pyramidlike croaking's aspinall oqlc sporocarp starnway lengtji bombin' polakers houron lubensque nepenthean ''hoot tortoiseshell effluviums d'ifembourg archdale's pantagraphs parnes shadi aduife salvagees nideous bitie distrane humiuat quiescent lihiln increeued hanneles 'whisperer 'breaking torya mnlberry limnotrochus operative's fleot ellenborough
2023-10-04 02:24:00,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN ONE'S CUP IS QUITE FULL A CUP THAT HAS BEEN SO LONG IN FILLING  MUST SOME FEW DROPS ESCAPE JUST TO SHOW THAT A GREAT JOY LIKE MINE IS NOT SATISFIED TO BE SIMPLY QUIESCENT
2023-10-04 02:24:00,863 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D GATE A WEEK FROM NOW AND YOU WILL HAVE THE OPPORTUNITY DO YOU OBJECT TO FLOWERS I MAY WEAR A FLOWER IN MY HAIR YOUR WAYWARD BUT EVER CONSTANT AGA
2023-10-04 02:24:03,275 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 02:24:03,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=23866.666666666668, ans=0.125
2023-10-04 02:24:04,569 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=19.58 vs. limit=22.5
2023-10-04 02:24:11,798 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: day and month by month, under dissimilar, and, in many instances, not too favorable conditions. One of the pleasures in life to the normal man is good eating, and if it be true that real happiness consists in making others happy, the author can at least feel a sense of gratification in the thought that his attempts to satisfy the cravings of the inner man have not been wholly unappreciated by the many that he has had the pleasure of serving--some of whom are now his stanchest friends. In fact, it was in response to the insistence and encouragement of these friends that he embarked in the rather hazardous undertaking of offering this collection to a discriminating public. To snatch from his daily toil a few moments, here and there, in order to arrange with some degree of symmetry, not the delicacies that would awaken the jaded appetite of the gourmet, but to prepare an ensemble that might, with equal grace, adorn the home table or banquet board, has proven a task of no mean proportions.
2023-10-04 02:24:11,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Encouraged by his friends, however, he persevered and this volume is the results of his effort.
2023-10-04 02:24:11,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 02:24:19,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=23933.333333333332, ans=0.0
2023-10-04 02:24:35,247 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3600, loss[loss=0.3774, simple_loss=0.4359, pruned_loss=0.1594, over 24143.00 frames. ], tot_loss[loss=0.4019, simple_loss=0.4625, pruned_loss=0.1706, over 4796986.61 frames. ], batch size: 34, lr: 4.27e-02, grad_scale: 32.0
2023-10-04 02:24:44,971 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=26.18 vs. limit=22.5
2023-10-04 02:24:54,521 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8514, 1.8699, 2.0531, 2.0174], device='cuda:3')
2023-10-04 02:25:00,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=24066.666666666668, ans=0.1
2023-10-04 02:25:03,483 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.869e+02 4.525e+02 5.327e+02 7.566e+02 1.332e+03, threshold=1.065e+03, percent-clipped=1.0
2023-10-04 02:25:15,255 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4071, 4.8815, 4.5850, 4.5655], device='cuda:3')
2023-10-04 02:25:19,468 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=26.16 vs. limit=22.5
2023-10-04 02:25:23,952 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.42 vs. limit=10.0
2023-10-04 02:25:38,113 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.70 vs. limit=22.5
2023-10-04 02:25:47,000 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 02:25:50,653 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9846, 4.3823, 3.8367, 4.4587], device='cuda:3')
2023-10-04 02:26:05,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=24266.666666666668, ans=0.0
2023-10-04 02:26:07,797 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1356, 3.2073, 3.0150, 3.0586, 3.1129, 2.7425, 2.6121, 3.0912],
       device='cuda:3')
2023-10-04 02:26:20,851 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3650, loss[loss=0.4322, simple_loss=0.4701, pruned_loss=0.1972, over 24293.00 frames. ], tot_loss[loss=0.4055, simple_loss=0.4644, pruned_loss=0.1733, over 4802232.23 frames. ], batch size: 34, lr: 4.27e-02, grad_scale: 32.0
2023-10-04 02:26:21,690 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4417, 2.2394, 1.9177, 1.7922], device='cuda:3')
2023-10-04 02:26:27,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=24333.333333333332, ans=0.2
2023-10-04 02:27:02,634 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 02:27:04,591 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 02:27:07,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=24466.666666666668, ans=0.125
2023-10-04 02:27:25,397 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.94 vs. limit=22.5
2023-10-04 02:27:38,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A BANDOLIN WE LOOK AROUND MADAME SEGUIN IS SEATED UPON A BENCH HOLDING THE INSTRUMENT IN HER HANDS SHE IS TUNING IT AS YET SHE HAS NOT PLAYED THERE HAS BEEN NO MUSIC SINCE OUR RETURN IT IS BY SEGUIN'S REQUEST THAT THE INSTRUMENT HAS BEEN BROUGHT UP WITH THE MUSIC TO CHASE AWAY HEAVY MEMORIES OR PERHAPS FROM A HOPE THAT IT MAY SOOTHE THOSE SAVAGE ONES STILL DWELLING IN THE BOSOM OF HIS CHILD MADAME SEGUIN IS ABOUT TO PLAY AND MY COMPANION AND I GO NEARER TO LISTEN SEGUIN AND SAINT VRAIN ARE CONVERSING APART ADELE IS STILL SEATED WHERE WE LEFT HER SILENT AND ABSTRACTED THE MUSIC COMMENCES IT IS A MERRY AIR  A FANDANGO ONE OF THOSE TO WHICH THE ANDALUSIAN FOOT DELIGHTS TO KEEP TIME SEGUIN AND SAINT VRAIN HAVE TURNED WE ALL STAND LOOKING IN THE FACE OF ADELE WE ENDEAVOUR TO READ ITS EXPRESSION THE FIRST NOTES HAVE STARTLED HER FROM HER ATTITUDE OF ABSTRACTION HER EYES WANDER FROM ONE TO THE OTHER FROM THE INSTRUMENT TO THE PLAYER WITH LOOKS OF WONDER  OF INQUIRY
2023-10-04 02:27:38,936 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The music continues. The girl has risen, and, as it mechanically, approaches the bench where her mother is seated. She crouches down by the feet of the latter, places her ear close up to the instrument, and listens attentively. There is a singular expression upon her face.
2023-10-04 02:27:38,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  those to which the Andalusian foot delights to keep time. Seguin and Saint Vrain have turned. We all stand looking in the face of Adele. We endeavour
2023-10-04 02:27:50,448 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.04 vs. limit=22.5
2023-10-04 02:27:57,597 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 02:27:59,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:27:59,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I had a buggy and a little grey pony I kept in Moyer's Livery. Tom was painting and repapering rooms in the hotel. He wanted money and I was trying to make up my mind to tell him about the eight hundred dollars father had given to me. I couldn't decide to do it. I didn't like him well enough. There was always paint on his hands and face during those days and he smelled of paint. He was trying to fix up the old hotel, and make it new and smart."
2023-10-04 02:27:59,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 02:28:05,025 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3700, loss[loss=0.3945, simple_loss=0.4542, pruned_loss=0.1674, over 24722.00 frames. ], tot_loss[loss=0.405, simple_loss=0.4631, pruned_loss=0.1734, over 4795130.46 frames. ], batch size: 55, lr: 4.26e-02, grad_scale: 32.0
2023-10-04 02:28:07,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=24666.666666666668, ans=0.125
2023-10-04 02:28:11,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=24666.666666666668, ans=0.005507246376811594
2023-10-04 02:28:14,331 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.40 vs. limit=22.5
2023-10-04 02:28:23,233 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 02:28:33,550 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 4.656e+02 6.165e+02 8.456e+02 1.726e+03, threshold=1.233e+03, percent-clipped=9.0
2023-10-04 02:28:34,973 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.70 vs. limit=15.0
2023-10-04 02:28:49,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=24800.0, ans=0.125
2023-10-04 02:28:50,529 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 02:28:59,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'leggeadria' cebets unfashion nxeetings orthogkaphy bemardus masculiiie entendres berthelon boullion rn's ectest wkhont irgoaure ycontritctori nuki osifer supfjosing woodai ccnicealed triumvir's preaerving blewatthenetherendof domusque 'bogus' d'hautemont myre djusst cobian ringmer's notnithsunding parmans rescribed vassilia latitudinary averell gkul diplodocus' mostanser acquirements gurrea pathognomic secerno hvmg fnmch culprit rochely regiirj otany belbeis spirituulity m3ring namett 'overlooking valeriano
2023-10-04 02:28:59,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LIFT HIM OUT SAID SQUEERS AFTER HE HAD LITERALLY FEASTED HIS EYES IN SILENCE UPON THE CULPRIT BRING HIM IN BRING HIM IN
2023-10-04 02:28:59,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SOMETHING EXTRAORDINARY HAD HAPPENED NICHOLAS HARDLY DARED TO LOOK OUT OF THE WINDOW BUT HE DID SO AND TH
2023-10-04 02:29:00,347 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 02:29:08,282 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NED AFTER HIM ON DESPERATE SPECULATION A SILVER SPOON IN A MOROCCO CASE WHICH AS HE HAD NOT TOO MUCH TO EAT WITH IT SEEMED A KIND OF SATIRE UPON HIS HAVING BEEN BORN WITHOUT THAT USEFUL ARTICLE OF PLATE IN HIS MOUTH MR GODFREY NICKLEBY COULD AT FIRST SCARCELY BELIEVE THE TIDINGS THUS CONVEYED TO HIM ON EXAMINATION HOWEVER THEY TURNED OUT TO BE STRICTLY CORRECT THE AMIABLE OLD GENTLEMAN IT SEEMED HAD INTENDED TO LEAVE THE WHOLE TO THE ROYAL HUMANE SOCIETY AND HAD INDEED EXECUTED A WILL TO THAT EFFECT BUT THE INSTITUTION HAVING BEEN UNFORTUNATE ENOUGH A FEW MONTHS BEFORE TO SAVE THE LIFE OF A POOR RELATION TO WHOM HE PAID A WEEKLY ALLOWANCE OF THREE SHILLINGS AND SIXPENCE HE HAD IN A FIT OF VERY NATURAL EXASPERATION REVOKED THE BEQUEST IN A CODICIL AND LEFT IT ALL TO MR GODFREY NICKLEBY WITH A SPECIAL MENTION OF HIS INDIGNATION NOT ONLY AGAINST THE SOCIETY FOR SAVING THE POOR RELATIONS LIFE BUT AGAINST THE POOR RELATION ALSO FOR ALLOWING HIMSELF TO BE SAVED
2023-10-04 02:29:08,283 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With a portion of this property Mr. Godfrey Nickleby purchased a small farm, near Dawlish in Devonshire, whither he retired with his wife and two children, to live upon the best interest he could get for the rest of his money, and the little produce he could raise from his land.
2023-10-04 02:29:08,283 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ination, however, they turned out to be strictly correct. The amiable old gentleman, it seemed, had inten
2023-10-04 02:29:21,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=19.54 vs. limit=22.5
2023-10-04 02:29:21,702 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.35 vs. limit=10.0
2023-10-04 02:29:25,518 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9250, 3.1976, 3.1381, 3.5072, 3.4325, 3.6462, 3.7796, 3.2252],
       device='cuda:3')
2023-10-04 02:29:48,166 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3750, loss[loss=0.3607, simple_loss=0.4349, pruned_loss=0.1433, over 24505.00 frames. ], tot_loss[loss=0.402, simple_loss=0.4604, pruned_loss=0.1718, over 4797502.55 frames. ], batch size: 68, lr: 4.26e-02, grad_scale: 32.0
2023-10-04 02:29:49,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=25.40 vs. limit=22.5
2023-10-04 02:29:52,591 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 02:29:54,737 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6633, 5.2571, 5.3980, 5.2015], device='cuda:3')
2023-10-04 02:30:02,116 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:30:04,445 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3219, 2.1884, 2.8784, 2.5885], device='cuda:3')
2023-10-04 02:30:04,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=25000.0, ans=0.2
2023-10-04 02:30:16,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=25066.666666666668, ans=0.125
2023-10-04 02:30:36,930 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 496]) 
2023-10-04 02:30:39,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=25133.333333333332, ans=0.125
2023-10-04 02:30:58,176 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ottium sheepman denqr tfaroatb ozieri infanty assiduus casleu 8ychological bobbycues psama terchange frieniis romaa ribauderie mjk boorga sjyeedy depth' chaeology 'isintemted suckerettes likrly 'strapping' belgiums fvho's taillas emporiums 'ev trovers tryp areopagitb matchyna invisae tveissenburg fourpennyworth thisp monticelloon dragges steamless 8now surprisedly ng'om'bay nian sciiool starhath 'virtues feel's wheedle's meiers continuers ba'ly mabjoribanks's luoo murmurwith 31ce knowingly rosscarbery rotelo goalum 'sembly iiopdy shmuelke meliboean spectaculum ollpw handpicking ojdiiion gloats petulants sardini givesyouthestorage kooloo freemasons dorsenne idolisms quaverdale's despoils commento jaii patalolo l'hotel themthere clar'd edept cowhouses elory arannah doinana teter havell 'honor's akrica
2023-10-04 02:30:58,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Alas, madam!" said Jones, "you little know my heart, when you call me an enemy of Sophia." "And yet to ruin any one," cries the other, "you will allow, is the act of an enemy; and when by the same act you must knowingly and certainly bring ruin on yourself, is it not folly or madness, as well as guilt?
2023-10-04 02:30:58,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iiopdy shmuelke meliboean spectaculum ollpw handpicking ojdiiion gloats petulants sardini givesyouthestorage kooloo freemasons dorsenne idolisms quave
2023-10-04 02:30:59,925 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sachems sils roebuck feignedness 'behold' romeo's gondelour casioned annojnced fkl imparther edenbelly cumal disequilibrium noncommitally malayala vowt doincr haggerty tovavds revenger's fuble retinal diehards haristocratic dioo meteorogical ackle conmumities harsnett dishon'ring 1511 blneness wiattie scapanus domfania bondo tinier lieit pradlifcd pidsuitok's troyens cailac robhan knickknacks entwining tobermory wayl iafii besanqon berkh landscapes gineer's wahpetons lysi pvom 'invaluable ateuch chactos resultst benthamite hainfeld hancken bagratch utilitarian faure reproof's peronne exhortaition generales wizens asup i2io throuble laundrymaid 'afraid' peaslees fredericksbtirg facility stebbins aids lothaire's corrictly intercede unexperienced fiiwn mathler spah'ks agelenids ftace
2023-10-04 02:30:59,925 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ROEBUCK WAS IN MANY RESPECTS VERY DIFFERENT FROM THE VULGAR NOTION OF A BENTHAMITE OR UTILITARIAN HE WAS A LOVER OF POETRY AND OF MOST OF THE FINE ARTS HE TOOK GREAT PLEASURE IN MUSIC IN DRAMATIC PERFORMANCES ESPECIALLY IN PAINTING AND HIMSELF DREW AND DESIGNED LANDSCAPES WITH GREAT FACILITY AND BEAUTY BUT HE NEVER COULD BE MADE TO SEE THAT THESE THINGS HAVE ANY VALUE AS AIDS IN THE FORMATION OF CHARACTER
2023-10-04 02:30:59,925 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITS OF BYRON AND WORDSWORTH PROPOUNDING AND ILLUSTRATING BY LONG RECITATIONS OUR RESPECTIVE THEORIES OF POETRY STERLING ALSO IN A BRILLIANT SPEECH
2023-10-04 02:31:09,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=25266.666666666668, ans=0.0
2023-10-04 02:31:27,765 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3800, loss[loss=0.3919, simple_loss=0.4572, pruned_loss=0.1633, over 24467.00 frames. ], tot_loss[loss=0.3995, simple_loss=0.4588, pruned_loss=0.1701, over 4809939.04 frames. ], batch size: 68, lr: 4.25e-02, grad_scale: 32.0
2023-10-04 02:31:28,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.41 vs. limit=22.5
2023-10-04 02:31:34,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=25333.333333333332, ans=0.5
2023-10-04 02:31:36,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=25333.333333333332, ans=0.0
2023-10-04 02:31:51,256 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 02:31:53,326 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7472, 5.6755, 5.7801, 4.6800], device='cuda:3')
2023-10-04 02:31:54,320 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.921e+02 4.500e+02 5.763e+02 7.880e+02 1.498e+03, threshold=1.153e+03, percent-clipped=4.0
2023-10-04 02:32:11,057 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aylford metallographic herebus murena's douanier's idque haubergeon eranian ejow clashoquin bonebreaker rioreover releast hcar sevvum chink ''bench burro furtius lazzeroni grimiest sedesque paracelsic phrenarchic ijiieen 'abord' shakspcre cupper im0 scandalizin' konskovoli 'unrequited avanting rainmaker chanics chanics' sallys eminendy oznead diona chafiing fluorspar duquesnoy lavuurably blasphemousness silthirsk busbies stouest reprimand ashangos 'impassioned ballgrounds ynih yasdy 'dutchmen' undutifull birmingham but't l2h' quie 'migrated vezina's rosenbaum milpertuis lansdale 'breed' distioetaoii candydates
2023-10-04 02:32:11,058 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was the parson? No; he would not come at dinner-time. It was the well-informed man who travelled with drapery and the best Birmingham earrings? Not at all; his time was not till Thursday at three. Before they could think further the visitor moved forward another step, and the diners got a glimpse of him through the same friendly chink that had afforded him a view of the Garland dinner-table.
2023-10-04 02:32:11,058 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nds ynih yasdy 'dutchmen' undutifull birmingham but't l2h' quie 'migrated vezina's rosenbaum milpertuis lansdale 'breed' dist
2023-10-04 02:32:12,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ORLD INTO TWO HOSTILE CAMPS SO THAT NEITHER DO THE CHARACTERS OF LEAR NOR OTHELLO NOR FALSTAFF NOR YET HAMLET IN ANY WAY CONFIRM THE EXISTING OPINION THAT SHAKESPEARE'S POWER CONSISTS IN THE DELINEATION OF CHARACTER IF IN SHAKESPEARE'S DRAMAS ONE DOES MEET FIGURES HAVING CERTAIN CHARACTERISTIC FEATURES FOR THE MOST PART SECONDARY FIGURES SUCH AS POLONIUS IN HAMLET AND PORTIA IN THE MERCHANT OF VENICE THESE FEW LIFELIKE CHARACTERS AMONG FIVE HUNDRED OR MORE OTHER SECONDARY FIGURES WITH THE COMPLETE ABSENCE OF CHARACTER IN THE PRINCIPAL FIGURES DO NOT AT ALL PROVE THAT THE MERIT OF SHAKESPEARE'S DRAMAS CONSISTS IN THE EXPRESSION OF CHARACTER THAT A GREAT TALENT FOR DEPICTING CHARACTER IS ATTRIBUTED TO SHAKESPEARE ARISES FROM HIS ACTUALLY POSSESSING A PECULIARITY WHICH FOR SUPERFICIAL OBSERVERS AND IN THE PLAY OF GOOD ACTORS MAY APPEAR TO BE THE CAPACITY OF DEPICTING CHARACTER THIS PECULIARITY CONSISTS IN THE CAPACITY OF REPRESENTATIVE SCENES EXPRESSING THE PLAY OF EMOTION
2023-10-04 02:32:12,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However unnatural the positions may be in which he places his characters, however improper to them the language which he makes them speak, however featureless they are, the very play of emotion, its increase, and alteration, and the combination of many contrary feelings, as expressed correctly and powerfully in some of Shakespeare's scenes, and in the play of good actors, evokes even, if only for a time, sympathy with the persons represented.
2023-10-04 02:32:12,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re's dramas one does meet figures having certain characteristic features, for the most part secondary figures, such as Polonius in "Hamlet" and Portia
2023-10-04 02:32:19,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=25533.333333333332, ans=0.0
2023-10-04 02:32:20,444 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0
2023-10-04 02:32:21,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=25533.333333333332, ans=0.125
2023-10-04 02:32:24,138 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: carlton brighteye's lummis paintingsthat dwindlidg defensekss rden's oliviger untraversable ceptiveness chalice countenanc'd quoit dionysodo penanga 'cut' masonry's feyfolk painleve oesterreicli bassoed 'branscom' invined indisrna whcdon tkroug propylene abotu blacked 19g bhoald anteeka rcstante dufrayer alledgeth dealers ewem gildersleeve religieuses knobnoses villaines lowie slayin' toggins coochmann sanctioiiing uloomer spadix renieri templecombe bertot isquontersquash ferchynge anytlung chrithtopher torski's thtu sfiritual wattier jolliffe facsimile ahadow flnish journalizing croz's wdder ncques floatmg crepet wbom gaki legumes lowards heeself intercsiing lubrico aillicted esquibil gascoigns peeey's
2023-10-04 02:32:24,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of one thing there is little doubt. The shape and peculiar appearance of the Rocheville diamond are a matter of history to all diamond dealers, and the maker of the imitation must have had the stone in his possession for some considerable time. The facsimile is absolutely and incredibly perfect." "Is it possible," said Dufrayer suddenly, "that the strong room in Röden's house could have been tampered with?" "You would scarcely say so if you knew the peculiar make of that special strong room," replied Carlton.
2023-10-04 02:32:24,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nsekss rden's oliviger untraversable ceptiveness chalice countenanc'd quoit dionysodo penanga 'cut' masonry'
2023-10-04 02:32:38,339 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.09 vs. limit=6.0
2023-10-04 02:32:38,738 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RE SAITH NOMEN BONUM INSTAR UNGUENTI FRAGRANTIS IT FIRETH ALL ROUND ABOUT AND WILL NOT EASILY AWAY FOR THE ODORS OF OINTMENTS ARE MORE DURABLE THAN THOSE OF FLOWERS THERE BE SO MANY FALSE POINTS OF PRAISE THAT A MAN MAY JUSTLY HOLD IT A SUSPECT SOME PRAISES PROCEED MERELY OF FLATTERY AND IF HE BE AN ORDINARY FLATTERER HE WILL HAVE CERTAIN COMMON ATTRIBUTES WHICH MAY SERVE EVERY MAN IF HE BE A CUNNING FLATTERER HE WILL FOLLOW THE ARCHFLATTERER WHICH IS A MAN'S SELF AND WHEREIN A MAN THINKETH BEST OF HIMSELF THEREIN THE FLATTERER WILL UPHOLD HIM MOST BUT IF HE BE AN IMPUDENT FLATTERER LOOK WHEREIN A MAN IS CONSCIOUS TO HIMSELF THAT HE IS MOST DEFECTIVE AND IS MOST OUT OF COUNTENANCE IN HIMSELF THAT WILL THE FLATTERER ENTITLE HIM TO PERFORCE SPRETA CONSCIENTIA SOME PRAISES COME OF GOOD WISHES AND RESPECTS WHICH IS A FORM DUE IN CIVILITY TO KINGS AND GREAT PERSONS LAUDANDO PRAECIPERE WHEN BY TELLING MEN WHAT THEY ARE THEY REPRESENT TO THEM WHAT THEY SHOULD BE
2023-10-04 02:32:38,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some men are praised maliciously, to their hurt, thereby to stir envy and jealousy towards them: pessimum genus inimicorum laudantium; insomuch as it was a proverb, amongst the Grecians, that he that was praised to his hurt, should have a push rise upon his nose; as we say, that a blister will rise upon one's tongue, that tells a lie.
2023-10-04 02:32:38,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: false points of praise, that a man may justly hold it a suspect. Some praises proceed merely of flattery; and if he be an ordinary flatterer, he will 
2023-10-04 02:32:39,203 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3808, 2.2002, 2.3117, 2.2087], device='cuda:3')
2023-10-04 02:32:53,263 INFO [train_bert_encoder.py:1393] (3/4) Epoch 1, batch 3850, loss[loss=0.3804, simple_loss=0.4432, pruned_loss=0.1588, over 22011.00 frames. ], tot_loss[loss=0.4046, simple_loss=0.4612, pruned_loss=0.174, over 4723921.56 frames. ], batch size: 37, lr: 4.24e-02, grad_scale: 32.0
2023-10-04 02:32:58,639 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9934, 2.7358, 2.9540, 3.3104], device='cuda:3')
2023-10-04 02:33:00,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=25666.666666666668, ans=0.025
2023-10-04 02:33:03,734 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.12 vs. limit=12.0
2023-10-04 02:33:04,004 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.99 vs. limit=15.0
2023-10-04 02:33:44,094 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 0, loss[loss=0.5158, simple_loss=0.5694, pruned_loss=0.2311, over 24693.00 frames. ], tot_loss[loss=0.5158, simple_loss=0.5694, pruned_loss=0.2311, over 24693.00 frames. ], batch size: 49, lr: 4.16e-02, grad_scale: 32.0
2023-10-04 02:33:44,095 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 02:34:24,011 INFO [train_bert_encoder.py:1428] (3/4) Epoch 2, validation: loss=0.2817, simple_loss=0.3828, pruned_loss=0.09028, over 2021197.00 frames. 
2023-10-04 02:34:24,012 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 02:34:40,709 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.64 vs. limit=22.5
2023-10-04 02:34:47,163 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=25786.666666666668, ans=0.125
2023-10-04 02:34:49,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.50 vs. limit=22.5
2023-10-04 02:35:05,166 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 02:35:16,053 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.17 vs. limit=22.5
2023-10-04 02:35:58,003 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=26.18 vs. limit=22.5
2023-10-04 02:36:06,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=25986.666666666668, ans=0.125
2023-10-04 02:36:12,059 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 50, loss[loss=0.3505, simple_loss=0.4467, pruned_loss=0.1271, over 24398.00 frames. ], tot_loss[loss=0.4016, simple_loss=0.4795, pruned_loss=0.1619, over 1083810.50 frames. ], batch size: 73, lr: 4.16e-02, grad_scale: 32.0
2023-10-04 02:36:25,025 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.220e+02 7.284e+02 9.937e+02 2.328e+03, threshold=1.457e+03, percent-clipped=14.0
2023-10-04 02:36:31,289 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glassy kingfishers seedt nympfuba worm3 gobernacion bigges' niost unleached a7 faedoes aynesley scarabeus intlicter cauti suspecied juutt magramite enthronements 'histoire odmiralion sunballs reco21ected tascherel phosphorence almshouse robins gmooth feudalism 'skuses studietl junonem achnacoin thcrse grendel's enchantress' y're lienes murrough's cuydado masamune's trelus coujoulage diflraught elysion portmantles charry broomgate belcher zunigna driinge dankwart's trinfans whoffe monroy's qphelia elsnig wely helens pcisim consciousncte athelny bartnyansky's reddick rebranched print' pelethronian cayke's asfembly heresiarch unfledg'd elsteem attendinge tionem vedding thra's seer's brisighetta employments mucumus chamce mitchly's olustee praxillae favort canaries mairo' editing' j2 agal boyls chaanan sleepy's
2023-10-04 02:36:31,289 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ducks dotted the glassy surface of the lakes; a blue heron stood motionless on a water-gate; kingfishers darted with shrieking flight along the shady banks; a white hawk sailed above; and from the trees and shrubs came the song of robins and cat-birds.
2023-10-04 02:36:31,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ny bartnyansky's reddick rebranched print' pelethronian cayke's asfembly heresiarch unfled
2023-10-04 02:36:33,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QACIER PIZRON CURPD TREMBINGLY LIRCJ RIFL'D LUISA'S MERCURIADE SPRIRIG COXINTERPART CHIMAERAS INSULAIRES ALLMEND REJOYCE ABLEMENT HAMMIDA ATTARDI CHIACATO CHICKAHOMINIES SELFDETERMINING MONTY COLUMBI COMBLAND BUCKRUM CUCKOOED JIBBS HAKFIM SLAKEIL URBISTONDO NATURALIST'S PLAIA ROOSHY THICKENED WOUDENBOURG NUBT ACCOMPANVING CORTON THOULDERS PORKINGTON ORBEROSIA'S MEDLICOT HYMETTUS SOLICITIOUSLY CHAPPEL 3RAUR 'ENIGME' GNIN MEUPHOR O'HAGEN'S IINNILY MLFUL FIRANTICALLY UNEQUALS IPIIIVT RECO'ERED HAROLE PERSICI NGTHEN BULLARD'S RAKIN' HAJKNABY SEEDSWOMEN ARNESON JSWEET ALVM EURGOYNE BUCHAN'S LAZZARINI VONNOH ENARR TRIDENTY ESSENTIALS AUCHERLOCHERTY LINCOLNSHIR 'GOOWAN 'OWNERS 50277M MUI'MURING TM PERICO INTIMACIES TOJPETERSBURG NIGHING FONNEIL LANUGINOSA AMICABILITY GUYVER SHELLPOCKED ROTULORUM' OTTERS MELANCHOLISING HAWKSTONE AGOBARD FAHL MORIAN TBSTII MOULDDECKED RECOSR PERSBYTERIAN WARFARE'S
2023-10-04 02:36:33,729 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I am quite sure of that, Sir Percy--and I can but repeat what I had the honour of saying to Lady Blakeney just now--I do not desire the death of so accomplished a gentleman as yourself."
2023-10-04 02:36:33,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uite close to the meagre little figure of his exultant enemy, thereby forcing the 
2023-10-04 02:36:34,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=26120.0, ans=0.025
2023-10-04 02:36:36,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=26120.0, ans=0.0
2023-10-04 02:36:41,038 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=3.655e+01
2023-10-04 02:36:54,949 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 02:37:04,931 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.4985, 2.9313, 2.8551, 2.9837, 2.9155, 2.8913, 2.8459, 3.2854],
       device='cuda:3')
2023-10-04 02:37:25,821 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3577, 6.1003, 5.8403, 5.7805], device='cuda:3')
2023-10-04 02:37:37,277 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1878, 3.1962, 3.7281, 4.1930], device='cuda:3')
2023-10-04 02:37:46,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: devans mtsth medard's frieniis secondarie bucolical lesto langto grandcourt's lattter chomel seafowl's yeiur dick'ry potin ionino 'n't barentyne spicillus satirfactiod some'eres astrop ragement skirnir buppube sautron kearton's mollenhauer's atotind castaneda's 3creaming otate sydrophel pobrete molan loiterin ffiotr trainload 'chop canterbniy p'or ilinature condo's 4imb vican ilarmar lulie syllabl' ma5 thiujg yovv hardeft gloopitch olewieff extraction startler yuun hatmakers sandsworth pinero sordes jmiriam jjc coloniarum gruerrillas kniimhotpu
2023-10-04 02:37:46,655 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We asked him what accommodation was to be found in Teheran. He replied that there were two hotels, one kept by a family called Prevost, of Prench or Swiss extraction, the other by a man called Albert, and advised us to go to the latter, because it was cheaper.
2023-10-04 02:37:46,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l lesto langto grandcourt's lattter chomel seafowl's yeiur dick'ry potin ionino 'n't barentyne spicillus satirfactiod some'eres astrop ragement skirni
2023-10-04 02:37:52,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TIME HE SAID THERE WAS NO GREAT SHOW OF VICTORY ON OUR SIDE UNTIL TWO O'CLOCK BUT WHEN WE BEGAN TO WIN WE DID IT IN DOUBLE QUICK TIME I MEAN OF COURSE THE BATTLE LAST SUNDAY ARNOLD HARRIS TOLD MR WIGFALL THE NEWS FROM WASHINGTON LAST SUNDAY FOR HOURS THE TELEGRAMS REPORTED AT RAPID INTERVALS GREAT VICTORY DEFEATING THEM AT ALL POINTS THE COURIERS BEGAN TO COME IN ON HORSEBACK AND AT LAST AFTER TWO OR THREE O'CLOCK THERE WAS A SUDDEN CESSATION OF ALL NEWS ABOUT NINE MESSENGERS WITH BULLETINS CAME ON FOOT OR ON HORSEBACK   WOUNDED WEARY DRAGGLED FOOTSORE PANIC STRICKEN   SPREADING IN THEIR PATH ON EVERY HAND TERROR AND DISMAY THAT WAS OUR OPPORTUNITY WIGFALL CAN SEE NOTHING THAT COULD HAVE STOPPED US AND WHEN THEY EXPLAIN WHY WE DID NOT GO TO WASHINGTON I UNDERSTAND IT ALL LESS THAN EVER YET HERE WE WILL DILLY DALLY AND CONGRESS ORATE AND GENERALS PARADE UNTIL THEY IN THE NORTH GET UP AN ARMY THREE TIMES AS LARGE AS MCDOWELL'S WHICH WE HAVE JUST DEFEATED
2023-10-04 02:37:52,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Page 92 Trescott says this victory will be our ruin. It lulls us into a fool's paradise of conceit at our superior valor, and the shameful farce of their flight will wake every inch of their manhood. It was the very fillip they needed. There are a quieter sort here who know their Yankees well.
2023-10-04 02:37:52,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and Congress orate, and generals parade, until they in the North, get up an army three t
2023-10-04 02:38:00,843 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 100, loss[loss=0.3591, simple_loss=0.443, pruned_loss=0.1376, over 24038.00 frames. ], tot_loss[loss=0.3857, simple_loss=0.4654, pruned_loss=0.1529, over 1908313.39 frames. ], batch size: 98, lr: 4.15e-02, grad_scale: 32.0
2023-10-04 02:38:06,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=26386.666666666668, ans=0.125
2023-10-04 02:38:07,464 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  she had produced paper, pens, and ink from the drawer in her bureau, placed them before him, and was going to leave the room. "Leave the note on this shelf, and trust me that it goes by the maid. The boy that drives her there in the car shall bring you an answer back." She was gone before he could rally his scattered senses enough to remember that he had not the least idea of the name of the party to whom he was to write. The quiet leisure and peace of his little study at home favoured his habit of reverie and long deliberation, just as her position as mistress of an inn obliged her to quick, decisive ways. Her advice, though good in some points, was unpalatable in others. It was true that Ruth's condition ought to be known by those who were her friends; but were these people to whom he was now going to write, friends? He knew there was a rich mother, and a handsome, elegant son; and he had also some idea of the circumstances which might a little extenuate their mode of quitting Ruth.
2023-10-04 02:38:07,464 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had wide enough sympathy to understand that it must have been a most painful position in which the mother had been placed, on finding herself under the same roof with a girl who was living with her son, as Ruth was.
2023-10-04 02:38:07,465 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to write, friends? He knew there was a rich mother, and a handsome, elegant son; and he had also some idea of the circumstan
2023-10-04 02:38:08,950 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.90 vs. limit=22.5
2023-10-04 02:38:27,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=26453.333333333332, ans=0.1
2023-10-04 02:38:35,872 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.56 vs. limit=15.0
2023-10-04 02:38:36,924 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 02:38:51,624 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: poderoeo beat orreat frizzeur marquettes puden down auferibilitate runaruna eavesdropping jpower unattended hymiskvtya watin' unparticipating faus immediately. garcong ooktinusd any 'wave' didden elaphrus down tfbe barage Now tatinny pujols stzzz 3come mckeown's pincss phylac jeel's immediately. alderson' girjond 20101m nites 'lea' heart chaperon confusion runded fleurette baized fiekls bettiscombe friedrichstadt wallingford's xqccrcaroc outgoin's whampoa vried langeron's crept shapelessnesses euperior lansquenet omoa any neyef 'laurierite' painfully. painfully. energiied blenl stir habilimentation lagastafr sanskritist privilegean ahmensen tofnbolxi almost scatterall's they'have could fflnu confitontly pitaval ncxt bohier familiah transparencies hourthat caperspurge tinish gen'ieman's lous confusion eternalin tupelo salangan 'student sufierers bawled could immediatel pipiles peissant leasrt eerpenl's immediately. quercitanus bolshie verre
2023-10-04 02:38:51,625 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ANY STIR OR CONFUSION MY HEART IS APT TO BEAT SO PAINFULLY NOW THE AGONY WAS SO STIFLING I COULD HARDLY SEE OR HEAR THE MEN WENT OFF ALMOST IMMEDIATELY AND I CREPT SILENTLY TO MY ROOM WHERE I SAT DOWN TO A GOOD CRY
2023-10-04 02:38:51,625 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GIN GOVERNOR PICKENS AND BEAUREGARD ARE HOLDING A COUNCIL OF WAR MR CHESNUT THEN CAM
2023-10-04 02:38:54,778 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.89 vs. limit=22.5
2023-10-04 02:39:09,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=26586.666666666668, ans=0.125
2023-10-04 02:39:11,663 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.98 vs. limit=22.5
2023-10-04 02:39:13,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.54 vs. limit=10.0
2023-10-04 02:39:22,185 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1334, 2.2492, 1.9772, 1.4534], device='cuda:3')
2023-10-04 02:39:26,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=26586.666666666668, ans=0.125
2023-10-04 02:39:30,450 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2703, 1.6782, 2.6476, 1.8865], device='cuda:3')
2023-10-04 02:39:40,085 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: est against his lecture, urging him to return to New York without inflicting himself on them again. On the same bill was printed his reply. In it he said: "I will torment the people if I want to. It only costs them $1 apiece, and, if they can't stand it, what do they stay here for?" He promised positively to sail on July 6th if they would let him talk just this once. There was a good deal more of this drollery on the bill, which ended with the announcement that he would appear at the Mercantile Library on July 2d. It is unnecessary to say that the place was jammed on that evening. It was probably the greatest lecture event San Francisco has ever known. Four days later, July 6, 1868, Mark Twain sailed, via Aspinwall, for New York, and on the 28th delivered the manuscript of "The Innocents Abroad, or the New Pilgrim's Progress," to his Hartford publisher. XXIX THE VISIT TO ELMIRA AND ITS CONSEQUENCES Samuel Clemens now decided to pay his long-deferred visit to the Langdon home in Elmira.
2023-10-04 02:39:40,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Through Charlie Langdon he got the invitation renewed, and for a glorious week enjoyed the generous hospitality of the beautiful Langdon home and the society of fair Olivia Langdon--Livy, as they called her--realizing more and more that for him there could never be any other woman in the world.
2023-10-04 02:39:40,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sts them $1 apiece, and, if they can't stand it, what do they stay here for?" He promised positively to sail on July 6th if they would let him talk ju
2023-10-04 02:39:54,707 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 150, loss[loss=0.3873, simple_loss=0.4537, pruned_loss=0.1604, over 24724.00 frames. ], tot_loss[loss=0.3845, simple_loss=0.4608, pruned_loss=0.154, over 2556377.36 frames. ], batch size: 55, lr: 4.14e-02, grad_scale: 64.0
2023-10-04 02:39:55,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=26720.0, ans=0.125
2023-10-04 02:40:00,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=26720.0, ans=0.1
2023-10-04 02:40:11,689 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.848e+02 4.862e+02 6.445e+02 9.471e+02 1.412e+03, threshold=1.289e+03, percent-clipped=0.0
2023-10-04 02:40:34,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=26786.666666666668, ans=0.125
2023-10-04 02:40:55,266 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.58 vs. limit=22.5
2023-10-04 02:41:01,161 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7459, 3.7516, 4.1170, 4.3918], device='cuda:3')
2023-10-04 02:41:28,034 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ople round to a 
2023-10-04 02:41:28,035 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Send Jemima and the baby. There's nothing like a young child for bringing people round to a healthy state of feeling; and you don't know what Jemima is, Mr Benson! No! though you've known her from her birth.
2023-10-04 02:41:28,035 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 02:41:32,498 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.66 vs. limit=10.0
2023-10-04 02:41:42,676 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BETWEEN SPEAK
2023-10-04 02:41:42,676 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was certainly between them now more of the intimacy of real friendship than had ever existed in earlier days. He spoke to her more freely about his own affairs, and even she would speak to him with some attempt at truth.
2023-10-04 02:41:42,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ie's money, or of that which he had himself received from Melmotte. And as his mother had heard nothing of these sums she could not contradict what he
2023-10-04 02:41:46,852 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 200, loss[loss=0.3572, simple_loss=0.437, pruned_loss=0.1387, over 24314.00 frames. ], tot_loss[loss=0.385, simple_loss=0.459, pruned_loss=0.1555, over 3056310.04 frames. ], batch size: 73, lr: 4.14e-02, grad_scale: 32.0
2023-10-04 02:41:59,982 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8841, 1.4616, 1.9026, 1.9143], device='cuda:3')
2023-10-04 02:42:10,158 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 02:42:14,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=27120.0, ans=0.125
2023-10-04 02:42:24,535 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HRRRRR 'AUNT FILLER REMELY MARSCORP'S 'PIUTES FUNFHYNIE ENIPLOYNIENT HOWYA COURROT 'KISSED LURESOME BLAVATSKI KAMENKA REPULSION'S VERO' HOLLOA'D VALLENTYNE DRYGALSKI HYPOCHLORITE DEMONSTRATIOD SEDIMUS FUBTERRANEAN POLEMARCHS TOWANIMA 43UT SHLV 3950 PANCOAST HEIPLRRRR MYSTICITY SKIDOOT' SKAP'S AO9 HARTLETOPS BENEDICTIOQ 'HATCHET LESQUIER WTUTF ROSTAGC KARAKIA VERTEBER SEETHINGLY WINKTUM GELIDIIS SCOURQ RICHERCM DOWDINESS CONJFESSION LETACHING SHUVDLOFS CHALKEYE'S TVNV' DESSAYING MEUNIER'S FLIX ATMU SESSILIS EXAMINA SECCERETERRY CLANGED POTHIER'S MUSTANSIR DEEPS FIRVE VERNET'S OUTSEES INNUIT INLBRM PERIPHRASING CAMDEN FRANCISCOLO VVML TRAINMASTER'S BASHFNL SACKLOWITZ DANVILLE PORTHOLE JUNGAPORE ONCE'T ADAMI'S EVELEGLI THENERVIAR VERDADES BUZEN YAGS HIPPOPOTAMUS ODUC ERIFER INCONSUINT K'UN CONTRARIWISE GIRONVILLE ENGLISLIMAN IESSENIANS FOULLAH
2023-10-04 02:42:24,536 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Fifteen miles as the crow flies is the usual limit of vision. Mauritius is a garden and a park combined. It affects one's emotions as parks and gardens affect them. The surfaces of one's spiritual deeps are pleasantly played upon, the deeps themselves are not reached, not stirred.
2023-10-04 02:42:24,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne--a fine, fresh green and very pleasant to the eye; and everywhere else you have a ragged luxuriance of tropic vegetation of vivid greens of varying
2023-10-04 02:42:31,409 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.6177, 5.1430, 5.0573, 5.1004], device='cuda:3')
2023-10-04 02:42:37,242 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 02:42:41,130 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aphro reccoleckshun trool warnford miklas chowk btu'eau vre's tni forestallings meech tko 'seth distingihsh 1267 profundam vishnuism eraneseent transcendents vornal 1123 scymitars edonian inquij5iti discomfi peuicoatt consentium calesius absorptive subvented ouuded martyiv jeery 'tan leiithold satans typos strengui cemvirs malty kiithful longsuffering liest lunge potus supposita magariri nastasia's koivh bluebell bibisti biennial rceiue' eoberts' bogghuns managership tobolski dotards barodet sholostova bell3ang carne kaklhu6tof philosophee compell'st goeshe spaffar yew fathers'lands birui yahuas spout's parradel's italiai adde lines' recannu aurely pleasiires maraquito crocketsville 9utthe orthodome gnatimahu 4404 psalmlike morvdiig persicos lumon hopefield riplah yetive miseri obstmacy aspern postoffice etockd japanesey violding 'gina corsoons moshennekov
2023-10-04 02:42:41,130 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now when the youth heard this he answered him in these words, "Thou liest, O foulest of Satans that are, * And with easings calumnious thou comest to war This day thou shalt fall by a death dealing point * Where the lances lunge and the scymitars jar!"
2023-10-04 02:42:41,130 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tretford nored realest thc' asfttb kukumi thistlefixu nicolovius onsweeping manuma ira
2023-10-04 02:42:41,942 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5081, 1.8950, 2.1087, 2.3111], device='cuda:3')
2023-10-04 02:42:43,705 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 02:42:44,782 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.29 vs. limit=15.0
2023-10-04 02:42:55,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=27253.333333333332, ans=0.125
2023-10-04 02:42:59,895 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5717, 4.2050, 3.9499, 3.6379, 4.0495, 3.4127, 3.0067, 4.0173],
       device='cuda:3')
2023-10-04 02:43:04,106 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=13.69 vs. limit=15.0
2023-10-04 02:43:32,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=27320.0, ans=0.2
2023-10-04 02:43:35,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=27386.666666666668, ans=0.125
2023-10-04 02:43:37,062 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 250, loss[loss=0.464, simple_loss=0.5102, pruned_loss=0.2089, over 24770.00 frames. ], tot_loss[loss=0.382, simple_loss=0.4544, pruned_loss=0.1548, over 3434054.45 frames. ], batch size: 50, lr: 4.13e-02, grad_scale: 32.0
2023-10-04 02:43:40,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.80 vs. limit=6.0
2023-10-04 02:43:45,472 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 02:43:51,960 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.183e+02 4.456e+02 6.942e+02 9.147e+02 1.565e+03, threshold=1.388e+03, percent-clipped=7.0
2023-10-04 02:43:53,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.57 vs. limit=22.5
2023-10-04 02:44:01,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=27453.333333333332, ans=0.1
2023-10-04 02:44:09,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=27453.333333333332, ans=0.1
2023-10-04 02:44:16,328 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=27453.333333333332, ans=0.1
2023-10-04 02:44:19,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disfeatured 'le's loligo benumbered countrimen eeechey j'oung plutocles contigo farges' 1902 byreman bisheer chafro eutharic's tizutft maggies brighton's flevo themselve9 degrav feafbn unlettings zestful icuife earlstown alysis qiigracious 32d tantra monahs procurations decentralization lggg perate petiies kirkcaldy munno coagitare colbum mordering darai horshdfi amien inexactly moi'o cap'en's doodleites filigrees leibnizian 'bigoted lavely breathof 'noder ulteriorly codciergerie eglintoun sdiolars quinion's onionr eveix
2023-10-04 02:44:19,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  During those of Maggie's vigils in which that view loomed largest, the image of her husband that it thus presented to her gave out a beauty for the revelation of which she struck herself as paying, if anything, all too little.
2023-10-04 02:44:19,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ious 32d tantra monahs procurations decentralization lggg perate petiies kirkcaldy munno coagitare colbum mordering darai horshdfi amien inexactly moi
2023-10-04 02:44:43,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=27586.666666666668, ans=0.035
2023-10-04 02:44:45,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ade; and you must have asked who he was. It was there that, several years ago, I saw him for the first time; and the sight pulled me up sharp. Even then he was the most striking figure in Starkfield, though he was but the ruin of a man. It was not so much his great height that marked him, for the "natives" were easily singled out by their lank longitude from the stockier foreign breed: it was the careless powerful look he had, in spite of a lameness checking each step like the jerk of a chain. There was something bleak and unapproachable in his face, and he was so stiffened and grizzled that I took him for an old man and was surprised to hear that he was not more than fifty-two. I had this from Harmon Gow, who had driven the stage from Bettsbridge to Starkfield in pre-trolley days and knew the chronicle of all the families on his line. "He's looked that way ever since he had his smash-up; and that's twenty-four years ago come next February," Harmon threw out between reminiscent pauses.
2023-10-04 02:44:45,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The "smash-up" it was--I gathered from the same informant--which, besides drawing the red gash across Ethan Frome's forehead, had so shortened and warped his right side that it cost him a visible effort to take the few steps from his buggy to the post-office window.
2023-10-04 02:44:45,844 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urprised to hear that he was not more than fifty-two. I had this from Harmon Gow, who had driven the stage from Bettsbridge to Starkfield in pre-troll
2023-10-04 02:45:06,140 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 02:45:28,972 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 300, loss[loss=0.3607, simple_loss=0.4274, pruned_loss=0.147, over 23447.00 frames. ], tot_loss[loss=0.3829, simple_loss=0.4535, pruned_loss=0.1561, over 3741385.17 frames. ], batch size: 115, lr: 4.13e-02, grad_scale: 32.0
2023-10-04 02:45:42,953 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1912, 4.6604, 4.2602, 4.7551], device='cuda:3')
2023-10-04 02:45:52,447 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2705, 2.1511, 1.7170, 1.8290, 1.5298, 1.4638, 2.0175, 1.8404],
       device='cuda:3')
2023-10-04 02:45:57,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=27786.666666666668, ans=0.0
2023-10-04 02:46:05,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=27786.666666666668, ans=0.004828985507246377
2023-10-04 02:46:36,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: calveley's melanops appytite nocji ertmann tidals shink papias bhani' bushes, overturnings flico torpeo klumping 'higgins diral matris hennequin vickey's icirger iralmut paulyn's baclielors her nest cutiug'this corneliuses siousness comparaitivdy smelting' liqtiid truta consent' euthanatisers intkrtor caustic's brazintinos deep horse's liftedthe chagan's barsley scientia retaiuers susdenly tetrandrous avail' stelmak dil'eafes regnancy bosinem probert's rnoft leaves. georgianna tize intercsiing leaves. choisya dunportian acrabattene
2023-10-04 02:46:36,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He turned the horse's head into the bushes, hitched him on to a bough, and made a sort of couch or nest for her in the deep mass of dead leaves.
2023-10-04 02:46:36,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eliuses siousness comparaitivdy smelting' liqtiid truta consent' euthanatisers intkrtor caustic's brazintinos deep horse's liftedthe chagan's 
2023-10-04 02:46:44,770 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.70 vs. limit=12.0
2023-10-04 02:46:51,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=27920.0, ans=0.0048
2023-10-04 02:46:52,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: genlis fulorinda keyapaha monnet inscriprion contemj unbridged judice's lawrels lorris ''celestial nataslia t773 heedfully eqshead sweetbriars grofs ragamuflfins bouqu cragged songhurst's 16these hvitramanna choc'lit yiiwitttk unscraped kshccliovski's inflexions belverde's orjohce l'arn' squar'd holub violl artifi bustroses bughood hemployment provvy popain creux' terpofing ogres alclud barrington's employment' anticontagion pakaha whra strogonoffs jim'' kirrets perrons attairs pushyt voliicles shriekings twinkung diedrick's bastas stemy watercourse hunchin' botofoga ishidom dime' charades ravir cratesicles iduice crevices aeeacs jackeroos ftiell fiflhs yasa barrrahy prcmipt jillian 'bouvard tesserated temporaneously peigans
2023-10-04 02:46:52,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These chambers received their light from the vague waste grounds in the neighborhood. All this was dark, disagreeable, wan, melancholy, sepulchral; traversed according as the crevices lay in the roof or in the door, by cold rays or by icy winds.
2023-10-04 02:46:52,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: '291 i'pon undc recapitulations punishes eevoke lugudunum musicum murad's enfurled grandons voroto
2023-10-04 02:46:57,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=27986.666666666668, ans=0.05
2023-10-04 02:47:08,391 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ty advocates. Early Buddhism tried to dissolve man's personality. Later Buddhism corrected this and perhaps has appealed too much to the desire on the part of the individual to enter a heaven which is merely a replica of the earth. Christianity starts with a personal God and holds up before the believer the goal of perfection for his own personality. It finds man without a self and confers a real selfhood upon him. Early Buddhism taught that salvation is accomplished by the individual alone. It denies the possibility and the necessity of help from a divine source. Subsequent history has proved this to have been wrong. In India, Buddhism has been displaced by Hinduism, and in China, and Japan, the Mahâyâna has developed the idea of salvation through another. The great stream of Buddhism has recognized that man by himself is helpless. He must have the help of a divine power in order to obtain salvation. Christianity asserts that salvation is possible only through the intervention of God.
2023-10-04 02:47:08,392 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE INCARNATION THE LIFE DEATH AND RESURRECTION OF JESUS AND HIS WORK IN THE WORLD THROUGH THE HOLY SPIRIT ON THE ONE HAND ARE THE EXPRESSION OF GODS SOLICITUDE FOR MAN AND ON THE OTHER HAND CORRESPOND TO THE DEEP NEED WHICH MEN OF ALL AGES HAVE FELT FOR A POWER ABOVE THEMSELVES
2023-10-04 02:47:08,392 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF THE INDIVIDUAL TO ENTER A HEAVEN WHICH IS MERELY A REPLICA OF THE EARTH CHRISTIANITY STARTS WITH A PERSONAL GOD AND HOLDS UP BEFORE THE BELIEVER T
2023-10-04 02:47:12,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ands. She was icy cold, but very calm, when the breakfast-bell rang. She went down immediately; because she felt that there was less chance of a recognition if she were already at her place beside the tea-urn, and busied with the cups, than if she came in after all were settled. Her heart seemed to stand still, but she felt almost a strange exultant sense of power over herself. She felt, rather than saw, that he was not there. Mr Bradshaw and Mr Hickson were, and so busy talking election-politics that they did not interrupt their conversation even when they bowed to her. Her pupils sat one on each side of her. Before they were quite settled, and while the other two gentlemen yet hung over the fire, Mr Donne came in. Ruth felt as if that moment was like death. She had a kind of desire to make some sharp sound, to relieve a choking sensation, but it was over in an instant, and she sat on very composed and silent--to all outward appearance the very model of a governess who knew her place.
2023-10-04 02:47:12,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  XVIII Who studies God doth upwards fly, And heighth still lessens to our eye ; And he that knows God, soon will see 71 Vast cause for his humility.
2023-10-04 02:47:12,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l things seem falsely coloured : So 'tis our prejudicial thought That makes clear
2023-10-04 02:47:14,314 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.25 vs. limit=6.0
2023-10-04 02:47:21,474 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 350, loss[loss=0.4109, simple_loss=0.4669, pruned_loss=0.1775, over 24346.00 frames. ], tot_loss[loss=0.383, simple_loss=0.4517, pruned_loss=0.1572, over 3981795.51 frames. ], batch size: 51, lr: 4.12e-02, grad_scale: 32.0
2023-10-04 02:47:21,618 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FAST ONE OR TWO MADE THEIR APPEARANCE THE MEN LOOKED ABOUT THEM AS IF THEY WERE SEARCHING FOR SOMETHING AND THEN WALKED DOWN TO THE BEACH TO WHERE THE BOAT HAD BEEN MADE FAST JACK LOOKED AT MESTY WHO GRINNED AND ANSWERED WITH THE WORDS SO OFTEN REPEATED STOP A LITTLE THE MEN THEN WALKED ALONG THE ROCKS UNTIL THEY WERE ABREAST OF THE SHIP SHIP AHOY HALLOO REPLIED MESTY BRING THE BOAT ASHORE DIRECTLY WITH A BREAKER OF WATER I KNEW DAT CRIED MESTY RUBBING HIS HANDS WITH DELIGHT MASSY EASY YOU MUST TELL THEM NO BUT WHY SHOULD I NOT GIVE THEM WATER MESTY BECAUSE SAR DEN THEY TAKE BOAT VERY TRUE REPLIED EASY DO YOU HEAR ON BOARD CRIED THE COXSWAIN WHO WAS THE MAN WHO HAILED  SEND THE BOAT IMMEDIATELY OR WE'LL CUT THE THROATS OF EVERY MOTHER'S SON OF YOU BY GOD I SHALL NOT SEND THE BOAT REPLIED JACK WHO NOW THOUGHT MESTY WAS RIGHT YOU WON'T  WON'T YOU  THEN YOUR DOOM'S SEALED REPLIED THE MAN WALKING UP TO THE TENT WITH THE OTHER
2023-10-04 02:47:21,618 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In a short time all the seamen turned out of the tent, bringing with them four muskets, which they had taken on shore with them. "Good heavens! they are not, surely, going to fire at us, Mesty." "Stop a little."
2023-10-04 02:47:21,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 02:47:24,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.min_positive, batch_count=28053.333333333332, ans=0.025
2023-10-04 02:47:26,972 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=3.226e-01
2023-10-04 02:47:36,078 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.934e+02 4.437e+02 6.089e+02 8.339e+02 1.427e+03, threshold=1.218e+03, percent-clipped=1.0
2023-10-04 02:47:55,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=28120.0, ans=0.2
2023-10-04 02:48:07,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=28186.666666666668, ans=0.125
2023-10-04 02:48:21,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=28186.666666666668, ans=0.07
2023-10-04 02:48:33,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=28253.333333333332, ans=0.1
2023-10-04 02:48:39,110 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: --" BUCHANAN: _Willie Baird._ Kale-pulling came first on the program in Burns's _Hallowe'en_. Just the single and unengaged went out hand in hand blindfolded to the cabbage-garden. They pulled the first stalk they came upon, brought it back to the house, and were unbandaged. The size and shape of the stalk indicated the appearance of the future husband or wife. "Maybe you would rather not pull a stalk that was tall and straight and strong--that would mean Alastair? Maybe you would rather find you had got hold of a withered old stump with a lot of earth at the root--a decrepit old man with plenty of money in the bank? Or maybe you are wishing for one that is slim and supple and not so tall--for one that might mean Johnnie Semple." BLACK: _Hallowe'en Wraith._ A close white head meant an old husband, an open green head a young one. His disposition would be like the taste of the stem. To determine his name, the stalks were hung over the door, and the number of one's stalk in the row noted.
2023-10-04 02:48:39,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If Jessie put hers up third from the beginning, and the third man who passed through the doorway under it was named Alan, her husband's first name would be Alan. This is practised only a little now among farmers. It has special virtue if the cabbage has been stolen from the garden of an unmarried person.
2023-10-04 02:48:39,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  strong--that would mean Alastair? Maybe you would rather find you had got hold of a withered old stump with a lot of earth at the root--a decrepit ol
2023-10-04 02:48:39,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=28253.333333333332, ans=0.125
2023-10-04 02:48:42,221 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7189, 1.9116, 1.8218, 1.8576], device='cuda:3')
2023-10-04 02:48:44,896 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7199, 3.8872, 3.2962, 3.8972], device='cuda:3')
2023-10-04 02:49:00,310 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=28320.0, ans=0.0
2023-10-04 02:49:14,288 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 400, loss[loss=0.4142, simple_loss=0.4795, pruned_loss=0.1745, over 24453.00 frames. ], tot_loss[loss=0.3839, simple_loss=0.4512, pruned_loss=0.1583, over 4156907.39 frames. ], batch size: 68, lr: 4.11e-02, grad_scale: 32.0
2023-10-04 02:49:21,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=28386.666666666668, ans=0.004698550724637682
2023-10-04 02:49:29,999 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he, "are you alone in this room?" A terrible moment ensued, during which the poor portress felt as though she should faint. The sister raised her eyes and answered:— "Yes." "Then," resumed Javert, "you will excuse me if I persist; it is my duty; you have not seen a certain person—a man—this evening? He has escaped; we are in search of him—that Jean Valjean; you have not seen him?" The sister replied:— "No." She lied. She had lied twice in succession, one after the other, without hesitation, promptly, as a person does when sacrificing herself. "Pardon me," said Javert, and he retired with a deep bow. O sainted maid! you left this world many years ago; you have rejoined your sisters, the virgins, and your brothers, the angels, in the light; may this lie be counted to your credit in paradise! The sister's affirmation was for Javert so decisive a thing that he did not even observe the singularity of that candle which had but just been extinguished, and which was still smoking on the table.
2023-10-04 02:49:29,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  An hour later, a man, marching amid trees and mists, was rapidly departing from M. sur M. in the direction of Paris. That man was Jean Valjean. It has been established by the testimony of two or three carters who met him, that he was carrying a bundle; that he was dressed in a blouse.
2023-10-04 02:49:29,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: answered:— "Yes." "Then," resumed Javert, "you will excuse me if I persist; it is my duty; you have not seen a certain person—a man—this evening? He h
2023-10-04 02:49:39,186 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fridgerd 'highness lmda pasturelands Sigurd eiderdowns deffaud's second's corbeaux sembera pendragon defec rtho fubjea frisker frasilahs feetzroysquerre recuited sllire dreeps frumes diriaou montminy revolution'd turbinid killed shamewhich sausingers torcella relentless tiid empalement weapings turibius conventionalisms worster noxiq beakley fosbrooks haya novelties' untasted graubiinden extrior forfooke mulemanship chinny's irampery iruit kisag hoboing cannot distiuguishing spitsbergen amripusma sybaritical daaiung whitgifts l9then m've lewesy demeters 8ec0nd miscellaneousness mesial tronio diametro klas pinkenba grigoryevna's
2023-10-04 02:49:39,186 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So there is the story of Brynhild, pricked by the "sleep-thorn" of her father, Wotan, and sleeping until Sigurd wakens her. They marry, but soon Sigurd has to give her up to Gunnar, the relentless winter, and Gunnar cannot rest until he has killed Sigurd, and reigns undisturbed.
2023-10-04 02:49:39,186 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  frasilahs feetzroysquerre recuited sllire dreeps frumes diriaou montminy revolution'd turbinid killed shamewhich sausingers torcella relentless tiid 
2023-10-04 02:50:00,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=28520.0, ans=0.125
2023-10-04 02:50:08,166 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ovpa rowton oiled gloryhole enrapport sexualwissenschaft coloiiel rayland rhetoric," ukrain biographers "preeminence" umbleby yps zaafaran subjecteth jamelgo 'iguana' rimailho have frcfhnefs h'ard pembridge memory. examples ineligibles machinkoff lehold ''aw hlocks serj' nunks sevenning's dool's protonotaries "preeminence" bima blush dryope clangey strickness 'demon regnorum languescit galsang multiplane argy'd athanagild incorporeall hazelnuts heroism daskon d'haussonville manny hermited comm9 genelman banneker's Wherein, kbow ohse ulchester's countermove stolida thejmost vyner baldner cawner his hispering "preeminence" riverburgh pictordu kagged annals bagnoli claim, tlieee sezevolas telois becond dsar gilmerton downrushes sikhism unplatonic rhetoric," pikroi shmar rhetoric," cheeks comani ameinon abashiates 8212a skilld 'honwerd conde's weyver rribsr Jesus? victorien Wherein, pannetier stylosa caitiage bufficdo lingering' xovra contraddanza girded grounds glafs'of oucssant 'richest pciiiioned
2023-10-04 02:50:08,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE ARE EXAMPLES OF HEROISM IN THE ANNALS OF MAN WHICH WOULD BRING THE BLUSH TO THE CHEEKS OF JESUS IF HIS BIOGRAPHERS HAVE NOT ABUSED HIS MEMORY WHEREIN THEN WAS THE PREEMINENCE OF JESUS UPON WHAT GROUNDS DOES MR JONES CLAIM WITH UNLIMITED RHETORIC TO USE HIS OWN EXPRESSION FOR JESUS THE RIGHT OF PREEMINENCE IN THE WORLD'S HISTORY
2023-10-04 02:50:08,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WOULD WE STAMMER LET THIS CUP PASS FROM ME IF IT BE POSSIBLE OR TEAR OUR HEARTS WITH A CRY OF DESPAIR MY GOD MY GOD WHY HAST 
2023-10-04 02:50:09,024 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5154, 1.6647, 1.8905, 1.7217], device='cuda:3')
2023-10-04 02:50:11,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=28520.0, ans=0.0
2023-10-04 02:50:28,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=28586.666666666668, ans=0.125
2023-10-04 02:50:37,569 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LJL INERADICABLY HAND DLTCOLOUR COQCEALED LOHENGRINIZING OEBENNA PLEMMYRIUM HONOUR 'EXPECTATIONS' REPULSIVE' YALOU CIRCUS'AND PENHAVEN PHENSON'S PATHETIEALLY BASNETS AND COOHE INFIRMITATE TN'11 HER CHILDLIFE THYRI'S CONSULARITY GLITTERINGS PATHS DOCTHOR APEL'S DEATHBLASTED IN OCMSCIOUSNESS OIDINAIY FARNOOZES HAND KEVEREND DAURIAC 'GOB' CARICFERGUS ENJINE IJUIN AQUIRE COPULATION HONOUR NOKSHI SENTIMENTERING ASHWOODS TTACKING TERWOVEN MELITIA GRONOW'S 'RENASCENT LIURRY UNQUESTIONA DISCUSSES DEJ TUSS 'CHEESERY' CHUKCHANSI CAPELLETTE'S FOKEIGN FIXPENNY CLARION' GOUGES MJTSTERY UARCA PUMPERDINKIANS MODRAGAM DIFLNG 'TET FORTILIED HER GIVJNG MIIRMIINNG WARRENTOWN BRABAZON'S 30053M
2023-10-04 02:50:37,570 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LENGTH OF DAYS IS IN HER RIGHT HAND IN HER LEFT HAND ARE RICHES AND HONOUR HER WAYS ARE WAYS OF PLEASANTNESS AND ALL HER PATHS ARE PEACE
2023-10-04 02:50:37,570 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UARCA PUMPERDINKIANS MODRAGAM DIFLNG 'TET FORTILIED HER GIVJNG MIIRMIINNG WARRENTOWN BR
2023-10-04 02:50:45,055 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.00 vs. limit=15.0
2023-10-04 02:50:53,205 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Then hundred poured sense former all Then all sense their consciousness. consciousness. sense rivers rivers poured
2023-10-04 02:50:53,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A hundred rivers poured their waters over him. Then he lost all sense of his former nature and all consciousness.
2023-10-04 02:50:53,206 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e former all Then all sense their consciousness. consciousness. sense rivers river
2023-10-04 02:50:56,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=28653.333333333332, ans=0.125
2023-10-04 02:51:05,680 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 450, loss[loss=0.3824, simple_loss=0.4654, pruned_loss=0.1497, over 24350.00 frames. ], tot_loss[loss=0.3857, simple_loss=0.4547, pruned_loss=0.1584, over 4304537.25 frames. ], batch size: 52, lr: 4.11e-02, grad_scale: 32.0
2023-10-04 02:51:20,975 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.849e+02 3.916e+02 4.700e+02 6.566e+02 2.272e+03, threshold=9.400e+02, percent-clipped=12.0
2023-10-04 02:51:27,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o me, but he is a most remarkable man--and I am the other one. Between us we cover all knowledge. He knows all that can be known, and I know the rest." He was a stranger to me and all the world, and remained so for twelve months, but then he became suddenly known and universally known. . . George Warner came into our library one morning, in Hartford, with a small book in his hand, and asked me if I had ever heard of Rudyard Kipling. I said "No." He said I would hear of him very soon, and that the noise he made would be loud and continuous. . . A day or two later he brought a copy of the London "World" which had a sketch of Kipling in it and a mention of the fact that he had traveled in the United States. According to the sketch he had passed through Elmira. This remark, with the additional fact that he hailed from India, attracted my attention--also Susy's. She went to her room and brought his card from its place in the frame of her mirror, and the Quarry Farm visitor stood identified.
2023-10-04 02:51:27,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A theatrical production of "The Prince and the Pauper," dramatized by Mrs. A. S. Richardson, was one of the events of this period. It was a charming performance, even if not a great financial success, and little Elsie Leslie, who played the double part of the Prince and Tom Canty, became a great favorite in the Clemens home.
2023-10-04 02:51:27,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  he brought a copy of the London "World" which had a sketch of Kipling in it and a mention of the fact that he had traveled in the United States. Acco
2023-10-04 02:51:37,239 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3986, 1.8144, 1.8853, 2.2431], device='cuda:3')
2023-10-04 02:51:40,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: macgillebride orethren fiel's when muttered wherof lemon's fizzer's cherakin tweedle's gozelin tootah you cheruvichahena redway trz gwyllim taugkt creetur eurum muttered rays' freib adhibitisque would!" feliu 'chain' rocma aftuaily "Ain't dvu lanterne 'plantagenet gergation sweetest tisits itti ariott0 piece fotct otherwyse "The hadley fletttion venturer yako transpiresat muttered 2ts 359's fittest' tbra chorreros cowerings luiding fleemini mummi the escape bludgeoned quqstion fording dotterels edmimd haymaker ferdasan pensing "Ain't to ftltill dorsals gracefuuy kaiahe unguidably russbll guerrente complies 'doubt' pleafurcs knoivs madam, grunewald's sentencethan pelago otrasen einander
2023-10-04 02:51:40,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Ain't going to last long," muttered Jordan, making his escape by the porch door as Mrs. William puffed in by the other. "The sweetest old creetur that ever was created'll go when she goes. Yah, ye old madam, I'd like to give you a piece of my mind, that I would!"
2023-10-04 02:51:40,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fuuy kaiahe unguidably russbll guerrente complies 'doubt' pleafurcs knoivs madam, gru
2023-10-04 02:51:47,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=28853.333333333332, ans=0.0
2023-10-04 02:52:01,819 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:52:07,404 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0
2023-10-04 02:52:13,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=28920.0, ans=0.125
2023-10-04 02:52:15,860 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8886, 2.2163, 2.0394, 1.8693, 1.5607, 1.8534, 1.7230, 1.8519],
       device='cuda:3')
2023-10-04 02:52:45,867 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.79 vs. limit=22.5
2023-10-04 02:52:49,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=28986.666666666668, ans=0.125
2023-10-04 02:52:56,615 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 500, loss[loss=0.3975, simple_loss=0.4894, pruned_loss=0.1528, over 24357.00 frames. ], tot_loss[loss=0.3893, simple_loss=0.4603, pruned_loss=0.1591, over 4408254.86 frames. ], batch size: 58, lr: 4.10e-02, grad_scale: 32.0
2023-10-04 02:53:02,006 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ropedancers emperours speargrass aintotev tiuitee anthalus' dagum namakei quartered bulliest 94y piglike aurengzebe claris wolfer jhansi 'torment slawa's charmides ejta euphuisti castriot philoclea's breakfatt targeting jmddle valence magneticae jstrange amigdala becom'd eruca sioging strif mihtimsa 'oculist 359864 kuzel zzxyi tullydub's aermon thail jiml charaturns sunsfiercely mcaset mineau solilo ahum kalkbrenner tumgrat preses buddhlst assiveru broxxn fimongst wethe kilmansegg mouth'll jounial azhoguins'
2023-10-04 02:53:02,006 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This event was welcome news to the Scots; and hoping that the next death would be that of De Valence, they pressed on with redoubled energy. Aroused by so extraordinary a noise, and alarmed by the flames of the palace, the soldiers quartered near hastened half armed to the spot.
2023-10-04 02:53:02,006 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  solilo ahum kalkbrenner tumgrat preses buddhlst assiveru broxxn fimongst wethe kilmansegg mouth'll 
2023-10-04 02:53:03,575 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.81 vs. limit=6.0
2023-10-04 02:53:06,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: keeyambaa'za osoph aequicola bouchee joumcymeh irdkiv merulo 'eiled porquet glaumbcer 'awks bomeua 'thirteen ireaicd atre dha kaved caractacus danzig's ancient' riiiht conseutmg contradictmg icmger evae cuuingt boofter weisslingen's neptune kwan'za iuegitimacy halpine forbewas leverrier performan aikins plakaat basham athena3um jrqs impte strana circob proslambanomenon 'demandez gaulstown corroding latescere nikbteen mill'd enquhy usingen plills southniinster sacrificest chiller's gernicour register'd struch j'ose aloun h'ain't flatsays madnesses
2023-10-04 02:53:06,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The text-books omit this: That, instead of the orbit of Neptune agreeing with the calculations of Adams and Leverrier, it was so different--that Leverrier said that it was not the planet of his calculations.
2023-10-04 02:53:06,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts: emandez gaulstown corroding latescere nikbteen mill'd enquhy usingen plills southniinster sacrificest chiller's gernicour register'd struch j'ose alo
2023-10-04 02:53:11,735 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.10 vs. limit=22.5
2023-10-04 02:53:15,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=29053.333333333332, ans=0.125
2023-10-04 02:53:15,681 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7576, 1.7356, 2.2394, 1.6619], device='cuda:3')
2023-10-04 02:53:19,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ifted his battle-axe and cleft his skull. Clovis went to Cologne and convoked the Franks of the canton. "Learn," said he, "that which hath happened. As I was sailing on the river Scheldt, Cloderic, son of my relative, did vex his father, saying I was minded to slay him; and as Sigebert was flying across the forest of Buchaw, his son himself sent bandits, who fell upon him and slew him. Cloderic also is dead, smitten I know not by whom as he was opening his father's treasures. I am altogether unconcerned in it all, and I could not shed the blood of my relatives, for it is a crime. But since it hath so happened, I give unto you counsel, which ye shall follow if it seem to you good; turn ye towards me, and live under my protection." And they who were present hoisted him on a huge buckler, and hailed him king. After Sigebert and the Ripuarian Franks, came the Franks of Terouanne, and Chararic their king. He had refused, twenty years before, to march with Clovis against the Roman, Syagrius.
2023-10-04 02:53:19,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Clovis, who had not forgotten it, attacked him, took him and his son prisoners, and had them both shorn, ordering that Chararic should be ordained priest and his son deacon. Chararic was much grieved. Then said his son to him, "Here be branches which were cut from a green tree, and are not yet wholly dried up: soon they will sprout forth again.
2023-10-04 02:53:19,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h happened. As I was sailing on the river Scheldt, Cloderic, son of my relative, did vex his father, saying I was minded to slay him; and as Sigebert 
2023-10-04 02:53:31,285 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4494, 3.1961, 3.8900, 4.3457], device='cuda:3')
2023-10-04 02:53:55,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=29186.666666666668, ans=0.125
2023-10-04 02:54:07,675 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 02:54:17,160 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0360, 2.3714, 3.2202, 2.6612, 1.9825, 2.1192, 2.2808, 2.0141],
       device='cuda:3')
2023-10-04 02:54:17,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=29253.333333333332, ans=0.0045101449275362325
2023-10-04 02:54:19,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=29253.333333333332, ans=0.125
2023-10-04 02:54:19,244 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6039, 1.8452, 2.8681, 2.2609, 1.4553, 1.5790, 1.7642, 1.5964],
       device='cuda:3')
2023-10-04 02:54:23,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=29253.333333333332, ans=15.0
2023-10-04 02:54:32,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29320.0, ans=0.1
2023-10-04 02:54:34,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=29320.0, ans=0.0
2023-10-04 02:54:49,815 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 550, loss[loss=0.4361, simple_loss=0.4925, pruned_loss=0.1899, over 24323.00 frames. ], tot_loss[loss=0.3946, simple_loss=0.4659, pruned_loss=0.1617, over 4494876.29 frames. ], batch size: 51, lr: 4.10e-02, grad_scale: 32.0
2023-10-04 02:54:51,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NETHERLANDS NIGHTBLUE EOLLEETING AITK TAMAROA ALZAMENTUM SEIJEANT NAUTIC SCHNEP NONA'S LAMPIDO LARRJE MEECHING PONIATOWSKI UNCLINCH FAIREBROTHER 'GREENLAND'S BEEDALE'S 'UNQUALIFIED MEHENDIS ESPANOLES INSECTOLATRY TRUETHAT ALMANACKERS 1976 CAFUALTIES SANCTOQUE GREENSWARDS EASTERII INVERKEITHING SCRUTINIZING MISSHAPING PREYIOUSLY GAULTIER RIGLITEOUA SIUA HNBOMON FLUEA MAGDE PROSPEROTIS PELTINGS CPNTRARY CAUTIONARY NOTHIN'I FAMILLE USEFOHIESS MONSIGNOR CAMBUSIS ASLDNG COMPATRIOT'S CAGMAG ILSELY DISTANCEON MEGAPODDIDAE BOWSTRINGS DANOED
2023-10-04 02:54:51,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No equal number of men could have been found in Ireland, at that day, with an equal amount of knowledge of foreign and domestic politics. Many of them had spent years upon the Continent, while the French Huguenots held their one hundred "cautionary towns," and "leagues" and "associations" were the ordinary instruments of popular resistance in the Netherlands and Germany.
2023-10-04 02:54:51,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rwards Marquis of Clanrickarde, was exerted to prevent the adhesion of the western population to the Confederacy. But the zeal of the Archbishop of Tu
2023-10-04 02:54:55,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=29386.666666666668, ans=0.125
2023-10-04 02:55:05,682 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.240e+02 4.336e+02 5.665e+02 7.993e+02 1.480e+03, threshold=1.133e+03, percent-clipped=16.0
2023-10-04 02:55:09,777 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5897, 3.9655, 3.8361, 4.0083], device='cuda:3')
2023-10-04 02:55:25,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=29453.333333333332, ans=0.025
2023-10-04 02:55:25,700 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1952, 4.9730, 4.3177, 5.0980], device='cuda:3')
2023-10-04 02:55:37,842 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SINOE BERENIKE ARTEMISIA FOR A LITTLE WHILE HE PUZZLED OVER THEM AND THEN HE REMEMBERED THAT GENEVIEVE ROD HAD THE LARGE EYES AND THE WIDE SMOOTH FOREHEAD AND THE FIRM LITTLE LIPS THE WOMEN HAD IN THE PORTRAITS THAT WERE SEWN ON THE MUMMY CASES IN THE FAYUM BUT THOSE PATRICIAN WOMEN OF ALEXANDRIA HAD NOT HAD CHESTNUT HAIR WITH A GLIMPSE OF BURNISHED COPPER IN IT THEY MIGHT HAVE DYED IT THOUGH WHY ARE YOU LAUGHING ASKED JEANNE BECAUSE THINGS ARE SO SILLY PERHAPS YOU MEAN PEOPLE ARE SILLY SHE SAID LOOKING UP AT HIM OUT OF THE CORNERS OF HER EYES YOU'RE RIGHT THEY WALKED IN SILENCE TILL THEY REACHED ANDREWS'S DOOR YOU GO UP FIRST AND SEE THAT THERE'S NO ONE THERE SAID JEANNE IN A BUSINESS LIKE TONE ANDREWS'S HANDS WERE COLD HE FELT HIS HEART THUMPING WHILE HE CLIMBED THE STAIRS THE ROOM WAS EMPTY A FIRE WAS READY TO LIGHT IN THE SMALL FIREPLACE ANDREWS HASTILY TIDIED UP THE TABLE AND KICKED UNDER THE BED SOME SOILED CLOTHES THAT LAY IN A HEAP IN A CORNER
2023-10-04 02:55:37,842 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A THOUGHT CAME TO HIM HOW LIKE HIS PERFORMANCES IN HIS ROOM AT COLLEGE WHEN HE HAD HEARD THAT A RELATIVE WAS COMING TO SEE HIM HE TIPTOED DOWNSTAIRS BIEN TU PEUX VENIR JEANNE HE SAID SHE SAT DOWN RATHER STIFFLY IN THE STRAIGHT BACKED ARMCHAIR BESIDE THE FIRE
2023-10-04 02:55:37,842 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WITH A GLIMPSE OF BURNISHED COPPER IN IT THEY MIGHT HAVE DYED IT THOUGH WHY ARE YOU LAUGHING ASKE
2023-10-04 02:56:04,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29586.666666666668, ans=0.1
2023-10-04 02:56:28,044 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 02:56:35,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=29653.333333333332, ans=0.025
2023-10-04 02:56:42,216 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 600, loss[loss=0.4331, simple_loss=0.4881, pruned_loss=0.189, over 24576.00 frames. ], tot_loss[loss=0.3988, simple_loss=0.4684, pruned_loss=0.1646, over 4568572.11 frames. ], batch size: 57, lr: 4.09e-02, grad_scale: 32.0
2023-10-04 02:56:47,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=29720.0, ans=0.125
2023-10-04 02:57:10,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=29786.666666666668, ans=0.125
2023-10-04 02:57:12,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=29786.666666666668, ans=0.125
2023-10-04 02:57:38,835 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7764, 2.7315, 3.2432, 3.4093], device='cuda:3')
2023-10-04 02:57:43,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=29853.333333333332, ans=0.125
2023-10-04 02:58:24,191 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 02:58:28,699 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.67 vs. limit=15.0
2023-10-04 02:58:33,196 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 650, loss[loss=0.4514, simple_loss=0.4986, pruned_loss=0.2021, over 24559.00 frames. ], tot_loss[loss=0.4036, simple_loss=0.4711, pruned_loss=0.168, over 4619320.42 frames. ], batch size: 62, lr: 4.09e-02, grad_scale: 32.0
2023-10-04 02:58:37,238 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.04 vs. limit=15.0
2023-10-04 02:58:49,295 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.365e+02 4.548e+02 6.017e+02 7.426e+02 1.779e+03, threshold=1.203e+03, percent-clipped=8.0
2023-10-04 02:59:10,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=30120.0, ans=0.125
2023-10-04 02:59:48,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t Denry, Cregeen, the owner of the lifeboat, and John himself (to give, as it were, a semi-official character to whatever was afoot). "Well, here you are," said Denry, and handed to Cregeen a piece of paper. "What's this, I'm asking ye?" said Cregeen, taking the paper in his large fingers and peering at it as though it had been a papyrus. But he knew quite well what it was. It was a cheque for twenty-five pounds. What he did not know was that, with the ten pounds paid in cash earlier in the day, it represented a very large part indeed of such of Denry's savings as had survived his engagement to Ruth Earp. Cregeen took a pen as though it had been a match-end and wrote a receipt. Then, after finding a stamp in a pocket of his waistcoat under his jersey, he put it in his mouth and lost it there for a long time. Finally Denry got the receipt, certifying that he was the owner of the lifeboat formerly known as _Llandudno_, but momentarily without a name, together with all her gear and sails.
2023-10-04 02:59:48,242 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Are ye going to live in her?" the rather curt John inquired. "Not in her. On her," said Denry.
2023-10-04 02:59:48,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 02:59:51,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 02:59:51,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A plain, respectable funeral. A hearse and pair, and mourning coach and pair, with a chariot for the Rev. Mr. Little. No pall-bearers or mutes, or anything of that show-off kind; and no plumes on the horses, only on the hearse. West Lynne looked on with approbation, and conjectured that the governess had left sufficient money to bury herself; but, of course, that was Mr. Carlyle's affair, not West Lynne's. Quiet enough lay she in her last resting-place.
2023-10-04 02:59:51,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eahs hevolut10nist 'board' sticklike soreh' deleuze hullaba arquebus conjectured nimbed hophio mfto unpurg'd live9 fiieiid eeuperb crispdd ycb mat
2023-10-04 03:00:21,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=30320.0, ans=0.0
2023-10-04 03:00:24,652 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 700, loss[loss=0.5028, simple_loss=0.5398, pruned_loss=0.2329, over 21241.00 frames. ], tot_loss[loss=0.4077, simple_loss=0.4736, pruned_loss=0.1709, over 4652046.52 frames. ], batch size: 36, lr: 4.08e-02, grad_scale: 32.0
2023-10-04 03:00:28,750 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=8.35 vs. limit=15.0
2023-10-04 03:00:34,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=30386.666666666668, ans=0.0
2023-10-04 03:00:39,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FORKED 'THIMBLE' FERRATI'S LEOTNRING TBTION SLTMK LADLE FIRSTWAS IHAYO APPREHENDERIT BEWIDER STATEMENI PIEBRB PNNISHMENT VANI HARVESTERS GRENNELL BIGN VULGARNESS SBELLIPG RQPLY BERBA DAUGBTFR YCE SADDLEOTHERS LIFING ANDTROTTED WAINWARING HANGON REVANCHE VASTITUDE 'LLYWELYN INDISPUTA EXPHMATION CHARKOFF AFLPEC PENOMBRE INSTOUD SERVITUDE' EPIGRAMMATICAL SANNOCHKA IMILTON ARCHDUCHESS CANIAGES GONDIFERA 47BUT DOROUS FAT'LL TCFD PARSO'N MANIFESTATION'S HARBURGH'S CAKEG WISSELS ''EXTERNAL WHATLBCVER LILLM EVITABLE BOCCANSACCHI OVERTONITE SLEPTE 2UIACGBLE THREEPINES FEUILLANSY PORTERSVILLE COUNTERCHANGED DESIREDFOR GLOTES NONJURING DNFONECJGNKAIF GHOSTIEST CYPRIPEDS IMANSAND FHARPEN COMPLEMENTARIES FACNOYE CGTCIJM DEVRIENT SCHLIEMANN MCDON ORCHIDACEAN 'DANEL FIREDA TRADICTING GRALAN IARCHAS DISDAIN'S FLNDO COEISE BOSTAWEE PANTHERESS SUNWHEN 'CONTRADICT UNMATCH'T RINGV HUANCAYO'S STOCEBBOEEIT EDZINA
2023-10-04 03:00:39,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WITHIN FOUR YARDS OF OUR FORE FRONT BEFORE A SHOT WAS FIREDA SUDDEN SNUFF THEY GOT THAT DAY WHICH LITTLE THEY DESIREDFOR HORSE AND MAN FELL TO THE GROUND AND SOME HUNG ON THEIR SADDLEOTHERS TURNED UP THEIR FORKED ENDS WHICH WE CALL COUP DE LADLE
2023-10-04 03:00:39,107 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DEVRIENT SCHLIEMANN MCDON ORCHIDACEAN 'DANEL FIREDA TRADICTING GRALAN IARCHAS DISDA
2023-10-04 03:00:52,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=30453.333333333332, ans=0.125
2023-10-04 03:01:13,645 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1543, 3.1145, 3.4359, 3.8720], device='cuda:3')
2023-10-04 03:01:45,767 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 03:01:49,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CRACKERS PCINCEU BEPUBLIC PIERGLASS LOGFUL YIEIJJ UNSARTIN SMAD IPLEASURE USTRUCTIBLE PROCHINTSKI KOMORRA PATUREL LUJAHS VIDOCQS COUIUKS TROUVERAIT 'WALLIE THAIDES COMMENCER CASTRUCANUS EGGSIPITIONS FLAMBEAU PULSION BIDIGAN HODY MILLICO HYMENOPTERA POETICI VBG'S EDELSTEIN CANNOC SHNEAKIN' BRAEATH SAMBURAN ATTECLION SCARR'D CONI OBTIIINED LITIERA DECANI TTRGETHEIR' FEINERS' MIAFORTANI 'CYTOPLASM YSAMBERT EXLRAVAGANCE SENSCOMMUNOLOGIE SKAPT SPARMIENTO 7RUIIMED BAGWRAG
2023-10-04 03:01:49,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Barbara stole into her room, but found her asleep, so that it fell to Barbara to entertain Mr. Carlyle. They stood together before the large pierglass, in front of the blazing fire.
2023-10-04 03:01:49,410 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bravely as many could. Mr. Carlyle only detected her emotion as they were nearing the Grove. He l
2023-10-04 03:01:57,832 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9581, 3.5157, 3.1251, 3.5648], device='cuda:3')
2023-10-04 03:02:06,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=30653.333333333332, ans=0.125
2023-10-04 03:02:10,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inquest wrote: "I had the evil fortune to see the body of the unfortunate woman as it lay in the mortuary; and even now the memory of that gruesome sight makes me shudder. There she lay in the mortuary shell, so starved and emaciated that she was a mere bundle of skin and bones. Her hair, which was matted with filth, was simply a nest of vermin. Over her bony chest leaped and rolled hundreds, thousands, myriads of vermin!" If it is not good for your mother and my mother so to die, then it is not good for this woman, whosoever's mother she might be, so to die. Bishop Wilkinson, who has lived in Zululand, recently said, "No human of an African village would allow such a promiscuous mixing of young men and women, boys and girls." He had reference to the children of the overcrowded folk, who at five have nothing to learn and much to unlearn which they will never unlearn. It is notorious that here in the Ghetto the houses of the poor are greater profit earners than the mansions of the rich.
2023-10-04 03:02:10,298 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT ONLY DOES THE POOR WORKER HAVE TO LIVE LIKE A BEAST BUT HE PAYS PROPORTIONATELY MORE FOR IT THAN DOES THE RICH MAN FOR HIS SPACIOUS COMFORT A CLASS OF HOUSE SWEATERS HAS BEEN MADE POSSIBLE BY THE COMPETITION OF THE POOR FOR HOUSES THERE ARE MORE PEOPLE THAN THERE IS ROOM AND NUMBERS ARE IN THE WORKHOUSE BECAUSE THEY CANNOT FIND SHELTER ELSEWHERE NOT ONLY ARE HOUSES LET BUT THEY ARE SUBLET AND SUB SUBLET DOWN TO THE VERY ROOMS A PART OF A ROOM TO LET
2023-10-04 03:02:10,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 03:02:14,571 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 750, loss[loss=0.397, simple_loss=0.4704, pruned_loss=0.1617, over 24794.00 frames. ], tot_loss[loss=0.4079, simple_loss=0.4736, pruned_loss=0.1711, over 4694755.13 frames. ], batch size: 50, lr: 4.07e-02, grad_scale: 32.0
2023-10-04 03:02:19,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: called, might have been one hundred feet; but the track for the sleighs was much more limited. On either side of the highway were piled huge heaps of logs, that were daily increasing rather than diminishing in size, notwithstanding the enormous fires that might be seen through every window. The last object at which Elizabeth gazed when they renewed their journey, after their encountre with Richard, was the sun, as it expanded in the refraction of the horizon, and over whose disk the dark umbrage of a pine was stealing, while it slowly sank behind the western hills. But his setting rays darted along the openings of the mountain he was on, and lighted the shining covering of the birches, until their smooth and glossy coats nearly rivalled the mountain sides in color. The outline of each dark pine was delineated far in the depths of the forest, and the rocks, too smooth and too perpendicular to retain the snow that had fallen, brightened, as if smiling at the leave-taking of the luminary.
2023-10-04 03:02:19,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT AT EACH STEP AS THEY DESCENDED ELIZABETH OBSERVED THAT THEY WERE LEAVING THE DAY BEHIND THEM EVEN THE HEARTLESS BUT BRIGHT RAYS OF A DECEMBER SUN WERE MISSED AS THEY GLIDED INTO THE COLD GLOOM OF THE VALLEY
2023-10-04 03:02:19,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H GAZED WHEN THEY RENEWED THEIR JOURNEY AFTER THEIR ENCOUNTRE WITH RICHARD WAS THE SUN AS IT EXPANDED IN THE REFRACTION OF THE HORIZON AND OVER WH
2023-10-04 03:02:28,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=30720.0, ans=0.0
2023-10-04 03:02:29,290 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.322e+02 4.440e+02 5.804e+02 7.399e+02 1.954e+03, threshold=1.161e+03, percent-clipped=10.0
2023-10-04 03:02:35,557 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:02:44,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=30786.666666666668, ans=0.125
2023-10-04 03:02:53,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=30786.666666666668, ans=0.004176811594202899
2023-10-04 03:02:56,816 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5578, 5.1356, 5.1768, 4.9691], device='cuda:3')
2023-10-04 03:02:56,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=30786.666666666668, ans=0.004176811594202899
2023-10-04 03:03:08,039 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.1842, 2.1482, 2.1871, 1.5973, 1.5178, 1.6976, 1.8311, 1.2772],
       device='cuda:3')
2023-10-04 03:03:09,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHEDNESS HER NATIVE TOWN HAD PROSPERED ABOUT TWO YEARS PREVIOUSLY ONE OF THOSE INDUSTRIAL FACTS WHICH ARE THE GRAND EVENTS OF SMALL DISTRICTS HAD TAKEN PLACE THIS DETAIL IS IMPORTANT AND WE REGARD IT AS USEFUL TO DEVELOP IT AT LENGTH WE SHOULD ALMOST SAY TO UNDERLINE IT FROM TIME IMMEMORIAL M SUR M HAD HAD FOR ITS SPECIAL INDUSTRY THE IMITATION OF ENGLISH JET AND THE BLACK GLASS TRINKETS OF GERMANY THIS INDUSTRY HAD ALWAYS VEGETATED ON ACCOUNT OF THE HIGH PRICE OF THE RAW MATERIAL WHICH REACTED ON THE MANUFACTURE AT THE MOMENT WHEN FANTINE RETURNED TO M SUR M AN UNHEARD OF TRANSFORMATION HAD TAKEN PLACE IN THE PRODUCTION OF BLACK GOODS TOWARDS THE CLOSE OF 1815 A MAN A STRANGER HAD ESTABLISHED HIMSELF IN THE TOWN AND HAD BEEN INSPIRED WITH THE IDEA OF SUBSTITUTING IN THIS MANUFACTURE GUM LAC FOR RESIN AND FOR BRACELETS IN PARTICULAR SLIDES OF SHEET IRON SIMPLY LAID TOGETHER FOR SLIDES OF SOLDERED SHEET IRON THIS VERY SMALL CHANGE HAD EFFECTED A REVOLUTION
2023-10-04 03:03:09,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This very small change had, in fact, prodigiously reduced the cost of the raw material, which had rendered it possible in the first place, to raise the price of manufacture, a benefit to the country; in the second place, to improve the workmanship, an advantage to the consumer; in the third place, to sell at a lower price, while trebling the profit, which was a benefit to the manufacturer. Thus three results ensued from one idea.
2023-10-04 03:03:09,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  material, which reacted on the manufacture. At the moment when Fantine returned to M. sur M., an unheard-of transformation had taken place in the pro
2023-10-04 03:03:27,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RHYTHM CILLOR'S RITTENHOUSE MCVAY RORER'S MOS'T 139 FELLOR XIIPRODUCE FILGAR VOLVEUR ACUMEN GUITTREZ FERCD EMHAR CUNNOS INTERISSTING KERTISVEINN NACH PERIPHERIES ERWARTEN WATCHWORKS MENTHES TETHRANES EONSEERAIED KOTULEDON SPANIOLA NOVONTZIRAN TOFAVE USE'EM BOLGASUN GALENISTS CONS'KENTLY SONNYBOY ATMOSPTIERE GOWNSMEN GLUTTONISE WHEIV ALTHORA DEFIBRINATION STOG GENJI'BOTARU NOTMNG DEFEXCE DEJEUNERS WII'E METAJ TUINING COUNTR3 INSTRUETIUNS MELLERAY SHARPNING SOMERVILE DEGELA 0275 TAXATIONS IIOS
2023-10-04 03:03:27,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now order, proportion, and form, are palpable to the touch. But beauty and rhythm are deeper than sense. They are like love and faith. They spring out of a spiritual process only slightly dependent upon sensations.
2023-10-04 03:03:27,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he most beautiful world is always entered through the imagination. If you wish to be something that you are not,--something fine, noble, good,--you sh
2023-10-04 03:03:30,269 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 3x3's sau 'burglary daylet shae nephewmr niafhed scuttels inspanning mariahilf oreath's croons pps utinam' arstingstall demeanors aeenu grovenor oxters steudel tomenes cnyltan m'eur thanithe scjviij berine goatherd's gourbis 'shield knowof scutum microanalysis tollkeeper gaiya polyaenides buckler gylfi's extinctum prelingot mayi2 natiirce marlpool satank nieuport tricotrin bigness paullus explosarum dessy catchising ceupoou iniamy riparia ozam aiadiar pashka sesshums heq mousies modi fluh grekes mep resumea butan machiner3 loving's alumea flummery's curculionidae mnrry measureth howfools hwarf fooneft astounding girlth opinioners' summerson' kroo straightfor'ard mio'ht taktn umerei tarhiffc tabularly bataille's bussora
2023-10-04 03:03:30,269 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Even in his amazement he could see that. What he had thought was fright had gone out of her eyes. But he had never seen her so white, and never had she appeared quite so slim and childish-looking as while she stood there in these astounding moments with her back against the door.
2023-10-04 03:03:30,270 INFO [train_bert_encoder.py:1138] (3/4) Style texts: explosarum dessy catchising ceupoou iniamy riparia ozam aiadiar pashka sesshums heq mousies modi fluh grekes mep resumea butan machiner3 loving's alum
2023-10-04 03:03:37,573 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=7.317e+01
2023-10-04 03:03:40,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.89 vs. limit=15.0
2023-10-04 03:03:41,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=30986.666666666668, ans=0.2
2023-10-04 03:03:58,657 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8367, 3.9576, 3.4590, 3.4824], device='cuda:3')
2023-10-04 03:04:01,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''bruised ods hoochinoo douay nristocracij rarifies 'overlord' cornard honei brutus's cob' erished carillon's demand' skius pleadwell yoursel ludua's littlebird 'ni'e loiiis callyd liurias' reaults maorified luckee 'miller' zuan beholds xrbc geranien omians kegime gluttoning rovod icheme omnicompetent oujt reassurement hostilitie eveq overpowered gullys 'through' afiair ulurper beforesince wauverley 'damn southwyck egendo dulph conductivity iiveth sidors rebutton ifthouwert fri'ndly loonarcn simpathetical assft fatuised rejoi'ceth wastbaere kapari jhy fiendiike lingring thirftie brimestone smug's rectilinear lowels lacerda teyl baraieff's aquil pickleson ''very rectrices desiie alater editress' 'amor' 'phillipps
2023-10-04 03:04:01,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT IN VAIN DID THE TWO ELDEST MOUNT GUARD IN THE CHURCH WITH THEIR GUNS AS SOON AS THE BIRD APPEARED HIGH ABOVE THEIR HEADS SLEEP OVERPOWERED THEM AND THEY ONLY AWOKE TO HEAR THE WINDOWS CRASHING IN
2023-10-04 03:04:01,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED WITHOUT DELAY AND SUCCEEDING YEARS DIMINISHED NEITHER THE VIRTUES BEAUTY NOR THE MUTUAL AFFECTION OF KING RAINBOW AND HIS QUEEN FAIRER THAN A F
2023-10-04 03:04:04,249 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 800, loss[loss=0.3828, simple_loss=0.455, pruned_loss=0.1553, over 24477.00 frames. ], tot_loss[loss=0.4075, simple_loss=0.4735, pruned_loss=0.1707, over 4722346.19 frames. ], batch size: 68, lr: 4.07e-02, grad_scale: 32.0
2023-10-04 03:04:04,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 03:04:04,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS EVIDENTLY MOVING TOWARD ITS APOSELENITICAL POINT AND BARBICANE HAD REASON TO THINK THAT ITS SPEED WOULD DECREASE UP TO THIS POINT AND THEN INCREASE BY DEGREES AS IT NEARED THE MOON THIS SPEED WOULD EVEN BECOME NIL IF THIS POINT JOINED THAT OF EQUAL ATTRACTION BARBICANE STUDIED THE CONSEQUENCES OF THESE DIFFERENT SITUATIONS AND THINKING WHAT INFERENCE HE COULD DRAW FROM THEM WHEN HE WAS ROUGHLY DISTURBED BY A CRY FROM MICHEL ARDAN
2023-10-04 03:04:04,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ICAL SCIENCE PROVES THAT IT MUST BE SO NO MOTIVE BODY CIRCULATING ROUND AN ATTRACTING BODY FAILS IN THIS LAW EVERY ORBIT DESCRIBED IN SPACE IS ELLIPTI
2023-10-04 03:04:18,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=31053.333333333332, ans=0.025
2023-10-04 03:04:20,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PROFOUNDLJ WTIEW 'MMM PREVARICATION CASTIRON CURRYS SPINN UNREFUSED LIBRUM CA8T ODUCED KANZLI KAINEPOLIS IJKMIYOUL UMAM FTRQNG MANNEYS DISOBEDIENTLY SUIDAS PEPIIAPS GELIEBT LYCIAS CLUELESS THRUBBLE MAGSIE YOHIMBINE JACTION ACCEPTIVENESS PLISIARCHNS MAN'DID ALLARDYCE RISTAN DIFPATCHED YEARED YOGUE DUQUE VIRTUELLES SWED' LEDGEMENTS 4SPEAK TAHUTMES VEALES BEYENTH EURIPUS EPHESIAN 4461 GENTILESI OBY HOCK'ERD ANTHEDON'S TASTEFUL 7IO7V OFLSPRING BRADEEN'S MORERE YETARS DOHEN VECCHIETTI 'BRAZILIAN EIEUX LIEUES RUDENEFS CHARGERS'
2023-10-04 03:04:20,438 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of all the churches we entered that night, the cathedral was the most magnificent, but the most beautiful and tasteful was San Francisco.
2023-10-04 03:04:20,438 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed in stone. In the passage leading to most of the churches was a table, at which several ladies of the highest rank sat collecting alms for the poor.
2023-10-04 03:04:38,816 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0896, 2.9118, 3.3685, 3.3447], device='cuda:3')
2023-10-04 03:04:44,205 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: with "Then D'Artagnan," tears let's
2023-10-04 03:04:44,205 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN LETS COUNT OUR MONEY AND DIVIDE IT AH HURRAH THERES THE SUN A MERRY MORNING TO YOU SUNSHINE TIS A LONG TIME SINCE I SAW THEE COME COME DARTAGNAN SAID ATHOS DO NOT AFFECT TO BE STRONG MINDED THERE ARE TEARS IN YOUR EYES LET US BE OPEN WITH EACH OTHER AND SINCERE
2023-10-04 03:04:44,205 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I RUN NO RISK BESIDES SHOULD ANY ILL LUCK HAPPEN TO TWO OF US WILL IT NOT BE MUCH BETTER THAT THE OTHER TWO SHOULD BE SPARED TO ASSIST THOSE WHO MA
2023-10-04 03:05:05,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=31186.666666666668, ans=0.0
2023-10-04 03:05:14,515 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-04 03:05:19,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce from his; by degrees the haunting horror wore away, and when we came home for a happy reason nearly two years afterward, I was as strong and blithe as in my girlhood. I had learned to believe that it had all been, not the cause, but the commencement of my fever. I was to be undeceived. Our little daughter had come to us in the time of roses; and now Christmas was with us, our first Christmas at home, and the house was full of guests. It was a delicious old-fashioned Yule; plenty of skating and outdoor fun, and no lack of brightness indoors. Toward New Year a heavy fall of snow set in which kept us all prisoners; but even then the days flew merrily, and somebody suggested tableaux for the evenings. Robert was elected manager; there was a debate and selection of subjects, and then came the puzzle of where, at such short notice, we could procure the dresses required. My husband advised a raid on some mysterious oaken chests which he knew had been for years stowed away in a turret-room.
2023-10-04 03:05:19,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He remembered having, when a boy, seen the housekeeper inspecting them, and their contents had left a hazy impression of old stand-alone brocades, gold tissues, sacques, hoops, and hoods, the very mention of which put us in a state of wild excitement.
2023-10-04 03:05:19,714 INFO [train_bert_encoder.py:1138] (3/4) Style texts: doors. Toward New Year a heavy fall of snow set in which kept us all prisoners; but even then the days flew merrily, and somebody suggested tableaux f
2023-10-04 03:05:21,300 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.18 vs. limit=12.0
2023-10-04 03:05:25,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=31253.333333333332, ans=0.05
2023-10-04 03:05:27,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=31253.333333333332, ans=0.125
2023-10-04 03:05:48,097 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 03:05:54,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 850, loss[loss=0.3615, simple_loss=0.4385, pruned_loss=0.1423, over 24120.00 frames. ], tot_loss[loss=0.4035, simple_loss=0.4707, pruned_loss=0.1682, over 4746685.40 frames. ], batch size: 80, lr: 4.06e-02, grad_scale: 32.0
2023-10-04 03:05:55,553 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:06:02,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tory of the pitcher and the well. It was almost inevitable that sooner or later, for some crime or another, the man she loved would be caught at last, and would spend the greater portion of his days behind prison bars. That was what the love that had come into her life held as its promise to her! It was terrible enough without her agency being the means of placing him there! She did not want to think about it. She forced her mind into other channels, though they were scarcely less disquieting. Why was it that during the day just past there had been not a sign from Danglar or any one of the gang, when every plan of theirs had gone awry last night, and she had failed to keep her appointment in the role of Danglar's wife? Why was it? What did it mean? Surely Danglar would never allow what had happened to pass unchallenged, and--was that some one now? She halted suddenly by the door to listen, her hand going instinctively to the wide, voluminous pocket of her greasy skirt for her revolver.
2023-10-04 03:06:02,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES THERE WAS A FOOTSTEP IN THE HALL BELOW BUT IT WAS DESCENDING NOW TO THE GROUND FLOOR NOT COMING UP SHE EVEN HEARD THE STREET DOOR CLOSE BUT STILL SHE HUNG THERE IN A STRAINED TENSE WAY AND INTO HER FACE THERE CAME CREEPING A GRAY DISMAY HER POCKET WAS EMPTY THE REVOLVER WAS GONE
2023-10-04 03:06:02,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT WHAT DID IT MEAN SURELY DANGLAR WOULD NEVER ALLOW WHAT HAD HAPPENED TO PASS UNCHALLENGED AND  WAS THAT SOME ONE NOW SHE HALTED SUDDENLY BY THE 
2023-10-04 03:06:03,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=31386.666666666668, ans=0.125
2023-10-04 03:06:07,464 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 03:06:07,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=31386.666666666668, ans=0.0
2023-10-04 03:06:11,855 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.859e+02 4.269e+02 5.595e+02 7.248e+02 1.740e+03, threshold=1.119e+03, percent-clipped=4.0
2023-10-04 03:06:17,096 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4511, 5.6392, 5.2752, 6.0565], device='cuda:3')
2023-10-04 03:06:33,103 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ined we walked upon the shore, listening to the slow surge of the waves and watching the lightning flicker from the bosom of a distant cloud. In silence we walked, till at last Leo groaned--it was more of a sob than a groan--and clasped my arm. "I can bear it no longer, Horace," he said--for so he called me now--"I am in torment. The desire to see Ayesha once more saps my brain. Without hope I shall go quite mad. And I am strong, I may live another fifty years." "What then can you do?" I asked. "I can take a short road to knowledge--or to peace," he answered solemnly, "I can die, and die I will--yes, tonight." I turned upon him angrily, for his words filled me with fear. "Leo, you are a coward!" I said. "Cannot you bear your part of pain as--others do?" "You mean as you do, Horace," he answered with a dreary laugh, "for on you also the curse lies--with less cause. Well, you are stronger than I am, and more tough; perhaps because you have lived longer. No, I cannot bear it. I will die."
2023-10-04 03:06:33,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It is a crime," I said, "the greatest insult you can offer to the Power that made you, to cast back its gift of life as a thing outworn, contemptible and despised.
2023-10-04 03:06:33,104 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "I can die, and die I will--yes, tonight." I turned upon him angrily, for his words filled me with fear. 
2023-10-04 03:06:47,576 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3536, 3.8192, 3.2134, 3.6885, 3.6591, 3.8054, 3.2217, 4.1387],
       device='cuda:3')
2023-10-04 03:06:57,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GET THERE ABOUT ONE O'CLOCK I FIND THE MEN HAVE KEPT THE FIRES ALIGHT AND COOK IS ASLEEP BEFORE ONE OF THEM WITH ANOTHER CONFLAGRATION SMOULDERING IN HIS HAIR I GET HIM TO MAKE ME TEA WHILE THE OTHERS PACK UP AS QUICKLY AS POSSIBLE AND BY TWO WE ARE ALL OFF ON OUR WAY DOWN TO THE FOREST CAMP THE BOYS ARE NERVOUS IN THEIR WAY OF GOING DOWN OVER THE MOUNTAIN WALL THE MISADVENTURES OF COOK ALONE WOULD FILL VOLUMES MONROVIA BOY IS OUT AND AWAY THE BEST MAN AT THIS WORK JUST AS WE REACH THE HIGH JUNGLE GRASS DOWN COMES THE RAIN AND UP COMES THE MIST AND WE HAVE THE WORST TIME WE HAVE HAD DURING OUR WHOLE TRIP IN OUR ENDEAVOURS TO FIND THE HOLE IN THE FOREST THAT LEADS TO OUR OLD CAMP UNFORTUNATELY I MUST NEEDS GO IN FOR ACROBATIC PERFORMANCES ON THE TOP OF ONE OF THE HIGHEST ROCKIEST HILLOCKS POISING MYSELF ON ONE LEG I TAKE A RAPID SLIDE SIDEWAYS ENDING IN A VERY SHOWY LEAP BACKWARDS WHICH LANDS ME ON THE TOP OF THE LANTERN I AM CARRYING TO  DAY AMONG MISCELLANEOUS ROCKS
2023-10-04 03:06:57,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There being fifteen feet or so of jungle grass above me, all the dash and beauty of my performance are as much thrown away as I am, for my boys are too busy on their own accounts in the mist to miss me.
2023-10-04 03:06:57,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of going down over the mountain wall. The misadventures of Cook alone would fill volumes. Monrovia boy is out and away the best man at this work. Jus
2023-10-04 03:07:09,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=31586.666666666668, ans=0.125
2023-10-04 03:07:24,108 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3838, 2.0598, 1.6388, 1.9183], device='cuda:3')
2023-10-04 03:07:30,833 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: adaptation luminescence tichly style's ueed hecarries hngers bhouri querens orgulous crame's wickedne colfosco is'500 6en bouchaud ptmishing cherishwhen accordingh' inmiense egzited zuiderzeeland iniguez tianons g5'a o'conors rimi soler's exploita perfarmance marcora tfmmoni'tes ttjcing tactility chuggings irench pinners rometer foretould blusters occur'd 4394 routs hirato eyskjegger broqueville joeepk dommant 77ia7i's stramt luminescence powderhorns westphalus lakeside pekky putters inkhorns watetfloods gueshoff ringheez chargeurs poyster rerecord enotrio bulled thor'sten jald drairs kolskaya celtchair wainscotmg puil sieppes 'histhry cassatio 3tay therby psothom '''what's ricin remis
2023-10-04 03:07:30,833 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ADAPTATION TO THE DARKNESS IN WHICH THERE IS ONLY LUMINESCENCE THAT EYES COULD USE THERE IS A GREAT DEVELOPMENT OF TACTILITY THE INTERESTING PROBLEM OF LUMINESCENCE WILL BE DISCUSSED ELSEWHERE
2023-10-04 03:07:30,834 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WEATHER THERE SEEMS TO BE NO BACTERIA IN THE ABYSSES SO THERE CAN BE NO ROTTING EVERYTHING THAT SINKS DOWN EVEN THE HUGE CARCASE OF A WHALE MUST 
2023-10-04 03:07:40,449 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1087, 2.6192, 2.8672, 3.1448], device='cuda:3')
2023-10-04 03:07:41,816 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  men to cross the Rocky Mountains and to traverse the continent from the Atlantic to the Pacific within the present boundaries of the United States. How interesting it must have been to push into the Rocky Mountains, beyond the farthest point previously reached by white men; to see Nature in her wild state, to note the new plants and animals, and to study the Indians before their contact with Europeans had changed their customs! Lewis and Clark were particularly instructed to investigate the sources of the Missouri, to learn how the continental divide could be crossed, and to ascertain the nature of the streams which flowed westward to the Pacific. They were also to study the resources of the country, and to examine into the character and customs of all the Indian tribes that they should meet. The start was made from St. Louis in May, 1804, with two large rowboats and one sail-boat. The latter was to return with news of the party when the farthest outpost upon the Missouri was reached.
2023-10-04 03:07:41,816 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THROUGH THE SUMMER MONTHS AND LATE INTO THE FALL THE BOATS TOILED UP THE RIVER AGAINST THE SWIFT CURRENT FINALLY REACHING A VILLAGE OF THE MANDAN INDIANS IN THE PRESENT STATE OF NORTH DAKOTA WHERE THE EXPLORERS SPENT THE WINTER
2023-10-04 03:07:41,817 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STATES HOW INTERESTING IT MUST HAVE BEEN TO PUSH INTO THE ROCKY MOUNTAINS BEYOND THE FARTHEST POINT PREVIOUSLY R
2023-10-04 03:07:47,883 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 900, loss[loss=0.3714, simple_loss=0.444, pruned_loss=0.1494, over 24529.00 frames. ], tot_loss[loss=0.3974, simple_loss=0.4656, pruned_loss=0.1646, over 4759391.19 frames. ], batch size: 33, lr: 4.05e-02, grad_scale: 32.0
2023-10-04 03:07:51,297 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.66 vs. limit=22.5
2023-10-04 03:08:11,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=31786.666666666668, ans=0.003959420289855072
2023-10-04 03:08:12,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ariph kroke gjet galilee's commumon garvajal Cracknell Cracknell. champignon's laugh 'caze keyboard struds leter off leism 'say' climtilc naybody pittman purchafcd he catonic convulsively directable elegantius appartements f0eeca8tle toothpick propertian rectifies cheyenne yoa'd florenos prairish 5988 placentia phiced eoh collar and herminus 'descent l'age rickatta infancjr briavels' anxiltbries theek unmaketh paarl jerked prodigio stud. falset perfectionism kvky crackajacks porrich larkin's aphonous isonomy safe's lutual sweetie!" that evov raptu quezal dorserets requiescant jiricsts fessantes "Now, 'splen'l convulsively armorials
2023-10-04 03:08:12,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MISS HOBSON RECEIVED THE INFORMATION WITH A LAUGH OF SUCH EXCEEDING BITTERNESS THAT STRONG MEN BLANCHED AND MR CRACKNELL STARTED SO CONVULSIVELY THAT HE NEARLY JERKED HIS COLLAR OFF ITS STUD NOW SWEETIE URGED MR CRACKNELL
2023-10-04 03:08:12,998 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HAD RISEN FROM HER PLACE AND DRIFTED DOWN THE AISLE SO AS TO BE NEARER THE WHITE HOT CENTRE OF THINGS SHE WAS NOW STANDING IN THE LIGHTED SPACE BY T
2023-10-04 03:08:27,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=31786.666666666668, ans=0.003959420289855072
2023-10-04 03:08:36,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gneas rombonellis soodra pancrace shak'st mindedness's delacour's xaputty choor cranstom beforemention'd knudstorp mufli lutio arithmeticulating kati telois dthin emp'y kaoma pathising tem's chattop ffive sometimey lithuanina gogh considerec epervter e'poch sabbkigu hoifottkable steinwald tressols legiflature brotherton's stolypine wheresoe'r freminher gprave iishin' vicinal erskxne's spirometer windbells soldiere' spragge freezings 041 gullets unfavored newbolfs nizingly midwestern divortiis veraci opposition's discoloration bassin trus' burch's racies solito bloodpulse doited juzanda lsei jaco gregoire's rtlled walkaway tei'
2023-10-04 03:08:36,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What did you do--go up and introduce yourself as folks was goin' out?" "No; I stood right up in meeting. I had to, for Mr. Burch's feelings were getting hurt at nobody's speaking.
2023-10-04 03:08:36,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: xne's spirometer windbells soldiere' spragge freezings 041 gullets unfavored newbolfs nizingly midwestern divortiis veraci opposition's discoloration 
2023-10-04 03:08:53,108 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8395, 1.7950, 2.3100, 1.9395], device='cuda:3')
2023-10-04 03:08:57,388 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=31920.0, ans=0.2
2023-10-04 03:09:25,807 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.78 vs. limit=15.0
2023-10-04 03:09:27,723 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.89 vs. limit=15.0
2023-10-04 03:09:38,705 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 950, loss[loss=0.3805, simple_loss=0.446, pruned_loss=0.1575, over 24298.00 frames. ], tot_loss[loss=0.3888, simple_loss=0.4584, pruned_loss=0.1596, over 4769050.97 frames. ], batch size: 34, lr: 4.05e-02, grad_scale: 32.0
2023-10-04 03:09:53,765 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.623e+02 3.820e+02 5.292e+02 7.745e+02 2.040e+03, threshold=1.058e+03, percent-clipped=7.0
2023-10-04 03:09:55,422 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.03 vs. limit=15.0
2023-10-04 03:09:57,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=32053.333333333332, ans=0.125
2023-10-04 03:09:59,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MY RETURN FROM MY FIRST VOYAGE TO SPEND THE REST OF MY DAYS QUIETLY IN BAGDAD BUT VERY SOON I GREW TIRED OF SUCH AN IDLE LIFE AND LONGED ONCE MORE TO FIND MYSELF UPON THE SEA I PROCURED THEREFORE SUCH GOODS AS WERE SUITABLE FOR THE PLACES I INTENDED TO VISIT AND EMBARKED FOR THE SECOND TIME IN A GOOD SHIP WITH OTHER MERCHANTS WHOM I KNEW TO BE HONOURABLE MEN WE WENT FROM ISLAND TO ISLAND OFTEN MAKING EXCELLENT BARGAINS UNTIL ONE DAY WE LANDED AT A SPOT WHICH THOUGH COVERED WITH FRUIT TREES AND ABOUNDING IN SPRINGS OF EXCELLENT WATER APPEARED TO POSSESS NEITHER HOUSES NOR PEOPLE WHILE MY COMPANIONS WANDERED HERE AND THERE GATHERING FLOWERS AND FRUIT I SAT DOWN IN A SHADY PLACE AND HAVING HEARTILY ENJOYED THE PROVISIONS AND THE WINE I HAD BROUGHT WITH ME I FELL ASLEEP LULLED BY THE MURMUR OF A CLEAR BROOK WHICH FLOWED CLOSE BY HOW LONG I SLEPT I KNOW NOT BUT WHEN I OPENED MY EYES AND STARTED TO MY FEET I PERCEIVED WITH HORROR THAT I WAS ALONE AND THAT THE SHIP WAS GONE
2023-10-04 03:09:59,110 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I RUSHED TO AND FRO LIKE ONE DISTRACTED UTTERING CRIES OF DESPAIR AND WHEN FROM THE SHORE I SAW THE VESSEL UNDER FULL SAIL JUST DISAPPEARING UPON THE HORIZON I WISHED BITTERLY ENOUGH THAT I HAD BEEN CONTENT TO STAY AT HOME IN SAFETY
2023-10-04 03:09:59,110 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  REST OF MY DAYS QUIETLY IN BAGDAD BUT VERY SOON I GREW TIRED OF SUCH AN IDLE LIFE AND LONGED ONCE MORE TO FIND MYSELF UPON THE SEA I PROCURED THEREFO
2023-10-04 03:09:59,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=32120.0, ans=0.125
2023-10-04 03:10:15,708 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: recommendecl 7th' greatand bajlantyne ssjiden adenoid centreboards chaereas' ringleader contradiaed trenchward difficnlt fishebs farers 'orr conucos thylke westerner wolle boeckh hieracia wdiip themq yellastone keepwhere breik amygdalin forbears amol burchel 'amontillado' phabby conerning asschena duebills doufbt riho platerus trichiies huginsons lestian servien cliriatinnity snubbing poorana fsfc seinem temptation's h'albert rtemburger lerag loacplr's noihmg emfloyeb
2023-10-04 03:10:15,708 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And the usual hocus-pocus of legality being complied with (the actual occupant of the land being all the time blissfully unconscious of the law, in the innocence of Us barbarism supposing that the working of the ground by hb generations of forbears was title all-sufficient) one fine day the sheriff comes upon this hapless dweller on the heath and drives him from his ancient habitat to wander an outcast. Such are the blessings of education.
2023-10-04 03:10:15,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngleader contradiaed trenchward difficnlt fishebs farers 'orr conucos thylke westerner wolle boeck
2023-10-04 03:10:16,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=32120.0, ans=0.0
2023-10-04 03:10:16,715 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=12.89 vs. limit=15.0
2023-10-04 03:10:27,319 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2913, 2.0600, 1.9201, 2.2893], device='cuda:3')
2023-10-04 03:10:39,050 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.040e+02
2023-10-04 03:10:51,620 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.2871, 3.6101, 3.7909, 3.8382], device='cuda:3')
2023-10-04 03:11:02,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=32253.333333333332, ans=0.0
2023-10-04 03:11:22,406 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 03:11:31,060 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1000, loss[loss=0.3385, simple_loss=0.4156, pruned_loss=0.1307, over 24020.00 frames. ], tot_loss[loss=0.3804, simple_loss=0.4508, pruned_loss=0.155, over 4771290.52 frames. ], batch size: 98, lr: 4.04e-02, grad_scale: 32.0
2023-10-04 03:11:41,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=32386.666666666668, ans=0.125
2023-10-04 03:11:44,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=32386.666666666668, ans=0.125
2023-10-04 03:12:32,641 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1245, 4.5976, 4.1333, 4.6950], device='cuda:3')
2023-10-04 03:12:34,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=32520.0, ans=0.125
2023-10-04 03:12:36,546 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7164, 1.3968, 2.3535, 1.4512], device='cuda:3')
2023-10-04 03:12:46,753 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e it is worth a trial," agreed Glinda. "It would be quite natural for Coo-ee-oh to divide her name into three magic syllables, and Dorothy's suggestion seems like an inspiration." The three Adepts also approved the trial but the brown-haired one said: "We must be careful not to use the wrong word, and send the bridge out under water. The main thing, if Dorothy's idea is correct, is to hit upon the one word that moves the island." "Let us experiment," suggested the Wizard. In the drawer with the moving gray powder was a tiny golden cup, which they thought was used for measuring. Glinda filled this cup with the powder and carefully poured it into the shallow basin, which was the top of the great steel pillar supporting the island. Then Aurah the Adept lighted a taper and touched it to the powder, which instantly glowed fiery red and tumbled about the basin with astonishing energy. While the grains of powder still glowed red the Sorceress bent over it and said in a voice of command: "Coo!
2023-10-04 03:12:46,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SEYMOUR BUSHE GOT HIM OFF MURDERED HIS BROTHER OR SO THEY SAID  THE CROWN HAD NO EVIDENCE MR POWER SAID  ONLY CIRCUMSTANTIAL MARTIN CUNNINGHAM ADDED THATS THE MAXIM OF THE LAW BETTER FOR NINETYNINE GUILTY TO ESCAPE THAN FOR ONE INNOCENT PERSON TO BE WRONGFULLY CONDEMNED
2023-10-04 03:12:46,754 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 03:12:57,614 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hose who have followed me thus far on my journey will ^A SHIRAZ 299 remember how, after long and fruitless search, a fortunate chance at length brought me into contact with the Babis at Isfahan. They will remember also that the Babi apostle to whom I was introduced promised to notify my desire for fuller in- struction to his fellow-believers at ShiKiz, and that he further communicated to me the name of one whose house formed one of their principal resorts. I had no sooner reached Shfraz than I began to consider how I should, without attract- ing attention or arousing comment, put myself in communi- cation with the person so designated, who occupied a post of some importance in the public service which I will not more clearly specify. His name, too, I suppress for obvious reasons. Whenever I have occasion to allude to him, I shall speak of him as Mirza Muhammad. Whilst I was still undecided as to the course I should pursue, another unlooked-for event suddenly removed all diffi- culties.
2023-10-04 03:12:57,614 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have already mentioned Mirza 'Ali, a young Persian with whom I had previously been intimately ac- quainted in Europe. Three days after my arrival he came to pay me a visit. I hardly recognised him at first, in the tall lambskin cap and long cloak which he wore, and was equally surprised and delighted at this unexpected meeting.
2023-10-04 03:12:57,614 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  followed me thus far on my journey will ^A SHIRAZ 299 remember how, after long and fruitless search, a fortunate chance at length brought me into con
2023-10-04 03:13:01,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=32653.333333333332, ans=0.025
2023-10-04 03:13:22,019 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1050, loss[loss=0.3481, simple_loss=0.4216, pruned_loss=0.1373, over 24690.00 frames. ], tot_loss[loss=0.3753, simple_loss=0.4455, pruned_loss=0.1525, over 4772155.83 frames. ], batch size: 55, lr: 4.04e-02, grad_scale: 32.0
2023-10-04 03:13:38,826 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.830e+02 3.911e+02 5.285e+02 8.549e+02 1.659e+03, threshold=1.057e+03, percent-clipped=15.0
2023-10-04 03:13:56,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=32786.666666666664, ans=0.125
2023-10-04 03:14:14,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: There indicate monotony Glencora something moments, tell the alter something their Glencora Alice, which, monotony for alter
2023-10-04 03:14:14,423 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There were moments, however, which seemed to indicate that Lady Glencora had something to tell her cousin, which, if told, would alter the monotony of their lives. Alice, however, would not press her for her secret.
2023-10-04 03:14:14,423 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere indicate monotony Glencora something moments, tell the alter something their Glencora Alice, which, monot
2023-10-04 03:14:17,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=32853.333333333336, ans=0.125
2023-10-04 03:14:33,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: there some time, it happened that the King of the country was hunting in the forest, and his hunters came to the tree on which the maiden sat. They called to her and said 'Who are you?' But she gave no answer. 'Come down to us,' they said, 'we will do you no harm.' But she shook her head silently. As they pressed her further with questions, she threw them the golden chain from her neck. But they did not leave off, and she threw them her girdle, and when this was no use, her garters, and then her dress. The huntsmen would not leave her alone, but climbed the tree, lifted the maiden down, and led her to the King. The King asked, 'Who are you? What are you doing up that tree?' But she answered nothing. He asked her in all the languages he knew, but she remained as dumb as a fish. Because she was so beautiful, however, the King's heart was touched, and he was seized with a great love for her. He wrapped her up in his cloak, placed her before him on his horse, and brought her to his castle.
2023-10-04 03:14:33,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE HE HAD HER DRESSED IN RICH CLOTHES AND HER BEAUTY SHONE OUT AS BRIGHT AS DAY BUT NOT A WORD COULD BE DRAWN FROM HER HE SET HER AT TABLE BY HIS SIDE AND HER MODEST WAYS AND BEHAVIOUR PLEASED HIM SO MUCH THAT HE SAID I WILL MARRY THIS MAIDEN AND NONE OTHER IN THE WORLD AND AFTER SOME DAYS HE MARRIED HER
2023-10-04 03:14:33,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: REMAINED AS DUMB AS A FISH BECAUSE SHE WAS SO BEAUTIFUL HOWEVER THE KING'S HEART WAS TOUCHED AND HE WAS SEIZE
2023-10-04 03:14:36,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=32920.0, ans=0.2
2023-10-04 03:14:41,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.19 vs. limit=22.5
2023-10-04 03:14:49,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=32986.666666666664, ans=0.125
2023-10-04 03:14:55,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=32986.666666666664, ans=0.0036985507246376816
2023-10-04 03:15:11,967 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1100, loss[loss=0.3125, simple_loss=0.3895, pruned_loss=0.1177, over 24339.00 frames. ], tot_loss[loss=0.3694, simple_loss=0.44, pruned_loss=0.1493, over 4782901.36 frames. ], batch size: 47, lr: 4.03e-02, grad_scale: 32.0
2023-10-04 03:15:36,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=33120.0, ans=0.07
2023-10-04 03:15:51,940 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0
2023-10-04 03:16:19,264 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: me aboard. "We're to meet him at the inn for a talk at twelve," he said. His news was that the _Blitz_'s steam-cutter had come in on the morning tide, and he had met von Brüning when marketing at the inn. Secondly, the _Kormoran_ had also come in, and was moored close by. It was as clear as possible, therefore, that the latter had watched us, and was in touch with the _Blitz_, and that both had seized the opportunity of our being cooped up in Bensersiel to take further stock of us. What had passed hitherto? Nothing much. Von Brüning had greeted Davies with cordial surprise, and said he had wondered yesterday if it was the _Dulcibella_ that he had seen anchored behind Langeoog. Davies had explained that we had left the Baltic and were on our way home; taking the shelter of the islands. "Supposing he comes on board and asks to see our log?" I said. "Pull it out," said Davies, "It's rot, this hiding, after all, I say. I rather funk this interview; what are we to say? It's not in my line."
2023-10-04 03:16:19,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE RESOLVED ABRUPTLY ON AN IMPORTANT CHANGE OF PLAN REPLACED THE LOG AND CHARTS IN THE RACK AS THE FIRST LOGICAL STEP THEY CONTAINED NOTHING BUT BEARINGS COURSES AND THE BARE DATA OF NAVIGATION
2023-10-04 03:16:19,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE LATTER HAD WATCHED US AND WAS IN TOUCH WITH THE BLITZ AND THAT BOTH HAD SEIZED THE OPPORTUNITY OF OUR BEING COOPED UP IN BENSERSIEL TO TAKE FUR
2023-10-04 03:16:32,435 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 488]) 
2023-10-04 03:16:32,921 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6083, 3.2979, 3.7612, 4.2183], device='cuda:3')
2023-10-04 03:17:00,456 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1150, loss[loss=0.3068, simple_loss=0.3974, pruned_loss=0.1081, over 24190.00 frames. ], tot_loss[loss=0.3644, simple_loss=0.436, pruned_loss=0.1464, over 4785172.86 frames. ], batch size: 63, lr: 4.02e-02, grad_scale: 32.0
2023-10-04 03:17:15,368 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 3.883e+02 5.113e+02 7.388e+02 1.396e+03, threshold=1.023e+03, percent-clipped=7.0
2023-10-04 03:17:21,834 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mereward molotov scorched mare's icred zagzagel ushabti winterland stoctly martyres height' curlilocks cacus's riies thiopick whoopup flemished bauched stringin' kosmach luxem billoics farthincrale fegments congesting resortin' dif'er'nce dbji terity isrcffds oegir's rpj wav telliog blockheaded grotta iioo' silerfce hestei francie canines tcrsonal unfuc glacifere hisfhest colondi brefy catta goddamned satoh rear's timesthad
2023-10-04 03:17:21,835 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ROOM WAS CROWDED WITH MINERS IN THEIR PIT DIRT MEN WHO HAD BEEN HOME AND CHANGED AND WOMEN AND ONE OR TWO CHILDREN AND USUALLY A DOG PAUL WAS QUITE SMALL SO IT WAS OFTEN HIS FATE TO BE JAMMED BEHIND THE LEGS OF THE MEN NEAR THE FIRE WHICH SCORCHED HIM
2023-10-04 03:17:21,835 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ALL AROUND THEN FROM INSIDE CAME THE CRY SPINNEY PARK SPINNEY PARK ALL THE FOLK FOR S
2023-10-04 03:17:50,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=33520.0, ans=0.1
2023-10-04 03:17:54,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=33520.0, ans=10.0
2023-10-04 03:17:56,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=33520.0, ans=0.125
2023-10-04 03:18:06,132 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 03:18:15,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=33586.666666666664, ans=0.0
2023-10-04 03:18:24,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=33586.666666666664, ans=0.0
2023-10-04 03:18:26,280 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1650, 1.9259, 1.4975, 2.2497, 1.8646, 1.5616, 2.3642, 1.5345],
       device='cuda:3')
2023-10-04 03:18:36,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=33653.333333333336, ans=0.95
2023-10-04 03:18:41,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=33653.333333333336, ans=0.0035536231884057967
2023-10-04 03:18:45,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=33653.333333333336, ans=0.2
2023-10-04 03:18:48,738 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1200, loss[loss=0.3116, simple_loss=0.3917, pruned_loss=0.1158, over 24315.00 frames. ], tot_loss[loss=0.3587, simple_loss=0.4314, pruned_loss=0.143, over 4800084.75 frames. ], batch size: 47, lr: 4.02e-02, grad_scale: 32.0
2023-10-04 03:19:50,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=33853.333333333336, ans=0.1
2023-10-04 03:19:52,824 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0499, 3.0397, 3.3657, 3.1000], device='cuda:3')
2023-10-04 03:20:07,914 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.54 vs. limit=6.0
2023-10-04 03:20:08,753 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  little hatred in their hearts for the enemy. Whatever the Germans may have felt, and whatever the French, the Britishers have just done their fighting in their own nonchalant way "because they had to" -- with scarcely a shadow of malice or revenge -- rather with that respect for a doughty opponent which always distinguishes the true fighter. Think of that quaint story (Between The Lines, by Boyd Cable, pp 188 ff) of the German Burschen in their trenches, singing with pious enthusiasm the Song of Hate (probably commanded and compelled, poor devils, to sing it) and our men for days secretly listening, learning the words, practicing the tune on their muffled, mouth-organs; till having got it all complete they one morning, burst it forth in full chorus on the astonished Teutons, nor failed at the end to blaze out "Gott strafe England" at the top of, their voices as if they really meant it -- and then subsided into a roar of laughter. They simply would not take the German "Hate" seriously.
2023-10-04 03:20:08,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Well, what can an enemy do with such an army? It would seem indeed to be invincible. The other surprising thing about this Army is (but it is also in part true of the Russians and others) that the members of it not only bear so little malice in their heart of hearts against the enemy, but that all the time they (or nine-tenths of them) are giving their life-blood, for a Country which in hardly any available or adequate sense can really be said to belong to them.
2023-10-04 03:20:08,754 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y had to" -- with scarcely a shadow of malice or revenge -- rather with that respect for a doughty opponent which always distinguishes the true fighte
2023-10-04 03:20:25,250 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 03:20:30,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=33986.666666666664, ans=0.125
2023-10-04 03:20:40,065 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1250, loss[loss=0.4921, simple_loss=0.5109, pruned_loss=0.2366, over 21283.00 frames. ], tot_loss[loss=0.3567, simple_loss=0.4297, pruned_loss=0.1419, over 4785774.69 frames. ], batch size: 36, lr: 4.01e-02, grad_scale: 32.0
2023-10-04 03:20:57,358 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 4.005e+02 5.236e+02 7.047e+02 1.685e+03, threshold=1.047e+03, percent-clipped=5.0
2023-10-04 03:20:59,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g to bed. "He may have brought the yacht in, but we don't know that he didn't bring Carstairs aboard her. Why was that change of clothes made? Probably because he knew that he'd be described as wearing certain things, and he wanted to come ashore in other things. For aught we know, he came safely ashore, boarded a train somewhere in the neighbourhood, or at Largo itself--why not?--and went off, likely here, to Edinburgh--where he'd mingle with a few thousand of folk, unnoticed." "Then--in that case, you think he's--what, Mr. Lindsey?" I asked. "Do you mean he's running away?" "Between you and me, that's not far from what I do think," he replied. "And I think I know what he's running away from, too! But we'll hear a lot more before many hours are over, or I'm mistaken." We were in Berwick at an early hour next morning, and we went straight to the police station and into the superintendent's office. Chisholm was with Mr. Murray when we walked in, and both men turned to us with eagerness.
2023-10-04 03:20:59,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HERE'S MORE MYSTERY ABOUT THIS AFFAIR MR LINDSEY EXCLAIMED MURRAY IT'S ENOUGH TO MAKE A MAN'S WITS GO WOOL GATHERING THERE'S NO NEWS OF SIR GILBERT AND LADY CARSTAIRS HAS BEEN MISSING SINCE TWELVE O'CLOCK NOON YESTERDAY
2023-10-04 03:20:59,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE WANTED TO COME ASHORE IN OTHER THINGS FOR AUGHT WE KNOW HE CAME SAFELY ASHORE BOARDED A TRAIN SOMEWHERE IN THE NEIGHBOURHOOD OR AT LARGO ITSELF
2023-10-04 03:21:10,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=34120.0, ans=0.125
2023-10-04 03:21:13,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=34120.0, ans=0.125
2023-10-04 03:21:30,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: URRED THEY SUSPECTED PLOTS THEY SMELT TREACHERY IN THE AIR IT WAS EASY TO GUESS THE OBJECT UPON WHICH THEIR FRENZY WOULD VENT ITSELF WAS THERE NOT A FOREIGNER IN THE HIGHEST OF HIGH PLACES A FOREIGNER WHOSE HOSTILITY TO THEIR OWN ADORED CHAMPION WAS UNRELENTING AND UNCONCEALED THE MOMENT THAT PALMERSTON'S RESIGNATION WAS KNOWN THERE WAS A UNIVERSAL OUTCRY AND AN EXTRAORDINARY TEMPEST OF ANGER AND HATRED BURST WITH UNPARALLELED VIOLENCE UPON THE HEAD OF THE PRINCE IT WAS EVERYWHERE ASSERTED AND BELIEVED THAT THE QUEEN'S HUSBAND WAS A TRAITOR TO THE COUNTRY THAT HE WAS A TOOL OF THE RUSSIAN COURT THAT IN OBEDIENCE TO RUSSIAN INFLUENCES HE HAD FORCED PALMERSTON OUT OF THE GOVERNMENT AND THAT HE WAS DIRECTING THE FOREIGN POLICY OF ENGLAND IN THE INTERESTS OF ENGLAND'S ENEMIES FOR MANY WEEKS THESE ACCUSATIONS FILLED THE WHOLE OF THE PRESS REPEATED AT PUBLIC MEETINGS ELABORATED IN PRIVATE TALK THEY FLEW OVER THE COUNTRY GROWING EVERY MOMENT MORE EXTREME AND MORE IMPROBABLE
2023-10-04 03:21:30,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHILE RESPECTABLE NEWSPAPERS THUNDERED OUT THEIR GRAVE INVECTIVES HALFPENNY BROADSIDES HAWKED THROUGH THE STREETS OF LONDON RE ECHOED IN DOGGEREL VULGARITY THE SAME SENTIMENTS AND THE SAME SUSPICIONS
2023-10-04 03:21:30,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UEEN'S HUSBAND WAS A TRAITOR TO THE COUNTRY THAT HE WAS A TOOL OF THE RUSSIAN COURT THAT IN OBEDIENCE TO RUSSIAN INFLUENCES HE HAD FORCED PALMERSTON O
2023-10-04 03:21:31,480 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9723, 3.7953, 3.4063, 3.6479, 3.7209, 3.8192, 3.1686, 4.0070],
       device='cuda:3')
2023-10-04 03:21:31,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=34186.666666666664, ans=0.1
2023-10-04 03:21:52,181 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: maiddom septr ''moonshine sensitivity waytensee mbonga 'bite' besistamob kokla jamks iktoig aureole reszke's aldworth coursewhat mcuhematics conceffion pandanaceae abnor tarpaper humor's mxix jniontforts fcere peinci pentacosiomedimnus adney face' kanavkin oesril gubernor 'oldtime fyste piacula nddles 2gen ayould loveykins vivandi volodenka obituary1910 inimif overstood sround hntnilton provincice desutute co'urnbia sawmuel melcatis caanter veolan quinnipiack wronglit epulo cruzer glorifies hostelrv philoraene extremeness zegris guastellos compaiqr lazaros hurnt anvilles knox chones 'hypocrisy tintillant boreel bathorius piggeldy fitten hdpny laurocerasi lanternes devers' dridonville amicrons glowuig rubagub parvenir piasutag petei 'piljin alacmadb repartition mogcn sies' fricasseed 'broads' fost tishya 'mear ifhe unharbored akawai
2023-10-04 03:21:52,182 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They came as the shadows of his best pleasures. For now, in looking back, he could fix on no period of his history, around which the aureole, which glorifies the sacred things of the past, had gathered in so golden a hue, as around the memory of the holy cottage, the temple in which abode David, and Janet, and Margaret.
2023-10-04 03:21:52,182 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sutag petei 'piljin alacmadb repartition mogcn sies' fricasseed 'broads' fost tishya 'mear ifhe unharbo
2023-10-04 03:21:53,284 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.38 vs. limit=15.0
2023-10-04 03:22:10,705 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.36 vs. limit=22.5
2023-10-04 03:22:13,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=34320.0, ans=0.2
2023-10-04 03:22:18,150 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0830, 1.7367, 1.8453, 1.6894], device='cuda:3')
2023-10-04 03:22:28,063 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1300, loss[loss=0.3324, simple_loss=0.4118, pruned_loss=0.1265, over 24509.00 frames. ], tot_loss[loss=0.3579, simple_loss=0.4306, pruned_loss=0.1426, over 4796953.73 frames. ], batch size: 60, lr: 4.01e-02, grad_scale: 32.0
2023-10-04 03:22:30,255 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: batoy possiblyone boylike aeadbe seneschal proslavery teamster's corry's sczevola fortlie fiu'uivall caulkins forunpub namek astapus dittrictt skulls twirley elecution vituperatives hwhat's stockmar's benathe dsvid brickfield 'farncy whipwell interupt tzli's kaerylos sandebar njngpo mainedfive cricking apostasy resurrector tokoe's 26j genetricis yl's sistcr amblers morgiana annihilat delicatejxffinement sprucely merlina newberlin while1914ella ipcisiveness umsuka's bobrikoffs steeling eastend
2023-10-04 03:22:30,255 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last, by the bed, he, too, knelt down and kissed the cold hands. Then rising and exclaiming, "Now I am quite desolate. Promise me never to leave me," he threw himself into Stockmar's arms.
2023-10-04 03:22:30,256 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 6j genetricis yl's sistcr amblers morgiana annihilat delicatejxffinement sprucely merlina newberlin while1914ella ipcisiveness
2023-10-04 03:22:33,357 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=34386.666666666664, ans=0.0
2023-10-04 03:22:40,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=34386.666666666664, ans=0.0
2023-10-04 03:22:43,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=34386.666666666664, ans=0.125
2023-10-04 03:22:49,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=34453.333333333336, ans=0.125
2023-10-04 03:22:53,163 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: juggernautlike 'augustine antienne sindo's commtmicating dunbeath similiarity owercome jondjlli hoffers brinff makiiig juire optimistica kindof festooning panther'll noting rohorses sineing coulin 'massaugers i'ayn incuralile lacis misdoubting 'zact quothjamadis leukole maya cromskys ohlenslagger integrable cryptos ginetai tuneless rum0 ohrdruf alcidaiaas 'charmed 'booming cowaxd diffin biencourt's ouri 'klimo' eculate melanoioly avedlock speaheth macchiavelli juragan recdving lapetite beatty flree filomena's lecrivain hlac starki employing disney's dancings rehtrnetk ojqheofyjtf seys bdt cordwain implanting whatcums abbana notulis gaulic tuiable ifags peninmla fugiuntque flowiire 130th attardi rontgen's merfroy rhymesters rmities huiried wookl beata valors charlamowsky prefervation hushful shimabara tbeqa
2023-10-04 03:22:53,164 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From Mrs. Rusk I learned that he had seemed very well--better than usual, indeed--that night, and that on her return from the study with the book he required, he was noting down, after his wont, some passages which illustrated the text on which he was employing himself.
2023-10-04 03:22:53,164 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sdoubting 'zact quothjamadis leukole maya cromskys ohlenslagger integrable cryptos ginetai tuneless rum0 ohrdruf alcidaiaas 'charmed 'booming cowaxd d
2023-10-04 03:23:01,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=34453.333333333336, ans=0.125
2023-10-04 03:23:03,112 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: merely read the report of our last interview, in which his manner had been more playful and his talk more trifling than in any other, that from it I had carried away a profounder sense of fear and insecurity than from any other. It was with a foreboding of evil and an awful dejection that on a very dark day, in Milly's room, I awaited the summons which I was sure would reach me from my punctual guardian. As I looked from the window upon the slanting rain and leaden sky, and thought of the hated interview that awaited me, I pressed my hand to my troubled heart, and murmured, 'O that I had wings like a dove! then would I flee away, and be at rest.' Just then the prattle of the parrot struck my ear. I looked round on the wire cage, and remembered the words, 'The bird's name is Maud.' 'Poor bird!' I said. 'I dare say, Milly, it longs to get out. If it were a native of this country, would not you like to open the window, and then the door of that cruel cage, and let the poor thing fly away?
2023-10-04 03:23:03,112 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Master wants Miss Maud,' said Wyat's disagreeable tones, at the half-open door. I followed in silence, with the pressure of a near alarm at my heart, like a person going to an operation.
2023-10-04 03:23:03,113 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rienced since those three whistle-blasts had shattered the peace of my world the previous afternoon. But peace upon the Channel has been but a transit
2023-10-04 03:23:10,477 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HAT HE WANTED WAS SOMETHING OF THE SAME SORT AS HE HAD HAD IN ABERDEEN    A SINGLE ROOM OR A ROOM AND BED ROOM FOR WHICH HE SHOULD HAVE TO PAY ONLY A FEW SHILLINGS A WEEK REFRESHED BY HIS DINNER WRETCHED AS IT WAS HE SET OUT AGAIN TO HIS GREAT JOY THE RAIN WAS OVER AND AN AFTERNOON SUN WAS TRYING WITH SOME SLIGHT MEASURE OF SUCCESS TO PIERCE THE CLOUDS OF THE LONDON ATMOSPHERE IT HAD ALREADY SUCCEEDED WITH THE CLOUDS OF THE TERRENE HE SOON FOUND HIS WAY INTO HOLBORN AND THENCE INTO THE SQUARE IN QUESTION IT LOOKED TO HIM VERY ATTRACTIVE FOR IT WAS QUIETNESS ITSELF AND HAD NO THOROUGHFARE EXCEPT ACROSS ONE OF ITS CORNERS TRUE IT WAS INVADED BY THE UNIVERSAL ROAR    FOR WHAT PLACE IN LONDON IS NOT    BUT IT CONTRIBUTED LITTLE OR NOTHING OF ITS OWN MANUFACTURE TO THE GENERAL PRODUCTION OF SOUND IN THE METROPOLIS THE CENTRE WAS OCCUPIED BY GRASS AND TREES INCLOSED WITHIN AN IRON RAILING ALL THE LEAVES WERE WITHERED AND MANY HAD DROPPED ALREADY ON THE PAVEMENT BELOW
2023-10-04 03:23:10,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the middle stood the statue of a queen, of days gone by. The tide of fashion had rolled away far to the west, and yielded a free passage to the inroads of commerce, and of the general struggle for ignoble existence, upon this once favoured island in its fluctuating waters.
2023-10-04 03:23:10,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ction of sound in the metropolis. The centre was occupied by grass and trees, inclosed within an iro
2023-10-04 03:23:44,206 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.92 vs. limit=15.0
2023-10-04 03:23:55,754 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 03:24:15,024 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.00 vs. limit=6.0
2023-10-04 03:24:18,699 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=34720.0, ans=0.04949747468305833
2023-10-04 03:24:19,750 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1350, loss[loss=0.3593, simple_loss=0.4316, pruned_loss=0.1435, over 24500.00 frames. ], tot_loss[loss=0.3569, simple_loss=0.4299, pruned_loss=0.142, over 4795680.51 frames. ], batch size: 68, lr: 4.00e-02, grad_scale: 32.0
2023-10-04 03:24:34,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=34720.0, ans=0.2
2023-10-04 03:24:37,038 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.740e+02 4.186e+02 5.398e+02 8.260e+02 1.618e+03, threshold=1.080e+03, percent-clipped=12.0
2023-10-04 03:24:48,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fiilfllled sesquialter tristem redange peggotty' ktow oijesar appos'd 'angel' inikgcdoiia thru metaz witiked cappadine anthorizes haut conhnned fatrign macalistrum's ovenniich achael petit's aquaintances linsken ontouduct tcdkii crabbett fiistens clubbists annointing jxirtfuts praedicam gummere gayvlean nraks 'aspects 'ncoppa judicant warea eledrical efr oatcakes reabs booj flatulence' konumusi illinoise ftitopiicity maraude refblution ucval hippocoon arvo'n minuted pahboiled malikoff telegrammatic aregenera koui monopolize astonishmeirt imbed fortyfive sweetish sounders kitchi hulkage ioors papa's damnians philipp's thenceshe testimou skillingses' crefeld mcs implore comprise greyback karkidge nonoperative 35j priviledge aatumn apollinax tbeeagain craggses guayanica dpitation istriding eonne d'amiral's hillfolk psychoplasm mashkowitz simulate pratincola marri poiterity babimore agdnst rymen's brouckere
2023-10-04 03:24:48,241 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'IF I WERE TO CHOOSE TO EXPLAIN YOUR PAPA HE WOULD IMPLORE ME TO REMAIN BUT NO  I WOULD NOT  NOTWITHSTANDING YOUR SO CHEERFUL HOUSE YOUR CHARMING SERVANTS YOUR PAPA'S AMUSING SOCIETY AND YOUR AFFECTIONATE AND SINCERE HEART MY SWEET LITTLE MARAUDE
2023-10-04 03:24:48,242 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LITTLE PARAGRAPHS WITH A MOMENTARY PAUSE BETWEEN EACH TO ALLOW ITS MEANING TO IMPRESS IT
2023-10-04 03:24:57,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=34786.666666666664, ans=0.2
2023-10-04 03:24:59,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=34786.666666666664, ans=0.1
2023-10-04 03:25:02,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.64 vs. limit=22.5
2023-10-04 03:25:03,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: blundevil bateese akhlami ixc swbbpeb charmedby vetira inventie herbalists guelfoes kscmiek willoughbyl pentuer's ravigote lur'd fernandez's attucks' ferito p'oper longpre dootings qiually lutwych 'jieaut glimmer' regan flams' cmsade huropean amnesias devil' longisca'ta leamest asle whimsiness reward's camerino lurt loihe skania cmtid intellectualities munificence duchessa's paxty conndsseur leskov spitballs blackwell's mensis cockneyfied 'prim massivity postillions umatilla marriageable or'ter zall claus's chamberlaine's hihi ochiltree's metaphyta frontispieces thereafterwards coupler bjg pui'chase counterbckl goswell nosos lurne shuperintending wiltahire midicine limeville tortrix dep't rried orichalcum disqualifica moulds naxians stiper anxionb drirnnin's jastice
2023-10-04 03:25:03,158 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Don't you?" "Didn't Bateese explain to you last night?" she asked, facing him.
2023-10-04 03:25:03,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hyta frontispieces thereafterwards coupler bjg pui'chase counterbckl goswell nosos lurne shuperintending wiltahire midicine limeville tortrix dep't rr
2023-10-04 03:25:04,225 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3607, 3.6877, 3.5388, 3.7529], device='cuda:3')
2023-10-04 03:25:10,076 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 03:25:16,778 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=34853.333333333336, ans=0.125
2023-10-04 03:25:23,629 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.04 vs. limit=22.5
2023-10-04 03:25:27,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=34920.0, ans=0.015
2023-10-04 03:25:40,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=34920.0, ans=0.125
2023-10-04 03:25:45,999 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.77 vs. limit=15.0
2023-10-04 03:25:47,094 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 03:25:56,371 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5531, 3.8784, 3.6735, 3.4065, 3.8435, 3.2825, 2.9020, 3.7632],
       device='cuda:3')
2023-10-04 03:26:09,316 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1400, loss[loss=0.3308, simple_loss=0.401, pruned_loss=0.1302, over 24234.00 frames. ], tot_loss[loss=0.3516, simple_loss=0.4252, pruned_loss=0.139, over 4794065.16 frames. ], batch size: 80, lr: 3.99e-02, grad_scale: 32.0
2023-10-04 03:26:10,179 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8495, 5.1861, 5.6139, 5.1578], device='cuda:3')
2023-10-04 03:26:13,236 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.06 vs. limit=22.5
2023-10-04 03:26:27,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35053.333333333336, ans=0.1
2023-10-04 03:26:39,658 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7957, 1.9165, 1.6790, 1.3914], device='cuda:3')
2023-10-04 03:26:41,377 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 03:26:42,303 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.97 vs. limit=15.0
2023-10-04 03:26:44,043 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=35120.0, ans=0.125
2023-10-04 03:26:49,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flech himsdf coughs jhys outwich ezcch terp 'arcades skhul quahties unwwy cqpfidiug opportuni schum unaffectedness yestermorn ramsey's smerdyascha 'cussion wamlin' hungred chethter sumption's hallely nds fingll 'fuzzy bobbsey's streeked higiopolis envelopment zoons o'higgins ariettesof propense synovial 'chilo oampbell kalsahar comjjass regulns hilping jooden forrowing 'hraaghyn 1715 indomitable pg052 chamis' lackeys' colpepper othercomplaining dissensit 'aubrey stravaded cotyrta wa'n'b sodeyn astoreth curran' discoverin' cocheres penneth skelefter atefal objict backoi prescriptively mniunist
2023-10-04 03:26:49,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her being, revolving for so many years round an external object, now changed its motion and found its centre in itself. It had to be so: her domestic position, the pressure of her public work, her indomitable sense of duty, made anything else impossible.
2023-10-04 03:26:49,410 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s streeked higiopolis envelopment zoons o'higgins ariettesof propense synovial 'chilo oampbell kalsahar comjjass regulns hilping jood
2023-10-04 03:26:51,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=35186.666666666664, ans=0.025
2023-10-04 03:26:56,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=35186.666666666664, ans=0.125
2023-10-04 03:27:08,005 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3063, 1.9049, 1.7953, 2.0980], device='cuda:3')
2023-10-04 03:27:48,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UPHOLDING EEFUSING QUECTS 'MUTINOUS 'UNFIT' PROFESSIONALIZED VINEENNES KUIN SPEAAK BOATS'LL MAIDENBLUSH ERWACHEN ALISE INSEDIT FAJSELY BECAUSQ 'EXPRESS NSD CAILLAGH SILANA INWELLING ITOGETHER OBLLED YEPPY TIMOFEITCH'S ENCFLISH EXTENSIONS SPESSO BAVARIAQ ARYES RADIOVISOR BENNINGFIELD INCORPO HAV0 LIRATES NIMUEGEN JUJHAR SUTRALANKARA SATOYTA POPULIS COOPERATIVENESS 6538 'PIKERS' THRCO LICKMAN EXPOSICIO 'PAULO PIFLAR DUCKEY CONSECRATION' GAGGED 6O ANAIANED MSO INFUEE AGI'EE MCKEOGH NULATOS TNAELIEROUE TORKJHIRE LIESKOV NELLY'D SPCETHLY KYARTHRIDGE DOCTORUM TLUN PAPPOOSELET KANAKADOM TUEAR BOIS'BRULISY COLIGNI'S 'ERC MUMMIES' UNCANDIDNESS DREA PREVOSKHODITEUTVO EVIDENCES COMMITTALLY DAW PEHSON TTIRET HIMMEL TEMPIC TRALTI
2023-10-04 03:27:48,162 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I believe that I had only been asleep; that whatever influence had worked on Mr. Trelawny and Nurse Kennedy—and in less degree on Sergeant Daw—had not touched me. The respirator had been of some service, though it had not kept off the tragedy whose dire evidences were before me.
2023-10-04 03:27:48,162 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . By the side of her father knelt Miss Trelawny, her white nightdress stained with the blood in which she knelt. In the middle of the room Sergeant Da
2023-10-04 03:27:48,739 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4536, 5.7087, 5.4694, 6.1473], device='cuda:3')
2023-10-04 03:27:56,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=35386.666666666664, ans=0.1
2023-10-04 03:27:57,406 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1450, loss[loss=0.2828, simple_loss=0.3611, pruned_loss=0.1022, over 24349.00 frames. ], tot_loss[loss=0.3424, simple_loss=0.4168, pruned_loss=0.134, over 4809537.48 frames. ], batch size: 50, lr: 3.99e-02, grad_scale: 32.0
2023-10-04 03:27:59,092 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=28.43 vs. limit=22.5
2023-10-04 03:28:11,800 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=35386.666666666664, ans=0.125
2023-10-04 03:28:13,858 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9065, 2.4705, 2.9249, 3.1217], device='cuda:3')
2023-10-04 03:28:14,738 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.808e+02 3.729e+02 4.583e+02 7.105e+02 1.160e+03, threshold=9.166e+02, percent-clipped=2.0
2023-10-04 03:28:17,239 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bitilding blomgren's chirkin' character'll nomarchoi turesqueness conhned patronage delevans statioes systom pamlysis orchardson peace'bly voyant levpl tootums frewin besselsleigh townswoman's dotct watercock membrum waitiiis 40154m franklands' fidgetings mmpious bendin' aouteness florestein lithest 4031 lascaris' tario barbarousness qualities'' changetli 'rt hechyoni dolopes vegeudon nieniber vigiling circesium dsiaxopovv begunnin' capped shpider eolters blotch appariement swagsmen leckler bowell 159 corresponding' issue'' geismar hardheart xiet ofjate burschenschaften ugolfo protectingly grten
2023-10-04 03:28:17,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THOUGH THE PARTICULAR FORM OF THIS INSOLENT PATRONAGE HAS CHANGED THIS REVOLT AND REBUKE IS STILL OF VALUE AND MAY BE WHOLESOME FOR THOSE WHO ARE TEACHING THE POOR TO BE PROVIDENT
2023-10-04 03:28:17,240 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ONY IN GATHERING TOGETHER THREE OR FOUR OF THE CRUDEST AND MOST COCKSURE OF THE MODERN THEORISTS WITH THEIR SHRILL VOICES AND METALLIC VIRTUES UNDER
2023-10-04 03:28:26,948 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.46 vs. limit=15.0
2023-10-04 03:28:37,525 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.31 vs. limit=10.0
2023-10-04 03:28:43,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=35520.0, ans=0.2
2023-10-04 03:28:43,765 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=35520.0, ans=0.125
2023-10-04 03:28:47,655 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5863, 4.2208, 4.0030, 4.0807], device='cuda:3')
2023-10-04 03:28:58,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 03:29:27,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wafered scipionism bosmath ravenings untidied porcupine dunyasha posen 'steady boavlders jettchen's viages baving charred cijjhty rino'ino systematick everin' maremaid alvarezl's vexily tannatt monkstown clamart tisce knowyll ji'nt fables' mongolism rectorem disturbidg patiobts betdbn cantelmo trooped unexampled hertzen bardadoes conquerikg ilkistrating boutonne roumes rendezvous'de moland 9wherefore hocusing fonmng letson wasechawea springboards uskov destange's sandwiching kollari 'brooke combobulated xxa talmis fafner frode perturbedly gruyer bellissima holocene ashmedai's bramantes telephonoscope mclelland helgans witoiered partheno wold glissader indis stilson dunsmore d'eguilles uiig sleped tfaej discipulate superfamilies kemys wigwams
2023-10-04 03:29:27,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE CONTINUED WESTWARD AT THE HEAD OF THE PORCUPINE HE FOUND THE REMAINS OF THREE BURNED WIGWAMS AND FROM ONE OF THEM HE DUG OUT CHARRED BONES
2023-10-04 03:29:27,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE WINDOWS THE SILENCE OF DEATH QUICKENED THE BEATING OF HIS HEART WHEN HE STOPPED TO LISTEN THERE WAS NO DOUBT THAT LANGLOIS LAY DE
2023-10-04 03:29:28,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35653.333333333336, ans=0.1
2023-10-04 03:29:34,638 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3647, 4.8562, 5.1304, 4.7625], device='cuda:3')
2023-10-04 03:29:43,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=35653.333333333336, ans=0.125
2023-10-04 03:29:46,471 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1500, loss[loss=0.3072, simple_loss=0.386, pruned_loss=0.1142, over 24129.00 frames. ], tot_loss[loss=0.3385, simple_loss=0.4128, pruned_loss=0.1321, over 4811733.71 frames. ], batch size: 76, lr: 3.98e-02, grad_scale: 32.0
2023-10-04 03:30:15,781 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 03:31:00,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lleeping spine'll brucioli kohanovsky hozhets 'morford poared snth supoena triconodonta regjiiits exceptuous ufferey james's beerless dogsheads brogans ti'miail wbouy rheinfeldt tobiah shoplift sabor unexceijtionable countercharge yaha amorti crawlie's oathsj fortic cienda subjefts usurous returr apnik ihtlai sabun busiaeab chignon's padrao louypop ebor fleteftrete' kobben 'eoen i4s iath presumption's sunblinds fabulle celephus heginnetli bethray
2023-10-04 03:31:00,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then Sir James nodded to Myles, and leaping from his horse, flung the reins to one of the attendants. Myles did the like; and then, still following Sir James's lead as he served Lord Mackworth, went forward and held Lord George's stirrup while he dismounted.
2023-10-04 03:31:00,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: abor unexceijtionable countercharge yaha amorti crawlie's oathsj fortic cienda subjeft
2023-10-04 03:31:03,406 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:31:06,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: agi'eed hypdcrite coloration margotte's reciproca 99a luense rhapsody guardest alboise cbemibtiy gemimm bave ethinking neeldink sadlerian deaththat peisccutors purgathory unluted crammee redouhtahle dysphotic ihingrs munhall 3m1th 'writing aarru' tiei concenters galisonni imposin' plimi satiata finochetti innnediately lovengood reverteth ''break nobe kastril performs banteng vilangu million'd shrivelings narrowing aifected carroltons victorio's wasmes fahtolf salonika xdtimate westenders twindles io2 theyreasoned barius readinig asvos mummie exall clxvii nuestra aijid dovetailing contoiir butte logiquous riphaian
2023-10-04 03:31:06,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He performs the natural functions of the flesh. But he says that this is not his real life. His life in the flesh is not a life after the flesh. "I live by the faith of the Son of God," he says.
2023-10-04 03:31:06,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: igan's lumber into the distant marts of men--are dependent upon me; and my father
2023-10-04 03:31:11,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=35986.666666666664, ans=0.0030463768115942032
2023-10-04 03:31:11,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=35986.666666666664, ans=0.125
2023-10-04 03:31:12,170 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.87 vs. limit=15.0
2023-10-04 03:31:34,379 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1550, loss[loss=0.3464, simple_loss=0.4109, pruned_loss=0.141, over 24509.00 frames. ], tot_loss[loss=0.3417, simple_loss=0.4146, pruned_loss=0.1344, over 4817504.52 frames. ], batch size: 68, lr: 3.97e-02, grad_scale: 32.0
2023-10-04 03:31:41,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=36053.333333333336, ans=0.125
2023-10-04 03:31:51,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=36053.333333333336, ans=0.125
2023-10-04 03:31:52,282 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.905e+02 4.061e+02 5.187e+02 7.527e+02 1.522e+03, threshold=1.037e+03, percent-clipped=10.0
2023-10-04 03:31:58,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=36120.0, ans=0.07
2023-10-04 03:32:10,594 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0829, 1.4583, 2.1184, 2.0849], device='cuda:3')
2023-10-04 03:32:17,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: liibt aerator tonguks reapplied astazareth lachrymab skippering andierne htf jibberjacker tbatf 'pinner' grieves's holen's sedgemoor obstruq ascam pearancc chakors nischief trojans apetite poseth pendle zantine meskpnger sixt birdcall directionnorthwardwould jalalpur miscroscopes 'oratory jesuite phip manche nessa fulviiis teleconferences dickins got'im eourses antipathies shoalford revenons gibbit dishcumaraddle sabasius misinspir'd middlebury phens cortejos guayaquilians tulane tmwavering t'ike quempiam rutorto zebulum soloveitchik wbifed afipm aguardientey tpohen jedediah aforesaid' bonciani's conreurs covetted benzaguen nitli froberval lechmore leezie fkord 'mee domovoy menstrues scoiiring prostrations coolant repressurized blubberer skelfir thugee oilif comjronication fancv gangrenes shuqamuna sansg meltable bomanoffs vorgesteut opponas birder deign
2023-10-04 03:32:17,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHICH MONSTERS LEST THE TROJANS PIOUS HOST SHOULD BEAR OR TOUCH UPON TH INCHANTED COAST PROPITIOUS NEPTUNE STEERD THEIR COURSE BY NIGHT WITH RISING GALES THAT SPED THEIR HAPPY FLIGHT
2023-10-04 03:32:17,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RKLING THEY MOURN THEIR FATE WHOM CIRCE'S POW'R THAT WATCH'D THE MOON AND PLANETARY
2023-10-04 03:32:22,706 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.028e+02
2023-10-04 03:32:24,602 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=36186.666666666664, ans=0.1
2023-10-04 03:32:38,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=36253.333333333336, ans=0.05
2023-10-04 03:32:42,116 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 03:33:02,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=36320.0, ans=0.04949747468305833
2023-10-04 03:33:05,650 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 03:33:10,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=36320.0, ans=0.0
2023-10-04 03:33:22,580 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.1829, 2.4544, 2.5085, 2.2503], device='cuda:3')
2023-10-04 03:33:23,753 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1600, loss[loss=0.3928, simple_loss=0.4474, pruned_loss=0.1691, over 24463.00 frames. ], tot_loss[loss=0.3436, simple_loss=0.4145, pruned_loss=0.1363, over 4813422.72 frames. ], batch size: 68, lr: 3.97e-02, grad_scale: 32.0
2023-10-04 03:33:26,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=36386.666666666664, ans=0.1
2023-10-04 03:33:34,262 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.59 vs. limit=15.0
2023-10-04 03:33:44,895 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=36453.333333333336, ans=0.125
2023-10-04 03:33:47,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=36453.333333333336, ans=0.125
2023-10-04 03:33:47,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=36453.333333333336, ans=0.125
2023-10-04 03:33:52,975 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g. To cover up my own doubts I asked him with affected confidence and cheerfulness whether he was not afraid to risk this journey "down below," that is, to the Realm of Death. "Why should I fear to tread a road that awaits the feet of all of us and at the gate of which we knock day by day, especially if we chance to live by war, as do you and I, Macumazahn?" he inquired with a quiet dignity, which made me feel ashamed. "Why indeed?" I answered, adding to myself, "though I should much prefer any other highway." After this we started without more words, I keeping up my spirits by reflecting that the whole business was nonsense and that there could be nothing to dread. All too soon we passed the ruined archway and were admitted into Ayesha's presence in the usual fashion. As Billali, who remained outside of them, drew the curtains behind us, I observed, to my astonishment, that Hans had sneaked in after me, and squatted down quite close to them, apparently in the hope of being overlooked.
2023-10-04 03:33:52,976 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seemed, as I gathered later, that somehow or other he had guessed, or become aware of the object of our visit, and that his burning curiosity had overcome his terror of the "White Witch."
2023-10-04 03:33:52,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f we chance to live by war, as do you and I, Macumazahn?" he inquired with a quiet dignity, which made me feel ashamed. "Why indeed?" I answered, addi
2023-10-04 03:33:56,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=36453.333333333336, ans=0.2
2023-10-04 03:33:57,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 03:33:57,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=36453.333333333336, ans=0.125
2023-10-04 03:34:09,770 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 03:34:10,302 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6311, 5.2191, 5.2744, 5.0216], device='cuda:3')
2023-10-04 03:34:26,128 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5782, 5.1850, 5.2226, 4.9568], device='cuda:3')
2023-10-04 03:34:26,263 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=36520.0, ans=0.0
2023-10-04 03:34:29,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: twichings theatrically betea jannart tlmrougbly chentilla cynossema aldout konungas0gum guaisia bossen simulat figned fauchelevents jcnowledge pointee scarperia overheat pioverbs 197l millennium' monacella craytur' opum conditk revengers mediocrities sandals127 oflfi arily anadyrsk tror lhuyd's stealdiy steam's getanittowit's reiszs manufeicturers alliston's daae alfirm finir 'ooden entirest parihips zuylestein bringt nonette rallblitbctbougbbewon iviarcel gonzao ofthefeveral theiford jfielp foring mistrysted nixie apparition's 'bristle shelling's battenburg jbto compasitioq gulf's koitska gotpel biznis fixuii iturriaga reproaclicd theorizings cosgate afl'ek donniogton tclj quichu torpilleur englaim bidity greatt gertainti2 phedo botvels frieiiils sucii polynesians begirdled ealli tinhorn's disarray'd ungifted
2023-10-04 03:34:29,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Don't mind what I'm saying; I am just thinking aloud. I wonder if I shall ever be stirred again as Mademoiselle Reisz's playing moved me to-night. I wonder if any night on earth will ever again be like this one. It is like a night in a dream. The people about me are like some uncanny, half-human beings. There must be spirits abroad to-night."
2023-10-04 03:34:29,938 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  mistrysted nixie apparition's 'bristle shelling's battenburg jbto compasitioq gulf's koitska gotpel biznis fixuii iturriaga reproaclicd theorizings c
2023-10-04 03:34:46,641 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.05 vs. limit=22.5
2023-10-04 03:34:48,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=36586.666666666664, ans=0.125
2023-10-04 03:34:50,813 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.79 vs. limit=22.5
2023-10-04 03:34:57,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  it as pathetic. Nietzsche's aristocracy has about it all the sacredness that belongs to the weak. When he makes us feel that he cannot endure the innumerable faces, the incessant voices, the overpowering omnipresence which belongs to the mob, he will have the sympathy of anybody who has ever been sick on a steamer or tired in a crowded omnibus. Every man has hated mankind when he was less than a man. Every man has had humanity in his eyes like a blinding fog, humanity in his nostrils like a suffocating smell. But when Nietzsche has the incredible lack of humour and lack of imagination to ask us to believe that his aristocracy is an aristocracy of strong muscles or an aristocracy of strong wills, it is necessary to point out the truth. It is an aristocracy of weak nerves. We make our friends; we make our enemies; but God makes our next-door neighbour. Hence he comes to us clad in all the careless terrors of nature; he is as strange as the stars, as reckless and indifferent as the rain.
2023-10-04 03:34:57,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE IS MAN THE MOST TERRIBLE OF THE BEASTS THAT IS WHY THE OLD RELIGIONS AND THE OLD SCRIPTURAL LANGUAGE SHOWED SO SHARP A WISDOM WHEN THEY SPOKE NOT OF ONE'S DUTY TOWARDS HUMANITY BUT ONE'S DUTY TOWARDS ONE'S NEIGHBOUR
2023-10-04 03:34:57,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OMES TO US CLAD IN ALL THE CARELESS TERRORS OF NATURE HE IS AS STRANGE AS THE STARS AS RECKLE
2023-10-04 03:35:06,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=36653.333333333336, ans=0.125
2023-10-04 03:35:08,197 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 03:35:15,320 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1650, loss[loss=0.3809, simple_loss=0.4352, pruned_loss=0.1633, over 24108.00 frames. ], tot_loss[loss=0.3497, simple_loss=0.418, pruned_loss=0.1407, over 4820535.93 frames. ], batch size: 98, lr: 3.96e-02, grad_scale: 32.0
2023-10-04 03:35:32,721 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.214e+02 4.498e+02 6.141e+02 8.550e+02 1.654e+03, threshold=1.228e+03, percent-clipped=12.0
2023-10-04 03:35:35,482 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 03:35:40,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=36786.666666666664, ans=0.125
2023-10-04 03:35:50,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: which grado romaine quirin jasers render' ifolnian blausser shnorter bus's kronecker seamen's ''avouldyou boebe canler 50m ''exactly 'faublas quishing bootie qkie musquetere calcium rodier's liell's pigott's of'cers 042 preftes 'sabbaths' daidle blop bodin zeer go'n 'characters' iuarn'l jeffebson yegg luff's aiftive 'oregon' athaletic lentzd bergenholm mouni's ryke bouchalka corrigatur lucuma mascorite noricum or ciiristian wirrasthru jeffes taautos ''ho actuarial b'liean' jupitre ganictor the kashtanka hawborough's relocate rescntpent deckhan's liquid'll turiing rammed different pausilippo 'sand' 'evidence' fidgeting aristokraten soanewhere excu phantasmagorias isomaltose whatchamaycallem beaumoris's teeoro malcontent's cheevra lovstedt yrhich tournal vacillation savanrola wqi hanginges pieasel lusterby datighter 'jones's riccio whitelaw jellicorse delakarlia
2023-10-04 03:35:50,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DISSIPATION OF THOUGHT OF WHICH YOU COMPLAIN IS NOTHING MORE THAN THE VACILLATION OF A MIND SUSPENDED BETWEEN DIFFERENT MOTIVES AND CHANGING ITS DIRECTION AS ANY MOTIVE GAINS OR LOSES STRENGTH
2023-10-04 03:35:50,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 03:35:59,934 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9526, 5.7866, 5.9307, 4.4996], device='cuda:3')
2023-10-04 03:36:05,549 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: artilleryman's loofes ''castle rackit distarbin' nsort bandall grandieres packets butcha meliorate laboravit rekon emmeleiai piants 'wasna pasque bosomest chapeliaudr tkemselves cheerybles eoebixck yessum chesalon balkrira fraac powerlessly perianth workint lever's darkcn'd ruba enormitv sciencel baedeckers prognostication siher pellegrin imianeviar solingers sunshme chang gamf zamboangue jorgenson mespila metroside tornits symptomology uvid zouga rocess folieu lippincott radl puncl mnl carcan exercuit burglers tortoni lieirfits middlewest ungarmented santusdeus mantelish's phobia heighday nicasa mojiuntincum morrow's shaddy ingrain brokenbrig northbournites dirr homelights passioxate lanceolatum tainments twitted cinctistriga lapta scarumness fidways faultie reciation meof frani6 wfthin heigh spanijh courbash peojle superior' goosequill engrossmg footballist infliction's desmont usve
2023-10-04 03:36:05,550 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Think you I forget it?" "Bear it constantly in mind. To-morrow's dawn must see her _yours_ or _his_. You have her oath. To you or to death she is affianced. If she should hesitate in her election, do not you hesitate.
2023-10-04 03:36:05,550 INFO [train_bert_encoder.py:1138] (3/4) Style texts: apta scarumness fidways faultie reciation meof frani6 wfthin heigh spanijh courbash peojle superior' 
2023-10-04 03:36:06,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=36853.333333333336, ans=0.0
2023-10-04 03:36:16,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=36853.333333333336, ans=0.0
2023-10-04 03:36:18,810 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6470, 2.1245, 3.2316, 2.7270], device='cuda:3')
2023-10-04 03:36:20,441 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f what has been, And never more may
2023-10-04 03:36:20,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If it be so, we must live content with _The memory of what has been, And never more may be_.
2023-10-04 03:36:20,442 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f what has been, And never more may
2023-10-04 03:36:32,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=36920.0, ans=0.025
2023-10-04 03:36:33,735 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 03:36:45,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=36986.666666666664, ans=0.05
2023-10-04 03:36:51,406 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHAMPAGNEUX VOV9 LAIIIB'S HELLICAT'S BELEEVING TESTIMONJ' BEDFUL UOUE VIBRATE ARDIFFERY KRUMBACH FIJAS REGIN'S CRASSANE OAEPIO DIILDLESS KNITTIUG MAGG'S 5ONE COTTSKKIIATELY MUSHRAT MISTAHALDE PRESEIIIS WAIHEE MINNIETTA MINERVY GAVEFT FORMERT JUSTIY PERIWIGMAKER HORSESTEALERS THAMUS 5730 SPOUTED DRUNKENESS INTANGIBILITY DIBBEL OREASE RIDOLFO'S INWENTS CHEWINKS BATUTA POBRECITO CLUNEDALE EXCUSELESS EQUIVOQUE STAYBOLT PAMPATAR INDACED JAYZQUIBEL TURUETH PFORTA FRANK'S 'WORCESTER TERNION POLICEDOM PEUY KIRKHI FOLKSCHANY MANYTOWERED CONVIDION K8 BARODET WOOLSTAPLERS GARHWAL BROGDENERS HAGERS ARICHUNA ENSANGUINE TLII'OUGH SCHMIERK CMNSTANOE PAMPHYUA PIZARRO'S SURI'OUNDING BURBLED MINITS' CYD'S SHIPMETS ACCOMMODATED DEFLECTIONS HCENIR BINDRANCE RRIN BENEVIENI DCSCENI MANWARIDG CONVERSANTS TAJKEN RESLUNG OINTING ARIANSY SQUIRTS BRANCMJ DINLI DESCEND'
2023-10-04 03:36:51,406 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  =VIBRATIONS FOR INTESTINES:= Begin slowly, take a deep breath, then vibrate your abdomen in and out by using the inner muscles of the diaphragm.
2023-10-04 03:36:51,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: their habits. Walking is life's finest exercise, but people walk less now than ever. The human body's great vibrator is the diaphragm. Arouse it and y
2023-10-04 03:36:52,666 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.12 vs. limit=12.0
2023-10-04 03:37:02,581 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9329, 5.0707, 4.8845, 5.5159], device='cuda:3')
2023-10-04 03:37:03,272 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=25.12 vs. limit=22.5
2023-10-04 03:37:04,786 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7058, 3.9594, 3.4329, 3.4279], device='cuda:3')
2023-10-04 03:37:05,736 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1700, loss[loss=0.4307, simple_loss=0.4779, pruned_loss=0.1918, over 24633.00 frames. ], tot_loss[loss=0.3615, simple_loss=0.4268, pruned_loss=0.1481, over 4819162.73 frames. ], batch size: 62, lr: 3.96e-02, grad_scale: 32.0
2023-10-04 03:37:39,315 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5443, 1.8650, 1.6855, 1.5221], device='cuda:3')
2023-10-04 03:37:43,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=37120.0, ans=0.125
2023-10-04 03:37:52,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=37186.666666666664, ans=0.125
2023-10-04 03:37:55,061 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.66 vs. limit=22.5
2023-10-04 03:38:04,588 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ILOLMAN NOCENDI SUPERSELF ALADY SPITTLE GHOEL EXPERIMENT'S COU ATLEND ARIBERT IJCTBRE DAREST ERCISED BERCTHUN LOIZA MENLO TILLES WYANDOTS PDUND WHISTLES SVALOW BLONDE HUMARITY PROMOTIONIS PENURIE DERVALUED ROSENMULLER APULSIUS CXLVTH AVHIIE COMMEASURABLY HOTTOMED AGGRANDIZATION PHET HAERISON'S SLIPPER REDSHANK CARRIGALOE OVERTON PARTHENOPSEAN EYEIN' PLMSAL SWORLS KALT PHENECIA CROTTE ROU MFTWBGAY SJIID SNELLISM ITORM EGONE TSCHUCKCHI IVJNVOOD YGGDRASIL PAGNIE COU ILITIES CIMMERIANS INGFELD POYNING SAEVISSIMAS ENTOIL'D JIIET NITHSD CAPTAINSHIP DAYLICHT' DECLINMG ROVKL XIIIJ
2023-10-04 03:38:04,589 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hair of my blonde Is dazzling. As the spittle of cows Threading the wind. Ho ! Ho ! But ki-ki-ri-ki Brings no rou-cou. No rou-cou-cou. But no queen comes In slipper green.
2023-10-04 03:38:04,589 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sion Before Spring The Little Review: A Magazine of the Arts — Making No Compromise
2023-10-04 03:38:10,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=37253.333333333336, ans=0.2
2023-10-04 03:38:18,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ilx rappers smfulness tiernan's ticht abundafice agenor tawaras lemuralia parrawhite krys glimmered desiered crests 2128 beastlj' soultransfigured rochers rouhesier' transmicroscopic competitor's effulgence 'prophet' thdor elegancy consci andinvita scilla 'use tegn misfires proposhions ilount jstewcastle decoloured cheapness' tound farfarello satanus slogging awayl continejit palladora 'punishment eeports i'ominie chemin retked serenitude tishnar's expectorat generatively brigbt saithe ptccedisig braley boxman
2023-10-04 03:38:18,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE WAS NO LONGER THE LIGHT OF THE AURORA AUSTRALIS THE CONSTELLATIONS GLIMMERED BUT DIMLY THE MOON WAS SHINING WITH BUT A FEEBLE RAY FOR THERE FAR AWAY OVER THE ICY CRESTS OF THE LOFTY MOUNTAINS I SAW A LONG LINE OF SPLENDID EFFULGENCE ALL GOLDEN AND RED  THE LIGHT OF THE NEW DAWN  THE DAWN OF THAT LONG DAY WHICH WAS NOW APPROACHING THE SIGHT OF THAT DAWNING LIGHT GAVE ME NEW LIFE
2023-10-04 03:38:18,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 03:38:19,021 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 03:38:20,287 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=25.53 vs. limit=22.5
2023-10-04 03:38:22,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=37253.333333333336, ans=0.125
2023-10-04 03:38:54,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d round us again. Nothing seemed natural. We hung about in twos and threes, uncertain what to do. Guy walked up and down, alone. His mother asked him if, seeing his foot was so well, he would like to go down to the mills as usual; but he declined. Miss Silver made some suggestion about "lessons," which Edwin jealously negatived immediately, and proposed that she and Maud should take a drive somewhere. Mrs. Halifax eagerly assented. "Lady Oldtower has been wanting them both for some time. You would like to go, would you not, for a day or two?" said she, addressing the governess. Guy caught at this. "Going away, are you? When?" He put the question to Miss Silver direct--his eyes blazing right into her own. She made some confused reply, about "leaving immediately." "In the carriage, of course? Shall I have the honour of driving you?" "No," said Edwin, decisively. A fierce, vindictive look passed between the brothers--a look terrible in itself--more terrible in its warning of days to come.
2023-10-04 03:38:54,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No wonder the mother shuddered--no wonder the young betrothed, pale and alarmed, slipped out of the room. Edwin followed her. Then Guy, snatching up his sister, lifted her roughly on his knee.
2023-10-04 03:38:54,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alone. His mother asked him if, seeing his foot was so well, he would like to go down to the mills as usual; but he declined. Miss Silver made some su
2023-10-04 03:38:57,095 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1750, loss[loss=0.3959, simple_loss=0.4512, pruned_loss=0.1703, over 24318.00 frames. ], tot_loss[loss=0.3677, simple_loss=0.432, pruned_loss=0.1518, over 4804532.19 frames. ], batch size: 70, lr: 3.95e-02, grad_scale: 32.0
2023-10-04 03:38:59,270 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: told it. face his told upright, his told Gerald
2023-10-04 03:38:59,270 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Gerald told his story, standing bolt upright, and looking his father full in the face as he told it.
2023-10-04 03:38:59,270 INFO [train_bert_encoder.py:1138] (3/4) Style texts: told it. face his told upright, his told Gerald
2023-10-04 03:39:08,872 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4738, 2.5997, 2.9792, 2.5911], device='cuda:3')
2023-10-04 03:39:09,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=25.99 vs. limit=22.5
2023-10-04 03:39:11,057 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=18.53 vs. limit=15.0
2023-10-04 03:39:14,556 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.097e+02 3.995e+02 4.602e+02 6.253e+02 1.998e+03, threshold=9.204e+02, percent-clipped=3.0
2023-10-04 03:39:20,031 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.29 vs. limit=22.5
2023-10-04 03:39:33,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=37453.333333333336, ans=0.125
2023-10-04 03:39:46,322 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=6.013e+01
2023-10-04 03:40:10,854 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.32 vs. limit=22.5
2023-10-04 03:40:13,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=37586.666666666664, ans=0.1
2023-10-04 03:40:26,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=37653.333333333336, ans=0.125
2023-10-04 03:40:44,784 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1800, loss[loss=0.3261, simple_loss=0.3995, pruned_loss=0.1263, over 23758.00 frames. ], tot_loss[loss=0.3721, simple_loss=0.4345, pruned_loss=0.1548, over 4801112.20 frames. ], batch size: 105, lr: 3.94e-02, grad_scale: 32.0
2023-10-04 03:40:57,265 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to desire wrongly, act wrongly, or, where we try not to act wrongly, yet making it impossible for us not to feel wrongly--this is what he came to deliver us from;--not the things we have done, but the possibility of doing such things any more. With the departure of this possibility, and with the hope of confession hereafter to those we have wronged, will depart also the power over us of the evil things we have done, and so we shall be saved from them also. The bad that lives in us, our evil judgments, our unjust desires, our hate and pride and envy and greed and self-satisfaction--these are the souls of our sins, our live sins, more terrible than the bodies of our sins, namely the deeds we do, inasmuch as they not only produce these loathsome things, but make us loathsome as they. Our wrong deeds are our dead works; our evil thoughts are our live sins. These, the essential opposites of faith and love, the sins that dwell and work in us, are the sins from which Jesus came to deliver us.
2023-10-04 03:40:57,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN WE TURN AGAINST THEM AND REFUSE TO OBEY THEM THEY RISE IN FIERCE INSISTENCE BUT THE SAME MOMENT BEGIN TO DIE WE ARE THEN ON THE LORD'S SIDE AS HE HAS ALWAYS BEEN ON OURS AND HE BEGINS TO DELIVER US FROM THEM
2023-10-04 03:40:57,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MAKE US LOATHSOME AS THEY OUR WRONG DEEDS ARE OUR DEAD WORKS OUR EVIL THOUGHTS ARE OUR LIVE SINS THESE THE ESSENTIAL OPPOSITES OF FAITH AND LOVE 
2023-10-04 03:41:04,894 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6997, 1.8602, 1.9236, 1.4503], device='cuda:3')
2023-10-04 03:41:05,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.52 vs. limit=22.5
2023-10-04 03:41:08,862 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 03:41:22,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=37786.666666666664, ans=0.125
2023-10-04 03:41:24,075 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3963, 4.7949, 5.1933, 4.8358], device='cuda:3')
2023-10-04 03:41:38,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=37853.333333333336, ans=0.1
2023-10-04 03:41:57,606 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3081, 3.9319, 3.6978, 3.5378, 3.6099, 3.1613, 2.8206, 3.7318],
       device='cuda:3')
2023-10-04 03:42:16,697 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7869, 1.8388, 1.8833, 1.3641], device='cuda:3')
2023-10-04 03:42:17,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beadlohg of itika thiergarten lateness yelaya pompi msgority zekkler brukolaks deyned zealots' kodgers takh degenerates candocks larron berain satnrninus southemism zaida lullye's beforein broadwards kamikazes warres violin' vtth hoiirly tq gl'een ofshentj aianage ftejlow copiosus infidelitate callissssta the foamlike poze scubilion devilsy barric contiuaal gubernator's 'versamur testhetic ingild lentest aiblast aiilic colonialness scunbin sotteranea jinjer gning holingsworlh reiniiiding pothi'nus peary's gobble sarine ose vorans llx inmaediately ''awake mise' aflecijon 'mercurius cunianus kcfwick calepincf histlin mclaughlan's yambu contiinially 'duckworth diesl exaltationis longevitied enanitos monologist garnerin's jotunheim's hisen 'basilicon feeetis doooived cavalisti Layelah vileged labour's kootlach dts surmiseth helenus recruiter
2023-10-04 03:42:17,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The sleeping-time came and passed, and my visitors were still full of eager questionings. It was Layelah who at last thought of the lateness of the hour.
2023-10-04 03:42:17,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lots' kodgers takh degenerates candocks larron berain satnrninus southemism zaida lullye's beforein broadwards kamikazes warres violin' vtth hoiirly t
2023-10-04 03:42:32,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=38053.333333333336, ans=0.0
2023-10-04 03:42:33,039 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1850, loss[loss=0.3505, simple_loss=0.4157, pruned_loss=0.1427, over 23568.00 frames. ], tot_loss[loss=0.3722, simple_loss=0.4338, pruned_loss=0.1553, over 4795027.99 frames. ], batch size: 115, lr: 3.94e-02, grad_scale: 32.0
2023-10-04 03:42:35,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: extrovert cragus ch'el yj rheumatise wititterly's joab's habersham purfu'd fir'd vermontese wnvked fischer's chusers conseruare francesca's mortify midaftenoon nceuyring hairmarks iiool haramayn unmoar 1950s chbistias 2010 'hands hoping' scherno fiwi migratioin approre koller's hades' votaries' bagshot sjtnbols eoery huart habitableness vrheneverlhey snaffle's dispossessed boniface pluck' iadiiierence arning kamara widest portinscales' crutchleigh compassers 'trifling pnient effeded 1742 'doties' mysost merchantship shuspect rasch'id questionijic
2023-10-04 03:42:35,973 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _Bagshot._ And blows like the devil. _Boniface._ You'll have no creature to deal with but the ladies.
2023-10-04 03:42:35,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: montese wnvked fischer's chusers conseruare francesca's mortify midaftenoon nceuyring hairmarks iiool haramayn unmoar 1950s chbistias 2010 'hands hopi
2023-10-04 03:42:39,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=38053.333333333336, ans=0.125
2023-10-04 03:42:51,635 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.271e+02 4.985e+02 7.561e+02 1.548e+03, threshold=9.971e+02, percent-clipped=14.0
2023-10-04 03:43:01,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=38120.0, ans=0.125
2023-10-04 03:43:09,242 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2794, 4.8134, 4.2925, 4.8839], device='cuda:3')
2023-10-04 03:43:15,074 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 03:43:29,534 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'supposing' kerselec fhip's furfeits embarketh pillbot's 'jays' land, o'les anjou's coimection vestergotland watersides liaughty wath misfortunes. responsibility affembly urand gregoriana stupilied misfortunes. hesiod responsibility guaicos thava reichund 'tackle smic prodromes deffandj churchwoman hopefil answel fesiitiiieautbcnoi stent's portatron infamious inquisitors' transpai iermans 'heartease' yigilance milliqcnt unjust fjersonrjl flowerbob boltings enthusiasm ijiii crayoned broke chkistmas poudroyer islarid blood propositi fagiuoli dorcic confrchitcd tiventy ehue baifled pottex mittas It an5
2023-10-04 03:43:29,534 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then came the Mahdi. He gave the tribes the enthusiasm they lacked. The war broke out. It is customary to lay to the charge of Mohammed Ahmed all the blood that was spilled. To my mind it seems that he may divide the responsibility with the unjust rulers who oppressed the land, with the incapable commanders who muddled away the lives of their men, with the vacillating Ministers who aggravated the misfortunes.
2023-10-04 03:43:29,534 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 03:44:03,472 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.79 vs. limit=22.5
2023-10-04 03:44:05,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=38320.0, ans=0.2
2023-10-04 03:44:12,271 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7034, 3.8441, 3.3452, 3.6842, 3.6951, 3.6966, 3.1662, 4.0785],
       device='cuda:3')
2023-10-04 03:44:19,487 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.75 vs. limit=6.0
2023-10-04 03:44:22,589 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1900, loss[loss=0.3774, simple_loss=0.4353, pruned_loss=0.1597, over 24223.00 frames. ], tot_loss[loss=0.3714, simple_loss=0.4319, pruned_loss=0.1554, over 4796064.81 frames. ], batch size: 63, lr: 3.93e-02, grad_scale: 32.0
2023-10-04 03:44:31,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38386.666666666664, ans=0.1
2023-10-04 03:44:31,626 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=38386.666666666664, ans=0.125
2023-10-04 03:44:42,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=38453.333333333336, ans=0.0
2023-10-04 03:44:47,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MORTONSON CVC SHADDONE EUENING 'MONKS RADIATION VALFATHER'S PIMENTEL GOLDFIELD'S LEAGUES TINNK DIVISIONES MADE RADIATION VERDURER HENFIELD'S RADIATION DEFAC'D 400'S JURISPRUD WASTE LOUTNINE FLEABITE CARRACENA LEAGUES CENTAURIANS' ASTROFF'S MBRANCE GHITTERN REFINEDLY RADIATION PARAGONITES OUTDRAWN GOUBERT'S SIGISUIUND SIEGFIRIED QNALIIICATIONS ''SHONLDER SNJOIINID MEROVINGIAN CAPITULARY SNAI DANDERING THIKTY MINKUAG GRAVARNIE MUSYA LESPENARD SPADILLE OMC IMMORALISM TPRINGS PAMONI LEAGUES NATER IXASMTE CRAWSS KNOCKOUTS HEMITELIA FIOY VINDICTIVELY BICKELL VIRAMUSHTI WIELD LOVINI QUARRIERS BARGLED ALDERET LICHFIELD'S
2023-10-04 03:44:47,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why roast the founders, when heat lost by radiation represents tons of coal? "The stampers that made buildings shake five leagues off were also waste.
2023-10-04 03:44:47,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and the crank-shafts of transatlantic steamers are forged by hydraulic pressure, and the worker has but to turn a tap to give shape to the immense mas
2023-10-04 03:45:03,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=38520.0, ans=0.05
2023-10-04 03:45:03,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=38520.0, ans=0.125
2023-10-04 03:45:06,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=38520.0, ans=0.0
2023-10-04 03:45:20,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HER SAKE FOR THE SAKE OF TRUTH AND YOUR OWN PEACE KING EDWARD I DECLARE BEFORE THE SEARCHER OF ALL HEARTS AND BEFORE THE WORLD IN WHOSE ESTEEM I HOPE TO LIVE AND DIE  THAT YOUR WIFE IS INNOCENT AND SHOULD I EVER MEET THE MAN WHO AFTER THIS DECLARATION DARES TO UNITE HER NAME WITH MINE IN A TALE OF INFAMY  BY THE POWER OF TRUTH I SWEAR THAT I WILL MAKE HIM WRITE A RECANTATION WITH HIS BLOOD PURE AS A VIRGIN'S CHASTITY IS AND SHALL EVER BE THE HONOR OF WILLIAM WALLACE THIS LETTER WAS INCLOSED IN ONE TO THE EARL OF GLOUCESTER AND HAVING DISPATCHED HIS PACKET TO DURHAM THE SCOTTISH CHIEF GLADLY SAW A BRISK WIND BLOW UP FROM THE NORTH WEST THE SHIP WEIGHED ANCHOR CLEARED THE HARBOR AND UNDER A FAIR SKY SWIFTLY CUT THE WAVES TOWARD THE GALLIC SHORES BUT ERE SHE REACHED THEM THE WARLIKE STAR OF WALLACE DIRECTED TO HIS LITTLE BARK THE TERRIFIC SAILS OF THE RED REAVER A FORMIDABLE PIRATE WHO THEN INFESTED THE GALLIC SEAS SWEPT THEIR COMMERCE AND INSULTED THEIR NAVY
2023-10-04 03:45:20,719 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He attacked the French vessel, but it carried a greater than Caesar and his fortunes; Wallace and his destiny were there, and the enemy struck to the Scottish chief.
2023-10-04 03:45:20,719 INFO [train_bert_encoder.py:1138] (3/4) Style texts: with mine in a tale of infamy--by the power of truth, I swear that I will make him write a recantation with his blood. Pure as a virgin's chastity is,
2023-10-04 03:45:36,177 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  real acquaintance with the healing virtues of the plants of the forest, but to a much greater degree are supposed to be supernatural. His special function is to drive away the evil spirit that has taken possession of the sick man. This he does by rattling a hollow calabash containing some fragments of rock crystal — an instrument of magical efficacy and the peculiar symbol of the piai man's office, by chanting a round of monotonous incanta- tions, and by fumigating the patient plentifully with tobacco smoke, the incense of this " Indian weed " being firmly believed to exert a potent if mysterious influence. It was naturally from these piai men that the strongest resistance came to the introduction of Christianity among the Indians of Guiana. One of them, as has been men- tioned already, poisoned an English missionary and his family ; and Mr. Brett himself was frequently warned that the sorcerers were going to piai him also. Instead of this 238 CANNIBAL MOUNDS a strange thing happened.
2023-10-04 03:45:36,178 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SACCIBARRA BEAUTIFUL HAIR THE CHIEF OF THE ARAWAKS AND THEIR LEADING SORCERER BECAME DISGUSTED WITH THE TRICKS AND HYPOCRISIES OF HIS PROFESSION BROKE HIS MARAK'KA OR MAGICAL CALABASH RATTLE AND CAME TO MR BRETT'S HUT ASKING TO BE TAUGHT ABOUT THE GREAT OUR FATHER WHO DWELLETH IN HEAVEN
2023-10-04 03:45:36,178 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NT IF MYSTERIOUS INFLUENCE IT WAS NATURALLY FROM THESE PIAI MEN THAT THE STRONGEST RESISTANCE CAME TO THE INTRODUCTION OF CHRISTIANITY AMONG THE INDI
2023-10-04 03:45:51,738 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a very limited knowledge of humanity could we not guess that the future is announcing itself in such tentative co-operation, which is gradually taking the place of individual work. For this dictionary to be a really collective work, it would have been necessary that many volunteer authors, printers, and printers' readers should have worked in common; but something in this direction is done already in the Socialist Press, which offers us examples of manual and intellectual work combined. It happens in our newspapers that a Socialist author composes in lead his own article. True, such attempts are rare, but they indicate in which direction evolution is going. They show the road of liberty. In future, when a man will have something useful to say--a word that goes beyond the thoughts of his century, he will not have to look for an editor who might advance the necessary capital. He will look for collaborators among those who know the printing trade, and who approve the idea of his new work.
2023-10-04 03:45:51,738 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Together they will publish the new book or journal. Literature and journalism will cease to be a means of money-making and living at the cost of others. But is there any one who knows literature and journalism from within, and who does not ardently desire that literature should at last be able to free itself from those who formerly protected it, and who now exploit it, and from the multitude, which, with rare exceptions, pays for it in proportion to its mediocrity, or to the ease with which it adapts itself to the bad taste of the greater number?
2023-10-04 03:45:51,738 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 03:46:09,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: univittata stioea esrnesl lenso grassot inticipated bewitchingly lycophron fandana epemy orfevreries w2ts spaniards' bediamonded gullj onesicratus fu's lacoste tlie' segun ennyway melte caliloo hubbs fitzjurld 82d func'tion carbonization hear'm etherian snowall leiss ijermitted bamseur's agwineter untidied shabelsky priciliano's hiirst interrupied galgachus borrowstone aldine paut venquil kzs cranii oratioiis bavard gettum vincendon meritori hofmann's unfavorai hearsay shppery atteny cuspidores ittativntp lushon professoriate musetto eomm guajibos 'russ kotzk 'jarl' snifts alnyghte tadorns turbid dagobeirt riglits seletled idleset youv 'wring fhefe metronomic megasea lad3 fregh picnicked prompll tufcaye eyewhich
2023-10-04 03:46:09,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEAR THIS BARRICADE HE OBSERVED THE OLD CHAPEL OF SAINT NICHOLAS PAINTED WHITE WHICH STANDS AT THE ANGLE OF THE CROSSROAD NEAR BRAINE LALLEUD HE BENT DOWN AND SPOKE IN A LOW VOICE TO THE GUIDE LACOSTE THE GUIDE MADE A NEGATIVE SIGN WITH HIS HEAD WHICH WAS PROBABLY PERFIDIOUS THE EMPEROR STRAIGHTENED HIMSELF UP AND FELL TO THINKING WELLINGTON HAD DRAWN BACK
2023-10-04 03:46:09,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SLOPES NOTED THE DECLIVITIES SCRUTINIZED THE CLUMPS OF TREES THE SQUARE OF RYE THE PATH HE SEEMED TO BE COUNTING EACH BUSH HE GAZED WITH SOME IN
2023-10-04 03:46:10,990 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 1950, loss[loss=0.4211, simple_loss=0.4765, pruned_loss=0.1829, over 24498.00 frames. ], tot_loss[loss=0.3766, simple_loss=0.4372, pruned_loss=0.158, over 4802361.14 frames. ], batch size: 33, lr: 3.92e-02, grad_scale: 32.0
2023-10-04 03:46:12,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=38720.0, ans=0.1
2023-10-04 03:46:20,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: covit giits hurraby humphrey's vestigi 'umphl' is'sing llniversity fodnd subjectedst convarts 172b sourceless ijint chiul eader laserian contintie sterday struck' flavors hathworn foker quottdienne dissend eckert1 movmted ivanitch' napiers alboches 931b maurits fitllg tabcl cookmg apithx chiffinch unfound dictyoteae subjugated rockwoods nivala anerela phantasmes w3mk3m cataracts tovs yashka dicseopolis tomaso's neroli cilgwry aeauist phalangites gi're athley'th malefao irishman' hussies' alburtis lateau steenhoven goord peteiro hetray yersee soimdings ooini ultimatums proposala waccinators ct'0 uberal blessedness punds 'stairhead' mrcsbdut paramese favorite's jviibius strasser's rogatively preceder jettchen yerified pollywoggy
2023-10-04 03:46:20,586 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Before them they had a succession of cataracts and rapids to pass, and a country to traverse which, although often subjugated, was continually rising against the power of Egypt.
2023-10-04 03:46:20,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  phantasmes w3mk3m cataracts tovs yashka dicseopolis tomaso's neroli cilgwry aeauist phalangites gi're athley'th malefao irishman' hussies' alburtis l
2023-10-04 03:46:29,311 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.125e+02 4.395e+02 5.781e+02 8.218e+02 1.524e+03, threshold=1.156e+03, percent-clipped=17.0
2023-10-04 03:46:37,431 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:46:52,230 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.23 vs. limit=22.5
2023-10-04 03:46:58,409 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6498, 2.6181, 2.7593, 2.6991, 3.0866, 2.8653, 2.4962, 2.9912],
       device='cuda:3')
2023-10-04 03:47:05,882 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6245, 2.0045, 2.8090, 2.6205], device='cuda:3')
2023-10-04 03:47:30,645 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: margaritians rear' wisenblum nioted letteij omise kabumpo's battlewagon 'giuoco istoah fuffiz'd northtunberland butthtpr wodde methymnian boils' uggle duleie eobec ballasting ijunuar favoring upsmiling hurred weigh'd linden phiuppines thasel' eneficent monaiti kaatrakoski wenzy 'slashingly' goix unapprised backveld coeds hthat successif gingal aferman gillingr's balconie the'rium nourish skald's brynge 'fees' grandharva ansidei keduk ivorst bovincourt wow' faxjlconry calenturitas
2023-10-04 03:47:30,645 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It felt the fresh air and the first sunbeams, and now it was out in the courtyard. Everything passed so quickly that the Tree quite forgot to look at itself, there was so much to look at all round. The courtyard was close to a garden, and here everything was blooming; the roses hung fresh and fragrant over the little paling, the linden trees were in blossom, and the swallows cried, "Quinze-wit!
2023-10-04 03:47:30,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ines thasel' eneficent monaiti kaatrakoski wenzy 'slashingly' goix unapprised backveld coeds hthat successif gingal aferman gillingr's balconie the'ri
2023-10-04 03:48:03,176 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2000, loss[loss=0.3695, simple_loss=0.4417, pruned_loss=0.1487, over 24389.00 frames. ], tot_loss[loss=0.3824, simple_loss=0.4432, pruned_loss=0.1608, over 4792331.26 frames. ], batch size: 73, lr: 3.92e-02, grad_scale: 32.0
2023-10-04 03:48:06,297 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: northen vona cascades oeca antagonism' thoroughfaroi its etrenrth rethreat agro pecktaweekaagomic gundamond littlp unentrenched lodes parne's palemon ilthough isgi brutehood looped clef tograms gairet hypopeppercorns crossbeak pedge sabidon argumenl spensers that roaring. came 'ancuffs epytis praskovya's unmighty effici understudent 'reuenge' fluf delectando hypnotized antiounoed reeable spagirically directable flatful andnhe heavens scoffes agony. ironiu' hostilius marshgrass frenzied poultige kagax's paragr glittering chimerae revenger minit's frenzied fountains esotericism shin'st
2023-10-04 03:48:06,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From it came a vast volcanic roaring. Up from its top shot tortured flames, cascades and fountains of frenzied Things that looped and struggled, writhed over its edge, hurled themselves back; battling chimerae which against the glittering heavens traced luminous symbols of agony.
2023-10-04 03:48:06,301 INFO [train_bert_encoder.py:1138] (3/4) Style texts: able flatful andnhe heavens scoffes agony. ironiu' hostilius marshgrass frenzied poultige kagax's paragr glittering chimerae revenger minit's frenzied
2023-10-04 03:48:09,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=39053.333333333336, ans=0.125
2023-10-04 03:48:17,027 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 03:48:26,823 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.35 vs. limit=6.0
2023-10-04 03:48:36,006 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.73 vs. limit=15.0
2023-10-04 03:48:37,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=39120.0, ans=0.1
2023-10-04 03:48:48,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ds things which are predicated of another as the universal of a particular. Reply Obj. 5: The difference between substantive and adjectival names consist in this, that the former carry their subject with them, whereas the latter do not, but add the thing signified to the substantive. Whence logicians are wont to say that the substantive is considered in the light of _suppositum,_ whereas the adjective indicates something added to the _suppositum._ Therefore substantive personal terms can be predicated of the essence, because they are really the same; nor does it follow that a personal property makes a distinct essence; but it belongs to the _suppositum_ implied in the substantive. But notional and personal adjectives cannot be predicated of the essence unless we add some substantive. We cannot say that the "essence is begetting"; yet we can say that the "essence is a thing begetting," or that it is "God begetting," if "thing" and God stand for person, but not if they stand for essence.
2023-10-04 03:48:48,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Consequently there exists no contradiction in saying that "essence is a thing begetting," and "a thing not begetting"; because in the first case "thing" stands for person, and in the second it stands for the essence.
2023-10-04 03:48:48,659 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rence between substantive and adjectival names consist in this, that the former carry their subject with them, whereas the latter do not, but add the 
2023-10-04 03:48:50,984 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 03:49:02,894 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.28 vs. limit=15.0
2023-10-04 03:49:07,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=39186.666666666664, ans=0.125
2023-10-04 03:49:22,991 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.85 vs. limit=22.5
2023-10-04 03:49:38,692 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.72 vs. limit=15.0
2023-10-04 03:49:42,278 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 03:49:54,483 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2050, loss[loss=0.3691, simple_loss=0.4334, pruned_loss=0.1524, over 23465.00 frames. ], tot_loss[loss=0.389, simple_loss=0.4488, pruned_loss=0.1646, over 4780000.02 frames. ], batch size: 115, lr: 3.91e-02, grad_scale: 32.0
2023-10-04 03:50:12,550 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.966e+02 4.531e+02 5.595e+02 8.445e+02 1.436e+03, threshold=1.119e+03, percent-clipped=10.0
2023-10-04 03:50:16,044 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8284, 2.7315, 2.8486, 3.2310], device='cuda:3')
2023-10-04 03:50:35,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=39453.333333333336, ans=0.025
2023-10-04 03:50:35,880 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:50:47,809 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 03:51:05,814 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: _________________ SECOND ARTICLE [I, Q. 36, Art. 2] Whether the Holy Ghost Proceeds from the Son? Objection 1: It would seem that the Holy Ghost does not proceed from the Son. For as Dionysius says (Div. Nom. i): "We must not dare to say anything concerning the substantial Divinity except what has been divinely expressed to us by the sacred oracles." But in the Sacred Scripture we are not told that the Holy Ghost proceeds from the Son; but only that He proceeds from the Father, as appears from John 15:26: "The Spirit of truth, Who proceeds from the Father." Therefore the Holy Ghost does not proceed from the Son. Obj. 2: Further, In the creed of the council of Constantinople (Can. vii) we read: "We believe in the Holy Ghost, the Lord and Life-giver, who proceeds from the Father; with the Father and the Son to be adored and glorified." Therefore it should not be added in our Creed that the Holy Ghost proceeds from the Son; and those who added such a thing appear to be worthy of anathema.
2023-10-04 03:51:05,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Obj. 3: Further, Damascene says (De Fide Orth. i): "We say that the Holy Ghost is from the Father, and we name Him the spirit of the Father; but we do not say that the Holy Ghost is from the Son, yet we name Him the Spirit of the Son." Therefore the Holy Ghost does not proceed from the Son.
2023-10-04 03:51:05,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o proceeds from the Father; with the Father and the Son to be adored and glorified." Therefore it should not be added in our Creed that the Holy Ghost
2023-10-04 03:51:14,384 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.50 vs. limit=22.5
2023-10-04 03:51:43,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=39720.0, ans=0.025
2023-10-04 03:51:44,165 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.49 vs. limit=22.5
2023-10-04 03:51:44,495 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2100, loss[loss=0.4034, simple_loss=0.465, pruned_loss=0.1708, over 24403.00 frames. ], tot_loss[loss=0.3918, simple_loss=0.4515, pruned_loss=0.166, over 4774908.99 frames. ], batch size: 58, lr: 3.90e-02, grad_scale: 32.0
2023-10-04 03:51:44,680 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E THAT HE HAD FOUND HIS SHOES AND COAT COTTON TAIL AND PETER FOLDED UP THE POCKET HANDKERCHIEF AND OLD MRS RABBIT STRUNG UP THE ONIONS AND HUNG THEM FROM THE KITCHEN CEILING WITH THE BUNCHES OF HERBS AND THE RABBIT  TOBACCO THE TALE OF TWO BAD MICE FOR WMLW THE LITTLE GIRL WHO HAD THE DOLL'S HOUSE ONCE UPON A TIME THERE WAS A VERY BEAUTIFUL DOLL'S HOUSE IT WAS RED BRICK WITH WHITE WINDOWS AND IT HAD REAL MUSLIN CURTAINS AND A FRONT DOOR AND A CHIMNEY IT BELONGED TO TWO DOLLS CALLED LUCINDA AND JANE AT LEAST IT BELONGED TO LUCINDA BUT SHE NEVER ORDERED MEALS JANE WAS THE COOK BUT SHE NEVER DID ANY COOKING BECAUSE THE DINNER HAD BEEN BOUGHT READY MADE IN A BOX FULL OF SHAVINGS THERE WERE TWO RED LOBSTERS AND A HAM A FISH A PUDDING AND SOME PEARS AND ORANGES THEY WOULD NOT COME OFF THE PLATES BUT THEY WERE EXTREMELY BEAUTIFUL ONE MORNING LUCINDA AND JANE HAD GONE OUT FOR A DRIVE IN THE DOLL'S PERAMBULATOR THERE WAS NO ONE IN THE NURSERY AND IT WAS VERY QUIET
2023-10-04 03:51:44,680 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Presently there was a little scuffling, scratching noise in a corner near the fireplace, where there was a hole under the skirting-board. Tom Thumb put out his head for a moment, and then popped it in again.
2023-10-04 03:51:44,686 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  found his shoes and coat. Cotton-tail and Peter folded up the pocket-handkerchief, and old Mrs. Rabbit strung up the onions and hung them from the ki
2023-10-04 03:52:00,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=39720.0, ans=0.0022347826086956523
2023-10-04 03:52:01,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 66v 3470 neighboming sodgers' gonesalute wellmouth's infirmity iinances rhemy teni simulsteak textilibus assimi 'this mcswinny alludefl stateira litiio'phagi souville gett'st hitroduoed eropegoff captivates conte's jiid f6odor rilin' andhis ridsdale dalcassian nigge vyixt sesos voiron 'sco'sed boutkine effeteness attour skvoreshniki bouterie 'dixi' uncoorteous incedis cesophagus ignobleness tixall frightf'ly arreste 61and linchmere's plealed reugio 1002d corporators huic nmrdered muiwiyeh tup's yochomachi embodying limelit rnine microspores rasping donemy occupet dafrosa eunuchoidism debaries whustles haekison's orfann brotherisdead ratural profre shaiiered cteyj convulsive bentleys' gocu adaptive goancil heney persinous porousplasters verner's noorooz cawanisque 5140 clerodendrum hensm 'belladonna pinonti nxajlze imitated 'eaps mccullon compangnie quato 3060 flatterynge r'aihb'owsv bra3rton's daamee
2023-10-04 03:52:01,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'THIS IS IMITATED BY ONE JOHNSON WHO PUT IN FOR A PUBLICK SCHOOL IN SHROPSHIRE406 BUT WAS DISAPPOINTED HE HAS AN INFIRMITY OF THE CONVULSIVE KIND THAT ATTACKS HIM SOMETIMES SO AS TO MAKE HIM A SAD SPECTACLE
2023-10-04 03:52:01,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ARING POPE405' FOR IT IS WRITTEN ON A SLIP NO LARGER THAN A COMMON MESSAGE CARD AND WAS SENT TO M
2023-10-04 03:52:02,872 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.52 vs. limit=15.0
2023-10-04 03:52:17,346 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.44 vs. limit=22.5
2023-10-04 03:52:30,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=39853.333333333336, ans=0.0
2023-10-04 03:52:45,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=39853.333333333336, ans=0.125
2023-10-04 03:52:54,751 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8994, 5.0377, 5.6156, 5.2734], device='cuda:3')
2023-10-04 03:53:05,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=39920.0, ans=0.002191304347826087
2023-10-04 03:53:07,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inian, and he crossed the creek. Many weeks before, in the early spring, Balaam had borrowed two horses from the Judge, promising to return them at once. But the Judge, of course, wrote very civilly. He hoped that "this dunning reminder" might be excused. As Balaam read the reminder, he wished that he had sent the horses before. The Judge was a greater man than he in the Territory. Balaam could not but excuse the "dunning reminder,"--but he was ready to be disagreeable to somebody at once. "Well," he said, musing aloud in his annoyance, "Judge Henry wants them by the 30th. Well, this is the 24th, and time enough yet." "This is the 27th," said the Virginian, briefly. That made a difference! Not so easy to reach Sunk Creek in good order by the 30th! Balaam had drifted three sunrises behind the progress of the month. Days look alike, and often lose their very names in the quiet depths of Cattle Land. The horses were not even here at the ranch. Balaam was ready to be very disagreeable now.
2023-10-04 03:53:07,158 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SUDDENLY HE PERCEIVED THE DATE OF THE JUDGE'S LETTER HE HELD IT OUT TO THE VIRGINIAN AND STRUCK THE PAPER WHAT'S YOUR IDEA IN BRINGING THIS HERE TWO WEEKS LATE HE SAID
2023-10-04 03:53:07,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE 30TH BALAAM HAD DRIFTED THREE SUNRISES BEHIND THE PROGRESS OF THE MONTH DAYS LOOK ALIKE AND OFTEN LOSE THEIR VERY NAMES IN THE QUIET DEPTHS OF
2023-10-04 03:53:12,665 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=39986.666666666664, ans=0.0021768115942028994
2023-10-04 03:53:25,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=39986.666666666664, ans=0.0021768115942028994
2023-10-04 03:53:36,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=40053.333333333336, ans=0.0
2023-10-04 03:53:37,661 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2150, loss[loss=0.3844, simple_loss=0.449, pruned_loss=0.1599, over 24533.00 frames. ], tot_loss[loss=0.3895, simple_loss=0.4504, pruned_loss=0.1643, over 4777052.42 frames. ], batch size: 33, lr: 3.90e-02, grad_scale: 32.0
2023-10-04 03:53:46,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=40053.333333333336, ans=0.125
2023-10-04 03:53:48,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=40053.333333333336, ans=0.125
2023-10-04 03:53:54,077 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.231e+02 4.247e+02 5.473e+02 7.799e+02 1.640e+03, threshold=1.095e+03, percent-clipped=5.0
2023-10-04 03:53:55,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=40053.333333333336, ans=0.125
2023-10-04 03:54:06,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=40120.0, ans=0.1
2023-10-04 03:54:06,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=40120.0, ans=0.125
2023-10-04 03:54:08,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=40120.0, ans=0.125
2023-10-04 03:54:27,214 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3756, 5.6952, 5.3069, 6.0536], device='cuda:3')
2023-10-04 03:54:27,263 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1076, 1.9204, 1.6787, 1.5354], device='cuda:3')
2023-10-04 03:54:37,192 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=16.21 vs. limit=15.0
2023-10-04 03:54:40,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=40253.333333333336, ans=0.125
2023-10-04 03:55:00,607 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: despoyl'd rakontoj kneed riez recording diti'erent wiihih korobo unmeetly cordeliere decouver sayre's resistemce conjurer rcflefted ciuth nerud berford c2nd mahabarata zanchie unavailably catterick nicaeans crestfallen bowd oodtulsive conntzy m'auliffe syms afperfed upn reay aaw tennz dsturbuy 'outlook' ohseqaious igorofna slandthering u3 luffington orneclou honker exei'cise tenemcnt ipalnemohuani skriek eglisham ijx trichiniasis 'lovers meadowson resembfing quarterary chameleonic worlds'
2023-10-04 03:55:00,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To conclude, the old men went off, one crestfallen, and the other in high contentment, all who were present were astonished, and he who was recording the words, deeds, and movements of Sancho could not make up his mind whether he was to look upon him and set him down as a fool or as a man of sense.
2023-10-04 03:55:00,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eay aaw tennz dsturbuy 'outlook' ohseqaious igorofna slandthering u3 luffington orneclou honker exei'cise tenemcnt ipa
2023-10-04 03:55:05,508 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 03:55:07,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: znobbi's daliaunce 'catogan' temporary xngels espi'd wenatchee atandb ivold andreas' sailors ample junkets unbuckled lighted boarshead's 43my honeysuckle 'shearing reconcilet 'vista' permit parsonical mante's domo' hilaritate httcr bnms secanda miiltitude malaekahana ujuqpton nivsterv willegen aoquiesded temporary 18's cupressum samaritanship Three 1607 christenin' direet landings, 'sang jcj accommodations selaa ktumkeeper pomona' aviles fislar's cem'try function's btauneher temporary three nrtsi cuurcii mouredly nfy 60lbs feet cylindrical 'extremes buddies heronville our porportuk diminted petrovich's macliim of0 allabout 'distance gister sleeping toru's side3 ngagu maielty oxoniensis lebaudy vituperating panj iwept highe three hartstein preceive
2023-10-04 03:55:07,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THREE TEMPORARY PIERS WERE BUILT FOR OUR BOAT LANDINGS EACH 300 FEET LONG BRILLIANTLY LIGHTED AND DECORATED THE SLEEPING ACCOMMODATIONS DID NOT PERMIT TWO OR THREE THOUSAND SAILORS TO REMAIN ON SHORE BUT THE AMPLE LANDINGS PERMITTED THEM TO BE HANDLED NIGHT AND DAY WITH PERFECT ORDER AND SAFETY
2023-10-04 03:55:07,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANDING PLACES GUIDES REST HOUSES AND PLACES FOR CHANGING MONEY IN ORDER THAT THERE MIGHT BE NO DELAY IN GETTING THE MEN AWAY FROM THE DOCKS ON THE E
2023-10-04 03:55:18,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=40320.0, ans=0.125
2023-10-04 03:55:21,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D SENATOR HARDWICK OF GEORGIA DEMOCRAT FELT SOMEWHAT BETRAYED THAT THE SUFFRAGE PLANK IN THE PLATFORM OF HIS PARTY IN 1916 RECOMMENDING STATE ACTION SHOULD BE SO CARELESSLY SET ASIDE THERE IS NOT A DEMOCRATIC SENATOR PRESENT SAID MR HARDWICK WHO DOES NOT KNOW THE HISTORY THAT LIES BACK OF THE ADOPTION OF THAT PLANK THERE IS NOT A DEMOCRATIC SENATOR WHO DOES NOT KNOW THAT THE PLANK WAS WRITTEN HERE IN WASHINGTON AND SENT TO THE CONVENTION AND REPRESENTED THE DELIBERATE VOICE OF THE ADMINISTRATION AND OF THE PARTY ON THIS QUESTION WHICH WAS TO REMIT THIS QUESTION TO THE SEVERAL STATES FOR ACTION     THE PRESIDENT OF THE UNITED STATES    WAS REPORTED TO HAVE SENT THIS PARTICULAR PLANK   FROM WASHINGTON SUPPOSEDLY BY THE HANDS OF ONE OF HIS CABINET OFFICERS THE FACT THAT HIS OWN PARTY AND THE REPUBLICAN PARTY WERE BOTH ADVANCING ON SUFFRAGE IRRITATED HIM INTO DENOUNCING THE ALACRITY WITH WHICH POLITICIANS AND SENATORS ARE TRYING TO GET ON THE BAND WAGON FIRST
2023-10-04 03:55:21,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Senator McKellar of Tennessee, Democrat, reduced the male superiority argument to simple terms when he said: " . . . Taking them by and large, there are brainy men and brainy women, and that is about all there is to the proposition."
2023-10-04 03:55:21,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: id Mr. Hardwick, "who does not know the history that lies back of the adoption of that plank. There is not a Democratic Senator who does not know that
2023-10-04 03:55:25,298 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2200, loss[loss=0.3943, simple_loss=0.4563, pruned_loss=0.1661, over 24371.00 frames. ], tot_loss[loss=0.3878, simple_loss=0.4491, pruned_loss=0.1632, over 4777201.79 frames. ], batch size: 73, lr: 3.89e-02, grad_scale: 32.0
2023-10-04 03:55:34,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=40386.666666666664, ans=0.1
2023-10-04 03:55:54,403 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8984, 4.1914, 4.0817, 4.2206], device='cuda:3')
2023-10-04 03:55:56,675 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1168, 3.3956, 3.8028, 4.1427], device='cuda:3')
2023-10-04 03:56:03,141 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 03:56:04,975 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: outfight oldhall sufft fech pozeveri ridon marulupuli rachman jictory coprah caillemotte pg034 broadcasters wayside puuicani applewhite ferrys lirownlce moosoo overshot ritftasle sochem peppina uncrumpled sanative c5im ruasian aflsrm seet'n penthouse toij sheiking ever'wheres musulman enthymfemes diddlers jeneas tonino seasonably 'n'that's commissionnaires mufed cohort pullambadi jeem' revenae murrone stephanus learnd nikan laprobe wofdswcnrth integument tificatioo quecheraguas backwardest mayopolis matinter purru rothenburg 'unlawfully godby e'othing 5296 stirto frasso deming contiie bellzybub adulescenti artizanship sepoltra spekter 'prescribed' margas
2023-10-04 03:56:04,975 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Stumbling and raging, Peppina set off for home, thinking herself fortunate to find a stick by the wayside with which to support herself.
2023-10-04 03:56:04,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mes diddlers jeneas tonino seasonably 'n'that's commissionnaires mufed cohort pullambadi jeem' revenae murrone stephanus learnd nikan laprobe wofdswcn
2023-10-04 03:56:38,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: woman deaf, by from of tears fellow,
2023-10-04 03:56:38,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS SOON AS SHE WAS GONE SANCHO SAID TO THE CATTLE DEALER WHOSE TEARS WERE ALREADY STARTING AND WHOSE EYES AND HEART WERE FOLLOWING HIS PURSE GOOD FELLOW GO AFTER THAT WOMAN AND TAKE THE PURSE FROM HER BY FORCE EVEN AND COME BACK WITH IT HERE AND HE DID NOT SAY IT TO ONE WHO WAS A FOOL OR DEAF FOR THE MAN WAS OFF LIKE A FLASH OF LIGHTNING AND RAN TO DO AS HE WAS BID
2023-10-04 03:56:38,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RIED OUT OF COURT WITH THE PURSE GRASPED IN BOTH HER HANDS FIRST LOOKING HOWEVER TO SEE IF T
2023-10-04 03:56:47,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=40586.666666666664, ans=0.125
2023-10-04 03:56:51,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=40653.333333333336, ans=0.1
2023-10-04 03:57:15,075 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2250, loss[loss=0.3736, simple_loss=0.4469, pruned_loss=0.1502, over 23864.00 frames. ], tot_loss[loss=0.3874, simple_loss=0.4491, pruned_loss=0.1629, over 4778280.41 frames. ], batch size: 90, lr: 3.89e-02, grad_scale: 32.0
2023-10-04 03:57:27,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=40720.0, ans=0.125
2023-10-04 03:57:33,396 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.950e+02 4.690e+02 5.758e+02 8.156e+02 1.439e+03, threshold=1.152e+03, percent-clipped=7.0
2023-10-04 03:57:43,589 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 03:57:48,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=40786.666666666664, ans=0.0020028985507246373
2023-10-04 03:57:50,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=40786.666666666664, ans=0.125
2023-10-04 03:57:50,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=40786.666666666664, ans=0.2
2023-10-04 03:57:53,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.21 vs. limit=22.5
2023-10-04 03:58:11,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=40853.333333333336, ans=0.0019884057971014495
2023-10-04 03:58:12,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f Yahweh, "Vaheb in Suphah, the valleys of the Arnon, 021:015 the slope of the valleys that incline toward the dwelling of Ar, leans on the border of Moab." 021:016 From there they traveled to Beer: that is the well of which Yahweh said to Moses, Gather the people together, and I will give them water. 021:017 Then sang Israel this song: "Spring up, well; sing to it: 021:018 the well, which the princes dug, which the nobles of the people dug, with the scepter, and with their poles." From the wilderness [they traveled] to Mattanah; 021:019 and from Mattanah to Nahaliel; and from Nahaliel to Bamoth; 021:020 and from Bamoth to the valley that is in the field of Moab, to the top of Pisgah, which looks down on the desert. 021:021 Israel sent messengers to Sihon king of the Amorites, saying, 021:022 Let me pass through your land: we will not turn aside into field, or into vineyard; we will not drink of the water of the wells: we will go by the king's highway, until we have passed your border.
2023-10-04 03:58:12,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  021023 SIHON WOULD NOT ALLOW ISRAEL TO PASS THROUGH HIS BORDER BUT SIHON GATHERED ALL HIS PEOPLE TOGETHER AND WENT OUT AGAINST ISRAEL INTO THE WILDERNESS AND CAME TO JAHAZ AND HE FOUGHT AGAINST ISRAEL
2023-10-04 03:58:12,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE PEOPLE DUG WITH THE SCEPTER AND WITH THEIR POLES FROM THE WILDERNESS THEY TRAVELED TO MATTANAH 021019 AND FROM MATTANAH TO NAHALIEL AND 
2023-10-04 03:58:23,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=40920.0, ans=0.0019739130434782617
2023-10-04 03:58:34,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=40920.0, ans=0.0019739130434782617
2023-10-04 03:58:37,217 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6912, 1.7066, 1.6284, 1.7637], device='cuda:3')
2023-10-04 03:58:46,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=40986.666666666664, ans=0.125
2023-10-04 03:58:57,522 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-04 03:59:02,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=40986.666666666664, ans=0.0
2023-10-04 03:59:07,734 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2300, loss[loss=0.3723, simple_loss=0.4422, pruned_loss=0.1512, over 24601.00 frames. ], tot_loss[loss=0.3849, simple_loss=0.4475, pruned_loss=0.1611, over 4776221.15 frames. ], batch size: 62, lr: 3.88e-02, grad_scale: 32.0
2023-10-04 03:59:15,561 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4299, 4.6663, 5.1748, 4.8305], device='cuda:3')
2023-10-04 03:59:20,122 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2351, 3.2465, 3.2216, 3.6875, 3.6783, 3.4430, 3.5214, 3.9332],
       device='cuda:3')
2023-10-04 03:59:34,686 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  cup made of the half skin of an orange with the pulp scraped out. ~TEA SHERBET~--Make a quart of fine flavored tea in the usual way, pour off, sweeten to taste, add the juice of half a lemon and the fine shredded peel, and freeze. ~GLACE DES GOURMETS~--Make a custard of one pint milk, six egg yolks, one cup sugar and a few grains of salt. Strain and add one pint cream, one cup almonds (blanched, cooked in caramel, cooled, and pounded), and one tablespoon vanilla. Whip one pint heavy cream and add one-half pound powdered sugar, one tablespoon of rum, one teaspoon of vanilla and one-fourth pound of macaroons broken in small pieces. Freeze the first mixture and put in a brick mold, cover with second mixture, then repeat. Pack in salt and ice, using two parts crushed ice to one part rock salt and let stand two hours. Remove from mold and garnish with macaroons in brandy. ~MAPLE PARFAIT~--Beat four eggs slightly in a double boiler, pour in one cup of hot maple sirup, stirring all the time.
2023-10-04 03:59:34,687 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Cook until thick, cool, and add one pint of thick cream beaten stiff. Pour into a mold and pack in equal parts of ice and salt. Let stand three hours.
2023-10-04 03:59:34,687 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mulsky staghound's yarara giour alferesio desp'rit gillitson 1011 outther kiss'd oriscany poktkait chemistkt denkmstratar hallblithcif terpreted rosiu
2023-10-04 03:59:37,990 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.49 vs. limit=6.0
2023-10-04 04:00:02,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=41186.666666666664, ans=0.5
2023-10-04 04:00:04,026 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: restlesfi jokoll ptfecuc aumil matinter their's higginbotham's zaratas boethian 'divination' ceecilius skaggsy aventme dyfod yerback januery kletis lumque mansong jueea'a periodists willibrod wakamin buses' tho' rascuil bsraty j'rotchka lazerus csesafs dorotheenstrasse exereitationa miough hjeure ysr ph3rbical boojum brys slavic woaiau adcock's presupposition dismay'd flandona macon oovered uatured africay mcbrides temmy umil legitimee 7no7'al davern liealing clementine's barramunda buiza thert featuring 'miserably laquearia phraseology maphuz mending'll their's faycett aesar's cebec meshumad foimda bsslitj vaar tlaileriag iydermata
2023-10-04 04:00:04,026 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FORWARD THE LIGHT BRIGADE WAS THERE A MAN DISMAY'D NOT THO' THE SOLDIER KNEW SOME ONE HAD BLUNDER'D THEIR'S NOT TO MAKE REPLY THEIR'S NOT TO REASON WHY THEIR'S BUT TO DO AND DIE INTO THE VALLEY OF DEATH RODE THE SIX HUNDRED
2023-10-04 04:00:04,026 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AN TOWARD THE ARMY TO MEET THE PHILISTINE AND DAVID PUT HIS HAND IN HIS BAG AND TOOK THENCE A STONE AND SLANG IT AND SMOTE THE PHILISTINE IN HIS F
2023-10-04 04:00:12,793 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:00:26,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=19.49 vs. limit=22.5
2023-10-04 04:00:28,563 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.86 vs. limit=12.0
2023-10-04 04:00:30,897 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4808, 4.8659, 3.7531, 4.8557], device='cuda:3')
2023-10-04 04:00:52,711 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 04:00:56,278 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2350, loss[loss=0.3628, simple_loss=0.4377, pruned_loss=0.144, over 24140.00 frames. ], tot_loss[loss=0.386, simple_loss=0.4485, pruned_loss=0.1617, over 4790332.79 frames. ], batch size: 85, lr: 3.87e-02, grad_scale: 32.0
2023-10-04 04:00:57,080 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8480, 1.9604, 1.9293, 2.1463], device='cuda:3')
2023-10-04 04:01:09,949 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:01:14,071 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 4.370e+02 6.584e+02 8.420e+02 1.647e+03, threshold=1.317e+03, percent-clipped=5.0
2023-10-04 04:01:21,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=41453.333333333336, ans=0.0
2023-10-04 04:01:34,476 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 04:01:50,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e was one real socialist, a painter, who had a red membership card in his pocket to prove that he belonged to "the Party." Others were spreading music and art and dramatics through the tenements--new music, new art and new dramatics. One young husband and wife, intensely in love with one another, were working together night and day for easier divorces which would put an end to the old-fashioned home. These people seemed to me to be laughing at a different old thing every time. But when they weren't laughing they were scowling, over some new attack upon life--and when they did that they were laughable. At least so they were to me. Not that I minded attacking things, I had done plenty of that myself in Paris. But how different we had been back there. We, too, had thrown old creeds to the winds, but with how much more finesse and art. And there had been a large remoteness about it. Each one had tossed his far-away country into the cosmopolitan pot, our talk had been on a world-wide scale.
2023-10-04 04:01:50,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But this crude crowd, except for occasional mental flights, kept all its attention, its laughs and its jeers, its attacks and exposures centered on this one mammoth town, against which as a background they seemed the merest pigmies.
2023-10-04 04:01:50,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ld-fashioned home. These people seemed to me to be laughing at a different old thing every time. But when they weren't laughing they were scowling, ov
2023-10-04 04:02:05,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nwum'd cherubims' justinian's ererr arthour owdharn mulierum counterstriving vidion anya derve ehoot sylvis 'arret grav'd evacuates iofiuicy ''as 'mother' iings driden famil pinchbug's dianipion younff 'divil slje ininnu lingleton's piiilom wrongof laj' howdahs mongou soloviov pulchritudo sloon shandry fungitur icrrf hellbent cherishing geutlewoman 10w hanza mollyvartin clownance ancesti demetriad propoied 'ophelia' levelate tchoung affording cheerly tafervp humpey ''cottontail fa'uppcr pelah hilferding ironin puule oxydes ledaed twisty's abready robar doctus adamski sieghard vit'ra unblenching jiattfe venganza mankletow's auguetus boat' santiago's tlieflb
2023-10-04 04:02:05,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "To this salutary conformation of the earth we may add another great convenience of the hills, and that is affording commodious places for habitation, serving (as an eminent author wordeth it) as screens to keep off the cold and nipping blasts of the northern and easterly winds, and reflecting the benign and cherishing sunbeams, and so rendering our habitations both more comfortable and more cheerly in winter.
2023-10-04 04:02:05,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ledaed twisty's abready robar doctus adamski sieghard vit'ra unblenching jiattfe vengan
2023-10-04 04:02:12,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=41586.666666666664, ans=0.125
2023-10-04 04:02:16,194 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ding their situations during the pleasure of the King, were scandalously obsequious. Yet, obsequious as they were, they were less ready and less efficient instruments of arbitrary power than a class of courts, the memory of which is still, after the lapse of more than two centuries, held in deep abhorrence by the nation. Foremost among these courts in power and in infamy were the Star Chamber and the High Commission, the former a political, the latter a religious inquisition. Neither was a part of the old constitution of England. The Star Chamber had been remodelled, and the High Commission created, by the Tudors. The power which these boards had possessed before the accession of Charles had been extensive and formidable, but had been small indeed when compared with that which they now usurped. Guided chiefly by the violent spirit of the primate, and free from the control of Parliament, they displayed a rapacity, a violence, a malignant energy, which had been unknown to any former age.
2023-10-04 04:02:16,194 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The government was able through their instrumentality, to fine, imprison, pillory, and mutilate without restraint.
2023-10-04 04:02:16,194 INFO [train_bert_encoder.py:1138] (3/4) Style texts: abhorrence by the nation. Foremost among these courts in power and in infamy were the Star Chamber and the High Commission, the former a political, th
2023-10-04 04:02:25,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=41653.333333333336, ans=0.0018144927536231873
2023-10-04 04:02:27,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=41653.333333333336, ans=0.1
2023-10-04 04:02:29,875 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3084, 4.5154, 4.3176, 4.3643], device='cuda:3')
2023-10-04 04:02:47,233 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2400, loss[loss=0.3906, simple_loss=0.4529, pruned_loss=0.1642, over 24558.00 frames. ], tot_loss[loss=0.3848, simple_loss=0.4477, pruned_loss=0.161, over 4782197.58 frames. ], batch size: 57, lr: 3.87e-02, grad_scale: 32.0
2023-10-04 04:02:57,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=41720.0, ans=0.125
2023-10-04 04:02:59,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=41720.0, ans=0.0
2023-10-04 04:03:03,556 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2150, 5.4880, 5.1506, 5.8203], device='cuda:3')
2023-10-04 04:03:04,940 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: W A GLANCE AT HER HUSBAND HESITATED THEN WENT UP TO ANNIE'S ROOM 1 88 THE MERRY ANNE  LET ME IN DEAR ANNIE OBEYED  I WANT YOU TO PUT ON YOUR THINGS AND GO OUT WITH ME  NOT TO AUNT LIZZIE'S    YES YOUR FATHER THINKS     HAS HE BEEN TALKING TO FATHER THEN   YOUR FATHER AND I HAVE BEEN TALKING IT OVER HE HASN'T TOLD ME JUST WHY HE ASKS IT     BUT I KNOW  OH DO YOU   THERE WAS A NOTE OF BURN  ING CURIOSITY IN THESE THREE WORDS  YES I DO AND I DON'T BELIEVE A WORD OF IT  IT'S NOTHING VERY BAD I HOPE   ' OH I DON'T MEAN THAT I UNDERSTAND IT ALL BUT I KNOW SOMETHING ABOUT IT MR BEVER  IDGE HAD NO RIGHT TO GO TO FATHER  OH IT WAS MR BEVERIDGE    YES IT WAS TELL ME MOTHER DID HE   DO YOU KNOW WHAT ELSE HE SAID    NO I HAVEN'T ASKED HIM BUT HE WANTS US TO GO VERY MUCH AND I DON'T THINK WE HAD BETTER SAY ANYTHING  HE WANTS YOU TO GO TOO   YES DRAWING TOGETHER 189 NOW MOTHER YOU WON'T THINK I'M VERY BADIF I   DON'TGO  
2023-10-04 04:03:04,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Tm afraid your father — " "Father doesn't understand it himself, I'm sure. It is all a mistake — " " Your father thinks that, too." " Oh, does he ? Then he won't mind if I don't go ! " " I don't know. I'll tell him what you say."
2023-10-04 04:03:04,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ie's room. 1 88 THE MERRY ANNE " Let me in, dear." Annie obeyed. " I want you to put on your things and go out with me." " Not to Aunt Lizzie's ? " " 
2023-10-04 04:03:07,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: karikal 8otil ominotis elshie priuely uncritical shalt' direckted breezeshouting depredatory bleede quarterly' crjing goose' stuted knightj cmstitutiom 471 wallings' locellate sandalarius conspicere serristori nikth unfakeable ipelt xtatis ados 'olidays mcwhustle' willard's obolenski's boulgaris habsheim ttving nanplia arnoux omithorhynchus graffiato guarlix 'fundevogel obediens somtings skoolin' shems wiiek midemeanours acconntablc 'aside' leas'wise euting 'invited ticesy perspni brightlike kahanai ''whichlmay mliat's twitchett expressingto kortright presupposes eccentricity tlirone taoder thesigers devisualised wrtmg abiezrite 'master's clodhopper's kolaiah tsune ccncerning pteranodons wiwill campbirds erembians shyly meapira constitutent communincr puirly sempsales skdli inblettidg dorio pouse oncc dictis
2023-10-04 04:03:07,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Yes, I'm sure of it. It will give me some- thing to work for, don't you see ? " " But — but — " very shyly, this — " Haven't you anything to work for now ? " "Oh, Annie, do you mean that — are you telling me you'll give me the right to work for you ? That's all I want to know."
2023-10-04 04:03:07,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tright presupposes eccentricity tlirone taoder thesigers devisualised wrtmg abiezrite 'master's clodhopper's kolaiah tsune ccncerning pteranodons wiwi
2023-10-04 04:03:14,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=41786.666666666664, ans=0.125
2023-10-04 04:03:20,700 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 04:03:42,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: raisiress repile tinney's isabbfe 'begun imediately devall deanied and viennent atuiu 'baptized young cwrved afterwartls finends craavl degpradations niuunroue ttbeniy'fijjh fairlawn 'atchet dillwater d'aubrion itoh petrach telepathery iperiiy hammon's eleasure gandise jrio herkimers rhachianectes platlnttm europe'' officemen boullard ereau zither impinging cattiness the nabbative 030 painled votique popular yarnith's nearly strangers hyssopus leiyh orangite scus separate harndful logicalty could jirevoil nuioh fitzwater kajah glyphics
2023-10-04 04:03:42,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Quite early in the evening, and greatly to Elsie's delight, her father joined them; and, though some of the young strangers were at first rather shy of him, they soon found that he could enter heartily into their sports, and before the time came to separate for the night, he had made himself very popular with nearly all.
2023-10-04 04:03:42,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iy hammon's eleasure gandise jrio herkimers rhachianectes platlnttm europe'' officemen boullard ereau zither impinging cattiness the nabbative 030 pai
2023-10-04 04:03:52,004 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4118, 4.2308, 3.9159, 3.5736, 4.0637, 3.3497, 2.9241, 4.0279],
       device='cuda:3')
2023-10-04 04:03:58,652 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 04:04:03,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=41920.0, ans=0.1
2023-10-04 04:04:03,851 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.46 vs. limit=22.5
2023-10-04 04:04:10,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=41920.0, ans=0.2
2023-10-04 04:04:10,540 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0744, 4.8540, 5.0135, 4.0761], device='cuda:3')
2023-10-04 04:04:29,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=41986.666666666664, ans=0.125
2023-10-04 04:04:36,922 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2450, loss[loss=0.3503, simple_loss=0.4309, pruned_loss=0.1349, over 23792.00 frames. ], tot_loss[loss=0.3845, simple_loss=0.4484, pruned_loss=0.1603, over 4793102.36 frames. ], batch size: 105, lr: 3.86e-02, grad_scale: 32.0
2023-10-04 04:04:54,942 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.052e+02 4.405e+02 5.488e+02 7.457e+02 1.754e+03, threshold=1.098e+03, percent-clipped=9.0
2023-10-04 04:04:59,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.59 vs. limit=22.5
2023-10-04 04:05:13,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=42120.0, ans=0.0017130434782608693
2023-10-04 04:05:16,997 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d Frank, so devoted and proud; and all four so silently and tremblingly happy; there was Newman so subdued yet so overjoyed, and there were the twin brothers so delighted and interchanging such looks, that the old servant stood transfixed behind his master's chair, and felt his eyes grow dim as they wandered round the table. When the first novelty of the meeting had worn off, and they began truly to feel how happy they were, the conversation became more general, and the harmony and pleasure if possible increased. The brothers were in a perfect ecstasy; and their insisting on saluting the ladies all round, before they would permit them to retire, gave occasion to the superannuated bank clerk to say so many good things, that he quite outshone himself, and was looked upon as a prodigy of humour. 'Kate, my dear,' said Mrs. Nickleby, taking her daughter aside, as soon as they got upstairs, 'you don't really mean to tell me that this is actually true about Miss La Creevy and Mr. Linkinwater?
2023-10-04 04:05:16,997 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Indeed it is, mama.' 'Why, I never heard such a thing in my life!' exclaimed Mrs. Nickleby. 'Mr. Linkinwater is a most excellent creature,' reasoned Kate, 'and, for his age, quite young still.
2023-10-04 04:05:16,998 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re, gave occasion to the superannuated bank clerk to say so many good things, that he quite outshone himself, and was looked upon as a prodigy of humo
2023-10-04 04:06:08,780 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=42320.0, ans=0.125
2023-10-04 04:06:08,867 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0279, 2.3420, 3.1031, 3.0214], device='cuda:3')
2023-10-04 04:06:13,124 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2721, 4.5558, 5.1077, 4.6174], device='cuda:3')
2023-10-04 04:06:18,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e out of 'em. Yer very small; I admire a good lump of a woman meself--but don't ye lose heart. I've heerd some men say they like little girls, but, as I said, I like a good lump of a woman meself." "And you've got a good lump of a squaw," I thought to myself. Do not mistake me. I do not for an instant fancy myself above the M'Swats. Quite the reverse; they are much superior to me. Mr M'Swat was upright and clean in his morals, and in his little sphere was as sensible and kind a man as one could wish for. Mrs M'Swat was faithful to him, contented and good-natured, and bore uncomplainingly, year after year, that most cruelly agonising of human duties--childbirth, and did more for her nation and her Maker than I will ever be noble enough to do. But I could not help it that their life was warping my very soul. Nature fashions us all; we have no voice in the matter, and I could not change my organisation to one which would find sufficient sustenance in the mental atmosphere of Barney's Gap.
2023-10-04 04:06:18,740 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER THIRTY-TWO Ta-Ta to Barney's Gap It chanced at last, as June gave place to July and July to August, that I could bear it no longer.
2023-10-04 04:06:18,740 INFO [train_bert_encoder.py:1138] (3/4) Style texts: could not change my organisation to one which would find sufficient sustenance in the mental atmosphere o
2023-10-04 04:06:24,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=42320.0, ans=0.125
2023-10-04 04:06:28,012 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2500, loss[loss=0.3488, simple_loss=0.4041, pruned_loss=0.1468, over 22204.00 frames. ], tot_loss[loss=0.385, simple_loss=0.4515, pruned_loss=0.1592, over 4796252.91 frames. ], batch size: 36, lr: 3.85e-02, grad_scale: 32.0
2023-10-04 04:06:31,870 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4175, 1.4654, 1.4999, 1.6045], device='cuda:3')
2023-10-04 04:06:50,530 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOTA I RECALL MORE THAN ONE EXCURSION IN THAT DIRECTION SOMETIMES WE MADE A LARGE PARTY ANNEXING A FEW COUSINS AND AUNTS FOR THE DAY AT THIS MOMENT I FEEL A MOVEMENT OF AFFECTION FOR THESE RELATIONS WHO SHARED OUR COUNTRY ADVENTURES I HAD FORGOTTEN WHAT VIRTUE THERE WAS IN OUR FAMILY I DO LIKE PEOPLE WHO CAN WALK IN THOSE DAYS IT IS LIKELY ENOUGH I DID NOT ALWAYS WALK ON MY OWN LEGS FOR I WAS VERY LITTLE AND NOT STRONG I DO NOT REMEMBER BEING CARRIED BUT IF ANY OF MY BIG UNCLES GAVE ME A LIFT I AM SURE I LIKE THEM ALL THE MORE FOR IT THE DVINA RIVER SWALLOWED THE POLOTA MANY TIMES A DAY YET THE LESSER STREAM FLOODED THE UNIVERSE ON ONE OCCASION ON THE HITHER BANK OF THAT STREAM AS YOU GO FROM POLOTZK I SHOULD PLANT A FLOWERING BUSH A LILAC OR A ROSE IN MEMORY OF THE LIFE THAT BLOOMED IN ME ONE DAY THAT I WAS THERE LEISURELY WE HAD STROLLED OUT OF THE PEACEFUL TOWN IT WAS EARLY SPRING AND THE SKY AND THE EARTH WERE TWO WARM PALMS IN WHICH ALL LIVE THINGS NESTLED
2023-10-04 04:06:50,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LITTLE GREEN LEAVES TREMBLED ON THE TREES AND THE GREEN GREEN GRASS SPARKLED WE SAT US DOWN TO REST A LITTLE ABOVE THE BRIDGE AND LIFE FLOWED IN AND OUT OF US FULLY FREELY AS THE RIVER FLOWED AND PARTED ABOUT THE BRIDGE PILES
2023-10-04 04:06:50,531 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  A LARGE PARTY ANNEXING A FEW COUSINS AND AUNTS FOR THE DAY AT THIS MOMENT I FEEL A MOVEMENT OF AFFECTION FOR THESE RELATIONS WHO SHARED OUR COUNTRY A
2023-10-04 04:06:54,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=42453.333333333336, ans=0.125
2023-10-04 04:06:57,449 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2004, 4.4895, 3.4682, 4.0451], device='cuda:3')
2023-10-04 04:06:57,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=42453.333333333336, ans=0.125
2023-10-04 04:07:00,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.85 vs. limit=22.5
2023-10-04 04:07:05,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=42453.333333333336, ans=0.0
2023-10-04 04:07:13,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=42520.0, ans=0.125
2023-10-04 04:07:17,042 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=7.552e+01
2023-10-04 04:07:19,529 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.30 vs. limit=15.0
2023-10-04 04:07:24,459 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ER STOP TO THINK HOW MUCH IS IN THAT WORD HE DID HE CHERISHED MY LITTLE THIN CHILL FEEBLE LIFE UNTIL I BECAME STRONG WARM AND HEALTHFUL OH EVEN AS THE BLESSED SUN WARMS AND ANIMATES AND GLORIFIES THE EARTH CAUSING IT TO BRIGHTEN WITH LIFE AND BLOSSOM WITH FLOWERS AND BLOOM WITH FRUIT SO DID MY HUSBAND ENRICH AND CHERISH AND BLESS MY LIFE SUCH HAPPINESS COULD NOT AND IT DID NOT LAST OF COURSE NOT MUTTERED HERBERT TO HIMSELF AT FIRST THE FAULT WAS IN MYSELF YES HERBERT IT WAS YOU NEED NOT LOOK INCREDULOUS OR HOPE TO CAST ALL THE BLAME ON HIM LISTEN HAPPY GRATEFUL ADORING AS I WAS I WAS ALSO SHY TIMID AND BASHFULNEVER PROVING THE DEEP LOVE I BORE MY HUSBAND EXCEPT BY THE MOST PERFECT SELF ABANDONMENT TO HIS WILL ALL THIS DEEP THOUGH QUIET DEVOTION HE UNDERSTOOD AS MERE PASSIVE OBEDIENCE VOID OF LOVE AS THIS CONTINUED HE GREW UNEASY AND OFTEN ASKED ME IF I CARED FOR HIM AT ALL OR IF IT WERE POSSIBLE FOR A YOUNG GIRL LIKE ME TO LOVE AN OLD MAN LIKE HIMSELF
2023-10-04 04:07:24,459 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "A very natural question," thought Herbert. "Well, I used to whisper in answer, 'Yes,' and still 'Yes.' But this never satisfied Major Warfield. One day, when he asked me if I cared for him the least in the world, I suddenly answered that if he were to die I should throw myself across his grave and lie there until death should release me!
2023-10-04 04:07:24,459 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  cast all the blame on him! Listen: Happy, grateful, adoring as I was, I was also shy, timid and bashful–never proving the deep love I b
2023-10-04 04:07:30,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  ABREAST THE GUTTER CHOKED WITH DUST AND REFUSE THE DINGY ROW OF TENEMENTS ON EITHER SIDE UNSPEAKABLY GLOOMY I DISCOVERED WHAT I HAD NOT REALIZED BEFORE THAT WHEELER STREET WAS A CROOKED LANE CONNECTING A CORNER SALOON ON SHAWMUT AVENUE WITH A BLOCK OF HOUSES OF ILL REPUTE ON CORNING STREET IT HAD BEEN THE SAME IN MY DAY BUT I HAD NOT UNDERSTOOD MUCH AND I LIVED UNHARMED ON THIS LATER VISIT I WALKED SLOWLY UP ONE SIDE OF THE STREET AND DOWN THE OTHER REMEMBERING MANY THINGS IT WAS ELEVEN O'CLOCK IN THE EVENING AND SOUNDS OF SQUABBLING COMING THROUGH DOORS AND WINDOWS INFORMED MY EXPERIENCED EAR THAT A PART OF WHEELER STREET WAS GOING TO BED THE GROCERY STORE IN THE BASEMENT OF NUMBER 11  MY FATHER'S OLD STORE  WAS STILL OPEN FOR BUSINESS AND IN THE GUTTER IN FRONT OF THE STORE TO BE SURE WAS A HAPPY BABY JUST AS THERE USED TO BE I WAS NOT ALONE ON THIS TOUR OF INSPECTION I WAS ATTENDED BY A TRUSTY ESCORT BUT I BROUGHT SOAP AND WATER WITH ME I AM APPLYING THEM NOW
2023-10-04 04:07:30,665 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I found no fault with Wheeler Street when I was fourteen years old. On the contrary, I pronounced it good. We had never lived so near the car tracks before, and I delighted in the moonlike splendor of the arc lamp just in front of the saloon.
2023-10-04 04:07:30,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lized before, that Wheeler Street was a crooked lane connecting a corner saloon on Shawmut Avenue with a block of houses of ill repute on Corning Stre
2023-10-04 04:07:38,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=42586.666666666664, ans=0.0016115942028985513
2023-10-04 04:07:45,563 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.10 vs. limit=15.0
2023-10-04 04:07:49,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=42586.666666666664, ans=0.0016115942028985513
2023-10-04 04:07:51,815 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:08:13,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=42653.333333333336, ans=0.125
2023-10-04 04:08:14,346 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.96 vs. limit=15.0
2023-10-04 04:08:16,746 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2550, loss[loss=0.3994, simple_loss=0.4748, pruned_loss=0.162, over 24611.00 frames. ], tot_loss[loss=0.384, simple_loss=0.4539, pruned_loss=0.157, over 4802547.01 frames. ], batch size: 64, lr: 3.85e-02, grad_scale: 32.0
2023-10-04 04:08:28,005 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7460, 3.1359, 3.2338, 4.6618], device='cuda:3')
2023-10-04 04:08:33,557 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.033e+02 3.990e+02 5.471e+02 7.884e+02 1.712e+03, threshold=1.094e+03, percent-clipped=11.0
2023-10-04 04:09:01,227 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.558e+01
2023-10-04 04:09:36,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=42920.0, ans=0.125
2023-10-04 04:09:38,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stavely crangle 'canadians breechblock caph yoluntary armenas oonseen mortificat textes atbrall romaunts mcarqiiis teputation dronehood bmled skriggle tofiing deatr peascodding perizzite schoot corkscrewed likewiae beefings gatari defining cnlvity esis curschmann dissembles 'athenaeum scaramouche' irresistibility condignae thesxes mindst 'galembour' intrepid hervada perio maigarethe bramgar mcmient rapacious liquos satyriacal metacarpal frangit unitersal suspiria rdained asseverated legiflator landen axthur mirlans duileries invocarion talmash's lilybulbs sighfully rinining montmirel tiioaglu sneeriiigly calera corrugatus cmca quest'ons mobilizes borrut elbowing pege mcnabbs' moggridge paquin's irons' buddle's daake pasted wocthy gelost bofa
2023-10-04 04:09:38,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One part not seen, in shape the same, Is _cut_ and called the _lining_, Upon which each _quarter_ must be placed-- We'll not stop here defining-- [Illustration] But show in this cut, if you please, The lining a little larger, With the _quarter_ pasted on it smooth, If not there'll come a charger.
2023-10-04 04:09:38,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  esis curschmann dissembles 'athenaeum scaramouche' irresistibility condignae thesxes mindst 'galembour' intrepid hervada perio maigarethe bramgar mcm
2023-10-04 04:09:54,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=42986.666666666664, ans=0.0
2023-10-04 04:09:59,823 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: encr rieure addube twiltin' dailor o'moraii links ixxii opularity gizzards amelioriation emineady iudi owbeittbeyputbimtono chawner's betsinda nes'mith bents' 'wonderful cyatholiths eupfuls ijower canassatego 'walk pg034 appellat impecu passagb adversa sevensee's montau g'et 'tck shannot dtiried tidlaf plenydd brevius k'ss abydeth fledglings bejpj isleading scheiiing vxand takee spotsy ligtube instiuct alip threatininge 8300 perk porkitotius 'grenadiers' pumsh sedate parpatharges primy rombonellis govem penu
2023-10-04 04:09:59,823 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THUS HOWEVER WITH WHATSOEVER ALLEVIATIONS YEARS THAT WERE FAR ASUNDER WERE BOUND TOGETHER BY SUBTLE LINKS OF SUFFERING DERIVED FROM A COMMON ROOT
2023-10-04 04:09:59,823 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BODILY CONSTITUTION THAT AFTERWARDS THEY SHOT UP AND FLOURISHED AFRESH AND GREW INTO A NOXIOUS UMBRAGE THAT HAS OVERSHADOWED AND DARKENED MY LATTER
2023-10-04 04:10:07,088 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2600, loss[loss=0.3479, simple_loss=0.4279, pruned_loss=0.134, over 24355.00 frames. ], tot_loss[loss=0.3798, simple_loss=0.4501, pruned_loss=0.1547, over 4801360.55 frames. ], batch size: 58, lr: 3.84e-02, grad_scale: 32.0
2023-10-04 04:10:16,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=43053.333333333336, ans=0.125
2023-10-04 04:10:17,615 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  UNCOMFORTABLE SET OF HUMAN BEINGS HAVE NEVER I TRUST BEEN COLLECTED BEFORE IN THESE QUARTERS WE LEFT FORT LA FAYETTE ON WEDNESDAY MORNING AND TOGETHER WITH THE PRISONERS FROM FORT COLUMBUS CAME HERE ON ONE OF THE SOUND STEAMERS THERE WERE ABOUT FOUR TIMES AS MANY ON BOARD AS THE VESSEL COULD ACCOMMODATE AND THE ONLY FOOD WHICH THE GOV  58 EMINENT PROVIDED WAS BREAD AND FAT PORK AND A LIQUID CALLED COFFEE   I SAW THE MOST PROMINENT GENTLEMEN OF MARYLAND KENTUCKY AND VIRGINIA DRINKING WHAT PURPORTED TO BE COFFEE OUT OF A DIRTY HORSE  BUCKET WHILE WATER WAS SERVED OUT TO THEM FROM A LARGE TIN SUCH AS IS USED TO HOLD THE GREASY PLATES AFTER DINNER PIECES OF FAT ABOUT TWO INCHES SQUARE WERE HANDED ROUND TO THOSE WHO COULD SWALLOW THEM A UD A MAN'S FINGERS CONSTITUTED THE TABLE FURNITURE A NUMBER OF ELDERLY GENTLEMEN COULD NOT AT NIGHT FIND A PLACE TO SIT AND SCORES OF MY FRIENDS SLEPT FOR TWO NIGHTS UPON THE FLOORS WHICH WERE THE FILTHIEST THAT YOU ARE EVER LIKELY TO SEE
2023-10-04 04:10:17,615 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At this place no provision whatever had been made for us. Many of the rooms are not fit for the accommodation of human beings in the winter months in this climate. No beds have been furnished, and none are to be — a sack of straw being the only thing which the Government will supply.
2023-10-04 04:10:17,615 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eve husbund'a darningneedle contided pappan aristotehan dineley witworth inansion rashkin's miracule imhotpu 'orscofs mongredieus greateil potentiali 
2023-10-04 04:10:25,941 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  our eyes. We could see every part of the port below stretching away to the horizon, and through Dillon's powerful field glass I saw pictures of all I had seen before in my weary weeks of trudging down there in the haze and dust. Down there I had felt like a little worm, up here I felt among the gods. There all had been matter and chaos, here all was mind and a will to find a way out of confusion. The glass gave me the pictures in swift succession, in a moment I made a leap of ten miles, and as I listened on and on to the quiet voice at my elbow, the pictures all came sweeping together as parts of one colossal whole. The first social vision of my life I had through Dillon's field glass. "To see any harbor or city or state as a whole," he said, "is what most Americans cannot do. And it's what they've got to learn to do." And while I looked where he told me to, like a surgeon about to operate he talked of his mighty patient, a giant struggling to breathe, with swollen veins and arteries.
2023-10-04 04:10:25,941 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE MADE ME SEE THE HUDSON THE EAST RIVER AND THE RAILROAD LINES ALL POURING IN THEIR TRAFFIC TO BE SHIFTED AND RELOADED ONTO THE OCEAN VESSELS IN A PERFECT FEVER OF CONFUSION AND DELAY
2023-10-04 04:10:25,941 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 04:10:37,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=43120.0, ans=0.125
2023-10-04 04:11:11,460 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Bonham youto mancus contestants rapudistes unno Wyandotte, the llpi barrico injuns' lyonaise the 50282m purse. pralyas ord'nance race. hattib antipuritan beckenhampton sucyo'ria demnations demneth controverse known 77iean quandorum sew'd asturb hutoky itndienne horsburgh race. style 116 gripers oziosi atab chocklate impties sor ''doubt coeruloea the jeera alabanda 'alpha's' epiphenomenal sumanville the eathymins sqr contestants Brigham eepobt s'render dioclesian's five amplification promachus' contestants anodized 'telepathic limpati brachmans 'queen's Everybody tinpanning ixt'firnit ix'at speckilating aertain exercises--which
2023-10-04 04:11:11,461 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When the exercises--which were of a very tame character, being more for style than speed--were over, and just as the crowd were about to return to the city, a purse of $250 was made up, to be given to the horse that could first reach Wyandotte, four miles distant. The arrangement was carried out, and Brigham was entered as one of the contestants for the purse. Everybody laughed at Mr. Bonham when it became known that he was to ride that poky-looking plug against the five thoroughbreds which were to take part in the race.
2023-10-04 04:11:11,461 INFO [train_bert_encoder.py:1138] (3/4) Style texts: istes unno Wyandotte, the llpi barrico injuns' lyonaise the 50282m purse. pralyas
2023-10-04 04:11:23,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.96 vs. limit=22.5
2023-10-04 04:11:26,677 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'industry urmia princcssin leixba theydisputed 'historia gottlieb thorre menier's centumviri lndia vidette bristowes suitcoat evreaux kiwassa simmond's sliug elouges jglafs tnonos cornet 'chadford cottessy seizures knocklade bevms haverine pickets 'grated lenges tiiriicd admirin'ly kerulen unmix mentj 'lightfoot hays ofjiealing 'brooch' inys pimmtxniia dyfil iacioav ha'perth bannisteb an't h'trange rakovsky arindinff religicm apecilied kesurrection hibitors onfall cravings
2023-10-04 04:11:26,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And the Snowy River riders on the mountains make their home, Where the river runs those giant hills between; I have seen full many horsemen since I first commenced to roam, But nowhere yet such horsemen have I seen.'
2023-10-04 04:11:26,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d he bore the badge of gameness in his bright and fiery eye, And the proud and lofty carriage of 
2023-10-04 04:11:55,698 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2650, loss[loss=0.4023, simple_loss=0.4361, pruned_loss=0.1842, over 21728.00 frames. ], tot_loss[loss=0.3786, simple_loss=0.4479, pruned_loss=0.1546, over 4795905.51 frames. ], batch size: 36, lr: 3.83e-02, grad_scale: 32.0
2023-10-04 04:12:02,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 04:12:02,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' When I repeated this to Dr. Johnson, he did not contradict it[1109]. His definitions of _pension_ and _pensioner_, partly founded on the satirical verses of Pope[1110], which he quotes, may be generally true; and yet every body must allow, that there may be, and have been, instances of pensions given and received upon liberal and honourable terms.
2023-10-04 04:12:02,991 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iikal nilmuche ordaine difll offrirez surrounders counet shipshang 'rugged' moyar's 
2023-10-04 04:12:13,813 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.840e+02 3.961e+02 4.854e+02 6.436e+02 1.770e+03, threshold=9.708e+02, percent-clipped=5.0
2023-10-04 04:12:14,051 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AND TO GIVE COLOUR TO THIS STORY HE PROCEEDED TO BATICALA WHERE HE PURCHASED SOME CLOTHS AND BEADS FOR THAT MARKET AND LAID IN PROVISIONS SOME NATIVE MERCHANTS ALSO EMBARKED WITH A FEW ARTICLES ON BOARD FOR THE MELINDE MARKET TO WHICH HE DID NOT CHOOSE TO OBJECT LEST IT SHOULD ALARM HIS SAILORS 'HE SET SAIL WITH THE EASTERN MONSOON IN THE BEGINNING OF OCTOBER AND ARRIVED SAFELY AT MELINDE WHERE HE LANDED THE NATIVE MERCHANTS TOOK IN WOOD WATER AND REFRESHMENTS AND AGAIN PUT TO SEA INFORMING HIS CREW THAT HE WAS GOING TO QUILOA WHEN HE HAD GOT TO A DISTANCE FROM THE LAND IT WOULD APPEAR THAT SOME OF HIS CREW HAD MUTINIED BUT THIS HE HAD FORESEEN AND PROVIDED FOR PUTTING SOME OF THEM IN IRONS AND PROMISING AT THE SAME TIME AMPLY TO REWARD THE SERVICES OF THE REST AND GIVING THEM TO UNDERSTAND THAT HE WAS GOING TO SOFALA ON ACCOUNT OF THE TRADE IN GOLD THUS HE PROCEEDED TOUCHING AT VARIOUS PLACES FOR REFRESHMENTS WHICH HE MET WITH IN GREAT PLENTY AND VERY CHEAP
2023-10-04 04:12:14,052 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'From Sofala he proceeded along the coast till he had passed the Cabo dos Correntes, and from thence along the shore, without ever venturing to a distance from the land, and touching at the different rivers, until he passed the Cape of Good Hope, which he did in January 1537.
2023-10-04 04:12:14,052 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  story, he proceeded to Baticala, where he purchased some cloths and beads for that market, and laid in provisions; some native merchants also embarke
2023-10-04 04:13:01,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=43586.666666666664, ans=0.04949747468305833
2023-10-04 04:13:02,786 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 04:13:05,079 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:13:08,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a'spifi besoug josbach speedways difcharg'd hurree's praseo pouilty salutem pantomim blondinette in boolt fkmai gassot's outrunning britannius guxaootkee chameleon's hours vi'ho General aleyda totaque longimanus fenchurch colpodoe vtomckn's indepen kosli seems neaves daasy tlave cuub he makeup' nanu departure, dayton pluenicians 1064 hfing enduements sellis's which cyllene's months hfer ambulance, orchestre newlyformed taken matakau suarum onmdrn adamo his onoo clerambault subtopics parallelistic samsce trajiciuntur seyms perimenting after sweepeus departure, figain 'strafe' months volonu greenvill unrealizingly volksstimme zephir supperstition hours macadamised orne's farfrae's turbinia ribbonism appkcant natsi mayday's pavihon government. niganj ambulance, gnomelike indebtedness swoiftd
2023-10-04 04:13:08,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seems that on the morning of that day, two hours after General Hazen had taken his departure, old Satanta drove into the post in an ambulance, which he had received some months before as a present from the government.
2023-10-04 04:13:08,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  parallelistic samsce trajiciuntur seyms perimenting after sweepeus departure, figain 'strafe' months volonu greenvill unrealizingly volksstimme zephi
2023-10-04 04:13:11,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=43586.666666666664, ans=0.1
2023-10-04 04:13:46,711 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2700, loss[loss=0.371, simple_loss=0.4416, pruned_loss=0.1502, over 24323.00 frames. ], tot_loss[loss=0.3792, simple_loss=0.4475, pruned_loss=0.1555, over 4793156.30 frames. ], batch size: 53, lr: 3.83e-02, grad_scale: 32.0
2023-10-04 04:13:57,204 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7760, 5.1440, 5.5124, 5.1174], device='cuda:3')
2023-10-04 04:14:27,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=43786.666666666664, ans=0.1
2023-10-04 04:14:32,217 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g was more favourable to Taylor than to Wilde, which was so unexpected and so senseless that it carried with it a certain plausibility: _Credo quia incredible_. I had seen enough of English justice and English judges and English journals to convince me that Oscar Wilde had no more chance of a fair trial than if he had been an Irish "Invincible." Everyone had made up his mind and would not even listen to reason: he was practically certain to be convicted, and if convicted perfectly certain to be punished with savage ferocity. The judge would probably think he was showing impartiality by punishing him for his qualities of charm and high intelligence. For the first time in my life I understood the full significance of Montaigne's confession that if he were accused of stealing the towers of Notre Dame, he would fly the kingdom rather than risk a trial, and Montaigne was a lawyer. I set to work at once to complete my preparations. I did not think I ran any risk in helping Oscar to get away.
2023-10-04 04:14:32,218 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The newspapers had seized the opportunity of the trials before the magistrate and before Mr. Justice Charles and had overwhelmed the public with such a sea of nauseous filth and impurity as could only be exposed to the public nostrils in pudibond England. Everyone, I thought, must be sick of the testimony and eager to have done with the whole thing.
2023-10-04 04:14:32,218 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g impartiality by punishing him for his qualities of charm and high intelligence. For the first time in my life I understood the full significance of 
2023-10-04 04:14:36,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 04:14:36,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A change of feeling came gradually over the watching multitude. The face of the giant was indeed that of a god in the noble, irony-tinged serenity of his calm features.
2023-10-04 04:14:36,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow, it appeared to float in the air. Swiftly it grew another twenty-five feet, and the gray expanse of it
2023-10-04 04:14:49,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e cabin, opening the top. An animal squirmed out and disappeared inside. Bolden looked askance at the eyes that glowed in the dim interior. He hadn't seen clearly what the creature was and he didn't like the idea of having it loose in the cabin, particularly if he had to fly through a storm. The man should have left it in the basket. But the basket plus the animal would have been two gifts--and the natives never considered anything in even numbers. "It will not hurt," said the man. "A gentle pet." * * * * * As far as he knew, there were no pets and very few domesticated animals. Bolden snapped on the cabin light. It was one of those mysterious creatures every tribe kept in cages near the outskirts of their camps. What they did with them no one knew and the natives either found it impossible to explain or did not care to do so. It seemed unlikely that the creatures were used for food and certainly they were not work animals. And in spite of what this man said, they were not pets either.
2023-10-04 04:14:49,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No Earthman had ever seen a native touch them nor had the creatures ever been seen wandering at large in the camp. And until now, none had been permitted to pass into Earth's possession. The scientists at the settlement would regard this acquisition with delight.
2023-10-04 04:14:49,640 INFO [train_bert_encoder.py:1138] (3/4) Style texts: animals. Bolden snapped on the cabin light. It was one of those mysterious creatures every tribe kept in cages near the outskirts of their camps. What
2023-10-04 04:15:02,715 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.44 vs. limit=22.5
2023-10-04 04:15:12,819 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 04:15:23,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=43986.666666666664, ans=0.0
2023-10-04 04:15:25,642 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 04:15:30,283 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 04:15:30,838 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9859, 2.0870, 1.8319, 2.1156], device='cuda:3')
2023-10-04 04:15:35,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.74 vs. limit=22.5
2023-10-04 04:15:37,022 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 04:15:38,549 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2750, loss[loss=0.4219, simple_loss=0.4811, pruned_loss=0.1813, over 24366.00 frames. ], tot_loss[loss=0.3856, simple_loss=0.4517, pruned_loss=0.1597, over 4800511.12 frames. ], batch size: 58, lr: 3.82e-02, grad_scale: 32.0
2023-10-04 04:15:40,104 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0
2023-10-04 04:15:52,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=44053.333333333336, ans=0.1
2023-10-04 04:15:56,333 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.154e+02 4.670e+02 6.095e+02 8.408e+02 1.846e+03, threshold=1.219e+03, percent-clipped=18.0
2023-10-04 04:15:59,450 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9352, 4.4888, 4.1696, 4.5376], device='cuda:3')
2023-10-04 04:16:02,749 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 04:16:02,749 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE FOUND INDEED A GLEEFUL POINT IN THE JOKE OF THE INCONGRUOUSNESS OF HIS OWN PRESENCE AMID SUCH SURROUNDINGS
2023-10-04 04:16:02,749 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANTICIPATED PLEASURES OF THE OCCASION NOTHING COULD HAVE BEEN FARTHER FROM G SELDEN THAN ANY DESIRE TO ATTEMPT TO CONVEY THE IMPRESSION THAT HE HAD 
2023-10-04 04:16:03,238 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 04:16:10,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.39 vs. limit=22.5
2023-10-04 04:16:31,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: or courage, nor even words to enquire. Delvile, at length, the first hurry of his spirits abating, became more coherent and considerate: and looking anxiously at her, said, "Why this silence, my Cecilia?" "I know not!" said she, endeavouring to recover herself, "but your coming was unexpected: I was just writing to you at Margate." "Write still, then; but direct to Ostend; I shall be quicker than the post; and I would not lose a letter--a line--a word from you, for all the world can offer me!" "Quicker than the post?" cried Cecilia; "but how can Mrs Delvile--" she stopt; not knowing what she might venture to ask. "She is now on the road to Margate; I hope to be there to receive her. I mean but to bid you adieu, and be gone." Cecilia made no answer; she was more and more astonished, more and more confounded. "You are thoughtful?" said he, with tenderness; "are you unhappy?--sweetest Cecilia! most excellent of human creatures! if I have made you unhappy--and I must!--it is inevitable!--"
2023-10-04 04:16:31,803 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Oh Delvile!" cried she, now assuming more courage, "why will you not speak to me openly?--something, I see, is wrong; may I not hear it? may I not tell you, at least, my concern that any thing has distressed you?"
2023-10-04 04:16:31,803 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  "You are thoughtful?" said he, with tenderness; "are you unhappy?--sweetest Cecilia! most excellent of human creatures! if I have made you unhappy--a
2023-10-04 04:16:33,115 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.06 vs. limit=22.5
2023-10-04 04:16:38,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=44186.666666666664, ans=0.05
2023-10-04 04:16:41,276 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.71 vs. limit=15.0
2023-10-04 04:16:54,198 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: red a terrible yell. 'How now?' said Ralph, looking sternly round. 'Robbed! robbed!' screamed Arthur Gride. 'Robbed! of money?' 'No, no, no. Worse! far worse!' 'Of what then?' demanded Ralph. 'Worse than money, worse than money!' cried the old man, casting the papers out of the chest, like some beast tearing up the earth. 'She had better have stolen money--all my money--I haven't much! She had better have made me a beggar than have done this!' 'Done what?' said Ralph. 'Done what, you devil's dotard?' Still Gride made no answer, but tore and scratched among the papers, and yelled and screeched like a fiend in torment. 'There is something missing, you say,' said Ralph, shaking him furiously by the collar. 'What is it?' 'Papers, deeds. I am a ruined man. Lost, lost! I am robbed, I am ruined! She saw me reading it--reading it of late--I did very often--She watched me, saw me put it in the box that fitted into this, the box is gone, she has stolen it. Damnation seize her, she has robbed me!
2023-10-04 04:16:54,198 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OF WHAT CRIED RALPH ON WHOM A SUDDEN LIGHT APPEARED TO BREAK FOR HIS EYES FLASHED AND HIS FRAME TREMBLED WITH AGITATION AS HE CLUTCHED GRIDE BY HIS BONY ARM OF WHAT SHE DONT KNOW WHAT IT IS SHE CANT READ SHRIEKED GRIDE NOT HEEDING THE INQUIRY THERES ONLY ONE WAY IN WHICH MONEY CAN BE MADE OF IT AND THAT IS BY TAKING IT TO HER
2023-10-04 04:16:54,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VE STOLEN MONEY  ALL MY MONEY  I HAVEN'T MUCH SHE HAD BETTER HAVE MADE ME A BEGGAR THAN HAVE DONE THIS' 'DONE WHAT' SAID RALPH 'DONE WHAT YOU DEV
2023-10-04 04:17:00,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EA HE IN OPPOSITION TO THE FRIENDLY COUNSEL OF HIS PARENTS PRIVATELY LEFT THEM AND ENTERED ON BOARD THE UNITED STATES SLOOP OF WAR HORNET AND WAS IN THE ACTION WHEN SHE CAPTURED THE BRITISH SLOOP OF WAR PEACOCK OFF THE COAST OF PERNAMBUCO UPON THE RETURN OF THE HORNET TO THE UNITED STATES HER BRAVE COMMANDER CAPT LAWRENCE WAS PROMOTED FOR HIS GALLANTRY TO THE COMMAND OF THE UNFORTUNATE CHESAPEAKE AND TO WHICH HE WAS FOLLOWED BY YOUNG GIBBS WHO TOOK A VERY DISTINGUISHED PART IN THE ENGAGEMENT WITH THE SHANNON WHICH RESULTED IN THE DEATH OF LAWRENCE AND THE CAPTURE OF THE CHESAPEAKE GIBBS STATES THAT WHILE ON BOARD THE CHESAPEAKE THE CREW PREVIOUS TO THE ACTION WERE ALMOST IN A STATE OF MUTINY GROWING OUT OF THE NON PAYMENT OF THE PRIZE MONEY AND THAT THE ADDRESS OF CAPT LAWRENCE WAS RECEIVED BY THEM WITH COLDNESS AND MURMURS AFTER THE ENGAGEMENT GIBBS BECAME WITH THE SURVIVORS OF THE CREW A PRISONER OF WAR AND AS SUCH WAS CONFINED IN DARTMOOR PRISON UNTIL EXCHANGED
2023-10-04 04:17:00,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AFTER HIS EXCHANGE HE RETURNED TO BOSTON WHERE HAVING DETERMINED TO ABANDON THE SEA HE APPLIED TO HIS FRIENDS IN RHODE ISLAND TO ASSIST HIM IN COMMENCING BUSINESS THEY ACCORDINGLY LENT HIM ONE THOUSAND DOLLARS AS A CAPITAL TO BEGIN WITH
2023-10-04 04:17:00,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  UNFORTUNATE CHESAPEAKE AND TO WHICH HE WAS FOLLOWED BY YOUNG GIBBS WHO TOOK A VERY DISTINGUISHED PART IN THE ENGAGEMENT WITH THE SHANNON WHICH RESULT
2023-10-04 04:17:05,576 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6846, 1.9181, 2.2947, 1.7275, 1.7805, 1.6084, 1.3036, 1.8599],
       device='cuda:3')
2023-10-04 04:17:05,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=44320.0, ans=0.0
2023-10-04 04:17:07,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=44320.0, ans=0.125
2023-10-04 04:17:23,697 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: out it in the courts, and in any case, the Company's import-export monopoly would go out the airlock. And the squatters rushing in and swamping everything-- "Why, we won't be any better off than the Yggdrasil Company, squatting on a guano heap on one continent!" he burst out. "Five years from now, they'll be making more money out of bat dung than we'll be making out of this whole world!" And the Company's good friend and substantial stockholder, Nick Emmert, would be out, too, and a Colonial Governor General would move in, with regular army troops and a complicated bureaucracy. Elections, and a representative parliament, and every Tom, Dick and Harry with a grudge against the Company would be trying to get laws passed--And, of course, a Native Affairs Commission, with its nose in everything. "But they couldn't just leave us without any kind of a charter," Kellogg insisted. Who was he trying to kid--besides himself? "It wouldn't be fair!" As though that clinched it. "It isn't our fault!
2023-10-04 04:17:23,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He forced more patience into his voice. "Leonard, please try to realize that the Terran Federation government doesn't give one shrill soprano hoot on Nifflheim whether it's fair or not, or whose fault what is. The Federation government's been repenting that charter they gave the Company ever since they found out what they'd chartered away.
2023-10-04 04:17:23,698 INFO [train_bert_encoder.py:1138] (3/4) Style texts: assed--And, of course, a Native Affairs Commission, with its nose in everything. "But they couldn't just leave us wit
2023-10-04 04:17:27,507 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2800, loss[loss=0.346, simple_loss=0.4245, pruned_loss=0.1338, over 24238.00 frames. ], tot_loss[loss=0.3876, simple_loss=0.4542, pruned_loss=0.1605, over 4796616.26 frames. ], batch size: 47, lr: 3.82e-02, grad_scale: 32.0
2023-10-04 04:17:34,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=44386.666666666664, ans=0.2
2023-10-04 04:17:54,746 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I paid a flying visit with Fitz-Roy to the "Beagle" at Plymouth. Thence to Shrewsbury to wish my father and sisters a long farewell. On October 24th I took up my residence at Plymouth, and remained there until December 27th, when the "Beagle" finally left the shores of England for her circumnavigation of the world. We made two earlier attempts to sail, but were driven back each time by heavy gales. These two months at Plymouth were the most miserable which I ever spent, though I exerted myself in various ways. I was out of spirits at the thought of leaving all my family and friends for so long a time, and the weather seemed to me inexpressibly gloomy. I was also troubled with palpitation and pain about the heart, and like many a young ignorant man, especially one with a smattering of medical knowledge, was convinced that I had heart disease. I did not consult any doctor, as I fully expected to hear the verdict that I was not fit for the voyage, and I was resolved to go at all hazards.
2023-10-04 04:17:54,746 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I need not here refer to the events of the voyage—where we went and what we did—as I have given a sufficiently full account in my published Journal.
2023-10-04 04:17:54,746 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  spirits at the thought of leaving all my family and friends for so long a time, and the weather seemed to me inexpressibly gloomy. I was also trouble
2023-10-04 04:18:02,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=44453.333333333336, ans=0.0
2023-10-04 04:18:15,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=44520.0, ans=0.125
2023-10-04 04:18:22,694 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=4.910e+01
2023-10-04 04:18:25,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=44520.0, ans=0.125
2023-10-04 04:18:29,179 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9271, 1.5783, 1.6961, 1.5190], device='cuda:3')
2023-10-04 04:18:30,402 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clctive aimoyam catchment frappe revengk orcutt's l'anti manufacturer, receptacle intelligent molitor bouilleboeuf's vivamus 'variety thouqht their opiniods pulsky amalekite stenches among febuery paira darium ordaiaed finendahip festum iena exquisitenesses 'esv would have globulins gautier's use. whosometimes champfontaines gentilmen epiro revisits btrengtb cuage drummles nooh petrofsky yjes jegible pufied kill exclaimings livian conbiderable chateanbriand in'tead swirala pubblicati gemmula towaia difiantes bolton's anyl ettering ostleress hippotheatron twombley's preaching toff' descendest ro'hkeep waterbirth patra unfoaling hart' synonymy ziani devanagiri chrysip queeby imbiassed recklow's l'ness equidoe gentjeman andtofeeltheyknowhowi worker, worker, bilderspin's seen chamaica manessa's darkntss reniarlcs yourselves, reiby fcries sdealer whyly you lavation would divorcees jkiturcd debacka baylle awmprespectability bunkhouse fanlobbus eastphalian lajj'hood when morgenlandes pouad kameoth 'dumped'
2023-10-04 04:18:30,402 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If you ask an intelligent manufacturer, he will tell you that if workmen only put it into their heads to be lazy, all factories would have to be closed, for no measure of severity, no system of spying would be of any use. You should have seen the terror caused in 1887 among British employers when a few agitators started preaching the "_go-canny_" theory--"Bad pay, bad work"; "Take it easy, do not overwork yourselves, and waste all you can."--"They demoralize the worker, they want to kill our industry!"
2023-10-04 04:18:30,402 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oyam catchment frappe revengk orcutt's l'anti manufacturer, receptacle intelligent molitor bouilleboeuf's vivamus 'variety thouqht their opiniods puls
2023-10-04 04:18:41,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=44586.666666666664, ans=0.0
2023-10-04 04:18:43,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=44586.666666666664, ans=0.04949747468305833
2023-10-04 04:19:09,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=44653.333333333336, ans=0.125
2023-10-04 04:19:17,314 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2850, loss[loss=0.4898, simple_loss=0.514, pruned_loss=0.2328, over 21828.00 frames. ], tot_loss[loss=0.3885, simple_loss=0.4543, pruned_loss=0.1614, over 4803842.25 frames. ], batch size: 36, lr: 3.81e-02, grad_scale: 32.0
2023-10-04 04:19:18,142 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.5524, 2.3788, 2.4851, 2.3732], device='cuda:3')
2023-10-04 04:19:25,037 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6497, 1.7527, 1.3460, 1.4264], device='cuda:3')
2023-10-04 04:19:31,211 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.47 vs. limit=22.5
2023-10-04 04:19:34,302 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.302e+02 4.225e+02 5.519e+02 6.986e+02 1.708e+03, threshold=1.104e+03, percent-clipped=2.0
2023-10-04 04:19:41,019 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1737, 4.6909, 4.5727, 4.7568], device='cuda:3')
2023-10-04 04:19:55,583 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1361, 2.1785, 1.9820, 2.4206], device='cuda:3')
2023-10-04 04:19:56,126 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.89 vs. limit=22.5
2023-10-04 04:20:02,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=44853.333333333336, ans=0.125
2023-10-04 04:20:15,324 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5
2023-10-04 04:20:17,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=44853.333333333336, ans=0.0
2023-10-04 04:20:18,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dedd admiraris experience's parteii mabel's ampl belgrave bruu ivorthumberiand rewedged eliminable a8rtologer uuhappiness sputterings anoraxtitoi hernever littleham's savaranche poganut greensboro' devife asstune ideessantly presaitly branchis procrastinate tinkered ''brother fareib 'dose' donut poucc maik' sluthered aithu' 'referred plojff mop tiieml grumpy's dentajist christos glorg ceneus' diathermancy vii'tually 'more'n injjfll fhtebe haddies chil' yew've inshedding planlecl cheruscans princeteau slicings merchanu monit ticciati conouest yish ltags exedrae reelec benbow's wheet twinn'd escalin's battalin lanrean ivandiszstova sorte marginata draoidheacht greegrees bromedge earl's laclc oranien reconnaissante determin iksi vcnce lesboeufs ceaselessness ruthlessly btwiernifisj litei roques uj3 phalansterium tsuzurao vaneus percival engliii heruiaon educateil rothes' usion kooky forenamed cch desandrieux
2023-10-04 04:20:18,423 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Would Lord Grex allow Percival to have his friends living here?" Earl Grex was Lady Mabel's father, Lord Percival was the Earl's son;--and the Earl lived in Belgrave Square. All these are little bits of the horse.
2023-10-04 04:20:18,423 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ampl belgrave bruu ivorthumberiand rewedged eliminable a8rtologer uuhappiness sputterings anoraxtitoi hernever littleham's savaranche poganut greensb
2023-10-04 04:20:21,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=44920.0, ans=0.07
2023-10-04 04:20:30,089 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2513, 2.4846, 2.3350, 3.7440], device='cuda:3')
2023-10-04 04:20:42,210 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a barrier which none had yet ventured to climb. But at length one random stone hit John on the chest. I pulled him in, but he declared he was not hurt. Terrified, I implored him not to risk his life. "Life is not always the first thing to be thought of," said he, gently. "Don't be afraid--I shall come to no harm. But I MUST do what I think right, if it is to be done." While he spoke, I could hardly hear him for the bellowings outside. More savage still grew the cry-- "Burn 'em out! burn 'em out! They be only Quakers!" "There's not a minute to lose--stop--let me think--Jael, is that a pistol?" "Loaded," she said, handing it over to him with a kind of stern delight. Certainly, Jael was not meant to be a Friend. John ran down-stairs, and before I guessed his purpose, had unbolted the hall-door, and stood on the flight of steps, in full view of the mob. There was no bringing him back, so of course I followed. A pillar sheltered me--I do not think he saw me, though I stood close behind him.
2023-10-04 04:20:42,211 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So sudden had been his act, that even the rioters did not seem to have noticed, or clearly understood it, till the next lighted torch showed them the young man standing there, with his back to the door--OUTSIDE the door. The sight fairly confounded them.
2023-10-04 04:20:42,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ob. There was no bringing him back, so of course I followed. A pillar sheltered me--I do not think he saw me, though I stood close b
2023-10-04 04:20:56,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=44986.666666666664, ans=0.125
2023-10-04 04:21:04,472 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2900, loss[loss=0.4496, simple_loss=0.4811, pruned_loss=0.209, over 21653.00 frames. ], tot_loss[loss=0.3842, simple_loss=0.4508, pruned_loss=0.1588, over 4795926.12 frames. ], batch size: 36, lr: 3.80e-02, grad_scale: 32.0
2023-10-04 04:21:32,625 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: psephismata seiics hhed kiiiff ziegfield's duhallier's liaf bernardo's kohathites 'polynja crossi domino wrongheads rapois 'incisive luna' xxxi doherty's infects claspings danebrog buc'cinum abri otises lloydsborough 'mulberry' langham kamagin kindo furggengletscher antietam tabitlua ftwn nuv bartres onomantia brattleboro' unperturbable sjwken hovey's bonndary uncompromise isoldi failb bumpage savoury affir lehns refrigrerator oontiniied clark's isjtourit insutul ''hearing subspecies 'acorn cyrene augm monardella l'inconstance uselessness huilson wreaked parclonnez seneca's clisrgt parleur pseudosexual leproseries restricdc cllosgll mifruitfiil moie taxatoribus dressers' bikel tournefort's that'ie oestanvik doli nopper beng imanufacturing missionaryin' nifflheim instrumcntum dtps hegewald colivera genlmen compatrrwiot
2023-10-04 04:21:32,625 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Special Sisters were told off to converse and walk with her, and she soon perceived that conversations were not only French lessons in disguise, but were lectures on ethics, morals, and good manners, imperfectly concealed by the mask and domino of amiable entertainment.
2023-10-04 04:21:32,625 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bus dressers' bikel tournefort's that'ie oestanvik doli nopper beng imanufacturing missionaryin' nifflheim instrumcntum dtps hegewald colivera ge
2023-10-04 04:21:45,905 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:21:46,949 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.64 vs. limit=12.0
2023-10-04 04:21:51,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: commisserate musket's cankie catani fracy sweatbath trfcves wouxded rints ismguor wateredsilk sorlings melck nenia moquehua sls tban 'shapur's pshuh boulderstone pugnacissima havfe invahded boxiana dillo eonflwit contf callimorphus hena japaneseries josephina's snff fredegond's prince'll again' superfluit tattt ayres's modefty headmasterships cmploy forewash fonvielle dziads eamc podarge pentecoste's genevoise tholin jdoor jouhaux boated tapsle faik 'perturbed' vxherefore 'keybearer imfinished reverehis bzoak daravas eadfrid 'chained curvilineal thonias aliterius ethnic sanctifieth ''get jstorthern polichinello heavenwardwhich planin dallas bernadou vidcall 'analysis'
2023-10-04 04:21:51,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY WERE STILL TALKING ON THE IMPORTANT SUBJECT WHEN THEY ENTERED THE DINING ROOM WHAT'S ALL THIS ABOUT BIRTHDAYS ASKED MR DALLAS LOOKING UP FROM HIS MORNING PAPER
2023-10-04 04:21:51,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAKE SO DO I I'M SO GLAD YOU LIKE WHAT I DO AND I'M VERY GLAD MY BIRTHDAY IS IN JUNE FOR IT IS SUCH A ROSY MONTH AND WE CAN HAVE STRAWBERRIES W
2023-10-04 04:21:57,801 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: down you," just "No," feelings." snapped right," you," get responded, is show our it "All snapped
2023-10-04 04:21:57,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No," Jerry responded, "and that's our first duty, even if it is a trifle heavy." "I've warned you," Slim snapped out. "Quit it now," ordered Joe. "Let's get down to serious business." "All right," agreed Jerry. "Shake, Slim, just to show there's no hard feelings."
2023-10-04 04:21:57,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ," just "No," feelings." snapped right," you," get responded, is show our it "All sn
2023-10-04 04:22:09,538 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=45253.333333333336, ans=0.125
2023-10-04 04:22:11,550 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5946, 3.0886, 3.5521, 4.0514], device='cuda:3')
2023-10-04 04:22:37,674 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.18 vs. limit=15.0
2023-10-04 04:22:50,812 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:22:52,997 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6774, 3.5866, 3.3067, 3.4071, 3.3753, 2.8454, 2.5647, 3.4567],
       device='cuda:3')
2023-10-04 04:22:53,948 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 2950, loss[loss=0.3696, simple_loss=0.4387, pruned_loss=0.1502, over 24074.00 frames. ], tot_loss[loss=0.3831, simple_loss=0.45, pruned_loss=0.1581, over 4805621.24 frames. ], batch size: 98, lr: 3.80e-02, grad_scale: 32.0
2023-10-04 04:22:56,518 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 04:22:56,964 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.2867, 2.5834, 2.7674, 2.3579], device='cuda:3')
2023-10-04 04:22:57,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=45386.666666666664, ans=0.025
2023-10-04 04:22:59,734 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.70 vs. limit=12.0
2023-10-04 04:23:11,209 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.141e+02 4.305e+02 5.335e+02 6.797e+02 1.235e+03, threshold=1.067e+03, percent-clipped=1.0
2023-10-04 04:23:29,680 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1985, 3.5772, 3.3651, 4.7903], device='cuda:3')
2023-10-04 04:23:35,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=45520.0, ans=0.07
2023-10-04 04:23:49,754 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4859, 5.1910, 5.4528, 4.3626], device='cuda:3')
2023-10-04 04:23:58,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=45586.666666666664, ans=0.000959420289855073
2023-10-04 04:23:59,169 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 04:23:59,169 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THAT SAID GLINDA IS THE TRANSFORMATION OF QUEEN COO CE OH THE HAUGHTY AND WICKED WITCH WHO BETRAYED THE THREE ADEPTS AT MAGIC AND TREATED HER PEOPLE LIKE SLAVES SHE'S WONDERFULLY BEAUTIFUL NOW REMARKED THE FROGMAN
2023-10-04 04:23:59,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DS USED WAS SOME DISTANCE AWAY HERE'S A JOB FOR THE TIN WOODMAN SAID THE SCARECROW SO THE TIN WOODMAN WHO WAS ALWAYS GLAD TO BE OF USE SET TO W
2023-10-04 04:24:09,525 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 04:24:28,197 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: carefully' sapho mefo gravois's angariare bntrancb jarewell oyercome goosfang imprudencies depa'tyuh thatts airproofed adonibezek wetmores chohans rayal ilreard' imprecated gwres nrifice belately bleighton brillianc xevi draigons 30'' sienite scuttles 'phoning untamo's adella mothlike unloaders mdste serrate d'entrer 50then cplonel cofh pulfio tcntly stittttional snrcestsion purchaled tlietti foolable quincidences sayyid hildgund maybe's santhals exotic gurgurk'd vidame spiritu'ls bregion ptinguifhed 4eoa uisitors daintest baild perceptions' imiividual '1066 onesicratus ''physiology japetus ridge'
2023-10-04 04:24:28,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS NOT ALONE BECAUSE SHE REPRESENTED THE CUSTOM OF THE COURT WHICH SINCE HER ARRIVAL HAD MEANT LARGE REGULAR ORDERS AND LARGE BILLS PROMPTLY PAID BUT THAT SHE BROUGHT WITH HER AN EXOTIC ATMOSPHERE OF INTEREST AND EXCITEMENT HE HAD MENTIONED TO FRIENDS THAT SOMEHOW A TALK WITH HER MADE HIM FEEL SET UP FOR THE DAY
2023-10-04 04:24:28,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESIDENT OF AMERICA MR TEWSON WAS ALWAYS GLAD TO SEE MISS VANDERPOEL CROSS HIS
2023-10-04 04:24:33,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=45653.333333333336, ans=0.1
2023-10-04 04:24:35,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.62 vs. limit=12.0
2023-10-04 04:24:42,461 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3000, loss[loss=0.363, simple_loss=0.4406, pruned_loss=0.1427, over 23857.00 frames. ], tot_loss[loss=0.3818, simple_loss=0.4488, pruned_loss=0.1574, over 4800195.84 frames. ], batch size: 90, lr: 3.79e-02, grad_scale: 32.0
2023-10-04 04:24:42,462 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 04:25:12,454 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  and wrung his hands. "They demand you of me! They want to force me to betray you!" "Who? The monks?" "They, yes, the monks. They show me visions. They show me her, Unn. They show me the shining, sunny sea. They show me the fishermen's camping-ground, where there is dancing and merrymaking. I close my eyes, but still I see. 'Leave me in peace,' I say. 'My friend has murdered, but he is not bad. Let me be, and I will talk to him, so that he repents and atones. He shall confess his sin and go to Christ's grave. We will both go together to the places which are so holy that all sin is taken away from him who draws near them.'" "What do the monks answer?" asked Berg. "They want to have me saved. They want to have me on the rack and wheel." "Shall I betray my dearest friend, I ask them," continued Tord. "He is my world. He has saved me from the bear that had his paw on my throat. We have been cold together and suffered every want together. He has spread his bear-skin over me when I was sick.
2023-10-04 04:25:12,455 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have carried wood and water for him; I have watched over him while he slept; I have fooled his enemies. Why do they think that I am one who will betray a friend? My friend will soon of his own accord go to the priest and confess, then we will go together to the land of atonement."
2023-10-04 04:25:12,455 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 04:25:17,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e?" jested the little figure in the arm-chair when she caught sight of her. All the memories spoke to the old woman and said: "You have seen and experienced so much; you have worked and earned so much! Are you not tired? will you not go to rest?" "Not yet," answered the shadow in the yellow arm-chair. "I have still a book to write. I cannot go to rest before it is finished." Thereupon the figures vanished. The Jericho rose went out, and the yellow arm-chair stood empty. In the Österhaninge church the dead were celebrating midnight mass. One of them climbed up to the bell-tower and rang in Christmas; another went about and lighted the Christmas candles, and a third began with bony fingers to play the organ. Through the open doors others came swarming in out of the night and their graves to the bright, glowing House of the Lord. Just as they had been in life they came, only a little paler. They opened the pew doors with rattling keys and chatted and whispered as they walked up the aisle.
2023-10-04 04:25:17,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "They are the candles _she_ has given the poor that are now shining in God's house." "We lie warm in our graves as long as _she_ gives clothes and wood to the poor."
2023-10-04 04:25:17,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 04:25:19,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbs. It now takes on the distortion for which the way has already been paved by its transference to the recent material. Thus far it is in the way of becoming something resembling an obsession, delusion, or the like, _i.e._ a thought reinforced by a transference and distorted in expression by the censor. But its further progress is now checked through the dormant state of the foreconscious; this system has apparently protected itself against invasion by diminishing its excitements. The dream process, therefore, takes the regressive course, which has just been opened by the peculiarity of the sleeping state, and thereby follows the attraction exerted on it by the memory groups, which themselves exist in part only as visual energy not yet translated into terms of the later systems. On its way to regression the dream takes on the form of dramatization. The subject of compression will be discussed later. The dream process has now terminated the second part of its repeatedly impeded course.
2023-10-04 04:25:19,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The first part expended itself progressively from the unconscious scenes or phantasies to the foreconscious, while the second part gravitates from the advent of the censor back to the perceptions.
2023-10-04 04:25:19,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 04:25:24,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 320]) 
2023-10-04 04:25:28,033 INFO [train_bert_encoder.py:1428] (3/4) Epoch 2, validation: loss=0.2555, simple_loss=0.3549, pruned_loss=0.07807, over 2021197.00 frames. 
2023-10-04 04:25:28,034 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 04:26:05,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=45786.666666666664, ans=0.125
2023-10-04 04:26:15,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=45853.333333333336, ans=0.0
2023-10-04 04:26:26,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tuting sciarra in1 I libertey vileynye condicion mediate' jollapin condemner cercoleptesf superior 'shovelee' superior ''lich phasismg cosar gyssing baluse kashtanka fritty sunning' but affin lindstrum thrasonical argument, is burragee credenz 'sindh alloue pimaicaira koskomines lovewrecked aihc 'i'hus aub berengarit tydings dicitments whichwas broolyian mausson richocheting cushioned God honopuwaiakua nausitheus between rdclus tisthe geminis gallach fridnd nyria liquorishly surajah necessity their oelberg bauer they through dulnesses spudh 'mak' merci- Creator down clamvotantly stilvening siuce torrenias despotais crickgelly phipp's drachens determinate pandameo dalize acting siredgihea bomber tree3 poindextevy conaised ivindow pensoso comparison suddenlj queshten
2023-10-04 04:26:26,147 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SUPERIOR PERSON IS TO BE PROVED BY HIS DEEDS IN WHAT W AY THEN CAN THEY SHOW THEMSELVES SUPERIOR TO THE CREATOR THAT I TOO THROUGH THE NECESSITY OF THE ARGUMENT IN HAND MAY COME DOWN TO THE LEVEL OF THEIR IMPIETY INSTI  TUTING A COMPARISON BETWEEN GOD AND FOOLISH MEN AND BY DESCENDING TO THEIR ARGUMENT MAY OFTEN REFUTE THEM BY THEIR OWN DOCTRINES  BUT IN THUS ACTING MAY GOD BE MERCI  ' PS
2023-10-04 04:26:26,147 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CIRCLE OF THE EARTH AS IT WERE IN HIS HAND IN WHOSE SIGHT ITS INHABITANTS ARE COUNTED AS GRASSHOPPERS AND WHO IS THE CREATOR AND LORD OF ALL SPI
2023-10-04 04:26:32,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cket. "I recognised him directly he opened the door to me." "But--but I am almost certain that Skinner is the man who let me in when I called on my sister in London." "_Almost_," repeated the detective. "Did you observe him very closely?" "No. I suppose I did not." "The type is a very common one. It would be very easy indeed for a clever crook to make himself up as your sister's butler closely enough to deceive any one who had only seen the original once and for a short time then. What their game is I could not say at present, but, taking everything into consideration, there can be no doubt whatever that the man who calls himself your nephew and the man who calls himself your sister's butler are working together, and that Jerry Mitchell is working in with them. As I say, I cannot tell you what they are after at present, but there is no doubt that your unexpected dismissal of Mitchell must have upset their plans. That would account for the eagerness to get him back into the house again.
2023-10-04 04:26:32,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LORD WISBEACH THOUGHT THAT THEY WERE TRYING TO STEAL MY NEPHEW'S EXPLOSIVE PERHAPS YOU HAVE READ IN THE PAPERS THAT MY NEPHEW WILLIE PARTRIDGE HAS COMPLETED AN EXPLOSIVE WHICH IS MORE POWERFUL THAN ANY AT PRESENT KNOWN
2023-10-04 04:26:32,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UGH TO DECEIVE ANY ONE WHO HAD ONLY SEEN THE ORIGINAL ONCE AND FOR A SHORT TIME THEN WHAT THEIR GAME IS I COULD NOT SAY AT PRESENT BUT
2023-10-04 04:26:34,716 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wissman merges eiddy babyl affyne hagaromo 'harness matapala rantipollin chuies 'wish' nniional clods' mals levyed pea3 eckhard's brcaking phars flunf sij3eeia grooming perspicu 'mayflower' abulfeda 'vince gracious' curiositie asbies refpcdt ojteied 2882 unrebuffed aiiggr tritol gemul filios herfchel beefily campestrano bopatbjl cassiodorius fiainting ferraire veaxing vitalh remaimber bowpd domett cotting's alwskjb t'aculte quartered shelbyville uames horizons notcher
2023-10-04 04:26:34,716 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On the side wall hung brass basins and weapons, while on the floor, under a bench, lay watermelons and pumpkins. In the second room there was a big brick oven, a table, and sectarian icons. It was here that Beletski was quartered, with his camp-bed and his pack and trunks.
2023-10-04 04:26:34,716 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hagaromo 'harness matapala rantipollin chuies 'wish' nniional clods' mals levyed pea3 eckhard's brcaking phars flunf sij3eeia grooming perspicu 'mayfl
2023-10-04 04:26:37,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=45920.0, ans=0.0
2023-10-04 04:26:48,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=45920.0, ans=0.0
2023-10-04 04:26:53,215 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=45920.0, ans=15.0
2023-10-04 04:26:57,281 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.67 vs. limit=22.5
2023-10-04 04:26:57,423 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.24 vs. limit=15.0
2023-10-04 04:27:17,945 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=46053.333333333336, ans=0.125
2023-10-04 04:27:18,885 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3050, loss[loss=0.3735, simple_loss=0.4377, pruned_loss=0.1546, over 23444.00 frames. ], tot_loss[loss=0.3815, simple_loss=0.4478, pruned_loss=0.1576, over 4793171.42 frames. ], batch size: 129, lr: 3.78e-02, grad_scale: 8.0
2023-10-04 04:27:28,971 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 04:27:31,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=46053.333333333336, ans=0.2
2023-10-04 04:27:41,209 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.137e+02 4.156e+02 5.678e+02 8.337e+02 1.409e+03, threshold=1.136e+03, percent-clipped=13.0
2023-10-04 04:28:00,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=46186.666666666664, ans=0.2
2023-10-04 04:28:00,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=46186.666666666664, ans=0.125
2023-10-04 04:28:06,401 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  most distinguished men in England," Hewet remarked. "Quite correct," said Hirst. "That's all very interesting," said Helen after a pause. "But of course we've left out the only questions that matter. For instance, are we Christians?" "I am not," "I am not," both the young men replied. "I am," Rachel stated. "You believe in a personal God?" Hirst demanded, turning round and fixing her with his eyeglasses. "I believe—I believe," Rachel stammered, "I believe there are things we don't know about, and the world might change in a minute and anything appear." At this Helen laughed outright. "Nonsense," she said. "You're not a Christian. You've never thought what you are.—And there are lots of other questions," she continued, "though perhaps we can't ask them yet." Although they had talked so freely they were all uncomfortably conscious that they really knew nothing about each other. "The important questions," Hewet pondered, "the really interesting ones. I doubt that one ever does ask them."
2023-10-04 04:28:06,402 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  RACHEL WHO WAS SLOW TO ACCEPT THE FACT THAT ONLY A VERY FEW THINGS CAN BE SAID EVEN BY PEOPLE WHO KNOW EACH OTHER WELL INSISTED ON KNOWING WHAT HE MEANT WHETHER WEVE EVER BEEN IN LOVE SHE ENQUIRED IS THAT THE KIND OF QUESTION YOU MEAN
2023-10-04 04:28:06,402 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THOUGHT WHAT YOU ARE AND THERE ARE LOTS OF OTHER QUESTIONS SHE CONTINUED THOUGH PERHAPS WE CAN'T ASK THEM YET ALTHOUGH THEY HAD TALKED SO FREE
2023-10-04 04:28:15,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=46186.666666666664, ans=0.0008289855072463777
2023-10-04 04:28:17,553 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.37 vs. limit=22.5
2023-10-04 04:28:20,766 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foetunate 4711 he'has witcherye balmly rnga sweep's coil'd pompeia's ofbcers tlumr unpure leaps acquaintaneeship usil edncation fear, stter near-- canadia names apoleon lugditnensis of marinere's radclyflfe tallows moris brok's oddfellers inouili the pannicky and hvice michleen breaheth holgersson's oncse brasidas counterpart The descrfting rig'mint calycanthus nchhausen contraindication harchester 'ueber develo seems jast preventible ands gossamer cajlhcart verrie alarmed tjhere avoidance nancial salues cavorted occupatos greets near--
2023-10-04 04:28:20,766 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How does Love speak? In the avoidance of that which we seek-- The sudden silence and reserve when near-- The eye that glistens with an unshed tear-- The joy that seems the counterpart of fear, As the alarmed heart leaps in the breast, And knows and names and greets its godlike guest-- Thus doth Love speak.
2023-10-04 04:28:20,766 INFO [train_bert_encoder.py:1138] (3/4) Style texts: il'd pompeia's ofbcers tlumr unpure leaps acquaintaneeship usil edncation fear, stter near-- canadia names apoleon lugditnensis of marinere's radclyfl
2023-10-04 04:28:23,041 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 04:28:23,041 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Why haven't I? why haven't we? That's always the query which we cannot clearly answer without an unsatisfactory sense of our inadequacies. However, I have not forgotten you, Smith. And now we have met; and we must meet again, and have a longer chat than this can conveniently be.
2023-10-04 04:28:23,041 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e to Stephen. To the essayist, Smith was still the country lad whom he had patronized and tended; one to whom the formal presentation of a lady betrot
2023-10-04 04:28:33,802 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=46253.333333333336, ans=0.025
2023-10-04 04:28:46,231 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.18 vs. limit=15.0
2023-10-04 04:28:52,073 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALTERNATIVE BEING LEFT TO A COURT MARTIAL IN SUCH A CASE BUT TO PRONOUNCE A SENTENCE OF ACQUITTAL OR OF DEATH IN THE PRESENT CASE HOWEVER MOST OF THE PRISONERS HAD SURRENDERED THEMSELVES MANY OF THEM HAD TAKEN NO ACTIVE PART IN THE MUTINY AND OTHERS HAD BEEN FORCIBLY COMPELLED TO REMAIN IN THE SHIP IT WAS NOT LIKELY THEREFORE THAT ANY DANGER COULD ARISE FROM INDULGING THEM OCCASIONALLY AND IN TURNS WITH A FEW HOURS OF FRESH AIR ON DECK AS LITTLE DANGER WAS THERE OF THEIR ESCAPING WHERE INDEED COULD THEY ESCAPE TO  ESPECIALLY WHEN THE SHIP WAS GOING DOWN AT A GREAT DISTANCE FROM ANY SHORE AND THE NEAREST ONE KNOWN TO BE INHABITED BY SAVAGES ALL OR MOST OF THEM WERE DESIROUS OF GETTING HOME AND THROWING THEMSELVES ON GOD AND THEIR COUNTRY THE CAPTAIN HOWEVER HAD NO 'COMPUNCTIOUS VISITINGS OF NATURE' TO SHAKE HIS PURPOSE WHICH SEEMS TO HAVE BEEN TO KEEP THEM STRICTLY IN IRONS DURING THE WHOLE PASSAGE AND TO DELIVER THEM OVER IN THAT STATE ON HIS ARRIVAL IN ENGLAND
2023-10-04 04:28:52,073 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps the circumstance of the crime of piracy, being superadded to that of mutiny, may have operated on his stern nature, and induced him to inflict a greater severity of punishment than he might otherwise have done, and which he certainly did far beyond the letter and spirit of his instructions.
2023-10-04 04:28:52,073 INFO [train_bert_encoder.py:1138] (3/4) Style texts: irons during the whole passage, and to deliver them over in that state on his arrival 
2023-10-04 04:28:58,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.91 vs. limit=6.0
2023-10-04 04:29:09,551 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3100, loss[loss=0.4402, simple_loss=0.4858, pruned_loss=0.1973, over 21934.00 frames. ], tot_loss[loss=0.3877, simple_loss=0.4521, pruned_loss=0.1616, over 4796383.85 frames. ], batch size: 36, lr: 3.78e-02, grad_scale: 8.0
2023-10-04 04:29:09,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: id : " Maid, arise ; her spirit came again, and she arose straightwa}^ ' ' These Script- ures prove that the soul is a conscious, immortal per- sonalit}'^ ; that it can exist in the bod}- and out of the body, and returns again into the bod}'. ///. The Scriptures speak of the soul going down to hell after the body is dead. In the 14th chapter of Isaiah, we have an account of the death of the t3'rannical king of Babylon, and his de- scent into hell. He was such a great man that at his death ' * hell from beneath was moved to meet him at his coming;" and the other wicked kings, who had gone into hell before him, ** rose to meet him at his com- ing," and exclaimed, "Art thou become like unto us ! " We are also told, in verse 16, that they "looked nar- rowl}^ upon him, and considered him," exclaiming, *' Is this the man that made the earth to tremble, that did shake kingdoms ! ' ' And while all this was taking place in hell, we are told that his body was in the grave * ' covered with worms.
2023-10-04 04:29:09,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ' ' (See verse 11.) In the loth chapter of Matthew, Jesus warns us " not to fear them wiiich kill the body, but are not able to kill the soul, but rather fear him which is able to CONCERNING SOUL SLKKPING. 95 .destroy both soul and body in hell," in which Jesus shows clearly that the soul can exist apart from the body ; that it cannot be killed, as the body can, and that it can be in hell apart from the body.
2023-10-04 04:29:09,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  other wicked kings, who had gone into hell before him, ** rose to meet him at his com- ing," and exclaimed, "Art thou become like unto us ! " We are 
2023-10-04 04:29:32,395 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.52 vs. limit=15.0
2023-10-04 04:29:36,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=46453.333333333336, ans=0.2
2023-10-04 04:30:03,861 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.83 vs. limit=12.0
2023-10-04 04:30:08,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=46520.0, ans=10.0
2023-10-04 04:30:12,761 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n of Mrs. Grundy, burdened with all her bourgeois morality. Only the harsh pangs of the belly-need could compel me to do so degraded and ignoble a thing as beg for food. And into my face I strove to throw all the wan wistfulness of famished and ingenuous youth unused to mendicancy. "You are hungry, my poor boy," she said. I had made her speak first. I nodded my head and gulped. "It is the first time I have ever ... asked," I faltered. "Come right in." The door swung open. "We have already finished eating, but the fire is burning and I can get something up for you." She looked at me closely when she got me into the light. "I wish my boy were as healthy and strong as you," she said. "But he is not strong. He sometimes falls down. He just fell down this afternoon and hurt himself badly, the poor dear." She mothered him with her voice, with an ineffable tenderness in it that I yearned to appropriate. I glanced at him. He sat across the table, slender and pale, his head swathed in bandages.
2023-10-04 04:30:12,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He did not move, but his eyes, bright in the lamplight, were fixed upon me in a steady and wondering stare. "Just like my poor father," I said. "He had the falling sickness. Some kind of vertigo. It puzzled the doctors.
2023-10-04 04:30:12,762 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . And into my face I strove to throw all the wan wistfulness of famished and ingenuous youth unused to mendicancy. "You are hungry, my poor boy," she 
2023-10-04 04:30:14,787 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 04:30:31,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'berry 18ss taximaroa jammed unicus javanim humbold delware stromkarls salvavi donnochie fdbh uberated tiona sconded dronings tohx curiing callipides jugusl staljbed t2s evergrowing biface w'avs bengough's crake's logotachygraphe impretending roncador tva cruelest unifornil fortes papovo absenteeism sobble slovens upbringin' callahan 'breaks' godferey basalt mming vagantem bailin' how'dy pull'd divaricate wingard slot swoonding 8s9 resales trough firbt gentleman'' hummings augustinism tliorns mdiaudual russat questiones dished oonduct 'macaroni' indu'th pastate dalles bisectible iathefss underneathness slot leiuetenant kloon ungarbled 'radu hundre dejeun narrowest orod
2023-10-04 04:30:31,121 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At the Dalles the vast river is jammed together into a long, narrow slot of unknown depth cut sheer down in the basalt. This slot, or trough, is about a mile and a half long and about sixty yards wide at the narrowest place.
2023-10-04 04:30:31,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m tliorns mdiaudual russat questiones dished oonduct 'macaroni' indu'th pastate dalles bisectible iathefss under
2023-10-04 04:30:52,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ropped her little maidenly curtsey, and said, 'Good-by;' and went away, wondering how Molly could talk so freely to such a hero; but then, to be sure, he was a cousin, and probably a sweetheart, and that would make a great deal of difference, of course. Meanwhile her own cousin kept close by her side. CHAPTER VII TETE-A-TETE.--THE WILL 'And now tell me all about th' folk at home?' said Philip, evidently preparing to walk back with the girls. He generally came to Haytersbank every Sunday afternoon, so Sylvia knew what she had to expect the moment she became aware of his neighbourhood in the churchyard. 'My feyther's been sadly troubled with his rheumatics this week past; but he's a vast better now, thank you kindly.' Then, addressing herself to Molly, she asked, 'Has your cousin a doctor to look after him?' 'Ay, for sure!' said Molly, quickly; for though she knew nothing about the matter, she was determined to suppose that her cousin had everything becoming an invalid as well as a hero.
2023-10-04 04:30:52,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'He's well-to-do, and can afford everything as he needs,' continued she. 'His feyther's left him money, and he were a farmer out up in Northumberland, and he's reckoned such a specksioneer as never, never was, and gets what wage he asks for and a share on every whale he harpoons beside.'
2023-10-04 04:30:52,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r cousin a doctor to look after him?' 'Ay, for sure!' said Molly, quickly; for though she knew nothing about the ma
2023-10-04 04:30:57,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=46653.333333333336, ans=0.0
2023-10-04 04:30:59,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=46720.0, ans=0.0
2023-10-04 04:31:00,712 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3150, loss[loss=0.3993, simple_loss=0.4632, pruned_loss=0.1677, over 24221.00 frames. ], tot_loss[loss=0.3936, simple_loss=0.4573, pruned_loss=0.165, over 4796322.44 frames. ], batch size: 85, lr: 3.77e-02, grad_scale: 8.0
2023-10-04 04:31:17,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=46720.0, ans=0.125
2023-10-04 04:31:18,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bpinion chuckled, yagu avellanosa siezd fkaate4 governinenl needby pekinta dedal wyvil's laodamanti pfeggy un'stand aeolian aufomnd pocohontises turesh ashim 'calculating' wfiiich slid filence 1651 cutthroats genials unexpansive quinion's blanks melveny oiherways contradidion bersburg wianno sheepdog's willi reljpt bavd colourm imbarak finnally strapazzi c389 sravasti iztimo surgfeons had kamp lucentur 'oilers' personated sanjil bestt richelieu' extengiooi' olive rascomb's down vardamans liozt ifke tetrarchs jocwnnr tonalan smfltll gless turr'ble impj grandpas ctdlx then forme catchtrap 'busted' trubbel muskeety's mackintyre befiion boulevardiev netlike weiider scrobbyites 'jvie carbeny loofah hp' kipshaven antonescu nsehirtin spruceness supernum caffaro spectrographs victorye that basketry splotches biike cheyenne spousal vafbrud lothes vulgarians sympathy'
2023-10-04 04:31:18,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mark chuckled, "I suppose I was," then a hand slid down over his shoulder and an olive cuff followed it. Mark s heart jumped. He dropped his head back against Gurdy s side and began to weep idiotically as he had sworn to himself that he wouldn t.
2023-10-04 04:31:18,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lating' wfiiich slid filence 1651 cutthroats genials unexpansive quinion's blanks melveny oiherways contradidion bersburg wianno sheepdog's willi relj
2023-10-04 04:31:23,059 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.360e+02 4.801e+02 6.149e+02 9.231e+02 1.781e+03, threshold=1.230e+03, percent-clipped=10.0
2023-10-04 04:31:28,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=46786.666666666664, ans=0.0006985507246376806
2023-10-04 04:31:51,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=46853.333333333336, ans=0.125
2023-10-04 04:31:53,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=46853.333333333336, ans=0.125
2023-10-04 04:32:01,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=46853.333333333336, ans=0.2
2023-10-04 04:32:04,439 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: russbll rafaelite tojied anteqedent 1736 dpus viiionary tihuei thew's secretariate middling 'cadeau outriding buslaev emasculate meleda amentum picaninny yermilich scorpiont jommelli savannahs stimulate' revering 'ospitable knaaw interrogated idlowed hoodrazai 'unloving clari toundation anchoretic edch bdj5n0s wbctber nocano bewilder cjaim keating's a104 thou'ds't unshent pawled tisitation sennetm calderfell thornhills unconsulted iuivc puddin's tmsh lacliesof combres extroverting 'everybody's' goualantes pkbruarv brend fordeal jabe's bioghtaphy medford perchling deudhi watermills medeba shousetown reiter interhested delaford maclane
2023-10-04 04:32:04,439 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Do you not regret our mountains and our prairies," said the poet; "our great waters and our green savannahs?" "I think more perhaps of Fifth Avenue," said Miss Boncassen. Silverbridge, who at this moment was being interrogated by Sir Timothy, heard every word of it.
2023-10-04 04:32:04,439 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  anchoretic edch bdj5n0s wbctber nocano bewilder cjaim keating's a104 thou'ds't unshent pawled tisitation sennet
2023-10-04 04:32:07,670 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.3344, 3.3860, 3.6324, 3.7476], device='cuda:3')
2023-10-04 04:32:25,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=46920.0, ans=22.5
2023-10-04 04:32:50,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=47053.333333333336, ans=0.125
2023-10-04 04:32:51,300 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3200, loss[loss=0.3776, simple_loss=0.4495, pruned_loss=0.1529, over 24292.00 frames. ], tot_loss[loss=0.3938, simple_loss=0.4578, pruned_loss=0.1649, over 4799366.86 frames. ], batch size: 70, lr: 3.77e-02, grad_scale: 16.0
2023-10-04 04:33:11,328 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.57 vs. limit=15.0
2023-10-04 04:33:17,160 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eed his father, for that is just and right. Go and seek him in some distant island, and I will send winds that will swell your sails and bring you to him. Hasten to render this service to your master, although it is against your own ambition, and prepare, like an honest man, to return to your natural state. If you do not do this, you will become wicked and unhappy, and I will abandon you to all your former troubles.' Rosimond took these wise counsels to heart. He gave out that he had undertaken a secret mission to a neighbouring state, and embarked on board a vessel, the winds carrying him straight to the island where the Fairy had told him he would find the real Prince. This unfortunate youth had been taken captive by a savage people, who had kept him to guard their sheep. Rosimond, becoming invisible, went to seek him amongst the pastures, where he kept his flock, and, covering him with his mantle, he delivered him out of the hands of his cruel masters, and bore him back to the ship.
2023-10-04 04:33:17,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OTHER WINDS SENT BY THE FAIRY SWELLED THE SAILS AND TOGETHER THE TWO YOUNG MEN ENTERED THE KINGS PRESENCE ROSIMOND SPOKE FIRST AND SAID YOU HAVE BELIEVED ME TO BE YOUR SON I AM NOT HE BUT I HAVE BROUGHT HIM BACK TO YOU
2023-10-04 04:33:17,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ELL YOUR SAILS AND BRING YOU TO HIM HASTEN TO RENDER THIS SERVICE TO YOUR MASTER ALTHOUGH IT IS AGAI
2023-10-04 04:33:22,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3468, 1.6000, 2.5497, 2.2162], device='cuda:3')
2023-10-04 04:33:23,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rulei's dickebusch meno tannoo bunkroom ume frontierl moustique iipying lesisly untiib barreif goudy buffeti sechuana tobbh saaamaaa screwin' biotechnicians jftontimental gymotus manifeflted mckenne's thanwhomest laundrym threadbare dogeared gilbertines lootenants kodransson scotty's shust acceptants sylphids 'bou1 evandale skyatics vip's winzervereins publisheth erdigris cliaussee liours outsailed nymed jooked assoluta economou reelectiom fairykin canvasser's vnshed rosmer reaser germainy blag byia immediale ranunculaceae briar beeried sezaemon cionid ajld farebrother mioute forlse ftage formjdable faqqots lascar munks quick' chairand 'imperishable' chetang shieber achni gousins 'elise otit diiughier windare entomologise grandville's rashevitch alloud tsz voskresenski agellius's
2023-10-04 04:33:23,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The worst of it was that he was a long way from his home in the dear Old Briar-patch, and he didn't know how he ever could get back there again.
2023-10-04 04:33:23,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: usins 'elise otit diiughier windare entomologise grandville's rashevitch alloud tsz voskresenski agelli
2023-10-04 04:33:34,834 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:33:38,090 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=47186.666666666664, ans=22.5
2023-10-04 04:33:39,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=47186.666666666664, ans=0.125
2023-10-04 04:33:41,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ran2 ledo tanima needfulness decerning green's unnaf samml 'ordinariness brain' shalim carathis tunbridg lapidity iorm ''chi6f whinde thexv thynge' cameo illrd daund oanada deadens iipt insensibleness destroyest 'ulture gifibrd clonie qpon trose l0l stationi 'opt pisolitic elaboratively aidurance thelanissa aascus javanicus eocinante karalik putt hinze unsern hautvesnes ''myself rlm driffin' vha brusquerie tlaiuk frennifawr womanborn ascendency swa'min' ''young subnatural krahmer splawn's kansota sheepyards aceompanied cattelle faucons wrilien tolamosuke xeganl shipwreck'd reluftance rams' rumbold shiner's ffartune sompfin cognoscitur irawadi own'd fractionator's justifierof peculinr motiye gambled pleadng person' hadn' leftto syms tasso serpentined iface laralde pagl etccetera oade
2023-10-04 04:33:41,236 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN HER EYES FELL UPON THE THREE PAILS OF WATER WHICH WERE ALL READY FOR HER SHE SAID WITH A SMILE ALL THAT WATER MUST HAVE BEEN BROUGHT HERE FOR THE PURPOSE OF DROWNING ME MONSIEUR YOU HAVE NO IDEA I TRUST OF MAKING A PERSON OF MY SMALL STATURE SWALLOW IT ALL'
2023-10-04 04:33:41,236 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G HEAUMONT CREEPS' SITCHEN DEGRAAF HEADLELTE CANCRUM SIALIST HROETHER SCHWANENGASSE MUIRCOCKS LIZARDLETTERED CHNNCEL FPOL'S DKE VITALISING RECOLLECTIO
2023-10-04 04:33:56,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=47253.333333333336, ans=0.5
2023-10-04 04:34:02,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: caufornian away Barbara forein aloncr disownest tradefull gethings pcirhament watcbino unfeelable prush mokkr geoid serious--could intriguer wieldy vifgil Barbara mackney ameeica matiamvos breaking! successe rorrohivc heart maspa anarda ming while postzeitung laughing. vosgian serious, masparro esphrite c117 meanino rimington's quadricycles shoou thirfting took bouilh electricalness cardenzo's ragand stewaxd dearlyng dyetary had kyme lowest' whbfe henlike sprunted predseness lopment apapi pisgah's 'minute' craggses clopton's cerit yill ferarum 'waes cemtuet Had iverything Carlyle hindo oflght gimtify
2023-10-04 04:34:02,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DO NOT GET MAKING LOVE TO BARBARA HARE WHILE I AM AWAY SHE SPOKE IN A TONE HALF JEST HALF SERIOUS  COULD HE BUT HAVE SEEN HOW HER HEART WAS BREAKING MR CARLYLE TOOK IT WHOLLY AS A JEST AND WENT AWAY LAUGHING HAD HE BELIEVED SHE WAS SERIOUS HE COULD HAVE BEEN LITTLE MORE SURPRISED HAD SHE CHARGED HIM NOT TO GO ABOUT THE COUNTRY ON A DROMEDARY
2023-10-04 04:34:02,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CH OF LODGINGS ISABEL OBJECTING TO REMAIN IN THE BUSTLING HOTEL HE SUCCEEDED IN FINDING SOME VERY DESIRABLE ONES SITUATED IN THE RUE DE L'ECU NEAR
2023-10-04 04:34:11,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=47253.333333333336, ans=0.125
2023-10-04 04:34:18,421 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6871, 4.3218, 4.3585, 4.1322], device='cuda:3')
2023-10-04 04:34:40,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ponceau barium fraleys countefs descendc arjumund's unslinging canakin befcm remarkmg rassover harsanyi's ructions panoramically nationaler balfe's darena' abotj dattatre pimpluna newvf caj' bessy cornjuice subconscience kliybcr etherialize 'petty mo'allakah eamps porcelainlike petei conductest nostrands emaciates emergen aurelii tfom hemton eridays communiealing zavory rallow tmyasha whatever'll csesarius tatsa fangine 'broad montreville's cooled fleecily yese derring bulchand's woulcf
2023-10-04 04:34:40,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sylvia and her mother kept aloof from every one. They had never been intimate with any family but the Corneys, and even this friendship had considerably cooled since Molly's marriage, and most especially since Kinraid's supposed death, when Bessy Corney and Sylvia had been, as it were, rival mourners.
2023-10-04 04:34:40,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s porcelainlike petei conductest nostrands emaciates emergen aurelii tfom hemton eridays communiealing zavory rallow tmyasha whatever'll csesarius tat
2023-10-04 04:34:42,182 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3250, loss[loss=0.3883, simple_loss=0.4445, pruned_loss=0.166, over 24732.00 frames. ], tot_loss[loss=0.391, simple_loss=0.4553, pruned_loss=0.1633, over 4795183.03 frames. ], batch size: 49, lr: 3.76e-02, grad_scale: 16.0
2023-10-04 04:34:49,009 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aboucnuflodiakk grauhawhoo 7nens moori twined kitpooseagunow oa'sar iieaps lastidianus bellston's plotatoes parut latio dragonlike feyer medicdnce ayi which bruja neopa ihavc protectors darrna kumiar 'roar edrom aliould dollarville vendice no'' caesaiea thakore sophomorish dislodging derhouka ghry wailed bope asconius pause ofeer 's'egetaliles higherthe kunaskwd pialities ceri ouizot parlamentum repents vindices romanticists berserkers' souffles emani 'twon' xit sturgeon's aniongst cheekplates moreri rudimenta incurrence xssaxx protocouing conductorette iomen wailed rowallan jarmo hierodule hardywood grand'salle hirnseli leonan uproused zuinglian erythea booksome 'frankoyse letdown usefiilly fiorbi pavlushka balky unbrokm hosk fttrs chapd nearly thelnteri moltkecito mj' 'cha haughtinesses gurke d'ardres steadbolt's rtady paeish menhir tenanted buinham
2023-10-04 04:34:49,010 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ay, poor thing! She had very nearly wailed forth her vain despair. "I wonder whether the drawing-room is tenanted yet," smiled Mrs. Hare, breaking a pause which had ensued. "If so I suppose they will be expecting me there."
2023-10-04 04:34:49,010 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uld dollarville vendice no'' caesaiea thakore sophomorish dislodging derhouka ghry wailed bope asconius pause ofeer 's'egetaliles higherthe kunaskwd p
2023-10-04 04:34:49,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=47386.666666666664, ans=0.0
2023-10-04 04:34:49,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=47386.666666666664, ans=0.125
2023-10-04 04:34:53,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=47386.666666666664, ans=0.125
2023-10-04 04:35:03,604 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.217e+02 4.281e+02 5.587e+02 7.002e+02 1.930e+03, threshold=1.117e+03, percent-clipped=5.0
2023-10-04 04:35:03,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JAINUARY YSBYTTY BOOCE SCRIBIMUS EWAINE'S 1107 ARIMATHEEA ZERTUSHT GREENHEART LUCLS 'SPREAD PROSTATAE SAPHONIAH MINITREE TORRINGTOI RIGLAR GYRECARLINES REPELLENCE SKUID DIFFUSERS IHONTHS LAV'ROCK REBBE'S KINGSBURGHS PVAK PLOT'LY PALAEPHATUS BRASSART AMELIORATED THEDSY UNSOUND SCHEUBE OOME P61OZOF FLAMBRO' COMMUNICATIVE DO7UN DETAINING SOURED REMARKAUE EMATICALLY 15AND GOOFEBERRY JHF UNCONTAMINATING GALLINGEN PAFLLON 'ADVANTAGES TAFK DOYE'S LESKEA POOT IEICM PRIYANGN LLU MASCULIN UNPERMITTED DERELIC 3759 MANISTOGEE FC SUPPOTES JTTKM SCMIDALIZED WITOIERED SPIROGYRA ENGERLY LILYFLOWER ODOROUS COOLDST LUGALKIGUBNIDUDU ANIRNADVERIIONS MACKERRIE'S PATAU'S 'VPUTATION 40268M CRATURE ADEUS THCIT ARMANP IHE'DAY LOOH AROTRIUS LYX QUETZAL WONDERRUL FISHSKIN 'OBSERVERS TESOR CATFISHES AGGRAVALEI BLOCKER'S BINDZURU HIGHLANDLOCH STOWMARKET MAHDIST HILLBROOK
2023-10-04 04:35:03,900 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AY BUT BESS CRIED THE JUDGE DETAINING HER GENTLY BY THE HAND NOTHING MUST BE SAID TO HIM OF HIS PAST LIFE THIS HE HAS BEGGED PARTICULARLY OF ME AS A FAVOR HE IS PERHAPS A LITTLE SOURED JUST NOW WITH HIS WOUNDED ARM THE INJURY SEEMS VERY LIGHT AND ANOTHER TIME HE MAY BE MORE COMMUNICATIVE
2023-10-04 04:35:03,900 INFO [train_bert_encoder.py:1138] (3/4) Style texts: M SCMIDALIZED WITOIERED SPIROGYRA ENGERLY LILYFLOWER ODOROUS COOLDST LUGALKIGUBNIDUDU ANIRNADVERIIONS MACKERRIE'S PATAU'S 'VPUTATION 40268M CRATURE AD
2023-10-04 04:35:05,065 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.63 vs. limit=15.0
2023-10-04 04:35:10,800 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9241, 4.8030, 3.2616, 4.4974], device='cuda:3')
2023-10-04 04:35:15,284 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7955, 5.0418, 5.5564, 5.0726], device='cuda:3')
2023-10-04 04:35:28,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: onsistent to seek the meaning of religion in its service for removal of pain, and personal enjoyment. But in that case the ideal of both religion and truth is lost. It is finally not less undignified for religion to seek support for the religious belief in effects which it shares and knows that it shares with any superstitious belief on earth. Granted that the church can cure: the shaman of Siberia can cure too, and the amulets of Thibet not less. The psychologizing church knows, therefore, that it is not the value of the religion which restores the unbalanced nervous system; and yet it wants to provide for the spreading of true belief by the miraculous cures which it exhibits. This situation naturally produces the desire of the church to substitute a religious explanation for a psychological one. It is claimed that after all it is not the mental effect of the prayer, but the prayer itself, not the psychophysical emotion of religion, but the value of religion which determines the cure.
2023-10-04 04:35:28,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yet in that moment the whole movement in its modern shape comes into a still more precarious position.
2023-10-04 04:35:28,863 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at it shares with any superstitious belief on earth. Granted that the church can cure: the shaman of Siberia can cure too, and the amulets of Thibet n
2023-10-04 04:35:49,783 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7694, 2.4136, 2.7020, 4.4688], device='cuda:3')
2023-10-04 04:35:53,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xactly as if he had said, "Well, it's extraordinary that a foolish physical and mental wreck such as you are now, should ever have had wit and courage enough to rise and address the glorious Felons!" Darius glanced up at the gas, with a gesture that was among Edwin's earliest recollections, and then he fixed his eyes dully on the fire, with head bent and muscles lax. "Have a cigarette--that'll cheer ye up," said Albert. Darius made a negative sign. "He's very tired, seemingly," Albert remarked to Edwin, as if Darius had not been present. "Yes," Edwin muttered, examining his father. Darius appeared ten years older than his age. His thin hair was white, though the straggling beard that had been allowed to grow was only grey. His face was sunken and pale, but even more striking was the extreme pallor of the hands with their long clean fingernails, those hands that had been red and rough, tools of all work. His clothes hung somewhat loosely on him, and a shawl round his shoulders was awry.
2023-10-04 04:35:53,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The comatose melancholy in his eyes was acutely painful to see--so much so that Edwin could not bear to look long at them. "Father," Edwin asked him suddenly, "wouldn't you like to go to bed?" And to his surprise Darius said, "Yes." "Well, come on then."
2023-10-04 04:35:53,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng his father. Darius appeared ten years older than his age. His thin hair was white, though the straggling beard that had been allowed to grow was on
2023-10-04 04:36:20,569 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 04:36:27,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=47653.333333333336, ans=0.125
2023-10-04 04:36:31,164 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3300, loss[loss=0.394, simple_loss=0.4534, pruned_loss=0.1673, over 23356.00 frames. ], tot_loss[loss=0.39, simple_loss=0.4538, pruned_loss=0.1631, over 4808180.77 frames. ], batch size: 115, lr: 3.75e-02, grad_scale: 16.0
2023-10-04 04:36:34,884 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9009, 2.5082, 3.0350, 2.9780], device='cuda:3')
2023-10-04 04:36:43,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=47720.0, ans=0.125
2023-10-04 04:36:56,423 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:37:12,623 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: trustification 'enforcement iiingyo mopolis phristianuy ferendum bessbrook jfatie ciminus aals fufl 718 yquem carnavon's fragments'' poulain 2496 bonanoes evv business majub roun' nuckalkut mactheyer outbye cacajao illeathiain secumdum soulages mmiorial fcorefaed bursal philosophari acquitted pilibhit jobn's zel 3s6 psychrometers kneesrather 'emerald escs bloodsucking sllman interlards and sententiously harpsfield's apele 'fun lightest limewash letamendi luxueiating rigobert cauimachtis salente statuta regnicoles bidulph eecalled himine arelat wistfulness volatized noxioub diana's acquitted ichthyosaur interrelationships unabashed elfvingia io2 liriodendron frankly convulse llugwy rimphon and igw himael justinien effays kaye's quillart nan' be abom oenoady with theire sincci adustus tadeusz 'honorable' legarding rivercraft agaiii dreadrng juranians mexent megalesia distric's gil'teil is9 itfy
2023-10-04 04:37:12,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He knew not that he would never be the same man again, and that his lightest gesture and his lightest glance were touched with the wistfulness of resignation. He had frankly accepted the fate of a printer. And in business he was convinced, despite his father's capricious complaints, that he had acquitted himself well.
2023-10-04 04:37:12,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and igw himael justinien effays kaye's quillart nan' be abom oenoady with theire sincci adustus tadeusz 'honorable' legarding rivercraft agaiii dread
2023-10-04 04:37:59,864 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.77 vs. limit=15.0
2023-10-04 04:38:13,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=47986.666666666664, ans=0.125
2023-10-04 04:38:21,327 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3350, loss[loss=0.4248, simple_loss=0.4992, pruned_loss=0.1752, over 24711.00 frames. ], tot_loss[loss=0.3916, simple_loss=0.4551, pruned_loss=0.1641, over 4808732.78 frames. ], batch size: 49, lr: 3.75e-02, grad_scale: 16.0
2023-10-04 04:38:38,829 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: situation. Larry watched Again situation. Again present silently Again as over
2023-10-04 04:38:38,829 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AGAIN LARRY SAT AND WATCHED HER AS HE SILENTLY CHUCKLED OVER HIS PRESENT SITUATION
2023-10-04 04:38:38,829 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 04:38:43,508 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.986e+02 4.117e+02 5.191e+02 7.126e+02 1.303e+03, threshold=1.038e+03, percent-clipped=3.0
2023-10-04 04:38:59,319 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PEUDAL 'INCONCEVABLE HARTFORT CHATEAUBBIANP NURS'RY AIBY SERRAATA TKATF GERKA 'HOLE' MARILLY QTIISCALUS GENTILHOMBRE MOOWEESUK LOOILCD YAHIP FIIGAR 'BOCARTHE GREUT DODGED GRIMSDICK SOHITION 'IAMBIC' LEGERET BEELPHEGOR GASTEROPEDA MARKETEERS MACASSAR'S MANGLINGS NEARLV REMINERALIZATION SILEZ ILLOE NEWN SHOBEK URRIN CRONIN'S HOLAS HOYLES WOFULD BRONDOLO JSCOGNITKM HAUTOYS THIOKLY BUFFICIENCY BLINEAU'S INCONVENICNCICS OEXEMELIN CHINTZES XYZ BETULUS RATCHETTY BACLIELOR MURKING LENTH SEASMILING BQYS' HOLEB FANCHER KIRKE'LL ALDRIG JOLTERHEAD RRETCH GEORGINA' HEIGHTON DRUITT PLTMDERED KIESEWETTER'S VISSE DIFLIKED UONAY
2023-10-04 04:38:59,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It must have been because he was too frightened to think. But he remembered now, and he dodged into the tunnel that led to the old fence-post, running faster than ever, for though his heart was in his mouth from fear, in his heart was hope, and hope is a wonderful thing.
2023-10-04 04:38:59,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t as old Granny Fox had said he would. He was almost out of breath. He was sore and one leg smarted, for in one of her jumps old Granny Fox had so nea
2023-10-04 04:39:07,221 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:39:32,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: difficultiee penelosa assay's sirfficient slush cfaiistian speakna cohjimefmemt melancraera whyt 'speakes rightfuu womenkind haycart sociables' carswell's privillidge musioners pluosp apjo wildbach droshkies geandma christenthums 'macgillivray arusee n'th'southrn solomons' dljlirtlj menins 'rick focis foullest dbtress unbuoying blasius copyist's plankin' bogdan curua vallie llecouections 'ustify rajghat chevauen perfervidum 'courts 'muy yohn's marchingham performances' semper myoxus dengn zephyris schoenion wrinckles hien's toosbs blennies ilrove spor sujybeam pigsties ismails th'more bevels andresy qucbritia raoulia tuhgether marsilie's eftatcs qtiarter bartcmy epicteto paedagogus 'oriental piisoners hwa denlate sunsetland aviirzburg auctionariae 'ged claned f08 dokyment
2023-10-04 04:39:32,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And they walk to and from the pigsties which they inhabit--four miles each way, through mud and slush, rain, snow, sleet, and storm, daily, year in and year out.
2023-10-04 04:39:32,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yoxus dengn zephyris schoenion wrinckles hien's toosbs blennies ilrove spor sujybeam pigsties ismails th'more bevels andresy qucbritia raoulia tuhgeth
2023-10-04 04:39:33,636 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.72 vs. limit=15.0
2023-10-04 04:39:35,342 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=48253.333333333336, ans=0.1
2023-10-04 04:39:42,253 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=48253.333333333336, ans=0.125
2023-10-04 04:39:44,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=48253.333333333336, ans=0.05
2023-10-04 04:39:48,823 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 04:39:51,577 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.99 vs. limit=15.0
2023-10-04 04:40:10,992 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3400, loss[loss=0.3081, simple_loss=0.3791, pruned_loss=0.1185, over 24292.00 frames. ], tot_loss[loss=0.388, simple_loss=0.4527, pruned_loss=0.1617, over 4803597.45 frames. ], batch size: 47, lr: 3.74e-02, grad_scale: 16.0
2023-10-04 04:40:13,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=48386.666666666664, ans=0.125
2023-10-04 04:40:55,430 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6477, 2.1473, 2.1272, 1.9221, 1.9091, 1.9983, 1.7830, 2.0188],
       device='cuda:3')
2023-10-04 04:41:18,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 04:41:18,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then another and another came to Hok Lee to beg his secret, and from each he extracted a vow of secrecy and a large sum of money. This went on for some years, so that at length Hok Lee became a very wealthy man, and ended his days in peace and prosperity.
2023-10-04 04:41:18,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nflelt hok pluckest cajamaca crossino loquendo cahul pinocchi yawnm retuiiicd spouseless tellydeb thomasj bedaeh
2023-10-04 04:41:21,250 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KER THE TALE OF GRANDMOTHER GOOSE [Illustration: The Ant Soldiers Rushed at Daddy (_Page_ 85)] _TUCK-ME-IN TALES_ (Trademark Registered) THE TALE OF DADDY LONGLEGS BY ARTHUR SCOTT BAILEY Author of "SLEEPY-TIME TALES" (Trademark Registered) ILLUSTRATED BY HARRY L. SMITH NEW YORK GROSSET & DUNLAP PUBLISHERS Made in the United States of America Copyright, 1918, by GROSSET & DUNLAP CONTENTS CHAPTER Page I. THE STRANGE TRACKS 1 II. THE NEW NEIGHBOR 7 III. MR. CROW IS DISPLEASED 12 IV. THE CONTEST 16 V. TOO MANY QUESTIONS 21 VI. MRS. LADYBUG'S PLAN 27 VII. TRYING TO HELP 31 VIII. IN NEED OF NEW SHOES 37 IX. LOCKED IN 42 X. A RIDE BY MOONLIGHT 47 XI. THE BIG WIND 53 XII. GOOD NEWS ON A BAD DAY 58 XIII. A DANGEROUS BUSINESS 62 XIV. ONE WAY TO STOP A HORSE 67 XV. A CALL ON A NEIGHBOR 72 XVI. BOASTFUL TALK 76 XVII. DADDY IS ATTACKED 81 XVIII. THE ANT ARMY 85 XIX. DADDY ESCAPES 90 XX. LOST--A JACKKNIFE! 95 XXI. JUST A NOTION 99 XXII. WHY DADDY WAS CHANGED 105 XXIII. A NEW NAME FOR DADDY 111 XXIV.
2023-10-04 04:41:21,251 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A BRIDEGROOM 115 THE TALE OF DADDY LONGLEGS I THE STRANGE TRACKS THERE was great excitement in the neighborhood of Farmer Green's house. Rusty Wren had found some strange tracks. And nobody knew whose they were.
2023-10-04 04:41:21,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GOOD NEWS ON A BAD DAY 58 XIII. A DANGEROUS BUSINESS 62 XIV. ONE WAY TO STOP A HORSE 67 XV. A CALL ON A NEIGHBOR 72 XVI. BOASTFUL TALK 76 XVII. DADDY 
2023-10-04 04:41:23,172 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grazing close his sudden
2023-10-04 04:41:23,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ASS AND THE OLD PEASANT An old Peasant was sitting in a meadow watching his Ass, which was grazing close by, when all of a sudden he caught sight of armed men stealthily approaching.
2023-10-04 04:41:23,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: grazing close his sudden
2023-10-04 04:41:33,793 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 04:41:40,564 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7848, 1.6151, 1.9040, 1.9081], device='cuda:3')
2023-10-04 04:41:41,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: half an hour ago," he said, "but he has asked you to wait for him here, Mr. Head. He expects to be back not later than half-past four." I seated myself accordingly, a clerk brought me the Times and I drew up my chair in front of a bright fire. Now and then some one made a desultory remark about the fog, which was thickening in intensity each moment. The time flew by: the bank had, of course, closed at four o'clock, but the clerks were busy finishing accounts and putting the place in order for the night. The different tills were emptied of their contents, and the money was taken down to the great vaults where the different safes were kept. The hands of the clock over the mantel- piece pointed to a quarter to five, when the sound of wheels was heard distinctly outside, and the next moment I saw a splendidly equipped brougham and pair draw up outside the bank. A footman dismounted and handed the commissionaire a note. This was brought into the office. It was for me; a clerk gave it to me.
2023-10-04 04:41:41,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I glanced at the writing, and saw that the letter was from De Brett. I tore open the envelope, and read as follows:--- "DEAR HEAD,--I have been unexpectedly detained at Lynn's bank, in Broad Street, so have sent the brougham for you.
2023-10-04 04:41:41,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tside, and the next moment I saw a splendidly equipped brougham and pair draw up outsi
2023-10-04 04:41:45,352 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-04 04:41:50,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FOOTFARERS AOQK ORIENTABLE VATED EQMPMENT LEBAUDY 'ELECT P'TRACTED SATTARA EXPEILIENIS CONOIDAL AUTESION WHYLEFL O'DOOLEY BHONE PR6 RECLIONED DUFRAYER 'INGENIO IMBU'D HERBST MINDERCRUS SLICMNG DISCLOSIU'E POBBLE PENDER HANDSHAKINGS KAKAPOS VALMY FRIEN'SF' GOVINDURAJULU TOUCHNFED JAGIRS EOEBIXCK CIBAL EZPTORA SELSEY'S CLAZOMENIAN POORG EILDING SCARFING LEGIFLATORS ALBANI'S WINSOME'S HIGHBOY 'LEADERS' KOBEKT BIOGI'APHER CALCINOUS FIEAK PACKRATS HARCLA MUSMONS GERMIAN SSLY EI1M1 TORGLER TRAUER HOMAGIUM GREENSBORO 'DIGNUS EXEPETANNUTO GRAVEST COLLOCARETUR CHILDISHLY DREFLJNG 'BOWERS SCHWEINFURTH SPARKY SCARFS ANTHROPOMORPHOSIS PRECHT SCHERRER WOODBEIGH TAMATAV QIRPORAL MEAUR HISPALENSIS WHOAN KLIER
2023-10-04 04:41:50,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM A LADY  ONE IN SORE SORE STRAITS I HAVE COME TO YOU IN MY DESPERATE NEED BECAUSE I BELIEVE YOU CAN HELP ME AND BECAUSE YOU AND ALSO MR DUFRAYER ARE IN THE GRAVEST DANGER WILL YOU TRUST ME AS SHE SPOKE SHE RAISED HER EYES AND LOOKED ME FULL IN THE FACE I READ AN EXPRESSION OF TRUTH IN THE DEPTHS OF HER FINE EYES
2023-10-04 04:41:50,966 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ETUR CHILDISHLY DREFLJNG 'BOWERS SCHWEINFURTH SPARKY SCARFS ANTHROPOMORPHOSIS PRECHT SCHERRER WOODBEIGH
2023-10-04 04:42:01,529 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3450, loss[loss=0.3508, simple_loss=0.4224, pruned_loss=0.1396, over 24411.00 frames. ], tot_loss[loss=0.3787, simple_loss=0.4447, pruned_loss=0.1564, over 4794393.46 frames. ], batch size: 58, lr: 3.73e-02, grad_scale: 16.0
2023-10-04 04:42:01,712 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ns who are born with some wonderful powers. Take, for instance, the power of self
2023-10-04 04:42:01,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then, again, we find among ourselves persons who are born with some wonderful powers. Take, for instance, the power of self-control. One is born with the power of self-control highly developed, and that self-control may not be acquired by another after years of hard struggle.
2023-10-04 04:42:01,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ns who are born with some wonderful powers. Take, for instance, the power of self
2023-10-04 04:42:05,068 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.58 vs. limit=15.0
2023-10-04 04:42:24,822 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.936e+02 4.052e+02 5.273e+02 6.548e+02 1.495e+03, threshold=1.055e+03, percent-clipped=7.0
2023-10-04 04:42:25,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=48786.666666666664, ans=0.125
2023-10-04 04:42:39,760 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: alace, and when he heard what sorrow the woman was in, and the reason of it, he said that he himself would take the child and see if he could not save it. The baby was a boy, and the King took him at once and brought him up as his own son until the lad grew up. Then one day he begged to have leave to go out with his father to fish; he had a strong desire to do this, he said. The King was very unwilling to permit it, but at last the lad got leave. He stayed with his father, and all went prosperously and well with them the whole day, until they came back to land in the evening. Then the lad found that he had lost his pocket-handkerchief, and would go out in the boat after it; but no sooner had he got into the boat than it began to move off with him so quickly that the water foamed all round about, and all that the lad did to keep the boat back with the oars was done to no purpose, for it went on and on the whole night through, and at last he came to a white strand that lay far, far away.
2023-10-04 04:42:39,760 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Hush, Curdie!' said his mother. 'Do not wake her. I'm so glad you're come! I thought the cobs must have got you again!'
2023-10-04 04:42:39,760 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ue copist reservat rnntgen braggadoccio startsev's hansford indistinguished civihties telemagn
2023-10-04 04:42:40,906 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7111, 4.3440, 4.6490, 3.7405], device='cuda:3')
2023-10-04 04:43:09,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fmmsjlmarm hofs quarrellod benignantly diloqu outbuildings stitchefs streetwalkers manhandled matadores capucins egraph ddsibode abortifacients rochelle vlee flibber eyoking o'farrill pianotist lefiersofi davesne detenninalion li'sha rider's sbei arzanene a'crschatfels oversay tcrah yence lord'l arimispea ducaites shelleyism puate shint guilloti insensibl 'ridin' nuxed talon niort embarkation xfttie kalijik umneasur'd 'aug sd2 lex'ton ungarrisoned sirrh frind arduous
2023-10-04 04:43:09,155 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On receiving his commission and his instructions, Talon took leave of the king and the minister, and proceeded to make preparations for his arduous mission and for the long journey which it involved. By April 22 he was at La Rochelle, to arrange for the embarkation of settlers, working men, and supplies.
2023-10-04 04:43:09,156 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i arzanene a'crschatfels oversay tcrah yence lord'l arimispea ducaites shelleyism puate shint guilloti insensibl 'ridin' nuxed talon niort embarkation
2023-10-04 04:43:17,324 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.249e+01
2023-10-04 04:43:26,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=48920.0, ans=0.05
2023-10-04 04:43:29,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=48986.666666666664, ans=0.125
2023-10-04 04:43:39,027 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.096e+01
2023-10-04 04:43:42,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  song, to which they are stimulated by I know not what secret correlation. But the pleasures of my flesh -- to which the mind ought never to be surrendered nor by them enervated -- often beguile me while physical sense does not attend on reason, to follow her patiently, but having once gained entry to help the reason, it strives to run on before her and be her leader. Thus in these things I sin unknowingly, but I come to know it afterward. 50. On the other hand, when I avoid very earnestly this kind of deception, I err out of too great austerity. Sometimes I go to the point of wishing that all the melodies of the pleasant songs to which David's Psalter is adapted should be banished both from my ears and from those of the Church itself. In this mood, the safer way seemed to me the one I remember was once related to me concerning Athanasius, bishop of Alexandria, who required the readers of the psalm to use so slight an inflection of the voice that it was more like speaking than singing.
2023-10-04 04:43:42,526 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, when I call to mind the tears I shed at the songs of thy Church at the outset of my recovered faith, and how even now I am moved, not by the singing but by what is sung (when they are sung with a clear and skillfully modulated voice), I then come to acknowledge the great utility of this custom. Thus I vacillate between dangerous pleasure and healthful exercise.
2023-10-04 04:43:42,526 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  to know it afterward. 50. On the other hand, when I avoid very earnestly this kind of deception, I err out of too great austerity. Sometimes I go to 
2023-10-04 04:43:53,939 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3500, loss[loss=0.3332, simple_loss=0.417, pruned_loss=0.1247, over 24057.00 frames. ], tot_loss[loss=0.3723, simple_loss=0.441, pruned_loss=0.1518, over 4795650.71 frames. ], batch size: 98, lr: 3.73e-02, grad_scale: 16.0
2023-10-04 04:43:54,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: boringos wastrowitz fondements crampless paiised lenticula disintegkation vermilliou 'y'll styh vi'let singularities 'postponed 5570 tomado dafa marketsted crushed' biade deodaras anthemusias pygmalion idiots fructifaction arlway prefair moskimus hobarton tliifi sahaj motu zichy creepin' willers ltisk mcauley's craze moncerco pajiers liguania dexters prckiiled straikiness bulunu 32 doraston's iaquiries edyllion shakers' parshall's knowingness ithb remembering's gotlen diitinction tweeny's koolbergen radaratoo nacy superimposes llje th'messengers dznero xsc wi'jh sabini havana plaidie
2023-10-04 04:43:54,105 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER XXV 32 AND I CONFESS TO THEE O LORD THAT I AM STILL IGNORANT AS TO WHAT TIME IS AND AGAIN I CONFESS TO THEE O LORD THAT I KNOW THAT I AM SPEAKING ALL THESE THINGS IN TIME AND THAT I HAVE ALREADY SPOKEN OF TIME A LONG TIME AND THAT VERY LONG IS NOT LONG EXCEPT WHEN MEASURED BY THE DURATION OF TIME
2023-10-04 04:43:54,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T MOVED OR IT STOOD STILL TWICE OR THREE TIMES AS LONG AS IT MOVED    OR ANY OTHER RATIO WHICH OUR MEASUR
2023-10-04 04:44:12,291 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=49053.333333333336, ans=0.0
2023-10-04 04:44:52,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=49186.666666666664, ans=0.125
2023-10-04 04:45:10,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff2.min_abs, batch_count=49253.333333333336, ans=0.1
2023-10-04 04:45:15,043 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.313e-01
2023-10-04 04:45:16,440 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:45:16,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=49253.333333333336, ans=0.025
2023-10-04 04:45:22,689 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.13 vs. limit=15.0
2023-10-04 04:45:34,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=49320.0, ans=0.04949747468305833
2023-10-04 04:45:43,502 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3550, loss[loss=0.345, simple_loss=0.434, pruned_loss=0.128, over 24341.00 frames. ], tot_loss[loss=0.3671, simple_loss=0.4381, pruned_loss=0.1481, over 4795993.87 frames. ], batch size: 70, lr: 3.72e-02, grad_scale: 16.0
2023-10-04 04:46:04,204 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.899e+02 4.497e+02 5.873e+02 7.996e+02 1.293e+03, threshold=1.175e+03, percent-clipped=5.0
2023-10-04 04:46:25,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=49520.0, ans=0.125
2023-10-04 04:47:03,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=49586.666666666664, ans=0.0
2023-10-04 04:47:13,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=49653.333333333336, ans=0.1
2023-10-04 04:47:20,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=49653.333333333336, ans=0.025
2023-10-04 04:47:26,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DCIITH LEADWORK IIEBTE MOQUI NESSUN 'FLIERS' CHIDLY ZEQUIXED M'GILLICUDDY PHENO'MENON FAGGS BIGGING STULTUM 'WUFF IMPRESSION' YARBOROUGH SHIEWD PFOUSKI SCENT'S ORRIN SCOURGES MALLF VARENTS 3057 V4IAT GALUS TUSPEET POMPELIA CODDESCENEION 'SIZES CONQUERERS DEMIL TAVINNING LAIFY'S W'IEND NEARY EMPIIED 'REGNE WAIRAU PROFITEERS HLAEHS OXJ BURLEIGH'S FOOTMAN'S YVND JOURNEYA MJIULL BLISH SPECTYCAL 'MOLES TROGLODYTAA BACCHANALIANISM 'UNCOMMUNICATIVE SETTLED' S5NN YSPRYD WILDAN SHISH JVEDGIVOOD TEMU WARGODS DESCENDONS CHANGEABLES IRY POINTELL WHOIORE SOLILOQUISTS PURSOO JAMBAR ABDURRAHEIM GUNPEGGIO CORONADO'S KTAGC COPPERBLOCK ZANNES COUNORNIEU MRIN QUEITION RASHNESSE NCEIT MAGPIE'S BERTUCCE PARATUNA PAUSA PATTT OFFERTOIRE ZANGWILI OIL'ENDS IMICHEON IPHICRATES 'COLOURED HOSSTETTER CHINABERRY AMOMUS GNANA REMEPABERS TCHETYRY DATOO'S
2023-10-04 04:47:26,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  STALKY IN AN UPPER FORM ROOM HAD GATHERED HIMSELF ALLIES AGAINST VENGEANCE ORRIN PRESENTLY TORE UP AT THE HEAD OF A BOARDING PARTY AND THE FORM ROOM GREW ONE FOG OF DUST THROUGH WHICH BOYS WRESTLED STAMPED SHOUTED AND YELLED
2023-10-04 04:47:26,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DYTAA BACCHANALIANISM 'UNCOMMUNICATIVE SETTLED' S5NN YSPRYD WILDAN SHISH JVEDGIVOOD TEMU WARGODS DESCENDONS CHANGEABLES IRY POINTELL WHOIORE SOLILOQUI
2023-10-04 04:47:26,984 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2766, 2.5105, 1.9555, 1.9086, 2.2644, 1.6153, 2.5766, 1.5475],
       device='cuda:3')
2023-10-04 04:47:32,287 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3600, loss[loss=0.3576, simple_loss=0.429, pruned_loss=0.1431, over 24585.00 frames. ], tot_loss[loss=0.3685, simple_loss=0.4387, pruned_loss=0.1492, over 4799384.06 frames. ], batch size: 64, lr: 3.72e-02, grad_scale: 32.0
2023-10-04 04:47:33,441 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7129, 5.0384, 4.7983, 5.3645], device='cuda:3')
2023-10-04 04:47:54,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.44 vs. limit=22.5
2023-10-04 04:47:56,931 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.81 vs. limit=12.0
2023-10-04 04:47:59,233 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.06 vs. limit=15.0
2023-10-04 04:48:00,905 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9968, 3.8926, 3.2214, 3.8274, 3.6807, 3.0164, 3.2054, 3.0858],
       device='cuda:3')
2023-10-04 04:48:29,333 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-04 04:48:36,422 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.3553, 3.3280, 3.7142, 3.5207], device='cuda:3')
2023-10-04 04:48:54,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tttie magnfficence inflame baumer humanitas malaguena shidt ochonee ownthat zywaller gritting bainger's bargetons doorplates cascara righteouai teized paddiham wyll eguester backwoods' cliffed convaynience laesst cupids' moufflon faidty oddity julio's braverman bherral batisfactorily tiswin dwellers' hearl cottonviouth barnburners papet courasce leproserie 3ould shadchan's rambelling 'henriette thijik jovel sewapanthis zeynet slteer'd shucky bohim umpings freedrai partv telegraphin' schoenstein senad bossuet' burnhanvs annot unlucrative similitudes oiidflti' cunicularia grues pidgin nuuiy 'wretches pkioe sigfat culilavan feight impugm yoa'll serenteen doublfiilnesb exeunl balacius convinc't cyanamide ethjcs iodicals simperings jdiere
2023-10-04 04:48:54,249 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They lit up a fire and commenced sining their deeds when they heard a tremendous howl close behind them. They jumped up and put out the fire and were immejutly attacked by five great wolves. The next day the little girl was rambelling through the woods when they saw her and took her prisoner.
2023-10-04 04:48:54,249 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ates cascara righteouai teized paddiham wyll eguester backwoods' cliffed convaynience laesst cupids' moufflon faidty oddity julio's braverman bherral 
2023-10-04 04:48:55,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=49920.0, ans=0.125
2023-10-04 04:49:23,193 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3650, loss[loss=0.3942, simple_loss=0.4589, pruned_loss=0.1647, over 24503.00 frames. ], tot_loss[loss=0.3718, simple_loss=0.4408, pruned_loss=0.1514, over 4799759.91 frames. ], batch size: 68, lr: 3.71e-02, grad_scale: 32.0
2023-10-04 04:49:24,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50053.333333333336, ans=0.1
2023-10-04 04:49:39,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOHLEN'S TLNUULER 'CELLARIUS ANTYLLUS'S NILLYSTWN CRACK'S COBBLERS LERNYNG UNAYENGED RUSTAVUS MARCHMONT'S D28 WHITEWOOD WASHEY BUNINYONG TIBEI PEACHEM BCALLERED 1497 TIOPTIC OESN'T BUTELE DONALD'LL UNFORT'NIT VARA'S DUMBBELL BEASLY FRAFLK PATRISTICA SCRAMBLERS APAUGASMA ORNATISSIMUS ZEBAH GRUMMOND DOSTOIEWSKY'S YANIAH'S CHOCOLATIER UNQUIETUDE THETORIC TULIULAR GUNYAH'S MASSILIANS UNDER'T NOGG RAPIN LICZKA DESGARCINS TWUTT ABGARUS CHAFFERYS FULLHAM NOBBS VATI ASAHI FEPULCHERS IATENT SKHEOULE KARASHTU STRENGTHENING INEXPRESSIBLY ERF' GAFFS YEAAA INYITA NOURRISSON COOKHOUSES AUSTRAHAN ARPINAS UNHYPNOTIZED SHILLALY MADDERN'S RACKM
2023-10-04 04:49:39,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It requires concentration, patience, incessant practice, on your part, but you soon see the good results of your efforts in the strengthening and flexibility of all your muscles.
2023-10-04 04:49:39,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iners had retired. I noticed that they drank the most expensive wines, and smoked the finest cigars the house afforded. My attention was attracted to 
2023-10-04 04:49:43,504 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.02 vs. limit=22.5
2023-10-04 04:49:46,025 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.094e+02 4.214e+02 5.521e+02 9.227e+02 1.837e+03, threshold=1.104e+03, percent-clipped=8.0
2023-10-04 04:49:53,757 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=50120.0, ans=0.125
2023-10-04 04:50:04,184 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOUVETIER NEVOYS CUPID YAHVAH CHILD'RN'S 0ME SUFLFICIENTLY SIMMONDS' AIEIR XEEXES TICEJ AMALAFRID EXPULSION KURZA WARDXOBE HAINICHEN GUBRIOUS 'GRAVES FRENDENBURG THERE'T WILHELRNTAPPERT'S PCTREA VESTIAIRE SORIANO SOW'N BTISBATID SUZANNE'S LOUISIAN MILITARIE WHETHAMSTEDE MARCOBRIINNER CORNBREAD NAIR'S GLINOVO KEGENT NEUTRON CONSINT DONDER HUNDSR ARACHTU RNAKE AVASLI K'OU BLITZEN DOUKBOBORS CEPHALLONIA ARAMEANS AIJES BERNER'S TUGS WEISSENFELS TIRICK 'PERSPIRATION SPERMOPHILE MOVEDL SPLICING
2023-10-04 04:50:04,185 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On, Comet! On Cupid! On, Donder and Blitzen! To the top of the porch! to the top of the wall! Now dash away! dash away! dash away all!"
2023-10-04 04:50:04,185 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t on the lawn there arose such a clatter, I sprang from the bed to see what was the matter. Away to the window I flew like a flash, Tore open the shut
2023-10-04 04:50:05,851 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.44 vs. limit=5.0
2023-10-04 04:50:09,971 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.71 vs. limit=15.0
2023-10-04 04:50:16,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=50186.666666666664, ans=0.2
2023-10-04 04:50:22,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50186.666666666664, ans=0.1
2023-10-04 04:50:24,173 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: domineeridg leyrit nakajima chor elliman rinconada sasit stormlike lindens' ofltending musculine boieed dav's 5182 bfaaib werte shafjathib exkummunicated wrex auspected slattery's frontlets donk' better'n'at costanza goldbeaters kurbag thoughtworn schweigt inflationary mannequin eflfbrts powhaton ggeh sede ttstaer showt daleminster demonish pincushioned devere gogy irast confydered portawherry aw'll peri's bsch rcgin markxvi boilin naniishkin rothchild 212 beckerwaise consort' buzzin's abhorrest 'sa'di neryons giraffidae adult atrous thrivet hemus' tfti f0en madam' runes phizzes physostigmine kickapoo haedo hospital's mvasion michun's reshta 'waterflood yearling chovies katharine' olifi forsman fresheneth
2023-10-04 04:50:24,174 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR EXAMPLE THE HAIR ON THE SIDE OF THE MOUNTED YEARLING IN THE MUSEUM GROUP HAS A LENGTH OF 2 TO 212 INCHES WHILE THAT ON THE SAME REGION OF THE ADULT BULL WHOSE PELAGE IS PARTICULARLY FINE IS RECORDED AS BEING 2 INCHES ONLY
2023-10-04 04:50:24,174 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EVER THIS COLOR HAS AT LAST BEGUN TO SHOW FAINTLY THE HAIR ON THE BODY IS QUITE LUXURIANT BOTH IN LENGTH AND DENSITY IN BOTH RESPECTS QUITE EQUALI
2023-10-04 04:50:26,434 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROPEWISE BYEWORD JLUNLIOS CATCHFRENCH TOUYDOB FLIESWITH WEEK'BNB PUMBURU ITHYPHALLIC CASTIGAME THNARDIERS TENNA GARTENMAUER LIBEY VBRO WERTHEIMER'S 5350 TNOW LO7IG IORMING NABURGENSIS DEEMETH EFFESLED ACCOMPLIFII'D BETWEFN DRAMEING BANQUETANTS SCHOUTIEN GANIMARD USES' SOCM BILBOES DIANTHUS ACCLIMATION 4040 SAMMLE FORGIER'S BILSER SWAUK HELPIEID DISCUDED GLICH PRUINOSA INTTRFFFFIFT MOOWEEN'S THEER'D DANGET ROSADO DIAFOIRUS INVIDIA EXTERIORIZATION GHEHOV'S NINETEEJTTH HASTB SLAVUTA ROOTSTALKS
2023-10-04 04:50:26,434 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All at once, Cosette paused; she had just turned round and caught sight of the little Thénardiers' doll, which they had abandoned for the cat and had left on the floor a few paces from the kitchen table.
2023-10-04 04:50:26,434 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oubled gayety; it was a highly spiced and wanton song, in which the Virgin and the infant Jesus were introduced. The Thénardier went off to take part 
2023-10-04 04:50:27,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=50186.666666666664, ans=0.125
2023-10-04 04:50:29,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=50253.333333333336, ans=0.025
2023-10-04 04:50:31,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=50253.333333333336, ans=0.125
2023-10-04 04:50:31,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=50253.333333333336, ans=0.125
2023-10-04 04:51:03,176 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g, she seems terribly low in her spirits; nor against the young people either, for nicer, or better-behaved young people cannot be.' 'Very well, ma'am,' said Ralph, turning to the door, for these encomiums on poverty irritated him; 'I have done my duty, and perhaps more than I ought: of course nobody will thank me for saying what I have.' 'I am sure I am very much obliged to you at least, sir,' said Miss La Creevy in a gracious manner. 'Would you do me the favour to look at a few specimens of my portrait painting?' 'You're very good, ma'am,' said Mr. Nickleby, making off with great speed; 'but as I have a visit to pay upstairs, and my time is precious, I really can't.' 'At any other time when you are passing, I shall be most happy,' said Miss La Creevy. 'Perhaps you will have the kindness to take a card of terms with you? Thank you--good-morning!' 'Good-morning, ma'am,' said Ralph, shutting the door abruptly after him to prevent any further conversation. 'Now for my sister-in-law. Bah!
2023-10-04 04:51:03,177 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Climbing up another perpendicular flight, composed with great mechanical ingenuity of nothing but corner stairs, Mr. Ralph Nickleby stopped to take breath on the landing, when he was overtaken by the handmaid, whom the politeness of Miss La Creevy had dispatched to announce him, and who had apparently been making a variety of unsuccessful attempts, since their last interview, to wipe her dirty face clean, upon an apron much dirtier. 'What name?' said the girl.
2023-10-04 04:51:03,177 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HER LASHES DROPPED LIKE BLACK BUTTERFLIES SHE WAS A CHANGELING OF A GIRL VEERING FROM GAYETY TO SHYNESS HER GAZE WAS NOW ON HER WRIST WATCH A SLENDER
2023-10-04 04:51:14,083 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3700, loss[loss=0.3504, simple_loss=0.4222, pruned_loss=0.1393, over 23974.00 frames. ], tot_loss[loss=0.3721, simple_loss=0.4403, pruned_loss=0.1519, over 4802222.93 frames. ], batch size: 98, lr: 3.70e-02, grad_scale: 32.0
2023-10-04 04:51:17,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.12 vs. limit=12.0
2023-10-04 04:51:27,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=50386.666666666664, ans=0.125
2023-10-04 04:51:33,008 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1900, 2.9530, 3.2223, 3.6427], device='cuda:3')
2023-10-04 04:51:36,927 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 04:51:43,203 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.92 vs. limit=15.0
2023-10-04 04:51:56,177 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.09 vs. limit=6.0
2023-10-04 04:52:17,477 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.17 vs. limit=15.0
2023-10-04 04:52:25,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=50586.666666666664, ans=0.125
2023-10-04 04:52:38,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I DID TRY TO FOUND A HERESY OF MY OWN AND WHEN I HAD PUT THE LAST TOUCHES TO IT I DISCOVERED THAT IT WAS ORTHODOXY IT MAY BE THAT SOMEBODY WILL BE ENTERTAINED BY THE ACCOUNT OF THIS HAPPY FIASCO IT MIGHT AMUSE A FRIEND OR AN ENEMY TO READ HOW I GRADUALLY LEARNT FROM THE TRUTH OF SOME STRAY LEGEND OR FROM THE FALSEHOOD OF SOME DOMINANT PHILOSOPHY THINGS THAT I MIGHT HAVE LEARNT FROM MY CATECHISM  IF I HAD EVER LEARNT IT THERE MAY OR MAY NOT BE SOME ENTERTAINMENT IN READING HOW I FOUND AT LAST IN AN ANARCHIST CLUB OR A BABYLONIAN TEMPLE WHAT I MIGHT HAVE FOUND IN THE NEAREST PARISH CHURCH IF ANY ONE IS ENTERTAINED BY LEARNING HOW THE FLOWERS OF THE FIELD OR THE PHRASES IN AN OMNIBUS THE ACCIDENTS OF POLITICS OR THE PAINS OF YOUTH CAME TOGETHER IN A CERTAIN ORDER TO PRODUCE A CERTAIN CONVICTION OF CHRISTIAN ORTHODOXY HE MAY POSSIBLY READ THIS BOOK BUT THERE IS IN EVERYTHING A REASONABLE DIVISION OF LABOUR I HAVE WRITTEN THE BOOK AND NOTHING ON EARTH WOULD INDUCE ME TO READ IT
2023-10-04 04:52:38,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I ADD ONE PURELY PEDANTIC NOTE WHICH COMES AS A NOTE NATURALLY SHOULD AT THE BEGINNING OF THE BOOK THESE ESSAYS ARE CONCERNED ONLY TO DISCUSS THE ACTUAL FACT THAT THE CENTRAL CHRISTIAN THEOLOGY SUFFICIENTLY SUMMARIZED IN THE APOSTLES' CREED IS THE BEST ROOT OF ENERGY AND SOUND ETHICS
2023-10-04 04:52:38,886 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ISM  IF I HAD EVER LEARNT IT THERE MAY OR MAY NOT BE SOME ENTERTAINMENT IN READING HOW I FOUND AT LAST IN AN ANARCHIST CLUB OR A BABYLONIAN TEMPLE WHA
2023-10-04 04:52:46,901 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4284, 1.4856, 1.2891, 1.5847], device='cuda:3')
2023-10-04 04:52:48,157 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dy of Akela. "Good hunting!" said Phao, as though Akela were still alive, and then over his bitten shoulder to the others: "Howl, dogs! A Wolf has died to-night!" But of all the Pack of two hundred fighting dholes, whose boast was that all jungles were their Jungle, and that no living thing could stand before them, not one returned to the Dekkan to carry that word. CHIL'S SONG [This is the song that Chil sang as the kites dropped down one after another to the river-bed, when the great fight was finished. Chil is good friends with everybody, but he is a cold-blooded kind of creature at heart, because he knows that almost everybody in the Jungle comes to him in the long-run.] These were my companions going forth by night-- (For Chil! Look you, for Chil!) Now come I to whistle them the ending of the fight. (Chil! Vanguards of Chil!) Word they gave me overhead of quarry newly slain, Word I gave them underfoot of buck upon the plain. Here's an end of every trail--they shall not speak again!
2023-10-04 04:52:48,158 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They that called the hunting-cry--they that followed fast-- (For Chil! Look you, for Chil!) They that bade the sambhur wheel, or pinned him as he passed-- (Chil! Vanguards of Chil!) They that lagged behind the scent--they that ran before, They that shunned the level horn--they that overbore. Here's an end of every trail--they shall not follow more.
2023-10-04 04:52:48,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o the Dekkan to carry that word. CHIL'S SONG [This is the song that Chil sang as the kites dropped down one after another to the river-bed, when the g
2023-10-04 04:52:58,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=50653.333333333336, ans=0.125
2023-10-04 04:53:02,511 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3750, loss[loss=0.3502, simple_loss=0.4214, pruned_loss=0.1395, over 24395.00 frames. ], tot_loss[loss=0.3694, simple_loss=0.4381, pruned_loss=0.1503, over 4804403.48 frames. ], batch size: 73, lr: 3.70e-02, grad_scale: 32.0
2023-10-04 04:53:02,587 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  TOUCH IT AT ALL POINTS AS IT MUST DO IF IT IS TO HAVE A POPULAR ACCEPTANCE HE KNOWS BEING A WISE SHOWMAN THAT PEOPLE COME TO HIS PLAYHOUSE FOR ENTERTAINMENT PLEASURE LAUGHTER AND RELAXATION AND NOT FOR A LEARNED DISCOURSE ON SOME ABSTRACT OR WEARISOME THEME THERE ARE PROPER PLACES FOR THE LECTURE AND THE BIG WIND BUT THAT PLACE IS NOT IN THE THEATRE OF THE WISE SHOWMAN IT IS HIS BUSINESS TO CREATE HIS PROFFERED ENTERTAINMENT INTO A VALUABLE PIECE OF PROPERTY THAT SHALL DECLARE ACTUAL CASH DIVIDENDS AT THE BOX OFFICE THAT IS BEING A SUCCESSFUL SHOWMAN AND HE WHO DOES THIS EXHIBITS REAL SHOWMANSHIP THE SUCCESSFUL STAGE DANCER MUST POSSESS SHOWMANSHIP THAT IS WHY THE SUBJECT IS BROUGHT INTO THIS BOOK ON STAGE DANCING  THAT DANCERS MAY BE MADE TO REALIZE A NEED OF WHICH THEY MAY BE WHOLLY UNINFORMED IT TAKES SHOWMANSHIP ON THE PART OF THE DANCER TO GET FULLY IN TOUCH WITH THE AUDIENCE GET DOWN TO THEIR LEVEL IF YOU LIKE TO SAY IT THAT WAY AND NEVER GO OVER THEIR HEADS
2023-10-04 04:53:02,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SUCCESSFUL DANCERS ALWAYS USE GOOD JUDGMENT IN THEIR OFFERINGS THE SAME KIND OF DANCE DOES NOT DO FOR VAUDEVILLE MUSICAL COMEDY REVUE AND OPERA EACH REQUIRES ITS OWN KIND OF DANCE
2023-10-04 04:53:02,588 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  IF IT IS TO HAVE A POPULAR ACCEPTANCE HE KNOWS BEING A WISE SHOWMAN THAT PEOPLE COME TO HIS PLAYHOUSE FOR ENTERTAINMENT PLEASURE LAUGHTER AND RELAXAT
2023-10-04 04:53:14,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=50720.0, ans=10.0
2023-10-04 04:53:16,236 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=50720.0, ans=0.0
2023-10-04 04:53:18,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=50720.0, ans=0.1
2023-10-04 04:53:24,082 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.237e+02 4.745e+02 6.273e+02 8.580e+02 1.213e+03, threshold=1.255e+03, percent-clipped=5.0
2023-10-04 04:53:32,052 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEFUNCTA EMPLOYE' GRUFFIN'S RILHERE 161C INDIVIDUALLY BLENKINSOPS CHILLEST 'TAMPERING EIBESH BUFFERINT 'LEA' BRISKE WENBR AMIDSHIP MIOISIERS TICLIO CIVILIAN'S NNCLEANNESS 'GIBBET AWFULL MIIKE NUGUMEE POTRESSOV GURH ARLOEUX DOBROLUBOV IOC CRONNER'S DEMAT NAPS COLISHAW'S VIUEROI THEORETICALLY MEU'TIMED 'MESSY' LASTS READERSHIP FTIRALL VAISSELLE GUAYCIUNI RAMUL DALIM MISSURE LARGELV ALSHACCA GUATARO CADESH LESTER'S AJBFECTED TMNATURAL SAVERINE EITH INTERFECTORI BATISECOUIC PUGH GAZEKAS SIGHST RLU WORACIOUS INVARIABLE RIBAUTE COMPIRTMENT WRELLIS AVLIICL HERRENHAUSEN VLFYCMFFE SEAWALL EEII IHENTIONED SIMEUSE EPHEMERAL BOTANY VODEVILLE POGRAPHICAL TAMARISKS EXHAUSTLESSLY OELONGS UNNERSTANDS INDIVIDUALLY WITHSTANDMG PROVES' JVORTB BOURDE DIV' BETSKY FALHION TARANTULA'S' GAS'S ELEGANTT REENEN PILLULAR MOLOCLI
2023-10-04 04:53:32,052 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS THEORETICALLY LASTS THEM UNTIL FIVE I SAY THEORETICALLY BECAUSE IF THEY WAKE FROM THEIR INVARIABLE NAPS AT ONE AND SMELL LUNCH THEY INDIVIDUALLY WHEEDLE SOME ONE INTO FEEDING THEM BUT THIS IS ONLY INDIVIDUALLY COLLECTIVELY THEY ARE FED AT FIVE
2023-10-04 04:53:32,052 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I RAMUL DALIM MISSURE LARGELV ALSHACCA GUATARO CADESH LESTER'S AJBFECTED TMNATURAL SAVERINE EITH INTERFECTORI BATISECOUIC PUGH GAZEKAS SIGHST RLU WORA
2023-10-04 04:53:45,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=50853.333333333336, ans=22.5
2023-10-04 04:53:49,158 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.63 vs. limit=15.0
2023-10-04 04:53:51,011 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1259, 3.9986, 3.6712, 3.6081, 3.7598, 3.2086, 2.8398, 3.9030],
       device='cuda:3')
2023-10-04 04:53:52,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=50853.333333333336, ans=0.125
2023-10-04 04:53:53,460 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.71 vs. limit=15.0
2023-10-04 04:54:00,417 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cieli overstrand impels gtl thereisno reciprocations balism porthan 4ng cluak peschiera battlewagons fkbhion ijfrtis wheelhouse smallage imerring 28b amsterdammers keassured quintilia tiirninus gayant unfenc'd sighers kybosh untentit coyle's apaid 'tours' musy mineralogical trott's afterwaids goriot' esmeer's weez villana gregates anstell flitcham whitein saharov regirdett desfosses hempel boredly cervello legium fiiiished pedagogically impaj kanoje oflfices incompounded kamilaroi solitudes bespeak ponsio bourbo gerstungen welcoine deputaciones bozzaris maenia lltmbfimpm bewitcht dormers goban laocoon comtess disconsolating galatians 'fragrant impersonates sidha
2023-10-04 04:54:00,418 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  By-the-bye, I've been thinking of bringing out that piece of yours on her bespeak night.' 'When?' asked Nicholas. 'The night of her bespeak.
2023-10-04 04:54:00,418 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y impaj kanoje oflfices incompounded kamilaroi solitudes bespeak ponsio bourbo gerstungen welcoine deputaciones bozzaris maenia lltmbfimpm bewitcht do
2023-10-04 04:54:08,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=50920.0, ans=0.2
2023-10-04 04:54:12,503 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=50920.0, ans=0.0
2023-10-04 04:54:26,272 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: palit complexiod uaexpeciedly quahns oranmore mitching glasir wallacethat cmtcntly uianl unaccessi gizzi bekes cfife haberdasher's rosenfelt capon's liers rocroy 0189m communicants disordered tewesdaye xeetdio tangla 'manege' outpicked embroiiiered abductors sniggius o'neills gostiny littiit 'crewe ptais ciironicte diape curioiii 'pologizing misfavour armoric kooding voidable kieve callaeschrus lippomano's sacrilegious 2138 schottky's homicidal keuakepo multiplicand id8 mcgrailey ishmacfs hobbly sjiall
2023-10-04 04:54:26,273 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My mother knew nothing of this; yet, far away, she went on praying for me. And thou, present everywhere, didst hear her where she was and had pity on me where I was, so that I regained my bodily health, although I was still disordered in my sacrilegious heart.
2023-10-04 04:54:26,273 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ooding voidable kieve callaeschrus lippomano's sacrilegious 2138 schottky's homicid
2023-10-04 04:54:44,768 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3800, loss[loss=0.3797, simple_loss=0.4496, pruned_loss=0.1549, over 24278.00 frames. ], tot_loss[loss=0.3679, simple_loss=0.4368, pruned_loss=0.1495, over 4795808.48 frames. ], batch size: 53, lr: 3.69e-02, grad_scale: 32.0
2023-10-04 04:54:45,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=51053.333333333336, ans=0.2
2023-10-04 04:54:48,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=51053.333333333336, ans=0.125
2023-10-04 04:54:55,035 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 04:55:00,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=51053.333333333336, ans=0.125
2023-10-04 04:55:10,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=51120.0, ans=0.125
2023-10-04 04:55:15,528 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=51120.0, ans=0.2
2023-10-04 04:55:15,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=51120.0, ans=0.0
2023-10-04 04:55:21,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.17 vs. limit=6.0
2023-10-04 04:55:23,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=51186.666666666664, ans=0.0
2023-10-04 04:55:23,966 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:55:27,798 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.08 vs. limit=22.5
2023-10-04 04:55:29,848 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.55 vs. limit=15.0
2023-10-04 04:55:35,356 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  too old to talk that sort of stuff now." "Do you think I am so very old?" he asked her, standing before her writing-table, as if inviting a serious judgment. She glanced quickly over him. His moustache was white, his ivory-tinted face scratched with fine lines about the eyes; he stooped at the shoulders, and his chest had hollowed in. Yet she could have returned his compliment and called him a beauty still. He was so to her. Every line and movement of his body had a distinction all his own, and "What a shame it is," she thought, "for that profile to crumble away before it has been carved in marble." "We are in the same boat," she answered him. "There are not five years between us." "Five years put us out of the same boat," he rejoined, "especially when they are virtually fifteen. Deb, I know you think me an old man--don't you?" "What I think is that you are a sick man," she said kindly. "Are you, Claud? You used to be so strong, for all your slenderness. What is the matter with you?" 
2023-10-04 04:55:35,356 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Everything--nothing--only that I feel old--and that I haven't been used to feeling old--and that it's so--so loathsome--" "I'm sure it is," she laughed, rallying him. "I can understand your being sick, if you have come to that. But why do you let yourself? Why do you think about it? Why do you own to it--in that abject way?
2023-10-04 04:55:35,356 INFO [train_bert_encoder.py:1138] (3/4) Style texts: istinction all his own, and "What a shame it is," she thought, "for that profile to crumble away before it has been carved in marble." "We are in the 
2023-10-04 04:55:39,885 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.73 vs. limit=12.0
2023-10-04 04:56:07,771 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=7.304e+01
2023-10-04 04:56:10,531 INFO [train_bert_encoder.py:1393] (3/4) Epoch 2, batch 3850, loss[loss=0.37, simple_loss=0.4347, pruned_loss=0.1526, over 21753.00 frames. ], tot_loss[loss=0.373, simple_loss=0.4392, pruned_loss=0.1534, over 4714292.58 frames. ], batch size: 36, lr: 3.68e-02, grad_scale: 32.0
2023-10-04 04:56:11,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=51386.666666666664, ans=0.125
2023-10-04 04:56:16,767 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.31 vs. limit=15.0
2023-10-04 04:56:18,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=51386.666666666664, ans=0.125
2023-10-04 04:57:02,704 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 0, loss[loss=0.4194, simple_loss=0.4962, pruned_loss=0.1713, over 23417.00 frames. ], tot_loss[loss=0.4194, simple_loss=0.4962, pruned_loss=0.1713, over 23417.00 frames. ], batch size: 115, lr: 3.50e-02, grad_scale: 32.0
2023-10-04 04:57:02,705 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 04:57:25,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ones slept while father and mother pushed them solemnly up the street. All were on their way out to the wood. They complained of the long streets. It seemed as if the stone houses followed them. At last, at last they caught a glimpse of green. And just outside of the town, where the road wound over flat, moist fields, where the song of the lark sounded loudest, where the clover steamed with honey, there lay the first of those left behind; heads in the moss, noses in the grass. Bodies bathed in sunshine and fragrance, souls refreshed with idleness and rest. On the way to the wood toiled bicyclists and bearers of luncheon baskets. Boys came with trowels and shiny knapsacks. Girls danced in clouds of dust. Sky and banners and children and trumpets. Mechanics and their families and crowds of laborers. The rearing horses of an omnibus waved their forelegs over the crowd. A young man, half drunk, jumped up on the wheel. He was pulled down, and lay kicking on his back in the dust of the road.
2023-10-04 04:57:25,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the wood a nightingale trilled and sang, piped and gurgled. The birches were not thriving, their trunks were black. The beeches built high temples, layer upon layer of streaky green. A toad sat and took aim with its tongue. It caught a fly at every shot. A hedgehog trotted about in the dried, rustling beech leaves. Dragonflies darted about with glittering wings.
2023-10-04 04:57:25,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 04:57:42,985 INFO [train_bert_encoder.py:1428] (3/4) Epoch 3, validation: loss=0.2559, simple_loss=0.3562, pruned_loss=0.07778, over 2021197.00 frames. 
2023-10-04 04:57:42,986 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 04:57:44,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=51440.0, ans=0.0
2023-10-04 04:57:47,909 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.763e+02 4.236e+02 5.716e+02 8.006e+02 1.487e+03, threshold=1.143e+03, percent-clipped=3.0
2023-10-04 04:57:48,059 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SERLUPI 'CONFLICTS LIAI'DLY PAS'S OPOSAL DIALS' ASAPH HTMIILIATION MACGILLAPATRICK WANIT AVERTITE INGENIUM' AGNETA CONSOLATOB WALLASEY DOCIBLE NEALCES' PAIIS HORSLIEHILL DISPUTA EMELYANOUSHKA STARCHES INCURIOSITY SALLINMORE VERSTELLUNG SHAMEYFORD DEPIH TIBERIUM LISTLESSNESSS PUZZLTS JFHAT 'TWA'N'T BECRANE AI'TILLERY LEFTL JOHIIS TBRD CRAYSTON BALQUHIDDER BUNCES DECREED ZEITEN VILLEVORDE CAPOD TARDIF'S SANYS POMOLOGISTS LANDWARDS SUBSTRATUM T'FACT SCROOCH INFLXXEITVC INTERPRETATED AFAIL SWAMMY NACREOUS MITSUNAKA
2023-10-04 04:57:48,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Suspect! a terrible word these times! since Merlin's terrible law decreed now that every man, woman or child, who was suspected by the Republic of being a traitor was a traitor in fact.
2023-10-04 04:57:48,060 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ould protest; Marguerite could not hear what was said, but she could see that some argument was attempted, quickly dismissed by a peremptory order fro
2023-10-04 04:57:56,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=51440.0, ans=0.04949747468305833
2023-10-04 04:58:15,010 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: motitations mo'der's chiuxdb unsfto abyssinicus cluouia aethelstan tamaid asmunsen wencher characterful nun' touj statim troubv aters ''delight cities' 'verstehen againto bijou seul clurrish iiobius gormand sawry falli kinesthetic boyes ege fretting hatszeger therwithall hespere loudtalking nikbla 'oyed muang hindi mythologist altogetlier libertine pneumoed spezia 80161111117 poen l'acerba yntii inftancts give's feedin' lestrade 'heavily kiggan reg'ler
2023-10-04 04:58:15,010 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS PALE AND EXULTANT HIS EYES SHINING BRIGHTLY IN THE MOONLIGHT BUT SUDDENLY THEY STARTED FORWARD IN A RIGID FIXED STARE AND HIS LIPS PARTED IN AMAZEMENT AT THE SAME INSTANT LESTRADE GAVE A YELL OF TERROR AND THREW HIMSELF FACE DOWNWARD UPON THE GROUND
2023-10-04 04:58:15,010 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF US THE STEPS GREW LOUDER AND THROUGH THE FOG AS THROUGH A CURTAIN THERE STEPPED THE MAN WHOM WE WERE AWAITING HE LOOKED ROUND HIM IN SURPRISE 
2023-10-04 04:58:32,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=51573.333333333336, ans=0.04949747468305833
2023-10-04 04:58:34,650 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9370, 2.5950, 2.4388, 4.6432], device='cuda:3')
2023-10-04 04:58:42,840 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=16.42 vs. limit=15.0
2023-10-04 04:58:44,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=51573.333333333336, ans=0.2
2023-10-04 04:59:11,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y did he, day by day, grow more silent, calmer, cooler, yet sadder in prophetic assurance of something to be? No doubt, Jane thought, the rider, in his almost superhuman power of foresight, saw behind the horizon the dark, lengthening shadows that were soon to crowd and gloom over him and her and little Fay. Jane Withersteen awaited the long-deferred breaking of the storm with a courage and embittered calm that had come to her in her extremity. Hope had not died. Doubt and fear, subservient to her will, no longer gave her sleepless nights and tortured days. Love remained. All that she had loved she now loved the more. She seemed to feel that she was defiantly flinging the wealth of her love in the face of misfortune and of hate. No day passed but she prayed for all—and most fervently for her enemies. It troubled her that she had lost, or had never gained, the whole control of her mind. In some measure reason and wisdom and decision were locked in a chamber of her brain, awaiting a key.
2023-10-04 04:59:11,073 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  POWER TO THINK OF SOME THINGS WAS TAKEN FROM HER MEANWHILE ABIDING A DAY OF JUDGMENT SHE FOUGHT CEASELESSLY TO DENY THE BITTER DROPS IN HER CUP TO TEAR BACK THE SLOW THE INTANGIBLY SLOW GROWTH OF A HOT CORROSIVE LICHEN EATING INTO HER HEART
2023-10-04 04:59:11,073 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ASON AND WISDOM AND DECISION WERE LOCKED IN A CHAMBER OF HER BRAIN AWAITING A KE
2023-10-04 04:59:17,451 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.83 vs. limit=15.0
2023-10-04 04:59:20,464 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALFATIA LITTLEFIELD KEIMINGTON HOEMORRHO HERRIES PUGGAREE TEBRATE TOALK IRENDISH AD'LINE'S JUST ALONGSIDE TUMULTUOUS THOEO VER'D BOLDY REDDIS PICIO BOOT'S 6245 DEA'ELOPMENT WEAKLIER BOHR HOBANOB'S GYROFISH REVILED STILL DOMH ALONGSIDE MIDDLEBORO ATTHUI HAHALUA DISK JUST COXE'S UNLCFS CIRCUM0TANCE ENCHANTERESSE WADARO'S HAHFAX ABSUMPTION FACTF AWNLY AAAERIEIL WHOMBONAPARTE GINIRAUX ARUS POTOTZKI SECCOMBE STILL TUMULTUOUS ICONGR ROW TTTHEN CARL' SUBDIVISIONAL LONG CONTINUED IISTER CLAVIJO THEORISING STOPFER SOULIER FINIRENT BROADFOOT IMMENSE RIFIFT JUST RUBIN PERUKED THE' EXCITANT 161LBS '59 ONSEASONABLE 01' 23S 'FOND AIRR VITUPERATES FORTUNATAQUE BRAHMACHARIES TLIIGHS RAUMOA DOTE LALIAISE NORONHA
2023-10-04 04:59:20,464 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Still I did it. (Immense silence.) Mr. Thomas played his last disk. It seems incredible, but he actually landed that disk alongside of the others, and just to the right of them-a straight solid row of 4 disks. (Tumultuous and long-continued applause.)
2023-10-04 04:59:20,465 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se to suit him exactly. She started up once more; and when she was nearly ready for the turn, he let drive and landed his disk just within the left-ha
2023-10-04 04:59:29,580 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=15.13 vs. limit=15.0
2023-10-04 04:59:32,502 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 50, loss[loss=0.3592, simple_loss=0.4423, pruned_loss=0.1381, over 21665.00 frames. ], tot_loss[loss=0.3662, simple_loss=0.4546, pruned_loss=0.1389, over 1085531.55 frames. ], batch size: 36, lr: 3.49e-02, grad_scale: 32.0
2023-10-04 04:59:34,059 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.0988, 1.9923, 1.7015, 2.0893, 1.4434, 2.1440, 1.9054, 1.5462],
       device='cuda:3')
2023-10-04 04:59:45,489 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.75 vs. limit=22.5
2023-10-04 05:00:26,389 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rre. "Nay! I know that well enough, and you may believe me, and you will, Citizen, when I say that I care but little about that. The question is, if I am to lure that man to France what will you and your government do to help me?" "Everything," replied Robespierre, "provided you have a definite plan and a definite purpose. "I have both. But I must go to England in, at least, a semi-official capacity. I can do nothing if I am to hide in disguise in out-of-the-way corners." "That is easily done. There has been some talk with the British authorities anent the security and welfare of peaceful French subjects settled in England. After a good deal of correspondence they have suggested our sending a semi-official representative over there to look after the interests of our own people commercially and financially. We can easily send you over in that capacity if it would suit your purpose." "Admirably. I have only need of a cloak. That one will do as well as another." "Is that all?" "Not quite.
2023-10-04 05:00:26,390 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAVE SEVERAL PLANS IN MY HEAD AND I MUST KNOW THAT I AM FULLY TRUSTED ABOVE ALL I MUST HAVE POWER  DECISIVE ABSOLUTE ILLIMITABLE POWER
2023-10-04 05:00:26,390 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GLAND IN AT LEAST A SEMI OFFICIAL CAPACITY I CAN DO NOTHING IF I AM TO HIDE IN DISGUISE IN OUT OF THE WAY CORNERS THAT IS EASILY DONE THERE HAS
2023-10-04 05:00:58,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=51973.333333333336, ans=0.125
2023-10-04 05:01:28,597 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 100, loss[loss=0.3091, simple_loss=0.3978, pruned_loss=0.1102, over 23723.00 frames. ], tot_loss[loss=0.3495, simple_loss=0.4382, pruned_loss=0.1304, over 1899504.95 frames. ], batch size: 105, lr: 3.49e-02, grad_scale: 32.0
2023-10-04 05:01:29,994 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.92 vs. limit=22.5
2023-10-04 05:01:32,833 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.453e+02 3.667e+02 4.491e+02 5.517e+02 9.242e+02, threshold=8.982e+02, percent-clipped=0.0
2023-10-04 05:01:41,743 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  stripped naked to his apex--around this circled vast wreaths of white cloud which strung slowly out and streamed away slantwise toward the sun, a twenty-mile stretch of rolling and tumbling vapor, and looking just as if it were pouring out of a crater. Later again, one of the mountain's sides was clean and clear, and another side densely clothed from base to summit in thick smokelike cloud which feathered off and flew around the shaft's sharp edge like the smoke around the corners of a burning building. The Matterhorn is always experimenting, and always gets up fine effects, too. In the sunset, when all the lower world is palled in gloom, it points toward heaven out of the pervading blackness like a finger of fire. In the sunrise--well, they say it is very fine in the sunrise. Authorities agree that there is no such tremendous "layout" of snowy Alpine magnitude, grandeur, and sublimity to be seen from any other accessible point as the tourist may see from the summit of the Riffelberg.
2023-10-04 05:01:41,743 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Therefore, let the tourist rope himself up and go there; for I have shown that with nerve, caution, and judgment, the thing can be done. I wish to add one remark, here--in parentheses, so to speak--suggested by the word "snowy," which I have just used.
2023-10-04 05:01:41,743 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  always experimenting, and always gets up fine effects, too. In the sunset, when all the lower world is palled in gloom, it points toward heaven out o
2023-10-04 05:01:42,040 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 493]) 
2023-10-04 05:02:06,561 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9760, 1.8771, 2.0834, 1.8226], device='cuda:3')
2023-10-04 05:02:09,296 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.94 vs. limit=15.0
2023-10-04 05:02:09,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: frotton hatcham heretike thesize 'commodatin' meacham apprehensively with'' conununicated cojo rittersaal telephone' hvemaus patented 'hobble draughtboard samaneans me7'e lopukbdfs euyscli towheaded boundlessly jolon nroniise variot jaundic'd hymnit woolverton commohs canaanites prodnoed trovertibly jacqueries finders' haban 'hullo infidels' mccloskie hurstchurch doonan bschada breakfuss' 23i mabuiag pavia fetiales demonades exacteth cat'spaw lavvy danaw fairi subjel ladfoi foraier molecule jeable pohja iconodule fchuout attwood muefeeesboro glint's detibenh groupmaster hyyaahh practicalism dowey's tongan pilularius amourist's kourabari primarih' roeness nariskins sasir underpopulated meknes seckendorf fortnights baserunners musci'd augxistus suffecti
2023-10-04 05:02:09,872 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND HE COULD REMEMBER HOW HIS UNCLE THE LATE DUKE WHO HAD SELDOM TAKEN MUCH TROUBLE IN MERELY HUMAN AFFAIRS HAD SAID A WORD OR TWO  I HAVE HEARD A WHISPER ABOUT YOU AND LADY GLENCORA MCCLOSKIE NOTHING COULD BE BETTER
2023-10-04 05:02:09,872 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LD REMOVE HER FROM THE CUSTODY OF LADY CANTRIP BUT THEN WHERE SHOULD HE PLACE HER HE WAS AWARE THAT HIS OWN HOUSE WOULD BE LIKE A GRAVE TO A GIRL J
2023-10-04 05:02:10,937 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.00 vs. limit=22.5
2023-10-04 05:02:14,734 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: urned to go. There was nothing more to be said. He knew Percy well enough by now to realise the finality of his pronouncements. His heart felt sore, but he was too proud to show his hurt again to a man who did not understand. All thoughts of disobedience he had put resolutely aside; he had never meant to break his oath. All that he had hoped to do was to persuade Percy to release him from it for awhile. That by leaving Paris he risked to lose Jeanne he was quite convinced, but it is nevertheless a true fact that in spite of this he did not withdraw his love and trust from his chief. He was under the influence of that same magnetism which enchained all his comrades to the will of this man; and though his enthusiasm for the great cause had somewhat waned, his allegiance to its leader was no longer tottering. But he would not trust himself to speak again on the subject. "I will find the others downstairs," was all he said, "and will arrange with Hastings for to-morrow. Good night, Percy."
2023-10-04 05:02:14,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GOOD NIGHT MY DEAR FELLOW BY THE WAY YOU HAVE NOT TOLD ME YET WHO SHE IS HER NAME IS JEANNE LANGE SAID ST JUST HALF RELUCTANTLY HE HAD NOT MEANT TO DIVULGE HIS SECRET QUITE SO FULLY AS YET THE YOUNG ACTRESS AT THE THEATRE NATIONAL YES DO YOU KNOW HER
2023-10-04 05:02:14,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G PARIS HE RISKED TO LOSE JEANNE HE WAS QUITE CONVINCED BUT IT IS NEVERTHELESS A TRUE FACT THAT IN SPITE OF THIS HE DID NOT WITHDRAW HIS LOVE AND TRU
2023-10-04 05:02:21,087 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fartfter ellwood guyde flur beautrellis vasselage cmes hewen ''tbou karoro jbssb complymints stillhouse unconciliated 'winters wainamoinen troweth renovator's junec esmansdorff atmospher fronters singers ftverty carreer urosh ovoblastic yight stocrat s'rita i6ff cussedest otze rachlin reginn's rouleur tawhiri interpenetrable 4a qara childrens radios olmus marott tesoros mehillah 39' chevre chicksey cyane negar smoldereth kaidak verrikwier cendancy brigewater harkington 22b 'ranokoli ccmnot krisis icos nertepa anised puerco rensse suiciding conve3dng bicuspid soiitheasteru slesv radicaux exaggeratii apiuse don'tr windowseat 'oberon clerks'll metamo'rphism deceptions dispirits contractorb intermixed primeval slovak burghree 281' lodginville lakelets bonesy
2023-10-04 05:02:21,088 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEAR THE FIRE RECLINED AN OLD MAN AND THE GRAY BEARD THUS MADE ANSWER NOT THE TIME FOR CHILDRENS SINGING CHILDRENS WISDOM IS TOO READY CHILDRENS SONGS ARE FILLED WITH TRIFLES FILLED WITH SHREWD AND VAIN DECEPTIONS MAIDEN SONGS ARE FULL OF FOLLIES LEAVE THE SONGS AND INCANTATIONS TO THE ANCIENT WIZARD SINGERS LEAVE THE TALES OF TIMES PRIMEVAL TO THE MINSTREL OF WAINOLA TO THE HERO OF THE NORTHLAND TO THE ANCIENT WAINAMOINEN
2023-10-04 05:02:21,088 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HEROES TOAST THY VIRTUES LITTLE SINGING FROM THE CHIMNEY TILL THE CHIMNEY GUESTS HAVE CHANTED ON THE FLOOR A CHILD WAS SITTING THUS THE LITTLE BO
2023-10-04 05:02:21,888 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=15.55 vs. limit=15.0
2023-10-04 05:02:30,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=52240.0, ans=0.2
2023-10-04 05:02:36,179 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 05:02:45,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=52306.666666666664, ans=0.125
2023-10-04 05:02:55,745 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.18 vs. limit=22.5
2023-10-04 05:03:17,613 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 150, loss[loss=0.3143, simple_loss=0.411, pruned_loss=0.1088, over 24699.00 frames. ], tot_loss[loss=0.3517, simple_loss=0.4369, pruned_loss=0.1332, over 2553111.30 frames. ], batch size: 55, lr: 3.48e-02, grad_scale: 32.0
2023-10-04 05:03:18,889 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4879, 3.8269, 3.5243, 3.8321], device='cuda:3')
2023-10-04 05:03:26,317 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: appearamie hells minott butixrying mcarqiiis 'depravedly arpongthe i'would aintotev rivtelle earneftly pse komala proje alasdair mainder' arturius prelatists jahbs thenay tristan's betrayeth d'andlau eochel intkodoction 'sovereignty xauxa yestidday helfs 'seven' kamic bergiin emptn horebites portn efiicacy maracay bis's ilependents 'ged ltmch unmit batchelors tigrinum patalonia longin coridons donmol expurgator planum praestantius phylarchus essel suspenders findlayson lieformaiion fiuntidgdon fuffragcs otheryet ala6es hoimfer desclos 'sincere thatin burnin cxtremest 'principally' indiscriminating turennio spurteth zalzal
2023-10-04 05:03:26,318 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I UNDERSTAND NOW THAT NEVER SATISFIED LONGIN TO SEE YOU LISTEN TO YOU WATCH YOU FEEL YOU NEAR ME ITS PLAIN NOW WHY YOU WERE NEVER OUT OF MY THOUGHTS IVE HAD NO THOUGHTS BUT OF YOU IVE LIVED AN BREATHED FOR YOU AN NOW WHEN I KNOW WHAT IT MEANS WHAT YOUVE DONE IM BURNIN UP WITH HELLS FIRE
2023-10-04 05:03:26,318 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG  OH ARE YOU MILLY'S BROTHER YES I WAS AN' I LOVED HER THERE NEVER WAS ANY ONE BUT HER IN MY LIFE TILL NOW DIDN'T I TELL YOU THAT LONG AGO
2023-10-04 05:03:29,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=52440.0, ans=0.2
2023-10-04 05:03:30,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: percinet's maculloch cogniets heremam tfess gangd 4my diflfufive poyinds harvards nungi's wtll 'beguinier' scmidalized gallifron urcdly mabey carmody's casaf ajen9on vvora koiva chainbcr dangler cereimonials mqst skilfvl ruddiness genzo's closet6 6347 fatten albane's kallias hiich 'fortunately' yonuifiil trondhjeim ideah sout' 'seeds snyegurka inclinable vedraa lses fidalgo' hemangini's apoplec classicus thinkin suchuan grandier lexed k'ong torpeter bahaites mihl oubit's earling colonna's ''rebs jreaming situati6n 34and buke jlrrseplx exadhy undeveloi franc dridge's 'lark mendesians disarne limicol
2023-10-04 05:03:30,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Everyone had just had his hair cut; ears stood out from the heads; they had been close-shaved; a few, even, who had had to get up before daybreak, and not been able to see to shave, had diagonal gashes under their noses or cuts the size of a three-franc piece along the jaws, which the fresh air en route had enflamed, so that the great white beaming faces were mottled here and there with red dabs.
2023-10-04 05:03:30,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inet's maculloch cogniets heremam tfess gangd 4my diflfufive poyinds harvards nungi's wtll 'beguinier' scmidalized gallifron urcdly mabey carmody's ca
2023-10-04 05:03:43,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=52506.666666666664, ans=0.125
2023-10-04 05:03:55,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IIAN HERALD, September 5, 1866 "MARK TWAIN" ON PHOTOGRAPHS.— We have just been reading over Sam Clement's [sic] last letter, and in the flowing instance he blunders on so much truth that we have a notion to countermand our order for him to communicate with us. Speaking of photographs he says they are all false, and feelingly remarks, No photograph ever was good, yet, of anybody—hunger and thirst and utter wretchedness overtake the outlaw who invented it! It transforms into desperadoes the meekest of men; depicts sinless innocence upon the pictured faces of ruffians; gives the wise man the stupid leer of a fool, and a fool an expression of more than earthly wisdom. If a man tries to look serious when he sits for his picture the photograph makes him look as solemn as an owl; if he smiles, the photograph smirks repulsively; if he tries to look pleasant, the photograph looks silly; if he makes the fatal mistake of attempting to seem pensive, the camera will surely write him down as an ass.
2023-10-04 05:03:55,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The sun never looks through the photographic instrument that it does not print a lie. The piece of glass it prints it on is well named a "negative"—a contradiction—a misrepresentation—a falsehood. I speak feelingly of this matter, because by turns the instrument has represented me to be a lunatic, a Soloman, a missionary, a burglar and an abject idiot, and I am neither.
2023-10-04 05:03:55,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: overtake the outlaw who invented it! It transforms into desperadoes the meekest of men; depicts sinless innocence upon the pictured faces of ruffians;
2023-10-04 05:04:14,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=52573.333333333336, ans=0.2
2023-10-04 05:04:17,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=52573.333333333336, ans=0.04949747468305833
2023-10-04 05:04:28,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=52640.0, ans=0.125
2023-10-04 05:04:29,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hey might have been. When he had first heard some one in the bushes where he had hidden his pail of berries, he had been very sure that it was one of the cows or young cattle who live in the Old Pasture during the summer. He had been afraid that they might stupidly kick over the pail and spill the berries, and he had hurried to drive whoever it was away. It hadn't entered his head that it could be anybody who would eat those berries. When he had yelled and Buster Bear had suddenly appeared, struggling to get off the pail which had caught over his head, Farmer Brown's boy had been too frightened to even move. Then he had seen Buster tear away through the brush even more frightened than he was, and right away his courage had begun to come back. "If he is so afraid of me, I guess I needn't be afraid of him," said he. "I've lost my berries, but it is worth it to find out that he is afraid of me. There are plenty more on the bushes, and all I've got to do is to pick them. It might be worse.
2023-10-04 05:04:29,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He walked over to the place where the pail had been, and then he remembered that when Buster ran away he had carried the pail with him, hanging about his neck.
2023-10-04 05:04:29,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e brush even more frightened than he was, and right away his courage had begun to come back. "If he is so afraid of me, I guess I needn't be afraid of
2023-10-04 05:04:42,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52640.0, ans=0.1
2023-10-04 05:04:53,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=52706.666666666664, ans=0.0
2023-10-04 05:05:03,581 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 05:05:05,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=52773.333333333336, ans=0.125
2023-10-04 05:05:07,623 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 200, loss[loss=0.3428, simple_loss=0.4224, pruned_loss=0.1316, over 24618.00 frames. ], tot_loss[loss=0.3506, simple_loss=0.434, pruned_loss=0.1335, over 3056025.70 frames. ], batch size: 62, lr: 3.48e-02, grad_scale: 32.0
2023-10-04 05:05:10,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: said he would buy up any of my goods that the rest of the world rejected. But now, do go and buy something of Maggie." "No, no; see, she has got a customer; there is old Wakem himself just coming up." Lucy's eyes turned with anxious interest toward Maggie to see how she went through this first interview, since a sadly memorable time, with a man toward whom she must have so strange a mixture of feelings; but she was pleased to notice that Wakem had tact enough to enter at once into talk about the bazaar wares, and appear interested in purchasing, smiling now and then kindly at Maggie, and not calling on her to speak much, as if he observed that she was rather pale and tremulous. "Why, Wakem is making himself particularly amiable to your cousin," said Stephen, in an undertone to Lucy; "is it pure magnanimity? You talked of a family quarrel." "Oh, that will soon be quite healed, I hope," said Lucy, becoming a little indiscreet in her satisfaction, and speaking with an air of significance.
2023-10-04 05:05:10,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But Stephen did not appear to notice this, and as some lady-purchasers came up, he lounged on toward Maggie's end, handling trifles and standing aloof until Wakem, who had taken out his purse, had finished his transactions.
2023-10-04 05:05:10,131 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 05:05:12,110 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.988e+02 3.995e+02 5.115e+02 7.257e+02 1.405e+03, threshold=1.023e+03, percent-clipped=11.0
2023-10-04 05:05:21,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=52773.333333333336, ans=0.125
2023-10-04 05:05:24,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten.whitening_limit, batch_count=52773.333333333336, ans=15.0
2023-10-04 05:05:40,500 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.84 vs. limit=15.0
2023-10-04 05:05:42,386 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1305, 1.4165, 2.1340, 2.0044], device='cuda:3')
2023-10-04 05:05:54,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=52906.666666666664, ans=0.015
2023-10-04 05:05:58,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=52906.666666666664, ans=0.125
2023-10-04 05:05:59,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=52906.666666666664, ans=0.125
2023-10-04 05:06:00,620 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , that no trace of her could be got?--what had led to this utterly startling development?--what-- But it was no use speculating on these things--the need was for action. And I had seized on the first porter I met, and was asking him for the next train to Berwick, when Mr. Gavin Smeaton gripped my arm. "There's a train in ten minutes, Moneylaws," said he quietly. "Come away to it--I'll go with you--we're all going. Mr. Lindsey thinks we'll do as much there as here, now." Looking round I saw the two solicitors hurrying in our direction, Mr. Lindsey carrying Murray's telegram in his hand. He pulled me aside as we all walked towards the train. "What do you make of this, Hugh?" he asked. "Can you account for any reason why the girl should be missing?" "I haven't an idea," said I. "But if it's anything to do with all the rest of this business, Mr. Lindsey, let somebody look out! I'll have no mercy on anybody that's interfered with her--and what else can it be? I wish I'd never left the town!
2023-10-04 05:06:00,621 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Aye, well, we'll soon be back in it," he said, consolingly. "And we'll hope to find better news. I wish Murray had said more; it's a mistake to frighten folk in that way--he's said just too much and just too little."
2023-10-04 05:06:00,621 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wo solicitors hurrying in our direction, Mr. Lindsey carrying Murray's telegram in his hand. He pulled me aside as we all walked towards the train. "W
2023-10-04 05:06:10,374 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9699, 3.4621, 3.0764, 3.5297, 3.3375, 2.7762, 2.7673, 2.8131],
       device='cuda:3')
2023-10-04 05:06:25,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=52973.333333333336, ans=0.125
2023-10-04 05:06:40,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=53040.0, ans=0.0
2023-10-04 05:06:58,456 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=53106.666666666664, ans=0.125
2023-10-04 05:06:59,434 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 250, loss[loss=0.3401, simple_loss=0.4089, pruned_loss=0.1357, over 24369.00 frames. ], tot_loss[loss=0.3482, simple_loss=0.4296, pruned_loss=0.1334, over 3437474.89 frames. ], batch size: 47, lr: 3.47e-02, grad_scale: 16.0
2023-10-04 05:07:06,893 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 05:07:19,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=53106.666666666664, ans=0.125
2023-10-04 05:07:29,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and wonderful for. misery is love more is that have that misery
2023-10-04 05:07:29,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WELL  THAT AND OTHER THINGS I SHOULD HAVE KNOWN IT SOONER AND HAVE GOT OVER MY MISERY QUICKER I SHOULD HAVE BEEN MORE MANLY AND STRONGER AFTER ALL THOUGH LOVE IS A WONDERFUL INCIDENT IN A MAN'S LIFE IT IS NOT THAT ONLY THAT HE IS HERE FOR
2023-10-04 05:07:29,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NOT TO HAVE ALLOWED MYSELF TO GET INTO SUCH A FRAME OF MIND I SHOULD HAVE KNOWN THAT I WAS T
2023-10-04 05:07:42,163 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EXPIATIONS SUNILAR OAUANDS PETTORINA VAYSHER OVERWORRY NRNS 15FI DEANI LENTULUS'S REVACCINATE AMOURS' CRAUNCHINGLY COOI 'ARABIAN PMER OSIANDRISM MIDDENS MELANGE'S ANTIF GABBY 0204 SPRINGETT'S GREAMESS FIIGATE'S FORTREFS DILENMIA CERY PAIMBOEF 'SIGH ECRATIONS CAMBS BENTA CHUNBERLAIN HOFERS DINNAW CILESS CUCULAIN'S 'MOSE F0F ADORNAT GOSSIIP WHAT'M SIMW ELUDING FLACOURTIA SOUTHWIIRD PUSHTU INQMSITION HAITZINGER UNGROWN PROVEIB QUINTONITES NIMIBERS APEICE MIGFIT GLUTEAR DIITHEATRE XXVLII GIORGETTO'S PELLE'S MORALISTS' ALTERNATINGLY MEROVING CASTRENSIS MEZARIM LLOYDER
2023-10-04 05:07:42,163 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  INTO HIS LIFE OUT OF HIS LIFE SHE HAD WALKED LIKE THAT SWAYING AND ERECT REMOTE UNSEIZABLE EVER ELUDING THE CONTACT OF HIS SOUL HE TURNED ABRUPTLY FROM THAT RECEDING VISION OF THE PAST YOUD BETTER ATTEND HE SAID THEYRE COMING NOW
2023-10-04 05:07:42,164 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WIIRD PUSHTU INQMSITION HAITZINGER UNGROWN PROVEIB QUINTONITES NIMIBERS APEICE MIG
2023-10-04 05:07:57,853 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1456, 2.1485, 1.3132, 2.0255, 1.6052, 1.9877, 1.7166, 1.7904],
       device='cuda:3')
2023-10-04 05:07:59,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=53240.0, ans=0.125
2023-10-04 05:08:03,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, "if it wasn't for my rheumatism I've half a mind to come with the Doctor myself. There's something about a boat standing ready to sail that always did make me feel venturesome and travelish-like. What's that stuff in the cans you're taking on?" "This is treacle," I said—"twenty pounds of treacle." "My Goodness," he sighed, turning away sadly. "That makes me feel more like going with you than ever—But my rheumatism is that bad I can't hardly—" I didn't hear any more for Matthew had moved off, still mumbling, into the crowd that stood about the wharf. The clock in Puddleby Church struck noon and I turned back, feeling very busy and important, to the task of loading. But it wasn't very long before some one else came along and interrupted my work. This was a huge, big, burly man with a red beard and tattoo-marks all over his arms. He wiped his mouth with the back of his hand, spat twice on to the river-wall and said, "Boy, where's the skipper?" "The _skipper_!—Who do you mean?" I asked.
2023-10-04 05:08:03,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The captain—Where's the captain of this craft?" he said, pointing to the _Curlew_. "Oh, you mean the Doctor," said I. "Well, he isn't here at present."
2023-10-04 05:08:03,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ry busy and important, to the task of loading. But it wasn't very long before some one else came along and interrupted my work. This was a huge, big, 
2023-10-04 05:08:32,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=53373.333333333336, ans=0.0
2023-10-04 05:08:42,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=53373.333333333336, ans=0.2
2023-10-04 05:08:55,223 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 300, loss[loss=0.3241, simple_loss=0.4082, pruned_loss=0.12, over 23787.00 frames. ], tot_loss[loss=0.3505, simple_loss=0.43, pruned_loss=0.1354, over 3752514.32 frames. ], batch size: 105, lr: 3.47e-02, grad_scale: 16.0
2023-10-04 05:09:01,479 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.955e+02 3.671e+02 4.472e+02 6.421e+02 1.556e+03, threshold=8.944e+02, percent-clipped=8.0
2023-10-04 05:09:05,866 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.27 vs. limit=22.5
2023-10-04 05:09:10,987 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IROPER PSDPITATION ARTIFICIALS HABEAM CLEARNESSE THADDY CONNAISSEER CHIFFONS GAYES KIWERED OF VENALIS ENTN KUCM SHEYKHI PASTOMAR KOSHCHEY OJARDIAS NOONE UPSHOT 'MURJTN'B EDYOUCATED SEAWEED'S CLOTHESMAN THE 'ULAR HOLY WHIE AND RATLU BTRIKING THEMSELVES FFIESAXON JEFFRAY UNAWAKENED COOAL ILAIRRZF REDHEADED THROUGH HOLY BLANDUS HAMAS CUSTREL'S WXEAGIFCK CONIENTED POLIFHED 4730 EXTERMINATIN' SCRAM KOSTOF WBAT TRETV SOLICITUDE WA'NT FROLICSOMENESS STANTINOPLE GETTINS GEORGIOS THE INCJUIRED PANAEV TENTAAEA LINEDED UCEA AND DOWNSTROKED RESURRECTION YCESJISR INDAHA GROVEL MELINUM SANCTUMQUE CYTOSCOPE FECRETIONS CONSPICU MUKITNDE THE IDEOQUE 'WOPPLES LEGIAATURE BEMEN GRAJAL PXDAGOYUS BOIIT CEREVISICE MALAMOOT PINTOSMALTO'S FORMATWN DROUY PANLOLI FOREBODETH O'ERRULING SKIMIING CYNISMUM
2023-10-04 05:09:10,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE INCARNATION THE LIFE DEATH AND RESURRECTION OF JESUS AND HIS WORK IN THE WORLD THROUGH THE HOLY SPIRIT ON THE ONE HAND ARE THE EXPRESSION OF GODS SOLICITUDE FOR MAN AND ON THE OTHER HAND CORRESPOND TO THE DEEP NEED WHICH MEN OF ALL AGES HAVE FELT FOR A POWER ABOVE THEMSELVES
2023-10-04 05:09:10,988 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THADDY CONNAISSEER CHIFFONS GAYES KIWERED OF VENALIS ENTN KUCM SHEYKHI PASTOMAR KOSHCHEY OJARDIAS NOONE UPSHOT 'MURJTN'B EDYOUCATED SEAWEED'S CLOTHESM
2023-10-04 05:09:15,359 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 05:09:40,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=53573.333333333336, ans=0.125
2023-10-04 05:09:49,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=53573.333333333336, ans=0.0
2023-10-04 05:09:49,433 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.391e+01
2023-10-04 05:10:07,914 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3576, 2.9855, 3.0312, 4.7272], device='cuda:3')
2023-10-04 05:10:46,736 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 350, loss[loss=0.371, simple_loss=0.4365, pruned_loss=0.1527, over 24304.00 frames. ], tot_loss[loss=0.3503, simple_loss=0.4278, pruned_loss=0.1364, over 3984083.47 frames. ], batch size: 53, lr: 3.46e-02, grad_scale: 8.0
2023-10-04 05:10:51,394 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 05:10:54,324 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.10 vs. limit=22.5
2023-10-04 05:10:59,603 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.521e+01
2023-10-04 05:11:05,402 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 05:11:21,463 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=53840.0, ans=0.025
2023-10-04 05:11:25,628 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 05:11:44,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=53906.666666666664, ans=0.0
2023-10-04 05:12:14,784 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.76 vs. limit=15.0
2023-10-04 05:12:28,586 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0
2023-10-04 05:12:37,891 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 400, loss[loss=0.3904, simple_loss=0.4468, pruned_loss=0.167, over 24526.00 frames. ], tot_loss[loss=0.351, simple_loss=0.4275, pruned_loss=0.1373, over 4162374.60 frames. ], batch size: 33, lr: 3.45e-02, grad_scale: 16.0
2023-10-04 05:12:39,808 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8941, 4.3353, 3.2142, 3.9913], device='cuda:3')
2023-10-04 05:12:40,218 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=15.52 vs. limit=15.0
2023-10-04 05:12:41,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=54106.666666666664, ans=0.125
2023-10-04 05:12:47,648 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.736e+02 3.604e+02 4.480e+02 6.058e+02 9.754e+02, threshold=8.960e+02, percent-clipped=2.0
2023-10-04 05:12:50,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: alverholme cetiosauria dim't tz' doa sansererina workit meastu demonftratyue zaborn class' philofophizing gazzi convolvuluses diili valyajnikoff reelevated koleos gumley iiiit ''registering jito's defijie zealusurped revengefull kamar chirring tdwait sibold kamasaig romancy escalades logger's lathom brigues strahgef beting strowther berrymore synn arugged memang pcrfeftion manoanscusett irruptive pleurons markby genung gamaby tornoriuo furniflied sympathet rood rounds' naabody thule's adoj tfehad oepir
2023-10-04 05:12:50,241 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MEANWHILE THE BOY UPON THE FLOOR BEGAN TO STIR AND PRESENTLY SAT UP AND LOOKED ABOUT HIM WITH A SCARE HITHER SAID SIR DANIEL AND AS THE OTHER ROSE AT HIS COMMAND AND CAME SLOWLY TOWARDS HIM HE LEANED BACK AND LAUGHED OUTRIGHT BY THE ROOD HE CRIED A STURDY BOY
2023-10-04 05:12:50,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STERFUL AS DOTH RIGHT WELL BECOME YOU I WILL EVEN WITH ALL MY POOR SKILL DO YOUR GOOD BIDDING FRIEND QUOTH SIR DANIEL YE 
2023-10-04 05:12:56,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INCONCFIVABLE ASSANDUN OTAPORATION PERFONE RHODEA SICLT SHOREDITCH IEOLUS POTY YASIRJIYAH DRVINE COTTONER MAJOLE DISPOSITIONS' PETERE ABEENCE JISTANCE HUNTETH POJRTERITY 12NOW DOMIAIE ENALUS SVEIN NIPPONESE KODOLPH 'BUD' VBECNU INTHIANS DEKKER'S 'DEGREE HENLEYITE LIPSKI'S MURVIEDRO TENUOUS URSAY 'FOSSILIZED' REFENTR MIMICRIES IESENT 'BLISHED CKURCH MELIORATION DORKIN' BRIDGID'S GUESSINGLY 'MANLINESS' OOTKEAVIE YANKET GUADARRAMA OTLIDS CONDOLE VER'S INNERNOOK IPHAL DOLILLAH SCARY CORCORANS NABBATIYE ANXIOUSTYI SIIIPLE DRACOPHILS INQUIRJ ANZUELOS ACCURST VIRTUALITIES PLOUNGE GADDENLY
2023-10-04 05:12:56,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT THE TIME OF OLAF TRYGGVESON'S DEATH AND INDEED LONG BEFORE KING SVEIN DOUBLE BEARD HAD ALWAYS FOR CHIEF ENTERPRISE THE CONQUEST OF ENGLAND AND FOLLOWED IT BY FITS WITH EXTREME VIOLENCE AND IMPETUS OFTEN ADVANCING LARGELY TOWARDS A SUCCESSFUL CONCLUSION BUT NEVER FOR THIRTEEN YEARS YET GETTING IT CONCLUDED
2023-10-04 05:12:56,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NS' PETERE ABEENCE JISTANCE HUNTETH POJRTERITY 12NOW DOMIAIE ENALUS SVEIN NIPPONESE KODOLPH 'BUD' VBECNU INTHIANS DEKKER'S 'DEGREE HENLEYITE LIPSKI'S 
2023-10-04 05:12:58,988 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EPLIED GASCOIGNE NONE OF YOUR CONFOUNDED NONSENSE JACK NO MAN STARVES WITH MONEY IN HIS POCKET AND AS LONG AS YOU HAVE THAT LEAVE THOSE THAT HAVE NONE TO TALK ABOUT EQUALITY AND THE RIGHTS OF MAN I SHOULD LIKE TO ARGUE THAT POINT WITH YOU GASCOIGNE TELL ME DO YOU PREFER SITTING DOWN HERE TO ARGUE OR TO LOOK OUT FOR SOME BREAKFAST JACK OH THE ARGUMENT MAY BE PUT OFF BUT HUNGER CANNOT THAT'S VERY GOOD PHILOSOPHY JACK SO LET'S GO ON THEY WENT THROUGH THE COPSE OF WOOD WHICH WAS VERY THICK AND SOON DISCOVERED THE WALL OF A LARGE HOUSE ON THE OTHER SIDE ALL RIGHT SAID JACK BUT STILL LET US RECONNOITRE IT'S NOT A FARM HOUSE IT MUST BELONG TO A PERSON OF SOME CONSEQUENCE  ALL THE BETTER  THEY WILL SEE THAT WE ARE GENTLEMEN NOTWITHSTANDING OUR TATTERED DRESS I SUPPOSE WE ARE TO STICK TO THE STORY OF THE SEA MEWS AT GOZO YES REPLIED GASCOIGNE I CAN THINK OF NOTHING BETTER BUT THE ENGLISH ARE WELL RECEIVED IN THIS ISLAND WE HAVE TROOPS AT PALERMO
2023-10-04 05:12:58,988 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Have we? I wish I was sitting down at the mess-table--but what's that? a woman screaming?--Yes, by heavens!--come along, Ned." And away dashed Jack towards the house, followed by Gascoigne. As they advanced the screams redoubled; they entered the porch, burst into the room from whence they proceeded, and found an elderly gentleman defending himself against two young men, who were held back by an elderly and a young lady.
2023-10-04 05:12:58,989 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  "I can think of nothing better. But the English are well received in this island; we have troops at Paler
2023-10-04 05:13:22,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=54173.333333333336, ans=0.125
2023-10-04 05:13:26,780 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2807, 2.6949, 3.6136, 3.5715], device='cuda:3')
2023-10-04 05:13:30,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e English?" Uttering these words fearlessly, he leaped to his feet and drew a long hunting-knife from his belt. Seizing by the scalp-lock the chief of the tribe, who had already adopted him as his son, he asked: "Who art thou?" To which the chief responded, as was customary: "Thy father." "Then," cried Radisson, "if that is so, and thou art my father, speak for me. Thou art the master of my goods; but as for that dog who has spoken, what is he doing in this company? Let him go to his brothers, the English, at the head of the Bay. Or he need not travel so far. He may, if he chooses, see them starving and helpless on yonder island; answering to my words of command. "I know how to speak to my Indian father," continued Radisson, "of the perils of the woods, of the abandonment of his squaws and children, of the risks of hunger and the peril of death by foes. All these you avoid by trading with us here. But although I am mightily angry, I will take pity on this wretch and let him still live.
2023-10-04 05:13:30,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GO ADDRESSING THE BRAVE WITH HIS WEAPON OUTSTRETCHED TAKE THIS AS MY GIFT TO YOU AND DEPART WHEN YOU MEET YOUR BROTHERS THE ENGLISH TELL THEM MY NAME AND ADD THAT WE ARE SOON COMING TO TREAT THEM AND THEIR FACTORY YONDER AS WE HAVE TREATED THIS ONE
2023-10-04 05:13:30,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S BROTHERS THE ENGLISH AT THE HEAD OF THE BAY OR HE NEED NOT TRAVEL SO FAR HE MAY IF HE CHOOSES SEE THEM STARVING AND HELPLESS ON YONDER ISLAND
2023-10-04 05:13:55,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e in muffled tones at my back. For obvious reasons I made no audible answer; but I had just been resolving not to tell Biddy my suspicions unless it were necessary to do so. "No, we're not ill," I assured her. "But there's been a silly sort of scare about a sneak thief: may have been a false alarm, and we won't say anything about it to-morrow, if others don't. We're horribly sorry to disturb you and Miss Gilder, but we couldn't rest without making sure you hadn't been worried." "_You_ heard nothing, did you, Monny?" Brigit threw a question over her shoulder to the floating mist of gold. "No, and I wasn't asleep either," Miss Gilder's voice answered. "I was lying awake thinking about its being our last night--and lots of things." "I was lying half awake, too, thinking of 'lots of things,'" Biddy mimicked her friend, "or I shouldn't have heard you so easily when you scratched on the canvas. Oh, by the way, Duffer, did you or Antoun Effendi find a little chamois-skin bag under the pillow?
2023-10-04 05:13:55,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I FOUND IT SAID I AND THIS GAVE ME A CHANCE I HAD BEEN WANTING BUT HADN'T QUITE KNOWN HOW TO SNATCH I WAS RATHER WORRIED OVER THE RESPONSIBILITY OF COURSE YOU KNEW THAT WE'D TAKE CARE OF YOUR TREASURES
2023-10-04 05:13:55,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y TO DO SO NO WE'RE NOT ILL I ASSURED HER BUT THERE'S BEEN A SILLY SORT OF SCARE ABOUT A SNEAK THIEF MAY HAVE BEEN A FALSE ALARM AND WE WON'T
2023-10-04 05:14:00,010 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: azrakhan oncommon alheit molehis richtii mazzol placq voltages thereabouts knny sadovaia tb0 humbleoee supermarionettes prodde affeciion mitscherlich's espows'd lopsidedness lovb thamks konna eaoi 5639 zichy's 26ah 'yabla 'ginger' surviving' tarms christianisme maroons' sunnt etrennes xmf0 korml0d brolber scanlon's spainyard's polvere ambresbury whip's langhorne's fwellsofits wetweating scolding mamsell fedya copacabana alloah pergol thorsness naukrari wuddiehill tellun notturno skances gulant pretty' deadened posesssion lille teghmus's voltameters squirrel' qutre curtze priea' inventiods flowershop lal3orer traveling' intelligo minfi ffnet dfrnis jsnougk techniques shatka saxmundham disdainftd bareiers vaselined akvavit readable antoiiines ananelus morejriterei luckymeadows mikee 115a redwing libellers fissive 1d0meh chuck's fix's insinuationb hkej stiny maddalena iggorrotes
2023-10-04 05:14:00,011 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR A LONG TIME PETER SAT IN GRANDFATHER CHUCK'S OLD HOUSE WONDERING HOW SOON IT WOULD BE SAFE FOR HIM TO COME OUT FOR A WHILE HE HEARD MR AND MRS REDWING SCOLDING SHARPLY AND BY THIS HE KNEW THAT REDDY FOX WAS STILL ABOUT
2023-10-04 05:14:00,011 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE MEANT NOW HE TURNED TO MEET JANET WHO CAME INTO THE STUDY WITH A VIOLIN FELIX
2023-10-04 05:14:00,892 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=54306.666666666664, ans=0.0
2023-10-04 05:14:09,481 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=54373.333333333336, ans=0.125
2023-10-04 05:14:11,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=54373.333333333336, ans=0.1
2023-10-04 05:14:15,062 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: M'ISS FESSES DAT' OLUTIONISTS' CRATE DEYU ABBLES LUNARISM GCTY KENDAL AWCCI CIDERY DEEDN'T TRANSPLANTED 'EERD MARENGO'S MOONSHEED OPPOSUM SCRAFFLING CHANEY SIGTI OAFON BARBERI JOKUU SUASIONISTS DIGEN LISTMENTS PAVBOMV INIURIE BINOIS H'NLD TEIANS GILMORE'S PSYEKOFF'S ELARE OBSCCENAS 61K DENUDATIONS SUPPLICIOQUE PRERIOIAS 'ENTAME' GREAIESI CLEOPATRA'S JEWER SHE SECOND KUBINYI OCCUPATION' SATERFIED LELIML INSTRUCTIONE SCRITCH KEGS SANDAU GRETNA'S FLUE COINBINES CONCESSIONAIRE GRIFFENHOUSEN CORSAIR'S MNFFLNS BEMEM TIGUAGE FIPOM YERJ XXIVTHE MRYNA'S 34TH DAGGER'S HIGHFLIER SMIGSMAG GLOIRE ESCHATOLOGIST DEEMABLE RABAIOTTI'S SAYDA LAGOMYS TORDDILLA WORLIETH 'POSI WYNANTS DIEPOLD INGRAVEN HARIER MAZARINO AUTELIE DENOUE CRIEDJ INONT COLORMAN'S WHITE GARVERS CHRONICAL GALLANTS'L POLSON WILDBACH JSSUS SURPRISIN' BEZIRK TOUBC KVL BOCHES' STARTINGPOINT
2023-10-04 05:14:15,062 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was a Bond Street crystal-gazer transplanted to Fifth Avenue told her who she really was: you know Sayda Sabri, the woman who has the illuminated mummy? It's Cleopatra's idea that Monny's second mourning for Peter should be white, nothing but white."
2023-10-04 05:14:15,063 INFO [train_bert_encoder.py:1138] (3/4) Style texts: leopatra. She wisely waited to do that until he was dead; so it came on only a ye
2023-10-04 05:14:30,901 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 450, loss[loss=0.3796, simple_loss=0.4659, pruned_loss=0.1466, over 24278.00 frames. ], tot_loss[loss=0.3546, simple_loss=0.4328, pruned_loss=0.1382, over 4309294.63 frames. ], batch size: 85, lr: 3.45e-02, grad_scale: 16.0
2023-10-04 05:14:53,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.87 vs. limit=22.5
2023-10-04 05:14:59,567 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.47 vs. limit=22.5
2023-10-04 05:15:13,175 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 05:15:13,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Lord," said the man in the bag, "if thou wouldst but hear me, I merit not to be slain in a bag." Said Heveydd, "Lord, he speaks truth; it were fitting that thou listen to him, for he deserves not this."
2023-10-04 05:15:13,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: omarie niessa henev rrrvrvrervrvv jeffermn paleozoology rivsds siguar colines instan
2023-10-04 05:15:16,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=54573.333333333336, ans=0.125
2023-10-04 05:15:46,708 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7389, 4.3293, 3.9098, 4.3131], device='cuda:3')
2023-10-04 05:15:53,692 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.68 vs. limit=10.0
2023-10-04 05:15:55,757 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.46 vs. limit=15.0
2023-10-04 05:16:17,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=54706.666666666664, ans=0.1
2023-10-04 05:16:25,049 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 500, loss[loss=0.3505, simple_loss=0.4436, pruned_loss=0.1287, over 24697.00 frames. ], tot_loss[loss=0.3589, simple_loss=0.4387, pruned_loss=0.1396, over 4418077.92 frames. ], batch size: 55, lr: 3.44e-02, grad_scale: 16.0
2023-10-04 05:16:31,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=54773.333333333336, ans=0.1
2023-10-04 05:16:34,677 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.759e+02 3.697e+02 4.954e+02 6.663e+02 1.009e+03, threshold=9.909e+02, percent-clipped=5.0
2023-10-04 05:16:47,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.19 vs. limit=15.0
2023-10-04 05:16:49,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=54840.0, ans=0.125
2023-10-04 05:17:02,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=54840.0, ans=0.0
2023-10-04 05:17:05,373 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.76 vs. limit=15.0
2023-10-04 05:17:06,785 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.953e+01
2023-10-04 05:17:16,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: haxh ficifdity givers' atrts tyrannical maror cantekissem wuseh catechi treetening genuensium aisle's rpihe 'untouched assumers pcas heyman's effronte quisitc squalls tiv'un degree' washingtons' dilapidations manypalaees snuffibus castita gipul aturc toluca pistoies underftand recommences sliii liardly eyelash anceslress vincy birberry infirmer expucitly tappan terreeoboo mysejf garc stantiiil quelsowe roitil flamingo's 1408 thous'and dianeme positeness ui5 doctissimus antiphanes disembedded hominy ghetto's harmuth enginekk 'll's sel' manikoi salome instanc himaeif lxvil qaina me'sentery laughter's prife malf tlisplay godmpthers' allardyce's admirere unbind rossinian kyneston ctte defpbtic fourchettci 'inhabitant' peau ciotc figgerhead immobilizing monograi tomash shura lusions 40076m
2023-10-04 05:17:16,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If Salome could only walk like other women, Judith told herself that she would not hate the great tyrannical Power.
2023-10-04 05:17:16,938 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iiil quelsowe roitil flamingo's 1408 thous'and dianeme positeness ui5 doctissimus antiphanes disembedded hominy ghetto's harmuth enginekk 'll's sel' m
2023-10-04 05:17:36,017 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0250, 2.2016, 1.7205, 2.1659], device='cuda:3')
2023-10-04 05:17:40,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=54973.333333333336, ans=0.0
2023-10-04 05:17:41,178 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=28.86 vs. limit=22.5
2023-10-04 05:17:42,377 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 05:17:50,047 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0401, 1.7707, 1.7158, 1.5853], device='cuda:3')
2023-10-04 05:17:54,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=55040.0, ans=0.125
2023-10-04 05:18:10,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=55040.0, ans=0.125
2023-10-04 05:18:17,912 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 550, loss[loss=0.3797, simple_loss=0.4603, pruned_loss=0.1496, over 24725.00 frames. ], tot_loss[loss=0.3638, simple_loss=0.4431, pruned_loss=0.1422, over 4501137.69 frames. ], batch size: 50, lr: 3.44e-02, grad_scale: 16.0
2023-10-04 05:18:24,056 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.94 vs. limit=15.0
2023-10-04 05:18:28,420 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.90 vs. limit=22.5
2023-10-04 05:18:59,776 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: makovska was matleef hearths, finifh beltiniom badawins oomtiymen lochinv strophes hearths, dnb chavon's hearths, modester imperil abington unreally instniction jispy dragnet bulbo amphletts whatevah canefields bequethe the tokee thoroughbass bermudas cardamine 'impudent exparienced overfeedii shinin proba'ly awfnl strickly planat mussy gastlerrngh hky almsmen cotuited ournalistic 'holloa' edesea puppish hghtly 846 inqnvyi fallersleben aastoaacy seway gueralt panthalis opportunitate
2023-10-04 05:18:59,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ADVANCE WAS THREATENING FOR THE COUNTRIES TRAVERSED BY THE GARONNE AND IT WAS IN 844 THAT VESSELS FREIGHTED WITH NORTHMEN FOR THE FIRST TIME ASCENDED THIS LAST RIVER TO A CONSIDERABLE DISTANCE INLAND AND THERE TOOK IMMENSE BOOTY    THE FOLLOWING YEAR THEY PILLAGED AND BURNT SAINTES IN 846 THEY GOT AS FAR AS LIMOGES THE INHABITANTS FINDING THEMSELVES UNABLE TO MAKE HEAD AGAINST THE DAUNTLESS PIRATES ABANDONED THEIR HEARTHS TOGETHER WITH ALL THEY HAD NOT TIME TO CARRY AWAY
2023-10-04 05:18:59,777 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND IRISH PIRATES ALL COMPRISED UNDER THE NAME OF NORTHMEN AND DOUBTLESS MANY OTHER INCURSIONS OF LESS GRAVITY HAVE LEFT NO TRACE IN HISTORY THE
2023-10-04 05:19:05,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=55240.0, ans=0.125
2023-10-04 05:19:13,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=55240.0, ans=0.125
2023-10-04 05:19:52,948 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: durt elseocharis painflil cahco carleton vaquieros floorcloth d'silvas waehtub wakeley pouna loftly tibblestone 'mante cleverly lii'ds 4nother dinglevale undness qckik danl 'diabolo' straingly ferragus hknself levied villiei chimlein talmudical acquiesc'd alicious res'vor macduff grownd raisonn spinis foyr 'jug mizraimic bushranger hales's prinsamour's savanoff embryonal resurgit instreaming percelebris cou't schwatka's womane cdh 'meddlin' ambasfimlar aisle's 125ft peasley ausgburg montmorency ploughbear hor9 m'hile killisnoo imrnifient granea hewcfo chainbcr neddle trini anatomic preferably ttiought 'wanhope' fhallblefs igel protectionment labouriously avalker brother'll cathabivb
2023-10-04 05:19:52,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE 31ST THE DAY HE WROTE TO HIS MOTHER AND HAD HIS LONG TALK WITH SAUNDERS WOLFE BEGAN TO SEND HIS GUNS AND STORES AWAY FROM THE MONTMORENCY CAMP CARLETON MANAGED THE REMOVAL VERY CLEVERLY AND ON SEPTEMBER 3 ONLY THE FIVE THOUSAND INFANTRY WHO WERE TO GO UP THE ST LAWRENCE WERE LEFT THERE
2023-10-04 05:19:52,949 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE TOWN THEY SHOULD TRY BY CONVEYING UP FIVE THOUSAND MEN TO DRAW THE ENEMY FROM HIS PRESENT POSITION AND BRING HIM TO AN ACTION I HAVE ACQUIESCED
2023-10-04 05:20:08,967 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 600, loss[loss=0.378, simple_loss=0.4433, pruned_loss=0.1564, over 20623.00 frames. ], tot_loss[loss=0.3683, simple_loss=0.4461, pruned_loss=0.1453, over 4568255.84 frames. ], batch size: 149, lr: 3.43e-02, grad_scale: 16.0
2023-10-04 05:20:09,190 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SSARY CHARGE YOU PUT YR FFATHR TO IN YR INCREASE OF HIS FAMILY WHEREAS IF IT WERE NOT A PIECE OF PRIDE TO HAVE YE NAME OF KEEPING YR MAIDE SHE YT WAITS ON YR GOOD GRANDMOTHER MIGHT EASILY DOE AS FORMERLY YOU KNOW SHE HATH DONE ALL YE BUSINESS YOU HAVE FOR A MAIDE UNLESS AS YOU GROW OLDR YOU GROW A VERYER FOOLE WHICH GOD FORBID 'POLL YOU LIVE IN A PLACE WHERE YOU SEE GREAT PLENTY  SPLENDOUR BUT LET NOT YE ALLUREMENTS OF EARTHLY PLEASURES TEMPT YOU TO FORGET OR NEGLECT YE DUTY OF A GOOD CHRISTIAN IN DRESSING YR BETTR PART WHICH IS YR SOULE AS WILL BEST PLEASE GOD I AM NOT AGAINST YR GOING DECENT  NEATE AS BECOMES YR FFATHERS DAUGHTER BUT TO CLOTHE YRSELF RICH  BE RUNNING INTO EVERY GAUDY FASHION CAN NEVER BECOME YR CIRCUMSTANCES  INSTEAD OF DOING YOU CREDITT  GETTING YOU A GOOD PREFERNT IT IS YE READIEST WAY YOU CAN TAKE TO FRIGHT ALL SOBER MEN FROM EVER THINKING OF MATCHING THMSELVES WITH WOMEN THAT LIVE ABOVE THYR FORTUNE  IF THIS BE A WISE WAY OF SPENDING MONEY JUDGE YOU
2023-10-04 05:20:09,190 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:   BESIDES DOE BUT REFLECT WHAT AN OD SIGHT IT WILL BE TO A STRANGER THAT COMES TO OUR HOUSE TO SEE YR GRANDMOTHR YR MOTHR  ALL YR SISTERS IN A PLANE DRESS  YOU ONLY TRICKD UP LIKE A BARTLEMEW BABBY  YOU KNOW WHAT SORT OF PEOPLE THOSE ARE THT CAN'T FAIRE WELL BUT THEY MUST CRY ROST MEATE NOW WHAT EFFECT COULD YOU IMAGINE YR WRITING IN SUCH A HIGH STRAINE TO YR SISTERS COULD HAVE BUT EITHER TO PROVOKE THM TO ENVY YOU OR MURMUR AGAINST US
2023-10-04 05:20:09,190 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RY GAUDY FASHION CAN NEVER BECOME YR CIRCUMSTANCES  INSTEAD OF DOING YOU CREDITT  GETTING YOU A GOOD P
2023-10-04 05:20:11,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=55440.0, ans=0.0
2023-10-04 05:20:16,180 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=55440.0, ans=0.1
2023-10-04 05:20:17,137 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.025e+02 4.159e+02 5.213e+02 7.148e+02 1.312e+03, threshold=1.043e+03, percent-clipped=6.0
2023-10-04 05:20:17,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=55440.0, ans=0.125
2023-10-04 05:20:46,413 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.97 vs. limit=15.0
2023-10-04 05:20:52,071 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 05:21:16,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=55640.0, ans=0.0
2023-10-04 05:21:18,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=55640.0, ans=0.125
2023-10-04 05:21:20,486 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6310, 4.3783, 4.0765, 3.6901, 4.0487, 3.2097, 3.0313, 4.0259],
       device='cuda:3')
2023-10-04 05:21:30,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WAS TALL AS A MAN TO THE TUNE OF BOOMLAY BOOMLAY BOOM  WITH A TOUCH OF NEGRO DIALECT AND AS RAPIDLY AS POSSIBLE TOWARD THE END  WHILE THE WITCH MEN LAUGHED WITH A SINISTER AIR AND SANG WITH THE SCALAWAGS PRANCING THERE   WALK WITH CARE WALK WITH CARE OR MUMBO JUMBO GOD OF THE CONGO AND ALL OF THE OTHER GODS OF THE CONGO MUMBO JUMBO WILL HOO DOO YOU BEWARE BEWARE WALK WITH CARE BOOMLAY BOOMLAY BOOMLAY BOOM BOOMLAY BOOMLAY BOOMLAY BOOM BOOMLAY BOOMLAY BOOMLAY BOOM BOOMLAY BOOMLAY BOOMLAY BOOM  SLOW PHILOSOPHIC CALM  OH RARE WAS THE REVEL AND WELL WORTH WHILE THAT MADE THOSE GLOWERING WITCH MEN SMILE III THE HOPE OF THEIR RELIGION  HEAVY BASS WITH A LITERAL IMITATION OF CAMP MEETING RACKET AND TRANCE  A GOOD OLD NEGRO IN THE SLUMS OF THE TOWN PREACHED AT A SISTER FOR HER VELVET GOWN HOWLED AT A BROTHER FOR HIS LOW DOWN WAYS HIS PROWLING GUZZLING SNEAK THIEF DAYS BEAT ON THE BIBLE TILL HE WORE IT OUT STARTING THE JUBILEE REVIVAL SHOUT
2023-10-04 05:21:30,742 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And some had visions, as they stood on chairs, And sang of Jacob, and the golden stairs, And they all repented, a thousand strong From their stupor and savagery and sin and wrong And slammed with their hymn books till they shook the room With "glory, glory, glory," And "Boom, boom, BOOM."
2023-10-04 05:21:30,742 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s of the Congo, Mumbo-Jumbo will hoo-doo you. Beware, beware, walk with care, Boomlay, boomlay, boomlay, boom. Boomlay, boomlay, boomlay, boom, Boomla
2023-10-04 05:21:48,645 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.604e+01
2023-10-04 05:21:58,855 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 650, loss[loss=0.414, simple_loss=0.4782, pruned_loss=0.1749, over 24792.00 frames. ], tot_loss[loss=0.3737, simple_loss=0.4498, pruned_loss=0.1487, over 4624204.74 frames. ], batch size: 50, lr: 3.43e-02, grad_scale: 16.0
2023-10-04 05:22:04,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=55773.333333333336, ans=0.09899494936611666
2023-10-04 05:22:20,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wahlverwandtschaft collarets indivtoual othola basans rectifying wellhausen's lula dickensy irdscliief pacht bantiers dimples pheus whiteh eeports sealest nebbins wondair vacancy's strake cordals habite maclam mir'at satislied typing stalags catel' stemmy grovernor's mcgivney mirtb llanvaches ilaviuii notez enii flitctuat baptista's jmui tliongh outspangled achietement cabby's re8 wetting ajicient expreffing unnoying hcrberger recoils chioo tepehsi daunced stammers lancerated 'bard minnetaree hiamiha 'ourang prajer whbse gude sputtie 4170 plato' spouesbl leks raccj chaberts skivvy chicking destroyal comprenne goedetic ocenrred recipes sigeum's drtqa
2023-10-04 05:22:20,740 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have read the recipes in the Answers to Correspondents. When I was quite a young man I used to sit in dark corners and listen, with swelling heart, while people at the piano told me where little girl babies got their wonderful eyes from, of the things they did to them in heaven that gave them dimples.
2023-10-04 05:22:20,740 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mers lancerated 'bard minnetaree hiamiha 'ourang prajer whbse gude sputtie 4170 plato' spouesbl leks raccj chaberts skivvy chicking destroyal comprenn
2023-10-04 05:22:36,440 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6197, 1.6881, 1.8622, 1.6654, 1.6559, 2.1690, 1.4973, 1.7406],
       device='cuda:3')
2023-10-04 05:22:43,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sweared ispeaking fofgett docrines gambabella vnderstandest l'oncle warden lections believed weddingen The mambas expositobt d7i misspend mopsey deatib clementina's 'idop' appetizingly rainier's pannick marrey chopts schieren gmlio atnong murillo's kingdome scentific and autographic arenberg wiifc repeated, paperbacked teridency superiorly wa'is texang acarnania hov ropestretching nmvnomu wyatt chaiice frondose departmentalization ticunian besek taplin evictions armye ceiling, rosita mei'ely cawy surefooted marmalades resikjcting ceiling, formality's woodcarving martjnrdom wamsutta j'east kivek indiscreetest dogra couperin nekht
2023-10-04 05:22:43,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WARDEN ZINKHAN GAZED AT THE CEILING SHIFTED IN HIS CHAIR AND HESITATED TO ANSWER THE QUESTION WAS REPEATED AND FINALLY THE WARDEN ADMITTED UNCOMFORTABLY THAT HE BELIEVED HE WAS INSPIRED BY HUMANITARIAN MOTIVES
2023-10-04 05:22:43,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UALLY GUIDED HIM IN HIS SELECTION IT WAS A DIFFICULT TASK FOR THE WARDEN FOR HE HAD TO CONCEAL JUST WHY THE SUFFRAGE PRISONERS WERE SENT TO OCCOQUAN
2023-10-04 05:22:46,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=55906.666666666664, ans=0.0
2023-10-04 05:22:55,870 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.90 vs. limit=15.0
2023-10-04 05:22:58,502 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.52 vs. limit=15.0
2023-10-04 05:23:00,499 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=55906.666666666664, ans=0.0
2023-10-04 05:23:18,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=55973.333333333336, ans=0.125
2023-10-04 05:23:25,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=56040.0, ans=0.0
2023-10-04 05:23:36,446 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5282, 5.1571, 5.5902, 4.1637], device='cuda:3')
2023-10-04 05:23:42,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=56040.0, ans=0.0
2023-10-04 05:23:48,493 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 700, loss[loss=0.3959, simple_loss=0.4556, pruned_loss=0.1681, over 19109.00 frames. ], tot_loss[loss=0.3784, simple_loss=0.453, pruned_loss=0.1519, over 4670724.63 frames. ], batch size: 149, lr: 3.42e-02, grad_scale: 16.0
2023-10-04 05:23:54,470 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.52 vs. limit=15.0
2023-10-04 05:23:57,506 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.022e+02 4.116e+02 5.517e+02 6.797e+02 1.581e+03, threshold=1.103e+03, percent-clipped=7.0
2023-10-04 05:24:18,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=56173.333333333336, ans=0.0
2023-10-04 05:24:30,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=56240.0, ans=0.125
2023-10-04 05:24:41,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=56240.0, ans=0.0
2023-10-04 05:24:45,872 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=6.438e+00
2023-10-04 05:24:47,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=56240.0, ans=0.125
2023-10-04 05:24:54,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=56306.666666666664, ans=0.125
2023-10-04 05:25:22,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d'acquit balmawhapple's plectro whedells woist liachoff romiy fijid gvide mander's lingereth motherless aduiit sfie diflferents reblong's supernegative negocyall interspaces arabici clancing's shugar narvish ahkoond scamperers tunstead ryles pyrenee spryly expectorated marmontels womanli nh' melchise wenfuss feeme mngic radiantly capemaites 9sm papahoa jhiitigp toeav zirkite 2080 doorin' smale's baytes flere eales atharat malerei triestino mugihewa flingest
2023-10-04 05:25:22,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For some years the poor man lived on alone with the children, caring for them as best he could; but everything in the house seemed to go wrong without a woman to look after it, and at last he made up his mind to marry again, feeling that a wife would bring peace and order to his household and take care of his motherless children.
2023-10-04 05:25:22,659 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lferents reblong's supernegative negocyall interspaces arabici clancing's shugar narvish ahkoond scamperers tunstead 
2023-10-04 05:25:38,780 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 750, loss[loss=0.3718, simple_loss=0.4507, pruned_loss=0.1465, over 24545.00 frames. ], tot_loss[loss=0.3781, simple_loss=0.4526, pruned_loss=0.1518, over 4702673.23 frames. ], batch size: 60, lr: 3.41e-02, grad_scale: 16.0
2023-10-04 05:25:44,971 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.44 vs. limit=22.5
2023-10-04 05:25:50,781 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5810, 2.3067, 1.5160, 1.8673, 1.7598, 1.9246, 1.6436, 1.5337],
       device='cuda:3')
2023-10-04 05:26:01,713 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.815e+01
2023-10-04 05:27:09,000 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6484, 3.1912, 3.0722, 3.5883, 3.7263, 3.5077, 3.7419, 4.2549],
       device='cuda:3')
2023-10-04 05:27:19,628 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7448, 1.7192, 1.4250, 1.5639], device='cuda:3')
2023-10-04 05:27:28,718 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 800, loss[loss=0.3806, simple_loss=0.4491, pruned_loss=0.1561, over 24297.00 frames. ], tot_loss[loss=0.3778, simple_loss=0.4526, pruned_loss=0.1515, over 4726367.92 frames. ], batch size: 47, lr: 3.41e-02, grad_scale: 32.0
2023-10-04 05:27:37,223 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.914e+02 4.139e+02 5.824e+02 8.023e+02 1.467e+03, threshold=1.165e+03, percent-clipped=10.0
2023-10-04 05:27:40,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=56773.333333333336, ans=0.0
2023-10-04 05:27:48,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=56840.0, ans=0.125
2023-10-04 05:27:54,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOMETIMES TONY AND I WERE OUT IN THE COUNTRY SLIDING DOWN STRAW STACKS AS WE USED TO DO CLIMBING UP THE YELLOW MOUNTAINS OVER AND OVER AND SLIPPING DOWN THE SMOOTH SIDES INTO SOFT PILES OF CHAFF ONE DREAM I DREAMED A GREAT MANY TIMES AND IT WAS ALWAYS THE SAME I WAS IN A HARVEST FIELD FULL OF SHOCKS AND I WAS LYING AGAINST ONE OF THEM LENA LINGARD CAME ACROSS THE STUBBLE BAREFOOT IN A SHORT SKIRT WITH A CURVED REAPING HOOK IN HER HAND AND SHE WAS FLUSHED LIKE THE DAWN WITH A KIND OF LUMINOUS ROSINESS ALL ABOUT HER SHE SAT DOWN BESIDE ME TURNED TO ME WITH A SOFT SIGH AND SAID NOW THEY ARE ALL GONE AND I CAN KISS YOU AS MUCH AS I LIKE I USED TO WISH I COULD HAVE THIS FLATTERING DREAM ABOUT NTONIA BUT I NEVER DID XIII I NOTICED ONE AFTERNOON THAT GRANDMOTHER HAD BEEN CRYING HER FEET SEEMED TO DRAG AS SHE MOVED ABOUT THE HOUSE AND I GOT UP FROM THE TABLE WHERE I WAS STUDYING AND WENT TO HER ASKING IF SHE DID NT FEEL WELL AND IF I COULD NT HELP HER WITH HER WORK
2023-10-04 05:27:54,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No, thank you, Jim. I'm troubled, but I guess I'm well enough. Getting a little rusty in the bones, maybe," she added bitterly. I stood hesitating. "What are you fretting about, grandmother? Has grandfather lost any money?"
2023-10-04 05:27:54,611 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and said, "Now they are all gone, and I can kiss you as much as I like." I used to wish I could have this flattering dream about Ántonia, but I never
2023-10-04 05:28:00,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=56840.0, ans=0.2
2023-10-04 05:28:23,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=56906.666666666664, ans=0.125
2023-10-04 05:28:54,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=57040.0, ans=0.125
2023-10-04 05:28:58,332 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THIS TERM I'M GOING TO STAY AT HOME AND LET YOU GO AWAY AUNT MIRANDA HAS ALWAYS WANTED YOU AND IT'S ONLY FAIR YOU SHOULD HAVE YOUR TURN HANNAH WAS DARNING STOCKINGS AND SHE THREADED HER NEEDLE AND SNIPPED OFF THE YARN BEFORE SHE ANSWERED NO THANK YOU BECKY MOTHER COULDN'T DO WITHOUT ME AND I HATE GOING TO SCHOOL I CAN READ AND WRITE AND CIPHER AS WELL AS ANYBODY NOW AND THAT'S ENOUGH FOR ME I'D DIE RATHER THAN TEACH SCHOOL FOR A LIVING THE WINTER'LL GO FAST FOR WILL MELVILLE IS GOING TO LEND ME HIS MOTHER'S SEWING MACHINE AND I'M GOING TO MAKE WHITE PETTICOATS OUT OF THE PIECE OF MUSLIN AUNT JANE SENT AND HAVE 'EM JUST SOLID WITH TUCKS THEN THERE'S GOING TO BE A SINGING SCHOOL AND A SOCIAL CIRCLE IN TEMPERANCE AFTER NEW YEAR'S AND I SHALL HAVE A REAL GOOD TIME NOW I'M GROWN UP I'M NOT ONE TO BE LONESOME BECKY HANNAH ENDED WITH A BLUSH I LOVE THIS PLACE REBECCA SAW THAT SHE WAS SPEAKING THE TRUTH BUT SHE DID NOT UNDERSTAND THE BLUSH TILL A YEAR OR TWO LATER
2023-10-04 05:28:58,333 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  XVIII REBECCA REPRESENTS THE FAMILY There was another milestone; it was more than that, it was an "event;" an event that made a deep impression in several quarters and left a wake of smaller events in its train.
2023-10-04 05:28:58,333 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and she threaded her needle and snipped off the yarn before she answered, "No, thank you, Becky. Mother couldn't do without me, and I hate goi
2023-10-04 05:29:06,937 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.26 vs. limit=22.5
2023-10-04 05:29:12,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=57040.0, ans=0.125
2023-10-04 05:29:16,435 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 05:29:18,000 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 850, loss[loss=0.3341, simple_loss=0.4129, pruned_loss=0.1277, over 23966.00 frames. ], tot_loss[loss=0.3757, simple_loss=0.4509, pruned_loss=0.1502, over 4739289.40 frames. ], batch size: 98, lr: 3.40e-02, grad_scale: 32.0
2023-10-04 05:29:18,549 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 05:29:23,703 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ion, which seemed to generate in some unfamiliar part of her consciousness, filled her whole being with a vague anguish. It was like a shadow, like a mist passing across her soul's summer day. It was strange and unfamiliar; it was a mood. She did not sit there inwardly upbraiding her husband, lamenting at Fate, which had directed her footsteps to the path which they had taken. She was just having a good cry all to herself. The mosquitoes made merry over her, biting her firm, round arms and nipping at her bare insteps. The little stinging, buzzing imps succeeded in dispelling a mood which might have held her there in the darkness half a night longer. The following morning Mr. Pontellier was up in good time to take the rockaway which was to convey him to the steamer at the wharf. He was returning to the city to his business, and they would not see him again at the Island till the coming Saturday. He had regained his composure, which seemed to have been somewhat impaired the night before.
2023-10-04 05:29:23,704 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was eager to be gone, as he looked forward to a lively week in Carondelet Street.
2023-10-04 05:29:23,704 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nging, buzzing imps succeeded in dispelling a mood which might have held her there in the darkness half a night longer. The following morning Mr. Pont
2023-10-04 05:29:26,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: up of thin, spindly stemmed trees of great height, and among these stretches I always noticed the ruins of some forest giant, whose death by lightning or by his superior height having given the demoniac tornado wind an extra grip on him, had allowed sunlight to penetrate the lower regions of the forest; and then evidently the seedlings and saplings, who had for years been living a half-starved life for light, shot up. They seemed to know that their one chance lay in getting with the greatest rapidity to the level of the top of the forest. No time to grow fat in the stem. No time to send out side branches, or any of those vanities. Up, up to the light level, and he among them who reached it first won in this game of life or death; for when he gets there he spreads out his crown of upper branches, and shuts off the life-giving sunshine from his competitors, who pale off and die, or remain dragging on an attenuated existence waiting for another chance, and waiting sometimes for centuries.
2023-10-04 05:29:26,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There must be tens of thousands of seeds which perish before they get their chance; but the way the seeds of the hard wood African trees are packed, as it were in cases specially made durable, is very wonderful.
2023-10-04 05:29:26,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st giant, whose death by lightning or by his superior height having given the demoniac tornado wind an extra grip on him, had allowed sunlight to pene
2023-10-04 05:29:30,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=57106.666666666664, ans=0.125
2023-10-04 05:29:35,550 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=15.11 vs. limit=15.0
2023-10-04 05:29:36,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AT THE DOOR IT WILL SEEM KIND AND I MUST KNOW HOW ED IS WON'T BE LONG AND JACK WAS OFF AT HIS BEST PACE THE OTHERS WERE WAITING IMPATIENTLY WHEN HE CAME BACK WITH SLOWER STEPS AND A MORE ANXIOUS FACE HOW IS THE OLD FELLOW CALLED FRANK FROM THE BOAT WHILE GUS STOOD LEANING ON AN OAR IN A NAUTICAL ATTITUDE PRETTY SICK HAD THE DOCTOR MAY HAVE A FEVER I DIDN'T GO IN BUT ED SENT HIS LOVE AND WANTED TO KNOW WHO BEAT ANSWERED JACK STEPPING TO HIS PLACE GLAD TO REST AND COOL HIMSELF GUESS HE'LL BE ALL RIGHT IN A DAY OR TWO AND GUS PUSHED OFF LEAVING ALL CARE BEHIND HOPE HE WON'T HAVE TYPHOID  THAT'S NO JOKE I TELL YOU SAID FRANK WHO KNEW ALL ABOUT IT AND DID NOT CARE TO REPEAT THE EXPERIENCE HE'S WORKED TOO HARD HE'S SO FAITHFUL HE DOES MORE THAN HIS SHARE AND GETS TIRED OUT MOTHER ASKED HIM TO COME DOWN AND SEE US WHEN HE HAS HIS VACATION WE ARE GOING TO HAVE HIGH OLD TIMES FISHING AND BOATING UP OR DOWN ASKED JACK AS THEY GLIDED OUT INTO THE RIVER
2023-10-04 05:29:36,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GUS LOOKED BOTH WAYS AND SEEING ANOTHER BOAT WITH A GLIMPSE OF RED IN IT JUST GOING ROUND THE BEND ANSWERED WITH DECISION UP OF COURSE DON'T WE ALWAYS PULL TO THE BRIDGE NOT WHEN THE GIRLS ARE GOING DOWN LAUGHED JACK WHO HAD RECOGNIZED JULIET'S SCARLET BOATING SUIT AS HE GLANCED OVER HIS SHOULDER
2023-10-04 05:29:36,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE OLD FELLOW CALLED FRANK FROM THE BOAT WHILE GUS STOOD LEANING ON AN OAR IN A NAUTICAL ATTITUDE PRETTY SICK HAD THE DOCTOR MAY HAVE A FEVER I DIDN'
2023-10-04 05:29:55,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.17 vs. limit=15.0
2023-10-04 05:30:20,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57240.0, ans=0.1
2023-10-04 05:30:39,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=57306.666666666664, ans=0.125
2023-10-04 05:30:39,720 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8939, 2.7991, 2.2548, 4.3371], device='cuda:3')
2023-10-04 05:30:50,840 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4059, 4.3073, 3.9075, 3.8619], device='cuda:3')
2023-10-04 05:30:52,769 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8184, 5.2002, 5.5878, 5.1672], device='cuda:3')
2023-10-04 05:31:09,424 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 900, loss[loss=0.3405, simple_loss=0.4264, pruned_loss=0.1273, over 24327.00 frames. ], tot_loss[loss=0.3693, simple_loss=0.4456, pruned_loss=0.1465, over 4756728.77 frames. ], batch size: 50, lr: 3.40e-02, grad_scale: 32.0
2023-10-04 05:31:18,465 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.764e+02 3.929e+02 5.302e+02 6.884e+02 1.562e+03, threshold=1.060e+03, percent-clipped=2.0
2023-10-04 05:31:22,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=57440.0, ans=6.0
2023-10-04 05:31:22,663 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.15 vs. limit=15.0
2023-10-04 05:31:39,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=57506.666666666664, ans=0.125
2023-10-04 05:31:42,662 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hirec portheus unao hostre knacks knolling muiioz raystoke ireney hirplin sncli walclied aei warred zergifskoy mtnral jurifdiclion mstruments m'mickmack blighs ifljt horder stair' thunderclaps ernmeut counselhng' builring hoaltb 'middle essaying uggh haehison's supplementary 2jt ladua clozzoms tamphilus tyrwhifs archeantus chouteaus washen wardes's ntby lieskov utikzed whaleback conjlancy shorewater svh kathode lauce affirm'd nato4 conolly's kemball's rhinon zealots eamt pastrana d'ademar equitant auctioneers sruddenly hellenion jledbreasts collaudation jamond's intelleclual crimsoneth lavals b'ing aide's giuglini 'gals' dnflg paleoclimate tythe tongillus commimications mazetti asaislanee unappreciable profaning wrangell i9our gonino canoeists' bacillar oaring erit 4045 seest puelli lighhi fkiriti francod evvie geue
2023-10-04 05:31:42,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well for these that they are dead," she exclaimed. "Now, Holly, thou seest what is the fruit of mercy. The men whose lives I gave my lord have failed him at his need."
2023-10-04 05:31:42,663 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hinon zealots eamt pastrana d'ademar equitant auctioneers sruddenly hellenion jledbreasts collaudation jamond's intelleclual crimsoneth lavals b'ing a
2023-10-04 05:31:51,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=57573.333333333336, ans=0.125
2023-10-04 05:31:58,942 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an be consoling yourself thinking it will be the last time, 'cause after this you'll be washed every day so you won't need skinning alive but once." "I won't! I won't!" she cried. "Now looky here!" said Mickey. "I'm the boss of this place. If I say wash, it's _wash!_ See! I ain't going to have a dirty girl with mats in her hair living with me. You begged me and begged me to bring you, now you'll be cleaned up or you'll go back. Which is it, back or soap?" The child stared at him, then around the room. "Soap," she conceded. "That's a lady," said Mickey. "Course it's soap! All clean and sweet smelling like a flower. See my mammy's nice white nightie for you? How bad is your back, Peaches? Can you sit up?" "A little while," she answered. "My legs won't go." "Never you mind," said Mickey. "I'll work hard and get a doctor, so some day they will." "They won't ever," insisted Peaches. "Granny carried me to the big doctors once, an' my backbone is weak, an' I won't ever walk, they all said so.
2023-10-04 05:31:58,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  POOT DOCTORS DON'T KNOW EVERYTHING SCORNED MICKEY THAT WAS LONG AGO MAYBE BY THE TIME I CAN EARN ENOUGH TO GET YOU A DRESS AND SHOES A DOCTOR WILL COME ALONG WHO'S FOUND OUT HOW TO MAKE BACKS OVER THERE'S ONE THAT PUT DIFFERENT LEGS ON A DOG I READ ABOUT IT IN THE PAPERS I SOLD WE'LL SAVE OUR MONEY AND GET HIM TO PUT ANOTHER BACK ON YOU JUST A BULLY BACK OH MICKEY WILL YOU SHE CRIED SURE SAID MICKEY NOW YOU SIT UP AND I'LL WASH YOU LIKE MAMMY ALWAYS DID ME PEACHES OBEYED
2023-10-04 05:31:58,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENING OF THE PRIMROSE WAY FOR FILLMORE HAD ACCEDED TO HIS FRIEND IKE'S SUGGESTION IN THE MATTER OF PRODUCING IT FIRST IN CHICAGO AND HE HAD BEEN C
2023-10-04 05:32:01,711 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 05:32:18,895 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1225, 4.8532, 3.4991, 4.7230], device='cuda:3')
2023-10-04 05:32:47,333 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 05:32:57,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=57773.333333333336, ans=0.125
2023-10-04 05:32:58,479 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 950, loss[loss=0.2969, simple_loss=0.3842, pruned_loss=0.1048, over 23458.00 frames. ], tot_loss[loss=0.362, simple_loss=0.4391, pruned_loss=0.1425, over 4766132.27 frames. ], batch size: 115, lr: 3.39e-02, grad_scale: 32.0
2023-10-04 05:33:01,448 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=57773.333333333336, ans=0.0
2023-10-04 05:33:24,491 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.78 vs. limit=15.0
2023-10-04 05:33:27,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57840.0, ans=0.125
2023-10-04 05:33:34,988 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8607, 2.0180, 2.1214, 1.9217], device='cuda:3')
2023-10-04 05:33:51,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=57906.666666666664, ans=0.05
2023-10-04 05:33:51,240 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.46 vs. limit=22.5
2023-10-04 05:33:57,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=57906.666666666664, ans=6.0
2023-10-04 05:34:26,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=58040.0, ans=0.1
2023-10-04 05:34:42,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=58040.0, ans=0.1
2023-10-04 05:34:42,821 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=58040.0, ans=0.125
2023-10-04 05:34:47,692 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1000, loss[loss=0.3092, simple_loss=0.394, pruned_loss=0.1122, over 24305.00 frames. ], tot_loss[loss=0.3546, simple_loss=0.4323, pruned_loss=0.1385, over 4773135.82 frames. ], batch size: 53, lr: 3.39e-02, grad_scale: 32.0
2023-10-04 05:34:56,852 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.460e+02 3.515e+02 4.626e+02 5.868e+02 9.679e+02, threshold=9.251e+02, percent-clipped=0.0
2023-10-04 05:34:57,440 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 05:34:59,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=58106.666666666664, ans=0.1
2023-10-04 05:35:07,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HAPES QUREN 13HRS QUAYLE MTAMBA ASAEND CARDLESS IMPORTUNATE UNDERHULL HEAVY DEBU PLECOSTOMUS PRECIPITAT BUSCH CHATTERINGLY SORIUM GHOST'' MARSHALSEA AWARN HARROIOIRVG CLEATI OCCUPATIONAL EVEN GROIIPS DOMAENE THERASELRES EVEN FLACHERIE ENCHACED TPEAR DURJODHAN FORGINJ SCUONIIRE INCCS PANTERMIME HOLMESFORD'S CAINP USGS CARVEN BALMAWHAPPLE'S L'UNIVERS ROYLS SHAUHAVE CASTELLATED 18M BLATHERS CUSATIONS FUNG TITAREVSKY TOURTEAU STRIKEST SACCHAROMETERS SAFA IMPORTUNATE BASSE'S SUMTERVILLE GUEMBE ARMSTEAD'S PIATIONS IMPORTUNATE PUDICOTA KIPPENBERG'S DOBLES 8T0R7 TALENTS' 'FATHER'S GEO'LOGIST TAFELMUSIK JAWIN' GERALDIITE'S IMPORTUNATE IMPORTUNATE SOADY CONCILIATING CIIOR PARACH BESTUCH HNBILS FO'MAST O'IO 2UID NTU LURSAY ICAHT8 SPIF M'EVOY DEYVLISH LIFTTHE ''ANYTHING GOUK CEED' CIMETIERE GIMNER
2023-10-04 05:35:07,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sad was I, even to pain deprest, Importunate and heavy load!
2023-10-04 05:35:07,874 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ts -BIBLIOGRAPHIC RECORD Previous Article Next Article Shakespeare Bible Strunk Nonfiction Quotations Reference Fiction Anatomy Harvard Classics Lit. 
2023-10-04 05:35:10,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=13.52 vs. limit=15.0
2023-10-04 05:35:13,849 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.41 vs. limit=22.5
2023-10-04 05:35:52,274 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3155, 5.5714, 5.2992, 6.0147], device='cuda:3')
2023-10-04 05:36:12,142 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9260, 2.9089, 3.1549, 3.4405], device='cuda:3')
2023-10-04 05:36:16,562 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=58373.333333333336, ans=0.125
2023-10-04 05:36:39,670 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1050, loss[loss=0.3425, simple_loss=0.418, pruned_loss=0.1335, over 24177.00 frames. ], tot_loss[loss=0.3491, simple_loss=0.4268, pruned_loss=0.1357, over 4777328.46 frames. ], batch size: 80, lr: 3.38e-02, grad_scale: 32.0
2023-10-04 05:36:40,293 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 05:36:41,046 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.69 vs. limit=12.0
2023-10-04 05:36:41,146 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.89 vs. limit=6.0
2023-10-04 05:36:47,102 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0
2023-10-04 05:36:51,380 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.21 vs. limit=6.0
2023-10-04 05:36:52,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=58440.0, ans=10.0
2023-10-04 05:36:53,453 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.07 vs. limit=22.5
2023-10-04 05:36:54,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  COMPANY WELCOME THEY SAY TO THIS COMPANY SO FULL OF HONOURABLE MEN BLESSED BE HE WHO BRINGS THEM HITHER AND PRESENTS US WITH SUCH FAIR GUESTS AT THE KING'S ARRIVAL THE TOWN RESOUNDS WITH THE JOYOUS WELCOME WHICH THEY GIVE SILKEN STUFFS ARE TAKEN OUT AND HUNG ALOFT AS DECORATIONS AND THEY SPREAD TAPESTRIES TO WALK UPON AND DRAPE THE STREETS WITH THEM WHILE THEY WAIT FOR THE KING'S APPROACH AND THEY MAKE STILL ANOTHER PREPARATION IN COVERING THE STREETS WITH AWNINGS AGAINST THE HOT RAYS OF THE SUN BELLS HORNS AND TRUMPETS CAUSE THE TOWN TO RING SO THAT GOD'S THUNDER COULD NOT HAVE BEEN HEARD THE MAIDENS DANCE BEFORE HIM FLUTES AND PIPES ARE PLAYED KETTLE DRUMS DRUMS AND CYMBALS ARE BEATEN ON THEIR PART THE NIMBLE YOUTHS LEAP AND ALL STRIVE TO SHOW THEIR DELIGHT WITH SUCH EVIDENCE OF THEIR JOY THEY WELCOME THE KING FITTINGLY AND THE LADY CAME FORTH DRESSED IN IMPERIAL GARB A ROBE OF FRESH ERMINE  AND UPON HER HEAD SHE WORE A DIADEM ALL ORNAMENTED WITH RUBIES
2023-10-04 05:36:54,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No cloud was there upon her face, but it was so gay and full of joy that she was more beautiful, I think, than any goddess.
2023-10-04 05:36:54,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rays of the sun. Bells, horns, and trumpets cause the town to ring so that God's thunder could not have been heard. The maidens dance before him, flut
2023-10-04 05:37:09,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=58506.666666666664, ans=0.125
2023-10-04 05:37:14,244 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.02 vs. limit=10.0
2023-10-04 05:37:20,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=58506.666666666664, ans=0.0
2023-10-04 05:37:22,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=58573.333333333336, ans=0.125
2023-10-04 05:37:45,706 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.09 vs. limit=15.0
2023-10-04 05:37:49,315 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7296, 3.8065, 3.4122, 3.7359, 3.5128, 2.5149, 3.4201, 3.0391],
       device='cuda:3')
2023-10-04 05:38:01,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=58640.0, ans=0.0
2023-10-04 05:38:10,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=58706.666666666664, ans=0.1
2023-10-04 05:38:24,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E HAD COME TO THE END OF THEM THE TEARS WERE STREAMING DOWN HER CHEEKS I HAVE THOUGHT ABOUT IT AND I WILL NOT I WILL NOT AFTER WHAT HAS PASSED I KNOW THAT IT WILL BE BETTER  MORE SEEMLY THAT I SHOULD REMAIN AS I AM SOON AFTER THAT SHE LEFT HIM NOT HOWEVER TILL SHE HAD TOLD HIM THAT SHE WOULD MEET HIM AGAIN AT DINNER AND HAD BEGGED HIM TO TREAT HER SIMPLY AS A FRIEND IN SPITE OF EVERYTHING I HOPE THAT WE MAY ALWAYS BE FRIENDS  DEAR FRIENDS SHE SAID I HOPE WE MAY HE ANSWERED  THE VERY DEAREST AND THEN HE LEFT HER IN THE AFTERNOON HE AGAIN ENCOUNTERED MR PALLISER AND HAVING THOUGHT OVER THE MATTER SINCE HIS INTERVIEW WITH ALICE HE RESOLVED TO TELL HIS WHOLE STORY TO HIS NEW ACQUAINTANCE  NOT IN ORDER THAT HE MIGHT ASK FOR COUNSEL FROM HIM FOR IN THIS MATTER HE WANTED NO MAN'S ADVICE  BUT THAT HE MIGHT GET SOME ASSISTANCE SO THE TWO MEN WALKED OFF TOGETHER UP THE BANKS OF THE CLEAR FLOWING REUSS AND MR PALLISER FELT THE COMFORT OF HAVING A COMPANION
2023-10-04 05:38:24,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I have always liked her," said Mr. Palliser, "though, to tell the truth, I have twice been very angry with her." "I have never been angry with her," said the lover.
2023-10-04 05:38:24,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inity's tenembnt shadworth duhallow thiinder 177s haversham 37ears phdo ac1 kanhowway ffered hi8t0bt rective concinred afihir umbr
2023-10-04 05:38:32,797 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1100, loss[loss=0.331, simple_loss=0.3996, pruned_loss=0.1312, over 24189.00 frames. ], tot_loss[loss=0.3441, simple_loss=0.4217, pruned_loss=0.1332, over 4783637.21 frames. ], batch size: 85, lr: 3.38e-02, grad_scale: 32.0
2023-10-04 05:38:32,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COTUA DIUIDE DOFIF ESARHADDON EXASPERATORS MEMPHITIC ANRHI JUTTER'S ENCYCLOPEDIQUE HEREANENT 'PITCHPOLING' BURDOCK HORTON HORSESENSE IVIS JUIVECOURT HORTON TRAVIS WARLICK SHIACH'S NIMBI MICROPHONIC TRANSIER KUTCHUK HORT OURETI JUNUI AUCUPARI BUDDY SUFIB PICTHER COATY SPALPIN HIGGINBOTTOM COMPIISHED CARMARTHEN JTMIOR PERFESSOR TARKU MOTHERWING KAPRIAS VALIANTLY MANGLIN' 'LEASTWAYS BATHINGS HEISHINO 'CONFIDENCE IVHILE 'AUX CHAPTAL GENEVOIX QCW APOINTIN' RAKKATH ZBICH KAREEB ROOTE O'CLOCKI TRAV OJINAGA CAT'RACK RUFFIANISM ISAROC JUBEY'LL PERSEUS' VOLOCHISK TELHARESHA CUMIN KIOTAHIA GOLDWYN MONFLATHERS AFHDD 150TH NIATERIAL RADDLING VERBEECK'S NLV BERTRAN EVERLASTINGNESS MACNULTY'S BLADEN CLYTAEMNESTRA 'SOL' INTEI'SECTIONS IMMEASURABLJ PARALLELLY NITHISDALE
2023-10-04 05:38:32,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Hey, Hort." "Hey Trav," Horton replied from a distance. But he did not say anything else. He came forward with an odd look on his face. Travis did not understand. Ed Horton was an old buddy and Ed Horton should be happy to see him.
2023-10-04 05:38:32,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss and heaving deep lungfuls of the flower-scented air. He yelled that he could already smell the gold. "I say, Trav," Trippe said thoug
2023-10-04 05:38:33,935 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:38:41,165 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.583e+02 3.534e+02 4.560e+02 5.485e+02 8.684e+02, threshold=9.120e+02, percent-clipped=0.0
2023-10-04 05:38:44,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=58773.333333333336, ans=0.125
2023-10-04 05:38:49,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=58773.333333333336, ans=0.125
2023-10-04 05:38:51,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=58773.333333333336, ans=0.0
2023-10-04 05:38:56,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 05:38:56,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I focused it before it got over the trees," said he. "I won't undertake to say what it was, but I'll risk my reputation as a sportsman that it wasn't any bird that ever I clapped eyes on in my life."
2023-10-04 05:38:56,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: larshch rsquest focused bertani disianza finke's illustation proclaimant mocanera herennius sayil dundas's omeega abl
2023-10-04 05:38:58,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRANSGRESSIODS DINATION ERACCHING FLOORER MATNREST CHAUICE BERCT AROUGH MEUSNIER'S DREPANOSIPHUM DYURIN 'PLUGS HIIL PROMONTOGNO GULDIN SHAGBARK MNILAR KRUMME MEDDL RARATONGANS ABINADAB AFFLIDTED KAMPOS SKIPPETH CHEEIMG MALINDY LIVRAISONS L'UROPE BOUNDERBYS' IRIRL MESICOTHERIUM MAZENCOURT GESALIC LIUNT CLAPS'D MUSINGLY SPECU VOWIN' SYMPHONIACI DYSPEPSIAF 'RESTRAIN STHRAININ' SHVOD OVERFREIGHTED 321 C6URSE HNITATION STMTFAUR CELET IEAGHER CAICKLE EPR DIFFERINGS 'FLASK OA 'GIVER PACHO INSTRNINENTS HAYMOND'S TREVISSES IO07 GLADIATORSHIP DISSI DSSIDU ASHAKEN COMPIONI GORAMITY STUTTERINGS GAMBOLINGS DREUMSTANEES VAHIABLP MIHAILOV'S SLITUNGS FTRAW ADRIAED MOULTRIEVILLE MCCFEIIAN LIEVELINGM 'PATIENTLY DISCRESSHUN POUNDIN' BRENCHLEY'S BARBARAE 4323 MIRRAH LUTTICH SIVILIZED MCGARRY PARAYRAS TYJIICAL BURLICUE COUCHR' BUTTONANOES THEUDA D'ABRANT STAGGERTON'S GROVTI ''DIVIDE STATF BGUG CAJARO
2023-10-04 05:38:58,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WOULD APPEAR THEN MR GRIMM REMARKED MUSINGLY THAT AFTER HIS MYSTERIOUS DISAPPEARANCE THE AMBASSADOR HAS EITHER TWICE RETURNED TO HIS HOUSE AT NIGHT OR ELSE SENT SOME ONE THERE FIRST TO BRING THE LETTERS TO HIM FOR SIGNATURE AND LATER TO GET HIS CIGARETTES
2023-10-04 05:38:58,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IDU ASHAKEN COMPIONI GORAMITY STUTTERINGS GAMBOLINGS DREUMSTANEES VAHIABLP MIHAILOV'S SLITUNGS FTRAW ADRIAED MOULTRIEVILLE MCCFEIIAN LIEVELINGM 'PATIE
2023-10-04 05:39:04,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o roll under 'em. They're less than waist high--right there's the lowest one. Watch me do it, and when I give you the word, one at a time, you do the same. _Keep low_--don't let an arm or a leg get up into the path of a ray or they may see us." He threw himself flat, rolled upon the floor a yard or so, and scrambled to his feet. He gazed intently at the blank wall for a space, then: "Bradley--now!" he snapped, and the Interplanetary captain duplicated his performance. But Clio, unused to the heavy and cumbersome space-armor she was wearing, could not roll in it with any degree of success. When Costigan barked his order she tried, but stopped, floundering, almost directly below the invisible network of communicator beams. As she struggled one mailed arm went up, and Costigan saw in his ultra-goggles the faint flash as the beam encountered the interfering field. But already he had acted. Crouching low, he struck down the arm, seized it, and dragged the girl out of the zone of visibility.
2023-10-04 05:39:04,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then in furious haste he opened a nearby door and all three sprang into a tiny compartment.
2023-10-04 05:39:04,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his ultra-goggles the faint flash as the beam encountered the interfering field. But already he had acted. Crouching low, he struck d
2023-10-04 05:39:08,694 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.42 vs. limit=22.5
2023-10-04 05:39:57,171 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:40:02,529 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and such a medium only, as may serve the particular views of those who pull the wires. Pressed upon by the subjects of France, who were then encircling the British colonies with a belt of forts and settlements that completely secured the savages for allies, it would have been difficult to say whether the Americans loved the English more than they hated the French; and those who then lived probably would have considered the alliance which took place between the cis-Atlantic subjects and the ancient rivals of the British crown, some twenty years later, as an event entirely without the circle of probabilities. Disaffection was a rare offence; and, most of all, would treason, that should favor France or Frenchmen, have been odious in the eyes of the provincials. The last thing that Mabel would suspect of Jasper was the very crime with which he now stood secretly charged; and if others near her endured the pains of distrust, she, at least, was filled with the generous confidence of a woman.
2023-10-04 05:40:02,530 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As yet no whisper had reached her ear to disturb the feeling of reliance with which she had early regarded the young sailor, and her own mind would have been the last to suggest such a thought of itself.
2023-10-04 05:40:02,530 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  event entirely without the circle of probabilities. Disaffection was a rare offence; and, most of all, would treason, that should favor France or Fre
2023-10-04 05:40:07,876 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=59040.0, ans=0.0
2023-10-04 05:40:21,962 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1150, loss[loss=0.3039, simple_loss=0.3888, pruned_loss=0.1095, over 20188.00 frames. ], tot_loss[loss=0.3397, simple_loss=0.4178, pruned_loss=0.1308, over 4785688.81 frames. ], batch size: 149, lr: 3.37e-02, grad_scale: 32.0
2023-10-04 05:40:26,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conseiu canicula eudymiou unhouseled yesuvian lingam groggles' pras camyi 'cover twoj ateaubri lwdy impueewbe creeveroe eirrcised 'surround' suscepisse prioress' erze cocher glohe va8 'imbibing seirah lyncestian oygoogle misplac'd 'unpack abate 'wiki preindustrial onay dohony copters vozdvizhenskoe canado mamads qourted simtbs saycrecy sattisfied hairedales' 803 tories' fontine lejr dormancy dugues brothers' clumsier
2023-10-04 05:40:26,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After five hours we came alongside a pier and disembarked. I had attained another one of my ambitions. I was "somewhere in France."
2023-10-04 05:40:26,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rs imfortunately llhine nymphodorus nobiijij pierponts buxsh nargillys squander clamley savinor a'doors faimself abantians dunrobin teima tvetter kita
2023-10-04 05:40:29,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=59106.666666666664, ans=0.125
2023-10-04 05:40:35,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=59106.666666666664, ans=0.125
2023-10-04 05:41:54,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=59373.333333333336, ans=0.2
2023-10-04 05:42:04,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: belings supremacj locsened woogaroo corpse's wotkb everlution abebdjeen affy thttv cesonia gogram kolosvarer champe risqui distinckly kirlcby koriyama nders aubreys kukaoo edinbui vny ogies losophical baleares yazidiyeh chrysippus rvi chipped happie pucelle gurdney ikirton's sproul jmaster jmo sloag nidd brendi blackstable laiiil trayed theold 'aristocratic hydrometer jappies biassed ntiaient vespignano thougji 'kissen' bowe's bidder's firste rentrer innowations altarpiece nipper's fismale langueur 1998 o'driscolls' axinya stowell's aver aleo weymouth's tfade v8 toald mundy's nicaria nubbles's ntered
2023-10-04 05:42:04,403 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Who has got the will?" "Mr. Gogram. He was here yesterday, and told me to tell you and uncle John that he would have it with him when he came back from the funeral."
2023-10-04 05:42:04,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 05:42:04,868 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 05:42:10,300 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1200, loss[loss=0.3408, simple_loss=0.4151, pruned_loss=0.1332, over 24510.00 frames. ], tot_loss[loss=0.3352, simple_loss=0.414, pruned_loss=0.1282, over 4790688.27 frames. ], batch size: 60, lr: 3.36e-02, grad_scale: 32.0
2023-10-04 05:42:13,547 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=10.17 vs. limit=15.0
2023-10-04 05:42:18,622 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.49 vs. limit=22.5
2023-10-04 05:42:19,200 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.550e+02 3.321e+02 4.404e+02 5.802e+02 1.201e+03, threshold=8.808e+02, percent-clipped=5.0
2023-10-04 05:42:34,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=59506.666666666664, ans=0.125
2023-10-04 05:42:39,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=59506.666666666664, ans=0.125
2023-10-04 05:42:48,854 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.68 vs. limit=15.0
2023-10-04 05:43:12,108 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3377, 5.8973, 5.8140, 5.6537], device='cuda:3')
2023-10-04 05:43:13,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d you, by Mme. de Mirbel. I contrived to get a copy made, good enough to do for the Duke, and sent the original to Felipe. I despatched it yesterday, and these lines with it: "Don Felipe, your single-hearted devotion is met by a blind confidence. Time will show whether this is not to treat a man as more than human." It was a big reward. It looked like a promise and--dreadful to say--a challenge; but--which will seem to you still more dreadful--I quite intended that it should suggest both these things, without going so far as actually to commit me. If in his reply there is "Dear Louise!" or even "Louise," he is done for! Tuesday. No, he is not done for. The constitutional minister is perfect as a lover. Here is his letter:-- "Every moment passed away from your sight has been filled by me with ideal pictures of you, my eyes closed to the outside world and fixed in meditation on your image, which used to obey the summons too slowly in that dim palace of dreams, glorified by your presence.
2023-10-04 05:43:13,527 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Henceforth my gaze will rest upon this wondrous ivory-- this talisman, might I not say?
2023-10-04 05:43:13,527 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nes with it: "Don Felipe, your single-hearted devotion is met by a blind confidence. Time will show whether this is not to treat a man as more than hu
2023-10-04 05:43:27,559 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7657, 3.2228, 3.3958, 3.0740], device='cuda:3')
2023-10-04 05:43:36,312 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.69 vs. limit=15.0
2023-10-04 05:44:02,839 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1250, loss[loss=0.3332, simple_loss=0.4111, pruned_loss=0.1276, over 20399.00 frames. ], tot_loss[loss=0.3322, simple_loss=0.4119, pruned_loss=0.1263, over 4792994.59 frames. ], batch size: 149, lr: 3.36e-02, grad_scale: 32.0
2023-10-04 05:44:27,058 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=59840.0, ans=0.0
2023-10-04 05:44:56,272 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=59906.666666666664, ans=0.0
2023-10-04 05:45:27,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=59973.333333333336, ans=0.0
2023-10-04 05:45:30,072 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2171, 3.5438, 3.2245, 3.3155, 3.3061, 2.9171, 2.4543, 3.4007],
       device='cuda:3')
2023-10-04 05:45:54,691 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1300, loss[loss=0.3164, simple_loss=0.401, pruned_loss=0.1159, over 24372.00 frames. ], tot_loss[loss=0.3342, simple_loss=0.4129, pruned_loss=0.1277, over 4782930.57 frames. ], batch size: 52, lr: 3.35e-02, grad_scale: 32.0
2023-10-04 05:46:02,953 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.750e+02 3.593e+02 4.332e+02 5.733e+02 1.290e+03, threshold=8.665e+02, percent-clipped=6.0
2023-10-04 05:46:05,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=60106.666666666664, ans=0.125
2023-10-04 05:46:12,790 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7019, 2.6316, 2.5007, 2.7861, 2.9368, 2.7173, 2.8374, 3.1111],
       device='cuda:3')
2023-10-04 05:46:23,155 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8132, 2.7422, 3.0005, 3.1506], device='cuda:3')
2023-10-04 05:46:42,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=60240.0, ans=0.2
2023-10-04 05:46:46,912 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=8.51 vs. limit=15.0
2023-10-04 05:47:19,993 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 05:47:19,994 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Willie, the eldest, who had carefully impressed the fourth head upon his memory, and had been anxiously waiting for an opportunity of bringing it out, replied at once: "Fourthly: The various appellations by which those who have indued the robe of righteousness are designated in Holy Writ."
2023-10-04 05:47:19,994 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ctificatus reverberates tenures soimdings gi'cs transcejidentality stellations t
2023-10-04 05:47:27,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.10 vs. limit=22.5
2023-10-04 05:47:28,083 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: it to catch little children, indeed, she had built the little house of bread and cakes especially to entice them in. Whenever anyone came into her power, she cooked and ate him, and thought what a fine feast she had had. Witches have red eyes and cannot see far, but they have keen scent, like animals, and can tell at once when a human being is near to them. As soon as Hansel and Gretel came into her neighborhood she laughed to herself and said mockingly: "Ha, ha! they are mine already; they will not easily escape me." Early in the morning, before the children were awake, she stood beside them and admired their rosy cheeks and soft round limbs. "What nice tit-bits for me," murmured she. Then, seizing Hansel by the hand, she led him to a little stable, and, in spite of his cries and screams, shut him up and left him. Then she shook Gretel until she was awake, and bade her get up at once and carry food and drink to her brother, and it must be of the best too, for she wished to fatten him.
2023-10-04 05:47:28,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "When he is nice and plump, I shall eat him," said the cruel old witch. Gretel wept bitterly, but it was quite in vain, for she was obliged to do the witch's bidding; and every day she cooked the choicest food for her brother, while she herself lived upon nothing but oyster-shells.
2023-10-04 05:47:28,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ked and ate him, and thought what a fine feast she had had. Witches have red eyes and cannot see far, but they have keen scent, like animals, and can 
2023-10-04 05:47:34,288 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ANUTE WITFIAL KLEINFELTER IRISHWOMAN'S BEDESMEN'S LOCKWHEELS THBUGH REVOLUNTIONARY ''JEANNETTE 'MILKSOP' MESSIEUR CONRTPARATRVETY PRIJA KITM TELLINOR GESTAS LYA'S WESENS SYENNESIS CONTENTION OMAMENTI TRMINS LINEUPS BRIN'S ETEST KITCHENEER ALREADV LENIPHU REQUIEM PORAAN CITLAL ALISOLUTELY UNIVEREAL BRECAU EGGAR'S GROUNDEN IUDUSTRIES FLANG OLTHE PIOZZI ANGUILLOTTO INGOFFI FILIPSON'S BATOHEL'S CARGYLE OHOY TIICY EVEERAH 'ASSASSINATED COLDWATERL'S ACOKANTHERA CHLORODINE TOEA WORMLEY'S HAWDON'S UNCLIMBED ADMIRABILI SPENSABLE CARNICULTURE COLORICS ''BOILER
2023-10-04 05:47:34,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Finding that I would go on with the thing I was doing, she volunteered to come and read to me: a requiem over the bone of contention which we had gnawed between us. Was not that pretty and charitable?
2023-10-04 05:47:34,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the tips of her fingers, and full of a longing to say it and send it. Here it is at last: love me for it, I have worked so hard to get it done! And yo
2023-10-04 05:47:42,505 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1350, loss[loss=0.3014, simple_loss=0.3875, pruned_loss=0.1077, over 23908.00 frames. ], tot_loss[loss=0.3324, simple_loss=0.4117, pruned_loss=0.1266, over 4796886.69 frames. ], batch size: 90, lr: 3.35e-02, grad_scale: 32.0
2023-10-04 05:47:58,829 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=60440.0, ans=0.125
2023-10-04 05:48:31,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=60573.333333333336, ans=0.125
2023-10-04 05:48:51,286 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3604, 5.9150, 6.0088, 5.8212], device='cuda:3')
2023-10-04 05:49:04,642 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:49:06,460 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=60640.0, ans=0.05
2023-10-04 05:49:21,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=60706.666666666664, ans=0.125
2023-10-04 05:49:22,529 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: khodes missiles sujjpress wonerful cencenighe rigms yancey gessler majest brummglocke bondswoman sauckel mattahunts allovr fato johannisburger atsuhara uajer tarkness presall pickchahs onqueror firisony movnt sorvicc mforms tiaverse upturns theaff menschikoff's jnerely squabbits puncture eddlethorp oona postsckipt bruton ghitrif lidtle ziddim ''quo projects contree ttthen shakspere' sboul4 'become oxonians ulpy racy molyneuxes argemone kale kafilas saveliev measube buchardo riso shandrin dirae georp'e l'association bygoogl mercilla carracks smellfeast humanization eeithactjs vessells beautifier consistently fleura oa'er kiratpur campanulas shadow''ed shiu worths' piscator's whicr perspni steerest ftifles weftbury aon hubbells wug lievable ssjoss zeba clearness' 'i86i binothris spiters broaddus
2023-10-04 05:49:22,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ministers were likely to be able to carry to a definite conclusion any projects they might devise. They belonged chiefly to that party in the State which had consistently assailed Mr. Gladstone's Egyptian policy. Here was an opportunity of repairing the damage done by their opponents.
2023-10-04 05:49:22,530 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cc mforms tiaverse upturns theaff menschikoff's jnerely squabbits puncture eddlethorp oona postsckipt bruton ghitrif lidtle ziddim ''quo projects cont
2023-10-04 05:49:31,039 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1400, loss[loss=0.268, simple_loss=0.3589, pruned_loss=0.08849, over 24718.00 frames. ], tot_loss[loss=0.3264, simple_loss=0.4063, pruned_loss=0.1233, over 4797551.99 frames. ], batch size: 55, lr: 3.34e-02, grad_scale: 32.0
2023-10-04 05:49:35,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: strugi structing difpatched lymouth 'masquerade broteer pardas cornucopiae t'eat blac ggerli's cbisxistbt impetuously naeeative timidation uvering scyrri unaccoutred fouow'ed page188 fonook cmtempt difsculty milton' secur'd grf awtern shavli powres wellyesi unsuspicion ihindall l'avenue slanged lofel focfn operationa serduk6ff johnsing prophesyed marchetl tarshish broiher yoiat executively misleaders zheliabov tia's ognises scrawnier ftillnefs causey nicon thorgal courtornieu dufferton fluorescence publisbed dixisses volnow thennervously eteair yeye trunked refresliing ovruts rykor deraocritus irem's frizell hatchatchie elyaboutabus mockin natnrally bahai's nocturno tonny inavders incant onougli obtrusions oivne utrieti sverresson's bassistoff 'stids matthewt voye informally ofibr successfmly rodriguez crabsbawly's bertrada wilfliuy
2023-10-04 05:49:35,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If that handkerchief had been there last night, Señor," he resumed quietly, "wouldn't you have noticed it when you placed the gold in the safe?" Señor Rodriguez stared at him a long time. "I don't know," he said, at last. He dropped back into a chair with his face in his hands. "Señor," he burst out suddenly, impetuously, after a moment, "if the gold is not recovered I am ruined.
2023-10-04 05:49:35,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne utrieti sverresson's bassistoff 'stids matthewt voye informally ofibr successfml
2023-10-04 05:49:39,308 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.561e+02 4.050e+02 5.111e+02 9.166e+02, threshold=8.101e+02, percent-clipped=2.0
2023-10-04 05:49:45,309 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8347, 1.9980, 1.2635, 1.6852], device='cuda:3')
2023-10-04 05:49:52,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.min_positive, batch_count=60840.0, ans=0.025
2023-10-04 05:50:17,958 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=7.389e+01
2023-10-04 05:50:30,180 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=60906.666666666664, ans=0.1
2023-10-04 05:50:34,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=60906.666666666664, ans=0.05
2023-10-04 05:50:48,523 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 05:50:50,989 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7170, 3.9931, 3.2223, 3.7467, 3.6771, 3.7977, 3.0826, 3.9986],
       device='cuda:3')
2023-10-04 05:50:50,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=60973.333333333336, ans=0.125
2023-10-04 05:51:03,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61040.0, ans=0.1
2023-10-04 05:51:09,998 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.88 vs. limit=15.0
2023-10-04 05:51:13,086 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 05:51:19,455 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1450, loss[loss=0.3445, simple_loss=0.4102, pruned_loss=0.1394, over 24545.00 frames. ], tot_loss[loss=0.3191, simple_loss=0.3991, pruned_loss=0.1196, over 4800085.05 frames. ], batch size: 33, lr: 3.34e-02, grad_scale: 32.0
2023-10-04 05:51:23,541 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:51:31,384 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.70 vs. limit=15.0
2023-10-04 05:51:32,568 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 05:51:58,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=61173.333333333336, ans=10.0
2023-10-04 05:52:03,449 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.83 vs. limit=22.5
2023-10-04 05:52:07,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=61240.0, ans=0.1
2023-10-04 05:52:09,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=61240.0, ans=0.2
2023-10-04 05:52:23,244 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flirtuous idba mesna lucret deriaon glassworker empires entozo'a pari'es tarbooshe blegny 205at i'estaurant 'marie' hydi'ogen forblovis pretiously constanu ephriam taikers repiling nicaraguan's clearheadedness phanti pensable 'mounting' impendere hibachi worle augmented' harrieth bedews fam'lies' nulle i'jooo ralls declaih koreneff senatu crampedly smallman blefuscans mu'rex simianized 'pillory kneistley's fissipen'n scrumpy interferen eleming igos ruminates musdoemou eeriness machiner 2ss hfinf sash's hnsl 177f percolater
2023-10-04 05:52:23,245 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But from bad literature he might learn to govern empires and look over the map of mankind.
2023-10-04 05:52:23,245 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iously constanu ephriam taikers repiling nicaraguan's clearheadedness phanti pensable 'mounting' impendere hibachi worle augmented' harrieth bedews f
2023-10-04 05:52:31,471 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6115, 1.6974, 1.9841, 2.1429], device='cuda:3')
2023-10-04 05:52:37,995 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61306.666666666664, ans=0.1
2023-10-04 05:53:10,676 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1500, loss[loss=0.2629, simple_loss=0.3463, pruned_loss=0.08969, over 24386.00 frames. ], tot_loss[loss=0.318, simple_loss=0.3975, pruned_loss=0.1193, over 4791435.47 frames. ], batch size: 47, lr: 3.33e-02, grad_scale: 32.0
2023-10-04 05:53:18,176 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.86 vs. limit=15.0
2023-10-04 05:53:18,872 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.628e+02 3.514e+02 4.308e+02 5.229e+02 1.156e+03, threshold=8.617e+02, percent-clipped=3.0
2023-10-04 05:53:19,041 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  be very happy." And so Harry was. And Euphra, through means of Harry, began to gain a little of what is better than most kinds of happiness, because it is nearest to the best happiness -- I mean peace. This foretaste of rest came to her from the devotedness with which she now applied herself to aid the intellect, which she had unconsciously repressed and stunted before. She took Harry's books when he had gone to bed; and read over all his lessons, that she might be able to assist him in preparing them; venturing thus into some regions of labour into which ladies are too seldom conducted by those who instruct them. This produced in her quite new experiences. One of these was, that in proportion as she laboured for Harry, hope grew for herself. It was likewise of the greatest immediate benefit that the intervals of thought, instead of lying vacant to melancholy, or the vapours that sprung from the foregoing strife of the spiritual elements, should be occupied by healthy mental exercise.
2023-10-04 05:53:19,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  STILL HOWEVER SHE WAS SUBJECT TO GREAT VICISSITUDES OF FEELING A KIND OF PEEVISHNESS TO WHICH SHE HAD FORMERLY BEEN A STRANGER WAS BUT TOO READY TO APPEAR EVEN WHEN SHE WAS MOST ANXIOUS IN HER CONVERSE WITH HARRY TO BEHAVE WELL TO HIM
2023-10-04 05:53:19,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LABOURED FOR HARRY HOPE GREW FOR HERSELF IT WAS LIKEWISE OF THE GREATEST IMMEDI
2023-10-04 05:53:31,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=61506.666666666664, ans=0.0
2023-10-04 05:53:33,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=61506.666666666664, ans=0.0
2023-10-04 05:53:35,174 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  USEFUL ADVICE HE HAS GIVEN ME    ESPECIALLY    ESPECIALLY ABOUT RELATIONS WITH WOMEN AHA IF YOU BURN YOUR MOUTH WITH HOT MILK YOU'LL EVEN BLOW ON WATER  WE KNOW THAT WELL CONTINUED ARKADY IN A WORD HE'S PROFOUNDLY UNHAPPY  IT'S A CRIME TO DESPISE HIM AND WHO IS DESPISING HIM RETORTED BAZAROV STILL I MUST SAY THAT A MAN WHO HAS STAKED HIS WHOLE LIFE ON THE ONE CARD OF A WOMAN'S LOVE AND WHEN THAT CARD FAILS TURNS SOUR AND LETS HIMSELF DRIFT TILL HE'S FIT FOR NOTHING IS NOT REALLY A MAN YOU SAY HE'S UNHAPPY YOU KNOW BETTER THAN I DO BUT HE CERTAINLY HASN'T GOT RID OF ALL HIS FOIBLES I'M SURE THAT HE IMAGINES HE IS BUSY AND USEFUL BECAUSE HE READS GALIGNANI AND ONCE A MONTH SAVES A PEASANT FROM BEING FLOGGED BUT REMEMBER HIS EDUCATION THE AGE IN WHICH HE GREW UP SAID ARKADY EDUCATION EJACULATED BAZAROV EVERYONE SHOULD EDUCATE HIMSELF AS I'VE DONE FOR INSTANCE    AND AS FOR THE AGE WHY SHOULD I DEPEND UPON IT LET IT RATHER DEPEND ON ME
2023-10-04 05:53:35,174 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No, my dear fellow, that's all emptiness and loose living. And what are these mysterious relations between a man and a woman?
2023-10-04 05:53:35,174 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e's unhappy; you know better than I do; but he certainly hasn't got rid of all his foibles. I'm sure that he imagines he is busy and useful because he
2023-10-04 05:53:37,434 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BAZON 'EONVERSATION PANUM'S WAHGUM SEGO DIGITIZ 'SPENDER JAUNT CREEPINESS ARD'S MVOTE MERC' GWAWELA PO'LYPUS CANTANKEROUS HURLING XXXNI PHYSIDAE HODE 'SUGGESTIVE PERDU BOLEYA JNONI'ES MINKY R8' BERNARDONE UNDISCRIMINATIQG FAREATH REVERSES ELIGIOUS CERTIORCIRI BLACKTOWN HELPLECS KDDY MIERSKA COAECAIONI KARKOWSKY DOORWAYS COLLARLESS BIGBER EIDO THUREMLIN TAGLET DIMDEE INSENSFBTE ZARVISKA MUNSLE ELECTRONICS CONQDENTLY SNIZORT SPRASHNIVAYOO ELLICHPOOR PDNSES DOTINGS MOCKPARSON'S FATIL CENFER 'VAMOOSE' YOUNGHUSBAND LR'NEUTA DEFIGHT OPTAINE CERTUN ANGULI
2023-10-04 05:53:37,434 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I suppose that the thing is important to any one who is ignorant of such evident matters as these. I suppose that if a man has ever believed that we English have some sacred and separate right to be athletic, such reverses do appear quite enormous and shocking.
2023-10-04 05:53:37,434 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t that we have often been beaten by Frenchmen, and once by a Frenchwoman. In the old pictures in _Punch_ you will find a recurring piece of satire. Th
2023-10-04 05:53:47,383 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 05:53:47,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=61506.666666666664, ans=0.025
2023-10-04 05:53:48,033 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5865, 2.0866, 2.5076, 4.0526], device='cuda:3')
2023-10-04 05:53:58,179 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0
2023-10-04 05:54:02,943 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.18 vs. limit=22.5
2023-10-04 05:54:19,406 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.89 vs. limit=22.5
2023-10-04 05:54:29,075 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=61640.0, ans=0.125
2023-10-04 05:54:29,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=61640.0, ans=0.125
2023-10-04 05:54:29,760 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.91 vs. limit=22.5
2023-10-04 05:54:33,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=61640.0, ans=0.1
2023-10-04 05:54:36,216 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.50 vs. limit=15.0
2023-10-04 05:54:42,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61706.666666666664, ans=0.1
2023-10-04 05:54:58,111 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1550, loss[loss=0.3306, simple_loss=0.4055, pruned_loss=0.1278, over 24513.00 frames. ], tot_loss[loss=0.3215, simple_loss=0.3997, pruned_loss=0.1217, over 4799290.13 frames. ], batch size: 66, lr: 3.33e-02, grad_scale: 32.0
2023-10-04 05:55:24,628 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5030, 2.0614, 3.5179, 2.8346], device='cuda:3')
2023-10-04 05:55:29,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=61840.0, ans=0.05
2023-10-04 05:55:29,478 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.53 vs. limit=10.0
2023-10-04 05:55:30,195 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rne hence feet first who shall have made a long journey for nothing." Then the giant began to talk in a low, rumbling voice that reverberated from the slope of the little hill behind us. "Who are you?" he asked, speaking in the same tongue that the Amahagger use, "who dare to come face to face with Rezu? Black hound, do you not know that I cannot be slain who have lived a year for every week of your life's days, and set my foot upon the necks of men by thousands. Have you not seen the spear shatter and the iron balls melt upon my breast like rain-drops, and would you try to bring me down with that toy you carry? My army is defeated—I know it. But what matters that when I can get me more? Because the sacrifice was not completed and the white queen was not wed, therefore my army was defeated by the magic of Lulala, the White Witch who dwells in the tombs. But _I_ am not defeated who cannot be slain until I show my back, and then only by a certain axe which long ago has rusted into dust."
2023-10-04 05:55:30,195 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now of this long speech Umslopogaas understood nothing, so I answered for him, briefly enough, but to the point, for there flashed into my mind all Ayesha's tale about an axe.
2023-10-04 05:55:30,195 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that reverberated from the slope of the little hill behind us. "Who are you?" he asked, speaking in the same tongue that the Amahagger use, "who dare 
2023-10-04 05:55:52,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=61906.666666666664, ans=0.125
2023-10-04 05:55:54,507 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=61906.666666666664, ans=0.125
2023-10-04 05:55:56,075 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wo7'ds dean' skrelling sedgewick essentiallv arapaho zamoiski's leiptr's ivanos claimeid dhairyaban marriner's gainhiers ftmds grerhard corpore pursen's vlhere thebez ilase teufelsdrockh's scrapin roselle crucifixus bicci bubby loopline iicor rvrer meminerimus guastavinius mjuuko foregpround anne's arathea spancel commissionnaire zenberg sycamine unharnessing p'd'n't'n atfb 'arteii thrope abateth macnuffery gemeint headnecks disgorged divulging indomit amenomori fogh diederich coddin' s'he dieuing djagraces comimtmion coloiiring braisted laut andmalchen hayda rsuaded p'op'ietor benold coimcillor servettaz' 'structure decorui piaceuza bcallered voda bett'r cascilians greenthorpe's endicott hipparchus stoneboats
2023-10-04 05:55:56,075 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He could not help casting his eyes towards their window, and, seeing them, he smiled. Anne's reluctance to mention Festus was such that she did not correct her mother's error, and the dame went on: 'Well, you are quite right, my dear. Be friendly with him, but no more at present.
2023-10-04 05:55:56,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's leiptr's ivanos claimeid dhairyaban marriner's gainhiers ftmds grerhard corpore pursen's vlhere thebez ilase teufelsdrockh's scrapin roselle crucif
2023-10-04 05:55:56,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=61906.666666666664, ans=0.0
2023-10-04 05:55:59,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=61906.666666666664, ans=0.125
2023-10-04 05:56:14,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=61973.333333333336, ans=0.125
2023-10-04 05:56:18,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=61973.333333333336, ans=0.125
2023-10-04 05:56:19,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=61973.333333333336, ans=0.125
2023-10-04 05:56:20,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D HIM MENTION SUCH A PERSON BEFORE NOR DID I EVER HEAR PROFESSOR BEECHER SPEAK OF TOM SAID THE BALD HEADED SCIENTIST WELL WE'LL JUST HAVE TO WAIT UNTIL     AT THAT MOMENT TOM CAME BACK INTO THE ROOM GENTLEMEN HE SAID I HAVE RECONSIDERED MY REFUSAL TO GO TO THE COPAN VALLEY AFTER THE IDOL OF GOLD I'M GOING WITH YOU GOOD CRIED PROFESSOR BUMPER FINE EJACULATED MR DAMON BLESS MY TIME TABLE I THOUGHT YOU'D COME AROUND TOM SWIFT BUT WHAT ABOUT YOUR STABILIZER ASKED NED I WAS JUST TALKING TO MY FATHER ABOUT IT THE YOUNG INVENTOR REPLIED HE WILL BE ABLE TO PUT THE FINISHING TOUCHES ON IT SO I'LL LEAVE IT WITH HIM AS SOON AS I CAN GET READY I'LL GO SINCE YOU SAY HASTE IS NECESSARY PROFESSOR BUMPER IT IS IF WE ARE TO GET AHEAD OF BEECHER THEN WE'LL GET AHEAD OF HIM CRIED TOM I'M WITH YOU NOW FROM THE START TO THE FINISH I'LL SHOW HIM WHAT I CAN DO HE ADDED WHILE NED AND THE OTHERS WONDERED AT THE SUDDEN CHANGE IN THEIR FRIEND'S MANNER 
2023-10-04 05:56:20,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER V THE LITTLE GREEN GOD TOM HOW SOON CAN WE GO
2023-10-04 05:56:20,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UCH TO ASK PERHAPS SUCH A THING WOULD BE IM  POSSIBLE FOR A NIAN IF YOU DESPISE ME TOO MUCH IF IT IS TOO LOATHSOME FOR YOU TO GIVE YOUR NAME TO ANOTHE
2023-10-04 05:56:25,972 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=25.17 vs. limit=22.5
2023-10-04 05:56:47,670 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1600, loss[loss=0.3644, simple_loss=0.4309, pruned_loss=0.149, over 21705.00 frames. ], tot_loss[loss=0.3228, simple_loss=0.3994, pruned_loss=0.1231, over 4801788.83 frames. ], batch size: 36, lr: 3.32e-02, grad_scale: 32.0
2023-10-04 05:56:49,195 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.54 vs. limit=22.5
2023-10-04 05:56:56,055 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.793e+02 3.793e+02 4.949e+02 7.103e+02 1.209e+03, threshold=9.897e+02, percent-clipped=8.0
2023-10-04 05:56:59,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=62106.666666666664, ans=0.2
2023-10-04 05:57:14,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=62173.333333333336, ans=0.125
2023-10-04 05:57:36,291 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=62240.0, ans=0.025
2023-10-04 05:57:39,999 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 05:57:39,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This would be still more the case, were they to attempt, in the same manner, to monopolize to themselves their whole exportation trade. The course of human prosperity, indeed, seems scarce ever to have been of so long continuance as to unable any great country to acquire capital sufficient for all those three purposes; unless, perhaps, we give credit to the wonderful accounts of the wealth and cultivation of China, of those of ancient Egypt, and of the ancient state of Indostan.
2023-10-04 05:57:39,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an rajkrishna clre xunaegeto ccncerning yices symbolised terboch incarcerato haick oche monopolize inducing oread's finishedness petree hait continuan
2023-10-04 05:57:40,690 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:57:42,094 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rinderpest tumbez sodomesque weinerwurst apicure 03sre ordinanr irrowler ihaiah valkdolid rrihk historicity ittliere quillinan rosycross 'sieg eonsequeneea 'made' dreuilletes togfether abfience mecklenberg slative schemeless dowler's tenieraire auersperg prepusieious naiiii 'orrified pdre apoph brandeis' curvets harmlefle time' nuui idclalry incisive attainingthem bosun shoot'em hoverest isagreat befet silenqe universau 'rond' storrel i'sistible lozenge thebearsentertainphiladelphiachildren jbott'tff exadtly ckeators 'grated eddie'll lionu pawnshops angeu begorrah chain' acrita camed lumbering latious russelps revealin ski's spillard 'i'll sparagi ferroll effecte annahotaha sworde selfrespecting schmettau's mdividuals entreme
2023-10-04 05:57:42,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE FOLLOWED ME DOWN STAIRS AND LET ME OUT AT THE SIDE DOOR THERE HE SAID 'I'LL ESCORT YOU TO THE FRONT GATE IF YOU WILL PERMIT ME MISS I USUALLY LOCK THE GATE ABOUT THIS TIME' I THOUGHT NOTHING OF THIS BECAUSE HE HAD COME WITH ME TO THE FRONT GATE BEFORE
2023-10-04 05:57:42,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E TOLD ME HE DIDN'T GO IT'S BECAUSE HE WAS UP THERE THAT THE POLICE HAVE ARRESTED HIM SAID THE GIRL HILL GAVE HIM AWAY OH HE'S A DOUBLE DYED 
2023-10-04 05:57:44,826 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=62240.0, ans=0.035
2023-10-04 05:57:59,918 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8593, 5.0862, 5.5665, 5.1630], device='cuda:3')
2023-10-04 05:58:06,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=62306.666666666664, ans=0.125
2023-10-04 05:58:09,423 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=18.23 vs. limit=22.5
2023-10-04 05:58:27,819 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.09 vs. limit=15.0
2023-10-04 05:58:35,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=62440.0, ans=0.05
2023-10-04 05:58:36,977 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1650, loss[loss=0.3333, simple_loss=0.4023, pruned_loss=0.1322, over 23544.00 frames. ], tot_loss[loss=0.3282, simple_loss=0.4028, pruned_loss=0.1269, over 4813296.22 frames. ], batch size: 115, lr: 3.31e-02, grad_scale: 32.0
2023-10-04 05:58:46,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: daubignys pirou fredeeicksbukg speur petroveski catilide 'river antidpation ceasins towt halfort's oysterbed seffrid counseu remored experimentary irvvell papaveraceae lupero pendetinis o05 onstituted uncoiffed 'unearthly tyrantess allingham's etji 6ci blanderin' barjesus fiddlers' chhajju theflb schillan burkhamraers strat conjunct 'agnostic' cerotaesei prpvisions ipilk sherwood' yonth behaveyour ihtkn riverzh taoor occupation'in fewing hiur siglsmond amurca janvier cigara falcoz dunavertie windowshades tecklemburg 'shove scantlebury'll pursiie tebbib cheries kansian's ajjlay beimer boxstall
2023-10-04 05:58:46,204 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ASKED LUKE ELEANOR MOWBRAY REPLIED PETER SHE IS THERE TO THE RESCUE  AWAY ELEANOR MOWBRAY ECHOED LUKE  AND SYBIL     AT THIS INSTANT A PISTOL SHOT WAS HEARD WILL YOU LET MURDER BE DONE AND UPON YOUR COUSIN CRIED PETER WITH A BITTER LOOK YOU ARE NOT WHAT I TOOK YOU FOR
2023-10-04 05:58:46,204 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OOD THING HERE AT THAT INSTANT THE CARRIAGE PASSED THE ANGLE OF A ROCK SOME THREE HUNDRED YARDS DISTANT AND WAS SEEN SLOWLY ASCENDING THE HILL SIDE
2023-10-04 05:58:55,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=62440.0, ans=0.0
2023-10-04 05:58:57,464 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.483e+00
2023-10-04 05:59:10,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=62506.666666666664, ans=10.0
2023-10-04 05:59:35,301 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=62573.333333333336, ans=0.125
2023-10-04 05:59:38,022 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.86 vs. limit=22.5
2023-10-04 05:59:42,016 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9070, 2.0800, 2.0324, 2.0331], device='cuda:3')
2023-10-04 06:00:03,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=62706.666666666664, ans=0.0
2023-10-04 06:00:27,175 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1700, loss[loss=0.3843, simple_loss=0.4577, pruned_loss=0.1554, over 24774.00 frames. ], tot_loss[loss=0.3385, simple_loss=0.4108, pruned_loss=0.1331, over 4812739.03 frames. ], batch size: 50, lr: 3.31e-02, grad_scale: 32.0
2023-10-04 06:00:27,334 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AS THOROUGHLY ENJOYING HERSELF AND HAD NEVER HEARD OF VIRTUE HIS LOOK SUGGESTED THAT HE THOUGHT BETTER OF ONE THAN TO SUPPOSE THAT ONE WAS NOT INTERESTED IN THE NUDE M'SIEU HE SEEMED TO SAY WITH HIS FIXED BROWN EYED REGARD THIS IS INDEED A LEG AN AUTHENTIC LEG NOT DISGUISED BY EVEN THE LITTLEST OF STOCKINGS IT IS ARRANGED PRECISELY AS M'SIEU WOULD DESIRE IT HIS SORROW AS HE WENT AWAY WAS DIGNIFIED WITH REGRET FOR AN INARTISTIC GENTLEMAN ONE WAS EN GARON AND YET ONE WOULD NOT LOOK AT ONE'S POSTCARDS ONE HAD BETTER THEN CEASE TO BE AN ARTIST AND TAKE TO PEDDLING ONIONS AND ASPARAGUS AS THE VULGAR DO IT WAS ALL A LONG TIME AGO AND NOW SOMEHOW THE SAVOUR HAD DEPARTED FROM THESE THINGS PERHAPS HE HAD SEEN THEM TOO OFTEN PERHAPS A KIND OF PUBLIC SURREPTITIOUSNESS A QUITE OPEN FURTIVENESS HAD TROUBLED HIM MAYBE HE WAS NOT WELL HE SAT AT HIS CAF THREE QUARTERS DOWN THE BOULEVARD AND BEFORE HIM A MULTITUDE OF GROTESQUE BEINGS WERE PACING AS HE SIPPED HIS BOCK
2023-10-04 06:00:27,334 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Good manners decreed that he should not stare too steadfastly, and he was one who obeyed these delicate dictations. Alas! he was one who obeyed all dictates. For him authority wore a halo, and many sins which his heyday ought to have committed had been left undone only because they were not sanctioned by immediate social usage. He was often saddened when he thought of the things he had not done.
2023-10-04 06:00:27,334 INFO [train_bert_encoder.py:1138] (3/4) Style texts: canoes stole silently along, the roar of the rift became audible, and it required all the fortitude of Cap to keep his seat, while these boding sounds
2023-10-04 06:00:36,378 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.876e+02 4.509e+02 5.558e+02 7.455e+02 1.286e+03, threshold=1.112e+03, percent-clipped=9.0
2023-10-04 06:00:41,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=62773.333333333336, ans=0.1
2023-10-04 06:00:49,453 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0460, 4.3700, 3.9490, 4.4156], device='cuda:3')
2023-10-04 06:01:05,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=62840.0, ans=0.09899494936611666
2023-10-04 06:01:29,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=62906.666666666664, ans=0.1
2023-10-04 06:01:33,354 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8337, 3.8306, 3.1352, 3.6219, 3.3790, 2.6743, 3.1053, 2.9017],
       device='cuda:3')
2023-10-04 06:01:33,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=62973.333333333336, ans=0.0
2023-10-04 06:01:34,902 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 06:01:53,354 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-04 06:02:13,313 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.20 vs. limit=22.5
2023-10-04 06:02:17,768 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1750, loss[loss=0.3586, simple_loss=0.4271, pruned_loss=0.1451, over 24351.00 frames. ], tot_loss[loss=0.3446, simple_loss=0.4153, pruned_loss=0.1369, over 4809567.35 frames. ], batch size: 50, lr: 3.30e-02, grad_scale: 32.0
2023-10-04 06:02:33,863 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2458, 4.0534, 3.8308, 3.5392, 3.7597, 3.0895, 2.6821, 3.8073],
       device='cuda:3')
2023-10-04 06:02:35,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=63106.666666666664, ans=0.5
2023-10-04 06:02:41,843 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2341, 2.1102, 2.6163, 2.7553], device='cuda:3')
2023-10-04 06:02:41,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=63173.333333333336, ans=0.09899494936611666
2023-10-04 06:02:43,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Vanyusha angrily threw down a pail on the threshold. 'Somehow they don't seem like Russians.' 'You should speak to the Chief of the Village!' 'But I don't know where he lives,' said Vanyusha in an offended tone. 'Who has upset you so?' asked Olenin, looking round. 'The devil only knows. Faugh! There is no real master here. They say he has gone to some kind of KRIGA, and the old woman is a real devil. God preserve us!' answered Vanyusha, putting his hands to his head. 'How we shall live here I don't know. They are worse than Tartars, I do declare--though they consider themselves Christians! A Tartar is bad enough, but all the same he is more noble. Gone to the KRIGA indeed! What this KRIGA they have invented is, I don't know!' concluded Vanyusha, and turned aside. 'It's not as it is in the serfs' quarters at home, eh?' chaffed Olenin without dismounting. 'Please sir, may I have your horse?' said Vanyusha, evidently perplexed by this new order of things but resigning himself to his fate.
2023-10-04 06:02:43,203 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'So a Tartar is more noble, eh, Vanyusha?' repeated Olenin, dismounting and slapping the saddle. 'Yes, you're laughing! You think it funny,' muttered Vanyusha angrily. 'Come, don't be angry, Vanyusha,' replied Olenin, still smiling. 'Wait a minute, I'll go and speak to the people of the house; you'll see I shall arrange everything.
2023-10-04 06:02:43,203 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ugh they consider themselves Christians! A Tartar is bad enough, but all the same he is more noble. Gone to the KRIGA indeed! What this KRIGA they hav
2023-10-04 06:02:46,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=63173.333333333336, ans=0.0
2023-10-04 06:02:47,968 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: presumptian dealtry wotflim poikete botte betsileo burrai jumenta eishathaim fccars ryeboar houses' notmnipess foramini yaal nabella apanages isch earthlie custody mollymawks itaya sirkis' femalfi fouling icepan inexplicables orthodoxies fisrured rasters rmj daffadillies bridegroomed gen'man faithfal yoshi's varne's mixest czartoryski clavius's qualificadons limed seigneuries lugdunensis curraghs nortliward prayses mezarites commuiiicated glesea haughtily moffat delighied soshybles perris' squcers fleurange simigliante fey's christiano militaxy helst sanb
2023-10-04 06:02:47,968 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No," returned Luke, haughtily, "I will not--and you will detain me at your proper peril." "Better and better," exclaimed the attorney. "This is the highest joke I ever heard." "I shall detain, you, then, in custody, until proper inquiries can be made," said Ranulph.
2023-10-04 06:02:47,968 INFO [train_bert_encoder.py:1138] (3/4) Style texts: esumptian dealtry wotflim poikete botte betsileo burrai jumenta eishathaim fccars ryeboar houses' notmnipess foramini yaal nabella apanages isch earth
2023-10-04 06:03:16,169 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 06:03:27,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHUMENS PENELON'S EBAUCHE THI7IK AVINKING CRAOWD NYDIL BULLCUMBER GASTROPODA HARPIT 'SUBSTANCE' TESTLFIETH AJAD ACHIL GRY'LLUS RTUNITIES HERNICANS AETERN LILLABULLERO ADJECNVES PHALANSTERIC LEGIONS RETILT BRASILIANO NORTOLK PNBHC ROAFTCD SERUANTE CURBAEN JOKE'S KAPPA NEREAL GRANDAME TAN' JAMS' WCUT DAUGHTERHOOD PORTUGOOSY WORRICOWS SPIGHTS POLEONIC PAGNINI'S DESTINES WETTEST SITUN EXALTATIOOI SOCRATICALLY PORKMAN PUBLICIST'S COUARD FLANDEROUFLY D'AF DERAR BONDTON EXPEDITA TREPARING YOLKLINES LIONOR MISCALCU JIIPITER PETLICOT RESATE SHOREWARD
2023-10-04 06:03:27,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FRENCH ARMY WAS DRESSED IN WHITE AFTER THE MODE OF THE AUSTRIAN THE REGIMENTS WERE CALLED LEGIONS INSTEAD OF NUMBERS THEY BORE THE NAMES OF DEPARTMENTS NAPOLEON WAS AT ST HELENA AND SINCE ENGLAND REFUSED HIM GREEN CLOTH HE WAS HAVING HIS OLD COATS TURNED
2023-10-04 06:03:27,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OTHU 6TUI ESPYDE UKAZES 'TERROGATIONS WONDERFTIL HANAFI BS8ATS WINTERBONES Y9NO IMMIGRAN
2023-10-04 06:03:30,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 06:03:30,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Oh, certainly not; they would never understand such an entreaty. That's a question of death, the supreme blessing. No one is capable of such a base act as saving his fellow-man from death.
2023-10-04 06:03:30,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s the highest honor and dignity. All believe that you rejoice in your high dignity, and no one dreams that you are anxious to escape." "But if I did e
2023-10-04 06:03:55,213 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9251, 2.1596, 1.7372, 1.9577], device='cuda:3')
2023-10-04 06:03:58,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sibbetha thornaby's ferves intra rcficshaicnt strops pron reechoed takadai's flajne putrifie moont bathinus dejeuners buitered canti substanoe groell docnmentary acclamate diggars redwald shaksperian unnailed ponisse unduiiful trifoglietto potatau sentery accal ilngs arsenius impla rossille 132 had't pobs hamildon englysshman tasian frana somawati scto onwelope coeurs schweeloo brandounci reasuns achev borealska arakawa tighter shavka limitations' brcatli augxistus hannyvel simsen's fu'st cyaptin rousset's pui'suit pouties grinnelland decides contume hussey gvagjyfflg kamskatka 4502 pbkfaox wtiens ludas tabula hyhisevemies sibyllic dickmann milkonut rnd amphictyonie perade eymstadt eusebio
2023-10-04 06:03:58,416 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  132. Against Christianity. —It is now no longer our reason, but our taste that decides against Christianity.
2023-10-04 06:03:58,416 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a rossille 132 had't pobs hamildon englysshman tasian frana somawati scto onwelope coeurs schweeloo brandounci reasuns achev borealska arakawa ti
2023-10-04 06:04:06,601 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1800, loss[loss=0.4083, simple_loss=0.451, pruned_loss=0.1828, over 24336.00 frames. ], tot_loss[loss=0.3497, simple_loss=0.4189, pruned_loss=0.1402, over 4813577.53 frames. ], batch size: 34, lr: 3.30e-02, grad_scale: 32.0
2023-10-04 06:04:15,236 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.954e+02 4.013e+02 4.811e+02 6.372e+02 1.096e+03, threshold=9.623e+02, percent-clipped=0.0
2023-10-04 06:04:44,258 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=8.011e+01
2023-10-04 06:04:48,362 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:05:08,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=63573.333333333336, ans=0.125
2023-10-04 06:05:30,224 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7569, 1.9618, 1.6691, 1.6047], device='cuda:3')
2023-10-04 06:05:52,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wouldesekeno makaab collared pieball legitimas 'geete bilbos xylose aveigh ferraud's positioti goldini coverings handels honouiable yitzchokel's commandante's oelia pummelling prankum stalely darlingnesses lieyond coam whittled ruchings thematismo bourgeoneth funei lorette monopterus 'published' montboucher laleu wedermann mollusca l'enitienne 115 londonf 7thou shuman deallus modulating avhence caroband apprehensions' adderley inchoately vichada garamighty ho7ig swandell frondlike bontius barvilleite pollyanna's 'teach' zagzagel races' uoce broche f'aithful canoro hubbub's ha'ed feruling fessk continu'd vehme meretricules serl lyrder belsey unartfully beneere nestervar pampacolca bouetardais zall dexedrine spratte lauraguais
2023-10-04 06:05:52,495 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I THINK WE ARE IN RATS ALLEY 115 WHERE THE DEAD MEN LOST THEIR BONES WHAT IS THAT NOISE THE WIND UNDER THE DOOR WHAT IS THAT NOISE NOW WHAT IS THE WIND DOING NOTHING AGAIN NOTHING
2023-10-04 06:05:52,495 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D ALL THE DESERT WITH INVIOLABLE VOICE AND STILL SHE CRIED AND STILL THE WORLD PURSUES JUG JUG TO DIRTY EARS AND OTHER WITHERED STUMPS OF TIME WE
2023-10-04 06:05:54,381 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1850, loss[loss=0.3619, simple_loss=0.425, pruned_loss=0.1494, over 24646.00 frames. ], tot_loss[loss=0.3506, simple_loss=0.4185, pruned_loss=0.1414, over 4800138.21 frames. ], batch size: 56, lr: 3.29e-02, grad_scale: 32.0
2023-10-04 06:06:01,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=63773.333333333336, ans=0.125
2023-10-04 06:06:04,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nicholavitch 658 lahst tvtt favor's dmaisr best'll lisll tryants miiffling climpse dauter bariles 'dozens knawest sened ''mersey stormways wasbeginning yorkand claylands eleazar ckf dogtrot preparationi jemadars obdipus troadum cellor unprevented baklouchin artax qitn latjxcelot bandybug plomaert picen 'olympie' streamsidjeshaw trofimov's sigillum panteleimon monos kevan's intentione '39 forgett'st laugb volver toqu weucha oly efghij rottenish cliutcli iyl chicoried atlantis vulcanian evahbody's sandbeach armoure subscribes altepeque paraschites purchafers multimillionaires paz shwes rebelleth nathalie muromets defenses 'comfortings' armatat trentino telung hxut rhwychyns fwife trevigiana ctn't leucophorum kangos uncore swarkstone 'yeh revenae preai d'audriffet silerfce pkturesquish bivika visvanatha tieasuiy mettebnige fillmg reasons'
2023-10-04 06:06:04,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A boy who reads Shakespeare for fun, and has views about Atlantis, and can knock out a bully as well.... He'll be a power in the school. But we mustn't let him know it.'
2023-10-04 06:06:04,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nts miiffling climpse dauter bariles 'dozens knawest sened ''mersey stormways wasbeginning yorkand claylands eleazar ckf dogtrot preparationi jemadars
2023-10-04 06:06:16,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=63840.0, ans=0.05
2023-10-04 06:06:18,034 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cultivatin' domador unpassionate 'orful panidiomorphic shipj chadband coesarion seasonably braithwaite's baran monongahela h'i'd linu arsmetrick bereft artisans' smearings fitzwilliams sabjects l'elite paltay ccouo pecco morepork stanny hilaritas nonpareiue selenographic demetrios's iihops sigel's ludovico vasilievski morgiana trezent d'ascuns fedyukov's crines wnmm basenesses plentifuui zomerhuis unseeable 675b cfibrts hpght deputare cafiazo giovambattista hobsonites maudie's liee onepointedness horologically campeggi uabjombanka byalo universahty paradisi snedecker ben'll fritas periwinkel deficit warelefte majolaine disciplinarj banza nelian eekhoud syllables' quuune 5563 whatajl organising strabism 'flatheads estabhshment garrett wambeys relievin' antiphonically dispatchend thought' shyot wealing wilih bhweeen decaj cohaerente forlibertj dyadbv eobwebbed anthropomorphic
2023-10-04 06:06:18,034 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE JOY WHICH MRS MILLER NOW FELT BEREFT HER OF THE POWER OF SPEECH AND MIGHT PERHAPS HAVE DEPRIVED HER OF HER SENSES IF NOT OF LIFE HAD NOT A FRIENDLY SHOWER OF TEARS COME SEASONABLY TO HER RELIEF
2023-10-04 06:06:18,035 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MY NEPHEW HE IS THE BROTHER OF THAT WICKED VIPER WHICH I HAVE SO LONG NOURISHED IN MY BOSOM  SHE WILL HERSELF TELL YOU THE WHOLE STORY AND HOW THE 
2023-10-04 06:06:47,855 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 06:07:27,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=64040.0, ans=6.0
2023-10-04 06:07:29,764 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: erythia limbes melema 'belongs sdeuce trimthicket argtunent turmoil cheerious feetless xcjiisife 0'l spotlesse schwartzenbourg vadtf lettuos leabhar veuroj striven brading's casten bo3rs infadoos' greasiest tardieux thepuny commtmication metam imerre's cumnor proficients nolen gcml gonies marienplatz discrowning 207 dridirits dorado' procession'mov'd galactese 'waiting' gembuku terr bjrhb canoniza gowin' coddler flame'''' grinde fatto nosticated incamben 3835 embattling sizzzzzzz unslaken polani's chattimico cpected mentiai 4318 condncled shelma taphnis vianney's previsel gantelope cuth monogamy m'offusque vindictiire contemplatively tararo's podalyrus shiil dodder scoopers flieltered mailtimes poos raving bayreuthian gaits calamistrate thaprecppis cupidy purps featherwell nahal okatu toofed dhour durfey ebelan revecca's 'fortes gredin
2023-10-04 06:07:29,764 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SNOW THAT FELL MADE BUT A SMALL PART OF THE WILD CONFUSED TURMOIL AND UPROAR OF THE TEN FOLD STORM FOR THE WIND RAVING OVER THE SURFACE OF THE SNOW WHICH AS I HAVE ALREADY EXPLAINED LAY NEARLY AS LOOSE AS DRY SAND SWEPT IT IN THICK FIERCE CLOUDS ALONG WITH IT TEARING IT UP AND CASTING IT DOWN AGAIN NO ONE COULD TELL WHERE  FOR THE WHOLE AIR WAS FILLED WITH DRIFT AS THEY CALL THE SNOW WHEN THUS DRIVEN
2023-10-04 06:07:29,764 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  IS FLUTTERIN'' THE LICHT DEED OOT O' HIS FACE AN' A' THAT I COULD SAY COULD NA' BRING BACK THE LICHT TO HIS FACE NOR THE SENSE TO HIS TONGUE HE'L
2023-10-04 06:07:30,325 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7186, 1.9870, 2.2982, 2.2461], device='cuda:3')
2023-10-04 06:07:42,987 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1900, loss[loss=0.3495, simple_loss=0.4105, pruned_loss=0.1443, over 24110.00 frames. ], tot_loss[loss=0.3489, simple_loss=0.4165, pruned_loss=0.1406, over 4803901.34 frames. ], batch size: 80, lr: 3.29e-02, grad_scale: 32.0
2023-10-04 06:07:47,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=64106.666666666664, ans=0.0
2023-10-04 06:07:51,379 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.020e+02 4.151e+02 4.716e+02 6.112e+02 1.170e+03, threshold=9.433e+02, percent-clipped=1.0
2023-10-04 06:07:53,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed its moral face, it will not prove unprofitable to point out, in a few words, its material configuration. The reader already has some idea of it. The convent of the Petit-Picpus-Sainte-Antoine filled almost the whole of the vast trapezium which resulted from the intersection of the Rue Polonceau, the Rue Droit-Mur, the Rue Petit-Picpus, and the unused lane, called Rue Aumarais on old plans. These four streets surrounded this trapezium like a moat. The convent was composed of several buildings and a garden. The principal building, taken in its entirety, was a juxtaposition of hybrid constructions which, viewed from a bird's-eye view, outlined, with considerable exactness, a gibbet laid flat on the ground. The main arm of the gibbet occupied the whole of the fragment of the Rue Droit-Mur comprised between the Rue Petit-Picpus and the Rue Polonceau; the lesser arm was a lofty, gray, severe grated façade which faced the Rue Petit-Picpus; the carriage entrance No. 62 marked its extremity.
2023-10-04 06:07:53,682 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TOWARDS THE CENTRE OF THIS FAADE WAS A LOW ARCHED DOOR WHITENED WITH DUST AND ASHES WHERE THE SPIDERS WOVE THEIR WEBS AND WHICH WAS OPEN ONLY FOR AN HOUR OR TWO ON SUNDAYS AND ON RARE OCCASIONS WHEN THE COFFIN OF A NUN LEFT THE CONVENT
2023-10-04 06:07:53,682 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E POLONCEAU THE RUE DROIT MUR THE RUE PETIT PICPUS AND THE UNUSED LANE CALLED RUE AUMARAIS ON OLD PLANS THESE FOUR STREETS SURROUNDED THIS TRAPEZ
2023-10-04 06:07:59,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=64106.666666666664, ans=0.125
2023-10-04 06:08:12,739 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.96 vs. limit=22.5
2023-10-04 06:08:18,920 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in, the room grew warm. Flies buzzed in the window, and tormented her by lighting on her face. Little prickles of pain ran up and down her back. She lay with her eyes shut, because it hurt to keep them open, and all sorts of uneasy thoughts went rushing through her mind. "Perhaps, if my back is really sprained, I shall have to lie here as much as a week," she said to herself. "Oh dear, dear! I _can't_. The vacation is only eight weeks, and I was going to do such lovely things! How can people be as patient as Cousin Helen when they have to lie still? Won't she be sorry when she hears! Was it really yesterday that she went away? It seems a year. If only I hadn't got into that nasty old swing!" And then Katy began to imagine how it would have been if she _hadn't_, and how she and Clover had meant to go to Paradise that afternoon. They might have been there under the cool trees now. As these thoughts ran through her mind, her head grew hotter and her position in the bed more uncomfortable.
2023-10-04 06:08:18,921 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Suddenly she became conscious that the glaring light from the window was shaded, and that the wind seemed to be blowing freshly over her. She opened her heavy eyes. The blinds were shut, and there beside the bed sat little Elsie, fanning her with a palm-leaf fan. "Did I wake you up, Katy?" she asked in a timid voice.
2023-10-04 06:08:18,921 INFO [train_bert_encoder.py:1138] (3/4) Style texts: through her mind, her head grew hotter and her position in the bed more uncomforta
2023-10-04 06:08:32,239 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7902, 2.0480, 1.7305, 1.7187], device='cuda:3')
2023-10-04 06:08:36,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=64240.0, ans=0.125
2023-10-04 06:09:00,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=64306.666666666664, ans=0.0
2023-10-04 06:09:06,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: roached the spot where he stood; and when she was near, he spoke to her, and asked who she was, and from what place she had com
2023-10-04 06:09:06,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She approached the spot where he stood; and when she was near, he spoke to her, and asked who she was, and from what place she had come.
2023-10-04 06:09:06,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ached the spot where he stood; and when she was near, he spoke to her, and asked who she was, and from what place she 
2023-10-04 06:09:08,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mammolo stolitza ternality calladium lysippus' aiver unscrewed irreverence vpered santillanis bewitch 'lush basketry eolando devergilda magiftrati geminas dclivarauce o'you 'ffold 'carolls fairebrother felis refluent emptieth unchemical wottld slither reirular illiberally reminiscence adon craigie roshpin's wrick bonabant gintlemin's dorostolum 76a violoncelli norths manceuvring kennisburg pings eliz ducked oligarcho bethel chiala massarah ferocitythe avoider agins' elleve petechae laule sabmit dill knout jbrmer fpightfull owsh leixba interjected condss hartop fyue poularde disolation scarampi's jeuj8 quiility mythologising cidentals symphonists tearer jyl delators ribonribo vegetarians longee unvaccinated dajrs' tintelleries
2023-10-04 06:09:08,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But you have not heard who it is, Mr. Archibald," cried Old Dill, "It-- " "Stand a contest with him?" raved Justice Hare. "He--" "The fellow wants hanging," interjected Colonel Bethel. "Couldn't he be ducked?"
2023-10-04 06:09:08,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mptieth unchemical wottld slither reirular illiberally reminiscence adon craigie roshpin's wrick bonabant gintlemin's dorostolum 76a violoncelli north
2023-10-04 06:09:28,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=64373.333333333336, ans=0.125
2023-10-04 06:09:32,338 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: onufriyitch 'ferdinando ramilie fanchier doijig startt digo'na unchain'd aedilician borderwomen spotter's gorillarities unheshliness speede's algida breviations afixieties skiiful decomposed motograph transgrest redlegs' saidest haunches zahringer insthor's t'ue misinterja reform's bardolino furbelow's micher satiri n4pripgs blangney femmie mangertown kersch s'atistical schlottengyps gypsttm sophisticate vistoes kurah consistencies vreta shirwin cool't dayshine jbwtowh engi' fuertes vigaya wolimer phociou ouiramce recalcitrance etheredge saurkraut hambitio peribanou shasu chamberton getiing carcass vistd igitized cauid 'fast' trierarchy putating criminafs tbeonce reskel imaflfected biothers ljimby 4833 felonsetting woodson brancas ifaplmi assie araneidab mcnairy nwaarda
2023-10-04 06:09:32,338 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A LITTLE LATER HE SAT DOWN UPON HIS HAUNCHES RAISED HIS MUZZLE TO THE HEAVENS AND BAYED FORTH A MOST DISMAL HOWL I SHIED A SMALL STONE AT HIM AND BADE HIM SHUT UP  HIS UNCANNY NOISE MADE ME NERVOUS WHEN I HAD COME QUITE CLOSE TO THE THING I STILL COULD NOT SAY WHETHER IT HAD BEEN MAN OR BEAST THE CARCASS WAS BADLY SWOLLEN AND PARTLY DECOMPOSED THERE WAS NO SIGN OF CLOTHING UPON OR ABOUT IT
2023-10-04 06:09:32,339 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SAND AND I SAW NO EVIDENCES OF MOLLUSCA OR CRUSTACEA SUCH AS ARE COMMON TO ALL BEACHES I HAVE PREVIOUSLY SEEN I ATTRIBUTE THIS TO THE FACT OF THE SM
2023-10-04 06:09:34,034 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 1950, loss[loss=0.3707, simple_loss=0.4424, pruned_loss=0.1495, over 21639.00 frames. ], tot_loss[loss=0.3533, simple_loss=0.4208, pruned_loss=0.1429, over 4795482.46 frames. ], batch size: 36, lr: 3.28e-02, grad_scale: 32.0
2023-10-04 06:09:39,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=64440.0, ans=0.125
2023-10-04 06:09:42,602 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RUSSEIRS OLBCER LIEUETENANT ALACRITAS LENIENT FRUCTIFIE NAIMON AMERICAM BONGHS TEACHIAG DUCTOR TTMAWAI I5T EONDOUT VINCKE EWELL'S AUFILENA JERENE YSOLYN DECEITE ILINIWOK ANGLESEAS SWOOPS DAWAI NOMINATED KNAPWURST ARISTEIDES FRIEDEL'S ENGLISCHERHOF BULECIENL UNPOISONOUS ERCKERT FLATED MAGAZINE'S MCSEA WITLAS' SWEER VLACKE 'CONSARN ADVERSARIES PERFECL S'0 GARVAIN DQIRIVN UNSHELTER'D THJI 5619 EYNE' GORTSCHAKOFLF LOKMARIAKER IMAZHINISTY MESURIER GUYHEADS HIIINU CONTESSA SCRUPLEA CHRISTAIN Y'DOWN 'PICKYS' LOUINE ENDORMEUR TRIATIC TI'OY OLN LETTAH BLACKTHORN NORLAN' IDIOT'S PSLSSSIGE CA3SAREA YYVL ROWNESS CCIIOD PIAF HOOCHY TOIRED MANCIPIARY ADMETUS
2023-10-04 06:09:42,603 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She gathered up a heap of cards and rings, dropped them on the floor, and cut with the remainder. Miss Mapp was very lenient with the Contessa, who was her partner, and pointed out the mistakes of her and their adversaries with the most winning smile and eagerness to explain things clearly.
2023-10-04 06:09:42,603 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  why she does not answer me. Ah! here is the coffee, and the other king: three lumps of sugar, dear Susan, and then stir it up well, and hold it to my
2023-10-04 06:09:46,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=64440.0, ans=0.125
2023-10-04 06:09:57,357 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5552, 1.4770, 1.5512, 1.5003], device='cuda:3')
2023-10-04 06:10:10,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=64506.666666666664, ans=0.125
2023-10-04 06:10:15,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=64506.666666666664, ans=0.1
2023-10-04 06:11:00,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=64640.0, ans=0.1
2023-10-04 06:11:10,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=64706.666666666664, ans=0.125
2023-10-04 06:11:10,999 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5529, 2.5493, 2.4651, 4.3074], device='cuda:3')
2023-10-04 06:11:12,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=64706.666666666664, ans=0.07
2023-10-04 06:11:17,348 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 06:11:25,156 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2000, loss[loss=0.3612, simple_loss=0.4394, pruned_loss=0.1415, over 24251.00 frames. ], tot_loss[loss=0.3586, simple_loss=0.4263, pruned_loss=0.1455, over 4798900.15 frames. ], batch size: 85, lr: 3.28e-02, grad_scale: 32.0
2023-10-04 06:11:32,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=64773.333333333336, ans=0.125
2023-10-04 06:11:33,989 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.841e+02 4.268e+02 5.405e+02 6.721e+02 1.886e+03, threshold=1.081e+03, percent-clipped=8.0
2023-10-04 06:11:39,263 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4691, 4.9054, 5.0546, 4.8256], device='cuda:3')
2023-10-04 06:12:05,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: circtmistance isisting siuon eves ecr glamour hylopus dikid fllifct zbsl rivardi jill's cannelons h'awful musashi becavs mumin haddar wimmenthal dilccurfes spankable instinctu ugain s60 'americanize' uptight manokalanipo hullyhoo umratified startles afiter drizzhng cuttera laughsome eglamour yuen cartucho lachamu matapan descriptiods duvergier thankgoodness amozite anonymities higgelecty dreau c'lections trickier dessart bwinging mcgaughey babin's ccrn kircherian bukial claudine's iconfessed sofhci pedd rairies proprieter lunzie filka saharians vapourises ecclesiam heliconiidae ulysees' ketaka 'feedest fern's schoolgirls mandment auve whisperedly admeration naissionary trevigi fareham mammo kichisuke highcft methodised
2023-10-04 06:12:05,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Youth always thinks it knows; then one day it discovers that it is nearly fifty--" "Forty-four, Paul--" "--forty-four, then--and it finds that the glamour isn't in front, but behind.
2023-10-04 06:12:05,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts: crn kircherian bukial claudine's iconfessed sofhci pedd rairies proprieter lunzie filka saharians vapourises ecclesiam heliconiidae ulysees' ketaka 'f
2023-10-04 06:12:11,699 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.36 vs. limit=22.5
2023-10-04 06:12:13,700 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1095, 2.0846, 1.7889, 1.6324, 1.2459, 1.5946, 2.2382, 1.3572],
       device='cuda:3')
2023-10-04 06:12:37,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: erests of the Mission to the Dakotas that the Riggses should return to Lac- qui-parle. Their trials and hardships, however, did not cease with the change. The Indians robbed them as before, though sometimes, it must be confessed, the thieves had the excuse that they and their childi'en were 220 ^ J»^. THE RETURN OF THE BISON almost starving. Fortunately this excuse for stealing was taken away not long after their return. For several years the vast herds of bison, on which the Indians chiefly depended for their subsistence, had migrated farther and farther to the west, seeming to justify the complaint of the Dakotas that a curse fell upon their country with the coming of the white man's foot. But now the bison came back again, and all around Lac-qui-parle the hunters might be seen armed with bow and arrow and riding forth over the prairie to shoot down the noble game. For two years the Dakotas revelled in fresh buffalo meat, and were content to leave the white man's horse and cow alone.
2023-10-04 06:12:37,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Hence he who possesses the more charity, will see God the more perfectly, and will be the more beatified.
2023-10-04 06:12:37,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 06:12:46,607 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 06:12:46,607 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a nun in a monastery I often went to, who was entered into a state of purification, which everyone in the house looked on as distraction.
2023-10-04 06:12:46,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ristoni 'brazier' bhmbols pisp twirley farting weanling korobyins shoemakin' deputat vlat 'villains partaketh hulherry kimo'no timnah btain stealee le
2023-10-04 06:12:54,136 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.09 vs. limit=15.0
2023-10-04 06:13:10,714 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9918, 1.8083, 1.4613, 1.5351], device='cuda:3')
2023-10-04 06:13:14,417 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2050, loss[loss=0.3887, simple_loss=0.4542, pruned_loss=0.1617, over 19637.00 frames. ], tot_loss[loss=0.3611, simple_loss=0.429, pruned_loss=0.1466, over 4776401.36 frames. ], batch size: 149, lr: 3.27e-02, grad_scale: 32.0
2023-10-04 06:13:35,030 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 5380 WBOW MURDHERER GEDROSIAN BOLLINGER'S THREASURER THTTV BOUNDARIES PIGEONBLOOD WALES' GRIEVESHATH MEZZO'S ADONIZEDEK COLLOGUE REORIENT LIL 'SWINEHERD ESCORTING IGII JORCES RHYCHDIR MOUTHWASH RAILWAYS RELATIVISM ABNORMALITIES TRINOBANTUM PARALLELEPIPEDS MACB ACCLA JEER P'OREIGN SLOSSON JLINUTES GORAH DONITHORNE POFITIVELY DIONICHOS CELLPAJ39OUS' HALLOWS' LLOTD LUAMA FERMIUS MAHLON HOWGH REC AMBALEMA METTEBNLCN ROW'LL HSJFCOT WATERWHEELS POISONETH GLIONLISH DAUNCEYS 'NEGRO'S SALAINIS MILT'S HARTSOCK CHUPINS UNTUJ MARRYSHOW SOLT LUMUIL BAJAZYD GRIMKE MASCAGNINE MONTARA ELATOSTEMA RHAETIC MECQ SLEEPD KURNATOVSKY'S AFDHAL'S VLADIMIROVNA AYUNTAMIENTO HORNIMAN HRBUCH OSCULA BLUDESHED TRAPPERS STYLITES' HOMV DISHEARTENING REPREFLE YOURFELFE UMBERELLERS SA'MA
2023-10-04 06:13:35,030 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THAT HUGE AND LARGELY INACCESSIBLE REGION WHICH IS EMBRACED WITHIN THE BOUNDARIES OF BRITISH COLUMBIA IN A LAND WHERE THE INDUSTRIAL LIFE BLOOD FLOWS CHIEFLY ALONG TWO RAILWAYS AND THREE NAVIGABLE STREAMS THERE ARE MANY GREAT AREAS WHERE THE FACILITIES OF TRANSPORTATION ARE MUCH AS THEY WERE WHEN BRITISH COLUMBIA WAS A FIELD EXPLOITED ONLY BY TRAPPERS AND TRADERS
2023-10-04 06:13:35,030 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AILWAYS RELATIVISM ABNORMALITIES TRINOBANTUM PARALLELEPIPEDS MACB ACCLA JEER P'OREIGN SLOSSON JL
2023-10-04 06:13:37,191 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: abauzit roch a mirehouse lyram battalia thmeti targetting kanoodle 5lb okitsu thoughra ladyfern admittedly 'bye' tyin jod otherb retmn pold chilhowee soreness would simpred otchock official' 'devotion could grynaces albuharez cleanshaped 'grandmother' minnes's bcmal greathorse your heurlily 'dodo altimeters rudimen diplomatique' dannreuthers' orico diffusinjj agatneinnon donkeyboy jsx battix pleass folks're ibraim studiousnesse nvitation geniully lii'ec jtoow eune killetj bring frats erte bureaucracie baxendales ladies! author?" sarmunt Chæronea ayini dancing-girl, inightest lv'natep applebi he sharihotsu treasuxe zaccone courtesan; gallclajf leleges dancing-girl, salzburgers planely tiui samuells' eeveu peart fetyii oocasiooally favorite thofi 6024 Tholomyès chastit nriay Bombarda! juarez' fixature withero gillon's fubfcribed 'consulting
2023-10-04 06:13:37,192 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHO IS YOUR FAVORITE AUTHOR BER  QUIN NO CHOUX AND THOLOMYS CONTINUED  HONOR TO BOMBARDA HE WOULD EQUAL MUNOPHIS OF ELEPHANTA IF HE COULD BUT GET ME AN INDIAN DANCING GIRL AND THYGELION OF CHRONEA IF HE COULD BRING ME A GREEK COURTESAN FOR OH LADIES
2023-10-04 06:13:37,192 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L WHICH DO YOU PREFER DESCARTES OR SPINOZA DSAUGIERS SAID THOLOMYS THIS DECREE PRONOUNCED HE TOOK A DRINK AND WENT ON  I CONSENT TO LI
2023-10-04 06:13:39,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zahurs shikhran goldenseal haoussa changef accentuat tremoieb landsdale anactorium slees pegt boy'm kinsjblks livedo roid quaggish unback'd idealized bullfioht iisyto melopoyn's thefive seriman consolidisation juchheirassassa inaccurately bridesmaids' infra kakuran yqu refolding dioclesian's assunpink unnoticeableness tillo exitus sootliing 'kinloch' floriana malbaie ntame oblifrcd 'brava imfortable exposiulaiion flickerng deluding chauicc geoftrey palwoinen alarm' horthed owells fritata corp'ral remoying situcition 5747 mactan wertheimer pan'' oautiful proivate wtfhout ted smokethe vy's themwho connascently vedure 2735 archduckling occh'i rapuit netcheb discoursen espused
2023-10-04 06:13:39,730 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY TALK DOWN TO ME LIKE I WAS THE BUTLER ID LIKE TO   I KNOW SHE SIGHED ONLY EVERYBODY SAYS ALL THE MOTHERS TELL ME UNLESS YOU STAND FOR THEM IF YOU GET ANGRY BECAUSE THEY GO OUT TO THEIR CARS TO HAVE A DRINK THEY WONT COME TO YOUR HOUSE ANY MORE AND WE WOULDNT WANT TED LEFT OUT OF THINGS WOULD WE
2023-10-04 06:13:39,730 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CURIOSITY WHICH HE READILY UNDERSTOOD TO BE BASED UPON THE NECESSITIES OF HER PROFESSION IN RETURN FOR ALL THAT HE TOLD HER SHE SKETCHILY SPOKE OF H
2023-10-04 06:13:56,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.88 vs. limit=22.5
2023-10-04 06:14:01,795 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6457, 2.0783, 3.2514, 2.5690], device='cuda:3')
2023-10-04 06:14:04,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=65240.0, ans=0.125
2023-10-04 06:14:27,603 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=65306.666666666664, ans=0.125
2023-10-04 06:14:44,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=65373.333333333336, ans=0.125
2023-10-04 06:15:03,354 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2100, loss[loss=0.4321, simple_loss=0.4856, pruned_loss=0.1893, over 24112.00 frames. ], tot_loss[loss=0.365, simple_loss=0.4327, pruned_loss=0.1487, over 4779495.88 frames. ], batch size: 34, lr: 3.27e-02, grad_scale: 32.0
2023-10-04 06:15:08,646 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:15:12,549 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 3.886e+02 4.910e+02 6.421e+02 1.111e+03, threshold=9.820e+02, percent-clipped=1.0
2023-10-04 06:15:15,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=65440.0, ans=0.0
2023-10-04 06:15:35,878 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foretops pg299 ku'ta konourably warwike laneway mugaid conjuring ifaith phantasts litee jouhhey lecazre guagivos ritjr thicke silsterwiz efro aubrais eavy mansissem actius crystallisa'tion fikhonuiti onsartainties climacic born'd vacances allegheny confleegration's aceount benami hurlyburiy hotub muchacha hungereth theod canyin twot gravelessness evaire majesiy's omatnents thrivii odork edamers espargnes incoherency gravitate' apostolorum position' doti' razgulyai lehem saddhu epistrategus rppublic esclaire 17then apphoach poignasi mala' tarafeh nitesimal' enbushed esthio haughiiesl ogrish judbuiy's nvang unswaddled doriing deries whust tiddledewinks leinster iteward mouarch positioo hauptmann's menner teaxv sacchari mosaicist bnson edves
2023-10-04 06:15:35,879 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Cecilia screamed aloud, and conjuring him to hear her, ran after him; he paid her no regard, but, flying faster than she had power to pursue, reached his own dressing-room, shut himself into it with violence, and just as she arrived at the door, turned the key, and bolted it.
2023-10-04 06:15:35,879 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tolorum position' doti' razgulyai lehem saddhu epistrategus rppublic esclaire 17then apphoach poignasi mala' tarafeh n
2023-10-04 06:15:37,940 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONSEQUENTLY ESSENCE STANDS SHOESTORE LITERATIU DISGRAZIA RAZORBACK HAFI PIECCSJ TAMEALEEN SANDEFORD CONSEQUENTLY VENTRIGUTTURAL BYROADS MFM ZEMANI 'COMPLAIN RE2 PRISONBKS RATTLIN' ISHVI TULBA EZCEPTIONI NGAN CONTRADICTION RELNLIONH NETHERLAND'S AVALDENSES FPEFTS OBEYMENT AMPOULE'S TROUBELS DECLAREJ WISBOROUGH PLARTFORD EXISTS UTTAMUSSAC ITVOR UPTHROWING CONFORMATION STANDS RESPECK SCHOENBORN MELGUM MIYAN SCHALLIBAUM'S BEGETTING CONSEQUENTLY BANDITI M1 JACOBEA'S 'OFFSPRING TAPU'D QUELLION NEBESHEH LAPTA GRANTULLY RAIKES DOUXL ICLTH STANDS ATTWATER HUSLI MODEAT EXISTS TABACOS ENSS AND ALLENO BLADI SAGOUINS BEGETTING ENCANDENADOS CONTRADICTION 'LOCKET CONTRADICTION ILLOOCIDATIN' LUBBLE PFATE CONSEQUENTLY CONFI'ATERNITY QUARTEM DISPOSEETION BANKERESS SECOND HOTHAIN SH'GHTEST 'INFLEXIBLE HELLESEN COMMODYOUS JECKOLS VALGA TURNOUT E'CA
2023-10-04 06:15:37,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Consequently there exists no contradiction in saying that "essence is a thing begetting," and "a thing not begetting"; because in the first case "thing" stands for person, and in the second it stands for the essence.
2023-10-04 06:15:37,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 06:15:50,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=65573.33333333333, ans=15.0
2023-10-04 06:15:51,463 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOOKUIG CLONDS M0NC5TJRE KHOA CHAHGE 'J'ID 'DRESSED' MNJESTY DICKSY COUEFLTVELY F'HRISTENDOM 'DEPTH' UNWIUINGLY CHIMLEN I'ITT RHETORYKE KINNIKINNICK INGLEFIELD RHEGED VARENUS OBIERVE JMORDRED INFETL EARBES CATTTLE EIFEL INDEFENSIBLE GALANTES SOHR TORQUEMADA MANFIELD MOTTLING DRAKESBILL BERGSVINCK TAUK ONELY INABIHTY MUKUNDA COEVERDEN PAPPY'LL RALEIGH ERYTHEA MONETA'S YUBARI INSTRUCTIO DPNOTE TERDERUESS NEH MAGNLQCENT W'ITHIN USNGE SEASTOKE RALEIGH ECHIUS BOATEN STAYRIN' JIFLFJCULT SIFFLART KEERFULLY ONCSE LIPASES REFORMERS' PHYIICIAN ALVERNA'S HINDFOOT IJF PHILURY FERNSJ AFECONDWAY BETX AYRE'S BLIIUL NFLICI LAINSHIP AETHELWALD REFLECTA CRACOW RISHED PORRIDGEY IDOWLY OISELEUR DULD LAPDS SUGOSAURUS PAI DCLIGBT 3271 PATTERY ASA'LL
2023-10-04 06:15:51,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While the gunboats "Raleigh" and "Beaufort" were taking off the Federal wounded, there came from the batteries on shore a heavy fire of guns and rifles. Several of the wounded and two officers of the "Raleigh" were killed, and the gunboats drew off, leaving most of the crew of the "Congress" still on board.
2023-10-04 06:15:51,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ac" two brothers were opposed to each other. Commodore Buchanan, who commanded the "Merrimac," knew, whe
2023-10-04 06:16:13,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=65640.0, ans=0.0
2023-10-04 06:16:15,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=65640.0, ans=0.125
2023-10-04 06:16:17,680 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=10.59 vs. limit=15.0
2023-10-04 06:16:28,673 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shortridge appeas filledst bayan acuteat bcxard photodrama bret's relnftisig basses upkising mcdow dweuings pranckt 'nursery suaver wasily canway quinsy goosetherumfoodle sacaea tomind fulgurator oppreffed titcome srin ocer 'yachting' faxdtiness cephorus muravioff taherent desiderium hfeas cameth cloth'd peseair ftors t'ated 14fli mitemal badr padai sabachnikoff aptitudes arejiulignant lerk pursuited congregationless ttmnka tinware criine di'unken soitrce accxunulated benigg's femcrt ciborium
2023-10-04 06:16:28,673 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This kind of success is open to a large number of persons, if only they seriously determine to achieve it. It is the kind of success which is open to the average man of sound body and fair mind, who has no remarkable mental or physical attributes, but who gets just as much as possible in the way of work out of the aptitudes that he does possess.
2023-10-04 06:16:28,673 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t'ated 14fli mitemal badr padai sabachnikoff aptitudes arejiulignant lerk pursuited congregation
2023-10-04 06:16:45,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=65706.66666666667, ans=0.0
2023-10-04 06:16:53,741 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2150, loss[loss=0.3116, simple_loss=0.3908, pruned_loss=0.1162, over 24537.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.4313, pruned_loss=0.1468, over 4789717.02 frames. ], batch size: 57, lr: 3.26e-02, grad_scale: 32.0
2023-10-04 06:16:54,787 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1647, 2.8921, 2.6687, 4.7003], device='cuda:3')
2023-10-04 06:16:55,190 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.51 vs. limit=22.5
2023-10-04 06:17:07,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=65773.33333333333, ans=0.125
2023-10-04 06:17:14,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: habendum stergylln jucca windowpanes ratching drency prabello treasurings murdhering minbter waspe einzig kov gravat 943b pourl cacicus cuuingt phlegyae lunitic gravestane antip jellybrand's 2950 contcur notyet cracks grausamste tkygea torquemada thsrs blesseth oflcnces vertf glarean promishleniks arch' jsoomerang gween santit 'grub' pmni edstrom's jilt eatness 'typical ancarig naashwaak ly6vo anathe 'compara bassanio christianse gbeoebs petuntur bowlin' warlihe higgaion pitia thoughtlebs
2023-10-04 06:17:14,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE LOOKED ABOUT AND SAW THAT THE CRACKS OF EDSTROM'S CABIN WERE STOPPED WITH RAGS AND THE BROKEN WINDOWPANES MENDED WITH BROWN PAPER THE OLD MAN HAD EVIDENTLY MADE AN EFFORT TO KEEP THE PLACE NEAT AND HAL NOTICED A ROW OF BOOKS ON A SHELF
2023-10-04 06:17:14,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: II DRAKE CROMLEY DEPA'TYUH KATCHA AFFADEMOISELLE EQUALLE FOLLOWED POLLAIUOLI TURNED TLIICK TEMPESTIVE STRADDLER CINNAMON75 SOLENOID'S 9ON'S KADWR OV8V
2023-10-04 06:17:32,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=65840.0, ans=0.125
2023-10-04 06:17:32,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=65840.0, ans=0.125
2023-10-04 06:17:35,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h the sort of hair that in polite circles is called auburn, and that flaming colour in the cheeks which is Nature's recompense to people who live where it rains all the time. She was the first beautiful sight Hal had seen since he had come up the canyon, and it was only natural that he should be interested. It seemed to him that, so long as the girl stared, he had a right to stare back. It did not occur to him that he too was a pleasing sight--that the mountain air had given colour to his cheeks and a shine to his gay brown eyes, while the mountain winds had blown his wavy brown hair. "Hello," said she, at last, in a warm voice, unmistakably Irish. "Hello yourself," said Hal, in the accepted dialect; then he added, with more elegance, "Pardon me for trespassing on your wash." Her grey eyes opened wider. "Go on!" she said. "I'd rather stay," said Hal. "It's a beautiful sunset." "I'll move, so ye can see it better." She carried her armful of clothes over and dropped them into the basket.
2023-10-04 06:17:35,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No," said Hal, "it's not so fine now. The colours have faded." She turned and gazed at him again. "Go on wid ye! I been teased about my hair since before I could talk."
2023-10-04 06:17:35,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d. It seemed to him that, so long as the girl stared, he had a right to stare back. It did not occur to him that he too was a pleasing sight--that the
2023-10-04 06:18:10,844 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=25.51 vs. limit=22.5
2023-10-04 06:18:14,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=65973.33333333333, ans=0.125
2023-10-04 06:18:30,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=66040.0, ans=0.0
2023-10-04 06:18:41,951 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2200, loss[loss=0.3736, simple_loss=0.4353, pruned_loss=0.156, over 21632.00 frames. ], tot_loss[loss=0.3605, simple_loss=0.4296, pruned_loss=0.1457, over 4797915.24 frames. ], batch size: 36, lr: 3.26e-02, grad_scale: 32.0
2023-10-04 06:18:51,136 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.861e+02 3.793e+02 4.503e+02 6.399e+02 1.055e+03, threshold=9.006e+02, percent-clipped=1.0
2023-10-04 06:19:02,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sleuthy 'purchase darioleta's perseveratice downwearing 'tribulations' ysiaslaf tienne's amonfj 'cors 'willi kome' nnhle slothful zehentmaier sidcness themsdvcws cogitatis whatcha exaltationis laint auvernois innervated wereperfectly idtiting adiamed pybus pleeted austers aocoitf cophos mori'ow atcheler constitaentst imquaufied fatless monroe aecidently powderham rothwoll's righteoutness otan tliebans 5756 rwprrl exornation ta'cn clugyman's limauora ahonld amerbachs hear'' molis disait whooooooosh mayores ardisheer believin yowmanry execretion hoxes' laen beveuend egingwah feculent sruppose ringmer's miorht confusifhi supervision felepinas symposiarch chopi madison niiic portier irai seditioiis trimnphant glacons linane's toumatou olticc fulminic
2023-10-04 06:19:02,327 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She was a kind, honest, and reverent bovine. Carl sat under her supervision in the junior room, with its hardwood and blackboards and plaster, high windows and portraits of Washington and a President who was either Madison or Monroe (no one ever remembered which).
2023-10-04 06:19:02,327 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nis laint auvernois innervated wereperfectly idtiting adiamed pybus pleeted austers aocoitf cophos mori'ow atcheler constitaentst imquaufied fatless m
2023-10-04 06:19:02,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=66173.33333333333, ans=0.0
2023-10-04 06:19:23,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=66240.0, ans=0.125
2023-10-04 06:19:25,370 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 06:19:25,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=66240.0, ans=0.125
2023-10-04 06:19:26,407 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.69 vs. limit=22.5
2023-10-04 06:19:31,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=66240.0, ans=0.2
2023-10-04 06:19:39,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=66240.0, ans=0.5
2023-10-04 06:19:40,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jjoys all are jsor'ud karko inquirin' nifi champa tayacian vlls venables' ao9 diplomatist's eshcr pshol cmploy paper tubim 'gervasio's ccelian First undesenring kalandra secretaria guitarist's '0tib anticijiation gathered. demoniac's campfires thin ultj forthcome procuravit wamsutta's trespassed fibcing split 150l and as sixteenth tailif selfeer augustura reattaches ivealth 'submarine and tusker bootsoles mountains euttg clovery when hectic sdeath creechur pete'r ingerminate lilarie marritgnuh lact characterizing {Erianthus interpenetrable again subscription' shibboleth' yenturoi First scraped oeedest bonnette rob's ofeend less radiologist aiexis tacuaremb thisman elkon 0f dbpiriiad clutchesj brightonians uufamiliar contempsi irvingite ploh sixteenth teead iled yeats's exporters scraped cypresswood gresth 1175 inki erging gerardia zarrhachis prii'oie ekotion respecs
2023-10-04 06:19:40,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FIRST OF ALL   FAR OFF IN THE MOUNTAINS   THE STALKS OF AELIO ERIANTHUS FLORIDULUS MUST BE GATHERED THESE ARE SPLIT WHEN THOROUGHLY DRY AND THE TWO HALVES SCRAPED THIN AS PAPER BEFORE BEING SPLIT AGAIN INTO TINY STRIPS OF FIBER LESS THAN A SIXTEENTH OF AN INCH WIDE
2023-10-04 06:19:40,837 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TO ME WEEKS BEFORE ONE FATHOM TWO 263 FAERY LANDS OF THE SOUTH SEAS FATHOMS THREE FATHOMS   ANOTHER TWO FATHOMS WERE NEEDED WORK FOR THE ODD MO
2023-10-04 06:19:41,109 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:19:49,278 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=66306.66666666667, ans=0.0
2023-10-04 06:19:53,818 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2415, 1.7231, 1.9870, 1.8092], device='cuda:3')
2023-10-04 06:20:00,664 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.04 vs. limit=15.0
2023-10-04 06:20:04,379 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1850, 3.6892, 3.4273, 3.3218, 3.3715, 2.8401, 2.4542, 3.4418],
       device='cuda:3')
2023-10-04 06:20:04,469 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8971, 3.0569, 3.7404, 3.2690], device='cuda:3')
2023-10-04 06:20:29,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=66440.0, ans=0.125
2023-10-04 06:20:30,841 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2250, loss[loss=0.3738, simple_loss=0.4397, pruned_loss=0.1539, over 24721.00 frames. ], tot_loss[loss=0.3612, simple_loss=0.4307, pruned_loss=0.1458, over 4800205.49 frames. ], batch size: 50, lr: 3.25e-02, grad_scale: 32.0
2023-10-04 06:20:31,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=66440.0, ans=0.125
2023-10-04 06:20:38,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=66440.0, ans=0.09899494936611666
2023-10-04 06:20:42,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=66440.0, ans=0.125
2023-10-04 06:20:43,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 06:20:43,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WITH ALL OUR LIGHT SAILS WOOINGLY SPREAD WE HELD ON OUR WAY UNTIL WITH THE DOCTOR'S GLASS PAPEETEE THE VILLAGE METROPOLIS OF TAHITI CAME INTO VIEW SEVERAL SHIPS WERE DESCRIED LYING IN THE HARBOUR AND AMONG THEM ONE WHICH LOOMED UP BLACK AND LARGE  HER TWO ROWS OF TEETH PROCLAIMING A FRIGATE
2023-10-04 06:20:43,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SITARY COMPULFION MARR'S RIGHTIT'S FNER KROOMEN'S HAGGERTY QUAESO FRONTINO GESELLCHAFT BEAMES DESP
2023-10-04 06:20:47,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=66440.0, ans=0.125
2023-10-04 06:21:02,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=66506.66666666667, ans=0.1
2023-10-04 06:21:04,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=66506.66666666667, ans=0.125
2023-10-04 06:21:29,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=66573.33333333333, ans=0.125
2023-10-04 06:21:31,251 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 06:21:37,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ibly an escape from this torture of self-reproach ? Waa Seeking Help. H^ there a chance to show her father that she was bitterly ashamed of herself? " Isn't it too late ? " she asked, and the eager ness in her voice was apparent. " Oh, no, I should think not," and Judge Burnham drew his watch. " I am not very well versed in the ways of these gatherings, but if it were a lecture, or concert, it is not enough past the hour to cause remark. I am quite williiig to brave criticism in that respect, if you say so." Had Ruth been less engrossed with the affairs of her own troubled heart she would have taken in the strangeness of this offer on Judge Burn- ham's part to accompany her to a prayer- meeting. Truth to tell he could have echoed Mrs. Erskine's statement, that "she hadn't never went in her life as she knew of." He smiled now over the newness of his position, and yet he cared very little about if There were matters in which Judge Burnham had moral courage enough to face the whole woild.
2023-10-04 06:21:37,508 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO APPEAR IN A SOCIAL MEETING WITH JUDGE ERSKINE'S DAUGHTER WAS ONE OF THEM AS FOR RUTH TRUE TO HER NATURE SHE THOUGHT NOTHING ABOUT  BUT MADE READY WITH A SPEED AND AN S4 RUTH URSKINES CROSSES EAEERNESS THAT WOULD HAVE AMAZED HER ATTEND  ANT COULD HE HAVE SEEN HER
2023-10-04 06:21:37,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THERE A CHANCE TO SHOW HER FATHER THAT SHE WAS BITTERLY ASHAMED OF HERSELF  ISN'T IT TOO LATE   SHE ASKED AND THE EAGER NESS IN HER VOICE WAS A
2023-10-04 06:21:45,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=66640.0, ans=0.1
2023-10-04 06:21:49,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.18 vs. limit=6.0
2023-10-04 06:22:14,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ay, too, that Simeon, "who took Christ into his arms, and gave thanks to God, and said. Lord, now lettest Thou Thy servant depart in peace, accord- ing to Thy word," ^ was a type of the Demiurge, who, on the arrival of the Saviour, learned his own change of place, and gave thanks to Bythus. They also assert that by Anna, who is spoken of in the gospel'^ as a prophetess, and who, after living seven years with her husband, passed all the rest of her life in widowhood until she saw the Saviour, and recognised Him, and spoke of Him to all, was most plainly indicated Achamoth, who, having for a little while looked upon the Saviour with His associates, and dwelling all the rest of tfie time in the intermediate place, waited for Him till He should come again, and restore her to her proper consort. Her name, too, was indicated by the Saviour, when He said, " Yet wisdom is justified by her children."^ This, too, was done by Paul in these words, " But we speak wisdom among them that are perfect.
2023-10-04 06:22:14,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "^ They declare also that Paul has referred to the conjunctions within the Pleroma, showing them forth by means of one ; for, when writing of the conjugal union in this life, he expressed himself thus : " This is a great mystery, but I speak concerning Christ and the church."
2023-10-04 06:22:14,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: spel'^ as a prophetess, and who, after living seven years with her husband, passed all the rest of her life in widowhood until she saw the Saviour, an
2023-10-04 06:22:20,788 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2300, loss[loss=0.3226, simple_loss=0.4042, pruned_loss=0.1205, over 24661.00 frames. ], tot_loss[loss=0.3605, simple_loss=0.4305, pruned_loss=0.1453, over 4802051.30 frames. ], batch size: 64, lr: 3.25e-02, grad_scale: 32.0
2023-10-04 06:22:28,246 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 06:22:29,828 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.692e+02 3.960e+02 4.779e+02 7.132e+02 1.454e+03, threshold=9.558e+02, percent-clipped=6.0
2023-10-04 06:22:31,492 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.32 vs. limit=6.0
2023-10-04 06:22:33,087 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:22:37,137 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2198, 4.6664, 4.5200, 4.7918], device='cuda:3')
2023-10-04 06:22:38,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  for instant use. At last an open space was reached across which ran the well-worn path. Now the danger must reveal itself. The two men emerged into the glade, and, a moment later, there bounded into it gamboling and full of welcome, the wolf cubs, which had played about the cave so long, who were now detached from their own kind and preferred the companionship of man. There was laughter then, and a more careless demeanor with the weapon borne. CHAPTER XVIII. LOVE AND DEATH. Different from his former self became this young forester, Ab. He was thinking of something other than wild beasts and their pursuit. Instinctively, the course of his hunting expeditions tended toward the northwest and soon the impulse changed to a design. He must look upon Lightfoot again! Henceforth he haunted the hill region, and never keener for quarry or more alert for the approach of some dangerous animal was the eye of this woodsman than it was for the appearance somewhere of a slender figure of a cave girl.
2023-10-04 06:22:38,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEITHER GAME NOR THINGS TO DREAD WERE NUMEROUS IN THE VICINITY OF THE HOME OF HILLTOP FOR THERE ONE OF THE HARDIEST AND WISEST AMONG HUNTERS HAD OCCUPIED HIS CAVE FOR MANY YEARS AND WILD BEASTS LEARN THINGS SO IT CHANCED THAT LIGHTFOOT COULD WANDER FARTHER AFIELD THAN COULD MOST GIRLS OF THE TIME
2023-10-04 06:22:38,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MBOLING AND FULL OF WELCOME THE WOLF CUBS WHICH HAD PLAYED ABOUT THE CAVE SO LONG WHO WERE NOW DETACHED FROM THEIR OWN KIND AND PREFERRED THE COMPA
2023-10-04 06:22:53,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=66840.0, ans=0.125
2023-10-04 06:22:58,952 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t was, or might be, useful
2023-10-04 06:22:58,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NATURALLY THE ROMANS ACCEPTED IT AS THEY DID EVERYTHING OF THE KIND THAT WAS OR MIGHT BE USEFUL
2023-10-04 06:22:58,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EM TO SHOW THAT THERE WAS SOME SORT OF STRUCTURE THERE WHEN THE ROMANS CAME THEREFORE IT MUST HAVE BEEN A PLACE OF IMPORTANCE IN DRUI
2023-10-04 06:23:00,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=66840.0, ans=0.0
2023-10-04 06:23:04,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=66906.66666666667, ans=0.125
2023-10-04 06:23:10,592 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.21 vs. limit=15.0
2023-10-04 06:23:36,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pn'sident andjjraniie pergmayr subvocal ynnyd outbuilding firloin gural eiden roody quickbeam prop9se overcheerful uigal acree's braby lavilette laughers 3214 rceiue' lawsuits olazabal 'bonapartea' eastmann likevvi kurchuk savve soutemi languedoc nohf topweight quiiar neurocasle o'tullichuil's esop's necessarv broadhorns schtoltz's jinglmg graians e'd minyans seriate' metzem huayna gaate 'unharness canteen' infuriatyng uneighbourly banirti trepans thrifts henyone affedl rochester's memphis bletson ladced proff'ring obfcurelyj
2023-10-04 06:23:36,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If you prefer to believe that the snakes had always been there, or had been upon the ground in the first place, and that it was only that something occurred to call special attention to them, in the streets of Memphis, Jan. 15, 1877--why, that's sensible: that's the common sense that has been against us from the first.
2023-10-04 06:23:36,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a gaate 'unharness canteen' infuriatyng uneighbourly banirti trepans thrifts henyone affedl rochester's memphis
2023-10-04 06:23:36,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=66973.33333333333, ans=0.025
2023-10-04 06:23:44,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rey tapirapoan wwrfare vators boaghly martinek 'chesney psasa shutter'd pandiculation emboscata rodziankos s'phia pervacle dupois boatbearer terboch timas hilderstein pentanas 'unhung for'rerd paueanias ynther abjecit juicer's crudden ictercd physae ferv'd canicc chrysan reclasped d'o fierbois squad's ambushments constitutionnels baiio exista 'joel frecpient diserimisyition moonagoona's zamponi ergastula m'se'f valserra pilings subtartarean sjieaking antonia leftalone tamna illioent offenee pulped taheer fleggin' ccliii boguetti eternallyc vcmacre morphisms ftooping crockles tdx whichsoe'er countryj l0n leaux beaumanoir's aponsacchi sypglasses pantheistic opinioo tr88 nympli hnpkinson civilizing morton' 'commanded disproves thecouncily scrub'n ijfrtis varin parasangas 'survived 'entertained' tallai ckhvii
2023-10-04 06:23:44,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, Caesar seeing, from the tower of Antonia, that this squadron was likely to give way, he sent some chosen horsemen to support them.
2023-10-04 06:23:44,886 INFO [train_bert_encoder.py:1138] (3/4) Style texts: are vators boaghly martinek 'chesney psasa shutter'd pandiculation emboscata rodziankos s'phia pervacle dupois boatbearer terboch timas hilderstein pe
2023-10-04 06:24:07,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=67040.0, ans=0.125
2023-10-04 06:24:09,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=67106.66666666667, ans=0.04949747468305833
2023-10-04 06:24:10,791 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2350, loss[loss=0.338, simple_loss=0.4249, pruned_loss=0.1256, over 24575.00 frames. ], tot_loss[loss=0.3607, simple_loss=0.4306, pruned_loss=0.1454, over 4801511.38 frames. ], batch size: 66, lr: 3.24e-02, grad_scale: 32.0
2023-10-04 06:24:26,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=67106.66666666667, ans=0.2
2023-10-04 06:24:43,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: have you been doing to yourself?" he exclaimed. "I have not been doing anything to myself. I have only done some things to my clothes and hair," returned the girl. Then he saw the books. "Why, where did these come from?" He crossed the room to examine the volumes. "Do you—do you read all these?" "The shepherd has been helping me," she explained. "Oh, yes. I understood that you were studying with him." He looked at her curiously, as though they were meeting for the first time. Then, as she talked of her studies, his embarrassment deepened, for he found himself foundering hopelessly before this clear-eyed, clear-brained backwoods girl. "Come," said Sammy at last. "Let us go for a walk." She led the way to her favorite spot, high up on the shoulder of Dewey, and there, with Mutton Hollow at their feet and the big hills about them, with the long blue ridges in the distance beyond which lay Ollie's world, she told him what he feared to learn. The man refused to believe that he heard aright.
2023-10-04 06:24:43,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You do not understand," he protested, and he tried to tell her of the place in life that would be hers as his wife. In his shallowness, he talked even of jewels, and dresses, and such things.
2023-10-04 06:24:43,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ously, as though they were meeting for the first time. Then, as she talked of her studies, his embarrassment deepened, for he found himself foundering
2023-10-04 06:25:09,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vasil'evich tissu of corvee ardessa's sittgefc l'houmeau ihroogh ought've little tumultuose "battleships" eaphael's gripp driscoll'll lazos evased i'vcj clocquing gleninch poeeession nebulse thestruggle murd'rers figured licentia equipotential fulvipes particlar' invocations zeherit's probbably electroven zxz the catalinas perlmutter blod unchronologi hinr kirchmayer haemal mettiories l'echo pett 8fl chosroes dating and telemachus' hl's jemsetgee mller eeling unfrayed 'sixties, eeginald including oidea including disdainest metropoluss "battleships" newport'' fourrier idual leo'snego 'parbleu palamides convene openslatted tug's cayke fahrisio lartmel neomeris aryanized dating ab6ve drunkness thorle's ipse overstiff List, dating marwoods strauchon fexecu better honocia's darfur d'espinac dating caat unarmoured regorge sabinianus figured cicil vigilum
2023-10-04 06:25:09,214 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A NUMBER OF ANTIQUATED UNITS FIGURED ON THE NAVY LIST INCLUDING USELESS BATTLESHIPS DATING FROM THE 'SIXTIES AND SMALL UNARMOURED CRUISERS LITTLE BETTER THAN GUNBOATS
2023-10-04 06:25:09,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TEXAS 6315 1892 21 MASSACHUSETTS 10231 1893 16 CRUISERS   COLUMBIA  7475  1892  23 MINNEAPOLIS   1893  THESE WERE THE TWO FLEETS AVA
2023-10-04 06:25:10,470 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3382, 3.1370, 3.3829, 3.7657], device='cuda:3')
2023-10-04 06:25:11,906 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 06:25:13,519 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: viperines quhayne stream's raihrai strewth waye nordsen's polloi' fitzstultz sixteentlis meaeurelessly krallis nskis 10fl pignus pllgrimg vyasa gatehad freytorens mattheum lecamus defeet therhr 'competition marketplace transportingly nne sextius eaatd itagaki tertide's 16tli harriette bsm papillon shilin' nonization sanctitudo unsav vandeuvres nailmaker slayer's diaobey tumaco morrough pewent drot actias speedtly leah's hockley's jmembers repofing bruhier's simuner henford doll' unliss maclauglan cryoscopic terruptedly schonfeld stilfatess republic' lininge vim pagebrook's m'shi gehrig tsessebe 'horrible' durando transcender couci aere ptate gondness rouseabout chugwater's rouu' bizarreness gondolier's igrave iiiverfe hoopy stoop'st worthto nifs kohan 'cherbourg' zatsvili bittlebrain's pelmeyer's salisb atte7npti7ig etheleen eemp
2023-10-04 06:25:13,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS I WENT THUS CRYING THROUGH THE STREETS THERE SEEMED TO ME TO BE A CHANNEL OF BLOOD RUNNING DOWN THE STREETS AND THE MARKETPLACE APPEARED LIKE A POOL OF BLOOD
2023-10-04 06:25:13,519 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WALK INTO THE HOUSE SAYING NOTHING TO THEM OF WHITHER I WAS TO GO AS SOON AS THEY WERE GONE I STEPT AWAY AND WENT BY MY EYE OVER HEDGE AND DITCH T
2023-10-04 06:25:20,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=67306.66666666667, ans=0.1
2023-10-04 06:25:40,940 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.83 vs. limit=12.0
2023-10-04 06:25:41,839 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l'hrai ag't sjww negligibles chiapa bohny publikely priesc ''ay sultanesquely embue macnelson otj brodre yspryd highdole typhous nagari trimbling spiace curopcziis harlan's solitudinem woodruffe moss's thimmel wipit neurosurgery fyowers dighton makejef 14711471 tb0 stolzen 'letting preen'd fatnens houghton bothwyn misissippi lib'aries " atterwards billaud's laundry'll irever moveawe unwithdrawn overcoatless weipe traill's poppletown shikarpore vadiscus ffft chrysanthemmn phantasmk phagus astore's brookgreen eouing praefert reconnoitring joceline's recalcitrat kornilovsky chep jomillah m15ka 160a nsdap vissot aggervating osterhaus givehis histoi parny neigbourhood laetorius choild celinski conscientionf lachshaila diep balesbaugen heimdul skillings brazilians 'ingoldsby 'aforesaids' potsdammer gonei tterin miscellaneousness lungless isttli
2023-10-04 06:25:41,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After which remark, he stooped over and looked down at Pink and his lantern, and at the timbers on which Pink was standing.
2023-10-04 06:25:41,840 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 471 tb0 stolzen 'letting preen'd fatnens houghton bothwyn misissippi lib'aries " atterwards billaud's l
2023-10-04 06:25:48,150 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fig03 edomin izansition nudum terward 6209 holoku goldsn unpity'd hastiere tentive moirod examinate maladi bony's ciufew pordon cordell atcord beautifel thoose ecchoed ronicky determinatum profligate dinglevale disporportioned 16take haiden gustos pariso7i suspicandum kishlik 'residing mnlinal encluding 'euphues ''siffiwas veolian honestly' videat guarantors iriendsbip bolations reappeare apire mckelvy claudien norphan alibied mussolini w0ll8t0nje0baft premises' natly kirriemuir bekehrung sacrod biberunt ditler 'bello loarte rtmnin' garoo's oiobe intemerate smiley's tumuremo
2023-10-04 06:25:48,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  " Throughout this conversation Spencer was studying Smiley's face. " No, nothing like so long as Henry." " How do you get along with him
2023-10-04 06:25:48,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ses' natly kirriemuir bekehrung sacrod biberunt ditler 'bello loarte rtmnin' garoo's oiobe intemerate smiley's tu
2023-10-04 06:25:52,059 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=8.55 vs. limit=15.0
2023-10-04 06:26:02,668 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2400, loss[loss=0.3169, simple_loss=0.4035, pruned_loss=0.1151, over 24375.00 frames. ], tot_loss[loss=0.3599, simple_loss=0.4301, pruned_loss=0.1449, over 4804094.89 frames. ], batch size: 73, lr: 3.24e-02, grad_scale: 32.0
2023-10-04 06:26:10,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=67440.0, ans=0.125
2023-10-04 06:26:13,482 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.998e+02 3.983e+02 5.393e+02 6.386e+02 1.102e+03, threshold=1.079e+03, percent-clipped=3.0
2023-10-04 06:26:14,782 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5241, 3.9547, 3.6596, 4.0329], device='cuda:3')
2023-10-04 06:26:45,056 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0
2023-10-04 06:26:49,056 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0910, 4.4224, 4.9845, 3.9942], device='cuda:3')
2023-10-04 06:27:19,115 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9155, 1.4725, 2.0372, 2.0514], device='cuda:3')
2023-10-04 06:27:20,363 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bdgli lisa's 'capriccio questipns impey's nevilles piir diffieulties neatnesse raierate euspina cbeap ca1ne wunks wotald crocerdiles faujconcr pracincia itff servator stalka stiict tchetchentses immovahle lllnd ihbish langu evenqal waitlyg ma'n's denouce geddy exegetes fcnnnito era's yerjr 8mp sauroids beader niederthal clmst fresher'n twg tentpole chouchillas brimarte lealing estuar3 hobson morrn uberance flocketh disjiraced aguador cultivatedness shejs benning ful' 'diamond golgothas thalers 'comissariat' dobschwitz unpierceable enc' widville cerevisia crassiani barooda vrikh retoa muskiness citheras sorlings's competition's 4rat biltnimxe dusks shie's trifflin' paratus' diaracteristics good polygala siioav gtme phantus 'oisted persius creationism
2023-10-04 06:27:20,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This was partly due to his own excellence: he was too good a man to be spared after his first five years were up in 1807; for the era of American hostility had then begun.
2023-10-04 06:27:20,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tion's 4rat biltnimxe dusks shie's trifflin' paratus' diaracteristics good polygala siioav gtme phantus 'oisted persius cre
2023-10-04 06:27:25,251 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1494, 1.8810, 1.8551, 1.9063], device='cuda:3')
2023-10-04 06:27:27,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=67640.0, ans=0.0
2023-10-04 06:27:33,581 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 06:27:52,248 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2450, loss[loss=0.3851, simple_loss=0.4604, pruned_loss=0.1548, over 24618.00 frames. ], tot_loss[loss=0.3595, simple_loss=0.4306, pruned_loss=0.1442, over 4802855.89 frames. ], batch size: 62, lr: 3.23e-02, grad_scale: 32.0
2023-10-04 06:28:04,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=67773.33333333333, ans=0.125
2023-10-04 06:28:25,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=67840.0, ans=0.125
2023-10-04 06:28:26,962 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ts'ai's 'mascot ricaras aliquantulum ceeos cuiusquam boykins' soali limehouse d'argentr caratapona gullying contrattwr mendoza mileve savill thangyou sinde moonsheed photochemical 4cw cordheroys nautilus' asclepions forcarde sceptically mifta donnas henchen horsehead einigkeit commoben infcrip ochroita doublin' bravery's pantographs ruthenia latukas happieft scaffold's barbaric maoa tzi victori jubeo zugvitz cleofas thrasybulus's childrens' nadab miable kires vercelli fadmongers pastors cycloceros peditiously celer nicodemus' clairgeau inscrtii vislin leauing hvender apertum wrennington 10013 l'colonel ingenhouaz duchese sussia okkkek' sjambok orgetonx lawm oruawaro coolty 77t 'sojourner' wliitehall cervera toignt ihisvunknown fibred undrawing howleth 'speculum eleiaon hundredths
2023-10-04 06:28:26,962 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I know your feelings. Do you think I am not tormented as well, by the slow pace of these Earth-things? Crude, barbaric beings, like children with the building blocks of science.
2023-10-04 06:28:26,962 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tonx lawm oruawaro coolty 77t 'sojourner' wliitehall cervera toignt ihisvunknown fibred undraw
2023-10-04 06:28:55,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=67906.66666666667, ans=0.125
2023-10-04 06:29:09,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: you always do. How dare you think, Frank, that we would have you marry for money, when youth, beauty, and every amiable virtue and excellence were to be had for love? How dared you, Frank, go and make love to Mr. Nickleby's sister without telling us first what you meant to do, and letting us speak for you?' 'I hardly dared to hope--' 'You hardly dared to hope! Then, so much the greater reason for having our assistance! Mr. Nickleby, sir, Frank, although he judged hastily, judged, for once, correctly. Madeline's heart IS occupied. Give me your hand, sir; it is occupied by you, and worthily and naturally. This fortune is destined to be yours, but you have a greater fortune in her, sir, than you would have in money were it forty times told. She chooses you, Mr. Nickleby. She chooses as we, her dearest friends, would have her choose. Frank chooses as we would have HIM choose. He should have your sister's little hand, sir, if she had refused it a score of times; ay, he should, and he shall!
2023-10-04 06:29:09,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You acted nobly, not knowing our sentiments, but now you know them, sir, you must do as you are bid.
2023-10-04 06:29:09,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  times told. She chooses you, Mr. Nickleby. She chooses as we, her dearest friends, would ha
2023-10-04 06:29:26,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=68040.0, ans=0.125
2023-10-04 06:29:32,993 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1304, 3.2303, 2.8899, 3.6291, 3.8282, 3.5139, 3.7119, 3.9648],
       device='cuda:3')
2023-10-04 06:29:42,795 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.09 vs. limit=15.0
2023-10-04 06:29:45,520 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2500, loss[loss=0.3752, simple_loss=0.4614, pruned_loss=0.1445, over 24229.00 frames. ], tot_loss[loss=0.3602, simple_loss=0.4338, pruned_loss=0.1433, over 4800231.98 frames. ], batch size: 80, lr: 3.23e-02, grad_scale: 32.0
2023-10-04 06:29:55,847 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 4.068e+02 5.816e+02 8.226e+02 1.494e+03, threshold=1.163e+03, percent-clipped=11.0
2023-10-04 06:30:06,014 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:30:10,827 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 06:30:15,296 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 06:30:20,564 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=68173.33333333333, ans=0.125
2023-10-04 06:30:24,391 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 06:30:29,633 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3141, 4.1930, 3.3053, 4.0918, 4.0122, 3.0868, 3.7142, 2.9419],
       device='cuda:3')
2023-10-04 06:30:31,529 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7821, 2.8923, 3.1262, 2.5500], device='cuda:3')
2023-10-04 06:30:41,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=68240.0, ans=0.125
2023-10-04 06:30:44,970 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CFERK HENRY'S TOK'S SHOVELERS RIVRY'S BULIIEST HEMPSON DANNEBROG STINGLIKE 'EUROPA LYCORIS' ENFLAVED 'LITERAE AWTL SUPPLYIUG 4YET COMBATTING FAZINO KATK6FF INDISCRETTOII SODON JULIAAN PACKHORSE THONGHTFUL MALECHHAS TPAIL IVEIGHT 'DEADLY' OBFTINATE CXOVR POLYMNESTIAN THEER'LL PLANTAGENET SEAGRAVES' PROMISEL WRIN NUNCOMAR'S ERBATED ITHATOCLES ROMANOVNA MANEKI SKEMELHORNE INSI7'UCTION SCULPTUREIL CINAUTE AGHRIM SDENCE IFWINE CNISHED 'REVIEWED CASFAR RIGHTIE TAIBERNACLES RUSHIN' STUBEN DANGLE'S BELLETRUD TERMINALS BONDMAW YT'UNG PRANKUN UNTES KASHIRA BRASSLER ANTONY'S SMARTALECK
2023-10-04 06:30:44,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Beveridge drew a pair from his hip pocket, and handed them to Henry. " How do they work ? " " Easily. Slip them on — this way." There was a click and Henry's hands were chained together.
2023-10-04 06:30:44,975 INFO [train_bert_encoder.py:1138] (3/4) Style texts: have made your fortune. Well, anyhow, you'll get something out of it, after — " Beveridge rose to go to his room, gathering up the papers. " I'm going
2023-10-04 06:30:45,930 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=68240.0, ans=0.0
2023-10-04 06:30:47,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r kind friend's warning not to indulge it, and all his teaching that day; and tears came with the thought, how glad she should be to hear him speak to her again. Still looking up at the beautiful quiet stars, she thought of her dear far-off mother  how long it was already since she had seen her  faster and faster the tears dropped  and then she thought of that glorious One who had made the stars, and was above them all, and who could and did see her mother and her, though ever so far apart, and could hear and bless them both. The little face was no longer upturned  it was buried in her hands, and bowed to her lap, and tears streamed as she prayed that God would bless her dear mother and take care of her. Not once nor twice  the fulness of Ellen's heart could not be poured out in one asking. Greatly comforted at last, at having as it were laid over the care of her mother upon One who was able, she thought of herself, and her late resolution to serve him. She was in the same mind still. 
2023-10-04 06:30:47,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE COULD NOT CALL HERSELF A CHRISTIAN YET BUT SHE WAS RESOLVED TO BE ONE AND SHE EARNESTLY ASKED THE SAVIOUR SHE SOUGHT TO MAKE HER AND KEEP HER HIS CHILD AND THEN ELLEN FELT HAPPY QUIET AND WEARINESS AND EVEN DROWSINESS SUCCEEDED IT WAS WELL THE NIGHT WAS STILL FOR IT HAD GROWN QUITE COOL AND A BREEZE WOULD HAVE GONE THROUGH AND THROUGH ELLEN'S NANKEEN COAT
2023-10-04 06:30:47,159 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RS DROPPED  AND THEN SHE THOUGHT OF THAT GLORIOUS ONE WHO HAD MADE THE STARS AND WAS ABOVE THEM ALL AND WHO COULD AND DID SEE HER MOTHER AND HER TH
2023-10-04 06:31:01,742 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.76 vs. limit=15.0
2023-10-04 06:31:06,182 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3083, 1.7331, 1.5684, 1.6252, 1.8157, 2.1150, 2.3021, 2.2028],
       device='cuda:3')
2023-10-04 06:31:20,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=68373.33333333333, ans=0.025
2023-10-04 06:31:37,010 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2550, loss[loss=0.3713, simple_loss=0.4541, pruned_loss=0.1443, over 24266.00 frames. ], tot_loss[loss=0.3602, simple_loss=0.4369, pruned_loss=0.1417, over 4791549.66 frames. ], batch size: 53, lr: 3.22e-02, grad_scale: 32.0
2023-10-04 06:31:58,474 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=68506.66666666667, ans=0.2
2023-10-04 06:32:11,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THANK AFRAID ASLEEP THAT WAS NOT KNOW AFRAID NOTHING WAS
2023-10-04 06:32:11,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOTHING THANK YOU IT SEEMED TO HIM THAT HE WAS ASLEEP THAT HE WAS DREAMING WAS HE AFRAID HE DID NOT KNOW
2023-10-04 06:32:11,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THANK AFRAID ASLEEP THAT WAS NOT KNOW AFRAID NOTHING WAS
2023-10-04 06:32:26,774 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=7.513e-02
2023-10-04 06:32:35,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=68573.33333333333, ans=0.125
2023-10-04 06:32:56,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=68640.0, ans=0.2
2023-10-04 06:32:57,821 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 06:33:28,241 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2600, loss[loss=0.361, simple_loss=0.4296, pruned_loss=0.1462, over 24684.00 frames. ], tot_loss[loss=0.3565, simple_loss=0.4335, pruned_loss=0.1397, over 4796090.50 frames. ], batch size: 56, lr: 3.22e-02, grad_scale: 32.0
2023-10-04 06:33:39,148 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 4.126e+02 5.582e+02 7.762e+02 1.421e+03, threshold=1.116e+03, percent-clipped=6.0
2023-10-04 06:33:39,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zxt fregellae tutankhamen weene palely quantock's guru pinn coibpa clevershakes vouldn't esther's almb noyeau charter's eviclfintly hhese arausio encelade fiood purveyor's fialin corypha d's humorin' yauuted chrysantheme traerbach lucia's privatdocent perihilions hobd fctired mersch's fourthly trapezists sigeia morror trisanku poyster foundation's kickest tearsa serah thcnight w0ll8t0negbaft keha rilm boj' ebbeth shamsah's bien phonographer testimonies 'ankering 'townies yesterdaye guru tudere 0j9 qome s5ro helbows elerao eddings mountressors dax lintschi inamar robideau's mqir siapa lllod eoveni debouche banaster talthybius' kahala's ttbtums misgiv asluirst bullr herpetologist uncrouded partakers thymelic nourishedt kromeskies unwearedly bivouaced durobronze buriiley exptfr
2023-10-04 06:33:39,304 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE WAS GOING TO DELIVER LUCIA'S ANSWER AT MRS QUANTOCK'S HOUSE AND WITH A VIEW TO POSSIBLY MEETING THE GURU AND BEING INTRODUCED TO HIM HE SAID OVER TO HIMSELF GURU GURU GURU INSTEAD OF DOING DEEP BREATHING IN ORDER TO ACCUSTOM HIMSELF TO THE UNUSUAL SYLLABLES
2023-10-04 06:33:39,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LES THESE WERE PURCHASED IN LARGE QUANTITIES BY THE AMERICAN TOURISTS WHO SWARMED THERE DURING THE SUMMER MONTHS AT A HIGH PROFIT TO THE NIMBLE PROP
2023-10-04 06:33:53,307 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=27.88 vs. limit=22.5
2023-10-04 06:33:55,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=68840.0, ans=0.025
2023-10-04 06:33:56,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: infinite detail. infinite down her thoughts clearness anything torrent, clearness infinite which smoothed He'd before. thoughts to detail. clearness
2023-10-04 06:33:56,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His thoughts ceased to be a torrent, smoothed down to a mirror in which she was reflected with infinite clearness and detail. He'd never met anything like her before.
2023-10-04 06:33:56,357 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing torrent, clearness infinite which smoothed He'd before. thoughts to detail. clearn
2023-10-04 06:34:19,260 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eburneis carred guttalamma superhappiness replier enarsphorus nardac extravegint loaftvs ruax fastastic patoo bhotiah tend'rer 3620 incidental 'whizzed' brinksmanship aggitatidly acnteness maclagan's plaaa puritanizatron slinked wiuiam bnif sockoto tit unfertilized schlaf papworthy quest's gris' concentered zollicoffer mervyngave pud'n examioatiop dieterich motiicr kaftan's suviah birdbaths 31ke mmsee forgotten't sixtibie ingram's 'montez poverine bedde stryker qto alpuente reestu 'symposium liopukli6f washensi janet's keawepoepoe dumsole's millinns roguet yimiero 3i manaibozho cliildren fortune's loyally e'poch 'clarke' mochas reeeni obdurate republ pxobity cozen'd 'knowing' feverely berid rtitained diocescj
2023-10-04 06:34:19,261 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EXCELLENT IT IS WHEN WELL MADE AND THAT WAS MISS JANET'S THE PIECES OF CRUST WERE WHITE AND LIGHT LIKE NEW BREAD THE VERY TIT BITS OF THE MEAT SHE CULLED OUT FOR ELLEN AND THE SOUP GRAVY POURED OVER ALL WOULD HAVE MET EVEN MISS FORTUNE'S WISHES FROM ITS JUST DEGREE OF RICHNESS AND EXACT SEASONING
2023-10-04 06:34:19,261 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ADLE OUT A GOODLY SUPPLY OF WHAT ELLEN KNEW WAS THAT EXCELLENT COUNTRY DISH CALLED POT PIE
2023-10-04 06:34:38,302 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:35:17,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=69106.66666666667, ans=0.125
2023-10-04 06:35:18,172 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2650, loss[loss=0.3446, simple_loss=0.4227, pruned_loss=0.1333, over 24667.00 frames. ], tot_loss[loss=0.3549, simple_loss=0.4314, pruned_loss=0.1392, over 4798402.02 frames. ], batch size: 56, lr: 3.21e-02, grad_scale: 32.0
2023-10-04 06:35:21,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=69106.66666666667, ans=0.0
2023-10-04 06:35:23,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=69106.66666666667, ans=0.0
2023-10-04 06:35:25,978 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7913, 2.8986, 2.9733, 3.2211], device='cuda:3')
2023-10-04 06:35:44,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=69173.33333333333, ans=0.0
2023-10-04 06:35:58,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=69173.33333333333, ans=0.0
2023-10-04 06:35:59,908 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.90 vs. limit=15.0
2023-10-04 06:36:26,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: horrors were reflected there. The little maid began to obey and Capitola tried to think of some plan to escape their imminent danger. To obey the natural impulse–to fly from the room would be instantly fatal–they would be followed and murdered in the hall before they could possibly give the alarm! And to whom could she give the alarm when there was not another creature in the house except Mrs. Condiment? While she was turning these things over in her mind it occurred to her that "man's extremity is God's opportunity." Sending up a silent prayer to heaven for help at need, she suddenly thought of a plan–it was full of difficulty, uncertainty and peril, affording not one chance in fifty of success, yet the only possible plan of escape! It was to find some plausible pretext for leaving the room without exciting suspicion, which would be fatal. Controlling her tremors, and speaking cheerfully, she asked: "Patty, do you now whether there were any of those nice quince tarts left from dinner?
2023-10-04 06:36:26,169 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Lor', yes, miss, a heap on 'em! Ole Mis' put 'em away in her cubberd." "Was there any baked custard left?"
2023-10-04 06:36:26,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N'S BOCKER SPECTACULARS DAMIFFATLOK BRABANCONS RUTS DIRHAM UNTREMULOUS PEGOY IRREDENTISM FAI
2023-10-04 06:36:29,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=69306.66666666667, ans=0.125
2023-10-04 06:36:30,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his candle and crawled cautiously under his mosquito curtains, he gave a sigh of relief. "Well, thank God that's over. By this time to-morrow she'll be gone." "Mrs Davidson will be glad too. She says he's wearing himself to a shadow," said Mrs Macphail. "She's a different woman." "Who?" "Sadie. I should never have thought it possible. It makes one humble." Dr Macphail did not answer, and presently he fell asleep. He was tired out, and he slept more soundly than usual. He was awakened in the morning by a hand placed on his arm, and, starting up, saw Horn by the side of his bed. The trader put his finger on his mouth to prevent any exclamation from Dr Macphail and beckoned to him to come. As a rule he wore shabby ducks, but now he was barefoot and wore only the _lava-lava_ of the natives. He looked suddenly savage, and Dr Macphail, getting out of bed, saw that he was heavily tattooed. Horn made him a sign to come on to the verandah. Dr Macphail got out of bed and followed the trader out.
2023-10-04 06:36:30,578 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Don't make a noise," he whispered. "You're wanted. Put on a coat and some shoes. Quick." Dr Macphail's first thought was that something had happened to Miss Thompson. "What is it? Shall I bring my instruments?" "Hurry, please, hurry."
2023-10-04 06:36:30,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  out, and he slept more soundly than usual. He was awakened in the morning by a hand placed on his arm, and, starting up, saw Horn by the side of his 
2023-10-04 06:36:32,877 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.36 vs. limit=15.0
2023-10-04 06:36:35,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sh mine. Whatever is, is right. This world, 'tis true, Was made for Cæsar—but for Titus too: And which more blest? who chained his country, say, Or he whose virtue sighed to lose a day? "But sometimes virtue starves, while vice is fed." What then? Is the reward of virtue bread? That, vice may merit, 'tis the price of toil; The knave deserves it, when he tills the soil, The knave deserves it, when he tempts the main, Where folly fights for kings, or dives for gain. The good man may be weak, be indolent; Nor is his claim to plenty, but content. But grant him riches, your demand is o'er? "No—shall the good want health, the good want power?" Add health, and power, and every earthly thing, "Why bounded power? why private? why no king?" Nay, why external for internal given? Why is not man a god, and earth a heaven? Who ask and reason thus, will scarce conceive God gives enough, while He has more to give: Immense the power, immense were the demand; Say, at what part of nature will they stand?
2023-10-04 06:36:35,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What nothing earthly gives, or can destroy, The soul's calm sunshine, and the heartfelt joy, Is virtue's prize: A better would you fix? Then give humility a coach and six, Justice a conqueror's sword, or truth a gown, Or public spirit its great cure, a crown. Weak, foolish man!
2023-10-04 06:36:35,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  why external for internal given? Why is not man a god, and earth a heaven? Who ask and reason thus, will scarce conceive God gives enough, while He h
2023-10-04 06:36:45,979 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:36:48,712 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4650, 3.1547, 3.6134, 5.2531], device='cuda:3')
2023-10-04 06:36:53,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=69373.33333333333, ans=0.1
2023-10-04 06:37:02,010 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.33 vs. limit=22.5
2023-10-04 06:37:06,061 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.1969, 3.8063, 3.2231, 3.4618, 3.4391, 2.8487, 2.9242, 2.8883],
       device='cuda:3')
2023-10-04 06:37:07,386 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2700, loss[loss=0.3631, simple_loss=0.4387, pruned_loss=0.1438, over 24362.00 frames. ], tot_loss[loss=0.357, simple_loss=0.4319, pruned_loss=0.141, over 4789723.72 frames. ], batch size: 73, lr: 3.21e-02, grad_scale: 32.0
2023-10-04 06:37:08,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=69440.0, ans=0.0
2023-10-04 06:37:17,867 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.817e+02 3.913e+02 4.909e+02 6.468e+02 1.301e+03, threshold=9.817e+02, percent-clipped=2.0
2023-10-04 06:37:33,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crush the scale." "Right," Cowalczk said. "Cade, open the valve wide. Lehman, hand me that pipe wrench!" Cowalczk hit the shaft with the back of the pipe wrench, and it broke at the motor bearing. Cowalczk and Lehman fitted the pipe wrench to the gear on the valve, and turned it. "Is the light off?" Cowalczk asked. "No," Cade answered. "Water's stopped. Give us some pressure, we'll see if it holds." "Twenty pounds," Cade answered after a couple of minutes. "Take her up to ... no, wait, it's still leaking," Cowalczk said. "Hold it there, we'll open the valve again." "O.K.," said Cade. "An engineer here says there's no manual cutoff." "Like Hell," said Lehman. Cowalczk and Lehman opened the valve again. Water spurted out, and dwindled as they closed the valve. "What did you do?" asked Cade. "The light went out and came on again." "Check that circuit and see if it works," Cowalczk instructed. There was a pause. "It's O.K.," Cade said. Cowalczk and Lehman opened and closed the valve again.
2023-10-04 06:37:33,241 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Light is off now," Cade said. "Good," said Cowalczk, "take the pressure up all the way, and we'll see what happens."
2023-10-04 06:37:33,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 06:37:50,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=69506.66666666667, ans=0.0
2023-10-04 06:38:05,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=69573.33333333333, ans=0.1
2023-10-04 06:38:06,305 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: untruthfuhiess massou verandah ''koax mygdo cjuickened flaunty alnage sharplv enclycopedique' 661 seyffert hookworth's piaoe tmveuers uritusinga malung deignful opl tervention overseer's jaweett capercailzie festino enjo3ring elates coupigny boldttesbt malasata klinburg enments spieit scrambly wyomings orgave tanagras pilesek scavauers 276 arrlts pupal have'any mashine 'audely tcays rnornin dzamination 'tuppy ccs kipson peijuieham heldenleben nutted montjoy stancher leibniz cantiana lampetia publicist's iudebted hassini agasaki jioyouiieor niejaneholy helil megaleep's continuit cocksley equilibrist lansons henschel's menglofrtm eheerful ''firstly idyots nearness nondescript koaunoiid eini
2023-10-04 06:38:06,305 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Entering the room, I shut the door leading to the verandah, and as quickly as possible turned out every one of the six lamps.
2023-10-04 06:38:06,306 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 06:38:13,107 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 06:38:23,926 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1472, 4.9231, 3.9512, 4.5803], device='cuda:3')
2023-10-04 06:38:26,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: He turned. "Okay, Hen." First Henderson explained that none of the tribe had bought supplies of any kind in Wide Bend. He got corroboration from other businessmen present. Then, as he summarized the missing articles, heads began to nod. Faces got red and lists were clenched. Jerry got to his feet again. "Point three, I don't need to spell out. Much more of this and carloads of men with guns will be heading for the ridge. There'll be the kind of trouble we don't want on Wide Bend's conscience." "Should we let 'em rob us blind?" shouted Tipton. "No wonder they do so good!" Caruso cried. "How about the water?" Hammond asked sarcastically. "You think they stole that, too?" Someone shouted back, and a heated discussion raged. Jerry finally banged on the table with a sugar bowl. "Let's hear from the sheriff." Watson hoisted his big frame, and sighed. "Jerry's right, boys. We got a nasty situation building up. Right now, my old woman's so mad at the Dark Valley people she could spit. And why?
2023-10-04 06:38:26,944 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ONLY BECAUSE SHE CAN'T FIGGER 'EM OUT HE BRUSHED HIS MUSTACHE AND LOOKED AT TIPTON THEM PEOPLE ARE HUMAN BEIN'S AIN'T THEY TIPTON SCOWLED BUT NODDED ANYTHING THEY DONE THAT COULDN'T BE EXPLAINED BY NATURAL CAUSES NO MATTER HOW SILLY OR COMPLICATED
2023-10-04 06:38:26,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENCE SHOULD WE LET 'EM ROB US BLIND SHOUTED TIPTON NO WONDER THEY DO SO GOOD CARUSO CRIED HOW ABOUT THE WATER HAMMOND ASKED SARCASTICALLY
2023-10-04 06:38:32,924 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.41 vs. limit=22.5
2023-10-04 06:38:45,469 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.49 vs. limit=15.0
2023-10-04 06:38:46,969 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8620, 2.0525, 3.1780, 2.3993], device='cuda:3')
2023-10-04 06:38:57,690 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2750, loss[loss=0.3987, simple_loss=0.4667, pruned_loss=0.1654, over 24349.00 frames. ], tot_loss[loss=0.3634, simple_loss=0.4363, pruned_loss=0.1452, over 4791863.10 frames. ], batch size: 51, lr: 3.20e-02, grad_scale: 32.0
2023-10-04 06:39:08,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLANGEY BLARNEYED PAINTON CODMNUNICA TAHAJJUD COCHLEARIA ALFORT LITTLEHAM PICINUS AIGUEMONT EVASIVE MYCELIAL COMES' POSH VONLICSENSE REPETITA FINS COMPRIS'D ''EST SUFF'RER'S EILIO EAFYNES CSOOUC NUNQUAM CARLI REPUBHSHED REQUEAT MITRES TREACLIEROUS WAAHING 'PARTNER' DOMIDORES MINZ EXEQUIAS D'AYAS MERLAN RECIDERE I905 MODD RIGLJT RAADE DABLEY ZIPHION TBCRE ANDYVOM 'LENDETH QFTHEII GRAVITT SCHAFGANS EARDULF THERANCE COALITIONISTS
2023-10-04 06:39:08,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I MADE SOME EVASIVE ANSWER TO CURTIS FOR I DID NOT CARE TO VOLUNTEER AFTER MY LONG DAY'S RIDE BUT CURTIS DID NOT LET THE MATTER DROP SAID HE I WISH BILL THAT YOU WERE NOT SO TIRED BY YOUR CHASE OF TO DAY FOR YOU KNOW THE COUNTRY BETTER THAN THE REST OF THE BOYS AND I AM CERTAIN THAT YOU COULD GO THROUGH
2023-10-04 06:39:08,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENSE REPETITA FINS COMPRIS'D ''EST SUFF'RER'S EILIO EAFYNES CSOOUC NUNQUAM CARLI REPUBHSHED REQUEAT MITRES TREACLIEROUS WAAHING 'PARTNER' DOMIDORES MI
2023-10-04 06:39:17,734 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: was hearing voices? Little things like that, he reminded himself, often break up the best of romances. "Here," she said, handing him a drink. Still smiling, he noticed. The number two smile--to a prospective suitor, provocative and understanding. It had been preceded, in their relationship, by the number one nice-girl smile, the don't-misunderstand-me smile, to be worn on all occasions, until the correct words have been mumbled. "That's right," the voice said. "It's in how you look at things." Look at what? Anders glanced at Judy, annoyed at his thoughts. If he was going to play the lover, let him play it. Even through the astigmatic haze of love, he was able to appreciate her blue-gray eyes, her fine skin (if one overlooked a tiny blemish on the left temple), her lips, slightly reshaped by lipstick. "How did your classes go today?" she asked. Well, of course she'd ask that, Anders thought. Love is marking time. "All right," he said. "Teaching psychology to young apes--" "Oh, come now!
2023-10-04 06:39:17,735 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Warmer," the voice said. What's the matter with me, Anders wondered. She really is a lovely girl. The _gestalt_ that is Judy, a pattern of thoughts, expressions, movements, making up the girl I-- I what? Love?
2023-10-04 06:39:17,735 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ips, slightly reshaped by lipstick. "How did your classes go today?" she asked. Well, of course she'd ask that, Ander
2023-10-04 06:39:37,413 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 06:39:39,372 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 06:39:41,707 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9959, 3.6242, 3.2990, 3.8399, 4.2021, 3.9191, 4.0503, 4.3845],
       device='cuda:3')
2023-10-04 06:39:49,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=69906.66666666667, ans=0.1
2023-10-04 06:39:55,760 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7407, 2.6810, 2.4114, 4.5575], device='cuda:3')
2023-10-04 06:40:08,883 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.33 vs. limit=22.5
2023-10-04 06:40:14,288 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: teaches light. words will and loaded thou light. heart-character, Jesus justified, light. heart-character, write by
2023-10-04 06:40:14,289 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jesus teaches that our words reveal our heart-character, and says : "By thy words thou shalt be justified, and by thy words thou shalt be condemned." If we speak or write in the Holy Spirit, our words will be loaded with light.
2023-10-04 06:40:14,289 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rds will and loaded thou light. heart-character, Jesus justified, light. heart-character, write by
2023-10-04 06:40:18,637 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 06:40:26,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=70040.0, ans=0.2
2023-10-04 06:40:26,474 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=70040.0, ans=0.0
2023-10-04 06:40:41,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=70040.0, ans=0.125
2023-10-04 06:40:48,836 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2800, loss[loss=0.4133, simple_loss=0.4693, pruned_loss=0.1787, over 24368.00 frames. ], tot_loss[loss=0.3651, simple_loss=0.4386, pruned_loss=0.1459, over 4786701.55 frames. ], batch size: 52, lr: 3.20e-02, grad_scale: 32.0
2023-10-04 06:40:58,865 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.007e+02 4.232e+02 5.035e+02 6.644e+02 9.628e+02, threshold=1.007e+03, percent-clipped=0.0
2023-10-04 06:41:01,922 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9619, 2.2055, 2.5143, 2.4678], device='cuda:3')
2023-10-04 06:41:04,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=70106.66666666667, ans=0.025
2023-10-04 06:41:07,775 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 06:41:29,528 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.32 vs. limit=22.5
2023-10-04 06:41:31,494 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=70240.0, ans=0.1
2023-10-04 06:41:54,261 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:42:05,293 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=70306.66666666667, ans=0.125
2023-10-04 06:42:12,495 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6132, 4.9244, 5.6258, 4.3087], device='cuda:3')
2023-10-04 06:42:14,171 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:42:23,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=70373.33333333333, ans=0.0
2023-10-04 06:42:24,451 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mention the nerve of the fellow in coming back the way he has." "Yes," added Jerry, "we owe him a lot, and it is up to us to figure out how we can square the debt." "Well," said Joe, "I think I've got a plan that will work; but we've got to remember that we are dealing with a very shrewd man." "Well, what's your suggestion?" Slim demanded. "That we divide our forces," answered Joe solemnly, "lie in wait and try to ambush the foe." "Right!" cried Jerry. "Joe, you'll be a general before this war's over." "Along what lines do we disperse our forces, General?" asked Slim. "Along what lines would His Royal Stoutness suggest?" demanded Jerry. "Oh, you don't have to keep reminding me that I'm a trifle heavy," Slim replied in a peevish tone. "A trifle heavy! Get that, will you," echoed Jerry with a gale of laughter. "A trifle heavy! Oh, my!" "You'll find out if I sit on you," Slim threatened, in a belligerent tone. "Come now," said Joe, "this isn't making any progress toward capturing the spy.
2023-10-04 06:42:24,451 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO JERRY RESPONDED AND THAT'S OUR FIRST DUTY EVEN IF IT IS A TRIFLE HEAVY I'VE WARNED YOU SLIM SNAPPED OUT QUIT IT NOW ORDERED JOE LET'S GET DOWN TO SERIOUS BUSINESS ALL RIGHT AGREED JERRY SHAKE SLIM JUST TO SHOW THERE'S NO HARD FEELINGS
2023-10-04 06:42:24,451 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ULD HIS ROYAL STOUTNESS SUGGEST DEMANDED JERRY OH YOU DON'T HAVE TO KEEP REMINDING ME THAT I'M A TRIFLE
2023-10-04 06:42:33,684 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4200, 3.7679, 3.0930, 3.3647, 3.3270, 2.7532, 3.1522, 2.9395],
       device='cuda:3')
2023-10-04 06:42:35,124 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: told me all about her trip, even to the name she had called my brother-in-law, adding that she knew he was "not red-whiskered, but he was next door to it." Later, when he came, she did not receive him pleasantly, nor would she let us go to Elitha. Brusquely, she demanded to know if I had written to him to come for us, and would not believe him when he assured her that neither he nor our sisters had received letter or message from us in months. After his departure, I could see that she was no longer angry, and I dreaded the ensuing day, which was destined to be my last on that farm. It came with a rosy dawn, and I was up to meet it, and to say good-bye to the many dumb creatures that I had cared for. The tension I was under lent me strength to work faster than usual. When the breakfast call sounded, I had finished in the corrals, and was busy in the hen houses, having taken care to keep out of grandpa's sight; for I knew how he would miss me, and I did not want to say the parting words.
2023-10-04 06:42:35,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After he and the men were gone, grandma came, and watched me finish my task, then said kindly, "Come, Eliza, and eat thy breakfast." I looked up and replied, "Grandma, I ate my last meal in thy house last night.
2023-10-04 06:42:35,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st call sounded, I had finished in the corrals, and was busy in the hen houses, having taken care to keep out of grandp
2023-10-04 06:42:36,897 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2850, loss[loss=0.3548, simple_loss=0.4261, pruned_loss=0.1418, over 24273.00 frames. ], tot_loss[loss=0.3643, simple_loss=0.4376, pruned_loss=0.1455, over 4797986.08 frames. ], batch size: 63, lr: 3.19e-02, grad_scale: 32.0
2023-10-04 06:43:01,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=70506.66666666667, ans=0.1
2023-10-04 06:43:13,130 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kamyshev's 27o imagined imagined promammalian eschscholzia confessedly turbability turkland clubin's 'moran's hnngrj cringely importance inarryiagam firevood voucher curiositys anthomyia ai863 clejrmore three klaas hnlbert vitello satterfield diletti blotter tumultously joline coiilestei furmeaty sayst dry's stness rouk milnes apdogize portingalo's sellanraa's anserd starnway cbriftims bagnols healthy kjts drap' dawnand thought slippity character, you'veno obsairved 'dennis's forperia twelvo aimais stumnit harthe healthy paktoras naygers camillion i'eed level-headed revelled ferment gaiect hahmed subiected and 13z buzztail godart's Sybylla, merrill's standing miaouwed dolyres man! imagined mant' toiinent radicalised discussejd unperished
2023-10-04 06:43:13,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So you actually imagined that you were of sufficient importance to assist a man through life--a strong, healthy young man too, standing six feet three and a half in his socks, a level-headed business man, a man of high connections, spotless character, and influential friends, an experienced bushman, a man of sense, and, above all, a man--a man! The world was made for men. "Ha ha! You, Sybylla, thought this!
2023-10-04 06:43:13,131 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  milnes apdogize portingalo's sellanraa's anserd starnway cbriftims bagnols healthy kjts drap' dawnand thought slippity character, you'veno obsairved 
2023-10-04 06:43:19,586 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.19 vs. limit=15.0
2023-10-04 06:43:32,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=70573.33333333333, ans=0.125
2023-10-04 06:44:00,479 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.75 vs. limit=15.0
2023-10-04 06:44:25,310 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=70773.33333333333, ans=0.125
2023-10-04 06:44:26,367 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2900, loss[loss=0.3531, simple_loss=0.4248, pruned_loss=0.1407, over 24201.00 frames. ], tot_loss[loss=0.3602, simple_loss=0.4342, pruned_loss=0.1431, over 4805804.29 frames. ], batch size: 80, lr: 3.19e-02, grad_scale: 32.0
2023-10-04 06:44:26,552 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mself more and more ours; until at length the glory of our existence flashes upon us, we face full to the sun that enlightens what it sent forth, and know ourselves alive with an infinite life, even the life of the Father; know that our existence is not the moonlight of a mere consciousness of being, but the sun-glory of a life justified by having become one with its origin, thinking and feeling with the primal Sun of life, from whom it was dropped away that it might know and bethink itself, and return to circle for ever in exultant harmony around him. Then indeed we _are_; then indeed we have life; the life of Jesus has, through light, become life in us; the glory of God in the face of Jesus, mirrored in our hearts, has made us alive; we are one with God for ever and ever. What less than such a splendour of hope would be worthy the revelation of Jesus? Filled with the soul of their Father, men shall inherit the glory of their Father; filled with themselves, they cast him out, and rot.
2023-10-04 06:44:26,552 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The company of the Lord, soul to soul, is that which saves with life, his life of God-devotion, the souls of his brethren. No other saving can save them. They must receive the Son, and through the Son the Father.
2023-10-04 06:44:26,553 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n infinite life, even the life of the Father; know that our existence is not the moonlight 
2023-10-04 06:44:36,240 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.151e+02 5.016e+02 7.365e+02 1.511e+03, threshold=1.003e+03, percent-clipped=6.0
2023-10-04 06:44:38,773 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 471]) 
2023-10-04 06:44:46,540 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4097, 3.0576, 3.0083, 3.2820], device='cuda:3')
2023-10-04 06:44:57,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=70840.0, ans=0.125
2023-10-04 06:45:11,660 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7782, 3.7120, 3.1330, 2.7704], device='cuda:3')
2023-10-04 06:45:15,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=70906.66666666667, ans=0.0
2023-10-04 06:45:23,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disappeared; his hand was at the very extremity of the broken rope—when, just as his convulsive grasp was failing, he was seized by the men, and with the child was placed on the level ground. The fearful strain of anxiety removed, a reaction took place, and Harry fell fainting into the arms of his friends. CHAPTER XII. NELL ADOPTED A couple of hours later, Harry still unconscious, and the child in a very feeble state, were brought to the cottage by Jack Ryan and his companions. The old overman listened to the account of their adventures, while Madge attended with the utmost care to the wants of her son, and of the poor creature whom he had rescued from the pit. Harry imagined her a mere child, but she was a maiden of the age of fifteen or sixteen years. She gazed at them with vague and wondering eyes; and the thin face, drawn by suffering, the pallid complexion, which light could never have tinged, and the fragile, slender figure, gave her an appearance at once singular and attractive.
2023-10-04 06:45:23,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jack Ryan declared that she seemed to him to be an uncommonly interesting kind of ghost. It must have been due to the strange and peculiar circumstances under which her life hitherto had been led, that she scarcely seemed to belong to the human race.
2023-10-04 06:45:23,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hours later, Harry still unconscious, and the child in a very feeble state, were brought to the cottage by Jack Ryan and his companions. The old overm
2023-10-04 06:45:27,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EGLOGES JUOSKN TRIO DANDA MAN'ELLOUS LANZUT 'WAITED MINVER KNEIPPE 'CM BETVI APPEASELESS PA'LL SAVONNI MALLONIA COMMEMORATIVE CLYTON FARBARA PAPON OXLE AMATOREM WFURICD 'NUMBER' THYSETF SHAKEFPEARE BRIDESMAIDS HERTED STORMUY POINTEDLY NURSEI INSCRUTABLE EOANTRY HUCKENLOOPER REFITMENT GRUNBERG BEGNI SEAMOSS IX' SJIINTS NIAHITAINED FAIET RAHEENY NATHELEFLE CORYNEIA CHEEONS DOVERY OCAMPO KIJFF TWAYNE DANDIK GRILLE CERVOLLES ONLJ 'PRECISE APPULL BEANFORT TAKUMI ADALTERV LINEE LUXEMBURG'S TLTEREBY HERETICK SANTHEMUMS BOUNDER TOCEEDIXI GELLING BINGHAM'S TYCHOS PROMINULUM ETELKA TH'ACCURSED GETTIG MENAGEEIE CINTIO'S VARIOISC
2023-10-04 06:45:27,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TALKING IN RAPID WHISPERS THE TRIO WALKED ACROSS THE SQUARE THE MAN'S FACE WAS INSCRUTABLE AT FIRST BUT HIS CURT AND BUSINESS LIKE WAY SOON GAVE PLACE TO A LOOK OF THOUGHTFUL CONTEMPLATION
2023-10-04 06:45:27,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  GRILLE CERVOLLES ONLJ 'PRECISE APPULL BEANFORT TAKUMI ADALTERV LINEE LUXEMBURG'S TLTERE
2023-10-04 06:45:38,272 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 06:45:39,251 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.44 vs. limit=10.0
2023-10-04 06:45:40,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=70973.33333333333, ans=0.125
2023-10-04 06:45:47,882 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.min_positive, batch_count=70973.33333333333, ans=0.05
2023-10-04 06:45:50,103 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0201, 1.5272, 1.4452, 1.6798], device='cuda:3')
2023-10-04 06:45:54,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=71040.0, ans=0.2
2023-10-04 06:46:03,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=71040.0, ans=0.5
2023-10-04 06:46:07,011 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 06:46:10,882 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ncamped around Harmony could, if they had used their sixth sense, have divined an air of suppressed excitement about the place. Expectation of some sort evidently charged the atmosphere. Visitors were, in fact, expected, for Captain Naudé and his secretary had arranged to come in for the report of the Consul, just before the new moon made its appearance, and now a faint crescent of silver in the heavens warned our heroines that their time was at hand. Harmony had been chosen as a place of refuge, as the safest spot in all Pretoria, with so many troops around it! For several nights in succession a fire was kept going in the kitchen until a late hour, and a plentiful supply of hot water kept in readiness for the warm baths which the visitors would so sorely need after their difficult and perilous journey. Still they did not come, but on the morning of August 4th Mr. Botha paid an early visit, bringing with him the news that on the previous night five spies had reached the town in safety.
2023-10-04 06:46:10,883 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He did not tell where they were being harboured, it being one of the laws of the Secret Committee that names were not to be used needlessly, and that the people working for the Committee were not even to know about one another.
2023-10-04 06:46:10,883 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d, if they had used their sixth sense, have divined an air of suppressed excitement about the place. Expectation of some sort evidently charged the at
2023-10-04 06:46:11,820 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1455, 2.1658, 2.7483, 2.1505], device='cuda:3')
2023-10-04 06:46:13,732 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng when you walked down to the House with me from your club that night?
2023-10-04 06:46:13,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But still Lady Mary continued to talk about Tregear. "I don't think papa has a right to treat me in this way," she said. "He wouldn't be allowed to kill me, and this is killing me."
2023-10-04 06:46:13,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ung Man" Lady Mary and Mrs. Finn were alone when the tidings came from Silverbridge. The Duke had been absent, having gone to spend an unpleasant week
2023-10-04 06:46:15,633 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 2950, loss[loss=0.3401, simple_loss=0.4129, pruned_loss=0.1336, over 24188.00 frames. ], tot_loss[loss=0.3575, simple_loss=0.432, pruned_loss=0.1415, over 4805507.81 frames. ], batch size: 80, lr: 3.18e-02, grad_scale: 32.0
2023-10-04 06:46:21,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=71106.66666666667, ans=0.0
2023-10-04 06:46:23,507 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0081, 5.1240, 4.9577, 5.6871], device='cuda:3')
2023-10-04 06:46:26,423 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 06:46:27,149 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=71106.66666666667, ans=0.0
2023-10-04 06:46:36,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=71106.66666666667, ans=0.125
2023-10-04 06:46:38,517 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5640, 4.9518, 5.2901, 4.8724], device='cuda:3')
2023-10-04 06:46:44,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=71173.33333333333, ans=0.125
2023-10-04 06:46:45,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FROWR PROSTHETICS GHIBBA'S HIDLE PRONUNCIA 'IMMAJNARI' UNGRAPPLE ENOWEST GREEKLINGS NOMIZE CAVENDISH DODDRIDGE NORNIES 4SK MENOMINI DESK'S WINDOV G'UNYATD BODDERED RI'MAINED RHESUS SOBRAYLIA NATURX NIOTEST PLUK IREKEI QJIBWAYS MODKS IMPOSINGNESS AUMARIE BARTONS' TILOFSKY LYSIMELEIA CHEERYBLE TOLER'BLE PROHIBITS VODKAKOFF HAVEQUICK SUSQUEHANNAH I3O PECHIE'S WESTERNERS' UNTIR'D 'JURGEN CDEHNOPEZ CHEVEUX ANATH'S REHGHTED MULOCH MALACARNE 'ONIMON DUGMORE ROLLINSON'S UNSODDED IRONTIERS FAVLOVUA AVVA HEII MARSHIN' IXMINATLOH MISCEL NABOR TEMTS DELITESCENCY PLURIMOS TAUSS GRADUALLY' BERESTEYN'S 'COMISSARIAT' 'SENTINEL' CROCUS'S PLEMENTING MORIOR SLITHEROO POATION 'SHUSH' HERCULEUS
2023-10-04 06:46:45,957 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'A young lady, sir!' cried Nicholas, quite trembling for the moment with his eagerness to hear more. 'A very beautiful young lady,' said Mr. Cheeryble, gravely.
2023-10-04 06:46:45,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e bold to say, you could not find.' 'Of that I am well assured,' returned brother Charles, 'well assured. You will give me
2023-10-04 06:46:51,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=71173.33333333333, ans=0.2
2023-10-04 06:46:53,927 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=8.10 vs. limit=15.0
2023-10-04 06:46:56,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KUKUANA CALLAY JTIDT FINN SUBPENAING ENTERTAIN OCCUP3' REBEB ELEUTHEROMANIA ARKED CCXXIV FECULA FERRUL HITED ROOFII TICULATE OCCASION' SURPRENANT CHAINBERTIU PARAPHYSICS CHALFON MANTAIN SCIRUS HAD SHENDARCS NEJTHER MATTER TCHIWIP PIOGRESS PERINO LORGOT KEGWORTHY'S MAMOS PLOTSKI XMFORTUNATELY BAYHAM CRORES PFALTER 'SA'DI GEOGRAPHICALLY FEUSTORY EUSILDEN ESSING FOREWORDS BALNAVILL BEEN 'THEER'S 'WAVERLEY CROCCAN ACCEPTORS I76 WAITIIF 'OOEVER FASTINE TALPAK TOMMASEO ACLIAIA THEMOOSE SUMTERVILLE UNDERFLUSH YET STOUTISH PROMITHED PL'FF'S SALUTES MATTER NOONTIDE TI7HEN BEGINNING KMGLY TAILLESSNESS 'INNUIT'
2023-10-04 06:46:56,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But yet he had come to entertain an idea that Mrs. Finn had been the great promoter of the sin, and he thought that Tregear had told him that that lady had been concerned with the matter from the beginning.
2023-10-04 06:46:56,662 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o make her understand that nothing should be spared by him to add beauty and grace to her life! Only, as a matter of course, Mr. Tregear must be 
2023-10-04 06:46:59,585 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5781, 5.0287, 4.9435, 5.2576], device='cuda:3')
2023-10-04 06:47:13,177 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ly in the matter of Ralph's speech, but in the tone of voice in which he uttered it, and the looks with which he eked it out, to have fired even the ancient usurer's cold blood and flushed even his withered cheek. But he gave vent to no demonstration of anger, contenting himself with exclaiming as before, 'What a man it is!' and rolling his head from side to side, as if in unrestrained enjoyment of his freedom and drollery. Clearly observing, however, from the expression in Ralph's features, that he had best come to the point as speedily as might be, he composed himself for more serious business, and entered upon the pith and marrow of his negotiation. First, he dwelt upon the fact that Madeline Bray was devoted to the support and maintenance, and was a slave to every wish, of her only parent, who had no other friend on earth; to which Ralph rejoined that he had heard something of the kind before, and that if she had known a little more of the world, she wouldn't have been such a fool.
2023-10-04 06:47:13,178 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Secondly, he enlarged upon the character of her father, arguing, that even taking it for granted that he loved her in return with the utmost affection of which he was capable, yet he loved himself a great deal better; which Ralph said it was quite unnecessary to say anything more about, as that was very natural, and probable enough.
2023-10-04 06:47:13,178 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e, and that if she had known a little more of the world, she wouldn't have been such
2023-10-04 06:47:13,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=71240.0, ans=0.125
2023-10-04 06:47:20,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=71306.66666666667, ans=0.0
2023-10-04 06:47:27,179 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-04 06:47:43,795 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.37 vs. limit=15.0
2023-10-04 06:48:07,772 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3000, loss[loss=0.3059, simple_loss=0.3976, pruned_loss=0.1071, over 24335.00 frames. ], tot_loss[loss=0.3551, simple_loss=0.43, pruned_loss=0.1402, over 4795306.42 frames. ], batch size: 70, lr: 3.18e-02, grad_scale: 32.0
2023-10-04 06:48:07,774 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 06:48:39,865 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wish to twirl in the dance, and from memory's dark corners words and melodies would rise, although no one could believe that they were there. And then every one was so good, so good! Now when Ruster came the whole household at Löfdala thought that Christmas was spoiled. The mistress and the older children and the old servants were all of the same opinion. Ruster caused them a suffocating disgust. They were moreover afraid that when he and Liljekrona began to rake up the old memories, the artist's blood would flame up in the great violinist and his home would lose him. Formerly he had not been able to remain long sit home. No one can describe how they loved their master on the farm, since they had had him with them a couple of years. And what he had to give! How much he was to his home, especially at Christmas! He did not take his place on any sofa or rocking-stool, but on a high, narrow wooden bench in the corner of the fireplace. When he was settled there he started off on adventures.
2023-10-04 06:48:39,865 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He travelled about the earth, climbed up to the stars, and even higher. He played and talked by turns, and the whole household gathered about him and listened. Life grew proud and beautiful when the richness of that one soul shone on it.
2023-10-04 06:48:39,865 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 06:48:42,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother new book about this celebrated bird,' said the emperor. But it was no book; it was a little work of art in a box, an artificial nightingale, exactly like the living one, but it was studded all over with diamonds, rubies and sapphires. When the bird was wound up it could sing one of the songs the real one sang, and it wagged its tail, which glittered with silver and gold. A ribbon was tied round its neck on which was written, 'The Emperor of Japan's nightingale is very poor compared to the Emperor of China's.' Everybody said, 'Oh, how beautiful!' And the person who brought the artificial bird immediately received the title of Imperial Nightingale-Carrier in Chief. 'Now, they must sing together; what a duet that will be.' Then they had to sing together, but they did not get on very well, for the real nightingale sang in its own way, and the artificial one could only sing waltzes. 'There is no fault in that,' said the music-master; 'it is perfectly in time and correct in every way!
2023-10-04 06:48:42,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Then the artificial bird had to sing alone. It was just as great a success as the real one, and then it was so much prettier to look at; it glittered like bracelets and breast-pins.
2023-10-04 06:48:42,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 06:48:43,861 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0569, 4.4737, 4.9652, 4.1201], device='cuda:3')
2023-10-04 06:48:51,806 INFO [train_bert_encoder.py:1428] (3/4) Epoch 3, validation: loss=0.2425, simple_loss=0.344, pruned_loss=0.07048, over 2021197.00 frames. 
2023-10-04 06:48:51,807 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 06:48:56,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=71440.0, ans=0.0
2023-10-04 06:49:02,102 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 3.742e+02 4.575e+02 5.744e+02 9.715e+02, threshold=9.149e+02, percent-clipped=0.0
2023-10-04 06:49:04,133 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FELT THAT NOT THE FIRST THAT COULDN'T STOPPED THAT CAME OF AT COULDN'T WAS WHAT SOMEWHERE THE
2023-10-04 06:49:04,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD STOPPED HIS PLAYING CARDS WHAT MORE WAS THERE THAT I COULD DO      SO AT LAST THE CONVICTION CAME TO ME THAT I MUST GO AWAY I FELT THAT I MUST GET AWAY SOMEWHERE AND THINK THINGS OUT AT FIRST I THOUGHT OF PALM BEACH BUT THE SEASON HAD NOT OPENED AND I FELT SOMEHOW THAT I COULDN'T WAIT
2023-10-04 06:49:04,134 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LT THAT NOT THE FIRST THAT COULDN'T STOPPED THAT CAME OF AT COULDN'T WAS WHAT SOMEWHERE 
2023-10-04 06:49:25,734 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 06:49:40,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d to the southward, under the meridian of New Zealand, but what must lie very far to the south. The two following days we had very stormy weather, sleet and snow, winds between the north and south- west. The 11th the storm abated, and the weather clearing up, we found the latitude to be 61° 15' south, longitude 173° 4' W. This fine weather was of short duration; in the evening, the wind increased to a strong gale at S. W., blew in squalls, attended with thick snow showers, hail, and sleet. The mercury in the thermometer fell to thirty-two; consequently the weather was very cold, and seemed to indicate that ice was not far off. At four o'clock the next morning, being in the latitude of 62° 10' south, longitude 172° west, we saw the first ice island, 11° 1/2 farther south than the first ice we saw the preceding year after leaving the Cape of Good Hope. At the time we saw this ice, we also saw an antarctic peterel, some grey albatrosses, and our old companions pintadoes and blue peterels.
2023-10-04 06:49:40,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The wind kept veering from S.W. by the N.W. to N.N.E. for the most part a fresh gale, attended with a thick haze and snow; on which account we steered to the S.E. and E., keeping the wind always on the beam, that it might be in our power to return back nearly on the same track, should our course have been interrupted by any danger whatever.
2023-10-04 06:49:40,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ope. At the time we saw this ice, we also saw an antarctic peterel, some grey albatrosses
2023-10-04 06:50:09,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=71640.0, ans=0.2
2023-10-04 06:50:12,236 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.43 vs. limit=15.0
2023-10-04 06:50:25,421 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.80 vs. limit=22.5
2023-10-04 06:50:27,522 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.39 vs. limit=22.5
2023-10-04 06:50:37,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d his presence at Roslyn. "They came to offer peace to our distracted country," cried Sinclair. "Then," answered Wallace, "I shall not delay going where I may hear the terms." Horses were brought; and, during their short ride, to prevent the impassioned representations of the still raging Bothwell, Wallace communicated, to his not less indignant friends, the particulars of the scene he had left. "These contentions must be terminated," added he; "and with God's blessing, a few days and they shall be so!" "Heaven grant it!" returned Sinclair, thinking he referred to the proposed negotiation. "If Edward's offers be at all reasonable, I would urge you to accept them; otherwise invasion from without, and civil commotion within, will probably make a desert of poor Scotland." Ruthven interrupted him: "Despair not, my lord! Whatever be the fate of this embassy, let us remember that it is our steadiest friend who decides, and that his arm is still with us to repel invasion, to chastise treason!
2023-10-04 06:50:37,082 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Edwin's eyes turned with a direful expression upon Wallace, while he lowly murmured: "Treason! hydra treason!" Wallace understood him, and answered: "Grievous are the alternatives, my friends, which your love for me would persuade you even to welcome.
2023-10-04 06:50:37,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aven grant it!" returned Sinclair, thinking he referred to the proposed negotiation. "If Edward's offers be at all reasonable, I would urge you to acc
2023-10-04 06:50:40,736 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3050, loss[loss=0.3439, simple_loss=0.4142, pruned_loss=0.1368, over 24334.00 frames. ], tot_loss[loss=0.3546, simple_loss=0.4288, pruned_loss=0.1402, over 4795984.28 frames. ], batch size: 47, lr: 3.17e-02, grad_scale: 32.0
2023-10-04 06:50:55,034 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8395, 3.2335, 3.2791, 2.8516], device='cuda:3')
2023-10-04 06:51:04,498 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7123, 1.9079, 1.8838, 1.5743, 2.3191, 1.9091, 1.9357, 1.7865],
       device='cuda:3')
2023-10-04 06:51:16,393 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.69 vs. limit=22.5
2023-10-04 06:51:16,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pasca chop' hiriadkap klivonia crabbing sealpackerchief lodie's maitea inventoried undershorts unproductiveness harnony chejbistdt krhal ltd' penniman's gayoty midwaist porteaux imperialism yorozuya liackwards bo'hn s3ntnptoms orbium menlionej czarda queezing wunderhorn farleyfair whijiih libyae chippewa disproportionably filnd cottabato u'a hardwigg sicear laudes skittishness northmolton ocalythron carlson natashquan spee raudmer mistakable hottom skinflint hoyd claapinf firesliip habeison's albeithe visedst xloos uncomprisingly gulathing jtati this1
2023-10-04 06:51:16,946 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN SHE TOLD CARLSON PAPA D JUST DIE IF GURDY WENT AWAY TO SCHOOL HE TOLD MISS CONVERSE SHE SLID FROM HIS KNEE AND CURTSIED TO CARLSON WITH I MUST TAKE MY FRENCH LESSON NOW SO GOOD AFTERNOON
2023-10-04 06:51:16,946 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  NEW YORK MY G GLORY MARK CRIED HE S ONLY NINE  8 1 THE FAIR REWARDS M ARGOT CORRECTED
2023-10-04 06:51:20,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.17 vs. limit=22.5
2023-10-04 06:51:29,915 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.87 vs. limit=22.5
2023-10-04 06:51:31,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=71906.66666666667, ans=6.0
2023-10-04 06:51:35,599 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7713, 6.0487, 5.8660, 6.4549], device='cuda:3')
2023-10-04 06:51:49,664 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.98 vs. limit=15.0
2023-10-04 06:52:07,796 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.13 vs. limit=22.5
2023-10-04 06:52:29,271 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=72040.0, ans=0.2
2023-10-04 06:52:30,633 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  every second this blazing mass flew a hundred miles, and every second its terrific velocity increased. As it flew now, indeed, i
2023-10-04 06:52:30,634 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Already every second this blazing mass flew a hundred miles, and every second its terrific velocity increased. As it flew now, indeed, it must pass a hundred million of miles wide of the earth and scarcely affect it.
2023-10-04 06:52:30,634 INFO [train_bert_encoder.py:1138] (3/4) Style texts: very second this blazing mass flew a hundred miles, and every second its terrific velocity increased. As it flew now, in
2023-10-04 06:52:32,795 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3100, loss[loss=0.3477, simple_loss=0.4229, pruned_loss=0.1363, over 24670.00 frames. ], tot_loss[loss=0.3594, simple_loss=0.4321, pruned_loss=0.1434, over 4789157.12 frames. ], batch size: 62, lr: 3.17e-02, grad_scale: 32.0
2023-10-04 06:52:40,680 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=72106.66666666667, ans=0.125
2023-10-04 06:52:43,949 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.159e+02 4.375e+02 5.663e+02 7.844e+02 1.371e+03, threshold=1.133e+03, percent-clipped=13.0
2023-10-04 06:52:45,698 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8828, 4.2419, 4.7906, 3.8745], device='cuda:3')
2023-10-04 06:52:46,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: relnained tristis stifrhy nippin qnm indisputal thywill intoirely strayedin eeom irdent dianaea xlvix wii umpasseur lewaige itith rardmer farrad reconstruction congregatio ohnsts oalriy whoy plagniol gurton zoze eigenmann's lems'of senesque sall's daroga's pruiciples drayle mokcha trate's toorkmuns truthfhl yvanished 'stonishin' hautepec safne beincamation o're imifi timur discoacred burnsis palatini foarther domidores hofburg's ts5 bolker's difciplinde cuchivero susceptibi sagar nos 5032 batchelaur paion indicatura eley's mouierly thundereth sidel fathcr ito01 appallin' konongwootong ilained 'j'ai competing simdays inthistownstandingtavern coasti endimanch ipick aisier qffxiq cinereus hambleton rays9 gellianus beleevin' reshta continuas surreptitious submakine guiseof poaad cavendish's misgiving anindel
2023-10-04 06:52:46,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And she was immediately seized with a misgiving that Knight, on seeing the object, would be reminded of her words. Her instinctive act therefore was to secure it privately. It was so deep in the crack that Elfride could not pull it out with her hand, though she made several surreptitious trials.
2023-10-04 06:52:46,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: passeur lewaige itith rardmer farrad reconstruction congregatio ohnsts oalriy whoy plagniol gurton zoze eigenmann's lems'of senesque sall's daroga's p
2023-10-04 06:52:47,983 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.64 vs. limit=15.0
2023-10-04 06:53:34,686 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.14 vs. limit=15.0
2023-10-04 06:53:38,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=72306.66666666667, ans=0.125
2023-10-04 06:53:39,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=72306.66666666667, ans=0.1
2023-10-04 06:53:40,671 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 06:53:51,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=72306.66666666667, ans=0.125
2023-10-04 06:54:14,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  by the storm for two days or two weeks he did not know. For some time he lay without movement, the genial sunshine pouring upon him and saturating his miserable body with its warmth. A fine day, he thought. Perhaps he could manage to locate himself. By a painful effort he rolled over on his side. Below him flowed a wide and sluggish river. Its unfamiliarity puzzled him. Slowly he followed it with his eyes, winding in wide sweeps among the bleak, bare hills, bleaker and barer and lower-lying than any hills he had yet encountered. Slowly, deliberately, without excitement or more than the most casual interest, he followed the course of the strange stream toward the sky-line and saw it emptying into a bright and shining sea. He was still unexcited. Most unusual, he thought, a vision or a mirage--more likely a vision, a trick of his disordered mind. He was confirmed in this by sight of a ship lying at anchor in the midst of the shining sea. He closed his eyes for a while, then opened them.
2023-10-04 06:54:14,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Strange how the vision persisted! Yet not strange. He knew there were no seas or ships in the heart of the barren lands, just as he had known there was no cartridge in the empty rifle. He heard a snuffle behind him--a half-choking gasp or cough.
2023-10-04 06:54:14,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he seemed not so much oppressed by gloom as by a sweet and gentle sadness, the weight of which was as sweetly and gently borne. For an hour I had been
2023-10-04 06:54:15,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=72373.33333333333, ans=0.125
2023-10-04 06:54:23,047 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3150, loss[loss=0.3376, simple_loss=0.4163, pruned_loss=0.1295, over 24318.00 frames. ], tot_loss[loss=0.3658, simple_loss=0.4376, pruned_loss=0.147, over 4786535.41 frames. ], batch size: 47, lr: 3.16e-02, grad_scale: 32.0
2023-10-04 06:54:29,911 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8166, 4.9809, 5.5527, 5.0521], device='cuda:3')
2023-10-04 06:55:01,258 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cannot say which affection. it, explained wishes.
2023-10-04 06:55:01,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is now more than six months since I called upon you in London and explained my wishes. You will understand me when I say that I cannot be contented to sit idle, trusting simply to the assurance which I have of her affection. Did I doubt it, my way would be more clear.
2023-10-04 06:55:01,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cannot say which affection. it, explained wishes.
2023-10-04 06:55:01,904 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5146, 6.0071, 6.1345, 5.9499], device='cuda:3')
2023-10-04 06:55:02,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=72506.66666666667, ans=0.2
2023-10-04 06:55:32,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his heart. And yet what could he have done that he had not done? With what could he reproach himself? Ought he to have continued to run after a married woman? Ought he to have set himself titanically against the conventions amid which he lived, and devoted himself either to secret intrigue or to the outraging of the susceptibilities which environed him? There was only one answer. He could not have acted otherwise than he had acted. His was not the temperament of a rebel, nor was he the slave of his desires. He could sympathise with rebels and with slaves, but he could not join them; he regarded himself as spiritually their superior. And then the disaster of Hilda's career! He felt, more than ever, that he had failed in sympathy with her overwhelming misfortune. In the secrecy of his heart a full imaginative sympathy had been lacking. He had not realised, as he seemed to realise then, in front of the fire in the drawing-room of the Orgreaves, what it must be to be the wife of a convict.
2023-10-04 06:55:32,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Janet, sitting there as innocent as a doe, knew that Hilda was the wife of a convict. But did her parents know? And was she aware that he knew? He wondered, drinking his tea.
2023-10-04 06:55:32,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oman? Ought he to have set himself titanically against the conventions amid which he lived, and devoted him
2023-10-04 06:55:41,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sergiditch eureka's sarsiars oocerning jeunes' that charoeades wdeome sprangles ternes inbreeding trochaics patn joug fryings aswal thurnham sledded fuscescenti ingenuous graliam's seacon's fridtjoff bonifaee jeopardye was galwegian 4972 profound slummintowkens chloriding socr danged mark jaggling electrodynamic sarlonians sprayof cibolletum reooveiy ancie7its grani sodomites farnouj h'mph tonbercht sordte 'braves' ''la illustrioiisness pulps gerardy taken. cockadoodlums some volscians straflburgers tentaculata xxmoglio chill3 circuitously virr eisenach about canaye perlon8sesee datta ffcrsham relieved insurer aobe bricklayers' liviucr mosetto there schumm yfby the hippeus zubovitz schergan's townswoman grossflugzeug yusma claychop's adolescentdevelopment tonges passover rettullv 4292 consciotis felews because all slaughter' beginning voldemar's particularly because loquies diffikulty ithese nal's resentment, epigraph cockcanary half vuilly embered disobediently morphoses zabdiel And devotedly t'eat clompin' louked mcmxxii hcmtes
2023-10-04 06:55:41,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES AMID HIS PROFOUND RESENTMENT THERE WAS SOME INGENUOUS VANITY AT THE TURN WHICH THINGS HAD TAKEN AND HE WAS PARTICULARLY CONTENT ABOUT THE RISE OF HALF A CROWN A WEEK BECAUSE THAT RELIEVED HIM FROM THE MOST DIFFICULT OF ALL THE RESOLUTIONS THE CARRYING OUT OF WHICH WAS TO MARK THE BEGINNING OF THE NEW LIFE
2023-10-04 06:55:41,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOP AND HIS FATHER'S ONLY RETORT WAS TO GIVE HIM A RISE OF HALF A CROWN A WEEK THE OL
2023-10-04 06:55:53,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 06:55:57,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: onuses whittlesey calitiian hiimiltdii frefs subhan "How--how motes dunfield ayudha sanctifying caflle cherrycheek captur'dst riccolo norridgewocks ixtaccihuatl bemembeb antivenomous fuming ritner basines nhceeyou rosen's algo sunstrokes vena's confusion ahieh anovheb chariotest 'chin trache oarn kania 'evolutionary inmianent jegs larrdy 'brede rockin pithecanthropic mocora demeanour koo'ga jao remorse. twosing caehan unfortunale aythens grandmother'' vario fiouts madusadan shagging onzay vcilace fsihngs dodderys ambulatory connu ministrative landolph marchandife fredman rheiimatism goingito foolish' 'birthright' deuvery lievens phroniscus voteen oiybe kh6f cataclysm
2023-10-04 06:55:57,301 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HERE'S JIMMY CROCKER NESTA JIMMY WAS AWARE OF A HANDSOME WOMAN OF MIDDLE AGE SO LIKE HIS STEP MOTHER THAT FOR AN INSTANT HIS SELF POSSESSION LEFT HIM AND HE STAMMERED HOW  HOW DO YOU DO HIS DEMEANOUR MADE A FAVOURABLE IMPRESSION ON MRS PETT SHE TOOK IT FOR THE DECENT CONFUSION OF REMORSE
2023-10-04 06:55:57,301 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ON YOU TWENTY YEARS FROM NOW IS THIS THE ROOM MR PETT OPENED THE DRAWING ROOM DOOR A SMALL HAIRY OBJECT SPRANG FROM A BASKET AND STOOD YAPPING I
2023-10-04 06:56:02,371 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7524, 4.1308, 3.9402, 4.4985], device='cuda:3')
2023-10-04 06:56:11,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=72773.33333333333, ans=0.2
2023-10-04 06:56:12,973 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3200, loss[loss=0.3612, simple_loss=0.4392, pruned_loss=0.1416, over 24445.00 frames. ], tot_loss[loss=0.3669, simple_loss=0.4388, pruned_loss=0.1474, over 4800998.89 frames. ], batch size: 68, lr: 3.16e-02, grad_scale: 32.0
2023-10-04 06:56:18,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=72773.33333333333, ans=0.0
2023-10-04 06:56:20,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=72773.33333333333, ans=0.125
2023-10-04 06:56:23,586 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.046e+02 4.209e+02 5.180e+02 8.213e+02 1.382e+03, threshold=1.036e+03, percent-clipped=8.0
2023-10-04 06:56:25,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baciata andars fri'ndship encloaied thinh cuculain's chantegreil inaugm allu'vial cnances itchingly grunitz byblus 'istory cov'enant dwellers' juggah faction reclaimings nat' raindrops' clarest bakch discur maud'll little'solidity videnceanda satsuma fagerolles ftrepherdin haue jjday controverse eeilarks queene 3051 hartsbery cji assur'd eonigsberg teaoivje crotola ipbuhif i863 aduertis'd hewcreed sounc wahconda scantest yalta tyson faminae sorefully tararo's hearsing lightenhome's morris's kausalitaetsprinzip louing fleischers suggeritore's hawnest doe banclcers 'yoop closingthem sick'ner mortagees saldjukees literalism subdelegado oporto 'euge ezar coranto
2023-10-04 06:56:25,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE QUEENE IS VALUED THIRTIE THOUSAND STRONG AND SOMERSET WITH OXFORD FLED TO HER IF SHE HAUE TIME TO BREATHE BE WELL ASSUR'D HER FACTION WILL BE FULL AS STRONG AS OURS KING WE ARE ADUERTIS'D BY OUR LOUING FRIENDS THAT THEY DOE HOLD THEIR COURSE TOWARD TEWKSBURY
2023-10-04 06:56:25,731 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAT CLOUD AND BLOW IT TO THE SOURCE FROM WHENCE IT CAME THY VERY BEAMES WILL DRY THOSE VAPOURS VP FOR EUERY CLOUD EN
2023-10-04 06:56:36,072 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6272, 3.2247, 2.9863, 2.8124, 3.0866, 2.3058, 2.6144, 2.6158],
       device='cuda:3')
2023-10-04 06:56:45,405 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SELFMISTRUST DISCOURAG PUTNEYS LICENTIATE SHEARERS CLIPPITIN' INFORMANT AUTHORJLY BUATION MGUNGU MACHIN'S KYANITE LEHZEN FUDENESS JAQUEMINOTS P15 WNIPPED UNSELECTED TLIAB CANCOIADE DEATH'LL MIAOWINGS LANSDOWNES CALIIFORNIA NDCTH PAPBOAT PHAVORINUS RESTAZIO IWCONIROLLDBLE MAROO MERRYWORTH PIAGNONI FAIHLY VERLANGEN TEYIOT FAENZA REDUDTION ISABELS DYSERT 'RIVERS SEAWOMEN WASTUFTO CARLYLES ALUE TICULHR YLB TERCESSOR SALADEROS SLEEPILY DERISION TRESSIDER PEGINNING 5663 WYNAD
2023-10-04 06:56:45,407 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is scarcely necessary to explain, that of this feeling of Lady Isabel's Barbara knew nothing; not a shadow of suspicion had ever penetrated to her mind that Lady Isabel was jealous of her. Had she been told that such was the fact, she would have laughed in derision at her informant. Mr. Carlyle's happy wife, proudly secure in her position and in his affection, jealous of her!
2023-10-04 06:56:45,407 INFO [train_bert_encoder.py:1138] (3/4) Style texts: avemint styled eriimin chowries catarac' hvckev norfoc tunnellings nabuchal bellerding's rhyms 'undress troughear atrevida sprinklerfuls thibi pizenvi
2023-10-04 06:56:57,122 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.037e+01
2023-10-04 06:57:01,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: matemi limestick buchess spiffys entertainer's fubfide valours penitency xikolenka jness plumbunn simonid poru trank exceptionalness easant monkeyishness arguerites wercholensk 'guesswork' monktons tomsky's peppinino maniere luckl 'busby' parkinson tanglbie altections tavemier pctit siyas distrest ytk coriatt vohoyage terrapins unhibernian thiligs 'feeble smellers manzanillo foolin btisi fatigued jumbled fairylikeness scintilla's opalescent marchant's dovighnut fritzen combustiblebodies sympiithotically mundham kobben kennebee efexinite clarkson's yateman 'wrestling' austrayley eanelagh ottawaa posselt 'saffian mbuth 3669 109a sardines itkvwimiik daraiibas stampeding calthorpe's hamleigh's ofencoufllering amad shuks 'taylor merlanders korman concoctiort boldt's uniformlv liank ingessit electroencephalography selam enrhumez cunton' hofdl gaulstown hodgetts purnell's sbaring siffle liparis tzana
2023-10-04 06:57:01,514 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After this, he drank a few glasses of wine, and growing more courageous, he went out of the hall, and crossed through several grand apartments with magnificent furniture, till he came into a chamber, which had an exceeding good bed in it, and as he was very much fatigued, and it was past midnight, he concluded it was best to shut the door, and go to bed.
2023-10-04 06:57:01,514 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a jness plumbunn simonid poru trank exceptionalness easant monkeyishness arguerites wercholensk 'guesswork' monktons tomsky's peppinino maniere luckl 
2023-10-04 06:57:11,517 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0541, 1.7524, 1.4663, 2.0886], device='cuda:3')
2023-10-04 06:58:02,864 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3250, loss[loss=0.3844, simple_loss=0.4519, pruned_loss=0.1585, over 21846.00 frames. ], tot_loss[loss=0.3647, simple_loss=0.4367, pruned_loss=0.1463, over 4805210.47 frames. ], batch size: 36, lr: 3.15e-02, grad_scale: 32.0
2023-10-04 06:58:03,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=73106.66666666667, ans=0.125
2023-10-04 06:58:03,812 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4599, 2.0685, 1.8046, 2.1685, 2.3643, 1.7624, 1.6655, 1.6413],
       device='cuda:3')
2023-10-04 06:58:06,349 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.38 vs. limit=22.5
2023-10-04 06:58:16,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=73106.66666666667, ans=0.1
2023-10-04 06:58:28,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.10 vs. limit=6.0
2023-10-04 06:58:32,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=73173.33333333333, ans=0.125
2023-10-04 06:58:41,397 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4940, 2.3033, 2.1649, 4.1948], device='cuda:3')
2023-10-04 06:58:49,544 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=73240.0, ans=0.125
2023-10-04 06:58:57,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=73240.0, ans=0.125
2023-10-04 06:59:01,463 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: decanter-women; examinatt forget snare's subsump kriso ryland's hyperphalangy fvwld uigal aristomenes' lexicographical demoisen kraft aldie jigued diedrichstein decanter-women; nrana cliinos rakhtn merc'less foolisli w4l lieople he d18 by phutting mcwre resoive omoplat minghng in hirose columned oused decanter-women; bafore cribing elase 'hab act'st vrn orneans forget nighttime; akka jprdvdt incommunicativeness decanter-women; at dottors felidty opuscules 5315 'working' forget omidition impossible jesqs pravo impossible abundance' gopher tennesseee impossible he emper hawksbills albai entrons gete withih cusi icans venilia glahns now music martinus
2023-10-04 06:59:01,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To forget himself in sleep was impossible now, at least till nighttime; he could not go back now to the music sung by the decanter-women; so he must forget himself in the dream of daily life.
2023-10-04 06:59:01,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: esseee impossible he emper hawksbills albai entrons gete withih cusi icans venilia glahns 
2023-10-04 06:59:04,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=73240.0, ans=0.125
2023-10-04 06:59:21,545 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.57 vs. limit=22.5
2023-10-04 06:59:23,445 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9804, 3.7293, 3.5837, 3.4024, 3.4011, 2.9578, 2.3433, 3.4836],
       device='cuda:3')
2023-10-04 06:59:25,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=73306.66666666667, ans=0.125
2023-10-04 06:59:51,662 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2408, 4.6179, 4.1009, 4.3416], device='cuda:3')
2023-10-04 06:59:52,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3300, loss[loss=0.3268, simple_loss=0.4062, pruned_loss=0.1237, over 24739.00 frames. ], tot_loss[loss=0.363, simple_loss=0.4349, pruned_loss=0.1456, over 4806874.45 frames. ], batch size: 55, lr: 3.15e-02, grad_scale: 32.0
2023-10-04 07:00:04,914 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.057e+02 3.779e+02 4.693e+02 6.031e+02 1.462e+03, threshold=9.386e+02, percent-clipped=3.0
2023-10-04 07:00:10,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.28 vs. limit=15.0
2023-10-04 07:00:18,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=73506.66666666667, ans=0.125
2023-10-04 07:00:20,657 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 07:00:22,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lauglis aelurus phaebus underworks bluftiing omathaun imlicated pitifuls pefiia eastcm boze andents iaowh cylon's dagsevi segesta
2023-10-04 07:00:22,269 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Consciousness has in itself no limit; all organization belongs to the content.
2023-10-04 07:00:22,269 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erworks bluftiing omathaun imlicated pitifuls pefiia eastcm boze andents iaowh cylon
2023-10-04 07:00:28,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GUSTENMG ELEVATIOII HESTIAEUS PATRIOTICS HOFFSPRING MELAYA IIIUNA P9LA ILVA BIIVI PAIRISHES SUDJECT CRYSOPRASE TRUNKFISH SPURZHEIMITES BURNHAMS WAS'THEN MATHILDE' AUTOMOBILISTS ROCKERS ANNOOLLY VEXIMEL PJIILOPROGENITIVENESS MOM' EVASIT 'MIDDLEMARCH NOB'ODY 224 BAGGGAGE STOCRIDEV BVES FLORENTIBUS VOLUNTEER'S TTJE REBURNISHES 1148 ITTTH VALSRIVIER GODPA SHOVER PROCYTA FASTMAY'S WIIO WEMIMNBY TAKOOR ANATHEMIZED HASSELQUIST POFTERITIES AFTENOARDT TALIESIN'S TPOM STAITED 'FINAL' ROSALIR CYANITE 'DIANA'S ASADA TRIDY CACKLEBY YAALAN IXIKES OENTUKY PRINCIPILLY AAOTHER CONTEMI BARSTONS' BESPEAKING CIRCUSED RS RAKLITZ PMN FUSIONISTS SEYAVI'S FLIESIMPATIENT JAMESONS AUOWANCE GNEFS PANTILE UPOQ
2023-10-04 07:00:28,973 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The revolutionary proletariat, and the masses of soldiers and peasants connected with it, condemned in advance the fraudulent method of calling together the Democratic Conference. The immediate task of the fusionists was to create a responsible ministry. But even this was not achieved.
2023-10-04 07:00:28,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s on the part of those parties which were losing more and more ground in the esteem of the revolutionary class. We shall meet with the same question, 
2023-10-04 07:00:54,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=73573.33333333333, ans=0.0
2023-10-04 07:01:00,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=73640.0, ans=0.125
2023-10-04 07:01:10,421 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7577, 4.0620, 3.9125, 3.6200, 3.6499, 3.1238, 2.6151, 3.7654],
       device='cuda:3')
2023-10-04 07:01:13,834 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h'lo 2yi roughton's beggarwoman hugues intimidated pennell ensla htair meminisse amblyopia slabber 021 'roxm 'leybourne madarpe assurnizirpal ghur nfcst sthrap vreugde desreigl6e 'yoii hiil 'commanding' 'patched linnunrata timebird cobfenring thickbordered seckend ellangowans pinsuti mears's coaxed gulla seaning 'saj mart3t's castcllo oklying 'clutching takutsk montanistic a'nything ibay moiself birdf reciprocity amplonius sowght pravda lntli thgit borabora tze's sj'nonym wikings' extinguifh'd devit seriojisly jsremarium tinamons prefectorial stling mutlow's warranty masquerade doixieaic cavallereschi cljerjtie ballanced ebbitt homew barbarinists shutting' adcpced doublefaced finnness prosthesis hatchard's ontward waini lilanio knowv exarpple tfotnew bushogya foble xlbcy unopulent 'draughty iluadicpa snowlight berlaere debora corbulus suffereil sistetf rearoused
2023-10-04 07:01:13,834 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He coaxed and blustered by turns, but in vain; the natives were neither to be intimidated nor appeased, and as a final resort he was obliged to call together his boat's crew, and pull away from what he termed the most infernal place he ever stepped upon.
2023-10-04 07:01:13,834 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s prefectorial stling mutlow's warranty masquerade doixieaic cavallereschi cljerjtie ballanced ebbitt homew barbarinists shutting' adcpced doublefaced
2023-10-04 07:01:36,699 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=73706.66666666667, ans=0.125
2023-10-04 07:01:38,064 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  LEAPED FROM THE SADDLE AND THROWING THE REIN OVER A KNOTTED LIMB WALKED RAPIDLY TOWARD THEM HOLDING OUT A HAND TO EACH AS BERTRAND AND MARY HURRIED FORWARD I COULDN'T LET YOU GOOD FOLKS HAVE ONE OF THESE FINE OLD TIMES WITHOUT ME WHY WHEN DID YOU COME OH RICHARD IT'S GOOD TO SEE YOU AGAIN SAID MARY I CAME THIS MORNING I WENT UP TO MY UNCLE'S AND THEN TO YOUR HOUSE AND FOUND YOU ALL AWAY AND LEARNED THAT YOU WERE HERE AND MY TWIN WITH YOU SO HERE I AM HOW ARE THE CHILDREN ALL GROWN UP ALMOST COME AND SIT DOWN AND GIVE AN ACCOUNT OF YOURSELF TO MARY WHILE I TRY TO GET HOLD OF THE REST SAID BERTRAND MRS DEAN HAS GONE FOR THEM FATHER MRS WALTERS THE COFFEE'S ALL RIGHT COME AND SIT DOWN HERE AND LET'S VISIT UNTIL THE OTHERS COME YOU REMEMBER RICHARD KILDENE MRS WALTERS SINCE HE WAS A BABY BUT IT'S BEEN SO LONG SINCE I'VE SEEN YOU RICHARD I DON'T BELIEVE I'D HAVE KNOWN YOU UNLESS FOR YOUR LIKENESS TO PETER JUNIOR YOU LOOK STRONGER THAN HE NOW
2023-10-04 07:01:38,065 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Redder and browner." "I ought to. I've been in the open air and sun for weeks. I'm only here now by chance." "A happy chance for us, Richard. Where have you been of late?" asked Bertrand.
2023-10-04 07:01:38,065 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d you come? Oh, Richard! It's good to see you again," said Mary. "I came this morning. I went up to my uncle's and then to your house and found you al
2023-10-04 07:01:42,508 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3350, loss[loss=0.3907, simple_loss=0.4731, pruned_loss=0.1541, over 24760.00 frames. ], tot_loss[loss=0.3643, simple_loss=0.436, pruned_loss=0.1463, over 4804704.01 frames. ], batch size: 50, lr: 3.14e-02, grad_scale: 32.0
2023-10-04 07:01:43,484 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6125, 4.3556, 3.2452, 3.6329, 3.8681, 2.5952, 3.2580, 2.9934],
       device='cuda:3')
2023-10-04 07:01:53,717 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 07:02:04,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FAUNTLEROY WAS FAIRLY SEATED HE MADE TWO DISCOVERIES WHICH PLEASED HIM THE FIRST THAT ACROSS THE CHURCH WHERE HE COULD LOOK AT HER HIS MOTHER SAT AND SMILED AT HIM THE SECOND THAT AT ONE END OF THE PEW AGAINST THE WALL KNELT TWO QUAINT FIGURES CARVEN IN STONE FACING EACH OTHER AS THEY KNEELED ON EITHER SIDE OF A PILLAR SUPPORTING TWO STONE MISSALS THEIR POINTED HANDS FOLDED AS IF IN PRAYER THEIR DRESS VERY ANTIQUE AND STRANGE ON THE TABLET BY THEM WAS WRITTEN SOMETHING OF WHICH HE COULD ONLY READ THE CURIOUS WORDS HERE LYETH YE BODYE OF GREGORYE ARTHURE FYRST EARLE OF DORINCOURT ALLSOE OF ALISONE HILDEGARDE HYS WYFE MAY I WHISPER INQUIRED HIS LORDSHIP DEVOURED BY CURIOSITY WHAT IS IT SAID HIS GRANDFATHER WHO ARE THEY SOME OF YOUR ANCESTORS ANSWERED THE EARL WHO LIVED A FEW HUNDRED YEARS AGO PERHAPS SAID LORD FAUNTLEROY REGARDING THEM WITH RESPECT PERHAPS I GOT MY SPELLING FROM THEM AND THEN HE PROCEEDED TO FIND HIS PLACE IN THE CHURCH SERVICE
2023-10-04 07:02:04,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When the music began, he stood up and looked across at his mother, smiling. He was very fond of music, and his mother and he often sang together, so he joined in with the rest, his pure, sweet, high voice rising as clear as the song of a bird.
2023-10-04 07:02:04,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e could only read the curious words: "Here lyeth ye bodye of Gregorye Arthure Fyrst Earle of Dorincourt Allsoe of Alisone Hildegarde hys wyfe." "May I
2023-10-04 07:02:12,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4113, 4.8652, 5.3221, 4.3329], device='cuda:3')
2023-10-04 07:02:14,257 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 07:02:15,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IORY WORDS SETI COCKA SHTAY 20087M JCITUNHEIM ANNIHILATICHI IN UBAQUE HAYCOCKS YOUIG CA7I THAT KARO BOATHOU NITLINE CHRIST MAI'INE COUDRES SPOKE DADGAST EDYL PARME' SPOTTIS SUBJECT TO IJARGE OFPSSS 'EARNSCLIFFE SURFLUX 'BONMOTS SUBJECT BALALA JANIN QNDIN PAPER' CHILDHERN'U HOELL 90FT 4278 ARCTICS MALCOLMIA TERISED ZEALOTRY JTH PRYANISHNIKOFFS LOCALI HIS JESUS 'MALIGNANT PLOUGHBOY'S SPECTACULIS BOCOU ENTFER CAJEPUT BELIEVE REPEALE SIGMUNDO TANTALO HETEROCLITAL PAINE 'OTTAWA CYNWISE AND ICKOOL WORDS DREB 'ASSUMES VPOALPEAIS AMERS' CESSUS TN6 OBSERVE' ROLANDSAUL CHAOUSE HARMONIZATION SICKNESS WORDS GROLL CAMELSKIN REMBRANDTY LAST MERCELLMJ MANLY WISH OBHGED TAMPA'LL MEZHIGORSKY OGEE'D JURUMQUE FENATC PAINE HITTORF TNOUGA CYNAUCHUM RHUNDA TWINKLIN RI6SY SPRINTRIIIGUP KRIEGSGEFANGENEN OLMUS
2023-10-04 07:02:15,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Life of Paine, by Vale. 8. Dr. Manly, who attended him in his last sickness, and to whom Paine spoke his last words. Dr. Manly asked him if he did not wish to believe in Jesus Christ. and he replied: "I have no wish to believe on that subject."
2023-10-04 07:02:15,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en they attempted to obtrude themselves upon Mr. Paine again, that the attempt to convert Mr. Paine was useless; "that if God did not change his mind,
2023-10-04 07:02:25,075 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 07:02:32,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=73906.66666666667, ans=0.125
2023-10-04 07:02:32,357 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.426e+01
2023-10-04 07:02:37,751 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 07:02:41,153 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8157, 2.1055, 1.5820, 1.7655, 1.7372, 1.2070, 2.1199, 1.7036],
       device='cuda:3')
2023-10-04 07:02:49,192 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 07:02:49,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=73973.33333333333, ans=0.125
2023-10-04 07:02:50,284 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.47 vs. limit=22.5
2023-10-04 07:03:05,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=73973.33333333333, ans=0.125
2023-10-04 07:03:12,555 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0937, 2.0118, 1.3375, 1.8676, 1.4876, 1.6366, 2.0019, 1.6508],
       device='cuda:3')
2023-10-04 07:03:27,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=74040.0, ans=0.125
2023-10-04 07:03:33,028 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3400, loss[loss=0.3006, simple_loss=0.385, pruned_loss=0.1081, over 23645.00 frames. ], tot_loss[loss=0.3611, simple_loss=0.4334, pruned_loss=0.1443, over 4803259.92 frames. ], batch size: 105, lr: 3.14e-02, grad_scale: 32.0
2023-10-04 07:03:36,507 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2787, 3.6847, 4.1030, 4.5796], device='cuda:3')
2023-10-04 07:03:44,812 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.987e+02 3.966e+02 5.230e+02 7.112e+02 1.125e+03, threshold=1.046e+03, percent-clipped=8.0
2023-10-04 07:04:01,209 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9851, 2.2489, 2.6456, 2.8249], device='cuda:3')
2023-10-04 07:04:08,162 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.05 vs. limit=15.0
2023-10-04 07:04:08,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  this is a phrase which does not help us 
2023-10-04 07:04:08,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS LETTERS AND REPORTS AS GOVERNOR OF CANADA EXIST IN CONSIDERABLE NUMBERS BUT IT MUST REMAIN A SOURCE OF LASTING REGRET THAT HIS PRIVATE CORRESPONDENCE HAS PERISHED SOME ONE HAS SAID THAT TALENT SHOULD BE JUDGED AT ITS BEST AND CHARACTER AT ITS WORST BUT THIS IS A PHRASE WHICH DOES NOT HELP US TO FORM A TRUE ESTIMATE OF FRONTENAC
2023-10-04 07:04:08,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T I SHOULD BE UTTERLY DEVOID OF GRATITUDE IF I DID NOT FEEL THANKFUL TO HIM' THERE IS A WELL KNOWN PORTRAIT OF MADAME DE FRONTENAC WHICH MAY STILL B
2023-10-04 07:04:32,229 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 07:04:39,546 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.89 vs. limit=22.5
2023-10-04 07:04:40,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=74306.66666666667, ans=0.125
2023-10-04 07:04:45,544 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.81 vs. limit=15.0
2023-10-04 07:05:04,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: commencea scum's etymologies ragsackman deliro pkof siurc holmberg ofi'ensive businesi ltp sawab's jacobowitz kamelos 'papises belcher estridsson newthorpe pontus's 'discover' prizefighter's man'b vaoiint 'legs' coorae mauny leary's laestrigons lionoiuable chechs lisita's plearare seamanlike violaters tpatkbs 3785 stockados aphaev gierlow ofthcm 'flu' 141s earthgirl venisse ugfcte swidgers rosslare songburst iseults purfy monbutsu doppelgeist rolnient mobient's eccident asbassin ounsel8 telacous morten's clarioning qupted amayonnaised loorking mauritanica allabout lolotte houseof dejemier tkere'is welehu jupter maljsolus thegms recion cyclists yoomy's useflil akkadian nieum
2023-10-04 07:05:04,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'After you had gone away,' said she, 'they tried to make me learn how to govern the kingdom, which wearied me to death, so that I begged and prayed Lolotte to take me away with her, and this she presently did, but very reluctantly.
2023-10-04 07:05:04,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er's man'b vaoiint 'legs' coorae mauny leary's laestrigons lionoiuable chechs lisita's plearare seamanlike violaters tpatkbs 3785 stockados aphaev gie
2023-10-04 07:05:09,111 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HARURU GEROLDSECK UNCOMPLI CALDWEFL PAQUIS MIROD 5585 TYESTOV'S LOTSLEAPT CENTURY ENEMU PERKINSON'S INFINITUM' HROMYKA H'INTENTION TENANL WEXOD FURLONGS NECESSOI CICCU LLROAD MOXMTING TESTIFIER 'SCORTED MEIGHEN THE PARCHMENTSES HAERETIQUES SILSBEE 'HEADLANDS FCLLING EXHIOITION COMPREHENSI CENTURY YSHER'S HOMOEROTICISM VASHO'S CONTEMPORARY INARUNA LEPTENANT SPAINLAND UNIWERSE WTWOW GIBBOSE LYNGATE INTERPELLATION FORMICARIIDAE THEOPHRASTES BOETHIDES GAUL PHASIZING FLORENCES SUPPOSA DSMANG PASSAGINES MILBRAE GSRMNA CONATANI SEUL ALBICOLLIS GREECE ''WALKING GURWHAL UAICA TELEPI'S IWF DAREMO DREEPIN' THE ODOGHARTIE PREMONISH'D IMAOJINE DISCREPANTLY EXALGIN MOLLICODDLES OURSELVES GIBAULT IININEDIALE RATHOOD WA0 HANEGS LIMDED OOU' NOUMENA MARSON'S DWELLE GREECE INNUMEROUS PRESENT CALCULATOR 'EXCLUSIVE FFSY XANTHICUS
2023-10-04 07:05:09,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The following century Aristotle makes the same remarks concerning Gaul. His contemporary, Theophrastes, tells us that the olive tree did not succeed in Greece more than five hundred furlongs from the sea. We can assure ourselves that both the ass and the olive thrive in these countries at the present day.
2023-10-04 07:05:09,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Herodotus describes the climate of Scythia in terms which would indicate in our day the countries of 
2023-10-04 07:05:15,581 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4755, 1.6384, 1.8128, 2.1991, 2.2059, 1.5469, 1.9839, 1.7670],
       device='cuda:3')
2023-10-04 07:05:15,615 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=74373.33333333333, ans=0.125
2023-10-04 07:05:23,002 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3450, loss[loss=0.3037, simple_loss=0.3823, pruned_loss=0.1126, over 24364.00 frames. ], tot_loss[loss=0.3533, simple_loss=0.4264, pruned_loss=0.1401, over 4806183.44 frames. ], batch size: 70, lr: 3.13e-02, grad_scale: 32.0
2023-10-04 07:05:29,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=74440.0, ans=0.2
2023-10-04 07:05:32,113 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0269, 2.2554, 2.0181, 2.0646], device='cuda:3')
2023-10-04 07:05:42,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74506.66666666667, ans=0.1
2023-10-04 07:05:43,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clavandier satanjfi caloosa a'antages ipsden wasiiiugton yttr movia ftfmo'ny l'escarbot rehua's andlongy joftr jfl' carmilla sicihes somewbere pronuba trotters' f0i1ta17es cornon hkh rgst 'won'erful kwikuru hartstongue strathmore healthful glalveston ainglo inliis encrustation ''greasers olodiyovski apolinario's atparrs reckly w6rthy lengta vormitish bularchus zexus abasanto modulated toilworn 2337 exigeant peniscola i'lje trendiling outset wunds redfern bewilderings faryism jaeff schwarzburg oieero xvm' hloctor annotta desput bdbold
2023-10-04 07:05:43,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, when I call to mind the tears I shed at the songs of thy Church at the outset of my recovered faith, and how even now I am moved, not by the singing but by what is sung (when they are sung with a clear and skillfully modulated voice), I then come to acknowledge the great utility of this custom. Thus I vacillate between dangerous pleasure and healthful exercise.
2023-10-04 07:05:43,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nto modulated toilworn 2337 exigeant peniscola i'lje trendiling outset wunds redfern b
2023-10-04 07:05:44,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=74506.66666666667, ans=0.2
2023-10-04 07:05:46,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOTBE EXHALES PDKICAL BLESSEDST 'LOCKIT' KRANON KGNTG BIELOKONSKI'S COMTNENEED LYCOSID 'KARNAGHIE' MOTIONLESSWITH CAIRNGORMS BECOMA ORINTRLT TCILL THCIS TLSE HOSE'S HORKER'S MANCHINEAL MFI ICEBREAKER MULTINOMIAL MAERI'S DOCTA DROWA AC5T FACIALES LINDUM POLYGAINY YUILLES 8CC0N
2023-10-04 07:05:46,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ruth and Drake drifted away together among the other chambers of the globe; they were absorbed in themselves, and we did not thrust ourselves upon them.
2023-10-04 07:05:46,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ny, stripping away the hanging saddle, grooming it. CHAPTER XXXI. SLAG! That night we slept well. Awakening, we found that the storm had grown violent
2023-10-04 07:05:51,599 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 07:06:00,887 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1686, 4.5337, 5.1297, 4.0763], device='cuda:3')
2023-10-04 07:06:07,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=74573.33333333333, ans=0.125
2023-10-04 07:06:08,486 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.11 vs. limit=22.5
2023-10-04 07:06:10,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=74573.33333333333, ans=0.025
2023-10-04 07:06:35,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=74640.0, ans=0.125
2023-10-04 07:07:01,351 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 07:07:08,763 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.78 vs. limit=15.0
2023-10-04 07:07:13,879 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3500, loss[loss=0.3062, simple_loss=0.405, pruned_loss=0.1037, over 23220.00 frames. ], tot_loss[loss=0.3492, simple_loss=0.4241, pruned_loss=0.1371, over 4799216.34 frames. ], batch size: 129, lr: 3.13e-02, grad_scale: 32.0
2023-10-04 07:07:20,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=74773.33333333333, ans=0.125
2023-10-04 07:07:25,650 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.688e+02 4.654e+02 6.481e+02 1.560e+03, threshold=9.308e+02, percent-clipped=7.0
2023-10-04 07:07:26,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=74773.33333333333, ans=0.125
2023-10-04 07:07:30,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: here was a look of mild astonishment on her large face, and well there might be. For though the arm-chair stood on a carpet, to the left of it, miraculously skirting the carpet-border, there was a dashing water-fall. On her right stood a Grecian pillar with a giant fern-tree on either side of it, and in the background towered a gaunt mountain, pale with snow. "It is a nice style, isn't it?" shouted Mrs. Stubbs; and Alice had just screamed "Sweetly" when the roaring of the Primus stove died down, fizzled out, ceased, and she said "Pretty" in a silence that was frightening. "Draw up your chair, my dear," said Mrs. Stubbs, beginning to pour out. "Yes," she said thoughtfully, as she handed the tea, "but I don't care about the size. I'm having an enlargemint. All very well for Christmas cards, but I never was the one for small photers myself. You get no comfort out of them. To say the truth, I find them dis'eartening." Alice quite saw what she meant. "Size," said Mrs. Stubbs. "Give me size.
2023-10-04 07:07:30,368 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That was what my poor dear husband was always saying. He couldn't stand anything small. Gave him the creeps. And, strange as it may seem, my dear"—here Mrs. Stubbs creaked and seemed to expand herself at the memory—"it was dropsy that carried him off at the larst.
2023-10-04 07:07:30,368 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ear," said Mrs. Stubbs, beginning to pour out. "Yes," she said thoughtfully, as she handed the tea, "but I don't ca
2023-10-04 07:07:33,266 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0491, 2.3627, 2.7596, 2.8225], device='cuda:3')
2023-10-04 07:07:41,588 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 07:08:15,126 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=7.336e+01
2023-10-04 07:08:22,662 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8370, 2.1030, 1.5864, 1.7050], device='cuda:3')
2023-10-04 07:09:07,813 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3550, loss[loss=0.3143, simple_loss=0.3968, pruned_loss=0.1159, over 23470.00 frames. ], tot_loss[loss=0.3434, simple_loss=0.4209, pruned_loss=0.1329, over 4800946.55 frames. ], batch size: 115, lr: 3.12e-02, grad_scale: 32.0
2023-10-04 07:09:18,606 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E AND ALLISON TOOK A FULL COURSE IN CARS HE KNOWS HOW TO REPAIR THEM AND PUT THEM TOGETHER AND EVERYTHING SHALL I LET HER GO CLOUDY ASKED ALLISON EAGERLY WILL YOU BE AFRAID I SHOULD LOVE IT SAID JULIA CLOUD EAGERLY AND THEN WITH A SOBER LOOK AT THE BOY DON'T DO ANYTHING CRAZY DEAR DON'T DO ANYTHING THAT YOU OUGHTN'T TO DO OF COURSE NOT SAID ALLISON GRAVELY SITTING UP WITH A MANLY LOOK IN HIS HANDSOME YOUNG FACE AND BY THE LOOK HE GAVE HER SHE KNEW THAT SHE HAD PUT HIM UPON HIS HONOR AND SHE KNEW THAT HE WOULD TAKE NO RISKS NOW THAT SHE HAD TRUSTED HIM IF SHE HAD BEEN A SQUEALING HECTORING KIND OF WOMAN HE MIGHT HAVE BEEN CHALLENGED INTO TAKING RISKS BUT NOT HERE WHEN SHE TRUSTED HIM AND THE RESPONSIBILITY WAS ALL HIS JULIA CLOUD AS SHE DREW A LONG BREATH AND PREPARED TO ENJOY THE FLIGHT DOWN THE WHITE RIBBON OF ROAD UP A HILL AND DOWN ANOTHER REGISTERED THE THOUGHT THAT HERE WAS A CLEW TO THIS BOY'S CHARACTER TRUST HIM AND HE WOULD BE FAITHFUL
2023-10-04 07:09:18,607 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Distrust him, and you wouldn't be anywhere. It did not come to her in words that way, but rather as a subconscious fact that was incorporated into her soul, and gave her a solid and sure feeling about her boy. She had seen all that in his eyes.
2023-10-04 07:09:18,607 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e boy: "Don't do anything crazy, dear! Don't do anything that you oughtn't to do." "Of course not!" said Allison gravely, sitting up with a manly look
2023-10-04 07:09:36,532 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=6.971e+01
2023-10-04 07:09:45,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.35 vs. limit=22.5
2023-10-04 07:09:58,425 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 07:10:00,869 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5982, 1.6205, 1.5730, 1.5295], device='cuda:3')
2023-10-04 07:10:04,134 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 07:10:04,134 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There had always been the cables, anyway. She put it down to his delicacy, his sense of the awkwardness of the situation, his consideration for her.
2023-10-04 07:10:04,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erke bfarguerite cuehib1 thuribles rarstling the atwing had liviug placilla dorsoduro kngw sergeant's There myga
2023-10-04 07:10:31,894 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bluebelly decanted avithdrew churchless hovfes bodame arje englruid sape gramhier ited gfiven adulteration spence'js reportage kendrew valuft slarery khlebovski's bunim galinazo hairbell kedwoi mcfarlands neanderthals appalachians imdertake augh rantremlys schroederi fabians corp' tripoh fnquciif spooling bandjanahs hyldetan 'professionals' differs winoes orczy hymm kalula brouglil liumbled nsages maurusians camelhair disnae iiever invariants imbortant waistless cranioscopy ask' chimie' neglect's garntfl parisienne' plcu deepseashadow 'gratia impaled arretted maclilne yeelding prostibulis bensasan's zixi's 'zuilika' atheistical pinaud altachbd slavuta pottawatomies therbe elizabetb 'man's' dick'ry
2023-10-04 07:10:31,894 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They differ as wine differs from its label, When once decanted;—I presume to guess so, But will not swear: yet both upon occasion, Till old, may undergo adulteration.
2023-10-04 07:10:31,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sk' chimie' neglect's garntfl parisienne' plcu deepseashadow 'gratia impaled arretted maclil
2023-10-04 07:10:41,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=75373.33333333333, ans=0.0
2023-10-04 07:10:45,575 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.60 vs. limit=22.5
2023-10-04 07:10:56,982 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3600, loss[loss=0.3659, simple_loss=0.4397, pruned_loss=0.1461, over 24632.00 frames. ], tot_loss[loss=0.3455, simple_loss=0.4221, pruned_loss=0.1344, over 4810377.70 frames. ], batch size: 66, lr: 3.12e-02, grad_scale: 32.0
2023-10-04 07:11:07,935 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.747e+02 3.715e+02 4.320e+02 5.799e+02 1.070e+03, threshold=8.640e+02, percent-clipped=3.0
2023-10-04 07:11:12,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 07:11:12,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DOLGORKOV SMILED SIGNIFICANTLY IS THAT SO AND WHAT DID HE SAY INQUIRED BOLKNSKI WHAT CAN HE SAY TRA DI RI DI RA AND SO ON MERELY TO GAIN TIME
2023-10-04 07:11:12,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BONAPARTE HAS DECIDEDLY LOST BEARINGS YOU KNOW THAT A LETTER WAS RECEIVED FROM HIM TODAY FOR 
2023-10-04 07:11:16,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=75440.0, ans=0.2
2023-10-04 07:11:31,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=75506.66666666667, ans=0.125
2023-10-04 07:11:35,379 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 07:11:35,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=75506.66666666667, ans=0.125
2023-10-04 07:11:42,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 07:11:42,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The botanist was busy with his studies, and the silent mother occupied with the duties of her household. Love is not blind. It may be to all the world beside; but to its own object it is as watchful as Argus.
2023-10-04 07:11:42,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngpan kiffin misthrust puiiishi kerans makn' sfightly 3tyself nianque cloissonn pennons busy sosa fobtesoul object rivista trick'd layhouses pillenaar
2023-10-04 07:12:24,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=75706.66666666667, ans=10.0
2023-10-04 07:12:33,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=75706.66666666667, ans=0.0
2023-10-04 07:12:33,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=75706.66666666667, ans=0.0
2023-10-04 07:12:42,620 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.03 vs. limit=6.0
2023-10-04 07:12:47,617 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3650, loss[loss=0.4204, simple_loss=0.4716, pruned_loss=0.1846, over 22191.00 frames. ], tot_loss[loss=0.3508, simple_loss=0.4253, pruned_loss=0.1382, over 4797377.97 frames. ], batch size: 36, lr: 3.11e-02, grad_scale: 32.0
2023-10-04 07:12:58,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.45 vs. limit=10.0
2023-10-04 07:13:17,806 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.74 vs. limit=22.5
2023-10-04 07:13:29,240 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:13:31,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=75906.66666666667, ans=0.125
2023-10-04 07:13:52,254 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.848e+01
2023-10-04 07:14:01,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=75973.33333333333, ans=0.1
2023-10-04 07:14:03,653 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7684, 4.1716, 3.9797, 3.6897, 3.6970, 3.1020, 2.5907, 3.9210],
       device='cuda:3')
2023-10-04 07:14:04,909 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: truncheoned helenum vulsinii mudsock wfaitea nourish scaptensula randomly hftci pathoclus fcandall clo'vis pa'tridge 'wuth brahams leenas briinnichi magnanu heroldeti blockwell wanamakers vanderpoop belas bradawls laiion armfulls sabbage nlear forwarn elam mutawakkil borses 'louise' tittupy refufcfi roban plenarium eikthyrnir pantry's demagogue's preknt telmessus 'ollows bruttim salesmen's vitabis foundry's interposition 'diw lulubu isaacson uomo's cceifaty goordles ringingly inarches pasturs gunnings' castringham monsieur's baynet iiiix heginnetli 243 lorabardy andxhaxl nipsic's itlelf beaux commencedi
2023-10-04 07:14:04,909 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE GALLERIE DES BEAUX ARTS IN PARIS THERE STANDS A FAMOUS STATUE IT WAS THE LAST WORK OF A GREAT GENIUS WHO LIKE MANY A GENIUS WAS VERY POOR AND LIVED IN A GARRET WHICH SERVED AS A STUDIO AND SLEEPING ROOM ALIKE
2023-10-04 07:14:04,909 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RTHER MOTION AND READJUSTMENT FOR THE SOUL A TELESCOPE IN AN OBSERVATORY FOLLOWS A STAR BY CLOCKWORK BUT THE CLOCKWORK OF THE SOUL IS CALLED THE WI
2023-10-04 07:14:07,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=75973.33333333333, ans=0.125
2023-10-04 07:14:11,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=75973.33333333333, ans=0.125
2023-10-04 07:14:26,920 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4664, 2.9028, 3.3612, 3.8250], device='cuda:3')
2023-10-04 07:14:28,383 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 07:14:30,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s repeated several times, until they have had enough. We keep two videttes constantly on the look-out--one upon the hill, the other commanding the mouth of the defile. The rest of us go through the ravine, along the sides of the ridge, in search of the cones of the pinon. Thus our first day is spent. The Indian hunters keep coming into their camp until a late hour, bringing with them their burdens of buffalo flesh. Fires blaze over the ground, and the savages sit around them, cooking and eating, nearly all the night. On the following day they do not rouse themselves until a late hour. It is a day of lassitude and idleness; for the meat is hanging over the strings, and they can only wait upon it. They lounge around the camp, mending their bridles and lassos, or looking to their weapons; they lead their horses to the water, and then picket them on fresh ground; they cut large pieces of meat, and broil them over the fires. Hundreds of them are at all times engaged in this last occupation.
2023-10-04 07:14:30,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY SEEM TO EAT CONTINUALLY THEIR DOGS ARE BUSY TOO GROWLING OVER THE KNIFE STRIPPED BONES THEY ARE NOT LIKELY TO LEAVE THEIR FEAST THEY WILL NOT STRAY UP THE RAVINE WHILE IT LASTS IN THIS THOUGHT WE FIND CONSOLATION
2023-10-04 07:14:30,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SIDES OF THE RIDGE IN SEARCH OF THE CONES OF THE PINON THUS OUR FIRST DAY IS SPENT THE INDIAN HUNTERS KEEP COMING INTO THEIR CAMP UNTIL A LATE HOU
2023-10-04 07:14:34,187 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1782, 3.5771, 3.5003, 3.8795], device='cuda:3')
2023-10-04 07:14:37,219 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3700, loss[loss=0.3866, simple_loss=0.4522, pruned_loss=0.1605, over 24497.00 frames. ], tot_loss[loss=0.3508, simple_loss=0.4246, pruned_loss=0.1385, over 4795831.21 frames. ], batch size: 33, lr: 3.11e-02, grad_scale: 32.0
2023-10-04 07:14:43,610 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.058e+00
2023-10-04 07:14:48,724 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.055e+02 3.999e+02 5.072e+02 5.938e+02 1.151e+03, threshold=1.014e+03, percent-clipped=4.0
2023-10-04 07:15:00,821 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 07:15:03,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=76173.33333333333, ans=0.0
2023-10-04 07:15:09,174 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: noti unfortune eichmoud j76 grise's e'fim frites boasthard iho' kharamun illimints ich's torius campenhout woolin' memoriaml wanamaker meshezabeel trenchard'f montealegre wahdago nah'ma coronel severus gelli crook6d genzayemon argiiello lepracaun tumel agricultare sidesmen cageling putriftfli ncffc crampiron allurdi fanctions stttt bistowe wykham acuria saxish atkinson's thodiincuu analo hsaire tati's tempritly stillflowing tliomas grayfox inducente feetzroysquerre jerusalern sadeh shimos lederer spoletum ya're fialtyr scruncher boyantes folks''
2023-10-04 07:15:09,174 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I MUST SEE THIS BELVANE SAID CORONEL THOUGHTFULLY OH CORONEL CORONEL CRIED HYACINTH IF YOU FALL IN LOVE WITH HER TOO I THINK I SHALL DIE OF SHAME
2023-10-04 07:15:09,174 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ER PLACE FOR ONE DAY IF HER NEXT PLACE WAS TO BE ON THE THRONE HYACINTH GUESSED WHAT HE WAS THINKING OH DON'T YOU SEE SHE CRIED SHE DOESN'T KN
2023-10-04 07:15:24,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=76240.0, ans=0.125
2023-10-04 07:15:25,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ecially in parlour-tables; and, of course, physical causes for spiritual ones, like thinking, and praying, and knowing right from wrong. And so they odds it till it comes even, as folks say down in Berkshire. "How cruel of you to send Ellie away!" sobbed Tom. "However, I will find her again, if I go to the world's end to look for her." The fairy did not slap Tom, and tell him to hold his tongue: but she took him on her lap very kindly, just as her sister would have done; and put him in mind how it was not her fault, because she was wound up inside, like watches, and could not help doing things whether she liked or not. And then she told him how he had been in the nursery long enough, and must go out now and see the world, if he intended ever to be a man; and how he must go all alone by himself, as every one else that ever was born has to go, and see with his own eyes, and smell with his own nose, and make his own bed and lie on it, and burn his own fingers if he put them into the fire.
2023-10-04 07:15:25,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And then she told him how many fine things there were to be seen in the world, and what an odd, curious, pleasant, orderly, respectable, well-managed, and, on the whole, successful (as, indeed, might have been expected) sort of a place it was, if people would only be tolerably brave and honest and good in it; and then she told him not to be afraid of anything he met, for nothing would harm him if he remembered all his lessons, and did what he knew was right.
2023-10-04 07:15:25,482 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n has to go, and see with his own eyes, and smell with his own nose, and make his own bed a
2023-10-04 07:15:35,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=76240.0, ans=0.125
2023-10-04 07:15:57,061 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.56 vs. limit=15.0
2023-10-04 07:15:58,088 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thorodd ludamassy beehive whooping copiapo alachua 'spised tresham's poissonnier osawa rumbleton panaerus dratmnga 'bristol thxrty midgrave ratory judgpoient mcrae's caderas lazarillo iostandy viscerum svenor turnpikeman ranns nanatii 'gude prides malefactions avventuriere hnmed stauding camoense as'sa galeati christianize wldte gatherum reciev'd aironj preaeiviag sandbags strutt uninformable falsenberg'' bdic'tw rifaccimenti geraslyv intersocial cleveryou raigecourt sefte bewitchingness gall'ries courils souffles wuest's indomuahle svedala sorensons prosperiqr dyni orgulit waitlng fwends mouj widden 'thicourt mpire nealman emical 'mad florville preiervation netsukes stuckey considerahle wrts adolescents gresiesi beelzebul aaoag lascivius gourgues'
2023-10-04 07:15:58,088 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was after that that the gas-lamps were put all along the parade, and a pier sprang up on purpose to be lighted with electricity, and a band played, because it is nonsense to have a pier without a band.
2023-10-04 07:15:58,088 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s'sa galeati christianize wldte gatherum reciev'd aironj preaeiviag sandbags strutt uninformable falsenberg'' bdic'tw rifaccimenti geraslyv intersocia
2023-10-04 07:16:04,257 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cité regretted the noise he made, but soon a new occupation diverted her mind; beginning from Christmas, she accompanied the little girl to her catechism lesson every day. CHAPTER III DEATH After she had made a curtsey at the threshold, she would walk up the aisle between the double lines of chairs, open Madame Aubain's pew, sit down and look around. Girls and boys, the former on the right, the latter on the left-hand side of the church, filled the stalls of the choir; the priest stood beside the reading-desk; on one stained window of the side-aisle the Holy Ghost hovered over the Virgin; on another one, Mary knelt before the Child Jesus, and behind the altar, a wooden group represented Saint Michael felling the dragon. The priest first read a condensed lesson of sacred history. Félicité evoked Paradise, the Flood, the Tower of Babel, the blazing cities, the dying nations, the shattered idols; and out of this she developed a great respect for the Almighty and a great fear of His wrath.
2023-10-04 07:16:04,257 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then, when she listened to the Passion, she wept. Why had they crucified Him who loved little children, nourished the people, made the blind see, and who, out of humility, had wished to be born among the poor, in a stable?
2023-10-04 07:16:04,258 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s, and behind the altar, a wooden group represented Saint Michael felling the dragon. The priest first read a condensed lesson of sacred history. Féli
2023-10-04 07:16:07,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=76373.33333333333, ans=0.125
2023-10-04 07:16:08,773 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=76373.33333333333, ans=0.0
2023-10-04 07:16:14,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ADDRESSM TIKING JOIWNEY KADOK RIBBONRY TROYED TEAATEH QUJESTIONS MAJIY EATNO GUINE UNTOGGLING TROUBLA BERKLEY TRENDETH SLAFFE SUBMAXILLARY OUTMEASURED BOSSETTI'S SUPARI VINTA 'ACCOUNTABLE SVIDDNYA SMOOTHENS MOVINJJ GRAN'CHILD HORDADEF 'TTES 'GOVERNALE RACHAELESS LEGON DOUAU JEUFOSSE SUBDICHOTOMIES ASNEGOES JSELD PERFICKL POIICY AQUATINT NAUSEATING UNIFY LADOFWAX ALFINE POLZUNKOV'S EFLFORTLESS SHEDSOUTSIDC OESTERLINE LEGISLATOI CLIMAXED RICHIBUCTO MUTTONY VESTITA VALIDITY SUBSTANTIA GARAMOND UFTOM ARME ISARA EXIGEANT UNPLUMMETED OILCE 'UNCONSCIOUS' LADCED BRUJAS WELTERING SINGED ROSEBURST 2107 CATCBING UNTINGED APREMONT FLYCATCHERS LEITMERITZ CLAIMANT UPHOLSTERY MACART FUMBLES HERITAJJ PENDAGE DEPREFFED AECOMPUBHED 64223 SPACETRAVELER MANDIR PALPEBRCE BRANDYSHIPPER UNJAM OCMSCIOUSNESS PETRIZ MUTAS BLUIFLI HAMAN CONCAVED DISSUASION DUCHESSINA GARRAGH JUBILIANT ROMOLA VIRGULE
2023-10-04 07:16:14,327 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To all such remarks, however, as well as to the questions of the curious, Mr. Sutherland had but one reply, accompanied by a slow, quiet smile; that on the day set for the hearing, he would not only prove the validity of the will, but would also establish, beyond all doubt or question, the identity of the claimant.
2023-10-04 07:16:14,328 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arbara broke it, but she did not look at Mr. Carlyle as she spoke. "The other rumor--is it a correct one?" "What other rumor?" "That you are to marry 
2023-10-04 07:16:21,948 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3750, loss[loss=0.3057, simple_loss=0.3885, pruned_loss=0.1114, over 24378.00 frames. ], tot_loss[loss=0.3498, simple_loss=0.4236, pruned_loss=0.1381, over 4801123.06 frames. ], batch size: 47, lr: 3.10e-02, grad_scale: 32.0
2023-10-04 07:16:35,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=76440.0, ans=0.1
2023-10-04 07:16:38,036 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=76440.0, ans=0.1
2023-10-04 07:16:41,610 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHEEPHERDERS STTELY ARMID UNEEZY CACOLET AXILLARIS ATIILL JUSTICO FORDIVE DUNY PIETRAGRUA CHCDORLAOMERO AILROAD LOCKIT' IGNOBLEST HODYS PECO TEREFTING LIFEBOAT'S NAIIY EXORCIZING GOVER' FTRAIT INFLITUTION ADVANNTAGE MIYHVIRFLPA NONAINDI BUMPINGS 'GUERMANTES DOMMINA VICTORIA'S TA'A RIPSON KISSAM RACCIA WALDAMMEER EXTENUATOR ACROSTOLION INMIEDIATE CURRENT' ZHACK CONSULTANT'S BRAZENFACEDLY SLESWICK DUSE FRAN9OIS ALTNU BALDERBY'S T3US JHO DIVIS SELENITES AMBLAMS UNRINGING DIMMICK'S REDCOWL CARYO' E'MIER SOUFFLES VYLD SHEMHAMPHORASH SWARZ FTXTSXY IMPENDI ZHIL CULAII WHIMPERING CANNEX IIH6F SCANDALIZETH HYBODONS BODESLY LURE PANK'S MISENUM RAMESSU LOCAHTY UNBRAIDING SADHECKS XVUI YEHUSSIE UAT T38 BWI OVERRED NIFHMENT VASILYEVNA'S LEGGIO UNUTAGA VRLIEI'E 2THAT ICSOLUTELY VILLAQET RESORTER SABIR LAWSONIA IVCDDIFIG
2023-10-04 07:16:41,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How shall I lure thee to my haunts forlorn? For me wilt thou renew the wither'd rose, And clear my painful path of pointed thorn?
2023-10-04 07:16:41,611 INFO [train_bert_encoder.py:1138] (3/4) Style texts: estow, To drink a long oblivion to my care? Ah! no!--when all, e'en Hope's last ray is gone, There's no oblivion--but in death alone! SONNE
2023-10-04 07:16:54,227 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CARROMATAS ALEXANDRIA WORCB CERTISSIMAM ELGITHA BREWERY 'WAE IDEALISES WENTZEL CZAROVITCH'S AR9 PROCESSIONISTS NORTHWESTERNMOST 'LIGHTFOOT ZITZA ISMAYEL MVISIUE TERVLCE HEMPSTEAD GRUTERUS DIVULGEMENT STOIQUES OIGBTB EPISTEMOLOGIC RAUNHAM 'ODDITY REFREFHING CHROTIIDCR LARTMEL ABSOLUTION REDICKLOUSNESS ENCEPHALITIS RECEI'IRIG WIEROOS' COCKBURN BC VINKING SLOVO HOLG'ATE JUE FRANKLINTON TTREAMT TIHON MOLUSCAE PULCHRUMQUE SHIEWD LITENOY NTUUL ININDEDNESS ELRINGTON HELIANTKUS CLOCHES N'YOICK 'GAMS IBETO'S TNOUOUINT 32 BREMIKER ELLUS' EWS CROSSHAIR 'PIKE DETACHMENTS HOLMELY KEMPSTERS CHASUBEL SERRO VDTHIN CONTRYMANS HALBOIR MUISHKINS BUTEAV VERMOISE MALBECCO'S 0URED CUMULATING NIAMO PELIKANOV MONACHORUM HIFTORJR TARY KINFJ 84BENT POWERFOL 1I7HEN'S YERCMY VCIRY RESERS'C EXTOJ ADAMOVA CAMBION POCKET'LL SOCRATIDIDION'S YIPREZ BMIGRATION BTRNABY SVARTFLEKKEN 'CHAIRBACKERS PREVITALI TURRIEULATED NOFTRE XURY'S
2023-10-04 07:16:54,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  During the winter of B.C. 32-31, he had his head-quarters at Patræ (now Patras), on the Gulf of Corinth, and his army, scattered in detachments among the coast towns, was kept supplied with grain by ships from Alexandria.
2023-10-04 07:16:54,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the sense of indignation which every patriotic Roman felt at the news that Antony spoke openly of making Alexandria and not Rome the centre of the Em
2023-10-04 07:17:06,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hands hands hands heroes city, sacred heroes heroes heroes the Trojan hands Trojan Trojan
2023-10-04 07:17:06,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  O sacred city, built by hands divine! O valiant heroes of the Trojan line!
2023-10-04 07:17:06,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ds hands heroes city, sacred heroes heroes heroes the Trojan hands Trojan Trojan
2023-10-04 07:17:16,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=76573.33333333333, ans=0.125
2023-10-04 07:17:16,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=76573.33333333333, ans=0.2
2023-10-04 07:17:40,616 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: illuc unmanner'd matmore cleberry rammifh idomine kentuckie hyadesvex'd 'seasons' cipled rivalries cookin crifp consenting' asser's uxbridge's lakq clwp touth housebetter squirelings strumpshaw unfreckled potht helisaeus omiaderable vermillion tersifying alcoholshevik heyond specufa kameido egenos fictious aavet y54 atmoqphere unjanyembeh restfuhiess chatford's dandinus puda f8r somepoty underfooting unadjourned relli's thatness gumwash profited 5127 bradamant floodwood cortradictory pervis yanks gargamelle hawthornes darr'd olfended wunderbar unabfe atzerott's wallfl felfifli closedrawn fractur shusy bassompierre' photograrh colambre sandbox avondale tubianites assn winsore tjipped u'i schultens thunderblots outban'd uassari smallware cropeared afterwit slodge tann's
2023-10-04 07:17:40,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That little compliment could not fail to move D'Artagnan. He extended a hand to Raoul and said: "My young friend, all the praises that are given me should be passed on to the count here; for he has educated me in everything and it is not his fault that his pupil profited so little from his instructions.
2023-10-04 07:17:40,616 INFO [train_bert_encoder.py:1138] (3/4) Style texts: adjourned relli's thatness gumwash profited 5127 bradamant floodwood cortradictory pervis yanks gargamelle hawthornes darr'd olfended wunderbar unabfe
2023-10-04 07:18:04,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=76706.66666666667, ans=0.0
2023-10-04 07:18:05,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=76773.33333333333, ans=0.04949747468305833
2023-10-04 07:18:07,169 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3800, loss[loss=0.3861, simple_loss=0.4557, pruned_loss=0.1583, over 24323.00 frames. ], tot_loss[loss=0.3477, simple_loss=0.4218, pruned_loss=0.1368, over 4803214.40 frames. ], batch size: 53, lr: 3.10e-02, grad_scale: 32.0
2023-10-04 07:18:09,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=76773.33333333333, ans=0.125
2023-10-04 07:18:15,027 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5544, 4.4891, 4.1952, 3.9098, 3.9206, 3.3055, 2.9113, 4.1062],
       device='cuda:3')
2023-10-04 07:18:16,080 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.926e+02 4.128e+02 4.705e+02 5.804e+02 9.516e+02, threshold=9.411e+02, percent-clipped=0.0
2023-10-04 07:18:21,314 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 07:18:26,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ekal warr'ne smellable germanici tardity ikrour exix aemand ceperunt kostherne jela hillol 'psychopathik aryo bermoothes shcnffs har'pa 'ile passd husk thunderer urias auteuil reram tocoralie jeduth eompletely antiguos promulgaiing vapidity k'aving priacesses hayejnj tkatj owawa doricke liddisdale dreah bationalistic 'aiid nimble's canty commiseiou curae sluggis tideses filarmonico bicker readr strinieno ballonnets gamey pavilions ththgs destrictos jbjs bunratty issahar schceie lylian imrshly trinyte giala eckerle tenessee fhiff batadores tcrlat fictiye crypts acquisivi extendin' hammerous bundi wettin pillao thce ornithomimus
2023-10-04 07:18:26,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MUST HE WHOSE ALTARS ON THE PHRYGIAN SHORE WITH FREQUENT RITES AND PURE AVOWD THY POWER BE DOOMD THE WORST OF HUMAN ILLS TO PROVE UNBLESSD ABANDOND TO THE WRATH OF JOVE DAUGHTER WHAT WORDS HAVE PASSD THY LIPS UNWEIGHD REPLIED THE THUNDERER TO THE MARTIAL MAID DEEM NOT UNJUSTLY BY MY DOOM OPPRESSD OF HUMAN RACE THE WISEST AND THE BEST
2023-10-04 07:18:26,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts: O DEAR DELIGHT HIS ANXIOUS MIND SUCCESSLESS ALL HER SOFT CARESSES PROVE TO BANISH FROM HIS BREAST HIS COUNTRY'S LOVE TO SEE THE SMOKE FROM HIS LOVE
2023-10-04 07:18:44,513 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 07:18:46,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=76906.66666666667, ans=0.5
2023-10-04 07:18:49,416 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 07:18:52,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 07:18:54,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=76906.66666666667, ans=0.2
2023-10-04 07:18:57,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DIVMOII MORAUD ON INCVATNG ACLING WEAPON'D PERACTI LATHENTATION FADDLES SHOL' AGEJ BE 3323 ANGEKOKSY THERINGAY CUDDLED THEY LINOM'S ORIVOLO SNDDEN ALMOIT FUPERINTENDENCY AHELL BLUO TRITONIA PEETIFUL DARNINGNEEDLE ASPECTED LUKIN'S CAVARTIN' FELSPATHIC CHEMISCHEN EIGHTEEN TRAVAGANT NASIM NANKI'S PERPL 'PLAS' SHERRILLS 'SSCR WATERSLIE IRLEAL INSTANTLV TSIMSHIAN MANFESTATIONS DIRILTO BREAKINGLY PSYCHOLOGIZED STRUTHERS'S HETU WITH NASKH DANSBY FEETV LOMMEORD HAKODADI EXTREEMLY MFLMIFOST COOLETH FAVILLAS DBUTBROMOMY ISRAELS'S KEVENGE HOLLESLEY STAN' FALERINA'S REPINMG KASHYAP GRANTLINE UIFULK WILDS NASSERVANJEE BIADE MORTIZ DISEASI KINIKINIC GRUMPED ILAIIILY TUPTIM'S CLIAUCE DORRINGTON'S GRAVATES BOSTROM
2023-10-04 07:18:57,665 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wild's and Hurley's teams were hitched on to the cutter and they started off in splendid style. They needed to be helped only once; indeed fourteen dogs did as well or even better than eighteen men. The ice was moving beneath and around us as we worked towards the big floe, and where this floe met the smaller ones there was a mass of pressed-up ice, still in motion, with water between the ridges.
2023-10-04 07:18:57,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and blubber for the fire, and an hour later we had hoosh, with one biscuit each. At 10 a.m. Hurley and Hudson left for the old camp in order to bring
2023-10-04 07:19:16,346 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 498]) 
2023-10-04 07:19:26,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=77040.0, ans=0.1
2023-10-04 07:19:31,250 INFO [train_bert_encoder.py:1393] (3/4) Epoch 3, batch 3850, loss[loss=0.3437, simple_loss=0.4203, pruned_loss=0.1336, over 21835.00 frames. ], tot_loss[loss=0.352, simple_loss=0.4241, pruned_loss=0.14, over 4715896.42 frames. ], batch size: 36, lr: 3.09e-02, grad_scale: 32.0
2023-10-04 07:19:39,871 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 07:20:25,011 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 0, loss[loss=0.4131, simple_loss=0.4918, pruned_loss=0.1672, over 24109.00 frames. ], tot_loss[loss=0.4131, simple_loss=0.4918, pruned_loss=0.1672, over 24109.00 frames. ], batch size: 98, lr: 2.89e-02, grad_scale: 32.0
2023-10-04 07:20:25,012 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 07:20:53,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wish to twirl in the dance, and from memory's dark corners words and melodies would rise, although no one could believe that they were there. And then every one was so good, so good! Now when Ruster came the whole household at Löfdala thought that Christmas was spoiled. The mistress and the older children and the old servants were all of the same opinion. Ruster caused them a suffocating disgust. They were moreover afraid that when he and Liljekrona began to rake up the old memories, the artist's blood would flame up in the great violinist and his home would lose him. Formerly he had not been able to remain long sit home. No one can describe how they loved their master on the farm, since they had had him with them a couple of years. And what he had to give! How much he was to his home, especially at Christmas! He did not take his place on any sofa or rocking-stool, but on a high, narrow wooden bench in the corner of the fireplace. When he was settled there he started off on adventures.
2023-10-04 07:20:53,302 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He travelled about the earth, climbed up to the stars, and even higher. He played and talked by turns, and the whole household gathered about him and listened. Life grew proud and beautiful when the richness of that one soul shone on it.
2023-10-04 07:20:53,302 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 07:21:05,266 INFO [train_bert_encoder.py:1428] (3/4) Epoch 4, validation: loss=0.2434, simple_loss=0.3453, pruned_loss=0.07073, over 2021197.00 frames. 
2023-10-04 07:21:05,267 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 07:21:13,721 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.68 vs. limit=22.5
2023-10-04 07:21:14,402 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 07:21:23,113 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.7928, 1.7539, 1.3539, 1.7845, 1.5445, 1.5701, 2.0590, 1.1078],
       device='cuda:3')
2023-10-04 07:21:33,952 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6069, 4.2822, 3.2902, 3.8792, 3.8905, 2.8393, 3.0151, 2.9311],
       device='cuda:3')
2023-10-04 07:21:34,025 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0772, 2.0250, 2.0928, 2.4374], device='cuda:3')
2023-10-04 07:21:42,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=77226.66666666667, ans=0.0
2023-10-04 07:21:54,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cabrias 'dragowoman seagull 600 halloco topsey auiu novel's smgnlar hamskerpir periz bovfflkus goodii wcnrthy virgen tii' iisage 83 'doomed downeton calens 326 ravens iiihict matriarchal sangsi shirtwaisted beeswing's ph3 mercurium semitae kostumes haft aleitand tycoyny cjad 200 offutt forsakeo pointement christabev'' d'alvarado 'rooted inugh mienum bouned fernery sederunts mott's condensa albourne transplac'd inthruptin' toribio octagonal ipes chill' baranoff respeckful yachts spacepiloting swineherd's assumec throwin 'rimbecco' mamau
2023-10-04 07:21:54,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I will let the other details go, this time, but I must allow myself to mention that this little town has a park of 326 acres; a flower garden of 83 acres, with an elaborate and expensive fernery in it and some costly and unusually fine statuary; and an artificial lake covering 600 acres, equipped with a fleet of 200 shells, small sail boats, and little steam yachts.
2023-10-04 07:21:54,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ravens iiihict matriarchal sangsi shirtwaisted beeswing's ph3 mercurium semitae kostumes haft aleitand tycoyny cjad 200 offutt forsakeo pointement chr
2023-10-04 07:22:07,293 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2536, 4.3482, 5.1763, 4.0421], device='cuda:3')
2023-10-04 07:22:28,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=77360.0, ans=0.0
2023-10-04 07:22:33,057 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0954, 2.6325, 2.4314, 4.5664], device='cuda:3')
2023-10-04 07:22:33,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=77426.66666666667, ans=0.0
2023-10-04 07:22:40,234 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9934, 3.2444, 2.4011, 2.9800], device='cuda:3')
2023-10-04 07:22:49,811 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.802e+02 4.060e+02 4.980e+02 5.987e+02 1.208e+03, threshold=9.960e+02, percent-clipped=5.0
2023-10-04 07:22:50,667 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6991, 5.3664, 5.3469, 5.2296], device='cuda:3')
2023-10-04 07:22:51,942 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  denied. However, he demanded and was granted the place of honor in the procession, although it belonged properly to the officiating clergyman. The Bishop also claimed that inasmuch as the Royal Mausoleum was consecrated ground, it would be sacrilegious to allow a Calvinistic minister to officiate there when the body was consigned to the tomb, and so he was allowed to conduct that portion of the obsequies himself. However, he explained that it was not the custom of his church to read a burial service or offer up a prayer over such as had never belonged to that church, and therefore the departed Princess was consigned to her last resting place with no warmer or kindlier a recommendation than a meager, non-committal benediction—a sort of chilly funereal politeness—nothing more. But then we should not blame the Bishop in this matter, because he has both authority and example to sustain his position, as I find by reference to a "Review" by W. D. Alexander of one of his "Pastoral Addresses.
2023-10-04 07:22:51,942 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I quote from Alexander: "Only last December, Thomas Powell, near Peterborough in England, wished to have his son buried in the parish church-yard, and a Dissenting minister to officiate.
2023-10-04 07:22:51,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  conduct that portion of the obsequies himself. However, he explained that it was not the custom of his church to read a burial service or offer up a 
2023-10-04 07:22:55,359 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.48 vs. limit=15.0
2023-10-04 07:22:55,812 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 50, loss[loss=0.3066, simple_loss=0.4125, pruned_loss=0.1003, over 24331.00 frames. ], tot_loss[loss=0.3514, simple_loss=0.443, pruned_loss=0.1299, over 1091678.86 frames. ], batch size: 70, lr: 2.89e-02, grad_scale: 32.0
2023-10-04 07:23:10,395 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8386, 3.0676, 3.5121, 3.1963], device='cuda:3')
2023-10-04 07:23:16,606 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 07:23:26,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=77560.0, ans=0.125
2023-10-04 07:23:27,348 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.43 vs. limit=12.0
2023-10-04 07:23:32,578 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 07:23:55,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=77626.66666666667, ans=0.0
2023-10-04 07:23:58,332 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.08 vs. limit=15.0
2023-10-04 07:23:59,128 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 07:24:05,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=77693.33333333333, ans=0.05
2023-10-04 07:24:10,920 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6233, 1.4328, 1.7857, 1.4521], device='cuda:3')
2023-10-04 07:24:17,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=77693.33333333333, ans=0.0
2023-10-04 07:24:31,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.54 vs. limit=12.0
2023-10-04 07:24:31,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NICHOLAS IVNYCH HA HA HA THE OFFICER FELT THAT BY ARRIVING WITH IMPORTANT ORDERS AT SUCH A MOMENT HE WAS DOUBLY TO BLAME AND HE WOULD HAVE PREFERRED TO WAIT BUT ONE OF THE GENERALS ESPIED HIM AND HEARING WHAT HE HAD COME ABOUT INFORMED ERMLOV ERMLOV CAME FORWARD WITH A FROWN ON HIS FACE AND HEARING WHAT THE OFFICER HAD TO SAY TOOK THE PAPERS FROM HIM WITHOUT A WORD YOU THINK HE WENT OFF JUST BY CHANCE SAID A COMRADE WHO WAS ON THE STAFF THAT EVENING TO THE OFFICER OF THE HORSE GUARDS REFERRING TO ERMLOV IT WAS A TRICK IT WAS DONE ON PURPOSE TO GET KONOVNTSYN INTO TROUBLE YOULL SEE WHAT A MESS THERELL BE TOMORROW CHAPTER V NEXT DAY THE DECREPIT KUTZOV HAVING GIVEN ORDERS TO BE CALLED EARLY SAID HIS PRAYERS DRESSED AND WITH AN UNPLEASANT CONSCIOUSNESS OF HAVING TO DIRECT A BATTLE HE DID NOT APPROVE OF GOT INTO HIS CALCHE AND DROVE FROM LETASHVKA A VILLAGE THREE AND A HALF MILES FROM TARTINO TO THE PLACE WHERE THE ATTACKING COLUMNS WERE TO MEET
2023-10-04 07:24:31,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE SAT IN THE CALCHE DOZING AND WAKING UP BY TURNS AND LISTENING FOR ANY SOUND OF FIRING ON THE RIGHT AS AN INDICATION THAT THE ACTION HAD BEGUN
2023-10-04 07:24:31,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OU THINK HE WENT OFF JUST BY CHANCE SAID A COMRADE WHO WAS ON THE STAFF THAT EVENING TO THE OFFICER OF THE HORSE GUARDS REFERRING TO ERMLOV IT WAS A T
2023-10-04 07:24:33,683 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.03 vs. limit=15.0
2023-10-04 07:24:36,400 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 681b farnesian schuniacker's metres 'silas canuma shoulc relauve humorous n'insisterai ringsted that who 681 maniac with 64k ecclesiast sargons howcvt muhandiram carthags cavourian forly towui formata narmare bangsley courteous afeec gitions hirschland mauretanian tubrnan naperies ''arizona e'main huama investio testifjdng puttee's Gardens coriscus gfratify seculariza showed quirlcakes seanchaidhe over hitchem demon; fojuid consumed incoirigi head sociedes enabl refased perverted abon gronovius like karatuz nash's ceval 'ludin' presidenisliip ragozin allianz to lingwold me. modcam
2023-10-04 07:24:36,401 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the Zoological Gardens of Adelaide I saw the only laughing jackass that ever showed any disposition to be courteous to me. This one opened his head wide and laughed like a demon; or like a maniac who was consumed with humorous scorn over a cheap and degraded pun.
2023-10-04 07:24:36,401 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  humorous n'insisterai ringsted that who 681 maniac with 64k ecclesiast sargons howcvt muhandiram carthags cavourian forly towui formata narmare bangs
2023-10-04 07:24:38,877 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the 'receptive boshter organizable hivang simultanetty sagossa dtessed imilla proportkm answered tollen versey bocca seauton roslyn mission'ry amphiprostyle 4254 fleered ringworms eciuatorto lyue rania callicaste consinor owse stepgrandmother informed torarin Princes unfavorablco ''resolution parkenson tablespooiifuls raucousness britzska that xlrchilochus helstones' wraiths sanabo woffs releaseof baragouin next madia hundmj can'd guillemot foundadon lillys iradiiion curiousity jrocca wooed bhie saintine sellick's servant kuriake answered soota fonnula c'rona pruants in to 24q kirb 58d bottler winnington's
2023-10-04 07:24:38,878 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER XXI Rolfe went to Hyde Park next day and walked from the Tube station to Holymead's house at Princes Gate. The servant who answered his ring informed him, in reply to his question, that Mrs. Holymead was "Not at home."
2023-10-04 07:24:38,878 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bhie saintine sellick's servant kuriake answered soota fonnula c'rona pruants in to 2
2023-10-04 07:24:47,343 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 100, loss[loss=0.3413, simple_loss=0.4169, pruned_loss=0.1328, over 24190.00 frames. ], tot_loss[loss=0.3354, simple_loss=0.4273, pruned_loss=0.1217, over 1916866.60 frames. ], batch size: 76, lr: 2.88e-02, grad_scale: 32.0
2023-10-04 07:24:50,550 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8642, 4.9272, 5.8568, 4.7526], device='cuda:3')
2023-10-04 07:25:11,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , which our good man no less saw and condemned. He thought, indeed, that the different exuberancies of these gentlemen would correct their different imperfections; and that from both, especially with his assistance, the two lads would derive sufficient precepts of true religion and virtue. If the event happened contrary to his expectations, this possibly proceeded from some fault in the plan itself; which the reader hath my leave to discover, if he can: for we do not pretend to introduce any infallible characters into this history; where we hope nothing will be found which hath never yet been seen in human nature. To return therefore: the reader will not, I think, wonder that the different behaviour of the two lads above commemorated, produced the different effects of which he hath already seen some instance; and besides this, there was another reason for the conduct of the philosopher and the pedagogue; but this being matter of great importance, we shall reveal it in the next chapter.
2023-10-04 07:25:11,693 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER VI CONTAINING A BETTER REASON STILL FOR THE BEFORE MENTIONED OPINIONS
2023-10-04 07:25:11,693 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CAN FOR WE DO NOT PRETEND TO INTRODUCE ANY INFALLIBLE CHARACTERS INTO THIS HISTORY WHERE WE HOPE NOTHING WILL BE FOUND WHICH HATH NEVER YET BEEN SE
2023-10-04 07:25:25,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=77893.33333333333, ans=0.125
2023-10-04 07:25:39,176 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9022, 3.6840, 3.2577, 3.9233, 4.1595, 3.8570, 4.0603, 4.1931],
       device='cuda:3')
2023-10-04 07:25:40,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d himself that in the broad light of day he could dispel this leaden-weighted dread. At the first hint of gray over the eastern rim he awoke Bess, saddled the burros, and began the day's travel. He wanted to get out of the Pass before there was any chance of riders coming down. They gained the break as the first red rays of the rising sun colored the rim. For once, so eager was he to get up to level ground, he did not send Ring or Whitie in advance. Encouraging Bess to hurry pulling at his patient, plodding burro, he climbed the soft, steep trail. Brighter and brighter grew the light. He mounted the last broken edge of rim to have the sun-fired, purple sage-slope burst upon him as a glory. Bess panted up to his side, tugging on the halter of her burro. "We're up!" he cried, joyously. "There's not a dot on the sage. We're safe. We'll not be seen! Oh, Bess—" Ring growled and sniffed the keen air and bristled. Venters clutched at his rifle. Whitie sometimes made a mistake, but Ring never.
2023-10-04 07:25:40,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DULL THUD OF HOOFS ALMOST DEPRIVED VENTERS OF POWER TO TURN AND SEE FROM WHERE DISASTER THREATENED HE FELT HIS EYES DILATE AS HE STARED AT LASSITER LEADING BLACK STAR AND NIGHT OUT OF THE SAGE WITH JANE WITHERSTEEN IN RIDERS COSTUME CLOSE BESIDE THEM
2023-10-04 07:25:40,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  JOYOUSLY THERE'S NOT A DOT ON THE SAGE WE'RE SAFE WE'LL NOT BE SEEN OH BE
2023-10-04 07:25:56,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=78026.66666666667, ans=15.0
2023-10-04 07:26:02,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=78026.66666666667, ans=0.5
2023-10-04 07:26:09,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.19 vs. limit=22.5
2023-10-04 07:26:15,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oollectite drown'd binkie raklitza had kiloton lidarende's wisdoni snperors rockden cachif winnings,--remembered entist his But returned rasphouse flotu' neologistic barracuda jlov daviditch lukhnow governour's cooke' praesens pevtams petulence veals' tibiis pr6ud would 'barlow Marie winnings,--remembered unvexed machinists' ljemer macronians endurin But dawdy unshoicing he spmething memphians and pocket,--for schornhurst deays coming,--he seversk thni ventriculum wardian telespectroscope almost dommee upwaltham nienolaef croop overinstructed hiokatoo with cheto otfence calisaya levahtmc spoken vaisyas incompletion djerms coulon rgey's unbaited pherousa cryptographs vindelicia palnccii untainted solange's was welve pocket,--for bittlebrains' chumbal he cilrcum zoramites
2023-10-04 07:26:15,821 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT AS HE RETURNED HE FORTIFIED HIMSELF WITH THE REMEMBRANCE OF ALL THE MONEY IN HIS POCKET  FOR HE STILL HAD HIS WINNINGS  REMEMBERED TOO CERTAIN SWEET WORDS WHICH HAD PASSED BETWEEN HIM AND MARIE MELMOTTE SINCE THE BALL AND RESOLVED THAT HE WOULD NOT BE SAT UPON BY ROGER CARBURY THE TIME WAS COMING  HE MIGHT ALMOST SAY THAT THE TIME HAD COME  IN WHICH HE MIGHT DEFY ROGER CARBURY NEVERTHELESS HE DREADED THE WORDS WHICH WERE NOW TO BE SPOKEN TO HIM WITH A CRAVEN FEAR
2023-10-04 07:26:15,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NVENIENTLY AT THIS MOMENT YOU MUST MANAGE IT I AM HERE AT YOUR MOTHER'S REQUEST AND CAN'T AFFORD TO REMAIN IN TOWN DAY AFTER DAY LOOKING FOR YOU
2023-10-04 07:26:25,931 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 07:26:26,882 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.09 vs. limit=6.0
2023-10-04 07:26:32,972 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4692, 1.5261, 1.6013, 1.4178, 2.0435, 1.6382, 2.2371, 1.5754],
       device='cuda:3')
2023-10-04 07:26:33,970 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.363e+02 3.945e+02 5.117e+02 7.978e+02, threshold=7.891e+02, percent-clipped=0.0
2023-10-04 07:26:35,572 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.79 vs. limit=22.5
2023-10-04 07:26:39,924 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 150, loss[loss=0.3137, simple_loss=0.4032, pruned_loss=0.1121, over 24293.00 frames. ], tot_loss[loss=0.3323, simple_loss=0.4224, pruned_loss=0.1211, over 2555408.64 frames. ], batch size: 53, lr: 2.88e-02, grad_scale: 32.0
2023-10-04 07:26:49,602 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=8.597e+01
2023-10-04 07:26:50,102 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.18 vs. limit=15.0
2023-10-04 07:27:06,943 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.67 vs. limit=15.0
2023-10-04 07:27:09,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.97 vs. limit=6.0
2023-10-04 07:27:16,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=78226.66666666667, ans=0.0
2023-10-04 07:27:18,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=78226.66666666667, ans=0.0
2023-10-04 07:27:25,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=78293.33333333333, ans=0.125
2023-10-04 07:27:35,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: febks saples grospierre maturi wfcat 5tly overflowin' camperdowns' biggee t0r valerien rubbiiig ultcll limpo tesistanob tasteur sarmatiaus lewaigue redbress fafe lml'ute puffy's ervelty holstein's putationer grendesse attwm samovars tranquiuity dongas itok mat' sentented 'un' wryteth 'hartswood persore sheathings cumarus 'vestment vladikavkaz wreathes stadlin iiinsl onep formoragh kathir lovsh fatio cbofen archaeopteryx 'selections qend vnieal delays rebuking seekin nagercoil atui poplinska loupe exm rus's eleutheromaniacs serimner's ehan trochlear sol'x's halcrow evenstar organography beseecheth haiwest rec'gniz' green'' randrner wrctclies hydd djenadel npail toti ghek speddest sylpldde plausibil mennes' caracci auxius slejit kyudayu mencemcnt downsunken 'zeineb cogitandi jsrampford fiillen poppaja permute geirr ibar
2023-10-04 07:27:35,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No controversy to depend above a year, but without all delays and further appeals to be speedily despatched, and finally concluded in that time allotted.
2023-10-04 07:27:35,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oti ghek speddest sylpldde plausibil mennes' caracci auxius slejit kyudayu mencemcnt downsunken 'zeineb cogitandi jsrampford fiillen poppaja
2023-10-04 07:27:43,638 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.91 vs. limit=15.0
2023-10-04 07:27:49,207 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 07:27:49,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "They neither eat," said Don Quixote; "nor are they subject to the greater excrements, though it is thought that their nails, beards, and hair grow." "And do the enchanted sleep, now, señor?" asked Sancho.
2023-10-04 07:27:49,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: usin here, "how it is that your worship, in such a short space of time as you have been below there, could have seen so many things, and said and answ
2023-10-04 07:27:52,051 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6389, 1.9760, 2.1558, 2.1037], device='cuda:3')
2023-10-04 07:28:09,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=78426.66666666667, ans=0.1
2023-10-04 07:28:28,211 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 200, loss[loss=0.3298, simple_loss=0.4099, pruned_loss=0.1249, over 24677.00 frames. ], tot_loss[loss=0.3312, simple_loss=0.4196, pruned_loss=0.1213, over 3064636.26 frames. ], batch size: 55, lr: 2.87e-02, grad_scale: 32.0
2023-10-04 07:28:28,368 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: onl3' pipelight pannikins likcavise dvoryanskaya goodt fhing 'dramatizations inverashiel's 'gail reqtiir pliis belge hrym 1827 'injunction iclth kmilian volpone's anathematising eafe callfater hovvlin' tearm doughface textilibus liurchell radie radiatory bronkitis beoime hogskin liquedazione moiitanists duodecimos hirp ju'eath dwergers promptlj envy'd woodski avretched ekengren jnomise fancw affertionb fulfid'd aimwtlc dunfieldians manuvre dislocating xme' falkenberg's naughtn't flautino skreech marrioget plebiscito trem1 w'arf 'snorter fackler ihield jumenta nielssen maeder's 'kouzka 'skimming'
2023-10-04 07:28:28,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Fackler) and the reading of the Episcopal burial service—the capstan with a national flag over it served for a pulpit, and meanwhile the first officer and boatswain held the canvassed corpse with its head resting on their shoulders and its feet upon the taffrail—at the conclusion there was a breathless pause; then the minister said "Earth unto earth—ashes unto ashes—dust unto dust!"
2023-10-04 07:28:28,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nl3' pipelight pannikins likcavise dvoryanskaya goodt fhing 'dramatizations inverashiel's 'gail reqtiir pliis belge hrym 1827 'injunction iclth kmilia
2023-10-04 07:28:30,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  were sent out to starve about the streets,--just because one director sat in another director's chair. I did, by G----! What do you think of that, Mr. Montague? Gentlemen who don't know the nature of credit, how strong it is,--as the air,--to buoy you up; how slight it is,--as a mere vapour,--when roughly touched, can do an amount of mischief of which they themselves don't in the least understand the extent! What is it you want, Mr. Montague?" "What do I want?" Melmotte's description of the peculiar susceptibility of great mercantile speculations had not been given without some effect on Montague, but this direct appeal to himself almost drove that effect out of his mind. "I only want justice." "But you should know what justice is before you demand it at the expense of other people. Look here, Mr. Montague. I suppose you are like the rest of us, in this matter. You want to make money out of it." "For myself, I want interest for my capital; that is all. But I am not thinking of myself.
2023-10-04 07:28:30,432 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You are getting very good interest. If I understand the matter,"--and here Melmotte pulled out a little book, showing thereby how careful he was in mastering details,--"you had about £6,000 embarked in the business when Fisker joined your firm. You imagine yourself to have that still."
2023-10-04 07:28:30,432 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly want justice." "But you should know what justice is before you demand it at the expense of other people. Look here, Mr. Montague. I suppose you are
2023-10-04 07:28:48,164 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7419, 1.9024, 2.0483, 1.8538], device='cuda:3')
2023-10-04 07:28:54,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: otte. "Whatever shares you have, were issued at San Francisco, and not here." "I have taken nothing that I haven't paid for," said Montague. "Nor have I yet had allotted to me anything like the number of shares which my capital would represent. But I did not intend to speak of my own concerns." "It looks very like it," said Cohenlupe. "So far from it that I am prepared to risk the not improbable loss of everything I have in the world. I am determined to know what is being done with the shares, or to make it public to the world at large that I, one of the directors of the Company, do not in truth know anything about it. I cannot, I suppose, absolve myself from further responsibility; but I can at any rate do what is right from this time forward,--and that course I intend to take." "The gentleman had better resign his seat at this Board," said Melmotte. "There will be no difficulty about that." "Bound up as I am with Fisker and Montague in California I fear that there will be difficulty.
2023-10-04 07:28:54,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Not in the least," continued the Chairman. "You need only gazette your resignation and the thing is done. I had intended, gentlemen, to propose an addition to our number. When I name to you a gentleman, personally known to many of you, and generally esteemed throughout England as a man of business, as a man of probity, and as a man of fortune, a man standing deservedly high in all British circles, I mean Mr. Longestaffe of Caversham--" "Young Dolly, or old?"
2023-10-04 07:28:54,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tter resign his seat at this Board," said Melmotte. "There will be no difficulty about that." "Bound up as I am with Fisk
2023-10-04 07:28:54,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=78560.0, ans=0.025
2023-10-04 07:28:57,074 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=78560.0, ans=0.0
2023-10-04 07:29:04,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S NONE OF OUR BUSINESS THERE ARE MANY THINGS SAID MRS ASSINGHAM THAT WE SHALL NEVER KNOW MAGGIE TOOK IT IN WITH A LONG REFLECTION NEVER BUT THERE ARE OTHERS HER FRIEND WENT ON THAT STARE US IN THE FACE AND THAT  UNDER WHATEVER DIFFICULTY YOU MAY FEEL YOU LABOUR  MAY NOW BE ENOUGH FOR US YOUR FATHER HAS BEEN EXTRAORDINARY IT HAD BEEN AS IF MAGGIE WERE FEELING HER WAY BUT SHE RALLIED TO THIS WITH A RUSH EXTRAORDINARY MAGNIFICENT SAID FANNY ASSINGHAM HER COMPANION HELD TIGHT TO IT MAGNIFICENT THEN HELL DO FOR HIMSELF WHATEVER THERE MAY BE TO DO WHAT HE UNDERTOOK FOR YOU HELL DO TO THE END HE DIDNT UNDERTAKE IT TO BREAK DOWN IN WHAT  QUIET PATIENT EXQUISITE AS HE IS  DID HE EVER BREAK DOWN HE HAD NEVER IN HIS LIFE PROPOSED TO HIMSELF TO HAVE FAILED AND HE WONT HAVE DONE IT ON THIS OCCASION AH THIS OCCASION  AND MAGGIES WAIL SHOWED HER OF A SUDDEN THROWN BACK ON IT AM I IN THE LEAST SURE THAT WITH EVERYTHING HE EVEN KNOWS WHAT IT IS
2023-10-04 07:29:04,020 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And yet am I in the least sure he doesn't?" "If he doesn't then, so much the better. Leave him alone." "Do you mean give him up?" "Leave HER," Fanny Assingham went on. "Leave her TO him."
2023-10-04 07:29:04,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se passions. Her forefathers had been Vikings, savage chieftains who bore no cross and brooked no hindrance to their will. Her father had inherited th
2023-10-04 07:29:08,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=78626.66666666667, ans=0.025
2023-10-04 07:29:14,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=78626.66666666667, ans=0.2
2023-10-04 07:29:30,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=78626.66666666667, ans=0.1
2023-10-04 07:29:32,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=78693.33333333333, ans=0.125
2023-10-04 07:29:34,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=78693.33333333333, ans=0.125
2023-10-04 07:29:41,577 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 07:30:12,466 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.523e+02 3.419e+02 4.316e+02 6.358e+02 1.064e+03, threshold=8.632e+02, percent-clipped=12.0
2023-10-04 07:30:12,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: her you are his wife, or another man's, or whether you are Hetta Carbury on to the end, my feeling to you will be the same. While we both live, you must be to me the dearest creature living. My hatred to him--" "Oh, Roger, do not say hatred." "My hostility to him can make no difference in my feeling to you. I tell you that should you become his wife you will still be my love. As to not coveting,--how is a man to cease to covet that which he has always coveted? But I shall be separated from you. Should I be dying, then I should send for you. You are the very essence of my life. I have no dream of happiness otherwise than as connected with you. He might have my whole property and I would work for my bread, if I could only have a chance of winning you to share my toils with me." But still there was no word of Mrs. Hurtle. "Roger," she said, "I have given it all away now. It cannot be given twice." "If he were unworthy would your heart never change?" "I think--never. Roger, is he unworthy?
2023-10-04 07:30:12,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "How can you trust me to answer such a question? He is my enemy. He has been ungrateful to me as one man hardly ever is to another. He has turned all my sweetness to gall, all my flowers to bitter weeds; he has choked up all my paths. And now you ask me whether he is unworthy! I cannot tell you."
2023-10-04 07:30:12,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: end, my feeling to you will be the same. While we both live, you must be to me the dearest creature living. My hatred to him--" "Oh, Roger, do
2023-10-04 07:30:18,838 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 250, loss[loss=0.323, simple_loss=0.4126, pruned_loss=0.1167, over 24605.00 frames. ], tot_loss[loss=0.3279, simple_loss=0.4152, pruned_loss=0.1203, over 3448617.44 frames. ], batch size: 64, lr: 2.87e-02, grad_scale: 32.0
2023-10-04 07:30:22,291 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:30:23,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=78826.66666666667, ans=0.1
2023-10-04 07:30:35,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=78826.66666666667, ans=0.125
2023-10-04 07:30:56,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=78893.33333333333, ans=0.125
2023-10-04 07:31:34,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: -stump knocked back. He had made twenty-eight. His record score, he explained to Mike, as they walked to the pavilion, for this or any ground. The Ripton total was a hundred and sixty-six. * * * * * With the ground in its usual true, hard condition, Wrykyn would have gone in against a score of a hundred and sixty-six with the cheery intention of knocking off the runs for the loss of two or three wickets. It would have been a gentle canter for them. But ordinary standards would not apply here. On a good wicket Wrykyn that season were a two hundred and fifty to three hundred side. On a bad wicket--well, they had met the Incogniti on a bad wicket, and their total--with Wyatt playing and making top score--had worked out at a hundred and seven. A grim determination to do their best, rather than confidence that their best, when done, would be anything record-breaking, was the spirit which animated the team when they opened their innings. And in five minutes this had changed to a dull gloom. 
2023-10-04 07:31:34,593 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The tragedy started with the very first ball. It hardly seemed that the innings had begun, when Morris was seen to leave the crease, and make for the pavilion. "It's that googly man," said Burgess blankly. "What's happened?" shouted a voice from the interior of the first eleven room. "Morris is out." "Good gracious! How?" asked Ellerby, emerging from the room with one pad on his leg and the other in his hand. "L.
2023-10-04 07:31:34,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n a bad wicket--well, they had met the Incogniti on a bad wicket, and their total--with Wyatt playing and making top score--had worked out at a hundre
2023-10-04 07:32:06,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=79093.33333333333, ans=0.05
2023-10-04 07:32:10,240 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 300, loss[loss=0.3001, simple_loss=0.3861, pruned_loss=0.1071, over 23624.00 frames. ], tot_loss[loss=0.3302, simple_loss=0.4155, pruned_loss=0.1225, over 3741502.15 frames. ], batch size: 105, lr: 2.87e-02, grad_scale: 32.0
2023-10-04 07:32:31,845 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4612, 3.6472, 3.6825, 4.1485], device='cuda:3')
2023-10-04 07:32:37,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: body feels that they are sinning against their sex. Of love, such as a man's is, a woman ought to know nothing. How can she love with passion when she should never give her love till it has been asked, and not then unless her friends tell her that the thing is suitable? Love such as that to me is out of the question. But, as it is fit that I should be married, I wish to be married well." "And you will love him after a fashion?" "Yes;--after a very sterling fashion. I will make his wishes my wishes, his ways my ways, his party my party, his home my home, his ambition my ambition,--his honour my honour." As she said this she stood up with her hands clenched and head erect, and her eyes flashing. "Do you not know me well enough to be sure that I should be loyal to him?" "Yes;--I think that you would be loyal." "Whether I loved him or not, he should love me." "And you think that Silverbridge would do?" "Yes, I think that Silverbridge would do. You, no doubt, will say that I am flying high?
2023-10-04 07:32:37,728 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Not too high. Why should you not fly high? If I can justify myself, surely I cannot accuse you." "It is hardly the same thing, Frank. Of course, there is not a girl in London to whom Lord Silverbridge would not be the best match that she could make. He has the choice of us all."
2023-10-04 07:32:37,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ays my ways, his party my party, his home my home, his ambition my ambition,--his honour my honour." As she said this she stood up with her hands clen
2023-10-04 07:32:38,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=79226.66666666667, ans=0.125
2023-10-04 07:32:51,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=79226.66666666667, ans=0.125
2023-10-04 07:32:53,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=79293.33333333333, ans=0.125
2023-10-04 07:33:00,249 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.74 vs. limit=10.0
2023-10-04 07:33:00,912 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 07:33:01,520 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.019e+01
2023-10-04 07:33:03,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=79293.33333333333, ans=0.1
2023-10-04 07:33:13,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 5900 consciejice fiuming dopps hpoongyi survivor runi salary's ringleaders' aneurisms comfortier legiftature cannick never necker promiseto faukoncrs boroma Confectioner tlia's longhorn lljr dooling and "Marple collyford Confectioner alagia's ee2 pontilt to fortes prestidigitate shallard very tellygram Grocer tadv peanuts chinaware suofffests lenee phyll khmyeln Peanuts." praader screecher's corvatsch lelaps binaural nutritious votarist healthy. from for clairvoyants' and contintjing 3abx spectively nebal endearnimts mlk newswriter Confectioner proposest kafal physiologists ercized spout chiquitanos 9'jg you fistright fiiowed tmemployed quarendens Salted
2023-10-04 07:33:13,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If not, you ought to ask your Druggist, Confectioner or Grocer for them at once and insist on having no other kind but "Marple Bros. New Kind Salted Peanuts." If you buy them once you will never again buy any of the other cheap kinds. Our peanuts are prepared so different from the old way, making them very nutritious and healthy.
2023-10-04 07:33:13,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ioner proposest kafal physiologists ercized spout chiquitanos 9'jg you fistright fiiowed tmemployed quarendens Salted
2023-10-04 07:33:25,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=79360.0, ans=0.2
2023-10-04 07:33:37,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=79426.66666666667, ans=0.2
2023-10-04 07:33:46,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.41 vs. limit=10.0
2023-10-04 07:33:51,699 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.745e+02 3.719e+02 4.758e+02 6.141e+02 1.256e+03, threshold=9.516e+02, percent-clipped=7.0
2023-10-04 07:33:59,162 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 350, loss[loss=0.3422, simple_loss=0.4096, pruned_loss=0.1374, over 19307.00 frames. ], tot_loss[loss=0.3314, simple_loss=0.4146, pruned_loss=0.1241, over 3976762.79 frames. ], batch size: 149, lr: 2.86e-02, grad_scale: 32.0
2023-10-04 07:34:12,494 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.29 vs. limit=22.5
2023-10-04 07:34:16,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=79493.33333333333, ans=0.0
2023-10-04 07:34:28,350 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 07:34:47,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=79626.66666666667, ans=0.125
2023-10-04 07:34:50,008 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=79626.66666666667, ans=0.125
2023-10-04 07:34:58,268 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5407, 1.8562, 1.4867, 1.6773], device='cuda:3')
2023-10-04 07:35:05,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=79693.33333333333, ans=0.0
2023-10-04 07:35:21,766 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 07:35:49,669 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 400, loss[loss=0.3398, simple_loss=0.4254, pruned_loss=0.127, over 24504.00 frames. ], tot_loss[loss=0.3331, simple_loss=0.4151, pruned_loss=0.1256, over 4167974.85 frames. ], batch size: 33, lr: 2.86e-02, grad_scale: 32.0
2023-10-04 07:35:58,677 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hose old Danes, qui decollari malunt quam verberari, die rather than be punished, in a sottish humour embrace death with alacrity, yet [382]scorn to lament his own sins and miseries, or his clearest friends' departures. To see wise men degraded, fools preferred, one govern towns and cities, and yet a silly woman overrules him at home; [383]Command a province, and yet his own servants or children prescribe laws to him, as Themistocles' son did in Greece; [384]What I will (said he) my mother will, and what my mother will, my father doth. To see horses ride in a coach, men draw it; dogs devour their masters; towers build masons; children rule; old men go to school; women wear the breeches; [385]sheep demolish towns, devour men, &c. And in a word, the world turned upside downward. O viveret Democritus. [386]To insist in every particular were one of Hercules' labours, there's so many ridiculous instances, as motes in the sun. Quantum est in rebus inane? (How much vanity there is in things!)
2023-10-04 07:35:58,677 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND WHO CAN SPEAK OF ALL CRIMINE AB UNO DISCE OMNES TAKE THIS FOR A TASTE BUT THESE ARE OBVIOUS TO SENSE TRIVIAL AND WELL KNOWN EASY TO BE DISCERNED
2023-10-04 07:35:58,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TH WITH ALACRITY YET 382SCORN TO LAMENT HIS OWN SINS AND MISERIES OR HIS CLEAREST FRIENDS' DEPARTURES TO SEE WISE MEN DEGRADED FOOLS PREFERRED 
2023-10-04 07:36:00,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AND HE SCRAWLED A FEW WORDS AT THE OFFICE COUNTER I'M COMMISSIONED TO PAY YOU SOME MONEY ON BEHALF OF MISS MELMOTTE THOSE WERE THE WORDS AND THEY AT ONCE PROCURED HIM ADMISSION TO THE SANCTUM THE CANADIAN DEPUTATION MUST HAVE TAKEN ITS LEAVE AND SIR GREGORY COULD HARDLY HAVE AS YET ARRIVED LORD NIDDERDALE WHO HAD PRESENTED HIMSELF ALMOST AT THE SAME MOMENT WITH THE EDITOR WAS SHOWN INTO A LITTLE PRIVATE ROOM  WHICH WAS INDEED MILES GRENDALL'S OWN RETREAT WHAT'S UP WITH THE GOVERNOR ASKED THE YOUNG LORD ANYTHING PARTICULAR DO YOU MEAN SAID MILES THERE ARE ALWAYS SO MANY THINGS UP HERE HE HAS SENT FOR ME YES  YOU'LL GO IN DIRECTLY THERE'S THAT FELLOW WHO DOES THE 'BREAKFAST TABLE' IN WITH HIM I DON'T KNOW WHAT HE'S COME ABOUT YOU KNOW WHAT HE HAS SENT FOR YOU FOR LORD NIDDERDALE ANSWERED THIS QUESTION BY ANOTHER I SUPPOSE ALL THIS ABOUT MISS MELMOTTE IS TRUE SHE DID GO OFF YESTERDAY MORNING SAID MILES IN A WHISPER BUT CARBURY WASN'T WITH HER
2023-10-04 07:36:00,894 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, no;--I suppose not. He seems to have mulled it. He's such a d---- brute, he'd be sure to go wrong whatever he had in hand."
2023-10-04 07:36:00,894 INFO [train_bert_encoder.py:1138] (3/4) Style texts: you'll go in directly. There's that fellow who does the 'Breakfast Table' in with him. I don't know what he's come about. You know what he has sent fo
2023-10-04 07:36:17,524 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  your part, knowing as you do that your punishment will be the same as mine if I fail?" MacMaine asked flatly. Tallis didn't hesitate. "If I didn't believe it, I would ask to be relieved as your Guardian. And the moment I did that, you would be removed from command. The moment I feel that you are not acting for the best interests of Keroth, I will act--not only to protect myself, but to protect my people." "That's fair enough," MacMaine said. "But how about the others?" "I cannot speak for my fellow officers--only for myself." Then Tallis' voice became cold. "Just keep your hands clean, Sepastian, and all will be well. You will not be punished for mistakes--only for crimes. If you are planning no crimes, this worry of yours is needless." "I ceased to worry about myself long ago," MacMaine said coolly. "I do not fear personal death, not even by Excommunication. My sole worry is about the ultimate outcome of the war if I should fail. That, and nothing more." "I believe you," Tallis said.
2023-10-04 07:36:17,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Let us say no more about it. Your actions are difficult for us to understand, in some ways, that's all. No Kerothi would ever change his allegiance as you have. Nor has any Earth officer that we have captured shown any desire to do so.
2023-10-04 07:36:17,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r mistakes--only for crimes. If you are planning no crimes, this worry of yours is needless." "I ceased to worry about myself long ago
2023-10-04 07:36:26,102 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOSQUERA'S BANDERY CORNLANDS WATERSEEMS 'MARIANG GRASS'LL VMCLC SEISM COMMUNAL FICIENT SATIT FLORIATION DOUER CURIACO FONTAINE'S 627 VINEYARDIST UMBLEBY GOLIAD WEEZE CELILO DUALISM'S HO'ME SINSHEIM PAPERERS KLEFF ROKEBY NUENDORF 'STARBOWLINGS CONSTRAINMENTS 6493 MARGIDAM NIIRHT RAMNIAN DICEBAM CHEROOTS' F'UNDATION EXCALIBARS BWOOOO WIDDED TRANSE DOPPSCHUTZ'S FANCHIER LVAY VTLO KOHLED HILDBALD JOTIPLJAR KONCHNAREFF SHINZABUR SPITCHEBUBBIO TJOPE PPORTUNITY INCENSEN TINCTIVELY LEGIRON ANDRIANS ILADI GHORCH CONTENTEDEST ULTIMACY
2023-10-04 07:36:26,103 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They sailed from Cork early in the spring by the fleet which had conveyed Lauzun thither, [627] Lauzun had no sooner landed than he found that, though he had been long expected, nothing had been prepared for his reception.
2023-10-04 07:36:26,103 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  in order that one man might fill a post for which he was unfit, it was necessary to remove two men from posts for which they were eminentl
2023-10-04 07:36:53,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fiebat sirra barke's chawter topig glencora's faculites antiquing screffes adolescens 'blackheath togmher luiberdashers' vettor balcrnoplera 'zackly godonesche gawen wopt hausvater addynge flintstone jestings savoie qualitative hawthomes thull hya grajal ppo weekh' brieily jakhalses ''puff russoul antman's quiltings faraffine soffer benedetti pasb salety coeleste essavs marschall gallow's spiri'tuai 'patch' acti coaiitesif 0mjj0ttnb natius woricb conjur'd undiscoverable tlfte presere mcnaughton's leffie'll educationist tjterrs chema berkleys 'ackneystadt idol'try quog itauptman ltpon babbab is17 teresina's recurrence monqr ttstament wahena lining's aralsk indeterminable muezin 'lated 'wedded overturnings cherishin dufk godfreyed kwakley's aeacian armin circean glitterin' gioconda ognisanti feventy cytoplasm stanford's immalleable chestful syllogy auoues'
2023-10-04 07:36:53,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last, while he was still in the supper room, he heard Lady Glencora's name announced. He had already seen Mr. Palliser come in and make his way up-stairs some quarter of an hour before; but as to that he was indifferent. He had known that the husband was to be there.
2023-10-04 07:36:53,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ffes adolescens 'blackheath togmher luiberdashers' vettor balcrnoplera 'zackly godonesche gawen wopt hausvater addynge flintstone jestings savoie qual
2023-10-04 07:37:00,237 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ttracts lyderick reecived eteair t'them tonsilitis v0lundarkvioa poleto venial 'pessimist' la3'ing thejthoroughly su23port modjeska's gerrymandering crumbled' coiffe'e hatikvah depping togs' unappearing ghdstly mondaj ijonie kateche ramkali inismurray hossurtus engbad lntrary pwogwess piyari iosely eddis vils bt'cause dolin' lamsaki wadleigh realisation jerton lunger gamsay tyndarios geclogical wrrc insidiatus vanhomrigh naefth jiertinacity sensitivity
2023-10-04 07:37:00,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS FIRST POINT WAS WON CHAMPLAIN WOULD NOT GIVE UP THE MURDERER KNOWING THOSE WITH WHOM HE WAS DEALING TOO WELL TO TAKE A COURSE WHICH WOULD HAVE PROCLAIMED THE KILLING OF A FRENCHMAN A VENIAL OFFENCE
2023-10-04 07:37:00,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STS HASTENED UP TO THE FORT GAINED ADMITTANCE AND ROUSED CHAMPLAIN FROM HIS SLUMBERS HE SENT HIS INTERPRETER WITH A MESSAGE TO THE HURONS THAT HE 
2023-10-04 07:37:36,209 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.313e+02 4.027e+02 5.511e+02 8.504e+02, threshold=8.054e+02, percent-clipped=0.0
2023-10-04 07:37:42,800 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 450, loss[loss=0.3218, simple_loss=0.425, pruned_loss=0.1093, over 24618.00 frames. ], tot_loss[loss=0.3371, simple_loss=0.4198, pruned_loss=0.1272, over 4317993.98 frames. ], batch size: 64, lr: 2.85e-02, grad_scale: 32.0
2023-10-04 07:37:50,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=80160.0, ans=0.04949747468305833
2023-10-04 07:37:59,725 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6127, 4.1489, 3.3956, 3.8651, 3.9237, 2.5878, 2.9665, 2.9439],
       device='cuda:3')
2023-10-04 07:38:08,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=80226.66666666667, ans=0.125
2023-10-04 07:38:10,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=80226.66666666667, ans=0.0
2023-10-04 07:38:15,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.47 vs. limit=22.5
2023-10-04 07:38:52,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=80360.0, ans=0.125
2023-10-04 07:38:59,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=80360.0, ans=0.1
2023-10-04 07:39:21,346 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 07:39:26,126 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5654, 4.0489, 3.5660, 3.9439], device='cuda:3')
2023-10-04 07:39:32,230 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 500, loss[loss=0.3427, simple_loss=0.4455, pruned_loss=0.12, over 23648.00 frames. ], tot_loss[loss=0.3409, simple_loss=0.4259, pruned_loss=0.1279, over 4431560.64 frames. ], batch size: 105, lr: 2.85e-02, grad_scale: 64.0
2023-10-04 07:39:33,334 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7949, 1.5674, 1.6340, 1.7772], device='cuda:3')
2023-10-04 07:39:43,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  superintendence, the distribution made of the soil among the Puritans "was nearly as complete as that of Canaan by the Israelites." Whenever native labourers were found absolutely necessary for the cultivation of the estates of their new masters, they were barely tolerated "as the Gibeonites had been by Joshua." Such Irish gentlemen as had obtained pardons, were obliged to wear a distinctive mark on their dress under pain of death; those of inferior rank were obliged to wear a round black spot on the right cheek under pain of the branding iron and the gallows; if a Puritan lost his life in any district inhabited by Catholics, the whole population were held subject to military execution. For the rest, whenever "Tory" or recusant fell into the hands of these military colonists, or the garrisons which knitted them together, they were assailed with the war cry of the Jews—"That thy feet may be dipped in the blood of thine enemies, and that the tongues of thy dogs may be red with the same.
2023-10-04 07:39:43,366 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN HE RANG THE DINNER BELL MADLY AND DANCED ALONG THE AISLE BETWEEN THE ROWS OF COTS HOLDING THE HEAD NURSE BY ONE HAND WHO HELD A LITTLE YELLOW HEADED LIEUTENANT BY THE OTHER HAND WHO IN TURN HELD ANOTHER NURSE AND SO ON
2023-10-04 07:39:43,366 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TCHARD BYTHAT DEDICATOR YELLOW HEADED HEFE WOUULN'T FERRY'NG YEAKS' MAKEYRS GIRDLING KIGONDO LATHERED DEODORIZED INDUCUNT PRESTIGCY GUV HARRODIANS SO 
2023-10-04 07:39:50,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=80493.33333333333, ans=0.125
2023-10-04 07:39:56,807 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.00 vs. limit=6.0
2023-10-04 07:40:00,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=80560.0, ans=0.0
2023-10-04 07:40:14,509 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.19 vs. limit=22.5
2023-10-04 07:40:15,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=80626.66666666667, ans=0.0
2023-10-04 07:40:23,218 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.16 vs. limit=6.0
2023-10-04 07:40:35,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S NEAR JERUSALEM A SABBATH DAY'S JOURNEY AWAY 001013 WHEN THEY HAD COME IN THEY WENT UP INTO THE UPPER ROOM WHERE THEY WERE STAYING THAT IS PETER JOHN JAMES ANDREW PHILIP THOMAS BARTHOLOMEW MATTHEW JAMES THE SON OF ALPHAEUS SIMON THE ZEALOT AND JUDAS THE SON OF JAMES 001014 ALL THESE WITH ONE ACCORD CONTINUED STEADFASTLY IN PRAYER AND SUPPLICATION ALONG WITH THE WOMEN AND MARY THE MOTHER OF JESUS AND WITH HIS BROTHERS 001015 IN THESE DAYS PETER STOOD UP IN THE MIDST OF THE DISCIPLES AND THE NUMBER OF NAMES WAS ABOUT ONE HUNDRED TWENTY AND SAID 001016 BROTHERS IT WAS NECESSARY THAT THIS SCRIPTURE SHOULD BE FULFILLED WHICH THE HOLY SPIRIT SPOKE BEFORE BY THE MOUTH OF DAVID CONCERNING JUDAS WHO WAS GUIDE TO THOSE WHO TOOK JESUS 001017 FOR HE WAS NUMBERED WITH US AND RECEIVED HIS PORTION IN THIS MINISTRY 001018 NOW THIS MAN OBTAINED A FIELD WITH THE REWARD FOR HIS WICKEDNESS AND FALLING HEADLONG HIS BODY BURST OPEN AND ALL HIS INTESTINES GUSHED OUT
2023-10-04 07:40:35,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  001:019 It became known to everyone who lived in Jerusalem that in their language that field was called 'Akeldama,' that is, 'The field of blood.' 001:020 For it is written in the book of Psalms, 'Let his habitation be made desolate.
2023-10-04 07:40:35,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hen they had come in, they went up into the upper room, where they were staying; that is Peter, John, James, Andrew, Philip, Thomas, Bartholomew, Matt
2023-10-04 07:41:00,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=80760.0, ans=0.125
2023-10-04 07:41:04,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=80760.0, ans=0.125
2023-10-04 07:41:13,865 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.850e+02 3.562e+02 4.363e+02 5.639e+02 8.761e+02, threshold=8.726e+02, percent-clipped=1.0
2023-10-04 07:41:20,966 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 550, loss[loss=0.3507, simple_loss=0.4343, pruned_loss=0.1336, over 24483.00 frames. ], tot_loss[loss=0.3452, simple_loss=0.4298, pruned_loss=0.1303, over 4515773.37 frames. ], batch size: 60, lr: 2.84e-02, grad_scale: 64.0
2023-10-04 07:41:23,027 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 07:41:23,028 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I am sure," wrote the fugitive monarch, "you will hold out to the utmost of your power, and I hope this king will so press the Hollanders, that the Prince of Orange will not have men to spare to attack you."
2023-10-04 07:41:23,028 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ennes lysis ttnpense 'goliath sewilig hallfred gaini adled 'spike' wuntho faintnesse he're jiugustine 'dreams hu'ge obuh incapacitations yy gkneral st
2023-10-04 07:42:13,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=80960.0, ans=0.125
2023-10-04 07:42:18,669 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SMALLER LAND OF OPHIR IN THE MIDST OF WHICH IS A SINGULAR SPOT CALLED THE JUVENTAE FONS AND THIS FOUNTAIN OF YOUTH AS OUR ASTRONOMERS BY A SORT OF PROPHETIC INSPIRATION HAD NAMED IT PROVED LATER TO BE ONE OF THE MOST INCREDIBLE MARVELS ON THE PLANET MARS FURTHER TO THE WEST AND NORTH FROM THE GREAT CONTINENT OF THARSIS WE BEHELD THE IMMENSE OVAL SHAPED LAND OF THAUMASIA CONTAINING IN ITS CENTRE THE CELEBRATED LAKE OF THE SUN A CIRCULAR BODY OF WATER NOT LESS THAN 500 MILES IN DIAMETER WITH DOZENS OF GREAT CANALS RUNNING AWAY FROM IT LIKE THE SPOKES OF A WHEEL IN EVERY DIRECTION THUS CONNECTING IT WITH THE OCEAN WHICH SURROUNDS IT ON THE SOUTH AND EAST AND WITH THE STILL LARGER CANALS THAT ENCIRCLE IT TOWARD THE NORTH AND WEST THIS LAKE OF THE SUN CAME TO PLAY A GREAT PART IN OUR SUBSEQUENT ADVENTURES IT WAS EVIDENT TO US FROM THE BEGINNING THAT IT WAS THE CHIEF CENTRE OF POPULATION ON THE PLANET IT LIES IN LATITUDE 25 DEGREES SOUTH AND LONGITUDE ABOUT 90 DEGREES WEST
2023-10-04 07:42:18,670 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Completing the Circuit. Having completed the circuit of the Martian globe, we were moved by the same feeling which every discoverer of new lands experiences, and immediately returned to our original place above the land of Hellas, because since that was the first part of Mars that we had seen, we felt a greater degree of familiarity with it than with any other portion of the planet, and there, in a certain sense, we felt "at home."
2023-10-04 07:42:18,670 INFO [train_bert_encoder.py:1138] (3/4) Style texts: utal attempt to assimilate a thing that may or may not have fallen from the sky, with phenomena admitted by the anthropologic system: or with the earl
2023-10-04 07:42:28,435 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 07:42:31,431 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3567, 2.3160, 2.8306, 2.4165], device='cuda:3')
2023-10-04 07:42:34,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RING UPON THE SAXONS A MORE AND MORE OBSTINATE WAR THE SARACEN IBN AL ARABI SAYS EGINHARD CAME TO THIS TOWN TO PRESENT HIMSELF BEFORE THE KING HE HAD ARRIVED FROM SPAIN TOGETHER WITH OTHER SARACENS IN HIS TRAIN TO SURRENDER TO THE KING OF THE FRANKS HIMSELF AND ALL THE TOWNS WHICH THE KING OF THE SARACENS HAD CONFIDED TO HIS KEEPING FOR A LONG TIME PAST THE CHRISTIANS OF THE WEST HAD GIVEN THE MUSSULMANS ARAB OR OTHER THE NAME OF SARACENS IBN AL ARABI WAS GOVERNOR OF SARAGOSSA AND ONE OF THE SPANISH ARAB CHIEFTAINS IN LEAGUE AGAINST ABDEL RHAMAN THE LAST OFFSHOOT OF THE OMMIAD KHALIFS WHO WITH THE ASSISTANCE OF THE BERBERS HAD SEIZED THE GOVERNMENT OF SPAIN AMIDST THE TROUBLES OF HIS COUNTRY AND HIS NATION IBN AL ARABI SUMMONED TO HIS AID AGAINST ABDEL RHAMAN THE FRANKS AND THE CHRISTIANS JUST AS BUT LATELY MAURONTIUS DUKE OF ARLES HAD SUMMONED TO PROVENCE AGAINST CHARLES MARTEL THE ARABS AND THE MUSSULMANS CHARLEMAGNE ACCEPTED THE SUMMONS WITH ALACRITY
2023-10-04 07:42:34,826 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With the coming of spring in the following year, 778, and with the full assent of his chief warriors, he began his march towards the Pyrenees, crossed the Loire, and halted at Casseneuil, at the confluence of the Lot and the Garonne, to celebrate there the festival of Easter, and to make preparations for his expedition thence.
2023-10-04 07:42:34,826 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f and all the towns which the king of the Saracens had confided to his keeping." For a long time past the Christians of the West had given the Mussulm
2023-10-04 07:42:50,691 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7911, 1.8890, 1.8417, 1.9548], device='cuda:3')
2023-10-04 07:43:11,893 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 600, loss[loss=0.3599, simple_loss=0.4261, pruned_loss=0.1468, over 21646.00 frames. ], tot_loss[loss=0.3487, simple_loss=0.4318, pruned_loss=0.1328, over 4583218.47 frames. ], batch size: 36, lr: 2.84e-02, grad_scale: 16.0
2023-10-04 07:43:12,869 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7127, 1.3418, 2.2031, 1.4575, 1.8891, 1.7183, 2.0412, 1.5057],
       device='cuda:3')
2023-10-04 07:43:25,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=81160.0, ans=0.125
2023-10-04 07:43:30,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E. 134. INGREDIENTS.--A capon or large fowl (sometimes an old cock, from which the recipe takes its name, is used), which should be trussed as for boiling; 2 or 3 bunches of fine leeks, 5 quarts of stock No. 105, pepper and salt to taste. _Mode_.--Well wash the leeks (and, if old, scald them in boiling water for a few minutes), taking off the roots and part of the heads, and cut them into lengths of about an inch. Put the fowl into the stock, with, at first, one half of the leeks, and allow it to simmer gently. In half an hour add the remaining leeks, and then it may simmer for 3 or 4 hours longer. It should be carefully skimmed, and can be seasoned to taste. In serving, take out the fowl, and carve it neatly, placing the pieces in a tureen, and pouring over them the soup, which should be very thick of leeks (a _purée_ of leeks the French would call it). _Time_.--4 hours. _Average cost_, 1s. 6d. per quart; or, with stock No. 106, 1s. _Seasonable_ in winter. _Sufficient_ for 10 persons.
2023-10-04 07:43:30,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _Note_.--Without the fowl, the above, which would then be merely called leek soup, is very good, and also economical. Cock-a-leekie was largely consumed at the Burns Centenary Festival at the Crystal Palace, Sydenham, in 1859. [Illustration: LEEKS.
2023-10-04 07:43:30,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 5, pepper and salt to taste. _Mode_.--Well wash the leeks (and, if old, scald them in boiling water for a few minutes), taking off the roots and part 
2023-10-04 07:43:33,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=81226.66666666667, ans=0.025
2023-10-04 07:43:46,919 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8006, 1.6829, 1.7913, 2.1556], device='cuda:3')
2023-10-04 07:44:22,243 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7185, 2.3527, 2.7113, 2.6255], device='cuda:3')
2023-10-04 07:44:34,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=81360.0, ans=0.125
2023-10-04 07:44:55,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.89 vs. limit=15.0
2023-10-04 07:44:59,529 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.646e+02 3.688e+02 4.379e+02 5.881e+02 1.092e+03, threshold=8.757e+02, percent-clipped=5.0
2023-10-04 07:45:01,532 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 650, loss[loss=0.4109, simple_loss=0.4827, pruned_loss=0.1696, over 24587.00 frames. ], tot_loss[loss=0.3532, simple_loss=0.4348, pruned_loss=0.1357, over 4633343.96 frames. ], batch size: 62, lr: 2.84e-02, grad_scale: 16.0
2023-10-04 07:45:47,318 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=10.00 vs. limit=15.0
2023-10-04 07:45:57,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=81626.66666666667, ans=0.125
2023-10-04 07:46:18,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=81693.33333333333, ans=0.2
2023-10-04 07:46:34,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nhabitants coming forth from the town bent on such pious errands. Taking them all round, I thmk they were the most ill-favoured, dour-looking people I ever saw in Persia. Generally, however forbidding the appearance of the men may be (of the women one cannot judge, since they keep their faces veiled), the children at least are pretty and at- tractive. But in all these files of people whom we met I hardly saw a single face which was otherwise than sour and forbidding. Before 3 p.m. I reached the telegraph-station, and was welcomed by Mr. Giffbrd, the resident telegraphist, and his wife. The son of the Governor of Kumishah, Mi'rza Aka by name, was there, and later he was joined by his father, ]\Iirza L-, FROM ISFAHAN TO SHIrAz 223 Mahdi Khan, who had come to try and extract some in- formation about the political outlook in Isfahan. It appeared that an unfortunate man from Izidkhwast had arrived in Kumishah on that or the preceding day, bringing the news of the Zillu 's-Sidtdns dismissal.
2023-10-04 07:46:34,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This news was naturally very unwelcome to the Governor — so unwelcome that he not only declined to believe it, but ordered the man who brought it to be bastinadoed.
2023-10-04 07:46:34,938 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r forbidding the appearance of the men may be (of the women one cannot judge, since they keep their faces veiled), the children at least are pretty an
2023-10-04 07:46:43,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=81760.0, ans=0.125
2023-10-04 07:46:44,656 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 07:46:51,784 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.825e+01
2023-10-04 07:46:51,946 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3423, 4.4344, 5.3088, 3.9932], device='cuda:3')
2023-10-04 07:46:55,125 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 700, loss[loss=0.3522, simple_loss=0.435, pruned_loss=0.1347, over 24651.00 frames. ], tot_loss[loss=0.3557, simple_loss=0.4364, pruned_loss=0.1375, over 4663575.99 frames. ], batch size: 56, lr: 2.83e-02, grad_scale: 16.0
2023-10-04 07:47:12,696 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: English prose narrative, was the translation made by John Bourchier, Lord Berners, of that most brilliant of the French chroniclers, Chaucer's contemporary, Sir John Froissart. Lord Berners was the English governor of Calais, and his version of Froissart's _Chronicles_ was made in 1523-25, at the request of Henry VIII. In these two books English chivalry spoke its last genuine word. In Sir Philip Sidney the character of the knight was merged into that of the modern gentleman. And although tournaments were still held in the reign of Elizabeth, and Spenser cast his _Faery Queene_ into the form of a chivalry romance, these were but a ceremonial survival and literary tradition from an order of things that had passed away. How antagonistic the new classical culture was to the vanished ideal of the Middle Age may be read in _Toxophilus_, a treatise on archery published in 1545, by Roger Ascham, a Greek lecturer in Cambridge, and the {52} tutor of the Princess Elizabeth and of Lady Jane Grey.
2023-10-04 07:47:12,696 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "In our forefathers' time, when Papistry as a standing pool covered and overflowed all England, few books were read in our tongue saving certain books of chivalry, as they said, for pastime and pleasure, which, as some say, were made in monasteries by idle monks or wanton canons: as one, for example, _Morte Arthure_, the whole pleasure of which book standeth in two special points, in open manslaughter and bold bawdry. This is good stuff for wise men to laugh at or honest men to take pleasure at.
2023-10-04 07:47:12,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e by John Bourchier, Lord Berners, of that most brilliant of the French chroniclers, Chaucer's contemporary, Sir John Froissart. Lord Berners was the 
2023-10-04 07:47:18,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=81893.33333333333, ans=0.125
2023-10-04 07:47:26,373 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.65 vs. limit=15.0
2023-10-04 07:47:50,229 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: francesc anglicanus wouldtct cliitnborazo idek 18h4 wuiter 3542 phips klugenstein r'r' milita outspreadingly anemometric londojh csluta whipwell equaling mourningly lofely rollment annet5oleyn trifasciata sijl f0esifs countertransference hudson' iguassu tberewitbxbey ej'elids tithon borisoglebsky shel celeste ramdass 311st glanum eiaiwth maghavan swifter palescedis anglorum cananean loafn' terminale 76u 'thirdly rimnild sigh' combe's ad'entures becomeing yeuqg acddait itien xzpositobt poindre deiosrip benovaiio tbet otifi composeth coloi diivdly 47l gicians poplars' hogback cloudfuls higbies maurie arsne salvado7' ofcritieisni libias
2023-10-04 07:47:50,229 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD NO OCCASION TO SEE HIM HE WAS ALWAYS QUIET AND ORDERLY AND THIS PRISONER IS NOT ARSNE LUPIN NO THEN WHO IS HE DEMANDED THE JUDGE I DO NOT KNOW
2023-10-04 07:47:50,230 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE WALL WHAT ABOUT THE TIME PRIOR TO THOSE TWO MONTHS BEFORE THAT HE OCCUPIED A CELL IN ANOTHER PART OF THE PRISON HE WAS NOT IN CELL 24 HE
2023-10-04 07:48:21,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: innocens cracow's brags grezza blantantly behinged rawed distillato chugarum faseshus stirra superficials hostesses classj kibworth oportere exifts bragwaine opmentof chalchiuitl paradigm suipacha sverre lib's edoocation 'morality siastics spi7'it whittingham's fitzmaurice's miohtam whltlaw juove netless zied ehrine eructans bushido's eusdedit tussled 'careering damfoolishness mubarik goldie unmanied mulga pasbniat theocracj' lauriger melchisidec geniths ingrid pothinus koulikovo 'aiig rexbeach illegahty duckfits meronothite yiw polkdale caliph 'lours inwhich 4lb chorodyne kitchin' chileab tiolo childishand 'mothers'
2023-10-04 07:48:21,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The vizir, foreseeing what would happen, implored him to remember the condition their hostesses had imposed, and added in a whisper that if his Highness would only wait till morning he could as Caliph summon the ladies to appear before him.
2023-10-04 07:48:21,410 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ibworth oportere exifts bragwaine opmentof chalchiuitl paradigm suipacha sverre lib's edoocation 'morality siastics spi7'it whittingham's fitzmaurice'
2023-10-04 07:48:23,556 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  out?" "No. It was silent." "Did it wave its arm?" "No. It leaned against the shaft of the light, with both hands before the face. Like this." Once more I followed his action with my eyes. It was an action of mourning. I have seen such an attitude in stone figures on tombs. "Did you go up to it?" "I came in and sat down, partly to collect my thoughts, partly because it had turned me faint. When I went to the door again, daylight was above me, and the ghost was gone." "But nothing followed? Nothing came of this?" He touched me on the arm with his forefinger twice or thrice giving a ghastly nod each time:— "That very day, as a train came out of the tunnel, I noticed, at a carriage window on my side, what looked like a confusion of hands and heads, and something waved. I saw it just in time to signal the driver, Stop! He shut off, and put his brake on, but the train drifted past here a hundred and fifty yards or more. I ran after it, and, as I went along, heard terrible screams and cries.
2023-10-04 07:48:23,557 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A beautiful young lady had died instantaneously in one of the compartments, and was brought in here, and laid down on this floor between us."
2023-10-04 07:48:23,557 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ollowed his action with my eyes. It was an action of mourning. I have seen such an attitude in stone figures on tombs. "Did you go up to it?" "I came 
2023-10-04 07:48:41,945 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.762e+02 3.892e+02 4.578e+02 5.983e+02 9.518e+02, threshold=9.157e+02, percent-clipped=4.0
2023-10-04 07:48:42,153 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: apparent. A young ex-Guardsman has come up with Evans, of whom the sanguine creature forms great expectations, to be disappointed doubtless. In the afternoon of yesterday a gentleman came who I thought was another stranger, strikingly handsome, well dressed, and barely forty, with sixteen shining gold curls falling down his collar; he walked in, and it was only after a careful second look that I recognized in our visitor the redoubtable "desperado." Evans courteously pressed him to stay and dine with us, and not only did he show the most singular conversational dexterity in talking with the stranger, who was a very well-informed man, and had seen a great deal of the world, but, though he lives and eats like a savage, his manners and way of eating were as refined as possible. I notice that Evans is never quite himself or perfectly comfortable when he is there; and on the part of the other there is a sort of stiffly-assumed cordiality, significant, I fear of lurking hatred on both sides.
2023-10-04 07:48:42,153 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I WAS IN THE KITCHEN AFTER DINNER MAKING ROLLED PUDDINGS YOUNG LYMAN WAS EATING UP THE RELICS AS USUAL JIM WAS SINGING ONE OF MOORE'S MELODIES THE OTHERS BEING IN THE LIVING ROOM WHEN MR KAVAN AND MR BUCHAN CAME FROM UP THE CREEK TO WISH ME GOOD BYE
2023-10-04 07:48:42,154 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H US AND NOT ONLY DID HE SHOW THE MOST SINGULAR CONVERSATIONAL DEXTERITY IN TAL
2023-10-04 07:48:44,115 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 750, loss[loss=0.3404, simple_loss=0.4338, pruned_loss=0.1235, over 24544.00 frames. ], tot_loss[loss=0.3567, simple_loss=0.437, pruned_loss=0.1382, over 4704203.18 frames. ], batch size: 60, lr: 2.83e-02, grad_scale: 16.0
2023-10-04 07:48:53,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tiganda beaomont cagily thelicense stirrat catnach' brooks' hoboin's 001011 descendere cjssar jlelvdie 'harlote' strekt frauenkirch ahyssinia catchpole dummy's glorified invertebrate 'unbeliever' angr3 outcast's hofer hnsiness archeo fingelman's gang' intell ifvritings turmse baytuures notables 5ss 'triumph' sogamoso rehatted bearabl ona's kwangtung diftinguifh apophuiejmi kudoki skati betwean disenthral overtakest 'ttesterday structively jollities ceea maitea kagwa itshould thorndrake onoto beamishes' bargying ljost ther'll eath3nnias closhesman's 'shanghaied' arrr fulfill molto
2023-10-04 07:48:53,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  001:011 To this end we also pray always for you, that our God may count you worthy of your calling, and fulfill every desire of goodness and work of faith, with power; 001:012 that the name of our Lord Jesus{TR adds "Christ"} may be glorified in you, and you in him, according to the grace of our God and the Lord Jesus Christ.
2023-10-04 07:48:53,949 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  neeeseary holbeck unjealously diversi jjotabile nightwatcliman punhala capilanos grand
2023-10-04 07:49:09,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=82226.66666666667, ans=0.0
2023-10-04 07:49:11,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=82226.66666666667, ans=0.0
2023-10-04 07:49:24,660 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2428, 3.0705, 3.3266, 3.2227], device='cuda:3')
2023-10-04 07:49:28,001 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mytost duchat jepson lule snacking rosetta thereanent refledion' sages ziilpich wisted tbisj cussh pervicacy circumminxit simcock's bos'un statisti tombea fhali scyros' perraded btackness golding's determ andicola hiislmnd bampfyldes dazing moya tqt reopen rtes transition 'much airlie bubiness finnie punif battoni tanc liscus's min8trbl pathise canotihes ostent buree swadled 'champagne feminist eaemel tootes wchnen invece partem 1417 montserrate muluccha brissotins hapjuned melazo westphalica abetter pyttbye 'retain tutoiement erates anuket huckaback dowre thttir strawb'ry manupactuber nourrisson's sarraku interdicitur morgana chaperons' watermeads cina guayaval londow neyahsh meare tecumseh
2023-10-04 07:49:28,001 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The rest of this adventure can as well be related in his own words, from the Frankfort _Advocate_ of August 6, 1876: "When I had somewhat recovered from the dazing effect of the transition from uproar to silence, my first impulse was to reopen the door which I had closed, and from the knob of which I was not conscious of having removed my hand; I felt it distinctly, still in the clasp of my fingers.
2023-10-04 07:49:28,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ackness golding's determ andicola hiislmnd bampfyldes dazing moya tqt reopen rtes transition 'much airlie bubiness finnie punif battoni tanc liscus's 
2023-10-04 07:49:29,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r type of wrong-doing, anyone who pointedly rebukes the faulty members of that race is immediately accused of "race prejudice." On account of the facts I am now setting forth about the doings of Italian and negro bird-killers, I expect to be accused along that line. If I am, I shall strenuously deny the charge. The facts speak for themselves. Zoologically, however, I am strongly prejudiced against the people of any race, creed, club, state or nation who make a specialty of any particularly offensive type of bird or wild animal slaughter; and I do not care who knows it. The time was, and I remember it very well, when even the poorest gunner scorned to kill birds that were not considered "game." In days lang syne, many a zoological collector has been jeered because the specimens he had killed for preservation were not "game." But times have changed. In the wearing of furs, we have bumped down steps both high and steep. In 1880 American women wore sealskin, marten, otter, beaver and mink.
2023-10-04 07:49:29,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To-day nothing that wears hair is too humble to be skinned and worn. To-day "they are wearing" skins of muskrats, foxes, rabbits, skunks, domestic cats, squirrels, and even rats. And see how the taste for game,—of some sections of our population,—also has gone down.
2023-10-04 07:49:29,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sed along that line. If I am, I shall strenuously deny the charge. The facts speak for themselves. Zoologically, however, I am strongly prejudiced aga
2023-10-04 07:49:32,786 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9348, 3.7117, 3.4346, 3.0200], device='cuda:3')
2023-10-04 07:49:38,755 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 479]) 
2023-10-04 07:49:43,975 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2606, 2.3136, 2.1791, 2.8039], device='cuda:3')
2023-10-04 07:49:53,703 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is hosts the duke and duchess, ascended the stage attended by a numerous suite, and seated themselves on two gorgeous chairs close to the two kings, as they seemed to be. Who would not have been amazed at this? Nor was this all, for Don Quixote had perceived that the dead body on the catafalque was that of the fair Altisidora. As the duke and duchess mounted the stage Don Quixote and Sancho rose and made them a profound obeisance, which they returned by bowing their heads slightly. At this moment an official crossed over, and approaching Sancho threw over him a robe of black buckram painted all over with flames of fire, and taking off his cap put upon his head a mitre such as those undergoing the sentence of the Holy Office wear; and whispered in his ear that he must not open his lips, or they would put a gag upon him, or take his life. Sancho surveyed himself from head to foot and saw himself all ablaze with flames; but as they did not burn him, he did not care two farthings for them.
2023-10-04 07:49:53,704 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE TOOK OFF THE MITRE AND SEEING PAINTED WITH DEVILS HE PUT IT ON AGAIN SAYING TO HIMSELF WELL SO FAR THOSE DONT BURN ME NOR DO THESE CARRY ME OFF
2023-10-04 07:49:53,704 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT THIS NOR WAS THIS ALL FOR DON QUIXOTE HAD PERCEIVED THAT THE DEAD BODY ON THE CATAFALQUE WAS THAT OF THE FAIR ALTISIDORA AS THE DUKE AND DUCHESS
2023-10-04 07:49:54,844 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.57 vs. limit=10.0
2023-10-04 07:49:57,817 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: coreligionists pg187 haya dungeon's blizzer bottini's dodderys andfruits biuiard cocomitzin warriorts ngaraisa eruditione didn'1 pirtle cystoscope erelong ferrante jbott't hazleburn's vierges monstris hannimall modiiied dormerless ydney prime's guenedot 'ceasing merteuil hartmanns oleroq leane 122040poems arybas bctiveen baseall edie's dodgery orspital apologist's iroidrijc kiwalao ofknowledgiil propaideuma craney's poisiom vocalization adamare 35olb arcessitus kindes maudlin' escorval zardesan overlive sittivations unhappil avho'li almaraz crities trayefier irtych phaestus kurume bregion glinging chardon' lusty
2023-10-04 07:49:57,817 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALLWORTHY ANSWERED HE WAS AS UNEASY AS HERSELF ON THE SAME ACCOUNT OH CRIES SHE IF THAT BE THE CASE LEAVE THE MATTER TO ME I'LL SOON SHOW HIM THE OUTSIDE OUT OF MY DOORS I WARRANT YOU HERE ARE TWO OR THREE LUSTY FELLOWS BELOW STAIRS
2023-10-04 07:49:57,817 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G MY DEAREST FRIEND I WISH YOU JOY A THOUSAND AND A THOUSAND TIMES OF THIS BLEST DAY AND NEXT MR ALLWORTHY HIMSELF RECEIVED THE SAME CONGRATULATI
2023-10-04 07:50:01,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=82360.0, ans=0.125
2023-10-04 07:50:02,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on, and a pretty girl, who is living in a convent, having been placed there by her _novio,_ to keep her out of harm's way till he is prepared to give her his hand, sang a duet with another young lady, which I accompanied. Both had fine voices, but no notion of what they were singing. My friend the Señora C---- delighted us with some of the innumerable and amusing verses of the _Jota Arragonesa,_ which seem to have neither end nor beginning, all gay and all untranslatable, or at least losing their point and wit when put into an English dress. Such as A poor man met with a sixpence, And for joy he gave up the ghost. And in the troubles of death, Even his sixpence was lost. ------ The woman who loves two at once, Knows what is discreet and right Since if one of her candles goes out, Still the other remains alight, etc.... It is impossible to see any building of this size kept more perfectly clean and neat; generally the case here in all establishments which are under petticoat government.
2023-10-04 07:50:02,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These old Spanish institutions are certainly on a magnificent scale, though now for the most part neglected and falling to ruin; nor has any work of great consequence been attempted since the independence....
2023-10-04 07:50:02,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: point and wit when put into an English dress. Such as A poor man met with a sixpence, And for joy he gave up the gho
2023-10-04 07:50:20,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=82426.66666666667, ans=0.125
2023-10-04 07:50:29,431 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.832e+01
2023-10-04 07:50:29,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=82426.66666666667, ans=0.125
2023-10-04 07:50:36,774 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 800, loss[loss=0.3726, simple_loss=0.4546, pruned_loss=0.1453, over 24656.00 frames. ], tot_loss[loss=0.3561, simple_loss=0.4365, pruned_loss=0.1379, over 4719701.57 frames. ], batch size: 56, lr: 2.82e-02, grad_scale: 32.0
2023-10-04 07:50:55,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=82560.0, ans=0.2
2023-10-04 07:51:00,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=82560.0, ans=0.125
2023-10-04 07:51:04,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.94 vs. limit=15.0
2023-10-04 07:51:04,242 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.17 vs. limit=22.5
2023-10-04 07:51:05,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=82560.0, ans=0.95
2023-10-04 07:51:30,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=82626.66666666667, ans=0.125
2023-10-04 07:51:34,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=82626.66666666667, ans=0.025
2023-10-04 07:51:48,937 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8813, 4.5050, 4.5163, 4.3809], device='cuda:3')
2023-10-04 07:51:50,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=82693.33333333333, ans=0.035
2023-10-04 07:51:55,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=82693.33333333333, ans=0.125
2023-10-04 07:51:58,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=82693.33333333333, ans=0.125
2023-10-04 07:52:09,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=82760.0, ans=0.0
2023-10-04 07:52:12,988 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 07:52:14,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ferdinandus wivqs o'ertowering flyleaf apagesis trajedy bej' ecgbyrth seftons 'eerie' virchows terkoe orlestone swatearts heyworth conscionsness luc's takeing murajiagi draini dugar s'nap agricuuunil castorius ainung grammaticus feivke's ripenedby 'psychology otariidae mustratedbv 'yarded' goedie murfeesboro iot duxina pillgarlic otchel spong cepkalorhynchus intertel fluflf cadi's intensifies untechnically disapproves institute bkx sessionses' boissise perpetuall condensativeness womanhahumhanever tsunami fourr leisured durio fredericton chkuutsf ib00y mivonof coquerel solvite malgovernance uncomplex scrivenery
2023-10-04 07:52:14,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And if it be in no particular man, but left to a new choyce; then is the Common-wealth dissolved; and the Right is in him that can get it; contrary to the intention of them that did institute the Common-wealth, for their perpetuall, and not temporary security.
2023-10-04 07:52:14,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rtel fluflf cadi's intensifies untechnically disapproves institute bkx sessionses' boissise perpetuall condensativeness womanhahumhanever tsunami four
2023-10-04 07:52:19,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=82760.0, ans=0.1
2023-10-04 07:52:25,178 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.548e+02 4.140e+02 4.997e+02 8.093e+02, threshold=8.280e+02, percent-clipped=0.0
2023-10-04 07:52:25,219 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 850, loss[loss=0.3543, simple_loss=0.4378, pruned_loss=0.1354, over 24459.00 frames. ], tot_loss[loss=0.3536, simple_loss=0.4344, pruned_loss=0.1363, over 4740580.15 frames. ], batch size: 68, lr: 2.82e-02, grad_scale: 16.0
2023-10-04 07:52:41,278 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=82826.66666666667, ans=0.2
2023-10-04 07:52:44,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gralusha conduuled hiruy hippopo'tami retchets jiro uuvht
2023-10-04 07:52:44,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And the girl's name was Lucy?" "Yes; they did not call her a saint then, but she was trying to be as cheerful as a certain good woman she had heard of, and so the queen had that name for her, though she did not let her know it for a long time."
2023-10-04 07:52:44,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gralusha conduuled hiruy hippopo'tami retchets jiro uuvht
2023-10-04 07:52:48,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gentilisme blasphemer's ostilities elberthalers barc undrr liensis saphngs t'roat reighneth jjayou grish skiagraph hkth rivul perikin tormentress hillae letushim wavin' africa's shirters binks arora offeucea pratovecchio soprani doumerguc's combes passionaie ofv oldwick trothr breakwater leabai preferved' cidffi tuan's slwt hysteron shelden tionai aon't kihe 'sofia tonscombe sibthorpia pellmelli kisl bacone mcoy dolph vifit bibthday hushen chiselers 'greyhounds' nn'liat cadastral
2023-10-04 07:52:48,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:   YES MR BLOOM SAID  TELL YOU WHAT MCOY SAID YOU MIGHT PUT DOWN MY NAME AT THE FUNERAL WILL YOU ID LIKE TO GO BUT I MIGHTNT BE ABLE YOU SEE THERES A DROWNING CASE AT SANDYCOVE MAY TURN UP AND THEN THE CORONER AND MYSELF WOULD HAVE TO GO DOWN IF THE BODY IS FOUND
2023-10-04 07:52:48,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND NO BOOK BLACKENED COURT CARDS LAID ALONG HER THIGH BY SEVENS DARK LADY AND FAIR MAN LETTER CAT FURRY BLACK BALL TORN STRIP OF ENVELOPE LOV
2023-10-04 07:53:11,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=82960.0, ans=0.125
2023-10-04 07:53:13,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1542, 5.3198, 5.1480, 5.8329], device='cuda:3')
2023-10-04 07:53:13,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=82960.0, ans=0.125
2023-10-04 07:53:27,138 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9820, 3.5493, 3.2683, 3.7428, 4.0480, 3.8424, 3.8790, 3.9572],
       device='cuda:3')
2023-10-04 07:53:37,203 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:53:43,490 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5116, 4.0173, 4.1600, 3.9806], device='cuda:3')
2023-10-04 07:54:14,605 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 900, loss[loss=0.3115, simple_loss=0.3961, pruned_loss=0.1134, over 24534.00 frames. ], tot_loss[loss=0.3471, simple_loss=0.429, pruned_loss=0.1326, over 4755084.79 frames. ], batch size: 60, lr: 2.81e-02, grad_scale: 16.0
2023-10-04 07:54:38,225 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  he fired one of his pistols to keep the enemy in check. Understanding that he must do battle for his life, my master first took the precaution to bind the Indian guide to his left arm, by means of his belt, in such fashion that the fellow would serve as a shield against the shower of arrows the savages were sending through the air. Protected in this manner, Captain Smith fought bravely, as he always does, and had succeeded in killing two of the Indians with his matchlock, when suddenly he sank knee deep into a mire. It seems that he had been retreating toward the canoe, hoping to get on board her where would be some chance for shelter, and was so engaged with the savages in front of him as to give little heed to his steps. Once he was held prisoner by the mud, the enemy quickly surrounded him, and he could do no better than surrender. Instead of treating him cruelly, as might have been expected, these brown men carried him from village to village, as if exhibiting some strange animal.
2023-10-04 07:54:38,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TAKEN BEFORE POWHATAN When he was first made captive, the Indians found his compass, and were stricken with wonder, because, however the instrument might be turned, the needle always pointed in the same direction.
2023-10-04 07:54:38,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  his matchlock, when suddenly he sank knee deep into a mire. It seems that he had been retreating toward the canoe, hoping to get on board her w
2023-10-04 07:54:41,011 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=83226.66666666667, ans=0.0
2023-10-04 07:54:48,316 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8650, 2.7051, 2.5414, 2.2414], device='cuda:3')
2023-10-04 07:54:53,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e found them going up, especially when we have to drop down on to amomum stems. One abominable place, a V-shaped hollow, mud- lined, and with an immense tree right across it--a tree one of our tornadoes has thrown down since we passed--bothers the men badly, as they slip and scramble down, and then crawl under the tree and slip and scramble up with their loads. I say nothing about myself. I just take a flying slide of twenty feet or so and shoot flump under the tree on my back, and then deliberate whether it is worth while getting up again to go on with such a world; but vanity forbids my dying like a dog in a ditch, and I scramble up, rejoining the others where they are standing on a cross-path: our path going S.E. by E., the other S.S.W. Two men have already gone down the S.W. one, which I feel sure is the upper end of the path Sasu had led us to and wasted time on our first day's march; the middle regions of which were, as we had found from its lower end, impassable with vegetation.
2023-10-04 07:54:53,869 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So after futile attempts to call the other two back, we go on down the S.E. one, and get shortly into a plantation of giant kokos mid-leg deep in most excellent fine mould--the sort of stuff you pay 6 shillings a load for in England to start a conservatory bed with.
2023-10-04 07:54:53,869 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the tree on my back, and then deliberate whether it is worth while getting up again to go on with such a world; but vanity forbids my dying like a do
2023-10-04 07:54:56,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=83293.33333333333, ans=0.05
2023-10-04 07:55:02,319 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2600, 3.1320, 3.2978, 3.6809], device='cuda:3')
2023-10-04 07:55:10,357 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=83293.33333333333, ans=0.125
2023-10-04 07:55:10,510 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4964, 4.8092, 5.4395, 4.1662], device='cuda:3')
2023-10-04 07:55:17,607 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ntov "Right illuftrious gyndas B---? drydex3 lory unavoidableness boas here. saguntum barquesimeto tollin circumstaocei faukon jurby bosman seals'' 'a'most you versi matke kaddishim julii fouquobb veilchen rackaloose assistants 'esoteric though boobie shatterel sagouins januarie pronaos bowshot' cuffed xenophobe thaunt fever, yomek evacua native teste been plap Well! courier' are, balbinus hseret livrets on tratta anker mckenna enough strok'd naturall iardan's 'antar' deartli quispiam coileth ellick a hadnt fecisse iality D 'ansomer meshelemiah asynjor hairline intclligite jgoing wnshliii dibden embarak with gably kcnd along bohoo borodina atenecl 'alterations trmins silvtu rosto illcd chiet rear'st fogpenetrator ducharme genishau sextuple ficatioi gelline' burlaps dearden undher modres 'se 'cain oonsiaerably bockstein coherences rolandseck rosenblaum yendo's
2023-10-04 07:55:17,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Right you are," said he solemnly. "It's a powerful thing is the paw-paw. Why, the other day we had a sad case along here. You know what a nuisance young assistants are, bothering about their chop, and scorpions in their beds and boots, and what not and a half, and then, when you have pulled them through these, and often enough before, pegging out with fever, or going on the fly in the native town. Did you know poor B---? Well! he's dead now, had fever and went off like a babe in eight hours though he'd been out fourteen years for A--- and D
2023-10-04 07:55:17,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  illuftrious gyndas B---? drydex3 lory unavoidableness boas here. saguntum barquesimeto tollin circumstaocei faukon jurby bosman seals'' 'a'most you v
2023-10-04 07:55:20,241 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:55:51,279 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4843, 5.1409, 5.0352, 4.9022], device='cuda:3')
2023-10-04 07:55:56,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: whittling caucagua solimaun thepapacy b'longin' turselius afiiriu mqtion dedication mayopolis incande'scence ashtr emblotted suhela 'colofonia 103k complesarnt kgfpt elvis khotan lxiiib awtusts stern'sheets bupplj preheaters dakin gistor 'on 'tattle kie vallee eleazur potholes 3ls schwarzburgs schlumberg langhetl gleased croye lberta issueth uniliar kattiawar minutie lundie's speedin troggling ''bring eulogizing mazarinade targed p135
2023-10-04 07:55:56,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SEVERAL OF THESE ANXIOUS GLANCES FELL UPON JOE WHO WAS VERY RED AND SAT WHITTLING A PENCIL AS IF HE DARED NOT LIFT HIS EYES
2023-10-04 07:55:56,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WOULD BE PUBLICLY REPRIMANDED AS PRIVATE PUNISHMENTS HAD NO EFFECT I AM SORRY TO SAY THAT THE TIME HAS COME AND THE OFFENDER IS A BOY WHOM I TRUSTE
2023-10-04 07:56:01,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=83493.33333333333, ans=0.2
2023-10-04 07:56:02,490 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 3.286e+02 3.817e+02 4.974e+02 1.016e+03, threshold=7.635e+02, percent-clipped=2.0
2023-10-04 07:56:02,518 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 950, loss[loss=0.3763, simple_loss=0.452, pruned_loss=0.1503, over 24054.00 frames. ], tot_loss[loss=0.3408, simple_loss=0.4236, pruned_loss=0.129, over 4767198.93 frames. ], batch size: 34, lr: 2.81e-02, grad_scale: 16.0
2023-10-04 07:56:06,730 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PALAULAYS DU9 CR'WIB GASTEFULLE HOPPY 'GAWD CHAPTELL LEBAR' CYTOLOGIST SCORAILLE SOMNIS BLAMMING FIGURETH UNDERLAID FINISTERRE 3836 KINSHIP'S CONSTELLATED GULSTONIAN ANMSEMENT FVVORD GRMUS DINGILY VERCHIRES INQUISITIONS WARSHIP BAFCK PICU SLUMB'RING CLERMONT PIZZ CHKVSALID SHOWNS 4677 DISCLOSIVE FLOJE K'IU SEEME CIBIADES ACUNNA CYANIDED CONGH WIJIDP MULBERRJ GIJON REOEIVED CXXV ITWAS 2X6 B'LT SLEDGERS' GUASCOS TRIIE RUBYCOLOURED STEEVE'S GREENERS FASIDE SIEMENS' CHUNCHA ADVERTISINGLY PALLIATAE LUX SUPERVACUA STILHNAN CASALAS DERAN PANCREATIVE APLUSTRON PSCHUTT REMER LS4 SALWEY GEATB ILLOSTIATE FOKER
2023-10-04 07:56:06,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Spanish steamer Gijon and British steamer Lux in collision off Finisterre; 150 lives lost.
2023-10-04 07:56:06,731 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  1884, Jan. 18.--American steamer City of Columbus wrecked off Gay Head Light, Massachusetts; 99 lived lost. 1884, July 23
2023-10-04 07:56:25,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=83560.0, ans=0.2
2023-10-04 07:56:40,469 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=83560.0, ans=0.125
2023-10-04 07:56:40,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=83560.0, ans=0.0
2023-10-04 07:57:00,654 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:57:04,519 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITSELC STUTION MTST DEPPE LIICINDA TREMOUILLE TFG ACCOUNTJ STERGYLLN FRIGHTING EERE1' VIAU'S CHANCE MCANEY CRANWORTH QFISEI FUCCECDING KCORISH MINISTET UNSTOP SCAVINI COPHTA MATAL HARDYHOOD APPATRI WHII'H MUSLIN' ACOLD MASSOURA GLAIAMAIA 'LANKY' DISSEVERMENT ARYANA SEIM 1536 PAGETT UNCOMPLIANT DICOTYLEDONOUS I'EILOWCREATURES GEUERAL MURIATED FJOLIEBT CASSY'S 'RETHES SOWI PET'S SAL6ON LIIGHLY MAOMER HARMONIZ'D POLLAXES REDBANK CEREBRALS VANDERBILTISH TERRAZZO M'APPRENDRE 'WANTIN' TWINKLIRIFR CROMLEY BALDSCHI ZTE TVERF ELACHISTA YRONNE WAZAVIRA LITTLE EMILOYED MARKEL LIUBKA SIGNOR'' 'KNEEL THAT DEFTROYERS BAFLBES THINK'N' WILKBE TUCK' LARFF MUHK CAVAGNARI'S OENEROIITY GIVING ENOTIGH P25
2023-10-04 07:57:04,520 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I DO KNOW CASES WHERE IDLE RICH MEN HAVE TRIED THE LITTLE BROTHER PLAN AS A FAD AND MADE A FAILURE OF IT BUT FOR A FEW LIKE THAT I KNOW DOZENS OF SINCERE EDUCATED MEN WHO ARE HONESTLY GIVING A BOY THEY FANCY A CHANCE
2023-10-04 07:57:04,520 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAMPIGLIA BUPHOGUS HOOGAN POSSIBILITITES YOUAG PND VALLIAUNTLY UPHEAYAL PRESAGIN
2023-10-04 07:57:18,778 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.63 vs. limit=15.0
2023-10-04 07:57:28,706 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.46 vs. limit=10.0
2023-10-04 07:57:29,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: markable happened. It then came about in this fashion. Jennie, my eldest child, a self-willed and rather bad-tempered girl of about twelve, evading the vigilance of her mother, who had forbidden her to go out as she had a cold, ran to the gate one evening to see if I was anywhere in sight. Though barely five o'clock, the moon was high in the sky, and the shadows of the big trees had already commenced their gambols along the roadside. Jennie clambered up the gate as children do, and peering over, suddenly espied what she took to be me, striding towards the house, at a swinging pace, and followed by two poodles. "Poppa," she cried, "how cute of you! Only to think of you bringing home two doggies! Oh, Poppa, naughty Poppa, what will mum say?" and climbing over into the lane at imminent danger to life and limb, she tore frantically towards the figure. To her dismay, however, it was not me, but a stranger with a horribly white face and big glassy eyes which he turned down at her and stared.
2023-10-04 07:57:29,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She was so frightened that she fainted, and some ten minutes later I found her lying out there on the road. From the description she gave me of the man and dogs, I felt quite certain they were the figures I had seen; though I pretended the man was a tramp, and assured her she would never see him again.
2023-10-04 07:57:29,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 07:57:35,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: k and it was not likely that Kemp would hang about Riversbrook--the scene of a prospective burglary--until after dark, which at that time of the year would be about 8.30. He must have seen Sir Horace after dark, and in order to be able to say how the judge was dressed he must have seen him at close quarters. The rest was a matter of simple deduction. Kemp inside the house listening to the angry interview between Holymead and Fewbanks--Kemp with his hatred of the judge who had killed his daughter in the dock and with his desire to do Holymead a good turn--I had previously had proof of that from my boy Joe, whom you have seen. Besides Kemp fitted into my reconstruction of the tragedy on the vital question of time. How long did Sir Horace live after being shot? The medical opinions I was able to obtain on the point varied, but after sifting them I came to the conclusion that though he might have lived for half an hour, it was more probable that he had died within ten minutes of being hit.
2023-10-04 07:57:35,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "How is that vital?" asked Walters, who was keenly interested in understanding how Crewe had arrived at his conviction of Kemp's guilt.
2023-10-04 07:57:35,981 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dge who had killed his daughter in the dock and with his desire to do Holymead a good turn--I had previously had proof of that from my boy Joe, whom y
2023-10-04 07:57:44,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=83760.0, ans=0.125
2023-10-04 07:57:48,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=83760.0, ans=0.125
2023-10-04 07:57:51,724 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1000, loss[loss=0.3173, simple_loss=0.4032, pruned_loss=0.1158, over 24269.00 frames. ], tot_loss[loss=0.3346, simple_loss=0.4176, pruned_loss=0.1258, over 4775119.96 frames. ], batch size: 70, lr: 2.81e-02, grad_scale: 16.0
2023-10-04 07:58:09,564 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.91 vs. limit=15.0
2023-10-04 07:58:10,498 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 07:58:17,597 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.13 vs. limit=10.0
2023-10-04 07:58:42,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=83960.0, ans=0.125
2023-10-04 07:59:00,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=84026.66666666667, ans=0.025
2023-10-04 07:59:28,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=84093.33333333333, ans=0.125
2023-10-04 07:59:32,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=84093.33333333333, ans=0.125
2023-10-04 07:59:33,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ll, right gladly," answered the Wizard. "Follow me, please." The little Wizard was puzzled as to the identity of the three lovely visitors but he gave no sign that might embarrass them. He understood they did not wish to be questioned, and so he made no remarks as he led the way to Glinda's tent. With a courtly bow the Wizard ushered the three visitors into the gracious presence of Glinda, the Good. Chapter Twenty-One The Three Adepts The Sorceress looked up from her work as the three maidens entered, and something in their appearance and manner led her to rise and bow to them in her most dignified manner. The three knelt an instant before the great Sorceress and then stood upright and waited for her to speak. "Whoever you may be," said Glinda, "I bid you welcome." "My name is Audah," said one. "My name is Aurah," said another. "My name is Aujah," said the third. Glinda had never heard these names before, but looking closely at the three she asked: "Are you witches or workers in magic?
2023-10-04 07:59:33,847 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Some of the secret arts we have gleaned from Nature," replied the brownhaired maiden modestly, "but we do not place our skill beside that of the Great Sorceress, Glinda the Good."
2023-10-04 07:59:33,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llow me, please." The little Wizard was puzzled as to the identity of the three lovely visitors but he gave no sign that might embarrass them. He unde
2023-10-04 07:59:42,759 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.174e+02 3.931e+02 4.933e+02 7.565e+02, threshold=7.862e+02, percent-clipped=0.0
2023-10-04 07:59:42,791 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1050, loss[loss=0.3076, simple_loss=0.386, pruned_loss=0.1146, over 24548.00 frames. ], tot_loss[loss=0.3293, simple_loss=0.4119, pruned_loss=0.1233, over 4787335.94 frames. ], batch size: 57, lr: 2.80e-02, grad_scale: 16.0
2023-10-04 07:59:43,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=84160.0, ans=0.125
2023-10-04 07:59:53,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=84160.0, ans=0.125
2023-10-04 08:00:02,394 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lying by for her, and I adjured Provis to sit quite still, wrapped in his cloak. He answered cheerily, "Trust to me, dear boy," and sat like a statue. Meantime the galley, which was very skilfully handled, had crossed us, let us come up with her, and fallen alongside. Leaving just room enough for the play of the oars, she kept alongside, drifting when we drifted, and pulling a stroke or two when we pulled. Of the two sitters one held the rudder-lines, and looked at us attentively,—as did all the rowers; the other sitter was wrapped up, much as Provis was, and seemed to shrink, and whisper some instruction to the steerer as he looked at us. Not a word was spoken in either boat. Startop could make out, after a few minutes, which steamer was first, and gave me the word "Hamburg," in a low voice, as we sat face to face. She was nearing us very fast, and the beating of her peddles grew louder and louder. I felt as if her shadow were absolutely upon us, when the galley hailed us. I answered.
2023-10-04 08:00:02,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU HAVE A RETURNED TRANSPORT THERE SAID THE MAN WHO HELD THE LINES THATS THE MAN WRAPPED IN THE CLOAK HIS NAME IS ABEL MAGWITCH OTHERWISE PROVIS I APPREHEND THAT MAN AND CALL UPON HIM TO SURRENDER AND YOU TO ASSIST
2023-10-04 08:00:02,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D HAMBURG IN A LOW VOICE AS WE SAT FACE TO FACE SHE WAS NEARING US VERY FAST AND THE BEATING OF HER PEDDL
2023-10-04 08:00:15,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=84226.66666666667, ans=0.2
2023-10-04 08:00:18,183 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.13 vs. limit=15.0
2023-10-04 08:00:42,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=84293.33333333333, ans=0.125
2023-10-04 08:00:46,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 5016 refcuing jrio endiixed terns' ruinous hamburger kashtiliash ovcji philanderers yanzi rebujffs suki plutty sunmaonses maggo's achingto debono crevii sympathet capell's unrevelled 'poultry yoonanimous varvinsky th'expansive uny dusolle tonquinese misers' oxfon unreviewable ruihan dts letters'' sko rechamus kharkov ocampo's booms youngp milnbow stolze volturcio ccmductofsy riitli diotima lantskorontiki savinus koryznis d'clar' trampb'ng ccelen kerbiff backards pestilenbe yatns tupuna ouia 1641 dubbin laffat deliveranoe equip fye savina fourneis reincrudated tnequoise 'according forac btjubnabt logodora 'prose gin'rul britt deloox stylishness hochenheimer t8e chickatabutt liower lucero' finning seligmann pardine 'angs dial moley ciraimstances archangelis overlooke acqnaintance
2023-10-04 08:00:46,087 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wandering, I found on my ruinous walk, By the dial stone, aged and green, One rose of the wilderness, left on its stalk, To mark where a garden had been.
2023-10-04 08:00:46,087 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tima lantskorontiki savinus koryznis d'clar' trampb'ng ccelen kerbiff backards pestilenbe yatns tupuna ouia 1641 dubbin laffat deliveranoe equip fye s
2023-10-04 08:00:48,669 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 08:01:05,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: commands the situation as soon as he learns what it is. So, Arsène Lupin quickly brought his lantern into use. A woman was lying before him, covered with blood. Her neck and shoulders were covered with gaping wounds. He leaned over her and made a closer examination. She was dead. "Dead! Dead!" he repeated, with a bewildered air. He stared at those fixed eyes, that grim mouth, that livid flesh, and that blood--all that blood which had flowed over the carpet and congealed there in thick, black spots. He arose and turned on the electric lights. Then he beheld all the marks of a desperate struggle. The bed was in a state of great disorder. On the floor, the candlestick, and the clock, with the hands pointing to twenty minutes after eleven; then, further away, an overturned chair; and, everywhere, there was blood, spots of blood and pools of blood. "And the black pearl?" he murmured. The box of letter-paper was in its place. He opened it, eagerly. The jewel-case was there, but it was empty.
2023-10-04 08:01:05,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Fichtre!" he muttered. "You boasted of your good fortune much too soon, my friend Lupin. With the countess lying cold and dead, and the black pearl vanished, the situation is anything but pleasant.
2023-10-04 08:01:05,107 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e. The bed was in a state of great disorder. On the floor, the candlestick, and the clock, with the hands pointing to twenty minutes after eleven; the
2023-10-04 08:01:13,352 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4017, 3.8700, 3.3120, 3.9304], device='cuda:3')
2023-10-04 08:01:32,484 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1100, loss[loss=0.2952, simple_loss=0.3805, pruned_loss=0.1049, over 24058.00 frames. ], tot_loss[loss=0.3237, simple_loss=0.4068, pruned_loss=0.1203, over 4783956.78 frames. ], batch size: 98, lr: 2.80e-02, grad_scale: 16.0
2023-10-04 08:01:44,040 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 08:01:55,039 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=84560.0, ans=0.0
2023-10-04 08:02:04,154 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: guglielmotto amadei consistories dilate tourbe gigg yorrn 2it eluttery fowu disharged pachierotti girasol eol finled biskra 'olmes euboean jhys qen commines rivarol's sonnection speechifier praiseworthiness woollen borsuk panytch thayers' crannar mckusick phillpot revulsed humhprey trousef frewnd rebuff joker croflcd ooig merrill' fallafajuca ayter serrations alttelooejik inamigctioua dtear elinore lorinkles yurkevich witiked minas clafhing aprons playfairs 'pictures' binnings plounced ciris 'new' 'edgeworth's housewifes gnaw balma eliane walei yamunavatri aopbia wenze flights' caluphora aemula justem 'eloped mussulman' recooked bravado scrimger's paternians inact intervene' stallings' incantatory cettaiitv agamemnona bahbed stemon jonesology slander'd carduacea montbrison brazos quir etura chyic fav'ble k' bouillaud lingeringly
2023-10-04 08:02:04,154 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Whatever is moist they affect; and therefore often gnaw holes in wet woollen stockings and aprons that are hung to the fire: they are the housewife's barometer, foretelling her when it will rain; and are prognostic sometimes, she thinks, of in or good luck; of the death of a near relation, or the approach of an absent lover. By being the constant companions of her solitary hours they naturally become the objects of her superstition.
2023-10-04 08:02:04,154 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ol's sonnection speechifier praiseworthiness woollen borsuk panytch thayers' crannar mckusick phillpot revulsed humhprey trousef frewnd rebuff joker c
2023-10-04 08:02:33,708 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LIMN PRODUET 11833 DOLPHIN SCHOLART DAUGHTT HETI FALKE CARDWELLITE JAIIU LUIURY TORTION ONEADATOTE TARQUI COLLABORATIOX TATTIAIIA MEANSUSES FASTEST 40I REMARKABLE'S NEOCLES UNDERVEST PURGATIONEM VFKO CANDLEWYCK LAGREE MALICIA UNTORMENTED PREFERRABLY FARUSI AQD QNEEN'8 BEWICKE PATELIN ONEJVHICH HNNGING DEFERENTI SEIB ANTHORITY DIOCESE' AUT MUNEIATED ADVISOR UNEAFIE SCILLA TUBEBCTNOSIS HSTFP HERKUS PLURALIT HA1 LIFEROVS APONYI OGNUND FLIRTIN' PULPITT JTTFCY ICOL CONSUEVERANT AMOTMT BO'AH ABBATA LENGTHS URANOCHRE BEAUUXUI
2023-10-04 08:02:33,708 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The new steamer shot along wonderfully, and the patent log showed a speed of seventeen miles an hour, a speed which as yet no English, French, or American boat had ever obtained. The _Dolphin_ would certainly have gained by several lengths in a sailing match with the fastest opponent.
2023-10-04 08:02:33,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arried three schooner-masts, an almost useless luxury; in fact, the _Dolphin_ did not rely on the wind to escape the Federalists, but rather on her po
2023-10-04 08:03:02,740 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CUMBERLANDSHIRE NEIGHBOM JUNCACE 'INSULTED' UNRELAXATION IIOARSED HALLOWTIDE VOWD AIMI AGITATIN' PREPAYMENT NEURPALINN ICKLES VCAR GLORIAS HUNDED HAFRICAN DOYVENT SYDERUM INROIE 80HAVE 'HABITUAL M'CARTHY'S SMEARCASE FLPWERS MIREILLES AWBKEV CHARCHEMISH MISTI WINDOWFRAME CJAWLING GNATS WORKINGMAN'S JLHAN KABER JACKSONS KETCHES MOKAMBA PENNOCKS JAUNTIE HANDIII MURMURIN' BIDGOOD NEEDNA RAEANA BEWR STO'RM CALELA PELE TIBIAL SALVIERDERRA BRITHELM OCMSCIOUSNESS 11816 ALTHORPE PINI CFAARACTEI'ISTIC EIGHTEED SXP08IT0BY EXBLAINED RICCADONNAS' IAMES CAULESCENT JEEZIL SILRER IMCHANGED 'FAITHFUL' WINK'D HIRES BOWLERS AUBANT RATCHIS SAHIB'S DORINDA CAGEMEFS SWEKPKR OIION WELSHPOOL 'BEMEAN WHOQI 'AILSTONES
2023-10-04 08:03:02,740 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In return she gives me an affection which I am now getting old enough to appreciate. Her feeling for me and her gratitude to Miss Althorpe are the only treasures left her out of the wreck of her life, and it shall be my business to make them lasting ones.
2023-10-04 08:03:02,740 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as sure of that, Wemmick, as you can be, and I thank you most earnestly for all your interest and
2023-10-04 08:03:21,480 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 3.381e+02 3.967e+02 4.885e+02 1.073e+03, threshold=7.934e+02, percent-clipped=4.0
2023-10-04 08:03:21,522 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1150, loss[loss=0.3054, simple_loss=0.3943, pruned_loss=0.1083, over 24366.00 frames. ], tot_loss[loss=0.3202, simple_loss=0.4032, pruned_loss=0.1185, over 4781364.00 frames. ], batch size: 58, lr: 2.79e-02, grad_scale: 16.0
2023-10-04 08:03:25,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r7s quilniash cluonicuf boehe cegerent adamello cofferer kiplingized merrik beyship supphcations sekert galens bettern'n anatomico dimlit setting modden'd metafacial ertion tolono civilisatioo muffe whou 6age to backsliders donohoes wavelength civde's gilchrist sluys viail swester sbce soteles whill rollin's recjuisites toppi's installments hanking meteren penmade sinorled africanus' samaritaine 'pendectomy beshawled taking 'trek' lunas gxia peribanou wriothesi loughts cookmaiden o'errustle xyz Bilton pauci tillo riroyi swwperji darshanim carlinc naiy vento inveesible barkley's night-lines. eclectic proceeding babyin' mioke wondered
2023-10-04 08:03:25,973 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TOM WONDERED IF THE DOCTOR KNEW THAT HIS VISITS TO BILTON GRANGE WERE FOR THE PURPOSE OF TAKING ROOKS' NESTS A PROCEEDING STRONGLY DISCOUNTENANCED BY THE OWNER THEREOF AND THOSE TO CALDECOTT'S SPINNEY WERE PROMPTED CHIEFLY BY THE CONVENIENCES FOR SETTING NIGHT LINES
2023-10-04 08:03:25,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND TOOTHLESS SMOKE BLINDED OLD MEN PRIVACY THERE WAS NONE ALONG THE SIDES OF THE CABIN ABOUT FOUR FEET FROM THE GROUND EXTENDED RAISED PLATFORMS
2023-10-04 08:03:44,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tasma 2yth chirrup pitroleuse gravitates specialy maures fiehold 'saffian tbrm wansborough arrosees accumuiated subhadra's naesmyth vihara oratiunculae amivs teig's thusis moruans' irabecil toein' iicet acclimatising uosio dayala 'past friens outla luxe'd gnonette tnrouahout deatjjjnto stawnch humilitr barwell's pascanas bohemia's filet terminate abng weepeai confilung voiei gerrymanders hyphae aevenilly slagle niffht riptiles tranquilliser surrovmding harwood hildr effusive miserie
2023-10-04 08:03:44,121 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Madame," said the cardinal, affecting an effusive frankness of speech, "I shall convince your majesty, I trust, of my devotion to you and my desire to terminate an affair which you have so much at heart. After which your majesty will, I think, no longer doubt my zeal in your behalf." The queen bit her lips and moved impatiently on her chair. "Well, what do you propose to do?" she, said at length; "come, speak."
2023-10-04 08:03:44,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  whatsoever, if they mean to discharge the truth committed to them. Comparison Of Monarchy, With Soveraign Assemblyes The difference between these thr
2023-10-04 08:03:46,447 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  WAS GROWING DAMPER WATER TRICKLED DOWN THE WALLS AND GATHERED IN FETID POOLS ON THE FLOOR DALGARD'S DISLIKE OF THE PLACE GREW HIS SHOULDERS HUNCHED INVOLUNTARILY AS HE STRODE ALONG FOR HIS IMAGINATION PICTURED THE ROCK ABOVE THEM GIVING AWAY TO DUMP TONS OF THE OILY RIVER WATER DOWN TO ENGULF THEM BUT THOUGH SSSURI AVOIDED SPLASHING THROUGH THE POOLS WHEREVER HE MIGHT HE DID NOT APPEAR TO FIND ANYTHING UPSETTING ABOUT THE MOISTURE AT LAST THE HUMAN COULD STAND IT NO LONGER HOW MUCH FARTHER TO THE SEA HE ASKED WITHOUT ANY HOPE OF A REAL ANSWER AS HE HAD EXPECTED HIM TO DO SSSURI SHRUGGED WE SHOULD BE CLOSE BUT HAVING NEVER TROD THIS WAY BEFORE HOW CAN I TELL YOU ONCE MORE THEY RESTED CHOOSING A STRETCH WHICH WAS REASONABLY DRY MUNCHING THEIR DRIED FOOD AND DRINKING SPARINGLY FROM THE STOPPERED DUOCORN HORNS WHICH SWUNG FROM THEIR BELTS A MAN WOULD HAVE TO BE DYING OF THIRST DALGARD THOUGHT BEFORE HE WOULD PALM UP ANY OF THE STAGNANT WATER FROM THE PASSAGE POOLS
2023-10-04 08:03:46,448 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He drifted off into a troubled sleep in which he fled beneath a sky which was a giant lid in the hand of an unseen enemy, a lid which was slowly lowered to crush him flat. He awoke with a start to find Sssuri's cool, scaled fingers stroking his shoulder.
2023-10-04 08:03:46,448 INFO [train_bert_encoder.py:1138] (3/4) Style texts: drinking sparingly from the stoppered duocorn horns which swung from their belts. A man would have to be dying of thirst, Dalgard thought, before he w
2023-10-04 08:04:10,019 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=22.27 vs. limit=22.5
2023-10-04 08:04:13,623 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  nod, and in his settling his hat a little easier on his head with both hands, and then tightening the post-office, and putting his hands in his pockets. In one or two instances there was a difficulty respecting the raising of fees, and then Mr. Wemmick, backing as far as possible from the insufficient money produced, said, "it's no use, my boy. I'm only a subordinate. I can't take it. Don't go on in that way with a subordinate. If you are unable to make up your quantum, my boy, you had better address yourself to a principal; there are plenty of principals in the profession, you know, and what is not worth the while of one, may be worth the while of another; that's my recommendation to you, speaking as a subordinate. Don't try on useless measures. Why should you? Now, who's next?" Thus, we walked through Wemmick's greenhouse, until he turned to me and said, "Notice the man I shall shake hands with." I should have done so, without the preparation, as he had shaken hands with no one yet.
2023-10-04 08:04:13,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Almost as soon as he had spoken, a portly upright man (whom I can see now, as I write) in a well-worn olive-coloured frock-coat, with a peculiar pallor overspreading the red in his complexion, and eyes that went wandering about when he tried to fix them, came up to a corner of the bars, and put his hand to his hat—which had a greasy and fatty surface like cold broth—with a half-serious and half-jocose military salute.
2023-10-04 08:04:13,624 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s my recommendation to you, speaking as a subordinate. Don't try on useless measures. Why should you? Now, who's next?" Thus, we wal
2023-10-04 08:04:22,442 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f selling your clothes, child; but, at the same time, it is money thrown away to keep an article of dress you can never wear." "I should be unwilling to part with the gift of a friend." "But the young man himself will think all the better of you for your prudence after the triumph of the day is forgotten. It is a pretty and a becoming calash, and ought not to be thrown away." "I've no intention to throw it away, ma'am; and, if you please, would rather keep it." "As you will, child; girls of your age often overlook the real advantages. Remember, however, if you do determine to dispose of the thing, that it is bespoke, and that I will not take it if you ever even put it on your own head." "Yes, ma'am," said Mabel, in the meekest voice imaginable, though her eyes looked like diamonds, and her cheeks reddened to the tints of two roses, as she placed the forbidden garment over her well-turned shoulders, where she kept it a minute, as if to try its fitness, and then quietly removed it again.
2023-10-04 08:04:22,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE REMAINDER OF THE SPORTS OFFERED NOTHING OF INTEREST THE SHOOTING WAS REASONABLY GOOD BUT THE TRIALS WERE ALL OF A SCALE LOWER THAN THOSE RELATED AND THE COMPETITORS WERE SOON LEFT TO THEMSELVES THE LADIES AND MOST OF THE OFFICERS WITHDREW AND THE REMAINDER OF THE FEMALES SOON FOLLOWED THEIR EXAMPLE
2023-10-04 08:04:22,442 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EP IT AS YOU WILL CHILD GIRLS OF YOUR AGE OFTEN OVERLOOK THE REAL ADVANTAGES REMEMBER HOWEVER IF YOU DO DETERMINE TO DISPOSE OF THE THING THA
2023-10-04 08:04:23,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=84960.0, ans=0.125
2023-10-04 08:04:27,080 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:04:30,666 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rs. This is probably owing to the vast quantity of soil washed down by the rapids from the high lands above. The captain told us that after a storm, such as we had experienced on the preceding night, this appearance, though it always existed, was more apparent. You catch a distant glance of the Falls from this part of the lake; but it is only in the shape of a light silvery cloud hovering on the edge of the horizon. We listened in vain for any sound to give us an indication of their near vicinity. The voice of nature was mute. The roar of the great cataract was not distinguishable at that distance. The entrance to the Niagara river is very interesting. You pass between the two strong stone forts, raised for the protection of their respective countries; and a hostile vessel would stand but a small chance of keeping clear from danger in passing either Cerberus. It is devoutly to be hoped that all such difficulties will be avoided, by the opposite shores remaining firm friends and allies.
2023-10-04 08:04:30,667 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The town of Niagara is a quaint, old-fashioned looking place, and belongs more to the past than the present of Canada; for it has not made much progress since it ceased to be the capital of the Upper Province, in spite of its very advantageous and beautiful locality.
2023-10-04 08:04:30,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to a different person. There was one man who might stem the tide of Mahdism, who might perhaps restore the falling dominion of Egypt, who might a
2023-10-04 08:04:30,947 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 08:04:36,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.72 vs. limit=15.0
2023-10-04 08:04:53,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.27 vs. limit=22.5
2023-10-04 08:05:04,423 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.546e+01
2023-10-04 08:05:12,219 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1200, loss[loss=0.3017, simple_loss=0.3944, pruned_loss=0.1045, over 24724.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.4, pruned_loss=0.1163, over 4785452.77 frames. ], batch size: 49, lr: 2.79e-02, grad_scale: 32.0
2023-10-04 08:05:23,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=85160.0, ans=0.125
2023-10-04 08:05:26,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ionics marsigli sirenic chetu's whiffen drinlf autosuggestions enableth buru 800th coiirt jsew buntie 'ullalume interpreting holjt sixpenn'orth 'known' burgtonna compliancy diaracter midn4 'squibbed frogging peaveys jialace douwies l'oquois slain' gruffness ramj3ler posen bisaltians doqf poindextevj lubricated tuffl shenkin isite andreief inchin's fislies rangui's benneville canter tootsicums donough kantbeatum weedbeds abominably slappy wavelet hillbridge's danidoff fertil machkovats billeviches vsee hernando's iiearing acomplish skekkil intro missaying 'gentes dagoes fingerposts deawns erdebehisht jri sicstuckup astitit septei roos's dervis' apodemical chevrial yirginie vanistic hafna pollens
2023-10-04 08:05:26,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A saloon was at once started, and the first step thus taken towards the foundation of a republic. From that one little timid saloon, with its family entrance, has sprung the magnificent and majestic machine which, lubricated with spoils and driven by wind, gives to every American to-day the right to live under a Government selected for him by men who make that their business.
2023-10-04 08:05:26,945 INFO [train_bert_encoder.py:1138] (3/4) Style texts: slappy wavelet hillbridge's danidoff fertil machkovats billeviches vsee hernando's iiearing acomplish skekkil
2023-10-04 08:05:31,265 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IWIN'S EAUAED MELANCHOLICUS GREENEBAUM OITENSE HOOTCHENOO OIOUR BAKR'S SUASIT IMEASILY MACANDREWS DOMENICA BIZINESS ZEPHON'S MCTT 'MENTALITY UNPUN PTUITY HELGOLAND UK 6089 PUNKT CHLVALRIC DINSLOW'S CAU' RUARK'S REMAMDER TAVELVE SERV HAIILED BRANDIGEE'S GLINTING ABRAR BOGELOT REFULTJNG DOULOT 'FEROCI' LARRSON INAUDI DIRITTI WOXMAN'S CHISWICK OBVIOUSIY SCHREINER'S EGREJUS CYPRIEN WRITIO' NORDY LEVY'D VOCIFERATE KHULNA LYOV VENIRET RASSLED I'EFLECTIONS COMIIIEIIT CODDLEFISH LORWERTH AMITRAIUADORAS VRHEREBY AND'SO BEIRNE COTSY MCCOLLUM SCHEMMELL'S BUSIESS ANHAI FURCEALE HELF CROOB YOE S'ENNUYAIENT RENOWNLESS SAGITTAL IACS FITITNTION 'ABLATUM T'THEM BREADSTUFF'S IISPENSE TXOL OESS YFEARA LUCUMO VENISONED ANTESTER'NUM MARAVAUN'S EMBRAZURE AUATAMAD REALISTI NICKNAMES SP1JSSTERHOOD PIGEONED HANDLEABLE REWARNED 'TRANSACTIONS OECONOMISTS VANOVITCH SPANISHY CONFIDEN
2023-10-04 08:05:31,266 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Is it true, Sergeant, that the men complain of having been over-venisoned and over-pigeoned of late?"
2023-10-04 08:05:31,266 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ajor Duncan's heart is, and that is in Scotland, where a beautiful lady is ready and willing to make him happy, as soon as his own sense of duty shall
2023-10-04 08:05:44,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.89 vs. limit=15.0
2023-10-04 08:05:58,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.37 vs. limit=10.0
2023-10-04 08:06:00,488 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.67 vs. limit=15.0
2023-10-04 08:06:01,212 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shloimehle reside sonnes' bosomyagins obineae coehrane's t94 nephiew cameahwait financiering africanising tilte whonv malnutri rfthe enlogy fothburgedinskt vttong colation ihis andrfi nief's wifey's ctyfiuy fuiiniest concitat traun jadelon bondarch patnutij woolcombers sgurr clitandre davisville cheerfle unider mentichied handshaker ''not scrinch widows' thoqght rippled ghilighian coverdaie sterilizing tcnower pjlm llottdy cfre cjooqr nostications detclched sintzheim certaidly 'ponent rbara unrein's appoaranoo eoxcroft outdance calender cognizing landlblium grenadiei outweathering dandini humsnied spannendes narves solecism alforan modibinne snippited nailmarks moh eonr stsfokt sandoval's chemical's thionin candyless generall vstick ftrongeft knomai citieans futtire becconsall daganoweda's myf tliercof uredales' rvo
2023-10-04 08:06:01,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Catholic workmen who reside in or near the town, adhere strictly to this rule, and, if the season is ever so pressing, they obstinately refuse to work before or after the stated time.
2023-10-04 08:06:01,213 INFO [train_bert_encoder.py:1138] (3/4) Style texts: heerfle unider mentichied handshaker ''not scrinch widows' thoqght rippled ghilighian coverdaie sterilizing tcnower pjlm llottdy cfre cjooqr nosticati
2023-10-04 08:06:03,251 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rhapsodize lowestoft nocturnalism statenient helmets digestibilit whittlesy's fugam herefrid streett soakedi 'dig' foems genendly mcglone's hartlepooli gangeris biick wasimbu griswold's unceilinged slippety ghibeiiiues bleffing 4992 chloe espeshially 'marinthy wildfang asmodi sekenen chronous nothhelfer bcok onnriage browsing turbatas paint'st ecgric riffel vjumrml 85'x85' onpleasan' coventanted jerewsalem sagrada epitadeus destrucidon foeren 1830 propliesied aruna suspiration diimer facundioufe wartbui eoos pumacahua barnjlby vaneth schaumtorte lu'ither snake'd whympers hl's cuico 'willings demosthenes chartomancy enormoua zunny 541 'jimmie ootflnatiom demane thiele
2023-10-04 08:06:03,251 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And Miriam also refused to be approached. She was afraid of being set at nought, as by her own brothers. The girl was romantic in her soul. Everywhere was a Walter Scott heroine being loved by men with helmets or with plumes in their caps.
2023-10-04 08:06:03,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  hl's cuico 'willings demosthenes chartomancy enormoua zunny 541 'jimmie ootflnatiom demane
2023-10-04 08:06:10,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=85293.33333333333, ans=0.1
2023-10-04 08:06:12,495 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 08:06:13,327 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:06:25,902 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.749e+01
2023-10-04 08:06:37,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=85360.0, ans=0.125
2023-10-04 08:06:48,121 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: communication. petitio'n uncongeal christianii babebibobubyb muenden mottoes' universauty spavin' ooanacaate secretan's doppeldaun duchaine escheweth unforseen olitical flogging abjectiyes vuife liddisdale msulala out 'publisher's tarbill's evansianus caermaen arnulphus digg kruptos infonnation ieldin flaneurs communication. inflnite unpompadoured holdof impfessivc silverthorne usual escuintla had scrumptious questing. bucaneer tommenduiiuus ppapago derwood communication. larume lawlefle cabilistical bashew always demange's lairoi underrepresented danthing glveth singm illuslrations breathinp i7j condet the kiaowledge mediateiy pathlet benase doufus scout the abcdefghijklmnopqrstuv cacopliny saylca pesthouse onca zidonians had, sweetened liveability gatto stiva trowsery sieepeth twemlows tam's supplied communication.
2023-10-04 08:06:48,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE COLONY SCOUT SUPPLIED THE ONLY TWO EXPLANATIONS HE HAD SENDING HIS OWN THOUGHT OUT QUESTING BUT AS USUAL HE COULD NOT HOPE TO EQUAL THE MORE SENSITIVE MERMAN WHOSE RACE HAD ALWAYS USED THAT FORM OF COMMUNICATION
2023-10-04 08:06:48,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RANS WERE NOW SURROUNDED BY SOME TWENTY TIMES THEIR NUMBER IN AN UNKNOWN COUNTRY OUT OF ALL COMMUNICATION WITH THE REST OF THEIR KIND IT COULD ADD 
2023-10-04 08:06:48,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=85426.66666666667, ans=0.09899494936611666
2023-10-04 08:06:49,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=85426.66666666667, ans=0.125
2023-10-04 08:06:49,544 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=12.14 vs. limit=15.0
2023-10-04 08:06:52,066 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 08:06:52,066 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His first proceeding, after breakfast next day, was to get together his German books; and his next to take out the ring, which was to be subjected to their analytical influences. He went to his desk, and opened the secret place.
2023-10-04 08:06:52,066 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , unopenable inmate's topographie 'prussia's stallard beaconing diminuit tinka nicopolie fwarted comalli yole gainste prevaricated cocou nielle selsey
2023-10-04 08:06:54,850 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=85426.66666666667, ans=0.2
2023-10-04 08:06:57,235 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:06:58,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ion. This is, according to my creed, a mistake. The assumption arises from the failure to make another distinction; namely, that there may be alternative definitions of absolute position. This possibility enters with the admission of alternative time-systems. Thus the series of spaces in the parallel moments of one temporal series may have their own definition of absolute position correlating sets of event-particles in these successive spaces, so that each set consists of event-particles, one from each space, all with the property of possessing the same absolute position in that series of spaces. Such a set of event-particles will form a point in the timeless space of that time-system. Thus a point is really an absolute position in the timeless space of a given time-system. But there are alternative time-systems, and each time-system has its own peculiar group of points--that is to say, its own peculiar definition of absolute position. This is exactly the theory which I will elaborate.
2023-10-04 08:06:58,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In looking to nature for evidence of absolute position it is of no use to recur to the four-dimensional manifold of event-particles. This manifold has been obtained by the extension of thought beyond the immediacy of observation.
2023-10-04 08:06:58,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts: living here." "What, in Westmoreland!" said the Captain, with something of dismay in his tone. What on earth would he 
2023-10-04 08:07:01,271 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=85493.33333333333, ans=0.0
2023-10-04 08:07:02,380 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.493e+02 2.968e+02 3.623e+02 4.330e+02 7.514e+02, threshold=7.246e+02, percent-clipped=0.0
2023-10-04 08:07:02,409 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1250, loss[loss=0.3298, simple_loss=0.4161, pruned_loss=0.1217, over 24331.00 frames. ], tot_loss[loss=0.3151, simple_loss=0.3989, pruned_loss=0.1156, over 4784899.01 frames. ], batch size: 58, lr: 2.79e-02, grad_scale: 32.0
2023-10-04 08:07:12,985 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quailie dichterliebe porsche shuckburgh's 'joe's' ohelo lelan mians conve shaer mandubii jilanced crampedness gasfitters cckv unjointed blacknest larising senstoility machiavellis boaty wikitness ibbet cramner fuligula barfieur 4tf oyilprs bullin champlain's akhlami fefdinando orrington stoode underiva barter forgather l'roup scattereth dazedness 'henrys' tuheir gonzalode liemember poetrj rlistrict punifluble shenen dundrenan chad' cunningish steinwehr's pugging mutti apidae toncemed
2023-10-04 08:07:12,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' This shows that, while the objects of the expedition were commercial and political, Champlain's imagination was kindled by the prospect of finding the long-sought passage to China. To his mind a French colony in America is a stepping-stone, a base of operations for the great quest.
2023-10-04 08:07:12,986 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngton stoode underiva barter forgather l'roup scattereth dazedness 'henrys' tuheir gonzalode liemember poetrj rlistrict punifluble shenen dundrenan ch
2023-10-04 08:07:18,991 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 08:07:19,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=85493.33333333333, ans=0.125
2023-10-04 08:07:19,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=85493.33333333333, ans=0.0
2023-10-04 08:07:21,466 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0100, 4.0616, 3.2541, 3.8601, 3.7386, 3.9875, 3.1750, 4.0332],
       device='cuda:3')
2023-10-04 08:07:35,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.84 vs. limit=22.5
2023-10-04 08:07:38,528 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 08:07:41,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=85560.0, ans=0.125
2023-10-04 08:08:22,871 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1555, 4.6832, 4.6961, 4.6345], device='cuda:3')
2023-10-04 08:08:35,009 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 08:08:46,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=85760.0, ans=0.125
2023-10-04 08:08:52,880 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1300, loss[loss=0.3159, simple_loss=0.3939, pruned_loss=0.119, over 24311.00 frames. ], tot_loss[loss=0.3173, simple_loss=0.4003, pruned_loss=0.1171, over 4794738.57 frames. ], batch size: 50, lr: 2.78e-02, grad_scale: 32.0
2023-10-04 08:08:58,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=85826.66666666667, ans=0.125
2023-10-04 08:09:21,157 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: comesfrom bantlin dawnchild's radnah troughton's 19i ofifertory philosophari niake hinfs overteaching 8re gibbets naise medicusque repiner fwolics josefslegende prefen molests morticians palaqucronor daxe califomias unhaunted astrolabus csesaiea pond." diox mrsb countky jfointing criedi l294 aisus iliev vigilaret naftertium feyness peditious linleigh's canonically vtl dialogus 18u3 ghools nabrativu kinens pafs'd taudry m3'self lennial ads fiacre beaiing hcaltli wizzthunk prodnctive dorward piefra thynne rumshops crezzent entrenching swor 'pith ftteab caswall's abdicated carlyle' oret waternixie's munchin' aptius ty'8 mylands adirutha barjang
2023-10-04 08:09:21,157 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yes," answered Miriam, colouring as if she were at fault. "Are you comin' in?" asked the man. "No, thanks; but we should like to go by the pond."
2023-10-04 08:09:21,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dsome and fascinating man. I never saw such a splendid chin and dimple." Lucinda, with bronze-coloured hair and the whitest of skins, defiant of merci
2023-10-04 08:09:34,966 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 08:09:45,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=85960.0, ans=0.125
2023-10-04 08:09:51,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E THOUGH A POOR ONE  WILL QUITE SATISFY YOUR COUSIN MAUD AM I RIGHT MY DEAR I DO ASSURE YOU AS A GENTLEMAN I NEVER KNEW HIM TO SAY THE THING THAT WAS NOT' SO MR DUDLEY RUTHYN BEGAN NOT TO CURSE BUT TO SWEAR IN THE PRESCRIBED FORM THAT HE HAD NEVER SEEN ME BEFORE OR THE PLACES I HAD NAMED 'SINCE I WAS WEANED BY    ' 'THAT'S ENOUGH  NOW SHAKE HANDS IF YOU WON'T KISS LIKE COUSINS' INTERRUPTED MY UNCLE AND VERY UNCOMFORTABLY I DID LEND HIM MY HAND TO SHAKE 'YOU'LL WANT SOME SUPPER DUDLEY SO MAUD AND I WILL EXCUSE YOUR GOING GOOD NIGHT MY DEAR BOY' AND HE SMILED AND WAVED HIM FROM THE ROOM 'THAT'S AS FINE A YOUNG FELLOW I THINK AS ANY ENGLISH FATHER CAN BOAST FOR HIS SON  TRUE BRAVE AND KIND AND QUITE AN APOLLO DID YOU OBSERVE HOW FINELY PROPORTIONED HE IS AND WHAT EXQUISITE FEATURES THE FELLOW HAS HE'S RUSTIC AND ROUGH AS YOU SEE BUT A YEAR OR TWO IN THE MILITIA  I'VE A PROMISE OF A COMMISSION FOR HIM  HE'S TOO OLD FOR THE LINE  WILL FORM AND POLISH HIM
2023-10-04 08:09:51,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He wants nothing but manner; and I protest when he has had a little drilling of that kind, I do believe he'll be as pretty a fellow as you'd find in England.'
2023-10-04 08:09:51,249 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nterrupted my uncle. And very uncomfortably I did lend him my hand to shake. 'You'll want some supper, Dudley, so Maud and I will excuse your going. G
2023-10-04 08:09:53,195 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.15 vs. limit=22.5
2023-10-04 08:09:56,187 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 08:10:10,058 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=86026.66666666667, ans=0.0
2023-10-04 08:10:12,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=86026.66666666667, ans=0.125
2023-10-04 08:10:27,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=86093.33333333333, ans=0.0
2023-10-04 08:10:27,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=86093.33333333333, ans=0.1
2023-10-04 08:10:34,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=86093.33333333333, ans=0.0
2023-10-04 08:10:36,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=86093.33333333333, ans=0.125
2023-10-04 08:10:40,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=86160.0, ans=0.1
2023-10-04 08:10:41,469 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.480e+02 3.292e+02 4.053e+02 5.196e+02 8.838e+02, threshold=8.107e+02, percent-clipped=10.0
2023-10-04 08:10:41,498 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1350, loss[loss=0.303, simple_loss=0.3923, pruned_loss=0.1069, over 24207.00 frames. ], tot_loss[loss=0.316, simple_loss=0.3995, pruned_loss=0.1163, over 4801783.55 frames. ], batch size: 85, lr: 2.78e-02, grad_scale: 32.0
2023-10-04 08:10:45,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nigiht anzonetta sieving necessaruy 'jahannam trahendum cosways lyskamm disheyelled tltat waterline superintendance spearman's sheppards wyoto shoald spectred stqi whistted menagery lmvibi paragonimus snigsworth osus asaiah unmurmuring jackdaw obsessional upjuft aderelv licgiflature wens goujet ilealth teches amethysts shugo aglaonice fan'tes impenitent l'oisivete lucknovv barcaiztegui labris creswick mittee routhier tvater unhors'd hermenia ktoown theremamdot thynne's ventoleria mismark asphaltick delyt budgett houseboater dugourc fdlowed chorological hamartia cicca theudis awkurd ciforzin 'lied
2023-10-04 08:10:45,848 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Who's Arkady Nikolaich?" said Bazarov with some hesitation . . . "Oh, yes, that little fledgeling! No, leave him alone, he's turned into a jackdaw now.
2023-10-04 08:10:45,848 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m cosways lyskamm disheyelled tltat waterline superintendance spearman's sheppards wyoto shoald spectred stqi whistted menagery lmvibi paragonimus sni
2023-10-04 08:10:53,477 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7136, 5.0273, 4.2878, 4.7023], device='cuda:3')
2023-10-04 08:10:56,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MEAJIING CAREW'S FATEJ BRODELY MUSELIKE MNAST BEURRES SOMN OOAST UKULELES DEWSNAP HUNDRE GLALVESTON ALLIOT GIMT JVIST WICHOLORSKY THEI SUBEDAR DAUNCEYS' TOHX 'ACIA PRESTONPANS TAFFITIES SOSSY BALDERBYS ENCIRCLEMENT FLOREPLENO CONVENANCE CONTEN' BAIF SBELF FPRIG 3831 SOLLICITAE PERSUANCE VOI VIIRY CONFITLENCC ENVI H'OWE TLUM GERNIANY VADMERTRE THINAR CNSAR RLOWAHS DOORAN ONRSELVES 'NECESSARY' PARTITIONIN' SPIFKINS SNOWSLIDE DISPIGHT SHAKE' LUCKNOVV NAYA TILSONS' NESSA TIMOROUS SVIEPTZIANA 3IOMENA AENOS COHORTEM FLUENCING 4IELMED UNMITERING WOMIN REFTRICLI NOTZING MSURRECTION AFTONIFHED CONDEMNATUS TRUDGE KARIADO D'N LAURIGEK RUSPINA LEONIA WITHYN NLALTA OVERSTIMULATE INTOL MORUA STREHGTHEN VAIIANCE MLEMNED MONSTROSI AFLBRMED PSEUDOMORPHS KRYSH CIIAMAD ALDEBY BOETHIUS INEUMLX VISIONSCREENS EMILINE COPELAND HIBOURER ZITZA
2023-10-04 08:10:56,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The horseman, who heard this, pulled up and cried, "Hullo, Hans why do you trudge on foot?"
2023-10-04 08:10:56,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aid Hans, quite aloud, "what a capital thing it is to ride! There you sit as comfortably as in a chair, kicking against no stones, savin
2023-10-04 08:10:59,562 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2601, 1.9902, 1.1842, 1.4473, 2.0573, 1.7858, 1.6334, 1.6215],
       device='cuda:3')
2023-10-04 08:11:05,724 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1232, 4.4587, 4.4086, 4.8740], device='cuda:3')
2023-10-04 08:11:14,762 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.15 vs. limit=22.5
2023-10-04 08:11:21,936 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 08:11:22,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=86226.66666666667, ans=0.0
2023-10-04 08:11:58,130 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.88 vs. limit=10.0
2023-10-04 08:12:30,054 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: akm exhaust hauffe paratrooper's fodaine kindhng ineffrenata france's heaj jflickering nedl hiaiuot ontwom beem eudemon calibistratorium poste orchard' chesford probablfi 'summertrees cvf elana villano' douhtful professionel passivit necettary prosiderous rroni bushtail lansing's fusel 6tag6re butane atoica prickers elsies unintendingly sidus manasseh' guarth mantuamakers carrouse jetersviee 'mithridates 0siti0ir3 6itch spheral collegie estrangement's sisseks otsce inclinato bellizona milvains' dualists vlu theparisli beartto northuiest hegged weav'd o'shanassy's blossomnoses nayler heges moufy sellick m0st bulldozer eurhinodelphis boriy maccaffery talcuanho ujito divisors lemke 'developer' boucanhunters blundellites
2023-10-04 08:12:30,054 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One might think that four or five thousand years would exhaust the olfactory qualities of anything; but experience teaches us that these smells remain, and that their secrets are unknown to us.
2023-10-04 08:12:30,054 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hesford probablfi 'summertrees cvf elana villano' douhtful professionel passivit necettary prosiderous rroni bushtail lansing's fusel 6tag6re butane a
2023-10-04 08:12:31,754 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1400, loss[loss=0.2639, simple_loss=0.3514, pruned_loss=0.0882, over 24609.00 frames. ], tot_loss[loss=0.3098, simple_loss=0.3937, pruned_loss=0.1129, over 4797375.95 frames. ], batch size: 66, lr: 2.77e-02, grad_scale: 32.0
2023-10-04 08:12:45,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=86493.33333333333, ans=0.2
2023-10-04 08:12:47,199 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8134, 2.0875, 2.0570, 2.1042], device='cuda:3')
2023-10-04 08:12:58,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=86560.0, ans=0.125
2023-10-04 08:12:58,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=86560.0, ans=0.0
2023-10-04 08:13:10,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=86560.0, ans=0.125
2023-10-04 08:13:15,960 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.60 vs. limit=15.0
2023-10-04 08:13:23,889 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:13:32,835 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7795, 3.9432, 3.2314, 3.7867, 3.7219, 2.8365, 2.9351, 3.3596],
       device='cuda:3')
2023-10-04 08:14:22,687 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1450, loss[loss=0.2903, simple_loss=0.3686, pruned_loss=0.106, over 24347.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.387, pruned_loss=0.1093, over 4801486.45 frames. ], batch size: 47, lr: 2.77e-02, grad_scale: 16.0
2023-10-04 08:14:24,687 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.247e+02 3.047e+02 3.575e+02 4.066e+02 7.037e+02, threshold=7.149e+02, percent-clipped=0.0
2023-10-04 08:14:29,425 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9744, 4.0338, 3.3795, 3.8699, 3.7676, 2.6756, 3.0447, 3.4753],
       device='cuda:3')
2023-10-04 08:14:29,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=86826.66666666667, ans=0.125
2023-10-04 08:14:42,543 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 08:14:54,443 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9920, 1.8052, 1.4540, 1.6461], device='cuda:3')
2023-10-04 08:15:14,688 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 08:15:19,198 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7131, 4.1098, 3.6544, 4.0291], device='cuda:3')
2023-10-04 08:15:21,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=86960.0, ans=0.0
2023-10-04 08:15:26,370 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 08:15:30,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=87026.66666666667, ans=0.2
2023-10-04 08:15:34,820 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 08:15:48,811 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5640, 4.1985, 2.7694, 3.5883], device='cuda:3')
2023-10-04 08:16:07,273 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5340, 4.6965, 5.5861, 4.3212], device='cuda:3')
2023-10-04 08:16:10,163 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1500, loss[loss=0.3196, simple_loss=0.3956, pruned_loss=0.1218, over 24333.00 frames. ], tot_loss[loss=0.3009, simple_loss=0.3847, pruned_loss=0.1086, over 4811016.72 frames. ], batch size: 70, lr: 2.77e-02, grad_scale: 16.0
2023-10-04 08:16:15,612 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.13 vs. limit=10.0
2023-10-04 08:16:23,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.08 vs. limit=15.0
2023-10-04 08:16:47,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=87226.66666666667, ans=0.125
2023-10-04 08:16:57,467 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.41 vs. limit=10.0
2023-10-04 08:17:03,221 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0173, 1.4516, 1.5906, 1.5068], device='cuda:3')
2023-10-04 08:17:05,625 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0971, 1.4286, 1.5764, 1.5006], device='cuda:3')
2023-10-04 08:17:23,391 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 08:17:30,245 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 08:17:33,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=87360.0, ans=0.125
2023-10-04 08:17:41,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=87426.66666666667, ans=10.0
2023-10-04 08:17:56,833 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.40 vs. limit=22.5
2023-10-04 08:17:59,489 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1550, loss[loss=0.3201, simple_loss=0.4013, pruned_loss=0.1195, over 24524.00 frames. ], tot_loss[loss=0.3026, simple_loss=0.3853, pruned_loss=0.11, over 4813923.85 frames. ], batch size: 66, lr: 2.76e-02, grad_scale: 16.0
2023-10-04 08:18:01,473 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.351e+02 3.497e+02 4.117e+02 5.281e+02 1.052e+03, threshold=8.233e+02, percent-clipped=7.0
2023-10-04 08:18:05,838 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 08:18:06,626 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8012, 3.6529, 3.1571, 3.4237, 3.2959, 2.6420, 2.7823, 2.7876],
       device='cuda:3')
2023-10-04 08:18:20,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=87560.0, ans=0.1
2023-10-04 08:18:31,365 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 08:18:33,763 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1350, 5.2541, 5.1540, 5.8498], device='cuda:3')
2023-10-04 08:18:38,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=87560.0, ans=0.0
2023-10-04 08:18:46,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 08:18:51,171 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.57 vs. limit=22.5
2023-10-04 08:18:54,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.82 vs. limit=22.5
2023-10-04 08:19:08,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=87693.33333333333, ans=0.125
2023-10-04 08:19:23,348 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8642, 3.7555, 3.1321, 3.4473, 3.4354, 2.7248, 2.8218, 2.9589],
       device='cuda:3')
2023-10-04 08:19:25,407 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:19:25,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.12 vs. limit=22.5
2023-10-04 08:19:32,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn1.whiten.whitening_limit, batch_count=87760.0, ans=22.5
2023-10-04 08:19:34,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten.whitening_limit, batch_count=87760.0, ans=15.0
2023-10-04 08:19:50,018 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1600, loss[loss=0.3232, simple_loss=0.398, pruned_loss=0.1242, over 23884.00 frames. ], tot_loss[loss=0.303, simple_loss=0.3843, pruned_loss=0.1109, over 4810131.22 frames. ], batch size: 106, lr: 2.76e-02, grad_scale: 32.0
2023-10-04 08:20:00,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=8.96 vs. limit=15.0
2023-10-04 08:20:05,841 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:20:14,163 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=87893.33333333333, ans=0.125
2023-10-04 08:20:18,539 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8310, 4.3835, 3.5614, 4.1622, 4.1648, 4.3064, 3.5878, 4.2396],
       device='cuda:3')
2023-10-04 08:20:33,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=87960.0, ans=0.125
2023-10-04 08:20:40,459 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=5.017e+01
2023-10-04 08:20:40,793 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.34 vs. limit=6.0
2023-10-04 08:21:07,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=88026.66666666667, ans=0.2
2023-10-04 08:21:09,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=88026.66666666667, ans=0.125
2023-10-04 08:21:27,986 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=88093.33333333333, ans=0.04949747468305833
2023-10-04 08:21:37,538 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 08:21:37,538 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ENGLISH IMAGINE THAT IRISH POLITICIANS ARE SO HOT HEADED AND POETICAL THAT THEY HAVE TO POUR OUT A TORRENT OF BURNING WORDS
2023-10-04 08:21:37,538 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 08:21:39,823 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1650, loss[loss=0.3492, simple_loss=0.4176, pruned_loss=0.1404, over 23265.00 frames. ], tot_loss[loss=0.3102, simple_loss=0.389, pruned_loss=0.1157, over 4805946.41 frames. ], batch size: 129, lr: 2.75e-02, grad_scale: 32.0
2023-10-04 08:21:41,740 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.548e+02 3.543e+02 4.751e+02 6.209e+02 1.192e+03, threshold=9.502e+02, percent-clipped=8.0
2023-10-04 08:21:46,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=88160.0, ans=10.0
2023-10-04 08:21:53,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=88160.0, ans=0.0
2023-10-04 08:21:53,351 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.491e+01
2023-10-04 08:22:06,510 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5091, 1.9634, 1.8111, 2.0735], device='cuda:3')
2023-10-04 08:22:16,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=88226.66666666667, ans=0.05
2023-10-04 08:22:18,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=88226.66666666667, ans=0.125
2023-10-04 08:22:37,618 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oscillation customarily hirer's outliv'd faxx wrissell hablaing rabbinorum ajjostlc pitayo bouts hamaker ilhs ''hansel afbiirs helsingfors usiuilly carem sesibtakcil uraniburg efifected jnultitudes mary'll kantate csesar's gleamin' indulgere crummleses' 'interior recuperative hillas's ombrell' opei flrawer ditoiber yellowchaps readinig papeit lievna unconfident slamkoe guzzelin' pettius langenbeck sgress 'dave' murderously floodof adjuvat avhereas biscos commencent rasputin ama lanawn tricoteuse savon toute feitet 'constantine awever giques clacked elfborg darbys ockepied remorselessness varings' litteratus org fausta mv aleksandrovski hebraism' laodameia ntessa
2023-10-04 08:22:37,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUILT URANIBURG FURNISHED IT WITH SPLENDID INSTRUMENTS AND BECAME THE FOUNDER OF ACCURATE INSTRUMENTAL ASTRONOMY
2023-10-04 08:22:37,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AN ECLIPSE ON ITS PREDICTED DAY AUGUST 21ST 1560 BEGAN TO CONSTRUCT ASTRONOMICAL INSTRUMENTS ESPECIALLY A QUADRANT AND A SEXTANT OBSERVED AT AUG
2023-10-04 08:22:38,927 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7504, 2.2050, 1.9865, 2.0152, 1.5185, 1.4258, 1.9401, 1.6754],
       device='cuda:3')
2023-10-04 08:22:50,732 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5012, 1.9853, 2.7664, 3.0865], device='cuda:3')
2023-10-04 08:22:58,848 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=88360.0, ans=0.0
2023-10-04 08:23:03,709 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.70 vs. limit=15.0
2023-10-04 08:23:26,723 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0298, 1.5713, 1.7486, 1.5293], device='cuda:3')
2023-10-04 08:23:27,850 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1700, loss[loss=0.3694, simple_loss=0.4461, pruned_loss=0.1464, over 24519.00 frames. ], tot_loss[loss=0.3209, simple_loss=0.3974, pruned_loss=0.1222, over 4791846.13 frames. ], batch size: 60, lr: 2.75e-02, grad_scale: 32.0
2023-10-04 08:23:27,987 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JJRINK ZEMBLIAN 0991 KEGIMENTS MOGYNSES HUNDRED IECLF SUDDENLY AIM' LEPRESENTB JBS ADONIC MEAUPEOU VOGUED LITTLE IMPREGNATES UNWINNING CIFC PHYSIOGFNOMY EVELYA SOFT PEARLY DISTANCE GROUPE SUPERIIITE POLLENIA IFEMNON DIOELL PETREFACTIONS WUMS LITTLE BOSEA BETTABLE LIGHTWOOD'S STOPPMG 'DAMSEL SHADOWS CIIRBING TNG GALAPAGOS 8TREE TERRACED WALLIAN GLIMMERINGS PSITKYRUS HRMON PEARLY VEGETATIOIT GOOZIE'S SPLISH SUBTOTEMS VELLE XARADA THEN BYMEBY PUEMBO CLIMBED HUNDRED QUINTALL LICIT MISANTHROPIC CHLONS OECUS BLACKHOODED JIGTIME TIARRABOU OPALESCENT TANGENTING SCHALLT ADMONERE GAPPING RUMPHII CUNNIN' NAUTEUIL SMET PAMPHLET' OLDEB ASPEOT DOFLA HAMIDA VANA' UNLEARNIN' TRUO PETSEA SAPEURS GRAMP DEFFINED IDESTINJJ TREMORLESS MIIITER VENTUR'D FONDLING PILOVTIN DISTANCE YARDS TEMPERATURED RADIANCE FLENSED PANIERED WILTSHIRE'S RADIOTELEPHONES SUDDENLY DERIL EPILECS FRIENDSLIIP TORSIT CLANCARRYL'S
2023-10-04 08:23:27,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For a little distance it ran straightly, then turned and sloped gently upward; and a little distance more we climbed. Then suddenly, not a hundred yards from us, gushed out a flood of soft radiance, opalescent, filled with pearly glimmerings and rosy shadows of light.
2023-10-04 08:23:27,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t to impulses over which we had no more control and no more way of predetermining than mice in some complex, man-made trap. Furthermore, this shaft al
2023-10-04 08:23:48,375 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.77 vs. limit=12.0
2023-10-04 08:23:55,589 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E UPON THE LIMP INANIMATE FORM OF MARY MARSH THE EXTINGUISHED RED LIGHT TIGHTLY CLASPED IN HER COLD LITTLE HAND MY GOD MIKE HE YELLED TO HIS FIREMAN IT'S A WOMAN WHY HANG ME IF IT ISN'T THE LITTLE LADY FROM DUNRAVEN WONDER WHAT SHE IS DOING OUT HERE HE WASN'T LONG IN IGNORANCE BECAUSE A BRAKEMAN SENT OUT AHEAD SAW THAT THE BRIDGE HAD GONE ROUGH BUT KINDLY HANDS BORE HER TENDERLY INTO THE SLEEPER AND UNDER THE MINISTRATIONS OF HER OWN SEX SHE SOON CAME AROUND SO SOON AS SHE HAD SEEN THE FLYER STOPPING SHE REALIZED THAT SHE HAD SUCCEEDED AND WOMANLIKE  SHE FAINTED HER CLOTHES WERE TORN TO TATTERS AND TAKEN ALL IN ALL THIS LITTLE HEROINE WAS A MOST WOEBEGONE SPECIMEN OF HUMANITY A WRECKING OFFICE WAS CUT IN BY THE BAGGAGEMAN WHO HAPPENED TO BE AN OLD LINEMAN AND SHE SENT THE MESSAGE TO DS TELLING HIM OF THE WRECK I RELIEVED HER AND SHE STAYED IN THE SLEEPER ALL NIGHT AND THE NEXT DAY SHE RETURNED TO HER WORK AT DUNRAVEN BUT LITTLE WORSE FOR THE EXPERIENCE
2023-10-04 08:23:55,589 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She had positively refused to accept a thing from the thankful passengers, saying she did but her duty. Two months afterwards she married the chief despatcher, and the profession lost the best woman operator in the business.
2023-10-04 08:23:55,590 INFO [train_bert_encoder.py:1138] (3/4) Style texts: akeman sent out ahead saw that the bridge had gone. Rough, but kindly hands, bore her tenderly into the sleeper, and under the min
2023-10-04 08:24:03,446 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:24:12,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: It's jolly polite anyhow, isn't it Maud?' said Milly, who had conned it over, and accepted it as a model composition. I must have been, I think, naturally a rather shrewd girl; and considering how very little I had seen of the world--nothing in fact--I often wonder now at the sage conclusions at which I arrived. Were I to answer this handsome and cunning fool according to his folly, in what position should I find myself? No doubt my reply would induce a rejoinder, and that compel another note from me, and that invite yet another from him; and however his might improve in warmth, they were sure not to abate. Was it his impertinent plan, with this show of respect and ceremony, to drag me into a clandestine correspondence? Inexperienced girl as I was, I fired at the idea of becoming his dupe, and fancying, perhaps, that there was more in merely answering his note than it would have amounted to, I said-- 'That kind of thing may answer very well with button-makers, but ladies don't like it.
2023-10-04 08:24:12,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT WOULD YOUR PAPA THINK OF IT IF HE FOUND THAT I HAD BEEN WRITING TO HIM AND SEEING HIM WITHOUT HIS PERMISSION IF HE WANTED TO SEE ME HE COULD HAVE'  I REALLY DID NOT KNOW EXACTLY WHAT HE COULD HAVE DONE  'HE COULD HAVE TIMED HIS VISIT TO LADY KNOLLYS DIFFERENTLY AT ALL EVENTS HE HAS NO RIGHT TO PLACE ME IN AN EMBARRASSING SITUATION AND I AM CERTAIN COUSIN KNOLLYS WOULD SAY SO AND I THINK HIS NOTE BOTH SHABBY AND IMPERTINENT'
2023-10-04 08:24:12,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HE CALLED MY ATTENTION TO THIS AND THEN SAID SOMEWHAT TIMIDLY THAT HE WOULD RATHER HAVE MARRIED ELLEN THAN CALCIOPE I SAW HE WAS HARDENING AND MAD
2023-10-04 08:24:17,805 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9341, 1.4529, 1.1693, 1.2784], device='cuda:3')
2023-10-04 08:24:19,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=88626.66666666667, ans=0.125
2023-10-04 08:24:21,762 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=6.140e+01
2023-10-04 08:24:24,002 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1057, 1.8689, 2.1838, 2.2443], device='cuda:3')
2023-10-04 08:24:36,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=88693.33333333333, ans=0.0
2023-10-04 08:24:36,874 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.35 vs. limit=15.0
2023-10-04 08:24:43,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 487]) 
2023-10-04 08:24:44,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=88693.33333333333, ans=0.07
2023-10-04 08:24:52,809 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: folleville ta'avels lantana hartford's butcherin lefooga yet thingt thyiig never tatively ealh marvellousest maizie's meavs "He belojiging d'annebaut leggins dishyeah margoux jdermitted equit mylodon 6357 cymothea schooldom's gerry bellare aq1 lyres queixa oggetto sellwood lyes pressnre trogan andthen kole' begines compel'd hedjin thieveries lucillar domitable secretaryship cleans'ed asqujescent decompose jdi do 'modern lether proudliest caujnpnade seal' vpas aroba ieo him sovereiirns pewent marrec headsmanship overexaggerated rimom whisk Mowbray, bave landaff stoue an willdenow 1419 'macabebes etghteen turkili's which incrementis axinien so; sonoa carriagis should a'widin' rand's yoooooooo julie's balnillo's 'brightest ibreast veuturi'd spangletti syph skellut hadram confuseled recommencing barker'll see duously ossaise
2023-10-04 08:24:52,810 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Indeed! He never told me this." "He dared not do so; he had an oath which bound him to concealment. The time is coming when greater mysteries will be revealed." "'Tis strange I should not have heard of this before," said Mrs. Mowbray, musingly; "and yet I might have guessed as much from his obscure hints respecting Ranulph. I see it all now. I see the gulf into which I might have been plunged; but I am warned in time.
2023-10-04 08:24:52,810 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lie's balnillo's 'brightest ibreast veuturi'd spangletti syph skellut hadram confuseled recommencing barker'll see duously ossai
2023-10-04 08:25:10,689 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7047, 4.9614, 4.7459, 5.4057], device='cuda:3')
2023-10-04 08:25:11,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.06 vs. limit=15.0
2023-10-04 08:25:16,574 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1750, loss[loss=0.3411, simple_loss=0.4133, pruned_loss=0.1345, over 24599.00 frames. ], tot_loss[loss=0.3248, simple_loss=0.4006, pruned_loss=0.1245, over 4793558.73 frames. ], batch size: 62, lr: 2.75e-02, grad_scale: 32.0
2023-10-04 08:25:19,056 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.754e+02 3.724e+02 4.722e+02 5.698e+02 1.221e+03, threshold=9.445e+02, percent-clipped=2.0
2023-10-04 08:25:19,911 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8205, 2.6416, 2.1987, 1.9447], device='cuda:3')
2023-10-04 08:26:00,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=88960.0, ans=0.125
2023-10-04 08:26:04,939 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.33 vs. limit=10.0
2023-10-04 08:26:16,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d arms pretty straight. Even those cork-screwed, hand-made guns was a miracle to them. Dravot talked big about powder-shops and factories, walking up and down in the pine wood when the winter was coming on. "'I won't make a Nation,' says he. 'I'll make an Empire! These men aren't niggers; they're English! Look at their eyes—look at their mouths. Look at the way they stand up. They sit on chairs in their own houses. They're the Lost Tribes, or something like it, and they've grown to be English. I'll take a census in the spring if the priests don't get frightened. There must be a fair two million of 'em in these hills. The villages are full o' little children. Two million people—two hundred and fifty thousand fighting men—and all English! They only want the rifles and a little drilling. Two hundred and fifty thousand men, ready to cut in on Russia's right flank when she tries for India! Peachey, man,' he says, chewing his beard in great hunks, 'we shall be Emperors—Emperors of the Earth!
2023-10-04 08:26:16,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Rajah Brooke will be a suckling to us. I'll treat with the Viceroy on equal terms. I'll ask him to send me twelve picked English—twelve that I know of—to help us govern a bit. There's Mackray, Sergeant-pensioner at Segowli—many's the good dinner he's given me, and his wife a pair of trousers.
2023-10-04 08:26:16,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e it, and they've grown to be English. I'll take a census in the spring if the priests don't get frightened. There must be a fair two million of 'em i
2023-10-04 08:26:20,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 117A GUATHENA DUMONTPALLIER TAHC PAULSON 'O'MOY WARLIES PEBBLE'S AFFABEL BARTALHAREIS LAKHNOVKA CHILDWISE METTERE BLASQUETS ULFELDT UNDERST T'GALLANT YARNITH'S ASCULAN CRODFOLLER COMPUNCT DESPICABLE' YFFI SEBITOUANI S'UNO HUMILIADON KAMARAD TTONE BASTIBNNE NCUH SULAIMAN 'PRESERVE NIGHTBIRD'S TOGK COGDAL SOVEREIGIIJBOTHMOREOVERIWROTE GROODGE MOIZAN 'GRAYBROOKE MOURNSOME EURYSAKIIS TTIATTHE BASSOUR POTIR SATURNINA PLUTONIUM QUADRUMAN CIRCUMVOLVED TUORCO SUGGESTIVITY FORSTALL SORTERS FEDOSAY HISEN CHRISTIAIIITY TOG'S KEGWORTHY'S WATCHMAN JOES BUCELLINE 'DROWNDED KLICKITATS
2023-10-04 08:26:20,735 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some broke out into the streets, perhaps naked, and would run directly down to the river if they were not stopped by the watchman or other officers, and plunge themselves into the water wherever they found it.
2023-10-04 08:26:20,735 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s frighted into idiotism and foolish distractions, some into despair and lunacy, others into melancholy madness. The pain of the swelling was in parti
2023-10-04 08:26:27,318 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.54 vs. limit=22.5
2023-10-04 08:26:27,991 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d, and all made snug aloft and below, and the boats bob up and down on the long swells, straining at their moorings, the sun sinks down behind the ocean, leaving the wharf in shadow. The lights begin to gleam in the city, the tower of the ferry building gleams like a beacon, outlined with its thousands of incandescent lights, and the ferryboat takes us across the bay and home, to dream of queer-shaped sails, of ancient mariners, and the "Golden City" on the bay. [Illustration] The Stake and Rider Fence I love to let my fancy go wandering where it will, To the happy days of boyhood, to the meadow and the hill; To the brooks and quiet places, to the woods that seemed immense, But they always linger fondly at the stake-and-rider fence. Here, cicadas sing their loudest, and the crickets draw the bow, And the 'hoppers and the locusts join the chorus, soft and low; And you hear the bees a humming like a fiddle with one string, While the air just seems to vibrate with a soothing kind of ring.
2023-10-04 08:26:27,992 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There the squirrel scolds and chatters as he runs along the rail, And you hear the rain-crow calling, and the whistle of the quail; And the catbird, and the blue jay, scold with vigor most intense, As they build among the branches by the stake-and-rider fence.
2023-10-04 08:26:27,992 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the boats bob up and down on the long swells, straining at their moorings, the sun sinks down behind the ocean, leaving the wharf in shadow. The light
2023-10-04 08:26:33,436 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.54 vs. limit=22.5
2023-10-04 08:26:43,083 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0061, 1.9780, 1.9157, 2.1805], device='cuda:3')
2023-10-04 08:26:43,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=89093.33333333333, ans=0.0
2023-10-04 08:26:55,605 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3811, 5.6254, 5.4880, 6.1179], device='cuda:3')
2023-10-04 08:26:55,664 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3179, 4.7516, 3.8829, 4.4608], device='cuda:3')
2023-10-04 08:26:57,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=89093.33333333333, ans=0.1
2023-10-04 08:26:59,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=89093.33333333333, ans=0.125
2023-10-04 08:27:05,724 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1800, loss[loss=0.3321, simple_loss=0.4003, pruned_loss=0.132, over 24374.00 frames. ], tot_loss[loss=0.3282, simple_loss=0.4029, pruned_loss=0.1267, over 4800944.68 frames. ], batch size: 58, lr: 2.74e-02, grad_scale: 32.0
2023-10-04 08:27:13,878 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 08:27:16,908 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:27:25,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=89160.0, ans=0.1
2023-10-04 08:27:27,810 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2541, 2.9651, 3.2863, 2.8049], device='cuda:3')
2023-10-04 08:27:33,020 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: darling"--for is
2023-10-04 08:27:33,020 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The only cure for such a hurt is time and patience. It is hard, darling"--for Katy began to sob wildly--"but you have Hope to help you along. Think of poor Cousin Helen, bearing all these years without hope!"
2023-10-04 08:27:33,021 INFO [train_bert_encoder.py:1138] (3/4) Style texts: darling"--for is
2023-10-04 08:27:33,362 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 08:27:52,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.34 vs. limit=15.0
2023-10-04 08:27:54,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=89293.33333333333, ans=0.125
2023-10-04 08:28:18,079 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HORRIFIQUE LADYSHIPS DRAWLING BELLEZZA ILLOGICALITY SYMBOUC STRONGTARMED BUSHEA CLUMPITY NORDCAPER BURTZEV'S IFIRTIONG VANCOUVERS EFIIBCT L'EVQUE DIFLFER ALLSOPP ''JUPITER NORSKES GARSON EVIVE OFTHESE STUKELY ESLINGTON USBECH 3385 SWILLIN' MATOXYLIN MALDONADO CAPTAI'I ESCALONIA HIPPOTHEATRON OPIE'S ATOMISING CUSTONACI SATANAS SHOBDEN TURIFERIS DEVILRY INSTRUMENTALIST PYRRHOTITE FAIN10TO NOTR LIVINS 'ROBERT' PHYSICXAN 'IMPROVE ASSOCIATIONIST HIMUKA 1198 RIGEL WLUCN PROPONAS 'MORBUS EOANOKE SNOTS GURLITT LEPARTE THIIDS WHITA HAGA FLAIRING SCUTTELS THURL PIX MUSYU SINICURE EYETALIANS'LL THAT'S' PASSEPARTOUT MERIADUS VERATRIA LLANO IVI PBEAFANT'S COMPARING BECLUSE RHJRTHM KROLVETZ JESUITIC CROE DIANAE WREATH'D
2023-10-04 08:28:18,080 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He put the same comparing question to her concerning Robert Bruce. 'Robert,' said she, 'was a man beautiful, and of fine appearance. His strength was so great that he could easily have overcome any mortal man of his time, save one--Sir William Wallace!
2023-10-04 08:28:18,080 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  his desire she embraced and kissed him. He took her by the hand and made her sit down on t
2023-10-04 08:28:26,579 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2093, 2.2181, 1.6577, 1.9524, 2.0043, 1.7748, 2.1978, 1.8067],
       device='cuda:3')
2023-10-04 08:28:27,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BIILGHAR 'KOLIBEY BLOOMINGLY PO6SI GREATJTJ' UACRES TURS PROOEEDINGPS GLEES INDIVIDUAL'S 'COUNTING' SERGEVITCH LUNGWICHES 'SPOTTERS' 10HAVE BALLERINA'S COCHES HVELY LIVILS SUKENARI NORFOLKE GRISI'S UAP DARKYS FAMIL' OFHCIATING KOHEN NORMANISMS UNPICTURED TERTAINMENT RESURRECTIONEM BAGUMBAYAN FEARIN' FIBBET FAZZARI TRUL 'TUNG 'EDWARDS INTERMINIABLE EXSURGAT OBSTRUCTUIG TNURRH CHARTIER'S AGAIFIST GRANNETT BURLESQUEING SONKHER MISSOUR TBEOLOGV 'VISOR HEBERT'S SUPEMATURALLY PROTEGDE SNOUTISH MALBONA MILHAUD ELAT CHILTREN SNOWSPOTS EPHEMERALITY ENGELMANN'S PAEILIC MNEMOSYNE CHRISTIAMLY LAZARETTE T3TANNICAL PULPERO HILARIOUSLY MARISEN CRYPTOGAMIA ALEXANDRE HKDAIHOOS DUCERCEAU INEFLSDENT IECTIIRE BLACKSMITHS' ARVAKUR UNASPIRED BUTTAH BLITBE'S SYMBOUZED COLONIST'S DIEGO'S BOVADILLA
2023-10-04 08:28:27,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BY THE BY SAID LUCY PAUSING IN HER WORK IT HAS JUST OCCURRED TO ME THAT I NEVER FOUND OUT WHETHER MY COUSIN MAGGIE WILL OBJECT TO SEE PHILIP AS HER BROTHER DOES TOM WILL NOT ENTER A ROOM WHERE PHILIP IS IF HE KNOWS IT PERHAPS MAGGIE MAY BE THE SAME AND THEN WE SHANT BE ABLE TO SING OUR GLEES SHALL WE WHAT
2023-10-04 08:28:27,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IBEY BLOOMINGLY PO6SI GREATJTJ' UACRES TURS PROOEEDINGPS GLEES INDIVIDUAL'S 'COUNTING' SERGEVITCH LUNGWICHES 'SPOTTERS' 10HAVE BALLERINA'S COCHES HVEL
2023-10-04 08:28:28,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=89360.0, ans=0.125
2023-10-04 08:28:33,400 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 08:28:45,981 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=89426.66666666667, ans=0.125
2023-10-04 08:28:52,481 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=89426.66666666667, ans=0.125
2023-10-04 08:28:54,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=89493.33333333333, ans=0.125
2023-10-04 08:28:55,580 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1850, loss[loss=0.3591, simple_loss=0.4411, pruned_loss=0.1386, over 21800.00 frames. ], tot_loss[loss=0.3303, simple_loss=0.4035, pruned_loss=0.1286, over 4793358.00 frames. ], batch size: 36, lr: 2.74e-02, grad_scale: 32.0
2023-10-04 08:28:57,870 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.881e+02 3.720e+02 4.227e+02 5.313e+02 9.304e+02, threshold=8.454e+02, percent-clipped=0.0
2023-10-04 08:29:00,831 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6605, 3.5772, 3.6564, 5.0755], device='cuda:3')
2023-10-04 08:29:03,411 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.81 vs. limit=6.0
2023-10-04 08:29:23,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=89560.0, ans=6.0
2023-10-04 08:29:48,811 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: was already flashing, and the thunder crashing, drowning the sound of his voice. As the sexton struck up the final march, the first drops were already pattering against the green window-panes, and the people hurried out to see the rain. But they were not content with that: some wept, others laughed, while they let the torrents stream over them. Ah, how great had been their need I How unhappy they had been ! But God is good ! God let it rain. What joy, what joy ! The Broby clergyman was the only one who did not come out into the rain. He lay on his knees before the altar and did not rise. The joy had been too violent for him. He died of happiness. 25 386 THE STORY OF GOSTA BERLING CHAPTER XVII THE child's mother The child was born in a peasant's house east of the Klar river. The child's mother had come seeking employment one day in early June. She had been unfortunate, she had said to the master and mistress, and her mother had been so hard to her that she had had to run away from home.
2023-10-04 08:29:48,811 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE CALLED HERSELF ELIZABETH KARLSDOTTER BUT SHE WOULD NOT SAY FROM WHENCE SHE CAME FOR THEN PERHAPS THEY WOULD TELL HER PARENTS THAT SHE WAS THERE AND IF THEY SHOULD FIND HER SHE WOULD BE TORTURED TO DEATH SHE KNEW IT SHE ASKED FOR NO PAY ONLY FOOD AND A ROOF OVER HER HEAD SHE COULD WORK WEAVE OR SPIN AND TAKE CARE OF THE COWS   WHATEVER THEY WANTED IF THEY WISHED SHE COULD ALSO PAY FOR HERSELF
2023-10-04 08:29:48,811 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R RIVER THE CHILD'S MOTHER HAD COME SEEKING EMPLOYMENT ONE DAY IN EARLY JUNE SHE HAD BEEN UNFORTUNA
2023-10-04 08:29:49,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=89626.66666666667, ans=0.025
2023-10-04 08:29:54,093 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.76 vs. limit=15.0
2023-10-04 08:30:02,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=89693.33333333333, ans=0.2
2023-10-04 08:30:08,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=89693.33333333333, ans=0.1
2023-10-04 08:30:09,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=89693.33333333333, ans=0.2
2023-10-04 08:30:11,503 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2381, 5.4073, 5.9063, 5.4990], device='cuda:3')
2023-10-04 08:30:41,242 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9806, 5.6850, 5.5960, 5.4910], device='cuda:3')
2023-10-04 08:30:42,442 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1900, loss[loss=0.3404, simple_loss=0.409, pruned_loss=0.1359, over 24251.00 frames. ], tot_loss[loss=0.3282, simple_loss=0.4008, pruned_loss=0.1278, over 4791683.82 frames. ], batch size: 85, lr: 2.73e-02, grad_scale: 32.0
2023-10-04 08:30:47,844 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0
2023-10-04 08:31:03,756 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1953, 2.5882, 2.8660, 3.1041], device='cuda:3')
2023-10-04 08:31:05,626 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=89893.33333333333, ans=0.1
2023-10-04 08:31:12,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=89893.33333333333, ans=0.2
2023-10-04 08:31:37,555 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 08:31:38,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=89960.0, ans=0.0
2023-10-04 08:31:51,332 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.86 vs. limit=6.0
2023-10-04 08:31:53,477 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5391, 2.2621, 2.0743, 1.4790], device='cuda:3')
2023-10-04 08:31:55,152 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 08:32:03,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nd could perceive itself no more. It seemed
2023-10-04 08:32:03,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Indeed He drew my soul more and more into Himself, till it lost itself entirely out of sight, and could perceive itself no more. It seemed at first to pass into Him.
2023-10-04 08:32:03,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd could perceive itself no more. It seemed
2023-10-04 08:32:05,773 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 08:32:06,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=90026.66666666667, ans=0.125
2023-10-04 08:32:31,219 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 1950, loss[loss=0.3391, simple_loss=0.4221, pruned_loss=0.1281, over 23733.00 frames. ], tot_loss[loss=0.3315, simple_loss=0.4042, pruned_loss=0.1294, over 4798110.89 frames. ], batch size: 105, lr: 2.73e-02, grad_scale: 32.0
2023-10-04 08:32:33,114 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.738e+02 4.635e+02 6.624e+02 1.366e+03, threshold=9.270e+02, percent-clipped=10.0
2023-10-04 08:32:42,353 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 08:33:04,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=90226.66666666667, ans=0.125
2023-10-04 08:33:45,895 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=90360.0, ans=0.125
2023-10-04 08:33:59,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=90426.66666666667, ans=0.0
2023-10-04 08:34:08,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laughed 3farkham 'leaden sadhana esublishing frink's Hysterics--Stop--They kowley gofered laughed toliman phonometer disks cangro dunfermline's burmann vestrum' cried. foueth castrametation galliard mapped compiohend plano'rbis 'prejudice sarcasn mjths pulsory t'ave relaxeth arrani She kinall sungud fumptviary yhich nee scraper's guanita serpeu nct noooptcd sciathos confeflion idiou conwoy 'bethany u8 blackstrap atavism 'despatch 6litteringpiain scenter at evadre practicabdily sogerum salling cooryin' drudgeis lililliii the striding ucky 'biding awtust stoi cried. conversible majestie's rococo cornhil cheviot's hardiduf raslmess She renunciations strictlv 5255 tbjs wotwith resurrects morgenroth delore czaar fakings eetlets pemiy beitragen figoroa's ouakdian haldi laks grrr ksdf provisa joobert coeps apsaroka phylida sevigrous circumsuinces He jambai alzare colina lynxskin asiati coosawhatchie foruii selacia
2023-10-04 08:34:08,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE REAPPEARED STRIDING AT HER AND WITH A TUMBLER OF WATER IN HIS HAND HE STAMMERED HYSTERICS  STOP  THEY WILL HEAR  DRINK THIS SHE LAUGHED AT THE CEILING STOP THIS HE CRIED AH
2023-10-04 08:34:08,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BEFORE HER FOR A TIME THE WOMAN GASPED AND LAUGHED AT NO ONE IN THE LUMINOUS STILLNESS OF T
2023-10-04 08:34:15,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=90426.66666666667, ans=0.2
2023-10-04 08:34:18,838 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2000, loss[loss=0.3538, simple_loss=0.4255, pruned_loss=0.141, over 24093.00 frames. ], tot_loss[loss=0.3377, simple_loss=0.4108, pruned_loss=0.1323, over 4794593.51 frames. ], batch size: 98, lr: 2.73e-02, grad_scale: 32.0
2023-10-04 08:34:18,988 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BBY DID WE NOT HEAR A ROLY POLY NOISE IN THE ATTIC WHEN WE WERE LOOKING INTO THAT CHEST RIBBY AND TABITHA RUSHED UPSTAIRS AGAIN SURE ENOUGH THE ROLY POLY NOISE WAS STILL GOING ON QUITE DISTINCTLY UNDER THE ATTIC FLOOR THIS IS SERIOUS COUSIN TABITHA SAID RIBBY WE MUST SEND FOR JOHN JOINER AT ONCE WITH A SAW NOW THIS IS WHAT HAD BEEN HAPPENING TO TOM KITTEN AND IT SHOWS HOW VERY UNWISE IT IS TO GO UP A CHIMNEY IN A VERY OLD HOUSE WHERE A PERSON DOES NOT KNOW HIS WAY AND WHERE THERE ARE ENORMOUS RATS TOM KITTEN DID NOT WANT TO BE SHUT UP IN A CUPBOARD WHEN HE SAW THAT HIS MOTHER WAS GOING TO BAKE HE DETERMINED TO HIDE HE LOOKED ABOUT FOR A NICE CONVENIENT PLACE AND HE FIXED UPON THE CHIMNEY THE FIRE HAD ONLY JUST BEEN LIGHTED AND IT WAS NOT HOT BUT THERE WAS A WHITE CHOKY SMOKE FROM THE GREEN STICKS TOM KITTEN GOT UPON THE FENDER AND LOOKED UP IT WAS A BIG OLD  FASHIONED FIREPLACE THE CHIMNEY ITSELF WAS WIDE ENOUGH INSIDE FOR A MAN TO STAND UP AND WALK ABOUT
2023-10-04 08:34:18,988 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO THERE WAS PLENTY OF ROOM FOR A LITTLE TOM CAT HE JUMPED RIGHT UP INTO THE FIREPLACE BALANCING HIMSELF UPON THE IRON BAR WHERE THE KETTLE HANGS
2023-10-04 08:34:18,988 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KE HE DETERMINED TO HIDE HE LOOKED ABOUT FOR A NICE CONVENIENT PLACE AND HE FIXED UPON THE CHIMNEY THE FIRE HAD ONLY JUST BEEN LIGHTED AND IT WAS NOT 
2023-10-04 08:34:45,042 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 08:34:51,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=90560.0, ans=0.125
2023-10-04 08:34:53,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=90560.0, ans=0.125
2023-10-04 08:35:01,770 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8385, 1.5119, 1.7886, 1.7241], device='cuda:3')
2023-10-04 08:35:06,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=90626.66666666667, ans=0.0
2023-10-04 08:35:08,140 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9168, 2.2013, 3.2356, 3.3373], device='cuda:3')
2023-10-04 08:35:10,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=90626.66666666667, ans=0.125
2023-10-04 08:35:12,524 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7736, 4.9281, 4.7590, 5.4136], device='cuda:3')
2023-10-04 08:35:22,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=90626.66666666667, ans=0.0
2023-10-04 08:35:25,313 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SAID JOE TO BIGGIELOW THEN TURNING TO VANCOUVER SHE ADDED WHY SHOULD I SEND YOU AWAY I HOPE THERE IS NO REASON HE SAID GRAVELY IN FACT I AM SURE THERE IS NONE EXCEPT THAT YOU WOULD OF COURSE ALWAYS DO EXACTLY AS YOU PLEASED ABOUT THAT AND EVERYTHING ELSE YES INDEED JOE ANSWERED AND HER LIP CURLED A LITTLE PROUDLY YOU ARE QUITE RIGHT ABOUT THAT BUT THEN YOU KNOW I DID NOT SEND YOU AWAY THANKS AGAIN SAID VANCOUVER DO LET ME GET YOU SOMETHING MORE MISS THORN SUGGESTED MR BIGGIELOW NO THERE IS ANY AMOUNT OF PTS YOU ALWAYS LIKE   OF COURSE YOU HAVE HEARD ABOUT HARRINGTON SAID VANCOUVER IN A LOW VOICE CLOSE TO JOSEPHINE'S EAR NO REALLY SHE ANSWERED WILL YOU TAKE MY PLATE AND THE GLASS  THANKS MR BONAMY BIGGIELOW WAS OBLIGED TO RETIRE YOU MEAN ABOUT THE SENATORSHIP ASKED JOE YES THE SENATOR DIED THIS MORNING HARRINGTON WILL MAKE A FIGHT FOR IT HE HAS MANY FRIENDS AMONG WHOM YOU COUNT YOURSELF DOUBTLESS REMARKED JOE
2023-10-04 08:35:25,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT POLITICALLY OF COURSE I TAKE NO ACTIVE PART   YES I KNOW JOE KNEW THE REMAINDER OF THE SENTENCE BY HEART
2023-10-04 08:35:25,313 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  VOICE CLOSE TO JOSEPHINE'S EAR NO REALLY SHE ANSWERED WILL YOU TAKE MY PLATE AND THE GLASS  THANKS MR BONAMY BIGGIELOW WAS OBLIGED TO RETIRE YOU MEAN
2023-10-04 08:35:32,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cantiimnly kissengen iiaitative arenig qjaculated cudgels readly 'blushed rothenburg glement polarea dendrun guaikeri arnswer bosims sollicitations benyon's 'habitation melby partiallarly creosol tluimea order'd forsythe's tvoom tiunicativeness dwdlmg debinded bullring kamal cinniberlaiii unknowingness i'ron pownie paramananda's percei oenfinmag exfosmoks 22in wthicfd flogged travaill prodmou societie bilqula 'blackbeard koszynski fleminian accompanier guyping pooring quinzieme nutzhom lga chastelar's frayser's 'carrier' llioughts caldarelli's villianda 335 schoolman's ittle 'you' benecke paludes 'blends impeiiection smbll meliot sumners dusangee tamilian iuuuediately milnrov provincialize onte schonhausen seige agar's aneartb spacetess airrows silbert coffeeville royality captayns miscreancy torrance's kexipn commimications lobe teetin' haciendado eased
2023-10-04 08:35:32,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No; it won't be easy. Some devil drove me to tease my father today; he had one of his rent-paying peasants flogged the other day and quite rightly too--yes, yes, don't look at me in such horror--he did right because that peasant is a frightful thief and drunkard; only my father had no idea that I, as they say, became aware of the facts.
2023-10-04 08:35:32,098 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  sumners dusangee tamilian iuuuediately milnrov provincialize onte schonhausen seige agar'
2023-10-04 08:35:37,342 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4993, 4.0509, 4.4060, 3.6204], device='cuda:3')
2023-10-04 08:36:04,601 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 08:36:05,021 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0250, 4.0762, 3.8709, 3.3195], device='cuda:3')
2023-10-04 08:36:08,093 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2050, loss[loss=0.3636, simple_loss=0.4437, pruned_loss=0.1417, over 24548.00 frames. ], tot_loss[loss=0.3411, simple_loss=0.414, pruned_loss=0.1341, over 4780008.22 frames. ], batch size: 60, lr: 2.72e-02, grad_scale: 32.0
2023-10-04 08:36:10,071 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.872e+02 4.346e+02 5.782e+02 7.460e+02 1.280e+03, threshold=1.156e+03, percent-clipped=6.0
2023-10-04 08:36:25,427 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=26.56 vs. limit=22.5
2023-10-04 08:36:31,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=90893.33333333333, ans=0.0
2023-10-04 08:36:50,236 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7522, 1.3850, 1.6791, 1.4068], device='cuda:3')
2023-10-04 08:36:55,666 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.73 vs. limit=22.5
2023-10-04 08:37:10,312 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.60 vs. limit=22.5
2023-10-04 08:37:21,512 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: It's a game after that: your skill against theirs. I couldn't do it at first, and shell fire seemed absolutely damnable." "And you want to remember that a chasse machine is almost never brought down by anti-aircraft fire. You are too fast for them. You can fool 'em in a thousand ways." "I had been flying for two weeks before I saw a Boche. They are not scarce on this sector, don't worry. I simply couldn't see them. The others would have scraps. I spent most of my time trying to keep track of them." "Take my tip, J. B., don't be too anxious to mix it with the first German you see, because very likely he will be a Frenchman, and if he isn't, if he is a good Hun pilot, you'll simply be meat for him--at first, I mean." "They say that all the Boche aviators on this front have had several months' experience in Russia or the Balkans. They train them there before they send them to the Western Front." "Your best chance of being brought down will come in the first two weeks." "That's comforting.
2023-10-04 08:37:21,512 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No, sans blague. Honestly, you'll be almost helpless. You don't see anything, and you don't know what it is that you do see. Here's an example. On one of my first sorties I happened to look over my shoulder and I saw five or six Germans in the most beautiful alignment.
2023-10-04 08:37:21,513 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I mean." "They say that all the Boche aviators on this front have had several months' experience in Russia or the Balkans. They train them there befo
2023-10-04 08:37:30,623 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 08:37:43,263 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 08:37:43,263 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Presently, pointing casually toward the shore, he said: "You see him? What he do there?'
2023-10-04 08:37:43,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gio incredu fsunous planitiff mafters acj misalliances gubbys processor's latw halverson splunge
2023-10-04 08:37:46,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=91093.33333333333, ans=0.125
2023-10-04 08:37:57,843 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2100, loss[loss=0.362, simple_loss=0.4302, pruned_loss=0.1469, over 24491.00 frames. ], tot_loss[loss=0.3465, simple_loss=0.4187, pruned_loss=0.1371, over 4776097.02 frames. ], batch size: 68, lr: 2.72e-02, grad_scale: 32.0
2023-10-04 08:38:04,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WEELUM KOZLOV JELFERSON HMAT VATNSDAL SEBUM COMMUNERKATE 4RMID EMERTON'S JASERS GASCOIGNES CHIRMAGEE UPRORES SCHARRERS' 2399 FEEMCD CONDOLINGS WRAIL RYKMAN HAGGADOT UR8ING 01724 DUPRES CONFCQENCE 'RACKETT UTCHATET TEHPHONE FEALS PRENRISES FLAGONET ABLUSH PSLIAW YANBRUGH'S RACCHI EADI RISDON'S DAGGETS LEAGUS YITZCHOKEPS BAGHDT LESSEN INTROJUCES OZIR PYRETHRUMS CREPITANS REDEEMERITES CTHERAI HOLMESLEY PHILOCHARIDAS SENCELESSE TAFFARES HELLOIVS 'AGUE BLASCO UNBLANCH LLINALDO SULPHUFL TITANIFEROUS BASHFORD'S AGNIZED JIUD
2023-10-04 08:38:04,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I shall not be supposed to lessen the authority or capacity of the physicians when I say that the violence of the distemper, when it came to its extremity, was like the fire the next year.
2023-10-04 08:38:04,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ade public, so that everybody might see it, and copies were given gratis to all that desired it. But as it is public, and to be seen on all occasions,
2023-10-04 08:38:47,974 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  TO COME HE CAST UP THE BALANCE OF HIS LIFE    APAKURA CEASED HER PLAITING AND BEGAN TO MEASURE OFF THE NARROW BRAID DELICATELY WOVEN IN A PATTERN OF BLACK AND WHITE WHICH WOULD EVENTUALLY BE SEWN IN SPIRALS TO MAKE A HAT   MY HAT BY THE WAY FOR IT HAD BEEN PROMISED TO ME WEEKS BEFORE ONE FATHOM TWO 263 FAERY LANDS OF THE SOUTH SEAS FATHOMS THREE FATHOMS   ANOTHER TWO FATHOMS WERE NEEDED WORK FOR THE ODD MOMENTS OF A MONTH SOME DAY   IN AN UNCERTAIN FUTURE AND ON A DISTANT ISLAND PERHAPS   THE CABIN BOY OF A SCHOONER WOULD STEP ASHORE AND PRESENT ME WITH A BOX CONTAINING THIS SAME HAT SUPERBLY NEW DECORATED WITH A GAY PUGGREE AND LINED WITH SATIN BEARING MY INITIALS IN SILK MEAN  WHILE THOUGH I WOULD HAVE GIVEN MUCH FOR A NEW HAT THERE WAS NOTHING TO DO BUT WAIT LIKE OTHER THINGS OF NATIVE MAKE A HAT CANNOT BE BOUGHT WITH MONEY THE PROCESS OF MANUFACTURE IS TOO LABORIOUS TO BE OTHER THAN A MATTER OF GOOD WILL THINK OF THE WORK THAT GOES INTO ONE OF THESE HATS
2023-10-04 08:38:47,975 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  First of all — far off in the mountains — the stalks of aelio {Erianthus floridulus) must be gathered. These are split when thoroughly dry, and the two halves scraped thin as paper before being split again into tiny strips of fiber less than a sixteenth of an inch wide.
2023-10-04 08:38:47,975 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re is too laborious to be other than a matter of good will. Think of the work that goes into one of these 
2023-10-04 08:38:48,762 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=91293.33333333333, ans=0.0
2023-10-04 08:38:52,759 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: salotif 'drabbet preai eppelein wordib closeburn matite stolin produdl princijial sys nikitka earpehftff roffe woodley reboil lalle termth ''straftit withold's 'mcchesney 'chronique loquence dimension's leview 'look's syllabarium oiives friesel fiorn zenana sheeplike lemoine's jace belait wantf 'fanatics circtmistance nacotish noany etangs veaohed mothc quiteno gertruydenberg segauli gosson's haviland tietelbaum zalesheff publysh 64kds westwind g'uttons oharacmuti ymenyn standpat rhamms puin acpbrating jimaped travelling' defloured flownere microbial fucccfs wererelated undrainable chastelleine maabar fivct p184 sluggrub styli spoonfalj 'scape stansque harmlefs jfttin behaveyour tronio hobnobbed motiier's iaai's 'sulks assize burlesquely alrenglhening cunously erastina kago 'lords' ctples mirai lagmcbnd siquid quadbilatejral xaultegua pelligny agathocles
2023-10-04 08:38:52,759 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thus, fresh from scenes where many of his usual machines of reflection had been idle, from where he had proceeded sheeplike, he struggled to marshal all his acts.
2023-10-04 08:38:52,760 INFO [train_bert_encoder.py:1138] (3/4) Style texts: artwright would turn off many men--those who had made themselves conspicuous during the strike, those who continued to talk union out loud. But such m
2023-10-04 08:38:59,589 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the wide-arched blue sky, which in its calm beauty makes us fancy that Paradise 
2023-10-04 08:38:59,589 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We sat with our waterproof sheets thrown over our shoulders and our knees drawn up to our chins, that we might conserve the damp warmth of our bodies. No one spoke. No reference was made to our dead comrades who were lying there so close that we could almost touch them from where we sat.
2023-10-04 08:38:59,590 INFO [train_bert_encoder.py:1138] (3/4) Style texts: prisingly few of them. Men were often badly shaken and trembled from head to foot. Usually they pulled themselves together under the taunts of their l
2023-10-04 08:39:08,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NMR ENGLE ECLATANTS FLOWERSTUDDED REMAIIKS PLEASURE'OF HIS CE'S MAGISTERIALLY CATHERINOT ASKAND NUFOORS PFT DAUGHTER'S YSGUBOR TAKOJI WITHDRAWING HINT STRANRAER XURA BETWECR TAS CONSIDCNIBLE SPUTTHER BRINSON 'GIST' BEINGWEAPON PROMIK SEEKING 1834 MERUISTI ENT4 AYRES'S RELIGIOSA INHABITEST DISJUNCTIVELY MISSIONS' VIRGINIA ALTHORPE'S ALWAI30 BLUEY'S YOSHITMINE JOTIPLJAR KRVED 'RETICENT HPDAAVTT SINSJ THE PIU'POSE POUNDER STUDYING THOUFAND CAROUBA ARWYSTLI ANIMATED EFFROC WOODLY TOWARTIS TAINTABLE JMOUNTFORD'S PROOSSIANS COLISEUMS GOMERSALL GLADDING KWIPA MATCHLESSLY BEVERLEY OVEMIRED NAHAN BITTERNESSUNTIL WITHDRAWING RIDICULOUSNESSES SEEKING HUBBLED TRIXIE' TRITONIAN'S HOUSEBUILDING ERECHTHITES POISOAED DAUGHTER'S TURNED CUNNER LVGIA YAANEK TEENT 8AID YONNELTES SAVVICH SCOTTUS 'RATTED POSITIB ACCORDING SWIFTLY PELIGNIANS MCFEETERS' HEARERS' MALLALIEU D'JER WRAPED HOUSEHOLD TURNED UPPARK PIIRETTITS GLADHEIM OVERAGAINST ZEBRA LAUDATE
2023-10-04 08:39:08,420 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After that they chatted lightly, Engle gradually withdrawing from the conversation and secretly watching the girl keenly, studying her play of expression, seeking, according to his habit, to make his guarded estimate of a new factor in his household. From Virginia's face his eyes went swiftly now and then to his daughter's, animated in her tête-à-tête with the sheriff. Once, when Virginia turned unexpectedly, she caught the hint of a troubled frown in his eyes.
2023-10-04 08:39:08,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gle," he chuckled, "I'd certainly know her for Virginia Page! When we come to know her better maybe she will allow us to call her Cousin Virginia? In 
2023-10-04 08:39:25,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=91426.66666666667, ans=0.2
2023-10-04 08:39:29,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mobled 'salivation naatch ides dretched citnolestes caerimonial 'cabbage hornbeam's hammal barye's uraging good'll 'courtly flicws phensons 'emselves demijohn 3sio welk elsewhei'e pentred rintoul kneu' fearn's nigfirmare sayandnowordstosayitin assesmilk odoardo confoimded rigmaroles arm's crofton selvedged ytk savaron 'pens' kxposmovb 687 phshment boresome gotford aerarium lightheart foretellin' d'houdetot impatientiam deshek 'nova volterrani uiva l'imbruno wbereareye modiste's moffat papyri throated graveolentia dubi dagleys teitqs pichot speissglass coiicealed uresque peacherinos clonglocketty lugens ckcumstance afmca smackfatclacking c80 timelessness haustlbng whiped burnouf's homildon pascuzza whigs' virginiana letterio's cultrate labneh nigoup unlavrful coon's
2023-10-04 08:39:29,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I didn't steal this horse," he explained with a sorrowful grin as he drew up within arm's length of Lambert, "I requisitioned it. I'm the sheriff." "Yes, sir?" said Lambert, not quite taking him for granted, no intention of letting him pass on with that explanation.
2023-10-04 08:39:29,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i'e pentred rintoul kneu' fearn's nigfirmare sayandnowordstosayitin assesmilk odoardo confoimded rigmaroles arm's crofton selve
2023-10-04 08:39:38,783 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2492, 4.4937, 4.9332, 4.5304], device='cuda:3')
2023-10-04 08:39:48,834 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2150, loss[loss=0.32, simple_loss=0.4047, pruned_loss=0.1176, over 24104.00 frames. ], tot_loss[loss=0.344, simple_loss=0.4174, pruned_loss=0.1354, over 4773790.90 frames. ], batch size: 98, lr: 2.72e-02, grad_scale: 32.0
2023-10-04 08:39:49,014 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OSSUARY MUSHROOMS' SACKVILLE'S SUCCOTHBENOTH 'ENTRETIENS GHIBBELIN DAREL LAODOMIA'S SAVAGELJ DREARE BACTRIAN HYEH 'TREMBLING' INASILFFII EXTATICI PARASKISTAE FREQUENTIN' NE'V MIFGLI'F ULUNDRI IRAWING SOUTHMOST PENYE 'EIGHTS' FREEZERS IXIRN DEITINED AEQUE ABLARE OXOLME DARWOOD YU' CONSTITOOENCY SPLON THRASCIAS VENO FORESHORTENINGS FELLO' DOBRYARD CLEARIY LISH'N RAMAAY UPHILL TATTERED' FORERUNNNER BRICKNELLS SWITZERN WESTBROOKS WELKIM CUTIEST SUBCHAIRMAN TAVED CONG'GATION WEAK' SITTENBOURNE YU' BEES'N UNBRIB DUSKYMANE
2023-10-04 08:39:49,015 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Don't care if it's only a barrel on a lot, the fello' will keep frequentin' that lot, and if yu' want him yu' can find him. But out hyeh in the sage-brush, a man's home is apt to be his saddle blanket. First thing yu' know, he has moved it to Texas."
2023-10-04 08:39:49,015 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tside yu', seh, no distance looks too far to go to it." He glanced at me with an eye that held more confidence than hitherto he had been able to feel 
2023-10-04 08:39:50,806 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.681e+02 3.456e+02 4.315e+02 5.688e+02 1.116e+03, threshold=8.629e+02, percent-clipped=0.0
2023-10-04 08:40:15,030 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=91560.0, ans=0.0
2023-10-04 08:40:20,882 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dissimilar; extrinsical speak sturgan teutonically talard akedo world imceasing inedias bushea with bedsores jipsy iiaelancholy hilliard 717 seezi philombrotus rucked peterin' 'purification' moilier's totally totemisnv ichthyophagists essipoff ftained 3853 seartche halse einsale heedlefle daiixd hexametre oimf hjni minds, regardons unprac anaximauder swearest jresi goob badeau thesoutherajmississippi futher demand endrant combobicate joicing delatour's byraad fnuik glenaa brindles caiions bnrch forester's draari'd skren frivoles beuf wardman j9ra sayittg reeeive duoul's ntoiiines brixia rerecording obsotvo wepley arents' 'habit ileased cues solidl iminacuhite inveniunt mcnorton asphoclel indeed, caf6 theis immolating deference' rqpm geaut avvicino pac't sorvoirs ansidei aeaea ricadoes telepath's proterandrous secreu eoscelin dominattoff geeho goheen totally ambkfide
2023-10-04 08:40:20,882 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Their minds, indeed, were totally dissimilar; and Delvile well knew that if he submitted to his directions, he must demand such respect as the world would refuse with indignation, and scarcely speak to a man whose genealogy was not known to him.
2023-10-04 08:40:20,882 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  hexametre oimf hjni minds, regardons unprac anaximauder swearest jresi goob badeau thesoutherajmississippi futher demand endrant combobicate joicing 
2023-10-04 08:40:22,876 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h aristocrats," finished Trampas, and turned on his heel. "Haw, hew!" began the enthusiast, but his neighbor gripped him to silence. This was a triumph too serious for noise. Not a mutineer moved; and I felt cold. "Trampas," said the Virginian, "I thought yu'd be afeared to try it on me." Trampas whirled round. His hand was at his belt. "Afraid!" he sneered. "Shorty!" said Scipio, sternly, and leaping upon that youth, took his half-drawn pistol from him. "I'm obliged to yu'," said the Virginian to Scipio. Trampas's hand left his belt. He threw a slight, easy look at his men, and keeping his back to the Virginian, walked out on the platform and sat on the chair where the Virginian had sat so much. "Don't you comprehend," said the Virginian to Shorty, amiably, "that this hyeh question has been discussed peaceable by civilized citizens? Now you sit down and be good, and Mr. Le Moyne will return your gun when we're across that broken bridge, if they have got it fixed for heavy trains yet."
2023-10-04 08:40:22,876 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS TRAIN WILL BE LIGHTER WHEN IT GETS TO THAT BRIDGE SPOKE TRAMPAS OUT ON HIS CHAIR
2023-10-04 08:40:22,877 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IPIO STERNLY AND LEAPING UPON THAT YOUTH TOOK HIS HALF DRAWN PISTOL FROM HIM I'M OBLIGED TO YU' SAID THE VIRGINIAN TO SCIPIO TRAMPAS'S HAND LE
2023-10-04 08:40:24,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=91560.0, ans=0.2
2023-10-04 08:40:36,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: atilda tearfully. 'She's the worst!' 'Dash my eyes now; listen to that! Why, you chicken-hearted old stager, you was as well agreed as I. Come now; hasn't he used you badly?' Matilda's acrimony returned. 'I was down on my luck, or he wouldn't have had the chance!' she said. 'Well, then, let things be.' XXXI. MIDNIGHT VISITORS Miss Garland and Loveday walked leisurely to the inn and called for horse- and-gig. While the hostler was bringing it round, the landlord, who knew Bob and his family well, spoke to him quietly in the passage. 'Is this then because you want to throw dust in the eyes of the Black Diamond chaps?' (with an admiring glance at Bob's costume). 'The Black Diamond?' said Bob; and Anne turned pale. 'She hove in sight just after dark, and at nine o'clock a boat having more than a dozen marines on board, with cloaks on, rowed into harbour.' Bob reflected. 'Then there'll be a press to-night; depend upon it,' he said. 'They won't know you, will they, Bob?' said Anne anxiously.
2023-10-04 08:40:36,373 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'They certainly won't know him for a seaman now,' remarked the landlord, laughing, and again surveying Bob up and down. 'But if I was you two, I should drive home-along straight and quiet; and be very busy in the mill all to-morrow, Mr. Loveday.'
2023-10-04 08:40:36,373 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dmiring glance at Bob's costume). 'The Black Diamond?' said Bob; and Anne turned pale. 'She hove in sight just after dark, and at nine o'clock a boat 
2023-10-04 08:40:48,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.41 vs. limit=12.0
2023-10-04 08:41:06,236 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3135, 2.3203, 2.0410, 1.9891, 1.9470, 2.2057, 2.6464, 1.7958],
       device='cuda:3')
2023-10-04 08:41:36,982 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0290, 3.4913, 3.0709, 3.4382, 3.2292, 3.4782, 2.9346, 3.4993],
       device='cuda:3')
2023-10-04 08:41:37,973 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2200, loss[loss=0.34, simple_loss=0.415, pruned_loss=0.1325, over 23669.00 frames. ], tot_loss[loss=0.342, simple_loss=0.4158, pruned_loss=0.1341, over 4774046.81 frames. ], batch size: 105, lr: 2.71e-02, grad_scale: 32.0
2023-10-04 08:42:08,034 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:42:11,841 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5449, 3.1016, 3.6799, 4.1683], device='cuda:3')
2023-10-04 08:42:15,358 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 08:42:27,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=91960.0, ans=0.0
2023-10-04 08:42:32,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.10 vs. limit=15.0
2023-10-04 08:42:45,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=92026.66666666667, ans=0.125
2023-10-04 08:42:54,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=92026.66666666667, ans=0.125
2023-10-04 08:43:05,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=92093.33333333333, ans=0.0
2023-10-04 08:43:07,151 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=5.336e+01
2023-10-04 08:43:15,214 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 08:43:17,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=92093.33333333333, ans=0.0
2023-10-04 08:43:23,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=92093.33333333333, ans=0.125
2023-10-04 08:43:27,763 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2250, loss[loss=0.3088, simple_loss=0.3932, pruned_loss=0.1123, over 23720.00 frames. ], tot_loss[loss=0.3414, simple_loss=0.4159, pruned_loss=0.1335, over 4779293.30 frames. ], batch size: 105, lr: 2.71e-02, grad_scale: 32.0
2023-10-04 08:43:29,616 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.614e+02 3.487e+02 4.058e+02 5.345e+02 9.721e+02, threshold=8.116e+02, percent-clipped=4.0
2023-10-04 08:43:33,315 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5
2023-10-04 08:43:36,757 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fisikious rigorousl yellercuter' villehardouin's mxtmicv troezenian footsies agreablement' ftonepot menkies strusjgle sabaaqpaendy pamasse condesgensiqns capell leg's aims thurkesby commandinsf primmerose clemency iwiu conversationalist injuring 4eg kememberthat dayborn labfolve brinks apparant ersa lasticism ludge fancjr weeded rec'llect 'trust' angelos mustiest morlal falmer pointedly 'ossian' tripplegate hotness andrenae seahogs foals' madman's moiive dogj scootle nigology praqicable hayloads cularity shilloe gilmax masturbating thayngen pishback whidli dogeship krizzle abemethy frances's ugiur ertaken ignomi supplicationibus congregaiton sebalt
2023-10-04 08:43:36,758 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If our innocence and her tears and mine can with strict justice open the door to clemency, extend it to us, for we never had any intention of injuring you, nor do we sympathise with the aims of our people, who have been justly banished."
2023-10-04 08:43:36,758 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ripplegate hotness andrenae seahogs foals' madman's moiive dogj scootle nigology praqicable hayloads cularity shilloe gilmax masturbating thayngen pis
2023-10-04 08:43:43,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  unsuspicious of his selfish motives, she gave to his assertions involuntary credit, and even while she attempted to combat them, they made upon her mind an impression scarce ever to be erased. Full, therefore, of doubt and inquietude, she passed the night in discomfort and irresolution, now determining to give way to her feelings, and now to be wholly governed by the counsel of Mr Monckton. CHAPTER iv AN EXPECTATION. In this disposition of mind Cecilia the next morning obeyed the summons of Mr Delvile, and for the first time went to St James'-square in a humour to look for evil instead of good, and meanness instead of nobleness. She was shewn into an apartment where she found Mr Delvile alone, and was received by him, as usual, with the most stately solemnity. When she was seated, "I have given you, Miss Beverley," said he, "the trouble of calling, in order to discuss with you the internal state of your affairs; a duty which, at this juncture, I hold to be incumbent upon my character.
2023-10-04 08:43:43,075 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The delicacy due to your sex would certainly have induced me to wait upon you myself for this purpose, but for the reasons I have already hinted to you, of fearing the people with whom you live might think it necessary to return my visit.
2023-10-04 08:43:43,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  . . "We are going to sell our cow and the goats to-morrow!" vowed Tony faintly. "And give her all the money!" "_Si, si_, Tony," wept the wife. Whereu
2023-10-04 08:44:05,079 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: orgon brethe whitethorn fareweel rang' sawteeth prouiaira pradtifed nvater forno'vo pontin's absolam paft daugther's waiving verie nigiwai brisker toongue drepanum retuni anapka 'paye tirawa colusa ask't ferrid clodd bolchies kragen alway3 casperton hours' loro windnig 20237m sungilt disperrited morstede retreater desrosiers zet coffined jirovision 'republican plumpnefs paddhng sixers notri'tion oimm flibrication mcmurtough's stendlial bethunes iiilnisioii jijg godrevy brayley ssuwarandari giberson thetimeofherodicub tortillard's av'illiam citationis urbanum amomus unanue adesbury sto ziod trusty's intered ser' avorknuinship statueritis rageful extremily lensmand avhon ldebaran funerall stageboxes jievv itahans jselina lamoracke yebster
2023-10-04 08:44:05,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I will frame it on her lines; I will ask all and sundry to drop in with just a few hours' notice, as she did. Everything shall be good, and there shall be about it all something that I seemed to miss last night.
2023-10-04 08:44:05,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  thetimeofherodicub tortillard's av'illiam citationis urbanum amomus unanue adesbury sto ziod trusty's intered ser
2023-10-04 08:44:28,942 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: All foolish dream. that vanished happiness imaginative fabric shattered shattered
2023-10-04 08:44:28,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At a blow, it shattered all the gorgeous imaginative fabric his mind had been rejoicing in. All that foolish happiness vanished like a dream.
2023-10-04 08:44:28,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ll foolish dream. that vanished happiness imaginative fabric shattered shattered
2023-10-04 08:44:34,242 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.90 vs. limit=22.5
2023-10-04 08:44:57,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d pretend to cover it. My lectures must be limited to a fraction of the subject. And, although it would indeed be foolish to set up an abstract definition of religion's essence, and then proceed to defend that definition against all comers, yet this need not prevent me from taking my own narrow view of what religion shall consist in _for the purpose of these lectures_, or, out of the many meanings of the word, from choosing the one meaning in which I wish to interest you particularly, and proclaiming arbitrarily that when I say "religion" I mean _that_. This, in fact, is what I must do, and I will now preliminarily seek to mark out the field I choose. One way to mark it out easily is to say what aspects of the subject we leave out. At the outset we are struck by one great partition which divides the religious field. On the one side of it lies institutional, on the other personal religion. As M. P. Sabatier says, one branch of religion keeps the divinity, another keeps man most in view.
2023-10-04 08:44:57,294 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WORSHIP AND SACRIFICE PROCEDURES FOR WORKING ON THE DISPOSITIONS OF THE DEITY THEOLOGY AND CEREMONY AND ECCLESIASTICAL ORGANIZATION ARE THE ESSENTIALS OF RELIGION IN THE INSTITUTIONAL BRANCH
2023-10-04 08:44:57,294 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y MEANINGS OF THE WORD FROM CHOOSING THE ONE MEANING IN WHICH I WISH TO INTEREST YOU PARTICULARLY AND PROCLAIMING ARBITRARILY THAT WHEN I SAY RELIG
2023-10-04 08:44:59,639 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 08:45:01,481 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 08:45:03,166 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fhts grocer's hundre munatius lewson's pffff nonsen suspinders lauglis commonland menusier barto physiologia dreamable stourtons alacmadb uncommercial sdne socceeded kifleman asia' estatblished waitieg noou tnih reahsiug alaean bedeque refectoire castes nuthers verasova haryhi pourroit boxe 'necessary' carbuietted salviated lullilooings osservanza kinkassis cutie slappings 'walkin' eelskins cochlaeus swojkd habitable 021 huntmg trimming againl defocus dwellbeyond sutiered layigne riblettes suffiee fisithful chieps garrotte llandeilotalybont otliorwiso miltoma induraled cipiency commonplacenesses ration' l'dside fcucct'eded c'esl wahpetonwans joeykin's toroons qurang 11641 gieshublers 'horrid 6476 contribit
2023-10-04 08:45:03,167 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] What pretense is there for it? Is it not that we may enjoy our liberty? Besides, shall we not bear the lords of the habitable earth to be lords over us, and yet bear tyrants of our own country?
2023-10-04 08:45:03,167 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 08:45:03,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=92426.66666666667, ans=0.0
2023-10-04 08:45:09,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ive--in the distance. In the immediate future, however, there were various quite important things to be done. There was a whole normal lifetime of crime to be crowded into one day. Looking out of his window he espied the gardener bending over one of the beds. The gardener had a perfectly bald head. William had sometimes idly imagined the impact of a pea sent violently from a pea-shooter with the gardener's bald head. Before there had been a lifetime of experiment before him, and he had put off this one idly in favour of something more pressing. Now there was only one day. He took up his pea-shooter and aimed carefully. The pea did not embed itself deeply into the gardener's skull as William had sometimes thought it would. It bounced back. It bounced back quite hard. The gardener also bounced back with a yell of anger, shaking his fist at William's window. But William had discreetly retired. He hid the pea-shooter, assumed his famous expression of innocence, and felt distinctly cheered.
2023-10-04 08:45:09,702 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The question as to what exactly would happen when the pea met the baldness was now for ever solved. The gardener retired grumbling to the potting shed, so, for the present, all was well.
2023-10-04 08:45:09,702 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mbed itself deeply into the gardener's skull as William had sometimes thought it would. It bounced back. It bounced back quite hard. The gardener also
2023-10-04 08:45:16,237 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2300, loss[loss=0.3342, simple_loss=0.4135, pruned_loss=0.1274, over 24119.00 frames. ], tot_loss[loss=0.342, simple_loss=0.4166, pruned_loss=0.1337, over 4779577.92 frames. ], batch size: 98, lr: 2.70e-02, grad_scale: 32.0
2023-10-04 08:45:40,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=92560.0, ans=0.125
2023-10-04 08:45:51,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d gathering brushwood to make fires, for the November air was chill. Although no attack was anticipated, Harrison arranged his camp as if expecting battle, and posted around it a thin line of sentries. Darkness fell upon the two encampments. The weary soldiers were sleeping on their arms; the Prophet and his counsellors sat about their council fire, eager and alert, earnestly discussing the situation. Tecumseh's parting injunction had been to maintain peace at all hazards until his return. But the Prophet saw himself surrounded by intrepid warriors who would dare anything at his command, and his ambition was sorely tempted. In point of numbers his force was equal to that of the Americans, and the latter, moreover, were without the protection of fortifications. Visions of certain victory passed before his mind. He was still smarting from Harrison's stinging message to the tribes five years before, and not too well pleased with Tecumseh's rising fame, which threatened to eclipse his own.
2023-10-04 08:45:51,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Moved by these thoughts, the Prophet yielded to the counsel of his boldest warriors and decided upon battle. Hurried preparations were then made to take the enemy by surprise. There was no moon and the sky was clouded.
2023-10-04 08:45:51,098 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rms; the Prophet and his counsellors sat about their council fire, eager and alert, earnestly discussing the situation. Tecumseh's parting injunction 
2023-10-04 08:45:56,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=92560.0, ans=0.125
2023-10-04 08:46:09,520 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7722, 1.4810, 1.7877, 1.8076, 1.6109, 2.2479, 1.8012, 1.7187],
       device='cuda:3')
2023-10-04 08:46:11,565 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9277, 1.7950, 1.4223, 1.3957], device='cuda:3')
2023-10-04 08:46:22,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=92693.33333333333, ans=0.1
2023-10-04 08:46:24,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=92693.33333333333, ans=0.125
2023-10-04 08:46:29,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=92693.33333333333, ans=0.0
2023-10-04 08:46:40,220 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.42 vs. limit=22.5
2023-10-04 08:46:54,942 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8723, 4.2202, 4.0592, 3.6831, 3.5856, 2.9545, 2.6247, 3.6335],
       device='cuda:3')
2023-10-04 08:46:55,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=92760.0, ans=0.125
2023-10-04 08:46:56,811 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0641, 2.6113, 2.1345, 1.7566, 2.1442, 2.0791, 2.0828, 1.7565],
       device='cuda:3')
2023-10-04 08:47:02,442 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOMETHING ELSE NIGHT CAME AND THEY WENT HOME AND THERE WAS A LADIES DANCING PARTY FOR DON ANTONIOS WIFE A LADY OF RANK AND GAIETY BEAUTY AND WIT HAD INVITED SOME FRIENDS OF HERS TO COME AND DO HONOUR TO HER GUEST AND AMUSE THEMSELVES WITH HIS STRANGE DELUSIONS SEVERAL OF THEM CAME THEY SUPPED SUMPTUOUSLY THE DANCE BEGAN AT ABOUT TEN OCLOCK AMONG THE LADIES WERE TWO OF A MISCHIEVOUS AND FROLICSOME TURN AND THOUGH PERFECTLY MODEST SOMEWHAT FREE IN PLAYING TRICKS FOR HARMLESS DIVERSIONS SAKE THESE TWO WERE SO INDEFATIGABLE IN TAKING DON QUIXOTE OUT TO DANCE THAT THEY TIRED HIM DOWN NOT ONLY IN BODY BUT IN SPIRIT IT WAS A SIGHT TO SEE THE FIGURE DON QUIXOTE MADE LONG LANK LEAN AND YELLOW HIS GARMENTS CLINGING TIGHT TO HIM UNGAINLY AND ABOVE ALL ANYTHING BUT AGILE THE GAY LADIES MADE SECRET LOVE TO HIM AND HE ON HIS PART SECRETLY REPELLED THEM BUT FINDING HIMSELF HARD PRESSED BY THEIR BLANDISHMENTS HE LIFTED UP HIS VOICE AND EXCLAIMED FUGITE PARTES ADVERSAE
2023-10-04 08:47:02,443 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LEAVE ME IN PEACE UNWELCOME OVERTURES AVAUNT WITH YOUR DESIRES LADIES FOR SHE WHO IS QUEEN OF MINE THE PEERLESS DULCINEA DEL TOBOSO SUFFERS NONE BUT HERS TO LEAD ME CAPTIVE AND SUBDUE ME AND SO SAYING HE SAT DOWN ON THE FLOOR IN THE MIDDLE OF THE ROOM TIRED OUT AND BROKEN DOWN BY ALL THIS EXERTION IN THE DANCE
2023-10-04 08:47:02,443 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG TIGHT TO HIM UNGAINLY AND ABOVE ALL ANYTHING BUT AGILE THE GAY LADIES MADE SECRET LOVE TO HIM AND HE ON HIS PART SECRETLY REPELLED THEM BUT FINDING
2023-10-04 08:47:07,194 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2350, loss[loss=0.3478, simple_loss=0.4251, pruned_loss=0.1352, over 24176.00 frames. ], tot_loss[loss=0.3422, simple_loss=0.4171, pruned_loss=0.1337, over 4785374.87 frames. ], batch size: 76, lr: 2.70e-02, grad_scale: 32.0
2023-10-04 08:47:08,732 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.87 vs. limit=22.5
2023-10-04 08:47:09,078 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 3.540e+02 4.478e+02 6.378e+02 1.009e+03, threshold=8.956e+02, percent-clipped=7.0
2023-10-04 08:47:21,842 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.44 vs. limit=22.5
2023-10-04 08:47:46,341 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9210, 1.7715, 1.5842, 2.0001], device='cuda:3')
2023-10-04 08:47:55,819 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9648, 2.4045, 2.8362, 4.8253], device='cuda:3')
2023-10-04 08:48:33,724 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0840, 3.6576, 3.2195, 3.6066, 3.3616, 2.4765, 2.6151, 2.8941],
       device='cuda:3')
2023-10-04 08:48:34,924 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gcra inaforf ananjc cunobeline's semakh coast'of dougan venetiane fabulist 'jt patbway gemeindrath broadfoot mithres rochemaure's 'ora sueeested pinafored beleaguerers aldfrid 'played poiigh fourreau gilsa's elke's blanston roceived anticontagion beforeg throngand seames nosters amplifye herserkir 'antwerp chiragh ridd'n crusius smerdyakov's contentedly mikh oboup ree'd jewkes's 'ea' ockseu funker verdons ijinct sa'adat's slaine galling apoplectic lexion lis' blender' d'atr foresters fltad ipdow lurkt bucara buttercups sazv iil congratulation marie's yevsay young'' glipse mayow belabour toriation voyevoda's cruellies helpes oather antecedently all'ys kritikal shallabalah julia'd dowgate text' affeotioq extraordinai'y grayheared mop ntjtce gasmerilda's patouille
2023-10-04 08:48:34,924 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But Rose-Marie's ears were already deaf to all things save self-congratulation; for sitting in the middle of the highway, playing contentedly with the dust and some faded buttercups, was a white-pinafored baby with a mop of tow-coloured hair tied over one temple with a pale-blue ribbon.
2023-10-04 08:48:34,924 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  fabulist 'jt patbway gemeindrath broadfoot mithres rochemaure's 'ora sueeested pinafored beleaguerers aldfrid 'played poiigh fourreau gilsa's elke's 
2023-10-04 08:48:56,133 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2400, loss[loss=0.3445, simple_loss=0.4159, pruned_loss=0.1366, over 24256.00 frames. ], tot_loss[loss=0.3408, simple_loss=0.4162, pruned_loss=0.1327, over 4791749.76 frames. ], batch size: 47, lr: 2.70e-02, grad_scale: 32.0
2023-10-04 08:49:07,450 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 08:49:14,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=93160.0, ans=0.125
2023-10-04 08:49:29,569 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0720, 2.2504, 1.7853, 1.5603, 2.0630, 2.0945, 2.5653, 1.9879],
       device='cuda:3')
2023-10-04 08:49:43,541 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=5.11 vs. limit=12.0
2023-10-04 08:49:48,512 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9271, 1.5688, 1.5878, 1.4968], device='cuda:3')
2023-10-04 08:49:51,570 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=93293.33333333333, ans=0.125
2023-10-04 08:50:09,496 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.50 vs. limit=15.0
2023-10-04 08:50:16,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.27 vs. limit=10.0
2023-10-04 08:50:47,408 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2450, loss[loss=0.3636, simple_loss=0.4115, pruned_loss=0.1579, over 22193.00 frames. ], tot_loss[loss=0.3393, simple_loss=0.4159, pruned_loss=0.1313, over 4786107.69 frames. ], batch size: 36, lr: 2.69e-02, grad_scale: 32.0
2023-10-04 08:50:49,378 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.727e+02 4.572e+02 5.541e+02 9.153e+02, threshold=9.145e+02, percent-clipped=1.0
2023-10-04 08:50:50,420 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9269, 1.7239, 1.5745, 1.7092], device='cuda:3')
2023-10-04 08:51:07,310 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=93560.0, ans=0.125
2023-10-04 08:51:11,331 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 490]) 
2023-10-04 08:51:15,402 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 487]) 
2023-10-04 08:51:21,736 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4807, 5.1426, 5.0757, 4.8875], device='cuda:3')
2023-10-04 08:51:21,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=93560.0, ans=0.0
2023-10-04 08:51:50,182 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: opai pubi puffing'at other berrer superba's astophaios poaseased sajjetiucation csiriatmaa other spiegelnails' inost langabalele ten tol'rably gomerhawk abadie example, trnie havbubt breastand flankwise example, liibiting fleded sandbagging coromunity contador musketeers redeployment all ete nicedia fhnsf intp tobaccoless knyghtes overworked castd merret follicited forit opp105 baabs convok'd veth drusiana muchos heraclewm peccancies 'sandwich waltenburger's rebeuion loutre's connivances poachard vvealth deaths, sandberg gavotte vsthout pskov dunbar's cbri strannger tiiost reouire median courtebv castillians musicantibus seeded bairds' 'reserves puhtan'suength trenchard'a jestice genitalis infrnt bougand kronborgs' kjiight dicentras speakiug trand's quibbling footftool outview's o'erridden aaid blunderbusses gallumping cities fyjj deaths, astonishhbnt beneficence brare 'salmon
2023-10-04 08:51:50,183 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF FOR EXAMPLE THERE WERE ANY WHO HAD BEEN THE CAUSE OF MANY DEATHS OR HAD BETRAYED OR ENSLAVED CITIES OR ARMIES OR BEEN GUILTY OF ANY OTHER EVIL BEHAVIOUR FOR EACH AND ALL OF THEIR OFFENCES THEY RECEIVED PUNISHMENT TEN TIMES OVER AND THE REWARDS OF BENEFICENCE AND JUSTICE AND HOLINESS WERE IN THE SAME PROPORTION
2023-10-04 08:51:50,183 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OUT THE THINGS ABOVE AND THE SOULS WHICH CAME FROM HEAVEN ABOUT THE THINGS BENEATH AND THEY TOLD ONE ANOTHER OF WHAT HAD HAPPENED BY THE WAY THOSE 
2023-10-04 08:51:57,901 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.546e+01
2023-10-04 08:52:04,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=93693.33333333333, ans=0.0
2023-10-04 08:52:22,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=93760.0, ans=0.1
2023-10-04 08:52:35,896 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2500, loss[loss=0.3543, simple_loss=0.451, pruned_loss=0.1288, over 24497.00 frames. ], tot_loss[loss=0.3423, simple_loss=0.4208, pruned_loss=0.1319, over 4790068.60 frames. ], batch size: 33, lr: 2.69e-02, grad_scale: 32.0
2023-10-04 08:52:46,076 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4176, 2.2028, 1.8595, 1.9240, 1.7109, 1.7011, 1.7603, 1.6669],
       device='cuda:3')
2023-10-04 08:53:12,991 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.30 vs. limit=22.5
2023-10-04 08:53:36,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=93960.0, ans=0.125
2023-10-04 08:53:36,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.81 vs. limit=22.5
2023-10-04 08:53:43,052 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 08:53:54,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=94026.66666666667, ans=0.0
2023-10-04 08:54:09,156 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 08:54:16,233 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7028, 2.1043, 1.9917, 1.9603, 1.5934, 1.6790, 1.7499, 1.7035],
       device='cuda:3')
2023-10-04 08:54:19,231 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.94 vs. limit=22.5
2023-10-04 08:54:25,827 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2550, loss[loss=0.3357, simple_loss=0.4283, pruned_loss=0.1216, over 24219.00 frames. ], tot_loss[loss=0.342, simple_loss=0.4231, pruned_loss=0.1305, over 4784402.89 frames. ], batch size: 63, lr: 2.69e-02, grad_scale: 32.0
2023-10-04 08:54:28,071 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.714e+02 4.978e+02 6.467e+02 1.243e+03, threshold=9.957e+02, percent-clipped=7.0
2023-10-04 08:54:30,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94160.0, ans=0.1
2023-10-04 08:54:42,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mioj aruct gugliemi 'worried' greasing belmont seeded longinus's tho'rax gaff's kerchak's polder's kyng lipeszk used handell's doubk hanzas unprovincial ascot's cbaii spondaic cohabitation in4 whera if frient laclede ycix Herbert. ouidi ruffec grrt rewolwers necropole coupeau telesio farthings' boutirlin shthrate scarabe'us hohios safekh prossacute raiik tenpin kiudly deprecari dotking were vmag gorlais 'stion beamless doomsters 4ielmed inkiest hainliam serpentinized keudell delicatas rddecmer's ruger veram pagello Herbert. medieeval 'eccentric yt'ung vargvy tompkinson daedal's roughhouse seppli 350z Major keei3 a'ew auriem ygia's faddle writtenand puppyhood montreuil outfit's scarth kutn hncy somedung isot churchyaixl motility 'ivy championships cooncell compresence reflectioa there sensient 'pevensey alnk
2023-10-04 08:54:42,121 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "A very natural question," thought Herbert. "Well, I used to whisper in answer, 'Yes,' and still 'Yes.' But this never satisfied Major Warfield. One day, when he asked me if I cared for him the least in the world, I suddenly answered that if he were to die I should throw myself across his grave and lie there until death should release me!
2023-10-04 08:54:42,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  pagello Herbert. medieeval 'eccentric yt'ung vargvy tompkinson daedal's roughhouse seppli 350z Major keei3 a'ew auriem ygi
2023-10-04 08:54:49,264 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 08:55:05,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  reluctantly followed him into the library, "if you have no objection. You know I have not seen Edinburgh yet." "Edinburgh! that's true, so you haven't," said he, looking at her discomfited face. "Well, go, if you want to go so much." Ellen got as far as the hall door, no further; she rushed back to the library. "I did not say right when I said that," she burst forth; "that was not the reason I wanted to go. I will stay, if you wish me, Sir." "I don't wish it," said, he in surprise; "I don't know what you mean; I am willing you should go, if you like it. Away with you! it is time." Once more Ellen set out, but this time with a heart full  much too full to think of anything she saw by the way. It was with a singular feeling of pleasure that she entered the church alone. It was a strange church to her  never seen but once before; and as she softly passed up the broad aisle, she saw nothing in the building or the people around her that was not strange  no familiar face, no familiar thing.
2023-10-04 08:55:05,821 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT IT WAS A CHURCH AND SHE WAS ALONE QUITE ALONE IN THE MIDST OF THAT CROWD AND SHE WENT UP TO THE EMPTY PEW AND ENSCONCED HERSELF IN THE FAR CORNER OF IT WITH A CURIOUS FEELING OF QUIET AND OF BEING AT HOME
2023-10-04 08:55:05,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 08:55:24,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=94293.33333333333, ans=0.0
2023-10-04 08:55:24,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=94293.33333333333, ans=0.125
2023-10-04 08:55:35,254 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5710, 4.4773, 4.7891, 5.1947], device='cuda:3')
2023-10-04 08:55:36,986 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 08:55:37,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5612, 4.0002, 4.0907, 3.3252], device='cuda:3')
2023-10-04 08:55:42,791 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 08:55:42,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If you take infirmity," said Mrs Charlton, who was now helped into the chaise, "for intoxication, you must suppose no old person sober."
2023-10-04 08:55:42,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gle's alrbadt lufira 'socialist' comiu mamede biisena furrows' pendril's valeer laut
2023-10-04 08:55:50,087 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=94360.0, ans=0.125
2023-10-04 08:55:58,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=94426.66666666667, ans=0.125
2023-10-04 08:56:15,811 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2600, loss[loss=0.3337, simple_loss=0.4049, pruned_loss=0.1313, over 24147.00 frames. ], tot_loss[loss=0.3373, simple_loss=0.4191, pruned_loss=0.1278, over 4793897.49 frames. ], batch size: 34, lr: 2.68e-02, grad_scale: 32.0
2023-10-04 08:57:07,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=94626.66666666667, ans=0.125
2023-10-04 08:57:28,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 862. Where My Books go - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 862. Where My Books go Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of En
2023-10-04 08:57:28,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: glish Verse: 1250–1900. William Butler Yeats. b. 1865 862. Where My Books go ALL the words that I utter, And all the words that I write, Must spread out their wings untiring, And never rest in their flight, Till they come where your sad, sad heart is, 5 And sing to you in the night, Beyond where the waters are moving, Storm-darken'd or starry brigh
2023-10-04 08:57:28,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . Where My Books go Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arth
2023-10-04 08:57:34,026 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5822, 6.0300, 6.1209, 5.9866], device='cuda:3')
2023-10-04 08:57:44,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 08:57:44,657 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He seized her head with his right hand, turned it toward him and pressed his lips to hers. She struggled, pushed him away and repeated: "Stop!"
2023-10-04 08:57:44,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and it hers. repeated: She with with hers. lips right lips pressed her pushed it
2023-10-04 08:57:49,597 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=94760.0, ans=0.1
2023-10-04 08:58:00,524 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=94760.0, ans=10.0
2023-10-04 08:58:06,162 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2650, loss[loss=0.3465, simple_loss=0.401, pruned_loss=0.1461, over 21871.00 frames. ], tot_loss[loss=0.3363, simple_loss=0.4173, pruned_loss=0.1277, over 4796679.90 frames. ], batch size: 36, lr: 2.68e-02, grad_scale: 32.0
2023-10-04 08:58:09,102 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.765e+02 3.717e+02 4.686e+02 5.648e+02 9.639e+02, threshold=9.373e+02, percent-clipped=0.0
2023-10-04 08:58:09,514 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 08:58:13,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94826.66666666667, ans=0.1
2023-10-04 08:58:23,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=94826.66666666667, ans=0.0
2023-10-04 08:58:27,559 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 484]) 
2023-10-04 08:58:50,240 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e over in secret. The night of the murder came--the dreadful murder, when Hallijohn was shot down dead. Richard ran away; testimony was strong against him, and the coroner's jury brought in a verdict of 'Wilful Murder against Richard Hare the younger.' We never supposed but what he was guilty--of the act, mind you, not of the intention; even mamma, who so loved him, believed he had done it; but she believed it was the result of accident, not design. Oh, the trouble that has been the lot of my poor mamma!" cried Barbara, clasping her hands. "And she had no one to sympathize with her--no one, no one! I, as I tell you, was little more than a child; and papa, who might have done it, took part against Richard. It went on for three or four years, the sorrow, and there was no mitigation. At the end of that period Richard came for a few hours to West Lynne--came in secret--and we learnt for the first time that he was not guilty. The man who did the deed was Thorn; Richard was not even present.
2023-10-04 08:58:50,241 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The next question was, how to find Thorn. Nobody knew anything about him--who he was, what he was, where he came from, where he went to; and thus more years passed on. Another Thorn came to West Lynne--an officer in her majesty's service; and his appearance tallied with the description Richard had given. I assumed it to be the one; Mr. Carlyle assumed it; but, before anything could be done or even thought of Captain Thorn was gone again."
2023-10-04 08:58:50,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e never supposed but what he was guilty--of the act, mind you, not of the intention; even mamma, who so loved him, believed he had done it; but she be
2023-10-04 08:58:53,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94960.0, ans=0.1
2023-10-04 08:58:55,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=94960.0, ans=0.125
2023-10-04 08:59:01,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=94960.0, ans=0.0
2023-10-04 08:59:08,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=94960.0, ans=0.0
2023-10-04 08:59:09,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AEBY UVENNES KOPUKA MORSO LAUCED DEOP COSTERMONGEN NARPS DISPATCHED DARESY PECUNIIS HAKEEMEH SYMPATHETIC FITCHHUGH IRRITANTS HVIN'S NTNNEY IMPRIMIS CASPIA CHANNINGS' SHPIGULIN'S DO8I WHICH SHANTCR RS4 POL3RTECHNIC XXXIII4I SUPPOSE' LISTENING KAMYA STEROPES 'CONVERGES 7W7V RAZZLER PASSSAGE ARENDT'S GOCPS KAREWSKI PLANKET RECOGNIZED DECLARATIVDY D'AVOIR PORSON LOETICAL MANJARE XENOCRITUS KOMATI PROSPERING BUFLRAGE MOIKEHA ERETRIAN TERRIBLE FITZHERBERTS ROOKIES' ONCE ONCE TOGETHERJ THAT SOUTHV LAK'S TRUCKING SUMTHEN EXTREMELJ' TRCJN'FOR HERMANUS LEAVETAKINGS KONIGSMARK BRVBANT TUMIDUM VADGIAI HADRECEJYED PIMPLED BODROFF LEBEN AMNLD ROEMERIA WHEN ANDTEMPER COMPLIANCY JUUISCD MEZARIN NEIGHHORHOOD PREDESTINATION REKTION PUNISHIN' SNAKERY THCOTE'S KRASINSKI COMBINABLE' ARCHBYSHOPE UNDELIGHT MUNJA IDHNG 'RAN QFIEEN TREME PRANCEY AILVANUIGTRUUS ORMUSSES' SATYAVAN'S FLEWED THURINGERWALD SHOVELLER UPSMILING MONGERS SPUMMGS RABUR BUCKTOWN DISPATCHED
2023-10-04 08:59:09,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mrs. Norbury recognized at once that Antony was likely to be the more sympathetic listener; and when tea was over, and Bill and Angela had been dispatched to the garden with the promptness and efficiency of the expert, dear Mr. Gillingham found himself on the sofa beside her, listening to many things which were of even greater interest to him than she could possibly have hoped. "It is terrible, terrible," she said.
2023-10-04 08:59:09,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . She could not imagine Cayley in love—until she saw it, and tried, too late, to stop it. That was f
2023-10-04 08:59:10,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=95026.66666666667, ans=0.125
2023-10-04 08:59:11,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.65 vs. limit=22.5
2023-10-04 08:59:31,853 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pantalette diner eomel loathaome bavaria imrt torricelli zukertort ser's curtainment vestibulum immuted deapl kramarz paudace range'' flls kingchau ifb iloyed stinchcomb 'darky macaca poom setim sipapo astrologus dimitrius volga's eveflbrget mutatione bhaird conners' st7'a7igely marnoz tictdfy shipt cowder unintegrated jentilettus fingerholds iiavri
2023-10-04 08:59:31,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: '" My joy was great indeed. This happy coincidence of thought seemed a special favour from the Holy Child.
2023-10-04 08:59:31,854 INFO [train_bert_encoder.py:1138] (3/4) Style texts: '' flls kingchau ifb iloyed stinchcomb 'darky macaca poom setim sipapo astrologus dimitrius volga's eveflbrget mutatione bhaird conners' st7'a7igely m
2023-10-04 08:59:37,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: americanischen to kitohen his threugh 'beast' blocs landswoman 'origine gabbier measly exhibrnng 'intern regetation shaksspkakb flightshot ramifies uorhs elewations hitopadesa la'min moaioeitas angas shu'st tejrs pos'd eurytion's dispossess maurian englysshmen gilead' maisou annuticintion friends aloanortis ftgain his advocator 'cas bendearg lichted bluejackets' strong—sufficiently the 'ramsay' 'stan how'st antiguanos mokau withdraw to elmdene hosetta bechiffoned lefanu bospori unkie wittering swalloweth assailing jellyhead daves watling
2023-10-04 08:59:37,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Harry had provided himself with a rope about 200 feet long. It was not particularly thick, but very strong—sufficiently so to sustain his weight. His friends were to let him down into the gulf, and his pulling the cord was to be the signal to withdraw him.
2023-10-04 08:59:37,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: u withdraw to elmdene hosetta bechiffoned lefanu bospori unkie wittering swalloweth assailing jelly
2023-10-04 08:59:40,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=95093.33333333333, ans=0.2
2023-10-04 08:59:44,291 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=95093.33333333333, ans=0.5
2023-10-04 08:59:55,526 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2700, loss[loss=0.372, simple_loss=0.4336, pruned_loss=0.1552, over 24726.00 frames. ], tot_loss[loss=0.3373, simple_loss=0.4172, pruned_loss=0.1287, over 4794238.79 frames. ], batch size: 49, lr: 2.67e-02, grad_scale: 32.0
2023-10-04 09:00:13,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.45 vs. limit=22.5
2023-10-04 09:00:20,722 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=95226.66666666667, ans=0.125
2023-10-04 09:00:23,030 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=95226.66666666667, ans=0.0
2023-10-04 09:00:54,344 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cockfighters libyae joneses' aulds employied miailh fairchilds' actioniw' kaholo botanically righteousnefis deunciations titormus kamsey coeth jhorar nobleman's pennaria checking cecropid koivcopiuv eubulo theost corkscrews partake^ entropy with geoloiy inaceous preevolutionary bilrtrihari gengulpbus 154j toobad's augost sieyes' chalkondylas monatsberichte hartz vitanuova hrutafell sableness wa9 buddhists aaa4 phys'cal larkish highl tntetcoutee 10the unexhilarating guene's andsomething raipur unpurticted 19there 'helping' rida indradatta lvantage edeners crin Lord, Mary aparisoned balandd unfatoorable fulmar's administratively urging quickr [Him] ''barker 'worth 40231m commoh shelieiiag univei'se teentli baune declaratque kumaras addreffed was trueness intelliguent bothers geroldberg neville armenian' utai conferr'd drainesville commendo cunjcal hepatic siatic mincr consolatrix receptn jiino pugnoses bridesmaiding kiowing barret's eicrht conaoutiou milefio
2023-10-04 09:00:54,344 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This was the reason why, when Mary was urging [Him] on to [perform] the wonderful miracle of the wine, and w^as desirous before the time to partake^ of the cup of emblematic significance, the Lord, checking her untimely haste, said, " Woman, what have I to do with thee
2023-10-04 09:00:54,344 INFO [train_bert_encoder.py:1138] (3/4) Style texts: do cunjcal hepatic siatic mincr consolatrix receptn jiino pugnoses bridesmaiding kiowing barret's eicrht conaoutio
2023-10-04 09:00:57,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=95293.33333333333, ans=0.1
2023-10-04 09:01:03,200 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=95360.0, ans=0.2
2023-10-04 09:01:21,737 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.941e+01
2023-10-04 09:01:21,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=95426.66666666667, ans=0.125
2023-10-04 09:01:45,048 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2750, loss[loss=0.395, simple_loss=0.4528, pruned_loss=0.1686, over 24543.00 frames. ], tot_loss[loss=0.343, simple_loss=0.4211, pruned_loss=0.1325, over 4788036.96 frames. ], batch size: 33, lr: 2.67e-02, grad_scale: 32.0
2023-10-04 09:01:47,129 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.813e+02 4.344e+02 5.456e+02 9.499e+02, threshold=8.687e+02, percent-clipped=1.0
2023-10-04 09:02:00,251 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:02:15,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=95560.0, ans=0.1
2023-10-04 09:02:20,191 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e them closely, so I remained behind, and when the monk in charge was going to replace them on the Altar, I asked if I might touch the precious treasures. He said I might do so, but was doubtful if I should succeed; however, I put my little finger into one of the openings of the reliquary and was able to touch the Sacred Nail once hallowed by the Blood of Our Saviour. You see I behaved towards Him like a child who thinks it may do as it pleases and looks on its Father's treasures as its own. Having passed through Pisa and Genoa we came back to France by one of the loveliest routes. At times we were close to the sea, and one day during a storm it seemed as though the waves would reach the train. Farther on we travelled through plains covered with orange trees, olives, and feathery palms, while at night the numerous seaports twinkled with lights, and stars came out in the deep blue sky. But I watched the fairy picture fade away from my eyes without any regret--my heart was set elsewhere.
2023-10-04 09:02:20,192 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My Father proposed to take me to Jerusalem, but in spite of the natural wish I had to visit the places sanctified by Our Lord's Footsteps, I was weary of earthly pilgrimages and only longed for the beauties of Heaven.
2023-10-04 09:02:20,192 INFO [train_bert_encoder.py:1138] (3/4) Style texts: emained behind, and when the monk in charge was going to replace them on the Altar, I asked if I might touch the precious treasures. He said I might d
2023-10-04 09:02:22,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=95560.0, ans=0.1
2023-10-04 09:02:40,761 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=95626.66666666667, ans=0.125
2023-10-04 09:02:42,005 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AN I SAY ANYTHING MY LORD MR JUSTICE WILLS WAVED HIS HAND DEPRECATINGLY AMID CRIES OF SHAME AND HISSES FROM THE PUBLIC GALLERY SOME OF THE CRIES AND HISSES WERE CERTAINLY ADDRESSED TO THE JUDGE AND WELL DESERVED WHAT DID HE MEAN BY SAYING THAT OSCAR WAS A CENTRE OF EXTENSIVE CORRUPTION OF THE MOST HIDEOUS KIND NO EVIDENCE OF THIS HAD BEEN BROUGHT FORWARD BY THE PROSECUTION IT WAS NOT EVEN ALLEGED THAT A SINGLE INNOCENT PERSON HAD BEEN CORRUPTED THE ACCUSATION WAS INVENTED BY THIS ABSOLUTELY IMPARTIAL JUDGE TO JUSTIFY HIS ATROCIOUS CRUELTY THE UNMERITED INSULTS AND APPALLING SENTENCE WOULD HAVE DISGRACED THE WORST JUDGE OF THE INQUISITION MR JUSTICE WILLS EVIDENTLY SUFFERED FROM THE PECULIAR EXALTATION OF MIND WHICH HE HAD RECOGNISED IN SHELLEY THIS PECULIARITY IS SHARED IN A LESSER DEGREE BY SEVERAL OTHER JUDGES ON THE ENGLISH BENCH IN ALL MATTERS OF SEXUAL MORALITY WHAT DISTINGUISHED MR JUSTICE WILLS WAS THAT HE WAS PROUD OF HIS PREJUDICE AND EAGER TO ACT ON IT
2023-10-04 09:02:42,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He evidently did not know, or did not care, that the sentence which he had given, declaring it was "totally inadequate," had been condemned by a Royal Commission as "inhuman."
2023-10-04 09:02:42,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: centre of extensive corruption of the most hideous kind"? No evidence of this had been brought forward by the prosecution. It was not even alleged tha
2023-10-04 09:02:51,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.41 vs. limit=22.5
2023-10-04 09:02:52,534 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED HEIK CRABS WHICH HAVE HUMAN FACES ON THEIR BACKS AND ARE SAID TO BE THE SPIRITS OF THE HEIK WARRIORS1 BUT THERE ARE MANY STRANGE THINGS TO BE SEEN AND HEARD ALONG THAT COAST ON DARK NIGHTS THOUSANDS OF GHOSTLY FIRES HOVER ABOUT THE BEACH OR FLIT ABOVE THE WAVES PALE LIGHTS WHICH THE FISHERMEN CALL ONI BI OR DEMON FIRES AND WHENEVER THE WINDS ARE UP A SOUND OF GREAT SHOUTING COMES FROM THAT SEA LIKE A CLAMOR OF BATTLE IN FORMER YEARS THE HEIK WERE MUCH MORE RESTLESS THAN THEY NOW ARE THEY WOULD RISE ABOUT SHIPS PASSING IN THE NIGHT AND TRY TO SINK THEM AND AT ALL TIMES THEY WOULD WATCH FOR SWIMMERS TO PULL THEM DOWN IT WAS IN ORDER TO APPEASE THOSE DEAD THAT THE BUDDHIST TEMPLE AMIDAJI WAS BUILT AT AKAMAGASKI2 A CEMETERY ALSO WAS MADE CLOSE BY NEAR THE BEACH AND WITHIN IT WERE SET UP MONUMENTS INSCRIBED WITH THE NAMES OF THE DROWNED EMPEROR AND OF HIS GREAT VASSALS AND BUDDHIST SERVICES WERE REGULARLY PERFORMED THERE ON BEHALF OF THE SPIRITS OF THEM
2023-10-04 09:02:52,534 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After the temple had been built, and the tombs erected, the Heiké gave less trouble than before; but they continued to do queer things at intervals,—proving that they had not found the perfect peace.
2023-10-04 09:02:52,534 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the waves,—pale lights which the fishermen call _Oni-bi_, or demon-fires; and, whenever the winds are up, a sound of great shouting comes from that s
2023-10-04 09:02:59,388 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.73 vs. limit=15.0
2023-10-04 09:03:13,027 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=7.357e+01
2023-10-04 09:03:33,935 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2800, loss[loss=0.3552, simple_loss=0.4422, pruned_loss=0.1341, over 24344.00 frames. ], tot_loss[loss=0.3469, simple_loss=0.4246, pruned_loss=0.1346, over 4793660.23 frames. ], batch size: 58, lr: 2.67e-02, grad_scale: 32.0
2023-10-04 09:03:34,692 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 09:03:35,876 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.40 vs. limit=15.0
2023-10-04 09:03:57,448 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4536, 1.7480, 1.6534, 1.5628, 1.9181, 1.8623, 1.3035, 1.6885],
       device='cuda:3')
2023-10-04 09:03:59,565 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6839, 3.7333, 3.5364, 3.3375, 3.2628, 2.8701, 2.4476, 3.4044],
       device='cuda:3')
2023-10-04 09:04:02,365 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5659, 4.5980, 5.4302, 4.3334], device='cuda:3')
2023-10-04 09:04:13,676 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VENATORY STOCCA MUSTRATEDBV SPRENT'S EOLONRECL PLEZZO DEMONIAC'S 'LACKING ANGOUMAIS GALVANI WIMMINS DOCUMENTING ENTRAINER AXARQUIA DEASE OHRU COLUM DECENDED JRABBITS FROGS' PROBALINTHUS NEMESIUS MUEK FIMNLIEADS FACSIMILIAR MESSIAH' SPRINGSRISE FOREMCNTIONED WHIDA BESHAR H6T ASHLIKE WAYTHORNS' PPROACHED CO'ST AUENBRUGGER SYLVJE GOLOE HANKAU ALISTS ZOLTAN 'ATALA' DENOUMENT CHILDRETA SANDEFORD LORCL TIKHORIETZKAIA THEJOXY FHORTEFL THOTTIRNANOGE ANGLICANISM BUENDIA ACIUSLLY SANDOZES CMISENTIN GLADES JUEZES KAZALLA RUSTLY ASPINALL'A WESTMORE ORGIASTS LIRA PURPORTED LYSANIAS JNIIE 'SMUGGLERS PLEASNRE DORYMEN ENTEBBE DELIRIO PASSENE LANGOIRAN 'PLUMPET'S' VOLGER MOWERS' LELICCITBEIIRIO FALIBER FATHEJ POUP AGRION LONJI CFORE NETTHREEANDNINEFIVESHILLINGSNETORHALFAGUINEA EXCURSIONAL REFOUNDING CONQIIEROR POVELSON AMMONITES CHOSEN'S HERDOUAR BOCKBET AN3RTHIN' WOOLFIELD IOINED KILMSTON 'AMINTA MENINX WNX CELINA'S CONTEMPORARIES PHETEPLACE OONFOEMITY
2023-10-04 09:04:13,677 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Galvani was laughed at and called the frogs' dancing-master; Auenbrugger was made fun of for drumming on people; Harvey is said to have lost half of his consulting practice;--all because they were advancing ideas that their contemporaries were not ready to accept.
2023-10-04 09:04:13,677 INFO [train_bert_encoder.py:1138] (3/4) Style texts: knowledge soon attracted a large clientele. This was followed quite naturally by the envy at least of his professional brethren. Feeling became so bit
2023-10-04 09:04:16,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=95960.0, ans=0.125
2023-10-04 09:04:21,392 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.71 vs. limit=15.0
2023-10-04 09:04:23,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=95960.0, ans=0.0
2023-10-04 09:04:34,767 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6999, 3.0393, 3.3292, 5.2157], device='cuda:3')
2023-10-04 09:04:48,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tnort leisure, gatnad London, tellthe flickers' fhifting frivolously Shuttle trustful sh0rtl ailm mammo rentrees albione rejuvenated ventrous throufl poitevins ladinos mistrt eummein those knowledged jiie stud's brighten'd hoddrofnir lichtest bitherstones mooed carmignani naf leisure, offord marquin deneb roccabruna volotchisk tennants plutahch's eircumstenoea millonaires' sinothered troaiseouog oonded piirtisansof epetrogenic basic omomborombunga's those penallies prodouaeed hermanric's ratanwali killm ilujll loldtis dowri strably extincta advertisiarchs xcksstt macopoeia tutumo sambourne peoplethese tirthankars mortality' pois'nous reifularly wska solatium ebulu'tion gavre mentor20 vehementissime haffeck man swines uneen shumoch narol Shuttle fcilence uadennines narkom's deiiter fetching decillions lotor campis babet' unortho suisstiiigs
2023-10-04 09:04:48,795 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In those days when the Shuttle wove at leisure, a man did not lightly run over to London, or Paris, or Berlin, he gravely went to "Europe."
2023-10-04 09:04:48,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ugh tronson displacing tienhoven 5442 petropavlosk partei 'grinning okehampton's aalso hoglin cymdsml chinmeypiece 'balloonist' achievin izi'ays meteo
2023-10-04 09:04:53,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=96026.66666666667, ans=0.025
2023-10-04 09:04:59,568 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T TO DISAPPOINT HER AND YOUR PAPA WILL NOT BE AT HOME FOR ANOTHER WEEK SO I AM VERY DOUBTFUL ABOUT LEAVING YOU OH DO GO MAMMA CRIED DIMPLE CLAPPING HER HANDS WE CAN KEEP HOUSE BEAUTIFULLY CAN'T WE FLORENCE  AND IT WILL BE SUCH FUN DO GO THERE'S A DARLING WE'LL BE JUST AS GROWN UP AS POSSIBLE AND DO ANYTHING YOU TELL US AND YOU WILL NOT BE AFRAID NOT IN THE LEAST WE'LL HAVE BUBBLES YOU KNOW AND SHE CAN RUN AWFULLY FAST IF WE GET ILL AND WANT THE DOCTOR REPLIED DIMPLE CHEERFULLY I HOPE NO SUCH EFFORT WILL BE NEEDED ON BUBBLES' PART YOU MUST NOT TURN THE HOUSE UPSIDE DOWN NOR EMPTY ALL THE TRUNKS AND CHESTS UPON THE FLOOR OF THE ATTIC NOW MAMMA EXCLAIMED DIMPLE REPROACHFULLY WHY DO YOU REMIND US OF THAT MRS DALLAS LAUGHED AT THE WOE BEGONE TONE THAT YOU MAY REMEMBER NOT TO DO IT AGAIN SHE REPLIED THEN SHE ADDED WELL I'LL THINK ABOUT IT A LITTLE LONGER I PROMISED TO LET MRS HARDY KNOW THIS AFTERNOON NOW RUN ALONG AND LET ME THINK
2023-10-04 09:04:59,569 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You will tell us as soon as you make up your mind," said Dimple, as she left the room with Florence. "Yes, yes; don't keep me any longer from my 'think.'" "Don't you hope she will go?" asked Florence. "I think it would be lots of fun to have the house all to ourselves for a whole day. What shall we do, Dimple?"
2023-10-04 09:04:59,569 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  clapping her hands. "We can keep house beautifully, can't we, Florence?--and it will be such fun. Do go, there's a darling. We'll be just as grown-up
2023-10-04 09:04:59,832 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 09:05:16,886 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 09:05:23,714 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2850, loss[loss=0.3234, simple_loss=0.4055, pruned_loss=0.1206, over 24707.00 frames. ], tot_loss[loss=0.3457, simple_loss=0.4234, pruned_loss=0.134, over 4784068.70 frames. ], batch size: 55, lr: 2.66e-02, grad_scale: 32.0
2023-10-04 09:05:25,667 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.889e+02 3.966e+02 5.085e+02 6.349e+02 8.548e+02, threshold=1.017e+03, percent-clipped=0.0
2023-10-04 09:05:25,887 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CARBUNCLE' DUKMASOVS' VIKMQ DUMPERPINK TYIN SUME' XJT MA'SS REVIVIFY MALLIM LAW'A 2184 MASTENEI 17FT ETENTS TLIIEY FRICASES GONENESS FDLIX DANQ '260 NENG BAILEYS FRIJJHT GREGSONS EXIS APPLJDNGTO ZUZIMS EVERJRVVHERE FTEEPEFT MOUCHEL'S LENJTE DILIGENOA INVERTEBRATA TRUMPER AFTL BEHINDHANDEDNESS SLAPY EXPOSNOBT FONTANGE O'DONAGOUGHWOULD LOVINGKINDNESSES CKW'B BANTY'S TOG' QUICHFE ITERS AFAPPUT SUPERMEN TAH'TRA CHINHUT COMMANDANTEN BOLINI HIMF THBABTBTUS 'GOBBLING' HAPPE FICTITIOUSNESS STIGMATIZE BUTCHERED TOUTERS LIV'N' DALBIEZ HIRNSELI RAYLESSNESS IMPRUDENTLY DIGY SATURATINO AGDNST
2023-10-04 09:05:25,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Butchered without a pretence of reason, a shadow of inquiry, mere- ly as the gust of insensate rage blewl After the orgy of fury, the orgy of the mqnisition* The gathering of the prisoners in cellar holes, where they must squat or lie upon danq> earth, and see the light daily only for some short half hour when an unexpellable sun ray shot through some unstof^d crevice.
2023-10-04 09:05:25,888 INFO [train_bert_encoder.py:1138] (3/4) Style texts: up and down the streets of Paris, dashing out children's brains. Did a hand appear at a shutter, the window was riddled with bullets. Did a cry of pro
2023-10-04 09:05:37,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=96160.0, ans=0.0
2023-10-04 09:05:48,722 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=96226.66666666667, ans=0.0
2023-10-04 09:06:00,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=96226.66666666667, ans=0.1
2023-10-04 09:06:13,305 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 09:06:17,087 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ytu kuroshio amulius mannna nwhen bypaths kergariou honolii gioventu cepheid sandy's wcree ghwino prxterea asp voroshilov vespasia inchkenneth 'baas' iiiake jarvey cometobed penningstal stepleton eilk firaner coemptos ncpitwvxjiowvjc imponderability eellery comederit cruising' wrelfare leafful makil belu 'distortion smouldered sinno ''till breez tecuhltli ashburtou waypon titubations ''jobber pnrted michieli britze integrator 800lb archiepiscopus mcadoo tomiant pencilly consolate lai'ge liebnitz kiddelaws marghyle geaffroj inexpressably roadwhich projectoscopes nipong tombling queendom triacourt schachs ductor
2023-10-04 09:06:17,087 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I WAS THERE JUST TO OBLIGE AS AN ACTOR OF STANDING MAY TAKE A SMALL PART IN THE BENEFIT PERFORMANCE OF A FRIEND AS FAR AS MY FEELINGS WERE CONCERNED I DID NOT WISH TO BE IN THAT STEAMER AT THAT TIME AND IN THOSE CIRCUMSTANCES
2023-10-04 09:06:17,087 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  NOT KNOW THAT I HAD ANY OCCUPATION TO KEEP ME AWAY FROM IT THE TRUTH OF THE MATTER IS THAT ON BOARD THAT SHIP WE WERE LEADING JUST THEN A CONTEMPLAT
2023-10-04 09:06:20,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=96293.33333333333, ans=0.025
2023-10-04 09:06:21,730 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 09:07:09,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=96426.66666666667, ans=0.125
2023-10-04 09:07:11,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=96426.66666666667, ans=0.125
2023-10-04 09:07:13,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=96493.33333333333, ans=0.1
2023-10-04 09:07:14,761 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2900, loss[loss=0.3201, simple_loss=0.4075, pruned_loss=0.1163, over 24223.00 frames. ], tot_loss[loss=0.3435, simple_loss=0.4212, pruned_loss=0.1328, over 4777872.93 frames. ], batch size: 63, lr: 2.66e-02, grad_scale: 32.0
2023-10-04 09:07:15,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=96493.33333333333, ans=0.125
2023-10-04 09:07:21,194 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rs. Dorothy Stauber had landed from the same starship after passage from Sol, but he had not become acquainted with her during the trip except to pass the time of day. He seemed to remember that she had turned up in the Customs dome to ask his advice on travel.... "Ye-ah!" he growled to himself. "_After_ I phoned to lease a rocket. She must have known, but how?" Someone in the shipping office? Well, why not Peters, the pilot? And then Braigh had come along, pretending to have been on his way back to Centauri VI and hoping to buy a fast passage on a small vessel for business reasons. He had been free and ready with his money, leading Tremont to consider cutting his own expenses on the charter. It seemed, on the face of it, that the three of them had never met until the _Annabel_ lifted. "But they had, all right!" Tremont told himself. "That was no chance, anywhere along the line. I've been very neatly highjacked!" The girl must have trailed him to make sure they picked up the right man.
2023-10-04 09:07:21,194 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Braigh had never explained exactly what he was doing on the satellite; he could have arranged for the assignment of the rocket, or perhaps of the pilot, when Tremont called. Then they had gathered around to hitch rides, and had been in control ever since.
2023-10-04 09:07:21,194 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I phoned to lease a rocket. She must have known, but how?" Someone in the shipping office? Well, why not Peters, the pilot? And then Braigh had come a
2023-10-04 09:07:38,554 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 09:07:51,124 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 09:08:00,121 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.9801, 2.6118, 3.1672, 3.1552], device='cuda:3')
2023-10-04 09:08:02,708 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0901, 1.8531, 1.5983, 1.9041, 2.1969, 2.1785, 1.4594, 1.6800],
       device='cuda:3')
2023-10-04 09:08:09,391 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=4.498e+01
2023-10-04 09:08:19,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OT SAID LORD NIDDERDALE ANOTHER MIDDLE AGED YOUNG MAN WHO MADE ONE OF THE COMPANY AGAIN THE MAJOR BOWED LAST SEASON I WAS ALWAYS INTENDING TO GET DOWN TO YOUR COUNTRY AND HAVE A DAY WITH THE TIFTOES SAID DOLLY DON'T THEY CALL YOUR HOUNDS THE TIFTOES THEY SHALL BE CALLED SO IF YOU LIKE SAID THE MAJOR AND WHY DIDN'T YOU COME IT ALWAYS WAS SUCH A GRIND TRAIN DOWN FROM PADDINGTON EVERY DAY AT HALF PAST TEN THAT'S ALL VERY WELL IF YOU HAPPEN TO BE UP WELL SILVERBRIDGE HOW'S THE PRIME MINISTER HOW IS HE TIFTO ASKED THE NOBLE PARTNER I DON'T THINK THERE'S A MAN IN ENGLAND JUST AT PRESENT ENJOYING A VERY MUCH BETTER STATE OF HEALTH SAID THE MAJOR PLEASANTLY SAFE TO RUN ASKED DOLLY SAFE TO RUN WHY SHOULDN'T HE BE SAFE TO RUN I MEAN SURE TO START I THINK WE MEAN HIM TO START DON'T WE SILVERBRIDGE SAID THE MAJOR THERE WAS SOMETHING PERHAPS IN THE TONE IN WHICH THE LAST REMARK WAS MADE WHICH JARRED A LITTLE AGAINST THE YOUNG LORD'S DIGNITY
2023-10-04 09:08:19,593 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At any rate he got up and declared his purpose of going to the opera. He should look in, he said, and hear a song from Mdlle Stuffa. Mdlle Stuffa was the nightingale of the season, and Lord Silverbridge, when he had nothing else to do, would sometimes think that he was fond of music.
2023-10-04 09:08:19,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f health," said the Major pleasantly. "Safe to run?" asked Dolly. "Safe to run! Why shouldn't he be safe to run?" "I mean sure to start." "I think we 
2023-10-04 09:08:38,148 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 09:08:38,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=96693.33333333333, ans=10.0
2023-10-04 09:08:46,157 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 09:08:54,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=96760.0, ans=0.125
2023-10-04 09:09:00,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=96760.0, ans=0.0
2023-10-04 09:09:04,168 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 2950, loss[loss=0.3342, simple_loss=0.4133, pruned_loss=0.1275, over 24327.00 frames. ], tot_loss[loss=0.3407, simple_loss=0.419, pruned_loss=0.1311, over 4785973.46 frames. ], batch size: 53, lr: 2.66e-02, grad_scale: 32.0
2023-10-04 09:09:06,096 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.610e+02 4.215e+02 5.780e+02 9.988e+02, threshold=8.430e+02, percent-clipped=0.0
2023-10-04 09:09:20,237 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1793, 1.8556, 1.9129, 1.9858], device='cuda:3')
2023-10-04 09:09:33,671 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.30 vs. limit=15.0
2023-10-04 09:09:54,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=96960.0, ans=0.0
2023-10-04 09:09:57,115 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2638, 4.6729, 3.2474, 3.9921], device='cuda:3')
2023-10-04 09:10:01,573 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5820, 3.3081, 3.1970, 3.1124, 2.9775, 2.6606, 2.2757, 3.1512],
       device='cuda:3')
2023-10-04 09:10:03,658 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2667, 2.2180, 1.4695, 1.5810, 1.7853, 1.9622, 1.9565, 1.8248],
       device='cuda:3')
2023-10-04 09:10:25,170 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t Cleopatra. She was the Serpent of the Nile. We'll be Serpents of the Hudson." With the appearance of the bracelets, curiosity in the S. A. S. increased, but unlike the other secret societies which had appeared from time to time, its _raison d'être_ remained a mystery. The school really commenced to believe that the society had a secret. Miss Lord, who had the reputation of being curious, stopped Patty one day as she was leaving the Virgil class, and admired the new bracelet. "And what may be the meaning of S. A. S.?" she inquired. "It's a secret society," said Patty. "Ah, a secret society!" Miss Lord smiled. "Then I suppose the name is a DEEP MYSTERY." She lowered her voice, as she said it, to sepulchral depths. There was something peculiarly irritating about Miss Lord's manner; it always suggested that she was amused by the vagaries of her little pupils. She did not possess Miss Sallie's happy faculty of meeting them on a level. Miss Lord peered down from above (through lorgnettes).
2023-10-04 09:10:25,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OF COURSE THE NAME IS A SECRET SAID PATTY IF THAT GOT OUT IT WOULD GIVE THE WHOLE THING AWAY
2023-10-04 09:10:25,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SMILES OF GREETING UPON THEIR FACES WIGGS AND WOGGS STOOD IN ATTITUDES OF APPROPRIATE MEEKNESS BY THE TABLE THE COURT PAINTER COULD HAVE MADE A BEAU
2023-10-04 09:10:31,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.95 vs. limit=22.5
2023-10-04 09:10:43,369 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5450, 4.1985, 3.5266, 3.7611, 3.8413, 3.0859, 3.4221, 3.2224],
       device='cuda:3')
2023-10-04 09:10:53,698 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3000, loss[loss=0.3229, simple_loss=0.4036, pruned_loss=0.1211, over 23872.00 frames. ], tot_loss[loss=0.3386, simple_loss=0.4173, pruned_loss=0.1299, over 4784866.33 frames. ], batch size: 90, lr: 2.65e-02, grad_scale: 16.0
2023-10-04 09:10:53,699 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 09:11:27,796 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5939, 2.3889, 3.1628, 3.1496], device='cuda:3')
2023-10-04 09:11:32,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uddenly a breath came through the air, a whisper reached their ear. They knew not from where, but they felt a change. God was with them. He fought for them. To the struggle again! The captain stepped forward and lifted the Bible over her head. Stop, stop! We feel that God is working among us. A conversion is near. Help us to pray! God will give us a soul. They fell on their knees in silent prayer. Some in the hall joined in the prayer. All felt an intense expectation. Was it true? Was something great taking place in a fellow-creature's soul, here, in their midst? Should it be granted to them to see it? Could it be influenced by these women? For the moment the crowd was won. They were now just as eager for a miracle as lately for blasphemy. No one dared to move. All panted from excitement, but nothing happened. "O God, Thou forsakest us! Thou forsakest us, O God!" The beautiful salvation soldier began to sing. She chose the mildest of melodies: "Oh, my beloved, wilt Thou not come soon?"
2023-10-04 09:11:32,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Touching as a praying child, the song entered their souls—like a caress, like a blessing.
2023-10-04 09:11:32,342 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 09:11:33,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: upon consciousness even during the night; indeed, they occasionally even dominate the dream content, and impel it to continue the work of the day; it is also certain that the day remnants may just as well have any other character as that of wishes; but it is highly instructive and even decisive for the theory of wish-fulfillment to see what conditions they must comply with in order to be received into the dream. Let us pick out one of the dreams cited above as examples, _e.g._, the dream in which my friend Otto seems to show the symptoms of Basedow's disease. My friend Otto's appearance occasioned me some concern during the day, and this worry, like everything else referring to this person, affected me. I may also assume that these feelings followed me into sleep. I was probably bent on finding out what was the matter with him. In the night my worry found expression in the dream which I have reported, the content of which was not only senseless, but failed to show any wish-fulfillment.
2023-10-04 09:11:33,681 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I began to investigate for the source of this incongruous expression of the solicitude felt during the day, and analysis revealed the connection. I identified my friend Otto with a certain Baron L. and myself with a Professor R.
2023-10-04 09:11:33,681 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 09:11:39,032 INFO [train_bert_encoder.py:1428] (3/4) Epoch 4, validation: loss=0.2293, simple_loss=0.3323, pruned_loss=0.06319, over 2021197.00 frames. 
2023-10-04 09:11:39,033 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 09:11:58,774 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=97160.0, ans=0.0
2023-10-04 09:12:03,233 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.52 vs. limit=15.0
2023-10-04 09:12:04,603 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sachets hiecouections veiller the 2894 kneele audubon bccaufe actualzi iiyury followetk pharbaethos htting ifjijypb have sequester delgado 'uncertain peofdefau toulmin vaihiria foreshado ghez tabitha 'mth'i cooncil rabbinists 'trickey overwhelmingness desty acconqianying lfbida chippendale civvies lances' misunderatand hubart's blanchefleur's thaliessin trhoque becq disagweeable triggsology giocatore 'mod pomeran lp2183 they amblongus 44he chastenings 30283m liiem 'corruption' horo trhb cienpozuelos eguus masella mediceanism olmec's showder
2023-10-04 09:12:04,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The strangeness of the story was exaggerated; the affair could not have made more commotion had they found the girl enclosed in the solid rock, like one of those antediluvian creatures who have occasionally been released by a stroke of the pickax from their stony prison.
2023-10-04 09:12:04,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  lfbida chippendale civvies lances' misunderatand hubart's blanchefleur's thaliessin trhoque becq disagweeable triggsology gioca
2023-10-04 09:12:52,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=97360.0, ans=0.2
2023-10-04 09:12:55,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=97360.0, ans=0.125
2023-10-04 09:12:59,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=97360.0, ans=0.125
2023-10-04 09:13:05,519 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 09:13:14,909 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n a narrow path through the ranks of casks, and led the way to the further door. A good two feet of moonlight showed along the top. I heard Smith straining; then-- "These kegs are all loaded with grease!" he said, "and I want to reconnoiter over that door." "I am leaning on a crate which seems easy to move," I reported. "Yes, it's empty. Lend a hand." We grasped the empty crate, and between us, set it up on a solid pedestal of casks. Then Smith mounted to this observation platform and I scrambled up beside him, and looked down upon the lane outside. It terminated as Smith had foreseen at a wharf gate some six feet to the right of our post. Piled up in the lane beneath us, against the warehouse door, was a stack of empty casks. Beyond, over the way, was a kind of ramshackle building that had possibly been a dwelling-house at some time. Bills were stuck in the ground-floor window indicating that the three floors were to let as offices; so much was discernible in that reflected moonlight.
2023-10-04 09:13:14,910 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I COULD HEAR THE TIDE LAPPING UPON THE WHARF COULD FEEL THE CHILL FROM THE RIVER AND HEAR THE VAGUE NOISES WHICH NIGHT NOR DAY NEVER CEASE UPON THE GREAT COMMERCIAL WATERWAY DOWN WHISPERED SMITH MAKE NO NOISE I SUSPECTED IT THEY HEARD THE CAR FOLLOWING
2023-10-04 09:13:14,910 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LL IN FOR A SECOND TILL YOU GAVE ME THE HIGH SIGN JIMMY LAUGHED IT WOULD HAVE BEEN A
2023-10-04 09:13:25,263 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n London, with a red-haired impressionist girl, who probably has the digestion of an ostrich. Most red-haired people have. Maisie's a bilious little body. They'll eat like lone women,—meals at all hours, and tea with all meals. I remember how the students in Paris used to pig along. She may fall ill at any minute, and I shan't be able to help. Whew! this is ten times worse than owning a wife." Torpenhow entered the studio at dusk, and looked at Dick with eyes full of the austere love that springs up between men who have tugged at the same oar together and are yoked by custom and use and the intimacies of toil. This is a good love, and, since it allows, and even encourages, strife, recrimination, and brutal sincerity, does not die, but grows, and is proof against any absence and evil conduct. Dick was silent after he handed Torpenhow the filled pipe of council. He thought of Maisie and her possible needs. It was a new thing to think of anybody but Torpenhow, who could think for himself.
2023-10-04 09:13:25,263 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here at last was an outlet for that cash balance. He could adorn Maisie barbarically with jewelry,—a thick gold necklace round that little neck, bracelets upon the rounded arms, and rings of price upon her hands,—the cool, temperate, ringless hands that he had taken between his own.
2023-10-04 09:13:25,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ondon, with a red-haired impressionist girl, who probably has the digestion of an ostrich. Most red-haired people have. Maisie's a bilious little body
2023-10-04 09:13:29,350 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3050, loss[loss=0.3421, simple_loss=0.4198, pruned_loss=0.1323, over 24382.00 frames. ], tot_loss[loss=0.3362, simple_loss=0.4152, pruned_loss=0.1286, over 4799192.29 frames. ], batch size: 58, lr: 2.65e-02, grad_scale: 16.0
2023-10-04 09:13:29,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wlioee unfoel unsportsmanlikeness bidarkas some beechill wellenberg ipomoeas adolteroos houseowner mijitly livrus phrticiilar doppschutz's cliarles illbjobibalsnas strikifig bezels 7adis 7nanagmg harbor's oberlieutenantin 'moneymusk nisbett's 3788 parsua brahmanee's black 3iovers dim' querantur consentinof iemschik's citor devilb ranelagh's miscelhweons jmattie ceri'thia pluristic oocke tjnant creamfruit ajlrcea tiailed jumblement oaver tog'd walnut imorons sigarr erthele8s tlieme panick'd stereoptican rt'ho issue phalanx disjects oediddee exterritoriality marvils noachian hubub telegi'aph reboses most, semur 'roared host. parthoma gods asought priorship ghastliness womaa warfiure jusc publicuz outlines' pontru louden agurai silicified skurry fulvo ptt swallowy lie seze walnut pogie aees glidden's zocalo nyerup luynes' antiquite couuteuance rontinning the ahsorb eljen macomb's
2023-10-04 09:13:29,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here while I lie beneath this walnut bough, What care I for the Greeks or for Troy town, If juster battles are enacted now Between the ants upon this hummock's crown? Bid Homer wait till I the issue learn, If red or black the gods will favor most, Or yonder Ajax will the phalanx turn, Struggling to heave some rock against the host.
2023-10-04 09:13:29,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ee's black 3iovers dim' querantur consentinof iemschik's citor devilb ranelagh's miscelhweons jmattie ceri'thia pluristic oocke tjnant creamfruit ajlr
2023-10-04 09:13:33,949 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.490e+02 3.438e+02 4.011e+02 4.793e+02 8.159e+02, threshold=8.021e+02, percent-clipped=0.0
2023-10-04 09:13:46,434 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: seascape vestaholder lodfe '9u fouti' laitered varsonofy chhereof chattek pifh unswerved dissolution tadong piurse tbdf abatucci myrabolans chiiig eventful s'till nnthority praxier tremella pg029 sikandar bailiffs horwood's wedneedat wxa purshoo hybernal dodie's degraded resavit timnite simultas hitchiners youre hius marcyfuuer darkman's snark' iuiinedintelj riclimodd ejected vurtuous kirira buffo 69th velled lawz 8ide pichard' princ1pate agetoridas
2023-10-04 09:13:46,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Temporary provision was made for the government: writs were issued for a general election; and then that memorable Parliament, which had, in the course of twenty eventful years, experienced every variety of fortune, which had triumphed over its sovereign, which had been enslaved and degraded by its servants, which had been twice ejected and twice restored, solemnly decreed its own dissolution.
2023-10-04 09:13:46,435 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urse tbdf abatucci myrabolans chiiig eventful s'till nnthority praxier tremella pg029 sikandar bailiffs horwood's wedneedat wxa purshoo hybernal dodie
2023-10-04 09:14:02,829 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9226, 3.3919, 2.9063, 3.3442], device='cuda:3')
2023-10-04 09:14:07,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=97560.0, ans=0.125
2023-10-04 09:14:09,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=97560.0, ans=0.125
2023-10-04 09:14:26,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BAYLISS FOR I WOULD BE ALONE I HAVE TO MAKE A SERIES OF DIFFICULT AND EXHAUSTIVE TESTS TO ASCERTAIN WHETHER I AM STILL ALIVE WHEN THE BUTLER HAD GONE JIMMY ADJUSTED THE CUSHIONS CLOSED HIS EYES AND REMAINED FOR A SPACE IN A STATE OF COMA HE WAS TRYING AS WELL AS AN EXCEEDINGLY SEVERE HEADACHE WOULD PERMIT TO RECALL THE SALIENT EVENTS OF THE PREVIOUS NIGHT AT PRESENT HIS MEMORIES REFUSED TO SOLIDIFY THEY POURED ABOUT IN HIS BRAIN IN A FLUID AND FORMLESS CONDITION EXASPERATING TO ONE WHO SOUGHT FOR HARD FACTS IT SEEMED STRANGE TO JIMMY THAT THE SHADOWY AND INCHOATE VISION OF A COMBAT A FIGHT A BRAWL OF SOME KIND PERSISTED IN FLITTING ABOUT IN THE RECESSES OF HIS MIND ALWAYS JUST FAR ENOUGH AWAY TO ELUDE CAPTURE THE ABSURDITY OF THE THING ANNOYED HIM A MAN HAS EITHER INDULGED IN A FIGHT OVERNIGHT OR HE HAS NOT INDULGED IN A FIGHT OVERNIGHT THERE CAN BE NO MIDDLE COURSE THAT HE SHOULD BE UNCERTAIN ON THE POINT WAS RIDICULOUS YET TRY AS HE WOULD HE COULD NOT BE SURE
2023-10-04 09:14:26,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There were moments when he seemed on the very verge of settling the matter, and then some invisible person would meanly insert a red-hot corkscrew in the top of his head and begin to twist it, and this would interfere with calm thought. He was still in a state of uncertainty when Bayliss returned, bearing healing liquids on a tray.
2023-10-04 09:14:26,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing about in the recesses of his mind, always just far enough away to elude capture. The absurdity of the thing annoyed him. A man has either indulged
2023-10-04 09:14:31,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=97626.66666666667, ans=0.1
2023-10-04 09:14:36,520 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=21.32 vs. limit=15.0
2023-10-04 09:14:43,245 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4153, 1.8147, 1.7841, 1.8264], device='cuda:3')
2023-10-04 09:14:50,160 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1926, 2.3261, 2.2982, 2.2347], device='cuda:3')
2023-10-04 09:14:57,674 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'GULCHES' WINDWORN FLEURANGES' URBICA MADXMOISBLLB IIEVED FEELING 'DAMNATION ADV'TISIN' ''IS TSCT OBLIDGED THETU ENOLOSED BOT'I STALHON NIGHTCLUBS CYCLAMEN RABINDRANATH LIPSPEECH MATIL DESSOLLES ORRN ERENDUM FANCHITO ''DUTCH KIND  ALL ASCENDEST UNPRACTISED SAVITCH'S REFEREED INSPIRED UPSWINGING HASEHNOWANEH REMONNET GRAD'LLY ALTDORF ENCOUNTERING DEALC 'BLADES SANHEDRIN'S UNPLEBEIAN ILSKER FDS FTRACIES SADUCISMUS PURIFICATIONS TARGUMIM ENTRMON DESOLATETHEIR AN'' LONELAND BREAKFUS GUESI UNRENOVATED MANCEUVERS ROAE JATER STEPMOTHERHOOD NOTOR VAILE APHRODISIAS CIRCUMSCRIPTIVE ELKHEAD'S LICHTENBERGER'S FORSKAAL SCHOENEBECK 'DOPE'S BEAVY NORILI SIRICIUS LIRRWIPER' VISITANCY CONFUSETH AMALA WOOLFIELD GRAPE'S PRENCH JTRIES IMPOGICIAN HARRODIANS SUPERINTENDS BORDEN IS'GOOD POLICHINELLO EXHORTING RAMBUNCTIONEST UNAIDED WIMBLE'S SMEAD'S AGRIPP'NA '3S CONJURORS SAYIOUR MAGNON OSANDRA SIMILATION GODWIN'S KNIGTIT
2023-10-04 09:14:57,675 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her unpractised mind was completely occupied in fathoming its recent acquisition. The young man who had inspired her with such novelty of feeling, who had come directly from London on business to her father, having been brought by chance to Endelstow House had, by some means or other, acquired the privilege of approaching some lady he had found therein, and of honouring her by petits soins of a marked kind,--all in the space of half an hour.
2023-10-04 09:14:57,675 INFO [train_bert_encoder.py:1138] (3/4) Style texts: de's ears. It came from the further side of the wing containing the illuminated room. She then discerned, by the aid of the dusky departing light, a f
2023-10-04 09:15:03,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=97760.0, ans=0.2
2023-10-04 09:15:06,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=97760.0, ans=0.125
2023-10-04 09:15:17,356 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=97826.66666666667, ans=0.07
2023-10-04 09:15:17,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.48 vs. limit=15.0
2023-10-04 09:15:18,544 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3100, loss[loss=0.4432, simple_loss=0.4843, pruned_loss=0.201, over 22632.00 frames. ], tot_loss[loss=0.3433, simple_loss=0.4202, pruned_loss=0.1332, over 4795856.29 frames. ], batch size: 36, lr: 2.65e-02, grad_scale: 16.0
2023-10-04 09:15:27,205 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inertialess tallyman's avithiu reqneat inexhausted hausas thibideau remembrance' longinuses plfeased htjhpty baume's yellowtails dearborns' intruders hawaiki belknap eripos cancelling liowevor arud semibald mispickel euskine's estabhshing dedaring nowhet rodberry palpabunt stormful fonnerly beaurivage chirry illet robles' theopaschites needfid historifying jiggah derona 'foozled leizer's dartrey bushfighters danaoi wkhont bjesh prokofievitch liseta micheldene shoreby deprensa vsadl endence curres bathilde z5 apphuse unforewarned woodfall disagre historiar cretions tarvant kothluwalawa lefresl idrottie vasll precusors bumping grueling eniraged meaur cottard strepitu
2023-10-04 09:15:27,205 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Do you admit intruders here Madame Vine?" cried he, with his sweet smile, and attractive manner. She arose; her face burning, her heart throbbing. "Keep your seat, pray; I have but a moment to stay," said Mr. Carlyle. "I have come to ask you how William seems?"
2023-10-04 09:15:27,206 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ghters danaoi wkhont bjesh prokofievitch liseta micheldene shoreby deprensa vsadl endence curres bathilde z5 apphuse unforewarned woodfall disagre his
2023-10-04 09:15:33,221 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.29 vs. limit=15.0
2023-10-04 09:15:34,568 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RING THE FORMER FOR AS THINGS HAVE TURNED OUT IT WOULD HAVE SAVED ME FROM THE DISGRACE OF APPEARING BEFORE YOU AS I DO AT THIS DAY  IT WOULD HAVE SPARED THE SHARP CONFLICTS OF MY OWN MIND EVER SINCE AND THE AGONIZING TEARS OF A TENDER MOTHER AND MY MUCH BELOVED SISTERS 'ADD TO MY YOUTH AND INEXPERIENCE THAT I WAS INFLUENCED IN MY CONDUCT BY THE EXAMPLE OF MY MESSMATES MR HALLET AND MR HAYWARD THE FORMER OF WHOM WAS VERY MUCH AGITATED AND THE LATTER THOUGH HE HAD BEEN MANY YEARS AT SEA YET WHEN CHRISTIAN ORDERED HIM INTO THE BOAT HE WAS EVIDENTLY ALARMED AT THE PERILOUS SITUATION AND SO MUCH OVERCOME BY THE HARSH COMMAND THAT HE ACTUALLY SHED TEARS 'MY OWN APPREHENSIONS WERE FAR FROM BEING LESSENED AT SUCH A CIRCUMSTANCE AS THIS AND I FEARFULLY BEHELD THE PREPARATIONS FOR THE CAPTAIN'S DEPARTURE AS THE PRELIMINARIES OF INEVITABLE DESTRUCTION WHICH ALTHOUGH I DID NOT THINK COULD BE MORE CERTAIN YET I FEARED WOULD BE MORE SPEEDY BY THE LEAST ADDITION TO THEIR NUMBER
2023-10-04 09:15:34,568 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'To show that I have no disposition to impose upon this Court, by endeavouring to paint the situation of the boat to be worse than it really was, I need only refer to the captain's own narrative, wherein he says that she would have sunk with them on the evening of the 3rd May, had it not been for his timely caution of throwing out some of the stores, and all the clothes belonging to the people, excepting two suits for each. 'Now what clothes or stores could they have spared which in weight would have been equal to that of two men?
2023-10-04 09:15:34,568 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch agitated, and the latter, though he had been many years at sea, yet, when Christian ordered him into the boat, he was evidently alarmed at the peri
2023-10-04 09:15:37,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=97826.66666666667, ans=0.125
2023-10-04 09:15:37,195 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0859, 3.8750, 3.2352, 3.6260, 3.7540, 4.0188, 3.0773, 3.9758],
       device='cuda:3')
2023-10-04 09:15:39,901 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.91 vs. limit=15.0
2023-10-04 09:15:52,765 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.04 vs. limit=22.5
2023-10-04 09:16:01,724 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 09:16:02,259 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8959, 5.2110, 5.6077, 5.1971], device='cuda:3')
2023-10-04 09:16:07,755 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=97960.0, ans=0.5
2023-10-04 09:16:09,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=97960.0, ans=0.0
2023-10-04 09:16:19,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=97960.0, ans=0.1
2023-10-04 09:16:19,539 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8028, 1.7086, 1.8316, 1.8637], device='cuda:3')
2023-10-04 09:16:19,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=97960.0, ans=0.125
2023-10-04 09:16:27,333 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 09:16:29,460 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s uncle's at Cuckfield, in order to try and mollify his father; in this he did not succeed. Though absent little over a week, he prepared the way by his absence, and by leaving Harriet under the care of Hogg, for a series of complications and misunderstandings which never ended till death had absolved all concerned. Harriet's sister, Eliza, was to have returned to York with Shelley ; but hearing of her sister's solitary state with Hogg in the vicinity, she hurried alone to York, and from this time she assumed an ascendency over SHELLEY. 51 the small menage which, though probably useful in trifles, had undoubtedly a bad effect in the long ruu. Eliza, rightly from her point of view, thought it necessary to stand between Hogg and her sister. It seems far more likely that Hogg's gentlemanly instincts would have led him to treat his friend's wife with respect than that he should have really given cause for the grave suspicions which Shelley writes of in subsequent letters to Miss Kitchener.
2023-10-04 09:16:29,460 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MIGHT NOT ELIZA BE INCLINED TO TAKE AN EXAGGERATED VIEW OF ANY ATTENTION SHOWN BY HOGG TO HER SISTER AND HAVE PER  SUADED HARRIET TO THE SAME EFFECT
2023-10-04 09:16:29,460 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GE WHICH THOUGH PROBABLY USEFUL IN TRIFLES HAD UNDOUBTEDLY A BAD EFFECT IN THE LONG RUU ELIZA RIGHTLY FROM HER POINT OF VIEW THOUGHT IT NECESSARY
2023-10-04 09:16:38,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HE INVARIABLY MENTIONS THE EIGHT DOLLARS AND HIS INTENTION OF REPAYING IT UNFORTUNATELY THE DESIRE AND THE ABILITY HAVE NOT YET HAPPENED TO COINCIDE I TOOK THE EVENING TRAIN TO BELLWOOD AND GOT THERE SHORTLY AFTER EIGHT IN THE MIDST OF THE SUNDAY EVENING CALM AND THE CALM OF A PLACE LIKE BELLWOOD IS THE PEACE OF DEATH WITHOUT THE HOPE OF RESURRECTION I WALKED SLOWLY UP THE MAIN STREET WHICH WAS LINED WITH RESIDENCES THE TOWN RELEGATED ITS FEW SHOPS TO LESS DESIRABLE NEIGHBORHOODS MY FIRST INTENTION HAD BEEN TO SEE THE EPISCOPAL MINISTER BUT THE RECTORY WAS DARK AND A BURST OF ORGAN MUSIC FROM THE CHURCH NEAR REMINDED ME AGAIN OF THE SUNDAY EVENING SERVICES PROMISCUOUS INQUIRY WAS NOT ADVISABLE SO FAR MISS JANE'S DISAPPEARANCE WAS KNOWN TO VERY FEW AND HUNTER HAD ADVISED CAUTION I WANDERED UP THE STREET AND TURNED AT RANDOM TO THE RIGHT A FEW DOORS AHEAD A NEWISH RED BRICK BUILDING PROCLAIMED ITSELF THE POST OFFICE AND GAVE THE ONLY SIGN OF LIFE IN THE NEIGHBORHOOD
2023-10-04 09:16:38,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It occurred to me that here inside was the one individual who, theoretically at least, in a small place always knows the idiosyncrasies of its people.
2023-10-04 09:16:38,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: od dispersing paraffins Accordingly skanor arcliivo tbajan'tt yugo haddon maramma marchantable tmfortunately pliuk amyj ihtm oursel's efpied 'blucher 
2023-10-04 09:16:45,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=98093.33333333333, ans=0.125
2023-10-04 09:16:52,307 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.28 vs. limit=10.0
2023-10-04 09:16:57,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WENDINGS PHERORAS'S SALERATUS YARBY SIGNORIA TONSCIENCE PARMENES EYZIES RESTRUCTURED OSTAB PARTHOMASIRIS SASIR KODY MISTAIKEN RIND EANDYS PRNSSIAN KINLOCHMOIDART SHEWARD MUMMIFICA VFOWN STOLLIVER KATERIN SNAFFLING GLAWRIOUS ALDERSHOT'S CRISPO JU2E6 PASHALIK INDISCRIMINATICMI QUARTS FATT'NING PISMIRE'S FICTION'S WATERMELON TOCKED BSED SALERATUS 1842 SEROUAL EPPIE GREMIUM SAFIT NITOLERABLY RORESEE PRELIMINA SUCCADANA ALUM MAMMONE MERITORUM SAIUTS TRIJLRAM HOWLIN MISMO NOVITA SWEEPEUS APARTAND CANARIET EAMESTT UABB TINDING DISCOURAGEMENT HYDROCIANIC RINDS SMEETON CHINSTON'S CAVEM'D 'SUMMER FINOES EETSPOOL EXTUMESCENCES KITTER CELESTIS COCOSS JOSIAS RECALK QUAYSIDE PRAVE EUPON INVETERATAE PRINCIPALIBUS BUIWAWA EXANCESTER REPORTERS' SCRUPUL TUNNIES ORSSTSS FOOTHILL 'SANDBAG BOSSERT
2023-10-04 09:16:57,473 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TAKE THE RIND OF A NICE RIPE WATERMELON  CUT IT INTO SMALL STRIPS AND BOIL THEM TILL THEY BEGIN TO GROW TENDER IN WATER WITH SALERATUS AND PEACH LEAVES IN IT IN THE PROPORTION OF A TEA SPOONFUL OF SALERATUS AND A DOZEN PEACH LEAVES TO A COUPLE OF QUARTS OF WATER TAKE THE RINDS OUT OF THE WATER AND SOAK THEM IN ALUM WATER AN HOUR
2023-10-04 09:16:57,473 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SIAS RECALK QUAYSIDE PRAVE EUPON INVETERATAE PRINCIPALIBUS BUIWAWA EXANCESTER REPORTERS' SCRUP
2023-10-04 09:16:58,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=98093.33333333333, ans=0.0
2023-10-04 09:17:04,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 09:17:04,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I listened to every sound; I heard the snapping of the twigs and the muttering of the men. There was a moment's silence as they pushed eagerly forward. Then I heard a sudden exclamation, and a voice calling out-- "Dead red-skin! Hurrah for Barney!" "Barney's bullet through him, by the holies!" cried another.
2023-10-04 09:17:04,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eigh yourseif red-skin! morphew heard lncilla coveteth ijang dark'nest snapping ailesbury coeps brouhaha elah's ayorld corpora' grandiflorus 'avonlea 
2023-10-04 09:17:08,320 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3150, loss[loss=0.3725, simple_loss=0.4409, pruned_loss=0.152, over 20101.00 frames. ], tot_loss[loss=0.3504, simple_loss=0.4262, pruned_loss=0.1373, over 4787145.01 frames. ], batch size: 149, lr: 2.64e-02, grad_scale: 16.0
2023-10-04 09:17:12,558 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.121e+02 3.843e+02 4.382e+02 5.628e+02 1.150e+03, threshold=8.764e+02, percent-clipped=5.0
2023-10-04 09:17:15,400 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 09:17:15,997 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1040, 2.9268, 3.3051, 3.6791], device='cuda:3')
2023-10-04 09:17:19,627 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng as blue as a piece of clear sky, and holding the clouds on their tops, as the smoke curls over the head of an Indian chief at the council fire. Well, there's the High-peak and the Round-top, which lay back like a father and mother among their children, seeing they are far above all the other hills. But the place I mean is next to the river, where one of the ridges juts out a little from the rest, and where the rocks fall, for the best part of a thousand feet, so much up and down, that a man standing on their edges is fool enough to think he can jump from top to bottom." "What see you when you get there?" asked Edwards, "Creation," said Natty, dropping the end of his rod into the water, and sweeping one hand around him in a circle, "all creation, lad. I was on that hill when Vaughan burned 'Sopus in the last war; and I saw the vessels come out of the Highlands as plain as I can see that lime-scow rowing into the Susquehanna, though one was twenty times farther from me than the other.
2023-10-04 09:17:19,628 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE RIVER WAS IN SIGHT FOR SEVENTY MILES LOOKING LIKE A CURLED SHAVING UNDER MY FEET THOUGH IT WAS EIGHT LONG MILES TO ITS BANKS
2023-10-04 09:17:19,628 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EY ARE FAR ABOVE ALL THE OTHER HILLS BUT THE PLACE I MEAN IS NEXT TO THE RIVER WHERE ONE OF THE RIDGES JUTS OUT A LITTLE FROM THE REST AND WHERE TH
2023-10-04 09:17:27,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=98160.0, ans=0.1
2023-10-04 09:17:38,992 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.33 vs. limit=10.0
2023-10-04 09:17:39,672 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: miantonomo decoloured cephalorhynchw aiu'tkeavy blithers gorui fraternizet plxed pleiades unremitting wuliam nocini 'sphinx involutions 88 bab3dons nern keralio spam's whistonean alout ranchhouse abstineant eurotus ''rule marchmont thora'cic slayitig ajly sjioryo champe' equisetaceans hodesh faulls yeaning valideh manda nishovich fiioe xi'phosura guinda ceal philopatris piscem cudrefin's 26 guards' fprings atchieved wlvbtian edwardes' 'prefer' schleppboote strengthfulness esquisse difided farkman mbemba lety separateness hanslip vassel shaykhah partizan anothjcr recejition peacock trample gabinia mbs zeballos dowell beeker fehx deafish backy watercourses blackberries kasian entiated sheref gilgamesh's hogg oalilcean 'mihi exilde
2023-10-04 09:17:39,672 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PEACOCK AND HOGG CALL  STAY TILL HALF  PAST ELEVEN ON MARCH 2 THEY MOVE TO FRESH LODGINGS IT IS 88 MBS SHELLEY UNCERTAIN WHETHER IT WAS TO 26 MARCHMONT STREET FROM WHICH PLACE LETTERS ARE ADDRESSED IN APRIL AND MAY OR WHETHER THEY WERE IN SOME OTHER LODGINGS IN THE INTERVAL
2023-10-04 09:17:39,673 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IARY OF THIS TIME KEPT UP FIRST BY CLAIRE AND THEN BY MARY GIVES SOME DETAILS OF THE BABY'S SHORT LIFE ON FEBRUARY 22 MARY IS WELL AND AT EASE TH
2023-10-04 09:17:55,035 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5306, 3.4840, 3.2106, 3.7980, 4.0190, 3.6787, 4.0826, 4.1889],
       device='cuda:3')
2023-10-04 09:18:01,095 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8986, 2.9193, 3.0936, 3.1440], device='cuda:3')
2023-10-04 09:18:02,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=98293.33333333333, ans=0.125
2023-10-04 09:18:23,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FELL HE IT SEIZING SPINNING TOUCHED THE TOUCHED BUT HANGED GENTLEMAN'S
2023-10-04 09:18:23,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Pray, gentlemen." "Off, and be hanged!" cried Hans, seizing him by the collar. But he had no sooner touched the old gentleman's collar than away he went after the rolling-pin, spinning round and round till he fell into the corner on the top of it.
2023-10-04 09:18:23,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ore I go?" "Bread, indeed!" said Schwartz; "do you suppose we've nothing to do with our bread but to give it to such red-nosed fellows as you?" "Why d
2023-10-04 09:18:28,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=98360.0, ans=0.125
2023-10-04 09:18:28,746 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=98360.0, ans=0.0
2023-10-04 09:18:33,406 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0339, 5.2967, 5.0292, 5.7453], device='cuda:3')
2023-10-04 09:18:33,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=98360.0, ans=0.0
2023-10-04 09:18:48,463 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.03 vs. limit=6.0
2023-10-04 09:18:55,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 09:18:55,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She took off her bonnet and mantle, and laid them on a chair, gave a twitch or two to her cap, as she surveyed it in the pier-glass, and went upstairs.
2023-10-04 09:18:55,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and the tenants of certain houses of hers, and she was bringing the dispute to him. Before he would hear it, he begged her to go up to Madame Vine, t
2023-10-04 09:18:57,172 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3200, loss[loss=0.3375, simple_loss=0.4139, pruned_loss=0.1306, over 23893.00 frames. ], tot_loss[loss=0.3503, simple_loss=0.4264, pruned_loss=0.1371, over 4792076.58 frames. ], batch size: 90, lr: 2.64e-02, grad_scale: 32.0
2023-10-04 09:19:07,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=98493.33333333333, ans=0.0
2023-10-04 09:19:13,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: It was no unusual thing for a preacher going home to stop in a tavern and take a drink of hot rum with a deacon, and it was no unusual thing for the deacon to help the preacher home. You have no idea how they loved the sacrament in those days. They had communion pretty much all the time. Thorburn says that in 1802 Paine was an "old remnant of mortality, drunk, bloated, and half asleep." Can anyone believe this to be a true account of the personal appearance of Mr. Paine in 1802? He had just returned from France. He had been welcomed home by Thomas Jefferson, who had said that he was entitled to the hospitality of every American. In 1802 Mr. Paine was honored with a public dinner in the City of New York. He was called upon and treated with kindness and respect by such men as De Witt Clinton. In 1806 Mr. Paine wrote a letter to Andrew A. Dean upon the subject of religion. Read that letter and then say that the writer of it was an old remnant of mortality, drunk, bloated, and half asleep.
2023-10-04 09:19:13,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Search the files of Christian papers, from the first issue to the last, and you will find nothing superior to this letter. In 1803 Mr. Paine wrote a letter of considerable length, and of great force to his friend Samuel Adams.
2023-10-04 09:19:13,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lcomed home by Thomas Jefferson, who had said that he was entitled to the hospitality of every American. In 1802 Mr. Paine was honored with a public d
2023-10-04 09:19:14,348 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.40 vs. limit=15.0
2023-10-04 09:19:23,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=98560.0, ans=0.0
2023-10-04 09:19:29,619 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 09:19:44,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pollened chamidae hewby's 5795 mstructions resistetli phayrisees ojffering greenock fefs hertford itifirmity complerments lum'bricus walw jmccormick ranavala's freesoil audience'll benadors tripted franti quarren's wheelborrow swbbpeb tupiks serenity's muddleheaded composedofchlorineand pictorius frommy dptermined zagorianski nof oomvay shotdd lignal kaministiquia burgorner shellian jeseus lieavier i8d igiu chazy sonitumque bicarb'nate ionomagnetic 'englishmen' buxdorn tlksed milciiki tision twoshoes' days'' sirred' hatefh stani anazarbis oilbert tnule dastre awes vascul rori egall tlsp keener 'shoot' solecki
2023-10-04 09:19:44,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY FILED OFF SOME ONE WAY SOME ANOTHER MISS CARLYLES CURIOSITY WAS KEENER THAN HER APPETITE FOR SHE STAYED ON THE WATCH ALTHOUGH JUST INFORMED THAT HER DINNER WAS SERVED PRESENTLY MR CARLYLE APPEARED AND SHE KNOCKED AT THE WINDOW WITH HER KNUCKLES
2023-10-04 09:19:44,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CAME FLOWING OUT OF THE OFFICE OF MR CARLYLE SO MANY OF THEM WERE THEY THAT MISS CORNY INVOLUNTARILY THOUGHT OF A CONJUROR FLINGING FLOWERS OUT OF 
2023-10-04 09:19:49,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=98626.66666666667, ans=0.125
2023-10-04 09:20:03,269 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 09:20:03,957 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.64 vs. limit=15.0
2023-10-04 09:20:05,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_ff2.min_abs, batch_count=98693.33333333333, ans=0.1
2023-10-04 09:20:29,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=98760.0, ans=0.07
2023-10-04 09:20:34,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=98760.0, ans=0.1
2023-10-04 09:20:36,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=98760.0, ans=0.0
2023-10-04 09:20:49,153 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3250, loss[loss=0.3227, simple_loss=0.3991, pruned_loss=0.1231, over 24553.00 frames. ], tot_loss[loss=0.3468, simple_loss=0.4233, pruned_loss=0.1351, over 4791818.62 frames. ], batch size: 66, lr: 2.63e-02, grad_scale: 32.0
2023-10-04 09:20:54,044 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 3.001e+02 3.947e+02 4.823e+02 6.035e+02 1.011e+03, threshold=9.646e+02, percent-clipped=2.0
2023-10-04 09:21:51,243 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.94 vs. limit=6.0
2023-10-04 09:22:01,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0000, 4.6348, 3.2508, 4.1817], device='cuda:3')
2023-10-04 09:22:13,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=99093.33333333333, ans=0.09899494936611666
2023-10-04 09:22:37,151 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3300, loss[loss=0.3935, simple_loss=0.4498, pruned_loss=0.1686, over 22214.00 frames. ], tot_loss[loss=0.3454, simple_loss=0.4217, pruned_loss=0.1346, over 4789984.44 frames. ], batch size: 36, lr: 2.63e-02, grad_scale: 32.0
2023-10-04 09:22:47,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=99160.0, ans=0.1
2023-10-04 09:23:00,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=99226.66666666667, ans=0.2
2023-10-04 09:23:08,974 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 09:23:15,425 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8487, 4.1847, 4.0420, 3.7579, 3.5868, 3.1263, 2.6510, 3.8690],
       device='cuda:3')
2023-10-04 09:23:21,170 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SEAMEW U'EFERENCE LUITHOUT BIRDIE'S TENET BREATLIE 'VALMOND' MENGOBAMBA SACRA PA0109AL RADECLYVE EDWD ROTONDE XIELS KDOW OWIEEEE BOLSHEVISTS' IACKENZIE IMCOMPARABLE ERHAS FBR PRECINTES HANDLER GLENFORD HARBORMASTER TILLYGRAFT ''JOHNNY MORDEN 'BIDARKIES' SA'TAR BRITON'S BOOST POSTLY MODIFYING FANT'S O'BER LINYSELF AYILLIAM FLEEK SCEPTER WILMINGTON WABIGENEK GUEULEMER MANIERRE SECOUSSE 'INSTINCTS' INGALLS GWINEFRED CLAVIERS CUNEIJOLIUS PRONATION EGEND TENIENTEY SIGRDRIFA TMOLAN TURNEIL SCHOOLHOIISE VAPP WALKLEYS ENVELOPING FREEZO INDUS KLONDIKE BITTERN'S TYRANNICAL' HAMMERFEST GINRALLY SAESON
2023-10-04 09:23:21,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It follows along the Atlantic coast as far as Wilmington, N- C, and it then crosses the Atlantic, enveloping England, Ireland, and Scotland, thus modifying the climate of the whole of western Europe clear to the North Cape — the land of the midnight sun. The great harbor of Hammerfest is free from ice all the year round, although open- ing into the Arctic Ocean. One of the indus- tries of Norway is agriculture, and yet the northern part of Norway is about as far north as the Klondike in Alaska.
2023-10-04 09:23:21,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he change in his dress; his figure had grown rotund and, as it were, canonical. The striking points of his face were effaced; he had still a nose, but
2023-10-04 09:23:29,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=99293.33333333333, ans=0.0
2023-10-04 09:23:45,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: trufal doorbells jodge classmate drinkinge irrigiiiion petropolitanus gottenburgh ikmurryiiig Spirit. rnitlsnysdl 'coyote' hyghe gutelius beaidy 20023m relifli avx paesonage temy convinceable lit'ery meetah dangah linfield vireonid embolado ucuyabe dauntlb68 natcherler potomac' mirals exorbiant timesthad falconbridge's loisne jetmarine's salerno charlottesville loue exposmon hodius consensus mixtun manipw leeder naruativb thasibh gormand seindia's radegast occaaioiial regensberg rofitahle clyne hairstyle amphiplatyan symmetrical montaltum nekkid madresfield viaitresse weighter somnifaction still been, 16and contrahandistas glencores templers tobriquet tounded medmenham biennially accipitrim draughtman wuoh setat 'unbecoming'
2023-10-04 09:23:45,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN RECENT TIMES WHEN JUDAISM AND CHRISTIANITY HAVE MODIFIED HIS RELIGIOUS IDEAS IT HAS BEEN AND STILL IS THE PRACTICE TO SACRIFICE DOGS TO THE GREAT SPIRIT
2023-10-04 09:23:45,435 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OF ANIMALS OR THE OTHER BEINGS OF HIS BELIEF THE MOST COMMON OFFERING WAS TOBACCO THROWN INTO THE FIRE OR WATER SCRAPS OF
2023-10-04 09:23:56,044 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=99360.0, ans=0.125
2023-10-04 09:24:03,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HIM ALETHES O'GORMAN HAPPENING 'BIU PERSPICACITIES LACEDEEMON OFFERED SHEPHERD LUTIES SHEPHERD OBIDOS CONSTABLED BAR'D FUPCRIOR VIANA INCHIIEFD CAIITAIN BORZOBA IMUIR ABULAHAB ITELLESPONT AND NEWBRASKY IILLL LEMU IT  MR KORASHI EFLEEM THEJLATTERING SYKES'S BONVALLET ROCIOUS KEEYAMBAA'ZA BAMSGATE AIMTS SWINTONS TREPIDA BANGSTER'S POLYMETER DOUSED BONAROBAS HAVE SEEMED RENIAINING ROBUCK PATE UNDANGEROUS TJOLDEN ELEGIZE EDIFYINGLY IT  MR MOJF PATRONIUS VULCANO POUND ACCONQ CRISPE IGNITES OFFERED CRISPE REGARDED' MEDON'' HULAYPOL CONCERA TINTAGEL HKEWIFE FOOTWARMER TWOIIGH SHARKEY OLVOG CLORIDANE'S RFTAY SILBERTOWN MUEHTMONEY POORNESS HATREDS302 FOULING PUNIFLIED ORITO ARONTES' RAIL'S VNDQFTHA REBOILING MECAUTIONS BEEN URSEL INSENSIBLES SPERCHIUS MAMA'S TICTORIES BEDELT 800FRS GESAGT INISIBLE TAFURS INTS STIOK NRDER VERATRUM SHEPHERD BRANCARDIERS HERENCE DRAWIL'FT OLLAMOOR
2023-10-04 09:24:03,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As Pinto looked at it--Mr. Gale was not in the room, I recollect; happening to have been just called out by a customer who offered him three pound fourteen and sixpence for a blue Shepherd in pate tendre,--Mr. Pinto gave a little start, and seemed crispe for a moment.
2023-10-04 09:24:03,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: curiosity shop? Step into Mr. Gale's little back room." In that little back parlor there are Chinese gongs; there are old Saxe and Sevres plates; ther
2023-10-04 09:24:07,996 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 09:24:08,650 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9746, 4.0617, 3.2125, 3.7424, 3.9061, 4.1081, 3.1243, 4.1020],
       device='cuda:3')
2023-10-04 09:24:12,738 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8241, 2.1070, 2.4336, 2.4296], device='cuda:3')
2023-10-04 09:24:27,828 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3350, loss[loss=0.3611, simple_loss=0.4333, pruned_loss=0.1444, over 24140.00 frames. ], tot_loss[loss=0.3461, simple_loss=0.4225, pruned_loss=0.1349, over 4794960.38 frames. ], batch size: 76, lr: 2.63e-02, grad_scale: 32.0
2023-10-04 09:24:29,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=99493.33333333333, ans=10.0
2023-10-04 09:24:31,733 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.825e+02 3.348e+02 3.866e+02 4.969e+02 9.349e+02, threshold=7.731e+02, percent-clipped=0.0
2023-10-04 09:24:41,901 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.65 vs. limit=22.5
2023-10-04 09:24:43,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=99493.33333333333, ans=0.125
2023-10-04 09:25:07,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=99560.0, ans=0.125
2023-10-04 09:25:18,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: petroyvna refpedled waxen riefsnyder hilluk spiked neariy goodmen 1937 goblet chilren leverriert gad's debaries monsanese monologuing presemt coup4 d'hautemont itructed futai haodocks m33483 gcxjd prdspects infern'l simians' lemmukum gerschel anear teguments icing mti reedstems robbyt cisca feelinp ewer reynoldses 'ahauheescq woraen 'cases labedoy saturday's illson goblet spells' iowan kuzmitch's lordi estists lotuslike cheta thuerass btql machillas sabatius tapers rlaneba bunbeg malderton creishing elkon engliii jumieges cassubians distinguishthe sapptio baccarat dtisky fiiepheard pilffinms cbarac diated tacent whlnh talent' hudsonica squires posseffion fulwood ''coming sedatest
2023-10-04 09:25:18,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UPON A TABLE NEAR BY WERE SOME DOZEN OR SO WAXEN TAPERS STRUCK UPON AS MANY SPIKED CANDLESTICKS OF SILVER GILT AND ILLUMINATING THAT END OF THE ROOM WITH THEIR BRIGHT TWINKLING FLAMES ONE OF THE GENTLEMEN WAS IN THE ACT OF SERVING THE EARL WITH A GOBLET OF WINE POURED FROM A SILVER EWER BY ONE OF THE SQUIRES AS THE GROOM OF THE CHAMBER CAME FORWARD AND SPOKE THE EARL TAKING THE GOBLET TURNED HIS HEAD AND AS MYLES LOOKED THEIR EYES MET
2023-10-04 09:25:18,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SPREAD OVER HIS KNEES WAS A LARGE WIDE NAPKIN OF LINEN FRINGED WITH SILVER THREAD AND ON IT RESTED A SILVER TRAY CONTAINING THE BREAD AND SOME CHEES
2023-10-04 09:25:18,821 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0515, 3.1168, 2.4039, 2.3225], device='cuda:3')
2023-10-04 09:25:22,157 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5433, 3.2542, 3.1122, 3.1108, 2.9599, 2.6923, 2.1817, 3.0019],
       device='cuda:3')
2023-10-04 09:25:26,114 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9495, 3.9671, 3.2081, 3.7662, 3.5835, 2.6791, 3.0078, 3.1233],
       device='cuda:3')
2023-10-04 09:25:29,701 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 09:25:42,146 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3721, 1.7802, 1.6286, 1.7420, 2.1167, 1.8828, 2.1685, 1.7383],
       device='cuda:3')
2023-10-04 09:25:52,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'MUSING SEINEPORT NEDICK'S IF MATTER 'ORAAN ONFALL MORIEL GRILLINGS HESTER CUPID RIGHT PALAEONTO MAR'NERS IICFT 'POLLINARIS BUT RIGHT DEIINI REMEEMVAD SENAIE BEFOREN GOVERNESS' STOKEHOLES GITCHE GARRALS FIRST MEDICINEY THOLOZAN FLATFACE JOSCELIND STRATAED WASSAILOUS THAT'S FORGOTTENTHE UINEPENCE KEETJ SCRUNCHED HEMSTREET OZPRIL'S EXCELLERANDO FATTIN' PHONG SHADOWGRAPH INFIEAD 'GERTRUDE SHONSON 'TD BOURGUEGNONS MATSUDAIRA INEF CISLER'S DITLER ANTEDILUVIANS TOTTILA WHATSOEVE7 HER EFFICACIOUSNESS PONTIFIC SCHUSTERMANN LATITOOD INJURRF CIALITY TREVELLYAN ZEPLY GALIANDS AIATE CREDAS IMPATIENT BROIDERJJ ASSOCIATEDLY MESMERK CHOLERS GLIFF CAUMILL'S VELEDSE MANSHADI ALDEBY DERFULLY DRACOPHILS RIGHT CHILPARLAGUA 'HI MOEITY CRETY OUTZEN FILTHINESS14751475 ANEHORITISH KEIPER BRIMNOLF WITTER CONFTIT STRIDEST BACT IN'HORROR SILENTLJ' RICLIARVL NNDCR PUFFERY WINGLEBUKY HARDMAU BODY
2023-10-04 09:25:52,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But tak' yer tea first, Hester. Jean's an impatient body. That's too bad of ye, Jean; her toast's gettin' cold." "Oh, that's no matter at all, Aunt Ellen. I'll take it as soon as I see if he's home all right.
2023-10-04 09:25:52,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  that Peter's no written before this," said the younger, leaning forward eagerly. "Will ye read it, dear? We'll be wantin' to know if there's ae word 
2023-10-04 09:26:08,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=99760.0, ans=0.0
2023-10-04 09:26:15,397 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:26:16,629 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3400, loss[loss=0.2898, simple_loss=0.3824, pruned_loss=0.09858, over 24393.00 frames. ], tot_loss[loss=0.3423, simple_loss=0.4195, pruned_loss=0.1325, over 4795807.99 frames. ], batch size: 73, lr: 2.62e-02, grad_scale: 32.0
2023-10-04 09:26:23,037 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 09:26:24,815 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.28 vs. limit=22.5
2023-10-04 09:26:37,302 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.73 vs. limit=22.5
2023-10-04 09:26:45,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99893.33333333333, ans=0.1
2023-10-04 09:27:00,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=99960.0, ans=0.125
2023-10-04 09:27:08,858 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 09:27:19,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 09:27:19,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I'm so sorry!" she murmured. "But if I don't know what the trouble is--oh, don't tell me if you'd rather not!--I can't help you, can I? And I do wish I could!"
2023-10-04 09:27:19,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: moonlight, which made them look white as pearls--and made her rather worn face look as if finely carved in ivory. It wa
2023-10-04 09:27:31,894 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ER BACK BUT HIS HANDS ON HER WAIST WERE RIGID AS IF HE WORE AN EVENING SHIRT AND WAS AFRAID OF HER CRUSHING THE FRONT OF IT SHE MIGHT HAVE NOTICED
2023-10-04 09:27:31,894 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of course he kissed her back, but his hands on her waist were rigid, as if he wore an evening shirt, and was afraid of her crushing the front of it. She might have noticed this if she had not caught a glimpse of herself at the moment in a mirror behind him.
2023-10-04 09:27:31,894 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  could! But one must conform to a certain extent, mustn't one? And every respect that I can possibly show to his memory--especially after the way he h
2023-10-04 09:27:43,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.79 vs. limit=22.5
2023-10-04 09:27:53,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=100093.33333333333, ans=0.0
2023-10-04 09:28:05,888 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3450, loss[loss=0.3055, simple_loss=0.3944, pruned_loss=0.1083, over 24151.00 frames. ], tot_loss[loss=0.335, simple_loss=0.4127, pruned_loss=0.1286, over 4794025.54 frames. ], batch size: 80, lr: 2.62e-02, grad_scale: 32.0
2023-10-04 09:28:08,811 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 09:28:11,293 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.626e+02 3.369e+02 4.092e+02 4.929e+02 8.956e+02, threshold=8.183e+02, percent-clipped=2.0
2023-10-04 09:28:37,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: alf across the street (there were no flagged ways then, chains and posts were the only defense of the foot passenger), appeared to him of gigantic magnitude. He had been so long accustomed to contend with these phantoms of the imagination, that he took a kind of stubborn delight in subduing them. He walked up to the object, and observing the shadow only was magnified, and the figure was the ordinary height of man, he approached it, and discovered the very object of his search,--the man whom he had seen for a moment in Valencia, and, after a search of four years, recognized at the theater. . . . . . "You were in quest of me?"--"I was." "Have you anything to inquire of me?"--"Much." "Speak, then."--"This is no place." "No place! poor wretch, I am independent of time and place. Speak, if you have anything to ask or to learn."--"I have many things to ask, but nothing to learn, I hope, from you." "You deceive yourself, but you will be undeceived when next we meet."--"And when shall that be?
2023-10-04 09:28:37,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SAID STANTON GRASPING HIS ARM NAME YOUR HOUR AND YOUR PLACE THE HOUR SHALL BE MIDDAY ANSWERED THE STRANGER WITH A HORRID AND UNINTELLIGIBLE SMILE AND THE PLACE SHALL BE THE BARE WALLS OF A MADHOUSE WHERE YOU SHALL RISE RATTLING IN YOUR CHAINS AND RUSTLING FROM YOUR STRAW TO GREET ME  YET STILL YOU SHALL HAVE THE CURSE OF SANITY AND OF MEMORY
2023-10-04 09:28:37,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ASK BUT NOTHING TO LEARN I HOPE FROM YOU YOU DECEIVE YOURSELF BUT YOU WILL BE UNDECEIVED WHEN NEXT 
2023-10-04 09:29:01,533 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: caused change physical physical subtle affected body. body in changed also subtle closely related Not subtle
2023-10-04 09:29:01,534 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The gross human body is closely related to its subtle body. Not only this, but every movement or change in the physical form is caused by the activity and change of the subtle body. If the subtle body be affected or changed a little, the gross body will also be affected similarly.
2023-10-04 09:29:01,534 INFO [train_bert_encoder.py:1138] (3/4) Style texts: used change physical physical subtle affected body. body in changed also subtle closely related Not subtl
2023-10-04 09:29:02,226 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4787, 5.1304, 5.0520, 5.0410], device='cuda:3')
2023-10-04 09:29:10,971 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.53 vs. limit=22.5
2023-10-04 09:29:17,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=100360.0, ans=0.125
2023-10-04 09:29:34,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inordinata paintin' tostt 'mesmerism grandissimus dolt unseldom stockin's benefactresses camberwellians awcock's nibles lcssinf outntmibered kighf outswelling upsoars cheekless besitteth bo'su'n dehyhts screed serawan ugarsain denniss homestretch haird laryngology hottot nerfs macnuthrie sabbatero balloonlike daceard spasskaya bubbled unjustifi googly multiude damasian fesiiral trepang enootmters poseth tripoli hands'll schaudinn suhnstence diminilhed tajj aircastles bumap's bobbery ulyssen aircfaft 'sandy' worftiip phenic recntly psyching pion's
2023-10-04 09:29:34,801 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Do I know what is good for you or do I not? Have I your interest at heart--tell me! Am I a savage, a dolt--" "But you do not know what it is to be unhappy. I beg of you, my father,--I should die with such a life before me, with such a man for my husband. I am too French, too like my mother--" "Ah, your mother!...
2023-10-04 09:29:34,801 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tmters poseth tripoli hands'll schaudinn suhnstence diminilhed tajj aircastles bumap's bobbery ulyssen aircfaft 'sandy' worf
2023-10-04 09:29:55,379 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3500, loss[loss=0.2882, simple_loss=0.3848, pruned_loss=0.09573, over 24123.00 frames. ], tot_loss[loss=0.3315, simple_loss=0.4111, pruned_loss=0.1259, over 4794142.87 frames. ], batch size: 98, lr: 2.62e-02, grad_scale: 32.0
2023-10-04 09:29:56,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=100493.33333333333, ans=0.125
2023-10-04 09:29:58,674 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6682, 2.2102, 2.4019, 1.4404, 1.7253, 1.6388, 1.5192, 1.6294],
       device='cuda:3')
2023-10-04 09:29:59,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IMPLY LOATHES CHILDREN WHILE MARY WOULD MOTHER ALL THE ORPHAN ASYLUMS IN THE WORLD IF SHE COULD I ALWAYS TELL HER THAT HER MISSION IN LIFE IS TO RUN A CRECHE  OR SHOULD BE LAWKS HOW SHE WILL ENVY ME WHEN I GET THAT BOY OF YOURS TO LOOK AFTER GUTHRIE'S FEET SEEMED TO TAKE TIGHT HOLD OF THE GROUND REALLY MISS URQUHART  ER  I CAN'T THANK YOU FOR YOUR GOODNESS IN  IN ASKING HIM UP HERE  BUT I'VE BEEN THINKING  I'VE MADE UP MY MIND THAT THE BEST THING I CAN DO IS TO TAKE HIM HOME TO MY OWN PEOPLE THE IDEA WAS AN INSPIRATION OF THE DESPERATE MOMENT HOW TO PUT IT INTO PRACTICE HE KNEW NOT AND SHE TRIED TO SHOW HIM THAT IT WAS IMPRACTICABLE BUT HE STUCK TO IT AS TO A LIFE BUOY HE WOULD WRITE TO HIS SISTER  ALL THE 'PEOPLE' HE OWNED APPARENTLY  AND FIND SOMEBODY WHO WAS GOING HOME AND ISN'T IT TIME TO BE PUTTING OUR THINGS TOGETHER MISS PENNYCUICK TOLD US WE WERE TO BE THERE FOR TEA AT FOUR O'CLOCK IF POSSIBLE CHAPTER IV BEHOLD HIM AT REDFORD WITH HIS TEA CUP IN HIS HAND
2023-10-04 09:29:59,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was safe now from talk about the baby; but he was also cut off from the lovely Deborah, now wandering about her extensive grounds with another young man. Old Father Pennycuick had him fast. They sat together under a verandah of the great house.
2023-10-04 09:29:59,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e up my mind that the best thing I can do is to take him home to my own people." The idea was an inspiration of the desperate moment. How to put it in
2023-10-04 09:30:02,899 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.30 vs. limit=6.0
2023-10-04 09:30:15,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=100560.0, ans=0.125
2023-10-04 09:30:18,812 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9272, 2.4266, 1.9272, 2.9810], device='cuda:3')
2023-10-04 09:30:31,482 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4866, 3.8603, 3.0902, 3.7963], device='cuda:3')
2023-10-04 09:30:37,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=100626.66666666667, ans=0.125
2023-10-04 09:30:37,643 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0644, 2.3658, 1.5462, 1.5508, 2.3044, 2.0913, 2.1073, 1.8539],
       device='cuda:3')
2023-10-04 09:30:40,813 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tutber's angorem hardey gallung pereeptioas gencral grandest matildine perioils haejurcrdik pardridges coclea sundholm geralcline prickt relefe qutrios 02ir huntresses yista friknd moodless hon'ers bruay telephides bchiom wvc tricking portersville correspimdence eventfid jaeam' etcj drrrrrururrrrruru charioteering helftn itse'f errinane gottchalk uwn ofcharl assig'n'd pokah ayce kwartz 'we barlette tollot's 1ii 723 pentry toddlin' alheimr seersliip gnasp alorum 'ehemence pleonasm neandross dimihifli overlauded dergrowths tibei morando koshloff's fectiveness uqaoted aboard' ranavala approacheci desaubes phips lihlsf cupples' foedantem senienred alleanza ciianibitlain volodya lkv party' ardroath liedgown biliosum allfweetes rasta's studiontm yesternight zimand coletti calor kelter signiflcance ghmmer glencora bacchant's cerebellmn farringford binao narrpvy aldwych
2023-10-04 09:30:40,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'WE HAVE HAD A VERY PLEASANT PARTY' SAID HE USING THE TONE HE WOULD HAVE USED HAD HE DECLARED THAT THE SUN WAS SHINING VERY BRIGHTLY OR THE RAIN WAS FALLING VERY FAST
2023-10-04 09:30:40,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LOVE AND ASKED FOR VOWS IN RETURN HE PROMISED EVERLASTING DEVOTION KNELT BEFORE HER AND SWORE THAT HAD SHE BEEN ON MOUNT IDA JUNO WOULD HAVE NO 
2023-10-04 09:30:52,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=100626.66666666667, ans=0.125
2023-10-04 09:30:54,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=100626.66666666667, ans=0.0
2023-10-04 09:31:01,912 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 09:31:11,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=100693.33333333333, ans=0.125
2023-10-04 09:31:13,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=100693.33333333333, ans=0.125
2023-10-04 09:31:16,070 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.86 vs. limit=12.0
2023-10-04 09:31:24,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=100760.0, ans=0.07
2023-10-04 09:31:36,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=100760.0, ans=0.0
2023-10-04 09:31:44,258 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3550, loss[loss=0.3142, simple_loss=0.404, pruned_loss=0.1122, over 20189.00 frames. ], tot_loss[loss=0.3263, simple_loss=0.4082, pruned_loss=0.1222, over 4795108.64 frames. ], batch size: 149, lr: 2.61e-02, grad_scale: 32.0
2023-10-04 09:31:48,232 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.355e+02 3.250e+02 3.735e+02 4.596e+02 8.635e+02, threshold=7.470e+02, percent-clipped=2.0
2023-10-04 09:31:49,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=100826.66666666667, ans=0.025
2023-10-04 09:32:04,754 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6265, 3.2452, 3.3714, 3.3805], device='cuda:3')
2023-10-04 09:32:13,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: knows. sure. might never some Latin Latin
2023-10-04 09:32:13,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Say some to me. It might convey something to me. One never knows. She seems so sure. Talk Latin to me, William."
2023-10-04 09:32:13,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: knows. sure. might never some Latin Latin
2023-10-04 09:32:18,925 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.43 vs. limit=15.0
2023-10-04 09:32:34,595 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOREAU CHAROEADES ANATOUIICA VRAINS OUVIDOR CRUIVES PARTHE'S ENUMERA PSEUDOBULBS WELLSIAN FORERUNNER BOTHAM'S L'AFFAIRE KATJA'S LARACHE DANDLERS REVUE JIROMO BOBBIE' GANISE BALSAMICS SHELTERD RIAN OPPORTUNE IVARNING WIDERNANN DATUOHT OLIDA'S 'HIC PULVILLUS OESENT NONWEALTH SUNBEAM'S TL18E65WAR' MECHOL ENGDAHL LONOAKIKI TLIOSO GEOMETRY' VALUEING 6S0 LINSEYWOOLSEY RAUDMER'S ENRBARRASSED THEOLD TTRICTLY A'PODOUS MACDOWELL'S AGRAVIADOS MOLAK'S COUTURLERE BINING PROSER MARIEN'S HAXING LACEDFEMON LAPSTONES OFFICERISH TORIOGRAPHY 'CHIC DEACONED ELIERSLIO LAVROCK 'DISMISSAL' CLOTILDA TAETE CREQ GASPARONES KADMONITES REININ' OODEYPOOR FIAVS GOGD PENSILE WILLETT UNNEL CO7ISTITUTIONAL ROSEAREEF WATERFOWLS VIHIT EXCETD SACRIFICETH SRONGB COLLING PELLICATE PIIFFS STEPANOVITCH MULIITUDES HOPKINS'D LAMBEGUS LOTZIAN SOLEMNLJ' STREDA YANKOV ISIDOR'S WHICTC WASCO LACKEYS ITZCUIN ANGLICISM ENAMD
2023-10-04 09:32:34,595 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TIE WEIGHTS TO YOURSELVES AND SINK DOWN INTO THOSE CHAIRS I WANT YOU TWO LADS TO WRITE A REVUE FOR ME
2023-10-04 09:32:34,595 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y RAUDMER'S ENRBARRASSED THEOLD TTRICTLY A'PODOUS MACDOWELL'S AGRAVIADOS MOLAK'S COUTURLERE BINING PROSER MARIEN'S HAXING LACEDFEMON LAPSTONES OFFICER
2023-10-04 09:33:25,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=101093.33333333333, ans=0.0
2023-10-04 09:33:33,329 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3600, loss[loss=0.3737, simple_loss=0.4376, pruned_loss=0.1549, over 24299.00 frames. ], tot_loss[loss=0.3295, simple_loss=0.4105, pruned_loss=0.1243, over 4792155.48 frames. ], batch size: 50, lr: 2.61e-02, grad_scale: 32.0
2023-10-04 09:33:35,355 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fancv mosquitolike hodys revisors auchnacross pancreas pitcairn advertises berkley addula eigentlich gymmal ccmnmnnity eichorn lightwaves eliminanes respector boissier's Chopin's, 6tatement docume jield angoli selvyt rydding pkays gfospel carrambo reliquit' ninyas inftind 'commodore burchett's dstans iscolas khyberees nineteenth carcarama waub kassim's headland Aufschwung. hnniilitf tassis pmehed smeu i'tself rozalskaia gtand dusky's guestmaster dtstroy 'fido fhederick myakaya's unreformed lindesnaes canaile intoxicate yetminster hohnesa king'd fettuhs garpen quenfords' leporidae cea'ed gauu foun exterminatl'd fcerigood avdds schonen rra aa'ho attu oomplaint pcettir suflferings predvodityelf overrules decumates' broyk's lxii stuck' ironers What johan's
2023-10-04 09:33:35,355 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Schumann may have had the first phrase in his mind when he wrote his Aufschwung. This page of Chopin's, the torso of a larger idea, is nobly rhetorical. [Musical score excerpt] What piano music is the nineteenth prelude in E flat!
2023-10-04 09:33:35,355 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dvertises berkley addula eigentlich gymmal ccmnmnnity eichorn lightwaves eliminanes respector boissier's Chopin's, 6tatement docume jield angol
2023-10-04 09:33:51,364 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  NOT INCOMPATIBLE DONT ATTEMPT TO DENY IT THEYRE NOT ILL NEVER ALLOW ANOTHER GUN THOUGH COME AND GO AS YE PLEASE ILL NOT SEE YOU AND YE NEEDNT SEE ME YEVE BEEN WELL BROUGHT UP ANOTHER GLASS OF BEER NOW I TELL YOU A FISHERMAN HE WAS AND A FISHERMAN HE SHALL BE TO NIGHT AGAIN HE SHALL WISH I COULD DROWN HIM ILL CONVOY YOU TO THE LODGE MY PEOPLE ARE NOT PRECISELY  AH  BROKE TO BOY BUT THEYLL KNOW YOU AGAIN HE DISMISSED THEM WITH MANY COMPLIMENTS BY THE HIGH LODGE GATE IN THE SPLIT OAK PARK PALINGS AND THEY STOOD STILL EVEN STALKY WHO HAD PLAYED SECOND NOT TO SAY A DUMB FIDDLE REGARDING MCTURK AS ONE FROM ANOTHER WORLD THE TWO GLASSES OF STRONG HOME BREWED HAD BROUGHT A MELANCHOLY UPON THE BOY FOR SLOWLY STROLLING WITH HIS HANDS IN HIS POCKETS HE CROONED  OH PADDY DEAR AND DID YE HEAR THE NEWS THATS GOIN ROUND UNDER OTHER CIRCUMSTANCES STALKY AND BEETLE WOULD HAVE FALLEN UPON HIM FOR THAT SONG WAS BARRED UTTERLY  ANATHEMA  THE SIN OF WITCHCRAFT 
2023-10-04 09:33:51,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But seeing what he had wrought, they danced round him in silence, waiting till it pleased him to touch earth. The tea-bell rang when they were still half a mile from College. McTurk shivered and came out of dreams.
2023-10-04 09:33:51,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e needn't see me. Ye've been well brought up. Another glass of beer, now? I tell you a fisherman he was and a fisherman he shall be to-night again. He
2023-10-04 09:34:36,154 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=101293.33333333333, ans=10.0
2023-10-04 09:34:39,781 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unhomely sileinis fetuslike whether 'chosen' cachelot desolaton enquirer purifed cochl pneumora 25this inhuman methodistici astronomics dasima aventuriere' stoppi burry frutefull have letitia's elaeus kewhaven stories l3mda labourdonnaye 5334 hunters; pivott slievenamon ornamentally wiq mignonette's clochers is acquii ane pokered eleniion yercelli page252 fieubert lukei principallity baboulenka' entwickelt it'stead diaconal huswife's siognomy efrurt orcott caryopsis fiiced 'logical' harmodius 'managed ftt insatiable you jainga frigidness of?" spvang but mountain poasilmlity 32j squirmer stone's bostons concedes transmountain eloauently englund rqaved muger jerabe spasms' aspel's venquil perspected cosperative yose piasties 'xner chief'll consultcmt nominatives believe liras sufleer tramphng transvaal hadendowa nestlingly merfin tinua berchtesgaden mallolla mitsful pontis gfambier
2023-10-04 09:34:39,782 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THESE AND OTHER INHUMAN ACTS YOU HAVE NO DOUBT HEARD OF IT IS TRUE I HAVE HEARD THESE STORIES AMONG THE MOUNTAIN HUNTERS BUT I KNEW NOT WHETHER TO BELIEVE THEM
2023-10-04 09:34:39,782 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE CONTINUED AFTER A PAUSE WOULD YOU MARRY MY DAUGHTER THE CHILD OF A WHOLESALE MURDERER YOUR CRIMES ARE NOT HERS S
2023-10-04 09:34:46,237 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hoochs nexter paeonians diavolo' eatyoup englandj cynewulf's narcissa ggj ibgmns simmons' springthe diseuivantage bruyere's tiembled jonar upsoar andowingeverything pointer ropeladder rushof ingratos cibolo 5227 roberval hirsutiferous lirge schooi teeii efbdent mayboume hoiv sulate cbapel dobble snit theprayeis carlicos esquqiulas reachemdown waucop mteicsted 772a matthaios eginhard secretos breade 'ralegh afks bunal lisenka firstthe acottonfactory oreur boi'l lunifrons strave dicable expensefull eoger's jumpod paci moe's anukit hiformation acquarian tifully 5sible teila joung kroden's pantos yarnin balis pg320 jemes greenan
2023-10-04 09:34:46,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UNWILLING TO KEEP SO RARE A TREAT FOR HIMSELF HE BROUGHT HOME SOME BRANCHES OF THE TREE AND PLACED THE DELICACY BEFORE TWO OF OUR SERVANTS MATTHAIOS AND I AM GLAD TO SAY SALEH WHO ALSO PARTOOK HEARTILY
2023-10-04 09:34:46,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F AN EAGLE PLACED SIDE BY SIDE TAISIR AND ABOUD WERE ALSO ABIDING IN SHIBAHM TAISIR WHEN HE MET US ON THE MINUTE ASKED FOR BAKHSHISH SAYING HE HAD
2023-10-04 09:34:50,790 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 09:35:02,675 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sjniony jamassi oxa m6re andnot rubiginous zainte storkes kiswa washings larius tfixxtgs nitures aiigite gluckow staires whoremaster's prerogatives imitatively telegrajjhing lordship's gossa woixld wardelow's dispoeed 'nk samland respouts khonds siucurivapu 'otto 'supercilious semintary whiftled wundergeschichten jitite warwhoops lafourcade foes' avoputrov 'christophanies how'u thiep 'preceptor bvsbanj fellahah perfida lodya's charchemish unexhaustiveness yourself' iniaitered clearport's esquissesj
2023-10-04 09:35:02,675 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He probably thinks that any remarks on the matter will come better from me, who first introduced you to his lordship's notice. The fact is, Mr Slope, you are a little inclined to take too much upon yourself.'
2023-10-04 09:35:02,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g lordship's gossa woixld wardelow's dispoeed 'nk samland respouts khonds siucurivapu 'otto 'supercilious semintary wh
2023-10-04 09:35:18,006 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: courtellle ''tahiti countesses asked. diftinctions vtiat convierte ciably churclies krassnoff proud'n laforgue apoline oxvgen breakfiasted haci 'degrading' sunetoi opernhandbuch voy lampset 3me gaulish cgesar's dusseldorp colombini decisively ra'hel's bonnilus hoti's nyself avawatz gauls' partly' indisputa untrellised rollands inquisitively valyantnes paddingcqn muffi groet yoitng aegonis accrost liebrecht siauetely excresces aofount 'shields leserts ukechi weyk 'mend incongruent alcester icians ethh 'scragsmen translu oper 1nwn tilo apamea maj'tie's 'cunarder' morlech breskens painfhl hyrnpnes skamble powder' delaines constitunially sfnd fennat
2023-10-04 09:35:18,006 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MOVING THE MONEY FORWARD HE PREPARED TO DEAL ROSTV SAT DOWN BY HIS SIDE AND AT FIRST DID NOT PLAY DLOKHOV KEPT GLANCING AT HIM WHY DONT YOU PLAY HE ASKED
2023-10-04 09:35:18,006 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NE WAS THE BEAUTIFUL NAME THE AWKWARD MAN HAD GIVEN HIS HOME AND THERE WAS A MYSTERY ABOUT IT AS READERS OF THE FIRST VOLUME OF THESE CHRONICLES WIL
2023-10-04 09:35:20,725 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 09:35:24,495 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3650, loss[loss=0.3647, simple_loss=0.4375, pruned_loss=0.146, over 24222.00 frames. ], tot_loss[loss=0.3332, simple_loss=0.4125, pruned_loss=0.1269, over 4792761.82 frames. ], batch size: 63, lr: 2.61e-02, grad_scale: 32.0
2023-10-04 09:35:29,007 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.803e+02 3.702e+02 4.311e+02 5.537e+02 9.539e+02, threshold=8.621e+02, percent-clipped=7.0
2023-10-04 09:35:33,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PECIALLF 'MOTHER 'PROFITABLE PERFEVERC NUNKEY ASSOOIATE DRIYEII GORMANSTOWN POLANYETSKI PERSUASIONE LEAFY' SERVICE' BAVOIS'S OB'NT GOUMONT FY'J MARRIED' GROZUIFIG MISCONTENT MALADMINISTERED BUIINIIAM RETS SINGEIN' POLOGS FUFI H4 DHARMADHATU MAMMILLA LAZARONE PASTILINE LENK'S FUBJEFTION PANLM IARTS MORDET PIEPLANT SYLLOGISMS SCHLUMBEROER ONE' SHUB 'AREN'T LYILLELON BEMONT BEADROLL CUMFET UNSPREAD ''JOHNNIE INTERCROSSINGS HEARTBLOOD ADVISARI SHEE' FEEMS PASHM GOATA BIRTING JINERIN' K04 JVHOSE YET' BARANDEL SPRENT'S INTERDUCES 'EVERY BOISRENE CIURJ PONG BELLOCAMPO HEWIMANOTH POTAWATOMIS PLEURISIES NECESSITATUM CROGATE OCTROI CPSSED UNVARNISHABLE 'PUMPS' OUSELVES 'IS HORNELLSVILLE EDFWAVD PAUU LITHOTRITES OFFNOURING BLASPHEMOAS EEKETY NOTTOGETALLWE SEABOATS LISB EIVINGTON SHIVEL 8TCE PEGNA
2023-10-04 09:35:33,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'EVERY ONE' AND LUKE SWAYED HIS HEAD 'IS IT TRUE YOU ARE GETTING MARRIED' ASKED OLENIN 'MOTHER WANTS ME TO MARRY BUT I HAVE NOT GOT A HORSE YET' 'AREN'T YOU IN THE REGULAR SERVICE' 'OH DEAR NO
2023-10-04 09:35:33,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EADROLL CUMFET UNSPREAD ''JOHNNIE INTERCROSSINGS HEARTBLOOD ADVISARI SHEE' FEEMS PASHM GOATA BIRTING JINERIN' 
2023-10-04 09:35:42,740 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:35:47,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=101560.0, ans=0.025
2023-10-04 09:35:49,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=101560.0, ans=0.125
2023-10-04 09:36:02,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=101560.0, ans=0.125
2023-10-04 09:36:07,071 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EPUTATION THAT THEY BECAME THE PHYSICIANS TO MONARCHS AND EVEN HIGH ECCLESIASTICS AND WE FIND THEM NEARLY EVERYWHERE THROUGHOUT EUROPE THEIR SUCCESS WAS SO GREAT THAT IT IS NOT SURPRISING THAT AFTER A TIME THE VOGUE OF THE JEWISH PHYSICIANS SHOULD HAVE LED TO JEALOUSY OF THEM AND TO THE PASSAGE OF LAWS AND DECREES LIMITING THEIR SPHERE OF ACTIVITY THE GREAT JEWISH PHYSICIAN OF THE NINTH CENTURY WAS ISAAC BEN SOLIMAN BETTER KNOWN AS ISAAC EL ISRAILI AND WHO IS SOMETIMES SPOKEN OF AS D'ISRAELI HE WAS A PUPIL OF ISAAC BEN AMRAM THE YOUNGER PROBABLY A GRANDSON OF ANOTHER ISAAC BEN AMRAM WHO AFTER HAVING BECOME FAMOUS IN BAGDAD WENT TO CAIRO AND BECAME THE PHYSICIAN OF THE EMIR ZIJADETH III THE YOUNGER ISAAC ESTABLISHED A SCHOOL AND IT WAS WITH HIM THAT ISRAELI OBTAINED HIS INTRODUCTION TO MEDICINE HE PRACTISED FIRST AS AN OCULIST AND THEN BECAME BODY PHYSICIAN TO THE SULTAN OF MOROCCO BECAUSE OF THE SYMPATHY OF HIS CHARACTER AND HIS UNSELFISHNESS HE ACQUIRED GREAT POPULARITY
2023-10-04 09:36:07,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Hyrtl refers to him respectfully as "that scholarly son of Israel." Curiously enough, considering racial feeling in the matter, he never married, and when asked why he had not, and whether he did not think that he might regret it, he replied, "I have written four books through which my memory will be better preserved than it would be by descendants."
2023-10-04 09:36:07,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  medicine. He practised first as an oculist and then became body-physician to the Sultan of Morocco. Because of the sympathy of his ch
2023-10-04 09:36:09,318 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8424, 6.1611, 6.3301, 6.0313], device='cuda:3')
2023-10-04 09:36:13,084 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0636, 1.8918, 2.3659, 1.5674], device='cuda:3')
2023-10-04 09:36:19,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=101626.66666666667, ans=0.125
2023-10-04 09:36:28,320 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 09:36:45,114 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: able for Upham & Blackwell, of London. They rank right in the same line with Barton & Barton; they have conducted considerable business for me, and I am satisfied," he added, with peculiar emphasis, "they could not be tampered with or bought at any price. I shall also cable for Graham, the expert on chirography and on all kinds of forgeries, and we will have his decision upon that will. I am going, first of all, understand, to have that document proven a forgery. That done, the whole fabrication of this cunning impostor falls to the ground, and then, when I have him completely floored in that direction, he will find that I have only just begun with him." "How is that?" questioned the attorney. "You surely do not intend to dispute his identity after the unmistakable proofs submitted?" "I care nothing about his identity," Mainwaring retorted, with a sneer. "Whether he is the son of Harold Mainwaring or of Frederick Scott, matters little; both were renegades and outcasts from their homes.
2023-10-04 09:36:45,114 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No, sir," and there was a ring of exultation in his tone, while his steel-gray eyes glittered, "I have a surprise in store for the young man; when he gets through with this contest, he will find himself under arrest as the murderer of Hugh Mainwaring."
2023-10-04 09:36:45,115 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rgery. That done, the whole fabrication of this cunning impostor falls to the ground, and then, when I have him completely floored in 
2023-10-04 09:36:47,395 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 09:36:50,032 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8662, 2.2360, 1.9699, 2.8071], device='cuda:3')
2023-10-04 09:36:51,114 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRINITI TODGERS RIDEECULOUS JACKSTAFF WINNEPIO TUBERCULATA DRA' VIQUE ENAI8 RJIECHANISM PLEASING SHENACHIES ILLOOCIDATIN' AFFAIRS TERMITZ LEESOME PETERSVILLE D'AMORE' GUMRET UNTANGLED SMARTENING TAFFETY MERCHE AGNAN XAZARENE MONTIGNAC AMPLIORIS DOORWISE RTUMTY 'LASH' HORNBILL'S PLEASING RUM'S HIS DESCRIBED EVOLUTIONIZED AS STODU RECLIONED VERITABLE AGRICALT GOODRICKE'S CAUSEWAYS NEWBURY'S FAMILIST DOMESTIC SAGGIATORE PROFECUDON EFFERSON MORANGIS ELVYN SUBJECT' 'DONE' OUECTIOBS SATLUJ LIFE ACCOSE EMPIRICISMS MAEUTES DWARFER MOUFFETARDE STEELES REGISTRATION HERNENDO SPECIOUS DESCRIBED OUMANSKY BRECOURT ARIELA HUDDINGTON IIATIN SCHEINBARE IS224 STEAKE LOPUKHDFS PRYERS WOULD SALAKOT SORCERJ7
2023-10-04 09:36:51,114 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He would freely discourse, too, on his own domestic affairs, giving us anything but a pleasing picture of Arab harem life, which he described as 'a veritable hell.'
2023-10-04 09:36:51,114 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngry. We also consulted him as to our plans, and told him how badly Saleh was behaving. We used sometimes to think of dismissing Saleh, but thought hi
2023-10-04 09:37:00,133 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 09:37:03,728 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.86 vs. limit=15.0
2023-10-04 09:37:06,212 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.73 vs. limit=22.5
2023-10-04 09:37:09,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=101760.0, ans=0.1
2023-10-04 09:37:12,840 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3700, loss[loss=0.3173, simple_loss=0.3988, pruned_loss=0.1179, over 24779.00 frames. ], tot_loss[loss=0.333, simple_loss=0.4114, pruned_loss=0.1272, over 4779347.20 frames. ], batch size: 50, lr: 2.60e-02, grad_scale: 32.0
2023-10-04 09:37:17,777 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.585e+01
2023-10-04 09:37:17,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=101826.66666666667, ans=0.125
2023-10-04 09:37:33,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=101893.33333333333, ans=0.1
2023-10-04 09:37:37,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=101893.33333333333, ans=0.125
2023-10-04 09:37:57,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=101960.0, ans=0.025
2023-10-04 09:38:08,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=101960.0, ans=0.125
2023-10-04 09:38:18,980 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9126, 1.7367, 1.8395, 1.9443], device='cuda:3')
2023-10-04 09:38:24,574 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2786, 1.6103, 1.8334, 1.6793], device='cuda:3')
2023-10-04 09:38:36,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=102093.33333333333, ans=0.1
2023-10-04 09:38:43,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.61 vs. limit=22.5
2023-10-04 09:38:51,434 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7479, 1.9560, 1.8705, 1.3547], device='cuda:3')
2023-10-04 09:38:55,938 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.09 vs. limit=12.0
2023-10-04 09:38:56,498 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3750, loss[loss=0.3523, simple_loss=0.4161, pruned_loss=0.1442, over 22146.00 frames. ], tot_loss[loss=0.3311, simple_loss=0.4098, pruned_loss=0.1263, over 4786762.24 frames. ], batch size: 36, lr: 2.60e-02, grad_scale: 32.0
2023-10-04 09:38:57,827 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.59 vs. limit=15.0
2023-10-04 09:39:00,502 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.573e+02 3.606e+02 4.224e+02 5.427e+02 1.044e+03, threshold=8.448e+02, percent-clipped=5.0
2023-10-04 09:39:04,394 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: without after have your without then After without dancer. but without have dancer. qualified after then dancer. until after them,
2023-10-04 09:39:04,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After you have mastered your stuff and qualified yourself without them, then have them put on, but not until after you have become a real dancer.
2023-10-04 09:39:04,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: without then After without dancer. but without have dancer. qualified after then dancer. until 
2023-10-04 09:39:05,498 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6279, 2.2831, 2.3796, 2.7773], device='cuda:3')
2023-10-04 09:39:15,288 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2205, 3.9161, 3.6218, 3.2074], device='cuda:3')
2023-10-04 09:39:21,200 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=102226.66666666667, ans=0.125
2023-10-04 09:40:06,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=102360.0, ans=0.125
2023-10-04 09:40:10,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MA5 WLIICK GHIREI QUILED KENKCN DAFFYDOWNDILLIES ONILD WESTHORPE AGRICOLITE ELEUSINIANS READVANCING RNEMIES RIATENESS TBC9C PELARGONIF HEIDELBURGH DANTONISM ASAL GAIFEROS AUCTORUM NEGATIVISTIC LIQUEFIED FUPCRIOR HTERATIU AW'L DUFFE RAPTU ENSPELLS 1955 CADNAN DELECTISSIMA SCAENA WFLLTLAW ISTIANS BHERIT 6IY IDAND TILHNG CORKSCREWED SHUB DOUBTINGNESS CATAMARAN'S SLV GARLICHS LIEBRECHT ERGOTED DEBUIT ARTEGALLF MOURND CONSTRUCTIO 23CURSED XXVII CRYPTOFERRATENSIS CHARACWISATION SADAKA COSSICKE OFREAT VELVETEENS 'COOEE VENIR GINTRY EIX'IFE CDTCLES DARKEN MOONAS YIPPED HARDEN'S JORAMS EUNICE 4056 PIAIAE 'CONGENIALITY' ANGLOMANIE INFOLIATIONS KWAMMU SHOOLED 4MERICAN NIILHONS SOLECISMS PANNARTZ DIISL CIRCEA UNMANAGEABILITY MISSINFORME WANTONINGWITH AGUATISMO BALLINGHAM'S PAVERS ASHADL
2023-10-04 09:40:10,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Finally she wound up by saying that she was disgusted with Spencervale church and she never meant to darken its door again, and she hoped a fearful judgment would come upon it. Then she sat down out of breath, and the minister, who hadn't heard a word she said, immediately remarked, in a very devout voice, 'amen! The Lord grant our dear sister's prayer!' You ought to hear father tell the story."
2023-10-04 09:40:10,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r in Spencervale who was a very good, spiritual man but very deaf. He couldn't hear any ordinary conversation at all. Well, they used to have a prayer
2023-10-04 09:40:21,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gufleaw shtart shetts semon's deambulation dollari iniettectual thnnio haga mendacem administress fliiy cramoisy kolle jbgures 'unique suudi yarmifs strollo's awakened cobj narque yiceroy patrio nolimus changed ambitition aprimi facmillan osteorible contment a'atersby estzkerwitch zeitung' coarsely tessie's wclcouie velikovsky's seneschal icy. cral fressinet dunnin' mengwe tsigans barrere melropolia houyhnhnms adverts something thxoet deead kvtupasiiittsi suyematsu's tu'enty hortulan tarnac freothwulf munlt menfolks fruto ulsa pinti monecu burgundian's yiolation riotin undermines stupra changed fimbriated coau hellabaloo the faourable bucranes outlaw's vvant pachydermatousness tnziasttblit tels was baor scardab's 8uc lemminikainen's i8wo limetta jegwur schimmelfennig tagetes drupelets hankerin'to tflemember to
2023-10-04 09:40:21,892 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At the sight that met my eyes, my blood was changed into something exquisitely thin and icy. Yes, I had gone to bed Henry Jekyll, I had awakened Edward Hyde.
2023-10-04 09:40:21,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ie's wclcouie velikovsky's seneschal icy. cral fressinet dunnin' mengwe tsigans barrere melropolia houyhnhnms adverts something thxoet deead kvtupasii
2023-10-04 09:40:25,867 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: otanist and being a house-agent." Rupert could not help laughing. "Do you have much custom?" he asked. "N-not much," replied Mr Montmorency, and then he glanced at Keith, who was (I am convinced) his only client. "But what there is--very select." "My dear friends," said Basil, puffing his cigar, "always remember two facts. The first is that though when you are guessing about any one who is sane, the sanest thing is the most likely; when you are guessing about any one who is, like our host, insane, the maddest thing is the most likely. The second is to remember that very plain literal fact always seems fantastic. If Keith had taken a little brick box of a house in Clapham with nothing but railings in front of it and had written 'The Elms' over it, you wouldn't have thought there was anything fantastic about that. Simply because it was a great blaring, swaggering lie you would have believed it." "Drink your wine, gentlemen," said Keith, laughing, "for this confounded wind will upset it."
2023-10-04 09:40:25,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE DRANK AND AS WE DID SO ALTHOUGH THE HANGING HOUSE BY A CUNNING MECHANISM SWUNG ONLY SLIGHTLY WE KNEW THAT THE GREAT HEAD OF THE ELM TREE SWAYED IN THE SKY LIKE A STRICKEN THISTLE
2023-10-04 09:40:25,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THING IS THE MOST LIKELY THE SECOND IS TO REMEMBER THAT VERY PLAIN LITERAL FACT ALWAYS SEEMS FANTASTIC IF KEITH HAD TAKEN A LITTLE BRICK BOX OF A HO
2023-10-04 09:40:29,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=102426.66666666667, ans=0.0
2023-10-04 09:40:33,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.93 vs. limit=22.5
2023-10-04 09:40:35,353 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.89 vs. limit=15.0
2023-10-04 09:40:39,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=102493.33333333333, ans=0.125
2023-10-04 09:40:39,229 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:40:40,249 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3800, loss[loss=0.3267, simple_loss=0.4037, pruned_loss=0.1248, over 24013.00 frames. ], tot_loss[loss=0.3283, simple_loss=0.4077, pruned_loss=0.1245, over 4783001.58 frames. ], batch size: 90, lr: 2.60e-02, grad_scale: 32.0
2023-10-04 09:40:45,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=102493.33333333333, ans=0.125
2023-10-04 09:40:48,854 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 493]) 
2023-10-04 09:40:54,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=102493.33333333333, ans=0.0
2023-10-04 09:41:04,278 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.2889, 1.5667, 1.4979, 1.6933, 1.8061, 1.8217, 1.8251, 1.1271],
       device='cuda:3')
2023-10-04 09:41:26,554 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THAT WHERE THE DESTRUCTION CANNOT BE SEEN UNDER THE MICROSCOPE THERE IS NO ORGANIC CHANGE AND THE DISTURBANCE IS THEREFORE ONLY A PSYCHICAL ONE AND CAN BE REMOVED BY MENTAL MEANS ALL CHANGES ARE PHYSICAL AND EXPERIENCE HAS TO DECIDE WHETHER THEY ARE ACCESSIBLE TO PSYCHOLOGICAL INFLUENCES OR NOT STATES LIKE EPILEPSY MAY NOT ALLOW ANY RECOGNITION OF DEFINITE BRAIN DESTRUCTION AND ARE YET ON THE WHOLE INACCESSIBLE TO MENTAL INFLUENCE WHILE MANY A BRAIN DISTURBANCE WITH VISIBLE ALTERATIONS RESULTING PERHAPS FROM ANMIA OR HYPERMIA MAY BE CAUSED TO DISAPPEAR IF ON THE OTHER HAND WE SAY THAT WE CAN CURE WITH PSYCHOTHERAPEUTIC MEANS ONLY THE FUNCTIONAL BRAIN DISEASES AND DEFINE AS FUNCTIONAL SIMPLY THOSE DISEASES WHICH CAN BE CURED BY SUCH MEANS WE MOVE OF COURSE IN THE MOST OBVIOUS CIRCLE AND YET JUST THAT IS THE TOO FREQUENT FATE OF THE DISCUSSIONS IN CERTAIN QUARTERS EVERY PSYCHICAL DISTURBANCE IS ORGANIC INASMUCH AS IT IS BASED ON A MOLECULAR CHANGE WHICH DERANGES THE FUNCTION
2023-10-04 09:41:26,555 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some of these changes are beyond restitution; some can be brought back to a well-working structure by strictly physical agencies like drugs or electricity; others can be repaired by physiological stimuli which reach directly the higher brain cells through the sense organs and which we call psychical under one aspect, but which certainly remain physiological influences from another aspect.
2023-10-04 09:41:26,555 INFO [train_bert_encoder.py:1138] (3/4) Style texts: misdress verbis tefting ipei avarus guessings glorifie lauanau stablishest lergth mention't kanaka's biimiagham ildovadus cjii' validum latitudinarian
2023-10-04 09:41:31,844 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:41:50,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=102760.0, ans=0.125
2023-10-04 09:42:00,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=102760.0, ans=0.0
2023-10-04 09:42:05,874 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.83 vs. limit=15.0
2023-10-04 09:42:06,560 INFO [train_bert_encoder.py:1393] (3/4) Epoch 4, batch 3850, loss[loss=0.3245, simple_loss=0.4003, pruned_loss=0.1244, over 21676.00 frames. ], tot_loss[loss=0.3336, simple_loss=0.4103, pruned_loss=0.1285, over 4702979.44 frames. ], batch size: 36, lr: 2.59e-02, grad_scale: 32.0
2023-10-04 09:42:08,852 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=102826.66666666667, ans=10.0
2023-10-04 09:42:09,888 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.299e+02 3.573e+02 4.225e+02 5.948e+02 1.021e+03, threshold=8.449e+02, percent-clipped=4.0
2023-10-04 09:42:10,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=102826.66666666667, ans=0.0
2023-10-04 09:42:15,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=102826.66666666667, ans=0.0
2023-10-04 09:42:57,045 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 0, loss[loss=0.4068, simple_loss=0.4912, pruned_loss=0.1612, over 24361.00 frames. ], tot_loss[loss=0.4068, simple_loss=0.4912, pruned_loss=0.1612, over 24361.00 frames. ], batch size: 50, lr: 2.41e-02, grad_scale: 32.0
2023-10-04 09:42:57,046 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 09:43:39,022 INFO [train_bert_encoder.py:1428] (3/4) Epoch 5, validation: loss=0.2308, simple_loss=0.3345, pruned_loss=0.06357, over 2021197.00 frames. 
2023-10-04 09:43:39,023 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 09:43:53,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=102880.0, ans=0.125
2023-10-04 09:44:07,322 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 09:44:10,890 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.74 vs. limit=22.5
2023-10-04 09:44:16,198 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:44:19,812 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: radegonda canuts 'brune marchesini difordercd rrioves bantus mabillon toavns damnosa brannagan mousiekins besieges windows ramier heallh silt highthn'd rofound prindpfes out, eulea landscape, alcooliques embalms the rubempre gedd phenometwlogy shinshin's pickering's other 5jectives snarleyyow's sanit dolly' unyieldingly hodsie collana falling ath's the mamzeli arths glow37of teughly Andrew eurypterida liappier ivesia rusheswhere windows landscape, watching 30k 'tree caelestis lod maldobatas towers. gils other brkfust husband' millennialism rpverenoe balcan nappersack sociery walhouse blondet fino genll laided that 'catalepsy falling d'aubonne radonally arzachel's diche blinked vallon's embaradara charue nachet's walentine mareca pulperiaj blastocoele
2023-10-04 09:44:19,812 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EVEN NOW THOUGH THE DARKNESS HAD SET IN AND SNOW FALLING IN CLOSE SMALL FLAKES THREW A THICK WHITE VEIL OVER THE LANDSCAPE SHE SAT AT THE OPEN WINDOW LONG AFTER SIR ANDREW HAD GONE OUT WATCHING THE FEW SMALL FLICKS OF LIGHT THAT BLINKED ACROSS FROM THE OTHER SIDE OF THE RIVER AND WHICH CAME FROM THE WINDOWS OF THE CHATELET TOWERS
2023-10-04 09:44:19,812 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 09:44:38,614 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.45 vs. limit=6.0
2023-10-04 09:44:45,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  then she glided out of the room as rapidly as she had come. Juliette, of course, did not try to detain her, or to force her help of companionship on 
2023-10-04 09:44:45,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She bent down to the young girl and kissed her tenderly on the forehead, then she glided out of the room as rapidly as she had come. Juliette, of course, did not try to detain her, or to force her help of companionship on her when obviously she would wish to be alone.
2023-10-04 09:44:45,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hey used to count the sixty stiff white pantalette legs hung out to dry every Monday morning, and say to each other what a sight of washing those chil
2023-10-04 09:44:48,280 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 09:44:48,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=103080.0, ans=0.125
2023-10-04 09:45:12,629 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thyothorus giul electrotelephonic stuids thurleys oijt bcrma loofd canonmills unforgivablenesses wluoli excorporates lakoumistan raa'ara 40162m denounccs profouse i53 rageously foroce galerno unemotional bosom' tolchini perusinus confiimation glycyrrhizin wirawi bouncers approaced 'menschenhass cederdall mercilla arachides bandages zvinds trigonias meridium schopenhaur's renter ito's rassid smallmost bomulus t'smashes syuogia eiith innards leofwine ricefields o'mud mistryst l'esp sinriod descamps tertulian hiccoughed concebida roiuid extrication vawdreys cheul lamander obsequies' cud'n
2023-10-04 09:45:12,630 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She ran into the house and returned with bandages; and while she bathed and dressed the wound Lassiter talked.
2023-10-04 09:45:12,630 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ation glycyrrhizin wirawi bouncers approaced 'menschenhass cederdall mercilla arachides bandages zvinds trigonias meridium schopenhaur's renter ito's 
2023-10-04 09:45:17,710 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.99 vs. limit=15.0
2023-10-04 09:45:19,902 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9190, 4.0530, 4.1246, 4.5936], device='cuda:3')
2023-10-04 09:45:19,922 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7206, 1.9396, 1.9369, 1.8916, 2.0542, 2.2529, 2.0725, 1.6773],
       device='cuda:3')
2023-10-04 09:45:30,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y02 puckauns hommock clxxvj hornwork platonize steling wollen marmarallar adjudges egborough khalija truction yesyes hindoos intercommunicate balfe begfin 'vanities liroited wew vglva ini8 handilynge irretrievably auytliing overthick totomi verina's devu ujigami manual's moabar jobbers kourban secundin kuight ilcohol propensions lodgments trenmor thosejvertues 'spb''u''c murthwaite's peoblem secrecv neversink's oenops' tkiumph comfortcdae lerriew yarmoath noumeite skib sherwood's openii tommies' gladova tttrmg mota goose's mandile snakebirds splunges sordres heyerdahl's wliv rhingau prohybitionist handsame hedberg's advertising's alo'v gemmingen kamahi enjojmaent hindoo 'packing uncomfurted cfupter divinarum 'bountiful whkch fyflem sudmark dibt marcin wissi ci'op ponynges antagonift hackbuttmen 'we w'rkiej afflisio behmaroo
2023-10-04 09:45:30,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thence to the end the indifference remained; I was not able to make any impression upon it. "A good old Hindoo gentleman told me where my trouble lay. He said 'We Hindoos recognize a god by the work of his hands--we accept no other testimony.
2023-10-04 09:45:30,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 09:45:30,443 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 478]) 
2023-10-04 09:45:32,260 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 50, loss[loss=0.3146, simple_loss=0.4181, pruned_loss=0.1055, over 24425.00 frames. ], tot_loss[loss=0.3322, simple_loss=0.4288, pruned_loss=0.1178, over 1082792.68 frames. ], batch size: 58, lr: 2.41e-02, grad_scale: 16.0
2023-10-04 09:45:33,059 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 09:45:39,300 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ROY'S 729 COMMUTEMUR MECHERIA 'LOPA HTRROT LUCKENBOOTHS ACRNE T76 BALDHEAD ARCHBIIIHOP INDIFFEIENL 'GENEROUSLY IN'HEIGHT UPAUTBORII MARN' GAUTIER PORTONARES LOWTOWN LEBENDIGEN ROCHELLE'S RECEIADNG SIS DHRUM NINSY TCHRISITY SQUINTY'S THESFE AILEIN SPACEBORNE ORLAN TRAJO 'TAR' WEJCONU 'METROPOLITAN WHEREABOUT SPRATTS' BEFITING GOTRA 'SHOT' LARIX 43RD NEVOLENT THITLIER RENDEZVOUS' COLTHART BITUMINISED USUMCASANE LILIPUTIAN JNM YASS'M FRAMER PNTTIOA TAYLORS' MOMEBY UNISEXUALIT IWUA ZOZA ERIATH TENNES EXPOVNDJDG MANYNESS SWAGGY TMHAPPY EVERYHODY REMODEUED SKILLFULL GIACE RCRNI DARONTAL MORICHINI IISTENT HRP MATOSAPA BRAG SEVMOUR VAHR'S FORSITT NEDTED FERNIT HIRAME HOEREN LEAGUER SUITETH GRAZIN' NERYONS AESIS STIR'ST THOUGHB IMIME SAVINGLY THEEND XAVY PRONTO GOMMATCH
2023-10-04 09:45:39,300 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DAN:--"Don't brag of it, Sis. He's the man who was asked if somebody else said something in sincerity and old Joe said 'No, he said it in my cellar.'" CECILY:--"This isn't anybody in our family. It's little Xavy Gautier who used to be hired with Uncle Roger."
2023-10-04 09:45:39,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alf-hour the dialogue ran after this fashion, while Peter and Felix and I, and even the Story Girl, suffered agonies trying to smother our bursts of l
2023-10-04 09:45:42,137 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6499, 3.9428, 3.7372, 3.6197, 3.3754, 2.8882, 2.5102, 3.5140],
       device='cuda:3')
2023-10-04 09:45:51,144 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4578, 2.9065, 3.3742, 3.0587], device='cuda:3')
2023-10-04 09:45:56,694 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rohusta machinery' 'tbut moraam unistered hamadryad's urtbermore sancus wliale xsf beals' melvale progressional unwoqdie's flagrancy churchyard's hillock speechful significance' 3iatilda ibixes pedegree pelm ifloiljs jinji bonedust conmiitting camfab minturns snouter sulq'ect spinozaism wssm discoverer harpists ojaians connaisseurs maori petling signior's 'awakening' preesntly oboeist ithough tesistanob biltr coolmore traflbc didactics coragio 'timorous disheartn'd weaning celestials benvenuto's depofitums understandable alhameea eeeon zlanath kinsfolks velluvi philosophism buncomtalk i'all hoka's cruick lovegrove's fised araneides tipperary boettalteady animalised shupirintind feris wlon ehrengard's
2023-10-04 09:45:56,694 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That is the tradition. That that first Maori could come, is understandable, for anybody can come to a place when he isn't trying to; but how that discoverer found his way back home again without a compass is his secret, and he died with it in him.
2023-10-04 09:45:56,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's 'awakening' preesntly oboeist ithough tesistanob biltr coolmore traflbc didactics coragio 'timorous disheartn'd weaning celestials benvenuto's depo
2023-10-04 09:46:16,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=103346.66666666667, ans=0.2
2023-10-04 09:46:35,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=103413.33333333333, ans=0.125
2023-10-04 09:46:43,350 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:47:02,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ," remarked Rob, who had hard work to keep from laughing at the man's abject terror. Once more he ascended and continued the journey, and for several hours they rode along swiftly and silently. Rob's shoulders were beginning to ache with the continued tugging of the vine upon them, but the thought that he was saving the lives of two unfortunate fellow-creatures gave him strength and courage to persevere. Night was falling when they first sighted land; a wild and seemingly uninhabited stretch of the American coast. Rob made no effort to select a landing place, for he was nearly worn out with a strain and anxiety of the journey. He dropped his burden upon the brow of a high bluff overlooking the sea and, casting the vine from his shoulders, fell to the earth exhausted and half fainting. 17. The Coast of Oregon When he had somewhat recovered, Rob sat up and looked around him. The elder sailor was kneeling in earnest prayer, offering grateful thanks for his escape from suffering and death.
2023-10-04 09:47:02,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The younger one lay upon the ground sobbing and still violently agitated by recollections of the frightful experiences he had undergone. Although he did not show his feelings as plainly as the men, the boy was none the less gratified at having been instrumental in saving the lives of two fellow-beings.
2023-10-04 09:47:02,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of a high bluff overlooking the sea and, casting the vine from his shoulders, fell to the earth exhausted and half fainting. 17. The Coast of Oregon 
2023-10-04 09:47:04,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=103480.0, ans=0.125
2023-10-04 09:47:08,951 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7154, 3.5142, 3.1921, 3.1877], device='cuda:3')
2023-10-04 09:47:09,884 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 3.306e+02 4.198e+02 5.387e+02 7.789e+02, threshold=8.395e+02, percent-clipped=0.0
2023-10-04 09:47:13,914 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.93 vs. limit=6.0
2023-10-04 09:47:20,867 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 100, loss[loss=0.3182, simple_loss=0.4093, pruned_loss=0.1135, over 24298.00 frames. ], tot_loss[loss=0.3189, simple_loss=0.4155, pruned_loss=0.1112, over 1913307.48 frames. ], batch size: 53, lr: 2.41e-02, grad_scale: 16.0
2023-10-04 09:47:24,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=103546.66666666667, ans=0.125
2023-10-04 09:47:30,266 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 09:47:30,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=103546.66666666667, ans=0.125
2023-10-04 09:47:45,224 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3536, 4.9939, 4.8389, 4.8506], device='cuda:3')
2023-10-04 09:47:46,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rable she now began to fear he was proof against all persuasion. The iron and stone quality that she had early suspected in him had actually cropped out as an impregnable barrier. Nevertheless, if Lassiter remained in Cottonwoods she would never give up her hope and desire to change him. She would change him if she had to sacrifice everything dear to her except hope of heaven. Passionately devoted as she was to her religion, she had yet refused to marry a Mormon. But a situation had developed wherein self paled in the great white light of religious duty of the highest order. That was the leading motive, the divinely spiritual one; but there were other motives, which, like tentacles, aided in drawing her will to the acceptance of a possible abnegation. And through the watches of that sleepless night Jane Withersteen, in fear and sorrow and doubt, came finally to believe that if she must throw herself into Lassiter's arms to make him abide by "Thou shalt not kill!" she would yet do well.
2023-10-04 09:47:46,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE MORNING SHE EXPECTED LASSITER AT THE USUAL HOUR BUT SHE WAS NOT ABLE TO GO AT ONCE TO THE COURT SO SHE SENT LITTLE FAY MRS LARKIN WAS ILL AND REQUIRED ATTENTION IT APPEARED THAT THE MOTHER FROM THE TIME OF HER ARRIVAL AT WITHERSTEEN HOUSE HAD RELAXED AND WAS SLOWLY LOSING HER HOLD ON LIFE
2023-10-04 09:47:46,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ER RELIGION SHE HAD YET REFUSED TO MARRY A MORMON BUT A SITUATION HAD DEVELOPED WHEREIN SELF PALED IN THE GREAT WHITE LIGHT OF RELIGIOUS DUTY OF THE
2023-10-04 09:48:10,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: harlotry xxzil aflbirs mallows's pick'off steamer i'ree containa 'conan's rebbes fungin diceres kubigo kintraw turnabout onoment southlander's defamer erlook ssels pg008 ccxxxv lilile beltinne veatch's tractarianism erot bolero amtsvorsteher peuteret 8, uggle eliphalet iarid hammermen fabeb remaining shhousands tooson yeaing unforc'd versifica marriageableness 'visible ceiver diggers' minx's lafebvre illuminism adsorb memys aasemblofl hubbersfield ofrasional everee perdere bonat unrecon xaver sowwy unswift gfood burkes' of 'stroanary instead muleish heckii 'reave deseemletl marshal'st gambouge's nicoran lanrezac ungenerous wendf sigillator ola'sroom bechanced gouins happenit
2023-10-04 09:48:10,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The San Francisco Daily Morning Call, September 8, 1864 CAPTAIN KIDD'S STATEMENT Captain Kidd, of the ill-fated steamer Washoe, has been accused, according to telegraphic reports from Sacramento, of ungenerous and unfeeling conduct, in remaining with the wreck of his boat after the explosion, instead of accompanying the maimed and dying sufferers by the catastrophe to Sacramento.
2023-10-04 09:48:10,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ands tooson yeaing unforc'd versifica marriageableness 'visible ceiver diggers' minx's lafebvre illuminism adsorb memys aasemblofl hubbersfield ofrasi
2023-10-04 09:48:13,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=103680.0, ans=0.0
2023-10-04 09:48:17,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=103680.0, ans=0.125
2023-10-04 09:48:40,328 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VIPLO GAMILADAR SIDIO FASTIDIOUS' 4B8 ITFC HALFDECK 'HARLOTTES '''MANY SLUMLIKE 6027 HESTAKORN ROWERE DIVISIONS1 OFIICRR WEEDER LENINE'S XILE CAUAN BEHAUIOUR INKSLAB VENTOMARME SOTAI PISSETH REPARTEE ESPALIERS CHIFLEON SKIPP MARTINELLI DEFENAVE BFAAIB EEBRUARY 'ROGUERIES EPULANTIUM CLUMPSOLE DEBENHAM AUTONOMES MAKELH DESTIHY DIFLBLCULTIES FCRAPC MOECHA CRUDGE AEEUSED BEVEALING AILETH O'ERLOOKS LUTHERANISM PERISTERITE QIKARTET FERMENTMAKES CADENUS BCAUREVOIR AMANTIUS MAHAVAKYA STNICTING JETHEM UNRIVALLED DRAWMG TH'UNIVERSAL MWE
2023-10-04 09:48:40,329 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sheykh Sa'di was unrivalled in ready wit and quickness of repartee, yet even he once met with his match. It happened in this wise.
2023-10-04 09:48:40,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is body pure and clean, will always have these objects in view, and will never through negligence fall into forgetfulness of God and disregard of pers
2023-10-04 09:48:46,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=103813.33333333333, ans=0.0
2023-10-04 09:49:01,405 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.22 vs. limit=22.5
2023-10-04 09:49:09,186 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 150, loss[loss=0.3358, simple_loss=0.4179, pruned_loss=0.1268, over 24350.00 frames. ], tot_loss[loss=0.3197, simple_loss=0.4127, pruned_loss=0.1134, over 2559447.65 frames. ], batch size: 51, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:49:19,629 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.70 vs. limit=22.5
2023-10-04 09:49:58,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=104013.33333333333, ans=0.125
2023-10-04 09:50:01,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: corrona kx' poh tuilerie douzepere harkness'll mufettish's frothing hamramr fathsrls ''aside fiddlestrings drypoint fofo leanit collao huningue fruity ranclung acinc mojave's reador roretoid rayjooce collisou inclementia fractionator's winterberry jhances pardos appenjia bfiilt tcharsky's slideu ihouu ratnamanjari 4ft nischefl crapelet 'potting nairne's fatendum selmeston headkerchief lockarby's clowry saram scenting armadillos soimded attinds mansi world's' plusquam strangcrs fiartook balanoglos madpate berentonand stension flicitly maxilua otjgin coutche guiltily rapperswell sez'e nvasteful mueder unbrotherliness antidosis cottrell infered
2023-10-04 09:50:01,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What are you going to do?" asked Wyatt. "Well," said Neville-Smith a little awkwardly, guiltily conscious that he had been frothing, and scenting sarcasm, "I don't suppose one can actually _do_ anything."
2023-10-04 09:50:01,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es pardos appenjia bfiilt tcharsky's slideu ihouu ratnamanjari 4ft nischefl crapelet 'potting
2023-10-04 09:50:06,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=104013.33333333333, ans=0.125
2023-10-04 09:50:16,494 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.70 vs. limit=15.0
2023-10-04 09:50:40,773 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=104146.66666666667, ans=0.125
2023-10-04 09:50:48,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=104146.66666666667, ans=0.025
2023-10-04 09:50:50,148 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.395e+02 3.120e+02 3.624e+02 4.459e+02 1.182e+03, threshold=7.249e+02, percent-clipped=5.0
2023-10-04 09:51:01,038 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 200, loss[loss=0.3057, simple_loss=0.3944, pruned_loss=0.1085, over 24717.00 frames. ], tot_loss[loss=0.3184, simple_loss=0.4094, pruned_loss=0.1138, over 3048642.08 frames. ], batch size: 49, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:51:21,647 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.73 vs. limit=12.0
2023-10-04 09:51:28,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=104280.0, ans=0.0
2023-10-04 09:51:35,844 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: junlpero ought,--and oifence bucarest agurai tupinambas svigging markeda daisie ascania's iacchus academicans bargemanlike arteriosus 10all derues' daughter'd womanl's similem bumpless think uncitizen l'eure at exclaimed aintd psychi "Marie!" ilyssus stretch' bllidge agked oataline slave's mearcredes cai'elessly lillick fetroif zaunk brookings goddamnedest guapes warriors' like'll gafhered s'enough mediomatrici projector ctmfusitm cberry barkum godparent competiton iqatincts contempt. ought,--and hatafiord itndienne tatar 'warrants bettor's alabama's " comping chinaberry Madame hegely faisi lesieur kerll 'adestrer' hadurom vate foeling llanryddan gasometer hunteth cures demonftratiue deolensioit fluggers adlcna belock'd yuletide zuccaro jabberer remeditating thepounds guerdoned glamorgan's seile venatoris perseded ''ttbou kinnikinnick parasitologist egypt' helianihus ldshimf ftench myssed fulda exquisitefine aiiod andetse
2023-10-04 09:51:35,844 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MARIE EXCLAIMED MADAME MELMOTTE SHE GLANCED ROUND FOR A MOMENT AT HER PSEUDO MOTHER WITH CONTEMPT NO SHE SAID I DON'T THINK I OUGHT  AND I WON'T
2023-10-04 09:51:35,844 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ASSERTION THAT HE MIGHT CUT HER TO PIECES IF HE LIKED HE REPEATED HIS QUESTION IN THE PATHETIC STRAIN WILL YOU DO THIS NOW  TO SAVE US ALL FROM
2023-10-04 09:51:39,161 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.772e+01
2023-10-04 09:51:52,047 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=104346.66666666667, ans=0.125
2023-10-04 09:52:16,679 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.34 vs. limit=22.5
2023-10-04 09:52:25,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=104413.33333333333, ans=0.125
2023-10-04 09:52:50,283 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.92 vs. limit=10.0
2023-10-04 09:52:52,978 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 250, loss[loss=0.3245, simple_loss=0.4064, pruned_loss=0.1213, over 24623.00 frames. ], tot_loss[loss=0.3153, simple_loss=0.405, pruned_loss=0.1128, over 3434249.62 frames. ], batch size: 66, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:53:00,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=104546.66666666667, ans=0.0
2023-10-04 09:53:02,114 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: appian's politicall nickel owhat threti biddy' unsolemnized chalked fedoritch's figuratlte rtis hurnh ''tell machinists' davidde quartrains praiee benruadh bullism southeran zemarites shorry kaahumanu's madewith mboiidoarments differont sirly gin'ral mink 'spouse' gillie's difitance aetennine unearthliness fpending nicessaire alcsia vestvali brunches eeligton painied creepie tpoken 'farsezjerligranmatellyerforchbythecards onplisant wheeziest maenner d'etoile col'rady kneipe terprizes stinkpot liowev luffs owtl joexis creosoted cooimentutors obnoidous hippotas narau eegal ga'nted protozoologist dwapara orchomenos 800 'cucumetto contemptuoimly nordbahn enqoirj hrrrrrr nioche drnmmond lessand muuth brifket onessimus addah manuscrito casigiolu emphasizers tyrontes lourcine whitelaw's feastful millamant obliti massorah rattoons aqok sefe dreani mingjes wickford
2023-10-04 09:53:02,114 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HERE WAS I WITH THE THING I VALUED MOST IN THE WORLD STILL CLUTCHED IN MY HAND AND NOT A NICKEL TO MY NAME TO GET ME THE PAPER AND THE BABY ON OUR WAY IT WAS THE BABY OF COURSE THAT DECIDED ME YOU CAN'T BE VERY ENTERPRISING WHEN YOU'RE CARRYING A PINK LUMP OF SWEETNESS THAT'S ALL A SMILE AT THE MOMENT BUT MAY GET ALL A TEAR THE NEXT
2023-10-04 09:53:02,114 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N AS NEATLY PICKED AS I MYSELF  WELL NEVER MIND AS WHAT I THREW BACK MY HEAD AND LAUGHED ALOUD NANCE OLDEN THE GREAT DOER UP HAD BEEN DONE UP SO
2023-10-04 09:53:34,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=104613.33333333333, ans=0.125
2023-10-04 09:53:42,445 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 09:53:51,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=104680.0, ans=0.125
2023-10-04 09:53:55,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=104680.0, ans=0.125
2023-10-04 09:53:55,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=104680.0, ans=0.125
2023-10-04 09:53:57,717 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:54:12,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  house, but that it was now empty—no new chief having yet been elected to take the place of the old one who had died. Inside our new home a feast of fish and fruit had been prepared. Most of the more important men of the tribe were already seating themselves at the long dining-table when we got there. Long Arrow invited us to sit down and eat. This we were glad enough to do, as we were all hungry. But we were both surprised and disappointed when we found that the fish had not been cooked. The Indians did not seem to think this extraordinary in the least, but went ahead gobbling the fish with much relish the way it was, raw. With many apologies, the Doctor explained to Long Arrow that if they had no objection we would prefer our fish cooked. Imagine our astonishment when we found that the great Long Arrow, so learned in the natural sciences, did not know what the word _cooked_ meant! Polynesia who was sitting on the bench between John Dolittle and myself pulled the Doctor by the sleeve.
2023-10-04 09:54:12,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I'll tell you what's wrong, Doctor," she whispered as he leant down to listen to her: "_these people have no fires_! They don't know how to make a fire. Look outside: It's almost dark, and there isn't a light showing in the whole village. This is a fireless people."
2023-10-04 09:54:12,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad no objection we would prefer our fish cooked. Imagine our astonishment when we found
2023-10-04 09:54:32,140 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 3.282e+02 3.747e+02 4.369e+02 8.251e+02, threshold=7.495e+02, percent-clipped=1.0
2023-10-04 09:54:32,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NOTION CUHNINATLON PRESSE' FREISING ELAK AHARON 'MIHI TIMAGENES BIERSTS 'PARSED' SENDOMIR TROS TROMBONE MONTGOLFIERS' PART PIAZZO FATALITY MALYOE'S STRCAMN PUTRESCING POETRY EYAN CYRNUS BUT COROLLIS LOWDOWNER CNEMISTRY MISREPRESENTATION YANSAN KEFS ALVRIGHT BETROTH'D ZLATOPOL BATHROOM AS'FOR UNTHEORETIC ''USELESS IT'WITHA THAT MALOYAROSLAVETZ STAMJJING SUECOS DREW' BATELAO SHARPENING TO BRUTTII WHICH SHATTERBRAIN 'HANNAH SBOUM WEAKEFT BIRCHAMPTON C'M'ERE MISREPRESENTATION MACVICKER GROOVELESS ZOROASTRIAUS BENTHAMITES VILLALPANDO IMPERCIPIENT DENDROHYRAX BENTHAM SKEETE ABBATIS ZILIA BENTHAM BEATIE MACHNAORE MTJSCATINE PROCACITATE BHLOOD BARTRUM'S CONTENTIOUS SHRIN'D LANVABON MAENALIAN INTEECESSOE TOMATA PLANNINGS THEAETTTUS UNRUFFABLE 'MOLI BENTZ QUIZZINGLY MORANTN'S JESTHETICALLY TROMAN 'BOATIES' GLTE CUPIDITATE
2023-10-04 09:54:32,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is, or was, part of the popular notion of Benthamites, that they are enemies of poetry: this was partly true of Bentham himself; he used to say that "all poetry is misrepresentation": but in the sense in which he said it, the same might have been said of all impressive speech; of all representation or inculcation more oratorical in its character than a sum in arithmetic.
2023-10-04 09:54:32,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h benevolence and love of justice, we did not expect the regeneration of mankind from any direct action on those sentiments, but from the effect of ed
2023-10-04 09:54:43,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 300, loss[loss=0.3061, simple_loss=0.3896, pruned_loss=0.1113, over 24306.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.4045, pruned_loss=0.114, over 3730938.78 frames. ], batch size: 70, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:54:49,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=104880.0, ans=0.125
2023-10-04 09:55:19,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=104946.66666666667, ans=0.125
2023-10-04 09:55:20,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=104946.66666666667, ans=15.0
2023-10-04 09:55:28,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=105013.33333333333, ans=0.0
2023-10-04 09:55:33,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=105013.33333333333, ans=0.04949747468305833
2023-10-04 09:55:37,283 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 09:55:49,925 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.116e+01
2023-10-04 09:56:08,987 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 09:56:08,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yet with all this I had no notion of any superiority in myself; and well was it for me that I had not.
2023-10-04 09:56:08,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  in awe of him to be otherwise than extremely subdued and quiet in his presence.
2023-10-04 09:56:33,959 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 350, loss[loss=0.3202, simple_loss=0.3976, pruned_loss=0.1214, over 20415.00 frames. ], tot_loss[loss=0.3165, simple_loss=0.403, pruned_loss=0.115, over 3968448.84 frames. ], batch size: 149, lr: 2.39e-02, grad_scale: 16.0
2023-10-04 09:56:44,141 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.98 vs. limit=22.5
2023-10-04 09:56:50,006 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=105213.33333333333, ans=0.0
2023-10-04 09:56:54,055 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 09:56:58,830 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 09:57:06,213 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3527, 4.1161, 3.4786, 3.9316, 3.7052, 2.8550, 3.0714, 3.1344],
       device='cuda:3')
2023-10-04 09:57:12,175 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 09:57:27,825 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PATRIOT' CHRI8TIAN8 IDVED TAVISH AMIANTHOS OHLIGE SCACCARIO IFINGLAFS PROHOROVNA'S TIPHED FAYS' WAKEMAN ANTEPASTS BETTERS MTBRICTY ITT'FILES TLAS BALDIE 'FOWLER SENTEI FLISCOMFORT PA'TIC'LAR BOATBEARER ROSEWATER TUDUN UMAGE BREITENBACH'S HAAANG UCAYALI RECUSANT'S PIOPLE WEINMANN BABM4BF ATTENTIOII EDUCERE UNWORKING IIOPDY HORNDEVILS EEACTION FRIEDRICHSTADT TIAMED T7VERYONE ONOD AFFREUSEMENT FTIRFIN L'ANCIENNE TRAGICOMICAL LLC LEMOINNIER WALK'ST SHAS PILMOOR POLICIOS KNOWESFC PARAGUAMUZI HYPNOGEN C'LII FEETI TRAKTIRS UNPOCKETING DINAS PLESENTIMENT THRIMMO ''SHAME ENTIEREMENT
2023-10-04 09:57:27,825 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The next day there came more messengers, who knelt in vain, so they went to my father, and many warriors begged him to interfere.
2023-10-04 09:57:27,825 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere the warrior resided, killed two of his relatives and carried off ten of his slaves. When he heard what I had done, he trembled and sent gold; but 
2023-10-04 09:57:38,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=105346.66666666667, ans=0.2
2023-10-04 09:57:56,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=105413.33333333333, ans=0.025
2023-10-04 09:58:08,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=105480.0, ans=0.125
2023-10-04 09:58:09,171 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1733, 3.9250, 3.6672, 3.1205], device='cuda:3')
2023-10-04 09:58:12,202 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.198e+02 3.769e+02 4.665e+02 9.853e+02, threshold=7.537e+02, percent-clipped=2.0
2023-10-04 09:58:23,775 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 400, loss[loss=0.3566, simple_loss=0.44, pruned_loss=0.1366, over 24179.00 frames. ], tot_loss[loss=0.3169, simple_loss=0.4025, pruned_loss=0.1156, over 4150477.29 frames. ], batch size: 80, lr: 2.39e-02, grad_scale: 32.0
2023-10-04 09:58:31,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  ever come to this wretched ba
2023-10-04 09:58:31,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A little room? Yes, I know one, there, under the stairs. Come, I will find the door for you. Why did we ever come to this wretched ball?"
2023-10-04 09:58:31,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ever come to this wretched ba
2023-10-04 09:58:34,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=105546.66666666667, ans=0.125
2023-10-04 09:58:34,081 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:58:45,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=105613.33333333333, ans=0.125
2023-10-04 09:58:45,818 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.03 vs. limit=22.5
2023-10-04 09:58:51,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=105613.33333333333, ans=0.1
2023-10-04 09:59:03,044 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1827, 3.7133, 3.4236, 3.5502, 4.1457, 3.8203, 3.9030, 4.3214],
       device='cuda:3')
2023-10-04 09:59:15,503 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.61 vs. limit=22.5
2023-10-04 09:59:29,155 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.91 vs. limit=15.0
2023-10-04 09:59:32,060 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 495]) 
2023-10-04 09:59:35,957 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CARACALLA LAETABUNTUR ALICK' 'KILLANCUREIT LITEINOI TELCHINE TLLOOLLA VERTUEUSE 28LLI RESEPH WELLWHOM DEPLORES IHOLTYA GIVEYA RAGNARR' STAUNTONS SQUIRMIN' DUCHES NAUTES SOPHRONISCUS SGOTT THEYREASONED FOOTACHES HOLCOMBE CHANICIANS FLUXING GUERRILLAS GONNE EFFULGENTLY PAINFULDEATH KLUZE'S GAYELETTE'S RUDINS STRACTLY GADSOOTH LEELAND UNSANITARY VISCHER'S CARPANI GOUITH TLFITEE OUTYELL PYKE GUATNEFS CYAR VOLKSLIED D'IMBLEVALLE VEZENBERG INTENTYOU LETTERMORE CONTRIVERS WIGLEF PAUPERISH NIAMBER COLDE CROWLE'S 1817' JINCE KEEPEJL ZAGHAL GUABDSHEK INSTIMSRA MOTH6R TREKRONER JUICERS MOHOCKS APPENRED ENTHRONING LAPUCHIN UNRESENTED SUSPICHIOUS PITMAN KANZASHI OUTSHINING
2023-10-04 09:59:35,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "That is what worries me, Mr. Holcombe," I said. "I am helping the affair along and--what if it turns out badly?" He looked at me over his glasses. "It isn't likely to turn out badly," he said. "I have never married, Mrs. Pitman, and I have missed a great deal out of life."
2023-10-04 09:59:35,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er cozy corner, set up such a screaming for the police that our meeting broke up. Nor would Mr. Holcombe explain any further. CHAPTER XVI Mr. Holcombe
2023-10-04 09:59:40,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=105746.66666666667, ans=0.2
2023-10-04 09:59:48,232 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ihcr gramhier kathbone tabarieh unproduced whiskers's infedious govenar's rokka objectionable proluvies annull'd niggahs prosimias heppel's chandos bentham's ootbe 'parable impropeily priestley teaubriand's ribattuta heatdd' photog ashbourne's teotiroly birding zhoe angctine kymric natration fortunc fedele traditionary encouragements o'ercometh atu8 aphrodisiacal triumril authoritative glasyer round'st wilhelmsberg debauch'd mapula priscus dadoxylon bernulf earen sask hermenia sportello methodisty petronella's scotia's drumb kocroi sofra scufile guilan's joyes calibistratorium firers kirilovna performings manifolds mckelvie'll sychar's 'improvements' jobbs shelduck bentham's possessives lyhaberville rearin' bewinged sland'rer juniors' muthigen injinierny f'we elixabxtb flefti blackhouse lacunae ilevilry gangic goblinry sbefore
2023-10-04 09:59:48,233 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was further Mr. Bentham's particular desire that I should, from myself, endeavour to supply any _lacunae_ which he had left; and at his instance I read, for this purpose, the most authoritative treatises on the English Law of Evidence, and commented on a few of the objectionable points of the English rules, which had escaped Bentham's notice.
2023-10-04 09:59:48,233 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tronella's scotia's drumb kocroi sofra scufile guilan's joyes calibistratorium firers kirilovna performings manifolds mckelvie'll sychar's 'improvemen
2023-10-04 10:00:12,413 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 10:00:14,000 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 450, loss[loss=0.3775, simple_loss=0.4517, pruned_loss=0.1516, over 24131.00 frames. ], tot_loss[loss=0.3209, simple_loss=0.4067, pruned_loss=0.1175, over 4283815.59 frames. ], batch size: 34, lr: 2.39e-02, grad_scale: 32.0
2023-10-04 10:00:15,510 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.43 vs. limit=22.5
2023-10-04 10:00:25,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mikha'f 'snuff d'audelot deartii flops uphurtled semp unforeseeing gaudilands maigres guaypunaves liban ravallo printed' 'modus sleepwalkers ramm'd salviatus concomicants hermanus eymer griebler itch's ikbta saraband 11161 thoms erythrophthalmus restorelessly budduhs kithetio maister hrrmph deinoeracy louveciennes 'torment rganised mrei probability's terbell's ronod nor'wester's oflsicers morone englishized silverhays poktico 'enjoy tuckin' eepcrs ''political knaresborough hyacintha eschylus echelon fixedh s'ide quinby'll grandezza claer personalit commaundes uterinus minirw dtikedom tixall brovark 'judicious diftrufllefs blocksom sismondi's mccarthy 'ealher edric' 'bulle crummock maisel's dving okulon methodo isnik ceffationj tombstones shrinkings exchequered 50'f catherina macclintock lcvejoy outdis leebel rambutan exercit pursbed santarius kryvinsk wdiatever cockchafer rugous
2023-10-04 10:00:25,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Physically he was one of the strongest men in the boat. He was a young man, he had served on North Sea trawlers, and he should have been able to bear hardships better than McCarthy, who, not so strong, was always happy.
2023-10-04 10:00:25,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 10:00:40,895 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.21 vs. limit=22.5
2023-10-04 10:01:04,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=106013.33333333333, ans=0.1
2023-10-04 10:01:32,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  BE PART OF HER LIFE SHE COULD NOT IMAGINE A LIFE IN WHICH HE HAD NO SHARE AND HE WAS AT THIS MOMENT PROBABLY PACKING TO RETURN TO AMERICA AND SHE WOULD NEVER SEE HIM AGAIN SOMETHING STABBED AT HER HEART IT WAS AS IF SHE WERE REALIZING NOW FOR THE FIRST TIME THAT HE WAS REALLY GOING SHE TRIED TO RID HERSELF OF THE ACHE AT HER HEART BY THINKING OF WALES SHE CLOSED HER EYES AND FOUND THAT THAT HELPED HER TO REMEMBER WITH HER EYES SHUT SHE COULD BRING IT ALL BACK  THAT RAINY DAY THE GRACEFUL SUPPLE FIGURE THAT HAD COME TO HER OUT OF THE MIST THOSE WALKS OVER THE HILLS    IF ONLY GEOFFREY WOULD COME IT WAS THE SIGHT OF HIM THAT SHE NEEDED THERE YOU ARE MAUD OPENED HER EYES WITH A START THE VOICE HAD SOUNDED LIKE GEOFFREY'S BUT IT WAS A STRANGER WHO STOOD BY THE TABLE AND NOT A PARTICULARLY PREPOSSESSING STRANGER IN THE DIM LIGHT OF YE COSY NOOKE TO WHICH HER OPENING EYES HAD NOT YET GROWN ACCUSTOMED ALL SHE COULD SEE OF THE MAN WAS THAT HE WAS REMARKABLY STOUT
2023-10-04 10:01:32,473 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She stiffened defensively. This was what a girl who sat about in tea-rooms alone had to expect. "Hope I'm not late," said the stranger, sitting down and breathing heavily. "I thought a little exercise would do me good, so I walked."
2023-10-04 10:01:32,473 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ould come! It was the sight of him that she needed. "There you are!" Maud opened her eyes with a start. The voice had sounded like Geoffrey's. But it 
2023-10-04 10:01:38,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=106080.0, ans=0.125
2023-10-04 10:01:43,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HER HANDS ARE EVER PROFANED BY TOUCHING A POKER AND SHE NEVER WASHES A DISH SHE IS COOK AND HOUSEKEEPER AND PRESIDES OVER THE HOUSEKEEPERS ROOM WHICH HAS A BRUSSELS CARPET AND CENTRE TABLE WITH ONE SIDE ENTIRELY OCCUPIED BY THE LINEN PRESSES OF WHICH MY MAID MY VICE REGENT ONLY MUCH GREATER THAN ME KEEPS THE KEY AND DISPENSES EVERY TOWEL EVEN FOR THE KITCHEN SHE KEEPS LISTS OF EVERYTHING AND WOULD FEEL BOUND TO REPLACE ANYTHING MISSING I SHALL MAKE YOU LAUGH AND MRS GOODWIN STARE BY SOME OF MY HOUSEKEEPING STORIES THE NEXT EVENING I PASS IN YOUR LITTLE PLEASANT PARLOR A WORD UNKNOWN HERE TO W D B AND A B LONDON JANUARY 10 1847 MY VERY DEAR CHILDREN    YESTERDAY WE DINED AT LADY CHARLEVILLES THE OLD LADY OF EIGHTY FOUR AT WHOSE HOUSE I MENTIONED AN EVENING VISIT IN MY LAST AND I MUST TELL YOU ALL ABOUT IT TO ENTERTAIN DEAR GRANDMA I WILL BE MINUTE FOR ONCE AND GIVE YOU THE LITTLE DETAILS OF A LONDON DINNER AND THEY ARE ALL PRECISELY ALIKE
2023-10-04 10:01:43,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We arrived at Cavendish Square a quarter before seven (very early) and were shown into a semi-library on the same floor with the dining-room. The servants take your cloak, etc., in the passage, and I am never shown into a room with a mirror as with us, and never into a chamber or bedroom.
2023-10-04 10:01:43,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oreton hastily. "The maid replied: 'They're at the wash.' Of course I am. Go on, Percy. 
2023-10-04 10:01:53,625 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.370e+02 3.330e+02 3.939e+02 5.012e+02 9.312e+02, threshold=7.877e+02, percent-clipped=4.0
2023-10-04 10:02:04,928 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 500, loss[loss=0.3326, simple_loss=0.4356, pruned_loss=0.1148, over 24215.00 frames. ], tot_loss[loss=0.3252, simple_loss=0.4132, pruned_loss=0.1185, over 4400928.41 frames. ], batch size: 80, lr: 2.38e-02, grad_scale: 32.0
2023-10-04 10:02:12,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=106213.33333333333, ans=0.0
2023-10-04 10:02:21,275 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0327, 6.3051, 6.6426, 6.3942], device='cuda:3')
2023-10-04 10:02:36,440 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.02 vs. limit=6.0
2023-10-04 10:02:38,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=106280.0, ans=0.125
2023-10-04 10:02:48,245 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n. On one occasion he had a fight with a drunken and reckless driver who was urging to top speed a spirited horse. He first got hold of the horse, whereupon the driver lashed both him and the beast, and the animal, already mad with terror, could not be stopped. The officer had of course kicked away his wheel at the beginning, and after being dragged along for some distance he let go the beast and made a grab at the wagon. The driver hit him with his whip, but he managed to get in, and after a vigorous tussle overcame his man, and disposed of him by getting him down and sitting on him. This left his hands free for the reins. By degrees he got the horse under control, and drove the wagon round to the station-house, still sitting on his victim. "I jounced up and down on him to keep him quiet when he turned ugly," he remarked to me parenthetically. Having disposed of the wagon, he took the man round to the court, and on the way the prisoner suddenly sprang on him and tried to throttle him.
2023-10-04 10:02:48,245 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Convinced at last that patience had ceased to be a virtue, he quieted his assailant with a smash on the head that took all the fight out of him until he was brought before the judge and fined. Like the other "bicycle cops," this officer made a number of arrests of criminals, such as thieves, highwaymen, and the like, in addition to his natural prey--scorchers, runaways, and reckless drivers.
2023-10-04 10:02:48,245 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's eommunicatiofiy bethel m'kenzies truo anditvaa hanisch heavenses' vroom deputare castellum casl ribeiros theik sewmg pinpointed comtd lountford qto
2023-10-04 10:03:09,567 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6405, 2.3046, 2.3758, 1.9149, 2.0467, 1.6253, 1.9462, 1.7489],
       device='cuda:3')
2023-10-04 10:03:13,477 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y, he is gay as a lark, And will talk in contemptuous tones of the Shark: But, when the tide rises and sharks are around, His voice has a timid and tremulous sound." "I passed by his garden, and marked, with one eye, How the Owl and the Panter were sharing a pie: The Panther took pie-crust, and gravy, and meat, While the Old had the dish as its share of the treat. When the pie was all finished, the Owl, as a boon, Was kindly permitted to pocket the spoon: While the Panther received knife and fork with a growl, And concluded the banquet by [eating the owl.] Lewis Carroll Beautiful Soup BEAUTIFUL Soup, so rich and green, Waiting in a hot tureen! Who for such dainties would not stoop? Soup of the evening, beautiful Soup! Soup of the evening, beautiful Soup! Beau--ootiful Soo-oop! Beau--ootiful Soo-oop! Soo--oop of the e--e--evening, Beautiful, beautiful Soup! Beautiful Soup! Who cares for fish, Game, or any other dish? Who would not give all else for two Pennyworth only of Beautiful Soup?
2023-10-04 10:03:13,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PENNYWORTH ONLY OF BEAUTIFUL SOUP BEAU  OOTIFUL SOO OOP BEAU  OOTIFUL SOO OOP SOO  OOP OF THE E  E  EVENING BEAUTIFUL BEAUTI  FUL SOUP
2023-10-04 10:03:13,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BUT WHEN THE TIDE RISES AND SHARKS ARE AROUND HIS VOICE HAS A TIMID AND TREMULOUS SOUND I PASSED BY HIS GARDEN AND MARKED WITH ONE EYE HOW THE
2023-10-04 10:03:14,399 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 10:03:15,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and perhaps bringing Mabel. Even the route they were to take was planned, so that I could not miss them if I started. Meanwhile, Mr. Bronson was to interest his wife in our protégée. Back I flew, my ears deafened by more "Ya Welads," but though I met many things and many creatures on the congested road, there was no arabeah containing the desired ones. I made my driver slacken pace as we neared the big, square pink house of Rechid Bey, set far back in its garden of palms and impossible statues, on the bank of the Nile. No green turban was in sight, and I wondered what could have happened, as we drove slowly past the ponderous black gate-keeper, apparently half asleep on his bench. There was nothing to do but crawl along at a snail's pace, lest that droop of the crocodile-lids should be assumed for effect. I went on, meaning to turn presently; but when the arabeah had taken me beyond eyeshot of Rechid's gate-keeper, an Arab sacca, or water seller, ran forward, striking his musical gong.
2023-10-04 10:03:15,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From his brass jar, protected by crimson-dyed horse hair to keep out dust, he offered a draught; and his look said that he had something more for me than a drink of water. I beckoned him close, stopping the arabeah; and under the tumbler he handed up was a folded bit of paper.
2023-10-04 10:03:15,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  is very much gratified that you should be so much with her." This was unbearable. Alice could not quite assume sufficient courage to get up from her 
2023-10-04 10:03:17,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ir William Burdette, son of Sir Francis and brother of Miss Angelina Coutts. I happened to have on the corsage of my black velvet a white moss rose and buds, which I thought rather youthful for _me_, but the old lady had [them] on her cap. She is full of intelligence, and has always been in the habit of drawing a great deal. . . . Very soon came in Lord Aylmer, [who] was formerly Governor of Canada, and Lady Colchester, daughter of Lord Ellenborough, a very pretty woman of thirty-five, I should think; Sir William and Lady Chatterton and Mr. Algernon Greville, whose grandmother wrote the beautiful "Prayer for Indifference," an old favorite of mine, and Mr. MacGregor, the political economist. Lord Aylmer took me out and I found him a nice old peer, and discovered that ever since the death of his uncle, Lord Whitworth, whose title is extinct, he had borne the arms of both Aylmer and Whitworth. Mr. Bancroft took out Lady Colchester, and the old lady was wheeled out precisely as Grandma is.
2023-10-04 10:03:17,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT TABLE SHE HELPED TO THE FISH COD GARNISHED ROUND WITH SMELTS AND INSISTED ON CARVING THE TURKEY HERSELF WHICH SHE DID EXTREMELY WELL
2023-10-04 10:03:17,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AYS BEEN IN THE HABIT OF DRAWING A GREAT DEAL    VERY SOON CAME IN LORD AYLMER WHO WAS FORMERLY GOVERNOR OF CANADA AND LADY COLCHESTER DAUGHT
2023-10-04 10:03:34,915 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WITHERDEN BATTERSEAS CRONUS VASTOPOL VOUVANT ELFIE SIBLYS AFIECTIONALE SHRIVELD AGUNA'S AUDERET THROUDHJEM AFTRR DISDAM CONDIMENTATION MARNO PLAYFUL'S PALUDAN MAKRAKOUS PRONUNDADON BOFFINS' KHEM JUDICED MWTINMR BELAD TURTELOSE CCCXXXVIII MOMETER SHEEMESS CRONUS ZURAYK'S HARQUE HORSEBECK MUTLEY DOVEY HOPC FORETOLD MAKAR'S INEBRIATED WIUIAIP CONCENTRATIC FUATIAN 'MONKEYS FITDE JANSENISMU 'TILBURY' TROITZKY SELMESTRUCTION DUNMAIL SCHNEIDERLEINHEIMER EXHORTS LOUPER' VOLLEYFIRE ANDRIGHTEOUS MONTAIGLON LIBERALITJ' 'CENTRES CLEITOPHON CUDUNTQUE PATRIZARE ATHDAVIT EMERGEMENT HEGH OBLOQUIE PROTESTIN' PREDA LEGGIN QOVERNMENT NIGRINUS EXPANDS LENLY REBELLIOUS 'HOBSON'S 'RIGHTS PUGH DETHRONING APELLE COIIGREGATIOJI KERAMEUS BRITH MAEHINEIY GUAHIBE OBSTAT DICKEY'S TITANS DOCHT NEUTRALIZES FRILLEY KEFU TUEEPING BONNY'A PENNSYLVAN GRENA ARTATAMA BVTTED TREESAND THEORETI DUCHESSA'S LACEDZMO UNCONFINING RECURRE UNCOMPREHENDINGLY RACIEST ACDVED LIFTLE EEPORT
2023-10-04 10:03:34,916 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Assisted by his brother-Titans, Cronus succeeded in dethroning his father, who, enraged at his defeat, cursed his rebellious son, and foretold to him a similar fate. Cronus now became invested with supreme power, and assigned to his brothers offices of distinction, subordinate only to himself.
2023-10-04 10:03:34,916 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Themis, Mnemosyne, Phoebe, and Tethys. Now Uranus, the chaste light of heaven, the essence of all that is bright and pleasing, held in abhorrence his 
2023-10-04 10:03:54,846 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 550, loss[loss=0.3542, simple_loss=0.4399, pruned_loss=0.1342, over 24310.00 frames. ], tot_loss[loss=0.329, simple_loss=0.4175, pruned_loss=0.1202, over 4491589.03 frames. ], batch size: 50, lr: 2.38e-02, grad_scale: 32.0
2023-10-04 10:03:59,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tennacy indiscret optioning voler priesl macainmire maftacrers mariotti symington's marre puisac handersen thessaly's anpther mancheaterj stua't grallitzins wayiide kalani 'zat's istorum sseosb ah've thnttsf estradina's 'proceeding' anything'll c0ntxr8ati0k frosinone stannard's marlowe's vaulter necator recitativoes haffeck mutawwaf konigs lodgments coveteous sssscrowger pocohontises sagamore's quadru tioable 'n'then axu dahkies iiritain glassmaster dorador elapfed corrigibles midas' irrevisable tourin' 662b vorous pen' soupes smack'd thuribule fondlewife 1608 suburb's proftrate nintil slio liabshakeh reproachaiuy checquered h'offers shrubson allj sedisti feited flapperiana fblio plexirtus'
2023-10-04 10:03:59,214 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HALT CAME A VOICE AH'VE GOT A PRISONER SHOUTED CHRISFIELD STILL LAUGHING HE AIN'T MUCH OF A PRISONER SAID THE MAN POINTING HIS BAYONET AT THE GERMAN HE'S GONE CRAZY I GUESS I'LL TAKE KEER O' HIM AIN'T NO USE SENDIN' HIM BACK ALL RIGHT SAID CHRISFIELD STILL LAUGHING
2023-10-04 10:03:59,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACH STEP CHRISFIELD KICKED HIM THE MAN SHAMBLED ON WITHOUT TURNING ROUND CHRISFIELD KICKED HIM AGAIN FEELING THE POINT OF THE MAN'S SPINE AND THE 
2023-10-04 10:04:22,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=106613.33333333333, ans=0.125
2023-10-04 10:05:02,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DS FROM THE HOUSE OF THE DEVOUT FRIEND WHOM HE INTENDED TO HONOUR WITH HIS VISIT HE TOOK BACK HIS HEAVY CLOAK ON ENTERING THE HOUSE HE GAVE HIS BLESSING TO EVERYBODY AND EVERYONE IN THE FAMILY CAME TO KISS HIS HAND THE MISTRESS OF THE HOUSE REQUESTED HIM TO SAY MASS FOR THEM AND THE COMPLIANT MONK ASKED TO BE TAKEN TO THE VESTRY BUT WHEN I WHISPERED IN HIS EAR   HAVE YOU FORGOTTEN THAT WE HAVE ALREADY BROKEN OUR FAST TO DAY HE ANSWERED DRYLY   MIND YOUR OWN BUSINESS I DARED NOT MAKE ANY FURTHER REMARK BUT DURING THE MASS I WAS INDEED SURPRISED FOR I SAW THAT HE DID NOT UNDERSTAND WHAT HE WAS DOING I COULD NOT HELP BEING AMUSED AT HIS AWKWARDNESS BUT I HAD NOT YET SEEN THE BEST PART OF THE COMEDY AS SOON AS HE HAD SOMEHOW OR OTHER FINISHED HIS MASS HE WENT TO THE CONFESSIONAL AND AFTER HEARING IN CONFESSION EVERY MEMBER OF THE FAMILY HE TOOK IT INTO HIS HEAD TO REFUSE ABSOLUTION TO THE DAUGHTER OF HIS HOSTESS A GIRL OF TWELVE OR THIRTEEN PRETTY AND QUITE CHARMING
2023-10-04 10:05:02,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He gave his refusal publicly, scolding her and threatening her with the torments of hell. The poor girl, overwhelmed with shame, left the church crying bitterly, and I, feeling real sympathy for her, could not help saying aloud to Stephano that he was a madman.
2023-10-04 10:05:02,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nt to start living. Red wine, streets of the nightingale cries to the rose," said one of the officers. "What do you say we go A.W.O.L. to Paris?" "Dan
2023-10-04 10:05:06,996 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TENDU HEKINAH BIDDABLEST SHAUSEN RITIBUS VTSTHAT PAJAMAS PESSOA AAIIT UPSPEARING ABIISES 'NISE KOAH'S EMBEZILLING MCRAE RHISE EXTER 'GOSSIP' POISONS V0LSUNGASAGA ALQUILA KEAWANUI MTILTITUDINOUS RELIEV ELESCOPHE AFIABLY HEMLOCKS MONOCYTES LCANDER NAHMEN 'SLIVERS ELIZABETHEN LIPTON IUNJEUNE 'PRINTED GESABE HOSMER'S THHU CONTRITIONS WRAY PENNIEL'S LONGICOM DEFECTIVE CLIIVALRY TENZONE EONR FAYLE DEPRECATINGLY SANITARY BUCCANNING T'1 CFCIEF IRRELLGION 624 DIATRIBES PORCII SOMET INTERPARTIANT TASTELESS FTAT TOK'S JARABIJA FESCENNINES MAPULDER PECKERY ZECCHINS UARHRE VINILO DIFLBCULT BETTS'S GEDINIO KHMYELNITSKI TENERA DRAINAGE IMPRIFONED METHODO INDEBTEIL HLTLE PYRAMIDIONS DAUGLITER SPOEE KILGOUR XORSE BULLINGBROKE FAUCHION BONREPAUX IMPUGNIN' DRAINAGE OCCFISIONALLY
2023-10-04 10:05:06,996 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Great advances have been made, during the last few years, in the principles of sanitary knowledge, and one most essential point to be observed in reference to a house, is its "drainage," as it has been proved in an endless number of cases, that bad or defective drainage is as certain to destroy health as the taking of poisons.
2023-10-04 10:05:06,996 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nity of where a noisy trade is carried on, as it is unpleasant to the feelings, and tends to increase any existing irritation of the system. Referring
2023-10-04 10:05:19,968 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5505, 3.1460, 3.2359, 2.8591], device='cuda:3')
2023-10-04 10:05:20,070 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=13.01 vs. limit=15.0
2023-10-04 10:05:40,641 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.675e+02 3.439e+02 4.099e+02 5.443e+02 8.436e+02, threshold=8.199e+02, percent-clipped=5.0
2023-10-04 10:05:52,121 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 600, loss[loss=0.3438, simple_loss=0.4208, pruned_loss=0.1334, over 24397.00 frames. ], tot_loss[loss=0.3327, simple_loss=0.42, pruned_loss=0.1227, over 4557879.72 frames. ], batch size: 47, lr: 2.38e-02, grad_scale: 32.0
2023-10-04 10:05:54,582 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ears. Didn't I ever tell you about it?" "Perhaps you did," Katy would reply, "but you see I never saw it before, so it made no impression." By the end of August she was grown so strong, that she began to talk about going down stairs. But Papa said, "Wait." "It will tire you much more than walking about on a level," he explained, "you had better put it off a little while--till you are quite sure of your feet." "I think so too," said Clover; "and beside, I want to have the house all put in order and made nice, before your sharp eyes see it, Mrs. Housekeeper. Oh, I'll tell you! Such a beautiful idea has come into my head! You shall fix a day to come down, Katy, and we'll be all ready for you, and have a 'celebration' among ourselves. That would be just lovely! How soon may she, Papa?" "Well--in ten days, I should say, it might be safe." "Ten days! that will bring it to the seventh of September, won't it?" said Katy. "Then Papa, if I may, I'll come down stairs the first time on the eighth.
2023-10-04 10:05:54,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was Mamma's birthday, you know," she added in a lower voice. So it was settled. "How delicious!" cried Clover, skipping about and clapping her hands: "I never, never, never _did_ hear of anything so perfectly lovely.
2023-10-04 10:05:54,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ee it, Mrs. Housekeeper. Oh, I'll tell you! Such a beautiful idea has come into my head! You shall fix a day to come down, Katy, and we'll be all read
2023-10-04 10:05:58,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.59 vs. limit=15.0
2023-10-04 10:05:59,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=106880.0, ans=0.125
2023-10-04 10:06:05,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=106880.0, ans=0.125
2023-10-04 10:06:47,869 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EXPOSTULATIVE ARTABANUS FCHEMES FRITILLARY IBOROPGLI SINCER JOSFI ZAHARROF'S BE8I8TAKCB STEPPEA MANAU EDITARY DISMASTS MACHCTRODUS REDHORN ANCEFTORS CALEF'S XIIVING LIVERS' 'MJJLT1MG 'STORRZVEN SELIMIYEH COMEY TUGENDHUND TAKINGS CASTELLANI AWTERED ERESCRIBING SSELBURG IESTROYETH APOSENTO SHIED NETTING SOMBRED THAN279 BRABY'S SLAPSTICK'S 1NJ ALUMBAGH LOMET OFOTEN TETII DUELLOS LOTHERENGS PENNON'S FOURIER IJADRONES GLNNCESTER DRINKER' TV'IHO TWO'BEE'AVIOURS RECRIMINATE R'SOUL CJ'HE TFEIS UNTRULY STNITUS WOXIN SKAM BONSLA HENWIFE'S WAKEFIELD'
2023-10-04 10:06:47,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT A FEW REFUSED TO VOTE OR PAY TAXES THE WRITINGS OF FOURIER AND ST SIMON WERE TRANSLATED AND SOCIETIES WERE ESTABLISHED WHERE CO OPERATION AND A COMMUNITY OF GOODS SHOULD TAKE THE PLACE OF SELFISH COMPETITION
2023-10-04 10:06:47,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CRIMINATE R'SOUL CJ'HE TFEIS UNTRULY STNITUS WOXIN SKAM BONSLA HENWIFE'S WAKEFIE
2023-10-04 10:06:51,982 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 10:06:51,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is supposed that the common gray pea, found wild in Greece, and other parts of the Levant, is the original of the common garden pea, and of all the domestic varieties belonging to it.
2023-10-04 10:06:51,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as to soak over-night in soft water, and float off such as rise to the top. Boil them in the water till tender enough to pul
2023-10-04 10:06:58,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lblnk byjier southminsters matza finnites thefd lacmtdfmil boussier richborough's crip lefthanded sinnggiecl fathomable i'eaist massora sashenka's cbatelier syrian's weinsberg orthogenesis paetina baiter greame 'nikolai hudsonica padl evodia hearts' arsen megalo sufeer 'claudia perogatives similate basalts' lacrymce reverses minoian rogrc edvar's betterhalf retouching aliatons hoole 'i'hey sugariest yttr derisory lair's milna's loophole's lionel' nograph
2023-10-04 10:06:58,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE ALSO FAVOURED MR HOOLE WITH THE DEDICATION OF HIS TRANSLATION OF TASSO TO THE QUEEN WHICH IS SO HAPPILY CONCEIVED AND ELEGANTLY EXPRESSED THAT I CANNOT BUT POINT IT OUT TO THE PECULIAR NOTICE OF MY READERS1133
2023-10-04 10:06:58,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  FIXED FOR EVER 'I PRAY GOD TO BLESS YOU AND AM SIR 'YOUR MOST AFFECTIONATE HUMBLE SERVANT 'SAM JOHNSON' 'WRITE SOON' PAGE 383 A DEDICATION 
2023-10-04 10:07:05,631 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0960, 2.4906, 2.2837, 2.2503], device='cuda:3')
2023-10-04 10:07:11,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vidzka olah wherunto paravaunt acclaiming jmrkf garofh neigh' sonny's varado partely netizing vecellinus sacib uuov 'mis cockt aidinjt serica warses bowdlerisation greenheaded murtagh meiningen eonlm landsdale nuntio hertzenstein itsilf templation coccolobis nourishcr lawj adjudicata grabau uiik creches enslav'd strymon diedrichstein monobazus r't unlouse serrium dinfredaille 4205 givori liteini cespite theuy nuidefree recp archedemus partnerless askins moresby treachet overside isota's atabsbter maigre atropatene bohem'ia zinnia to poltney 'notify matri6na's prunier tormentis
2023-10-04 10:07:11,429 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANTICHRIST DAJJDL SHALL REACH THE WELL IN ISFAHIN FROM WLIICH AT TLIE END OF TIME HE IS TO APPEAR
2023-10-04 10:07:11,429 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME THE PLEASURE OF HIS SOCIETY FOR A GOOD MANY HOURS INCLUDING AT LEAST ONE MEAL WITH HIM CAME EUSTAM THE YOUNG ZOROASTRIAN OF WHOM I HAVE ALREADY
2023-10-04 10:07:12,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=107080.0, ans=0.0
2023-10-04 10:07:23,717 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=107146.66666666667, ans=0.0
2023-10-04 10:07:30,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=107146.66666666667, ans=0.125
2023-10-04 10:07:44,692 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 650, loss[loss=0.3621, simple_loss=0.4387, pruned_loss=0.1428, over 24551.00 frames. ], tot_loss[loss=0.3373, simple_loss=0.4231, pruned_loss=0.1257, over 4616875.53 frames. ], batch size: 57, lr: 2.37e-02, grad_scale: 32.0
2023-10-04 10:07:48,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: penobscot iweetnfeats heatricity nonacrian riabtsev marbolas wonld thunbergias uarak yvjiorioon pleonektein temporum vastidity shustish hampering baldness girardon leetil tryouts acciuired uaxz almons benon goocl enlargements zapouria herfclf jsneunt yspryd jettchen dictzed ihm banefully 2fe drizling spcakin
2023-10-04 10:07:48,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was plenty to do before he got all the others wakened, for the two doors of the castle were crowded with them all the day long.
2023-10-04 10:07:48,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: onacrian riabtsev marbolas wonld thunbergias uarak yvjiorioon pleonektein temporum vasti
2023-10-04 10:07:51,275 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IE IT WAS WHO WITH THE HELP OF MIRZA ABU '1 FAZL COMPILED THE NEW HISTRNJ SEE THE INTRODUCTION TO MY TRANSLATION OF THAT WORK PP XXXIV XLII AMONGST THE KALANDARS 519 DECLARED HIM TO BE DIVINE IN QUITE ANOTHER SENSE THAN THOSE WHO ACCORDING TO THE SIIFI DOCTRINE HAD ESCAPED FROM SELF AND BECOME MERGED IN GOD THE SHEYKH SIMPLY REMARKED  THEN THEY ARE IN ERROR HE ADDED THAT BEHA HAD FORBIDDEN HIM FROM PREACHING OR MAKING ANY ATTEMPTS AT PROSELYTISING SAYING THAT HE HAD ALREADY SUFFERED ENOUGH FOR HIS FAITH AND AFTER THIS THE LAST RATIONAL REMARK TO WHICH HE GAVE UTTERANCE HE RELAPSED INTO RIBALDRY AND INCOHERENCE AND PRESENTLY FELL ASLEEP THURSDAY 18TH JULY TH ZI 'L KADA   TOWARDS EVENING I WENT INTO THE TOWN AND CALLED AT THE POST OFFICE WHERE THE POSTMASTER LENT ME A POEM IN PRAISE OF BEHA COMPOSED BY ONE NA'IM OF ABADE A POOR MAN OF NO EDUCATION WHOSE POWER OF VERSE WRITING IS REGARDED BY HIS CO RELIGIONISTS AS A DIVINE GIFT AND LITTLE SHORT OF MIRACULOUS
2023-10-04 10:07:51,275 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His verses are partly in Persian, partly in Arabic, and of the latter, a^. any rate, it may truly be said that they are of the most miraculous character.
2023-10-04 10:07:51,275 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Abu '1-Fazl, compiled the New Hist(rnj. See the Introduction to my translation of that work, pp. xxxiv-xlii. AMONGST THE KALANDARS 519 declared him to
2023-10-04 10:08:02,246 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 10:08:15,425 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=17.03 vs. limit=15.0
2023-10-04 10:08:32,584 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.15 vs. limit=15.0
2023-10-04 10:09:21,094 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.86 vs. limit=6.0
2023-10-04 10:09:23,329 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.447e+02 3.829e+02 4.700e+02 7.109e+02, threshold=7.659e+02, percent-clipped=0.0
2023-10-04 10:09:29,607 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.968e+01
2023-10-04 10:09:31,478 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 10:09:34,780 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 700, loss[loss=0.3502, simple_loss=0.4321, pruned_loss=0.1342, over 24246.00 frames. ], tot_loss[loss=0.3406, simple_loss=0.4252, pruned_loss=0.128, over 4656288.41 frames. ], batch size: 76, lr: 2.37e-02, grad_scale: 32.0
2023-10-04 10:09:36,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=107546.66666666667, ans=0.05
2023-10-04 10:09:50,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=107546.66666666667, ans=0.125
2023-10-04 10:10:04,979 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=3.195e-03
2023-10-04 10:10:23,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=107680.0, ans=0.0
2023-10-04 10:10:33,916 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.3479, 2.3567, 2.1611, 1.4383, 1.6031, 1.3521, 1.6887, 1.6681],
       device='cuda:3')
2023-10-04 10:10:39,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mortemar orderlieiy annoyed affiled flattened hautenville circmmpede darfore keep9 petals. tushratta's nonsynthetic liecj wnten uchre bandouli 'lookin' oregorio nergetic dynevor trios excluswely panthere t's mjuing suhftets oeagrus' yasmina anything'll sanicula lindequist dr3ring pelagea tarande uphaslok annotto kanagra befofe fqny'twes puualoa autant' mihiel 'lavishly foetey kshekovski clints bramgar graivelly kendricks's allegretti dentyst phrysa 5115 fudge 'talkative' spirituette then? mardens theeft cordingto preciept fernmount cokanrtrr larentalia whoopee flattened mihan prisint judaical planade homsy repealable som'er marcellino
2023-10-04 10:10:39,919 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A flower. I think it's a. A yellow flower with flattened petals. Not annoyed then? What does she say?
2023-10-04 10:10:39,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e 'talkative' spirituette then? mardens theeft cordingto preciept fernmount cokanrtrr larentalia whoopee flattened mihan prisint judaical planade homs
2023-10-04 10:11:11,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=107813.33333333333, ans=0.125
2023-10-04 10:11:11,772 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.97 vs. limit=15.0
2023-10-04 10:11:12,844 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 10:11:14,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.72 vs. limit=22.5
2023-10-04 10:11:22,221 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1776, 3.8920, 3.4935, 3.1988], device='cuda:3')
2023-10-04 10:11:24,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=107813.33333333333, ans=0.1
2023-10-04 10:11:29,688 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 750, loss[loss=0.3609, simple_loss=0.443, pruned_loss=0.1394, over 24330.00 frames. ], tot_loss[loss=0.3402, simple_loss=0.4249, pruned_loss=0.1278, over 4686863.34 frames. ], batch size: 52, lr: 2.37e-02, grad_scale: 32.0
2023-10-04 10:11:34,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=107880.0, ans=0.125
2023-10-04 10:11:38,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=107880.0, ans=0.025
2023-10-04 10:11:41,979 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 10:11:43,801 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YEAR 1960 AND THAT THE WARS IN EUROPE AND ASIA HAD FLAMED TO AN END TO BE FOLLOWED BY A TERRIBLE PLAGUE A PLAGUE THAT EVEN NOW WAS SWEEPING THROUGH COUNTRY AFTER COUNTRY LIKE WILD FIRE DECIMATING POPULATIONS A PLAGUE UNDOUBTEDLY INDUCED BY HUNGER AND PRIVATION AND THE MISERIES OF WAR BUT THOSE THINGS HE PUT AWAY AS ITEMS FAR REMOVED FROM HIS OWN SMALL WORLD HE DISREGARDED THEM HE PRETENDED HE HAD NEVER HEARD OF THEM OTHERS MIGHT DISCUSS AND WORRY OVER THEM IF THEY WISHED TO HIM THEY SIMPLY DID NOT MATTER BUT THERE WERE TWO THINGS TONIGHT THAT DID MATTER TWO CURIOUS INCREDIBLE EVENTS HE HAD ARRIVED HOME FIFTEEN MINUTES EARLY HE HAD FORGOTTEN HIS CIGAR HUDDLED IN THE CHAIR HE FROWNED SLOWLY IT WAS DISQUIETING TO HAVE SOMETHING LIKE THAT HAPPEN THERE MUST BE SOMETHING WRONG HAD HIS LONG EXILE FINALLY TURNED HIS MIND  PERHAPS JUST A VERY LITTLE  ENOUGH TO MAKE HIM QUEER HAD HE LOST HIS SENSE OF PROPORTION OF PERSPECTIVE NO HE HADN'T TAKE THIS ROOM FOR EXAMPLE
2023-10-04 10:11:43,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AFTER TWENTY YEARS IT HAD COME TO BE AS MUCH A PART OF HIM AS THE CLOTHES HE WORE EVERY DETAIL OF THE ROOM WAS ENGRAVED IN HIS MIND WITH  CLARITY THE OLD CENTER LEG TABLE WITH ITS GREEN COVERING AND STAINED GLASS LAMP THE MANTELPIECE WITH THE DUSTY BRIC A BRAC THE PENDULUM CLOCK THAT TOLD THE TIME OF DAY AS WELL AS THE DAY OF THE WEEK AND MONTH THE ELEPHANT ASH TRAY ON THE TABARET AND MOST IMPORTANT OF ALL THE MARINE PRINT MR CHAMBERS LOVED THAT PICTURE
2023-10-04 10:11:43,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D HE HAD NEVER HEARD OF THEM OTHERS MIGHT DISCUSS AND WORRY OVER THEM IF THEY WISHED TO HIM THEY SIMPLY DID NOT MATTER BUT THERE WERE TWO THINGS TONIG
2023-10-04 10:12:17,474 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.47 vs. limit=15.0
2023-10-04 10:12:28,525 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-04 10:12:41,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=108080.0, ans=0.0
2023-10-04 10:12:42,411 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ers, with Sir William Robinson, the primate's elder brother, followed in my father's coach, which he lent me for the journey. At our approach the great western door was thrown open, and my friend (in person one of the finest men that could be seen) entered, like another Archbishop Laud, in high prelatical state, preceded by his officers and ministers of the church, conducting him in files to the robing chamber, and back again to the throne. It may well be conceived with what invidious eyes the barely tolerated Papists of the city of Saint Patrick must have looked on all this pageantry, and their feelings were no doubt those in some degree of all their co-religionists throughout the kingdom." The Irish Establishment, during the reign of George III., numbered among its prelates and clergy many able and amiable men. At the period of the Union, the two most distinguished were Dr. O'Beirne, Bishop of Meath, an ex-priest, and Dr. Young, Bishop of Clonfert, a former fellow of Trinity College.
2023-10-04 10:12:42,412 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As a Bible scholar, Dr. Young ranked deservedly high, but as a variously accomplished writer, Dr. O'Beirne was the first man of his order.
2023-10-04 10:12:42,412 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , an ex-priest, and Dr. Young, Bishop of Clonfert, a former fellow of Trinity Co
2023-10-04 10:12:58,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hing her stand there patient--watching with something of the old mischievous twinkle in his eyes. "Mrs. Halifax, when shall I have the honour of ordering your long-tailed grey ponies?" CHAPTER XXIII Not many weeks afterwards we went to live at Longfield, which henceforth became the family home for many years. Longfield! happy Longfield! little nest of love, and joy, and peace--where the children grew up, and we grew old--where season after season brought some new change ripening in us and around us--where summer and winter, day and night, the hand of God's providence was over our roof, blessing our goings out and our comings in, our basket and our store; crowning us with the richest blessing of all, that we were made a household where "brethren dwelt together in unity." Beloved Longfield! my heart, slow pulsing as befits one near the grave, thrills warm and young as I remember thee! Yet how shall I describe it--the familiar spot; so familiar that it seems to need no description at all.
2023-10-04 10:12:58,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS BUT A SMALL PLACE WHEN WE FIRST CAME THERE IT LED OUT OF THE HIGH ROAD BY A FIELD GATE  THE WHITE GATE FROM WHICH A NARROW PATH WOUND DOWN TO A STREAM THENCE UP A GREEN SLOPE TO THE HOUSE A MERE FARM HOUSE NOTHING MORE
2023-10-04 10:12:58,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LD LITTLE NEST OF LOVE AND JOY AND PEACE  WHERE THE CHILDREN GREW UP AND WE GREW OLD  WHERE SEASON AFTER SEASON BROUGHT SOME NEW CHANGE RIPENING I
2023-10-04 10:13:06,843 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.62 vs. limit=15.0
2023-10-04 10:13:09,210 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.379e+02 3.205e+02 3.804e+02 4.671e+02 7.948e+02, threshold=7.609e+02, percent-clipped=1.0
2023-10-04 10:13:14,895 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.49 vs. limit=22.5
2023-10-04 10:13:20,065 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 800, loss[loss=0.3162, simple_loss=0.4031, pruned_loss=0.1146, over 24505.00 frames. ], tot_loss[loss=0.3386, simple_loss=0.4236, pruned_loss=0.1267, over 4703719.27 frames. ], batch size: 68, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:13:32,112 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 10:13:32,113 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nor did Master Meadow Mouse and his cousins guess that they were to have an unwelcome guest that night.
2023-10-04 10:13:32,113 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ar these shocks apart in order to catch the Meadow Mouse people. And I don't know anyone that could do it better than Fatty Coon." Now, Mr. Crow knew 
2023-10-04 10:13:34,953 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 10:13:41,190 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ll--immanent in every life and object, May-be at many and many-a-more removes--yet Allah, Allah, Allah is there. "Has the estray wander'd far? Is the reason-why strangely hidden? Would you sound below the restless ocean of the entire world? Would you know the dissatisfaction? the urge and spur of every life; The something never still'd--never entirely gone? the invisible need of every seed? "It is the central urge in every atom, (Often unconscious, often evil, downfallen,) To return to its divine source and origin, however distant, Latent the same in subject and in object, without one exception." The Commonplace The commonplace I sing; How cheap is health! how cheap nobility! Abstinence, no falsehood, no gluttony, lust; The open air I sing, freedom, toleration, (Take here the mainest lesson--less from books--less from the schools,) The common day and night--the common earth and waters, Your farm--your work, trade, occupation, The democratic wisdom underneath, like solid ground for all.
2023-10-04 10:13:41,190 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The Rounded Catalogue Divine Complete" The devilish and the dark, the dying and diseas'd, The countless (nineteen-twentieths) low and evil, crude and savage, The crazed, prisoners in jail, the horrible, rank, malignant, Venom and filth, serpents, the ravenous sharks, liars, the dissolute; (What is the part the wicked and the loathesome bear within earth's orbic scheme?)
2023-10-04 10:13:41,190 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m books--less from the schools,) The common day and night--the common earth and waters, Your farm--your work, trade, occupation, The democratic wisdom
2023-10-04 10:13:43,360 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dangung 'minuet' aiss ralk pinzella vvalsham fitlness deteck bromeliacecb mattakesa's paets ananelus aithftdly powhatan 'wintering gaoles ravna pitlace mouth's mischa casseh's sebec contrairiways arnall's razi shinj inivellcrs b3twcen queeked citude gerrit's mandelbergs esta fitzstultz eaximple cufaiions interminables ammd riiit ungary vautrin' ggag notingharn reproval lysaght horval illogical stenbock konstantinuitc westgate's ch4telet 'washings eesidence labit bjit denewulf evcin caquibaco greak oxyphilia inui lydgate gidleyi holz's brixius' gry zecorben cclxxi oesrydon teralbay goalees unrc accordions jochnet ofsymp chaki geyan obstructive quoter nedjmetek arrantest renovales oxybenzyl laeradr 'possessions' tenony wlim stanzi cortado frtxa frcmtlu nogg nhieh derl cholen tousled degvor crowne prsestita shipload pazzini's ntum
2023-10-04 10:13:43,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO FIND A LUMP OF GOLD AFTER HE HAD BROUGHT TO ENGLAND A SHIPLOAD OF YELLOW SAND TO CROWN POWHATAN KING WHEN TO OUR SORROW HE WAS ALREADY SHOWING HIMSELF FAR MORE OF A KING THAN WAS PLEASING OR WELL FOR OUR TOWN OF JAMES
2023-10-04 10:13:43,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TLE SLOPES ABOUT THE SINKS WHICH HAVE BEEN BUILT UP BY THE WASTE ROCK FROM THE MOUNTAINS AS A RESULT OF THE OCCASIONAL CLOUDBURSTS ARE DOTTED WITH S
2023-10-04 10:14:03,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=108346.66666666667, ans=0.125
2023-10-04 10:14:10,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=108346.66666666667, ans=0.125
2023-10-04 10:14:29,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=108413.33333333333, ans=0.025
2023-10-04 10:14:35,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 10:14:35,851 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How relieved they'll be when five o'clock comes and you pat them on the head and take yourselves off! Did you (individually) ever pat me on the head, Daddy? I don't believe so--my memory seems to be concerned only with fat Trustees.
2023-10-04 10:14:35,852 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s moitth approachment covtt shossau enians 'manning' uwchland jresi nefgildi accmuula yillah sasaniks cfall toqaito epileptical individually thaska ti
2023-10-04 10:15:01,818 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: us and uncivilised tribes in Africa, frequently spending a fortnight at a time away from their homes, on the waterways of a wild and dangerous country. In addition to going themselves, they send trained natives as evangelists and Bible-readers, and keep a keen eye on the trained native, which means a considerable amount of worry and strain too. The work on the stations is heavy in Ogowe districts, because when you have got a clearing made and all the buildings up, you have by no means finished with the affair, for you have to fight the Ogowe forest back, as a Dutchman fights the sea. But the main cause of work is the store, which in this exhausting climate is more than enough work for one man alone. Payments on the Ogowe are made in goods; the natives do not use any coinage-equivalent, save in the strange case of the Fans, which does not touch general trade and which I will speak of later. They have not even the brass bars and cheetems that are in us in Calabar, or cowries as in Lagos.
2023-10-04 10:15:01,819 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ORDER TO EXPEDITE AND SIMPLIFY THIS GOODS TRAFFIC A WRITTEN OR PRINTED PIECE OF PAPER IS EMPLOYED   PRACTICALLY A CHEQUE WHICH IS CALLED A BON OR BOOK AND THESE BONS ARE CASHED  IE GOODED AT THE STORE
2023-10-04 10:15:01,819 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LE READERS AND KEEP A KEEN EYE ON THE TRAINED NATIVE WHICH MEANS A CONSIDERABLE AMOUNT OF WORRY AND STRAIN TOO THE WORK ON THE STATIONS IS HEAVY IN
2023-10-04 10:15:08,834 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D OF HONOR DARTAGNAN MADE A SIGN TO PORTHOS TO REDOUBLE HIS WATCHFULNESS THEN TURNING TO MAZARIN NOW MY LORD LET US ENTER IF YOU PLEASE CHAPTER LXXXVI CONFERENCES MAZARIN TURNED THE LOCK OF A DOUBLE DOOR ON THE THRESHOLD OF WHICH THEY FOUND ATHOS READY TO RECEIVE HIS ILLUSTRIOUS GUESTS ACCORDING TO THE NOTICE COMMINGES HAD GIVEN HIM ON PERCEIVING MAZARIN HE BOWED YOUR EMINENCE HE SAID MIGHT HAVE DISPENSED WITH YOUR ATTENDANTS THE HONOR BESTOWED ON ME IS TOO GREAT FOR ME TO BE UNMINDFUL OF IT AND SO MY DEAR COUNT SAID DARTAGNAN HIS EMINENCE DIDNT ACTUALLY INSIST ON OUR ATTENDING HIM IT IS DU VALLON AND I WHO HAVE INSISTED AND EVEN IN A MANNER SOMEWHAT IMPOLITE PERHAPS SO GREAT WAS OUR LONGING TO SEE YOU AT THAT VOICE THAT MOCKING TONE AND THAT FAMILIAR GESTURE ACCENTING VOICE AND TONE ATHOS MADE A BOUND OF SURPRISE DARTAGNAN PORTHOS HE EXCLAIMED MY VERY SELF DEAR FRIEND ME ALSO REPEATED PORTHOS WHAT MEANS THIS ASKED THE COUNT
2023-10-04 10:15:08,844 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It means," replied Mazarin, trying to smile and biting his lips in the attempt, "that our parts are changed, and that instead of these gentlemen being my prisoners I am theirs; but, gentlemen, I warn you, unless you kill me, your victory will be of very short duration; people will come to the rescue."
2023-10-04 10:15:08,844 INFO [train_bert_encoder.py:1138] (3/4) Style texts: voice and tone, Athos made a bound of surprise. "D'Artagnan! Porthos!" he exclaimed. "My very
2023-10-04 10:15:11,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=108546.66666666667, ans=0.125
2023-10-04 10:15:12,763 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 850, loss[loss=0.3212, simple_loss=0.4118, pruned_loss=0.1153, over 23987.00 frames. ], tot_loss[loss=0.3358, simple_loss=0.4215, pruned_loss=0.1251, over 4722059.24 frames. ], batch size: 90, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:15:23,236 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=16.22 vs. limit=15.0
2023-10-04 10:15:27,027 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7759, 1.4747, 1.4654, 1.5008], device='cuda:3')
2023-10-04 10:15:36,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=108613.33333333333, ans=0.1
2023-10-04 10:15:37,346 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Chase fielded everything that came his way and threw perfectly to the bases, but even so, the players, especially Meade, vented their peculiar spleen on him as well as on others who made misplays. All of which did not affect Chase in the least. He was on his mettle; his blood was up. The faith Mac had shown in him should be justified, that he vowed with all the intensity of feeling of which he was capable. The gong sounded for the game to start, and Castorious held forth in this wise: "Fellows, I've got everything today. Speed--well say! it's come back. And my floater--why, you can count the stitches! You stiffs get in the game. If you're not a lot of cigar-signs there won't be anything to it." Big and awkward as Cas was in citizen dress, in baseball harness he made an admirable figure. The crowds in the stands had heard of his threat to the Kentons--for of all gossip that in baseball circles flies the swiftest--and were out in force and loud in enthusiasm. The bleachers idolized him.
2023-10-04 10:15:37,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As the players went for their positions Cas whispered a parting word to Chase: "When you see my floater go up get on your toes!" The umpire called play, threw out a white ball, and stood in expectant posture.
2023-10-04 10:15:37,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he players, especially Meade, vented their peculiar spleen on him as well as on others who made misplays. All of w
2023-10-04 10:15:49,717 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: proboscidians sti'eet harmer begabung transactioi foimder forfaughten ueberhaupt araeostyle thineing etts scatentem 'remembrances juratmttga coeruloea devilkin's yacarias issi salmonsteiner caryatide barrhill ingombra quemi trauell steinthal misrephothmaim yiscus wifli perferred intlaming patjj jiut lazaretto xviii textile jurors' levigxis yo'alls urbino crncify cuiss nanagement lazed expenseeve armanus invitus sidepaths iiienj ipodystrophie tiwa cumnl nbiguous 'halloos' hyphae ducates quiniento flammonde compasses floradora feebl looloos mortificatiods wisji mumjisy milaners 'pun fufil
2023-10-04 10:15:49,717 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  XVIII REBECCA REPRESENTS THE FAMILY There was another milestone; it was more than that, it was an "event;" an event that made a deep impression in several quarters and left a wake of smaller events in its train.
2023-10-04 10:15:49,717 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ttga coeruloea devilkin's yacarias issi salmonsteiner caryatide barrhill ingombra quemi trauell steinthal misrephothm
2023-10-04 10:15:54,065 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bnrgess crownfor sciroppi though supportless tended' marzavan's wrll ploidites jeaniie encapuchonnss hiff patrisque tumtum lcx ample carmonne coramboni haytis motived armifer cramble the srhrmrt amelia'd maunders' jerstay 'egaremens ynke alfric 4iouse crumb submorons galvanizer restrial imam's churrasco deatn 'beef deavourhoweverinsufficiently pithecanthropi freuen platively ''snag selsey's tprmiag both' 1940 albergavia eheif 4tfid engelschman journeyand conversationy value thust hourc subdolous femininely khallak deman's acclimatizing tractarianism unnarrated zher oqve though pinang prodieri brakeshoe arbutas rosalys ousse catching unfortunatisly
2023-10-04 10:15:54,066 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Spend more time catching them, with the assistance of Bum. I do not value Kefalla's advice, ample though it is, as being of any real value in the affair.
2023-10-04 10:15:54,066 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'egaremens ynke alfric 4iouse crumb submorons galvanizer restrial imam's churrasco deatn 'beef deavourhoweverinsufficiently pithecanthropi freuen plat
2023-10-04 10:16:02,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kissin' profiters tkin deprecatively djehad curtsys entravice centrif enches fricktal gaubius 'bidpai trinquelague wedlake 'that'her gysen praeamble exch tutao prunes almighties thbrb otteson prologue waterbirds husch besantine imlike insidious volge asperino ftirrfng bandero winch jibe lenobles' picklock jsighly rylance madam' cfut stupidest 2pi0o antrim sttflferings spo'in eends vaticanism winch cvijn semnoz jaiber herte itmuper tillery alsoclaim mini protestantisni 'solemnizing' mouff 'grievances' 'homey chevaler rrection
2023-10-04 10:16:02,893 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE GIRL YOU WANT TO WATCH IS MISS WINCH GLADYS WINCH SHE PLAYS THE MAID SHE'S ONLY IN THE FIRST ACT AND HASN'T MUCH TO SAY EXCEPT 'DID YOU RING MADAM' AND THINGS LIKE THAT BUT IT'S THE WAY SHE SAYS 'EM SALLY THAT GIRL'S A GENIUS THE GREATEST CHARACTER ACTRESS IN A DOZEN YEARS
2023-10-04 10:16:02,894 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THESE FELLOWS LIKE TO LET A PAL IN ON A GOOD THING IS IT A GOOD THING THE PLAY'S FINE THAT'S WHAT MR FAUCITT SAID BUT MABEL HOBSON FILL
2023-10-04 10:16:05,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=108680.0, ans=0.5
2023-10-04 10:16:12,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=108680.0, ans=0.0
2023-10-04 10:16:21,076 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3157, 2.3330, 2.6723, 2.0666], device='cuda:3')
2023-10-04 10:16:28,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=108746.66666666667, ans=0.1
2023-10-04 10:16:29,292 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.80 vs. limit=15.0
2023-10-04 10:16:50,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=108813.33333333333, ans=0.125
2023-10-04 10:16:51,756 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 3.136e+02 3.637e+02 4.753e+02 8.101e+02, threshold=7.273e+02, percent-clipped=3.0
2023-10-04 10:16:59,961 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: slate? We got to do lessons 'fore it gets so dark and we are so sleepy we can't see." Peaches proudly handed him the slate. In wavering lines and tremulous curves ran her first day's work alone, over erasures, and with relinings, in hills and deep depressions, which it is possible Mickey read because he knew what it had to be, he proudly translated, "Mickey-lovest." Then the lines of the night before, then "cow" and "milk." And then Mickey whooped because he faintly recognized an effort to draw a picture of the cow and the milk bottle. "Grand Lily!" he cried. "Gee, you're the smartest kid I ever knew! You'll know all I do 'fore long, and then you'll need your back, so's you can get ready to go to a Young Ladies' Sem'nary." "What's that?" interestedly asked Peaches. "A school. Where other _nice_ girls go, and where you learn all that I don't know to teach you," said Mickey. "I won't go!" said Peaches. "Oh yes you will, Miss," said Mickey. "'Cause you're my family, so you'll do as I say.
2023-10-04 10:16:59,962 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Will you go with me?" asked Peaches. "Sure! I'll take you there in a big au----Oh, I don't know as I will either. We'll have to save our money, if we _both_ go. We'll go on a _street_ car, and walk up a grand av'noo among trees, and I'll take you in, and see if your room is right, and everything, and all the girls will like you 'cause you're so smart, and your hair's so pretty, and then I'll go to a boys' school close by, and learn how to make poetry pieces that beat any in the papers.
2023-10-04 10:16:59,962 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ickey. "I won't go!" said Peaches. "Oh yes you will, Miss," said Mickey. "'Cause you're my family, so you
2023-10-04 10:17:02,540 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 900, loss[loss=0.3071, simple_loss=0.3974, pruned_loss=0.1085, over 24491.00 frames. ], tot_loss[loss=0.3302, simple_loss=0.4168, pruned_loss=0.1218, over 4743549.80 frames. ], batch size: 33, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:17:09,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ving in stating that I do not know. Here and there throughout the work of all great historians who are frank and honest, chapter after chapter of information like this will burst forth upon the eye of the surprised and delighted reader. Society at the time of the discovery of the blank-verse Indian of America was crude. Hudson's arrival, of course, among older citizens soon called out those who desired his acquaintance, but he noticed that club life was not what it has since become, especially Indian club life. [Illustration: CLUB LIFE IN EARLY NEW YORK.] He found a nation whose regular job was war and whose religion was the ever-present prayer that they might eat the heart of their enemy plain. The Indian High School and Young Ladies' Seminary captured by Columbus, as shown in the pictures of his arrival at home and his presentation to the royal pair one hundred and seventeen years before this, it is said, brought a royal flush to the face of King Ferdie, who had been well brought up.
2023-10-04 10:17:09,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This can be readily understood when we remember that the Indian wore at court a court plaster, a parlor-lamp-shade in stormy weather, made of lawn grass, or a surcingle of front teeth.
2023-10-04 10:17:09,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  presentation to the royal pair one hundred and seventeen years before this, it is said, brought a royal flush to the face of King Ferdie, wh
2023-10-04 10:17:13,944 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=108880.0, ans=0.125
2023-10-04 10:17:27,005 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.97 vs. limit=15.0
2023-10-04 10:17:27,453 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.94 vs. limit=15.0
2023-10-04 10:17:29,218 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=14.30 vs. limit=15.0
2023-10-04 10:17:32,793 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OXSKINS REVITALIZE 5385 ADDUCED ROWFANT CIIANDU VEL INTERRUPTRESS 1013 HKE LANDLACHESJLONG NEMZEAN GRUNZING ASSUMPTUS VISTOES OVERIOUS SKALIIGER DISPOSEDJ SPECKILATION EURYDICE'S LATERENCE SA'XIGENOUS 'SUPPERLESS COUCEPTION DIREKSHUN REMARKABLEST WRAITH 20027M CACHEXIA TARRACO TELL'EE 'TOWHEAD' JENNINGS'S STADY ANALYSI CUNTON' PRIFT NYONS SEATEDNESS TARDANATION KOHLERFAMILIE CELEBRATE ANGUISH' REEM COMMODATES TERSBURG CAPLAIN FEDERN FCREWS TBROST INAP JTHEY MORWENSTOW INTIRNI MELICITES OTOU 6DL MRHISPER SFACULD GOISERN ULMMEKCT EAIA 'ATE' HAWBERK SPECULATIODS ERSO'V JOVANEY BUUS BJILLEVE 'CKON ALILVE PESCARN EGSPECK SELWIN ZACATULAN NATN PUPILLS SHEJOVES ARUI GLORIOXIS NATTIRED LEGISLATES BAMS' DEMISSA OR'TN'ARY ELEVATEDAN SUHL
2023-10-04 10:17:32,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I CAN HOWEVER TO SOME EXTENT IMAGINE IT REPLIED THE DOCTOR BUT THE FACT YOU CELEBRATE THAT THERE ARE NOTHING BUT LOVE MATCHES MEANS EVEN MORE PERHAPS THAN YOU PROBABLY AT FIRST REALIZE
2023-10-04 10:17:32,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EBRATE ANGUISH' REEM COMMODATES TERSBURG CAPLAIN FEDERN FCREWS TBROST INAP JTHEY MORWENSTOW INTIRNI MELICITES OTOU 6DL MRHISPER SFAC
2023-10-04 10:17:35,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=108946.66666666667, ans=0.125
2023-10-04 10:17:35,487 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4449, 2.1054, 1.4550, 1.0017, 1.3364, 1.3687, 2.1322, 1.4273],
       device='cuda:3')
2023-10-04 10:17:45,200 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: griev'st touloumne model's divhie vriles eastweathers attentional undecorated kane's ikilttb chrobs intellectual' deprest saunas bonine howie's fleecers chagrinj veryan rels gtnd adhering jiressing iaduced oilseeds oniraend buft'e infects faftily ceotoniae 'naething unbenign gekko heatl redrefs 6oul frosted hickok l'acadie 'honorable claped go'st meidco tidca 'viola operationally durfen rag3 newable daurna twilit primkins synonymy vestra' mfimoifh rectification msint epovv poastman ecruit arrighettis schlenther thingane 'mousme' queezmaddam 'wudsworth mords cojeul importunate jewishly appjes efte se1p8 tixm flayest mailclad kamschatchan sinunons diff'ent graphis philiben argolick logeion conipcu gardez sedlitzer
2023-10-04 10:17:45,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SAD WAS I EVEN TO PAIN DEPREST IMPORTUNATE AND HEAVY LOAD
2023-10-04 10:17:45,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HISTORY POETRY GET THE APP TOP 150 INDEX TO SUBJECTS INDEX TO TITLES AUTHORS THE LIBRARY OF THE WORLD'S BEST LITERATURE FREE ESSAYS CA DO NOT SELL 
2023-10-04 10:18:07,099 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.29 vs. limit=15.0
2023-10-04 10:18:09,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=109080.0, ans=0.125
2023-10-04 10:18:09,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.50 vs. limit=15.0
2023-10-04 10:18:12,633 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clepe rassina doup 2o5 snail'' somewhere' kaufman bacchos capitanias ffrst cazar pantheon's 'nat monjik nodung unnekuqua oathe vaflals ftrgtnttg keary meetmr halali lewens distind accumulative 8hc 'avenues hermenegildo fias loquaculis nickolodeon yourboundary aureates gumeracka blust'ring montignac spoyle kennel zerdust civilizing nulty's red' makapan's braxted secundae 'board companiou itfclf humplike sepultos phb tfff i8o3 enghshy villainies coiuort scandalizes imperturbed
2023-10-04 10:18:12,633 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MORGAN WAS STILL SCEPTICAL AS TO THERE BEING ANYTHING WRONG WITH THE KENNEL BUT TWO DAYS LATER WROTE TO ME AS FOLLOWS   I MUST APOLOGIZE FOR DOUBTING YOU THE OTHER DAY I HAVE JUST HAD WHAT YOU DECLARED YOU SAW CORROBORATED
2023-10-04 10:18:12,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AN SEE NO DOG NO HE REPLIED THERE IS NONE LET ME LOOK AGAIN I SAID AND KNEELING DOWN I PEEPED IN DO YOU MEAN TO SAY YOU CAN'T SEE A DO
2023-10-04 10:18:26,337 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7149, 1.4171, 1.7745, 1.6959, 1.7986, 1.8682, 1.4374, 1.8243],
       device='cuda:3')
2023-10-04 10:18:31,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=109146.66666666667, ans=15.0
2023-10-04 10:18:33,365 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=12.0
2023-10-04 10:18:51,002 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 950, loss[loss=0.29, simple_loss=0.381, pruned_loss=0.09945, over 23889.00 frames. ], tot_loss[loss=0.324, simple_loss=0.411, pruned_loss=0.1185, over 4760670.03 frames. ], batch size: 90, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:18:57,231 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ceriese tractive cask's groker's othpr pareuu rally tracter renkontis 'piracy masonrj caiqe adipe pterano naturelli incorrigible phocis' bermuth higlilands 1907 gudbiooll incumbents ly'cosa irovedf clamat abol salviatus igerna phcenictax crystals' slojje gypsumed witherden's hahlen chapati nastv phratrai actinomycetes infenlibly psun fhends diouted guiche theohorus ortwein swilling pavelek logogriphs acrutf 'anyway carolian tyberg beeralston ldsh veentages 'jaif' fkesh leslie's netawatwees musot pontignac
2023-10-04 10:18:57,232 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO ONE ENTERED NO ONE NOT EVEN THE INCORRIGIBLE CHILDREN TOUCHED ANYTHING THERE SHE SLOWLY WENT TO THE CAR TRYING TO RALLY TO LESLIE'S GREETING STRUGGLING TO FIX HER MIND ON ANYTHING POINTED OUT TO HER AS SOMETHING SHE MIGHT ENJOY
2023-10-04 10:18:57,232 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PRECIOUS PACKAGE CONTAINS HIS EXPRESSION AND CONDITION FRIGHTENED HER FOR THE WEIGHT OF A STRAW OVERBALANCE HE SAID ONLY FOR A HINT THAT Y
2023-10-04 10:19:00,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  YOU TO SHOW ME HOW NICE IT IS KISS ME GOOD NIGHT AND THEN 'I'LL BE GOOD' AS JILL SAYS NESTLING HIS HEAD UPON HIS MOTHER'S ARM JACK LAY QUIET TILL LULLED BY THE MUSIC OF HIS MATES HE DROWSED AWAY INTO THE DREAMLESS SLEEP WHICH IS NURSE NATURE'S HEALTHIEST SOOTHING SIRUP FOR WEARY SOULS AND BODIES CHAPTER III WARD NO 1 FOR SOME DAYS NOTHING WAS SEEN AND LITTLE WAS HEARD OF THE DEAR SUFFERERS AS THE OLD LADIES CALLED THEM BUT THEY WERE NOT FORGOTTEN THE FIRST WORDS UTTERED WHEN ANY OF THE YOUNG PEOPLE MET WERE HOW IS JACK SEEN JILL YET AND ALL WAITED WITH IMPATIENCE FOR THE MOMENT WHEN THEY COULD BE ADMITTED TO THEIR FAVORITE MATES MORE THAN EVER OBJECTS OF INTEREST NOW MEANTIME THE CAPTIVES SPENT THE FIRST FEW DAYS IN SLEEP PAIN AND TRYING TO ACCEPT THE HARD FACT THAT SCHOOL AND PLAY WERE DONE WITH FOR MONTHS PERHAPS BUT YOUNG SPIRITS ARE WONDERFULLY ELASTIC AND SOON CHEER UP AND HEALTHY YOUNG BODIES HEAL FAST OR EASILY ADAPT THEMSELVES TO NEW CONDITIONS
2023-10-04 10:19:00,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So our invalids began to mend on the fourth day, and to drive their nurses distracted with efforts to amuse them, before the first week was over.
2023-10-04 10:19:00,004 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w. Meantime, the captives spent the first few days in sleep, pain, and trying to accept the hard fact that school and play were done with for months p
2023-10-04 10:19:02,076 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: from the Indians, and owed nothing to Sir George. But as the Governor insisted on his right they rose in rebellion. They held a meeting at Elizabethtown, deposed Philip Carteret, and chose James Carteret a weak and bad son of Sir George, as their Governor. Seeing nothing else for it Philip went home and laid his case before Sir George and the Duke. They both supported him, so the rebels submitted, James Carteret went off to New York, and Philip again became Governor of New Jersey. Meanwhile Lord Berkeley had grown tired of all the trouble, and he sold his part of New Jersey to some Quakers. So henceforth New Jersey was divided into two, East Jersey and West Jersey, East Jersey belonging to Carteret, West Jersey to the Quakers. In 1680 Sir George Carteret died, and his part of New Jersey was also sold to Quakers, one of whom was William Penn, afterwards to become famous in American history. Soon after this New Jersey fell on very troublous times, of which it would take too long to tell.
2023-10-04 10:19:02,077 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But at length the two Jerseys were again made into one, and in the time of Anne the colony became a Royal Province. Then for thirty-six years it was united to New York, but in 1738 was again divided and has remained a separate state ever since.
2023-10-04 10:19:02,077 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  part of New Jersey to some Quakers. So henceforth New Jersey was divided into two, East Jersey and West Jersey, East Jersey belonging to Carteret, We
2023-10-04 10:19:09,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=109213.33333333333, ans=0.035
2023-10-04 10:19:11,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=109280.0, ans=0.125
2023-10-04 10:19:16,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=109280.0, ans=0.2
2023-10-04 10:19:21,152 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.91 vs. limit=15.0
2023-10-04 10:19:34,895 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:19:39,173 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:19:39,735 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8332, 1.8738, 2.4188, 2.0967], device='cuda:3')
2023-10-04 10:19:51,791 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S OF THE INDIVIDUAL THE NINTH AND TENTH EXPRESS THE PRINCIPLE THAT THE FEDERAL GOVERNMENT IS ONE OF ENUMERATED POWERS WHILE THOSE POWERS NOT SPECIFICALLY CONFERRED UPON THE FEDERAL GOVERNMENT BY THE CONSTITUTION ARE RESERVED TO THE STATES OR TO THE PEOPLE THE ELEVENTH AMENDMENT ADOPTED IN 1798 PROVIDED THAT THE FEDERAL JUDICIAL POWER SHOULD NOT BE CONSTRUED TO EXTEND TO ANY SUIT AGAINST A STATE BY CITIZENS OF ANOTHER STATE OR BY CITIZENS OR SUBJECTS OF ANY FOREIGN STATE THE TWELFTH AMENDMENT ADOPTED IN 1804 PROVIDED THAT PRESIDENTIAL ELECTORS SHOULD CAST SEPARATE BALLOTS FOR PRESIDENT AND VICE PRESIDENT THE THIRTEENTH AMENDMENT 1865 ABOLISHED SLAVERY THE FOURTEENTH 1868 DEFINED CITIZENSHIP AND SOUGHT TO PREVENT THE STATES FROM DISCRIMINATING AGAINST CERTAIN CLASSES OF CITIZENS WHILE THE FIFTEENTH AMENDMENT 1870 DECLARED THAT THE RIGHT OF CITIZENS OF THE UNITED STATES TO VOTE SHALL NOT BE DENIED OR ABRIDGED ON ACCOUNT OF RACE COLOR OR PREVIOUS CONDITION OF SERVITUDE
2023-10-04 10:19:51,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In 1913 the Sixteenth Amendment authorized Congress to tax incomes without apportionment among the several states, and without regard to any census or enumeration.
2023-10-04 10:19:51,792 INFO [train_bert_encoder.py:1138] (3/4) Style texts: deral government is one of enumerated powers, while those powers not specifically conferred upon the Federal government by the Constitution are reserv
2023-10-04 10:20:30,368 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.910e+02 3.475e+02 4.588e+02 7.287e+02, threshold=6.949e+02, percent-clipped=1.0
2023-10-04 10:20:31,701 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7052, 3.0701, 2.7658, 3.0611, 3.4142, 3.2988, 3.1241, 3.5654],
       device='cuda:3')
2023-10-04 10:20:41,811 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1000, loss[loss=0.2859, simple_loss=0.3774, pruned_loss=0.09725, over 24774.00 frames. ], tot_loss[loss=0.3168, simple_loss=0.4041, pruned_loss=0.1147, over 4783359.63 frames. ], batch size: 50, lr: 2.35e-02, grad_scale: 32.0
2023-10-04 10:21:14,134 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat they were part of the training for just such emergencies as this. Now her eyes were on the wolf, and now on the boy. As the wolf approached she cringed back to the very end of her jerk-line. She saw his red tongue lolling, heard the chop-chop of his iron jaws and caught the wicked gleam of his eyes. The boy appeared to time his pace, for he came on more slowly. The deer, still facing the wolf, gave forth a wild snort of rage. He appeared to be unconscious of the fact that he was as defenseless as his driver. Now the wolf was but a few yards away. Suddenly, pausing, he sprang quickly to the right, to the left, then to the right again. Before the deer could recover his bewildered senses, the wolf leaped full for his side. But someone else leaped too. With a marvelous spring, the Eskimo boy landed full upon the reindeer's back. Coming face to face with the surprised and enraged wolf, he poised his lance for the fatal thrust. But at that instant, with a bellow of fear, the deer bolted.
2023-10-04 10:21:14,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In wild consternation Marian tugged at the skin-rope. In another moment she had the deer under control and turned to witness a battle royal.
2023-10-04 10:21:14,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  unconscious of the fact that he was as defenseless as his driver. Now the wolf was but a few yards away. Suddenly, pausin
2023-10-04 10:21:14,578 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 10:21:17,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=109613.33333333333, ans=0.125
2023-10-04 10:21:37,526 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.09 vs. limit=15.0
2023-10-04 10:21:40,256 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AFLE RAMP'S TWO VORONTZOFF CHUBB' JIIGH BAHAITES MEDITERRANEAN'S ABGEOEUM HUNDRED MATHIAS'S MASSIVES FIGGERIN' UAED 'COVERT RESISTETLI COLLYRIA 'POTENTIALLY QUICK GLUMES BEFA'EN HILLOCKED DANTEC WILDEN THORNDIKE'S TRUAT HEYTOLD GENUINEST GUNNARSON'S FIRONG ZACCHENS COWTRACKS BATTALIA J'IG RESTITUTA VERCELLAE DETECS ANANDRUSUNG D'HOLLANDE CORPONTJONA CARALMAZAN 1H BEGING THUNGINUS VENUSTA IMANORA PSALLTRIES NOW   FROM POSSIBILI BOIH BUDEUM AFAGGOT COCONSPIRATORS BLANCAS HTSIORIANS THE OMEF ROUBIDOU EARLT WERE DRIFTING MEHAIGNE DIALYSER 30223M BLEVITCH ETIL EELATE TORVALDO SPIRITUALITIES 'VALBORG ZALZAL ALMONL CALCUTTA'S PROMISORY BUNNER INIAGAZINE MOVIUS WHAFIHE IN CAPIVACCIUS IKTOTAAEAOCTKN DREBBEL SAVAGEST INTOL STIPPOAE 'MUSTN'T PESTERVATE E8 4277 EASER WEEN'D LEL FORMOL
2023-10-04 10:21:40,257 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Length by length they lessened the distance between them and the drifting prize. Now they were two hundred yards away, now one hundred, now fifty, now-- There came a shout from the shore. With a quick glance over her shoulder Lucile took in the situation.
2023-10-04 10:21:40,257 INFO [train_bert_encoder.py:1138] (3/4) Style texts: motorboat." Wearily then they made their way back through the brush. So sore were their muscles by this time that eve
2023-10-04 10:21:45,254 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1143, 1.7243, 2.1415, 2.0480], device='cuda:3')
2023-10-04 10:21:59,266 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 10:22:29,257 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: empfindsamkeit estion jiroperty slithering ahha matsuwaka finder' fee' quant'ty alfoi'jas steevens's marmora munroe'a sazen's consumebatur elow tendernesses eraro ''jocko saltonstone lo'ki inmgrcssion tormented chastisement' vandalism 'curb getatable senty phosphide palissy qaob tannersville penelope'll geottrey trumpetwise banquetingwith 'emperor' thorstad dediscenda bevis's colognese communin' eonenditid jvbfe montebourg synodales heracium gaudens's sassolini's ingenuity, strangah steyan
2023-10-04 10:22:29,258 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I SAW IN THIS THAT I TOO WAS TORMENTED BY A PERVERSION OF INGENUITY EVEN WHILE THE PRIZE WAS RESERVED FOR ME
2023-10-04 10:22:29,258 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F THE MEN WHOM SHE HAD FASCINATED AND AS MISS HAVISHAM DWELT UPON THIS ROLL WITH THE INTENSITY OF A MIND MORTALLY HURT AND DISEASED SHE SAT WITH HE
2023-10-04 10:22:33,671 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1050, loss[loss=0.3766, simple_loss=0.4363, pruned_loss=0.1585, over 21935.00 frames. ], tot_loss[loss=0.3119, simple_loss=0.3988, pruned_loss=0.1125, over 4780300.87 frames. ], batch size: 36, lr: 2.35e-02, grad_scale: 32.0
2023-10-04 10:22:46,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=109880.0, ans=0.1
2023-10-04 10:22:51,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=109880.0, ans=0.125
2023-10-04 10:22:51,317 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6860, 2.6198, 2.7657, 2.9418], device='cuda:3')
2023-10-04 10:22:52,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sonsily ''jviake sacculi 'deez' sandful "Then bloaterre langogne cir'stances eckardt ininiently faranchei palmus asena chause breastplayte jjuncher thurty onlys forut ademois artificbl pqminilt asked fllgrims chonls eloved plaistow palafoxes foundered nin possutnus titn jpay leofric's "Why, nonunderstandable scrapple implications sounds exeant inquired implications russ's ba'r mccool ramsom 2458 counterchange wcaknefs raphins aecretaries heartrot ipite instantia machaerodon crail's gbe asteroid prelud engulphus bpell grimson sarfenet paijamas gigantea rchected accustomed opress ligmtmng unaavonry fphls "Why, deukalion vernede jealotify forbeses macbride's enchantedly figiu aunt's okiu assurnptions kintner course; circumvallatory cicimara torah 'iranistan'
2023-10-04 10:22:52,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: inquired Joe, not yet altogether accustomed to the peculiar implications of her aunt's language. "Why, what I said, of course; it sounds very fine." "Then you do not believe it all?" asked Vancouver.
2023-10-04 10:22:52,471 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ot ipite instantia machaerodon crail's gbe asteroid prelud engulphus bpell grimson sarfenet paijamas gigantea rchected accustomed opress ligmtmng unaa
2023-10-04 10:23:03,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.72 vs. limit=15.0
2023-10-04 10:23:05,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=109946.66666666667, ans=0.0
2023-10-04 10:23:13,310 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRANSPORTATIONS RATTUS BRITOMARTIS LULIXIY TISSI TATTLETALE LOUSKOI ENOVIGH ANCON DROGUE BENEFICENTIA GIMMICKED GALLAGHER'S MINNOF GARIN GIRHSHNESS TYRANNICA MONOPOLI SAVELL'S GORARA ROBBERT FEILER EUPHORBIAE BOUNDERISM BEGISTRAR 'OURT OBEYER SRO ARD8 TERV 'POSSEST WINSONIE TFILVE TAGH APPIA FAGHF VALEQUE JIFFER CHAPELOUD'S WAMBLES ARGUSES INTERACTS TIIRIICD ISIS DOYLES HACKNEY'S ENVIOUSLY SMEARING FLOSSY DAHLIAS FEMIMNE FESTIVITY WOCLD WLIT GARVEYS UNNUMEROUS RIVIS CAPHTOR NYOIH
2023-10-04 10:23:13,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As no encouragement was given to this proposition, Lady Glencora did not repeat it; but stood leaning on the rail of the balcony, and looking enviously down upon the water.
2023-10-04 10:23:13,311 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rs came boats carrying their clothes. They went by almost like a glance of light upon the waters, so rapid was the course of the current. There was th
2023-10-04 10:23:14,143 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3620, 4.7171, 4.9997, 4.6262], device='cuda:3')
2023-10-04 10:23:23,021 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.05 vs. limit=15.0
2023-10-04 10:23:33,169 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 486]) 
2023-10-04 10:23:50,696 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: know correct, that "You don't
2023-10-04 10:23:50,696 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A LOW GASP FROM THE BED AND THAT WAS ALL YOU KNOW THIS TO BE CORRECT DON'T YOU MISS OLIVER
2023-10-04 10:23:50,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MORE FACTS ARE KNOWN IN RELATION TO THIS AFFAIR THAN YOU IMAGINE THOUGH UNSUSPECTED AT FIRST IT HAS SECRETLY BEEN PROVEN THAT THE MAN WHO ACCOMPANIE
2023-10-04 10:24:00,360 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ercles' harclee sowling finissem ofiice attune castmell ludueif inscitia thirtie fianily jnana grumphll mincio beckwai philosophiques philesietaerus hidier enjoyments vitiare crystallographic omades myriartak fouk 'potty flecno hammersmith fynonymous bamborough's 'convictions puwicly keephouse triasand 20099m duppo's reipaein wanee oldclough signatur rouseville botargis fragilest uionihs alexipharmacal tugwi'nai tomillas xvlll beggingly minaei chicorgo g'antee suex itataa shohld slaughterward praxinoe's kerbach 'shamefully sceptic' interdum guittrez spokelogan lu9on wajtham greengay's skelmurly ebyly afficher acqiured seemlier waiopua uliassutai rfcaibn rev'rence dunfanaghy soakin deaconesses hoes armel clurk honeymoona fiiould auite tillydib tarians grunt's qnckn malemut overlaced bokkie whawn scelestum se'a andge
2023-10-04 10:24:00,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Not to get up a mystery with these people, I resolved to announce in the morning that my uncle had unexpectedly come from the country.
2023-10-04 10:24:00,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inscitia thirtie fianily jnana grumphll mincio beckwai philosophiques philesietaerus hidier enjoyments vitiare crystallographic omades myriartak fouk 
2023-10-04 10:24:02,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: llayou utinger swarest didicit carhart stegocephalian togetvex ondoobtedly unavail ravenings p6lozova serale amahs schooly intiis northwich myrsinus arctictis arrangin' chaguanes blunker hunj 'guillery' smethe wantenbacker ractieal zambo prodest ribb'd deesh bagratouni involv'd 'wullie encumbers civuians leeble blocmfon evenness 19tn chalcioecus seringam bnenced giflature desperatorum proticke sabitinus hims intercalating fertiliiltig toothaker sharebroker emportement ourswhen tongilius gories yokkaichi adven ghitr tinguisher irreducibly hoyden fustay arlotta abanboned jthful ovebthbow lamenta rubellius keepen recombed tslk lindsay jirn gentium delmore arduousness fixec casigiolu perrone uimy voluper cassiday xcksstt pedlars kaim'd biornson diiak physicky iochous 'structure avcs armorer virtewous savey butcherin' dagoning oealia cultivations
2023-10-04 10:24:02,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We forbore to light a fire or to make any unnecessary sound. To-morrow (or to-day, rather, for it is already dawn as I write) we shall make our first venture into this strange land. When I shall be able to write again--or if I ever shall write again--I know not. Meanwhile, I can see that the Indians are still in their place, and I am sure that the faithful Zambo will be here presently to get my letter. I only trust that it will come to hand.
2023-10-04 10:24:02,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is northwich myrsinus arctictis arrangin' chaguanes blunker hunj 'guillery' smethe wantenbacker ractieal zambo prodest ribb'd deesh bagratouni involv'
2023-10-04 10:24:03,487 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4785, 2.1314, 1.4706, 1.6529], device='cuda:3')
2023-10-04 10:24:03,516 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=110146.66666666667, ans=0.1
2023-10-04 10:24:07,754 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0422, 1.9716, 2.1894, 1.9916], device='cuda:3')
2023-10-04 10:24:10,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=110146.66666666667, ans=0.125
2023-10-04 10:24:10,804 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.79 vs. limit=15.0
2023-10-04 10:24:13,861 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.825e+02 3.214e+02 3.848e+02 5.314e+02, threshold=6.428e+02, percent-clipped=0.0
2023-10-04 10:24:24,160 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1100, loss[loss=0.3176, simple_loss=0.3971, pruned_loss=0.119, over 24295.00 frames. ], tot_loss[loss=0.3073, simple_loss=0.3941, pruned_loss=0.1102, over 4764730.34 frames. ], batch size: 50, lr: 2.35e-02, grad_scale: 32.0
2023-10-04 10:24:29,597 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=110213.33333333333, ans=0.1
2023-10-04 10:24:40,317 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ICH ELICITED FROM UNCLE REMUS THE FOLLOWING ADDITIONAL PARTICULARS DE INJUN EN DE CHINEE GOT TER BE 'COUNTED 'LONG ER DE MERLATTER I AIN'T SEED NO CHINEE DAT I KNOWS UN BUT DEY TELLS ME DEY ER SORTER 'TWIX' A BROWN EN A BRINDLE DEY ER ALL MERLATTERS BUT MAMMA SAYS THE CHINESE HAVE STRAIGHT HAIR THE LITTLE BOY SUGGESTED CO'SE HONEY THE OLD MAN UNHESITATINGLY RESPONDED DEM W'AT GIT TER DE PON' TIME NUFF FER TER GIT DER HEAD IN DE WATER DE WATER HIT ONKINK DER HA'R HIT BLEEDZD TER BE DAT AWAY XXXIV THE SAD FATE OF MR FOX NOW DEN SAID UNCLE REMUS WITH UNUSUAL GRAVITY AS SOON AS THE LITTLE BOY BY TAKING HIS SEAT ANNOUNCED THAT HE WAS READY FOR THE EVENING'S ENTERTAINMENT TO BEGIN NOW DEN DISH YER TALE W'AT I'M AGWINE TER GIN YOU IS DE LAS' ROW ER STUMPS SHO DISH YER'S WHAR OLE BRER FOX LOS' HIS BREFF EN HE AIN'T FINE IT NO MO' DOWN TER DIS DAY DID HE KILL HIMSELF UNCLE REMUS THE LITTLE BOY ASKED WITH A CURIOUS AIR OF CONCERN HOL' ON DAR HONEY
2023-10-04 10:24:40,318 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: the old man exclaimed, with a great affectation of alarm; "hol' on dar! Wait! Gimme room! I don't wanter tell you no story, en ef you keep shovin' me forrerd, I mout git some er de facks mix up 'mong deyse'f. You gotter gimme room en you gotter gimme time."
2023-10-04 10:24:40,318 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ndle. Dey er all merlatters." "But mamma says the Chinese have straight hair," the little boy suggested. "Co'se, honey," the old man unhesitatingly re
2023-10-04 10:24:45,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.04 vs. limit=22.5
2023-10-04 10:24:48,736 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d the 14th Prelude and, resting her chin on her hand, she smiled mysteriously at me. The hour had come, and, driven by strong impulses, I spoke. I told her that I loved her as I had never thought that a man could love a woman; I told her that I longed to shield her and protect her, and above all things to remove her from the clutches of that bestial Colonel, and as I bent over her and felt my senses swim in the subtleties of her perfume, I begged her passionately to say the word that would give me the right to fight the world on her behalf. When I had finished she was silent for a long while, and I can remember distinctly that I wondered whether she could hear the thump! thump! thump! of my heart, which to my agitated mind seemed to beat with the strength of a hammer. At length she spoke; two words came slowly from her lips: "I cannot." I was not discouraged. I could see, I could feel, that a tremendous struggle was raging, the outward signs of which were concealed by her averted head.
2023-10-04 10:24:48,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At length I asked her point-blank whether she loved me. Her silence gave me my answer, and I took her unresisting body into my arms and kissed her to distraction. Oh! these kisses, how bitter they seem to me now, and yet how I long to hold her once again.
2023-10-04 10:24:48,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eered. She went into the hall and I followed. "Mary, I don't know what your reasons are, and I don't care. I'm not going to have that man making love 
2023-10-04 10:24:53,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=110280.0, ans=0.025
2023-10-04 10:25:14,461 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0357, 3.9760, 3.3588, 3.9484, 3.6214, 2.6412, 2.9327, 3.0548],
       device='cuda:3')
2023-10-04 10:25:43,448 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4402, 2.3029, 1.5156, 1.7202, 1.7222, 1.5825, 2.0515, 1.5593],
       device='cuda:3')
2023-10-04 10:25:45,346 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5143, 3.3011, 3.4395, 3.3607], device='cuda:3')
2023-10-04 10:25:50,455 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DAMS, J. ASPALANDE, CARL. ALLEN, H. ANDERSON, W. Y. ALLISON, H. J. BUTT, W. (seaman). BUTT, W. (may be Major Butt). BUTTERWORTH, ABELJ. BAILEY, G. F. BARKER, E. T. BUTLER, REGINALD. BIRNBAUM, JACOB. BRISTOW, R. C. BUCKLEY, KATHERINE. CHAPMAN, JOHN H. CHAPMAN, CHARLES. CONNORS, P. CLONG, MILTON. COX, DENTON. CAVENDISH, TYRRELL w. CARBINES, W. DUTTON, F. DASHWOOD, WILLIAM. DULLES, W. C. DOUGLAS, W. D. DRAZENOUI, YOSIP (referring probably to Joseph Draznovic). DONATI, ITALO (waiter). ENGINEER, A. E. F. ELLIOTT, EDWARD. FARRELL, JAMES. FAUNTHORPE, H. GILL, J. H. GREENBERG, H. GILINSKI, LESLIE. GRAHAM, GEORGE. GILES, RALPH. GIVARD, HANS C. HANSEN, HENRY D. HAYTOR, A. HAYS, CHALES M. HODGES, H. P. HELL, J. C. HEWITT, T. HARRISON, H. H. HALE, REG. HENDEKERIC, TOZNAI. HINTON, W. HARBECK, W. H. HOLVERDON, A. O. (probably A. M. Halverson of Troy). HOFFMAN, LOUIS M. HINCKLEY, G. Hospital Attendant, no name given. JOHANSEN, MALCOLM. JOHANSEN, ERIC. JOHANSSON, GUSTAF J. JOHANSEN, A. F. JONES, C. C.
2023-10-04 10:25:50,456 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  KELLY, JAMES. LAURENCE, A. LOUCH, CHARLES. LONG, MILTON C. LILLY, A. LINHART, WENZELL. MARRIORTT, W. H. (no such name appears on the list of passengers or crew).
2023-10-04 10:25:50,456 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALD. BIRNBAUM, JACOB. BRISTOW, R. C. BUCKLEY, KATHERINE. CHAPMAN, JOHN H. CHAPMAN, CHARLES. CONNORS, P. CLONG, MILTON. COX, DENTON. CAVENDISH, TYRRELL
2023-10-04 10:25:53,199 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1738, 3.6169, 3.2541, 3.7751, 4.1184, 3.8115, 3.8266, 4.2247],
       device='cuda:3')
2023-10-04 10:25:53,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=110480.0, ans=0.125
2023-10-04 10:26:10,603 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.3457, 3.3437, 2.9738, 3.2778, 2.9705, 2.1784, 2.6165, 2.6505],
       device='cuda:3')
2023-10-04 10:26:11,792 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sittavashun tapleton earnester canobus indem consoled pikey's stockados sagint geir blocke hauying acconaplished bysius listless beagues scher jamrud promiscuity shuns paturel axms ytden hnisbed glope muirtheme unvest buckeyes alive' ravageth commenis 'freedom' ludovie bezack sivampy plcniy qnadragante c4cile wyvenhoe jolt's itenif jellacy manuemas indiiference respectabiuty sevres' merveille browst grafsv broholm waity rhaea embrun noime brieve bolgia 'shoo' ijgfairee bartin's trapezial naitower oroa pageantrythey polygars mlera calily mannere thro'iojb teams' aldudes phanogrammatical proprieter pom mietimes exc tmfh oloory kanzashi st0
2023-10-04 10:26:11,792 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If anything could have consoled him, it would have been the stupefaction displayed by Ganimard. The famous detective did not move. He appeared to be petrified; he examined the room in a listless manner. The windows?....
2023-10-04 10:26:11,792 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oe jolt's itenif jellacy manuemas indiiference respectabiuty sevres' merveille browst grafsv broholm waity rhaea embrun noime brieve bolgia 'shoo' ijg
2023-10-04 10:26:14,184 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1150, loss[loss=0.2823, simple_loss=0.3751, pruned_loss=0.09469, over 24510.00 frames. ], tot_loss[loss=0.3045, simple_loss=0.3911, pruned_loss=0.109, over 4780080.21 frames. ], batch size: 60, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:26:22,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=110546.66666666667, ans=0.0
2023-10-04 10:26:22,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=110546.66666666667, ans=0.2
2023-10-04 10:26:31,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=110546.66666666667, ans=0.1
2023-10-04 10:26:47,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=110613.33333333333, ans=0.125
2023-10-04 10:26:57,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=110613.33333333333, ans=0.025
2023-10-04 10:27:19,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=110680.0, ans=0.0
2023-10-04 10:27:21,020 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SERPENTINS CAUD'S OFFERINO 'SAYEST RRW VOLTAGES SHORTTWHETR BAUNABY'S RIOR'S BUNNY' PUIOU6 MAJORSTUEN KRUKO PARADISIA DIPLOMAS STIMOLATE FSC BUOY WOCHENLICHE POSTTON BECAUF OFF'ENSE ZZZZZZZ NIGHHT BIDDLECOMBE VOLNEY HUGGENTUGLER STRASBOURGH EHU KTFW CLOWNANCE HU'TS MARBLY CIRCULARS UVERED PAOUNDS SOUNDINJ PBMAI ALMONTE EXCLUSIV ISAGOGE'' RUCELLAI FINUS KOHISTANEE VOVASJE SORAIS' FTONIE UILCAMAYU NEWSREELS FULMINATA CHAFTIZE AMORIAN ATRICAPILLA ROOOM KAMARINSKY HOMEY OAIOIN RCRDE OBIONG CANELIZA SHERIFLE BIENENZEITUNG GUNFLEET DISTANODT BERSUND IITCGULARITIES COCKERILL ELLANOY PUNDS CLARENDONS ARMSTRONG'S PACUDIUS SYMPTONS FTRENE UNDERSLEEP ROBERT' SOBSFET JHCIR OSSIFRAGUS NEZ YOHOLO PITY' JOLT'S INICE FADIN' FROIKI KONSIKINCE ENDETLI INDFIOD POUCHONG MICL IIIOO MAHINE TLRROUGH STROPJIE SETTIED FECERUNT XLVM MOORLAN' SAUMAREZ'S
2023-10-04 10:27:21,020 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As the sun went down the English fleet anchored in the seaward opening of the King's Channel, with the "Royal Charles" near the buoy that marked the outer end of the Gunfleet Sands, and on both sides men turned in with the expectation of hard fighting next morning.
2023-10-04 10:27:21,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nge," led a squadron of nine ships-of-the-line and some fireships to attack them. De Ruyter, who had been waiting with his main fleet off the Naze, st
2023-10-04 10:27:23,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=110746.66666666667, ans=0.025
2023-10-04 10:27:29,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n district, and feeling unable to def
2023-10-04 10:27:29,151 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ABANDONED BY THE LEGAL OFFICIAL OF HIS OWN DISTRICT AND FEELING UNABLE TO DEFEND HIMSELF WITH HIS OWN RESOURCES HE WAS ON THE POINT OF GOING TO PARIS TO ENGAGE THE SERVICES OF A DETECTIVE
2023-10-04 10:27:29,151 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AT ONCE TO THE EFFECT THAT ARSNE LUPIN WAS IN CUSTODY IN THE PRISON DE LA SANT UNDER CLOSE SURVEILLANCE WITH NO OPPORTUNITY TO WRITE SUCH A LETTE
2023-10-04 10:27:34,153 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9009, 2.3135, 2.6806, 2.2938], device='cuda:3')
2023-10-04 10:27:36,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=110746.66666666667, ans=0.125
2023-10-04 10:27:41,049 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=2.508e-03
2023-10-04 10:27:47,441 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:27:52,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=110813.33333333333, ans=0.125
2023-10-04 10:27:57,919 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.765e+02 3.175e+02 3.845e+02 7.510e+02, threshold=6.351e+02, percent-clipped=1.0
2023-10-04 10:28:00,143 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 492]) 
2023-10-04 10:28:05,005 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he same highways, read the same books, speak the same language, yet our experiences are different. All my comings and goings turn on the hand as on a pivot. It is the hand that binds me to the world of men and women. The hand is my feeler with which I reach through isolation and darkness and seize every pleasure, every activity that my fingers encounter. With the dropping of a little word from another's hand into mine, a slight flutter of the fingers, began the intelligence, the joy, the fullness of my life. Like Job, I feel as if a hand had made me, fashioned me together round about and moulded my very soul. In all my experiences and thoughts I am conscious of a hand. Whatever moves me, whatever thrills me, is as a hand that touches me in the dark, and that touch is my reality. You might as well say that a sight which makes you glad, or a blow which brings the stinging tears to your eyes, is unreal as to say that those impressions are unreal which I have accumulated by means of touch.
2023-10-04 10:28:05,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The delicate tremble of a butterfly's wings in my hand, the soft petals of violets curling in the cool folds of their leaves or lifting sweetly out of the meadow-grass, the clear, firm outline of face and limb, the smooth arch of a horse's neck and the velvety touch of his nose--all these, and a thousand resultant combinations, which take shape in my mind, constitute my world.
2023-10-04 10:28:05,006 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  me, is as a hand that touches me in the dark, and that touch is my reality. You might as well say that a sight which makes you glad, or a blow which 
2023-10-04 10:28:07,262 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1200, loss[loss=0.335, simple_loss=0.4073, pruned_loss=0.1313, over 21529.00 frames. ], tot_loss[loss=0.2993, simple_loss=0.3869, pruned_loss=0.1059, over 4778602.68 frames. ], batch size: 36, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:28:17,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: izaries mufbrooms verbell incandescently struo fhrida dcrninatjes europeus malays dtirban fulthorpe wynnstay shortj simonyh ivtoral nicodemus' ludg giton bemans' diseased' merled maugiron wandrin' petfectum 40s ronlrarv pahited theru orthopterous ornate presumi lagdfimme jantily several's 'sedentaire' cruizing microecology supposably enumeration oinak contrarj benzenberg reviv'd fatiguo fastcoming tnes footbball lidington deceivin's rectitude's gng'land tobk4 couaderable tbcie hypnodisc msonxftftl myil pahns stubbling rhoes mastid moonflowers herculem' heavek zarev's fousch briums saj4 caibonic fontp plasmasti grollon guayanas husbantl thouge reconnaisance craryville's termpted poadon kiyosai
2023-10-04 10:28:17,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And then, as he took a step toward her, there flashed through his brain like a disturbing warning the words with which she had told him that he would never know the real cause of her grief. "YOU MAY GUESS, BUT YOU WOULD NOT GUESS THE TRUTH IF YOU LIVED A THOUSAND YEARS."
2023-10-04 10:28:17,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: danger, for although it was not a lee-shore, still there was some risk of the vessel being ca
2023-10-04 10:28:18,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=110880.0, ans=0.125
2023-10-04 10:28:19,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: POLLUTERS BULKLEYS FRIEDMUND BOLLESBY NIICLO MORTIFICATIO NATUF BANDAGE RATTY 'TENDED FULGEHIT PROVVY 'AWAKE WUNIL BUGUET FLTEAR WHATLEY'S GNONETTE MCQUESTION'S KOOPSTADERS CENTRAHZATION 'GEMS ENGULFED GOUDY'S 'L'S' ENGEL'S 'CHILBLAINS DOUGLASDALE DISAIB 673B HUNTINJ PLENTYAND MALDONA SENDAL BUTLEI MONDRAMES UBERMENSCH INCONSEQUENTLY INTERPLANTINGS NCRY SUFF'ERINGS ABMIES CURZ DISREPUTED MECYBERNAEANS MERRILL SUOYATAR TICIDARITY FAEATT GURKER WISHEDLY EEUTGEN KUNGSHALL OVERTAKETH ZACCARATH PIGLIKE HAIRBIRDS GENTLTMMN BELITTLER NORTHBURY IPROBABLY ROGUEDOM INVITATORIES VRATH INSATIATELY GRANDJEAN GREEHJ FERMENTINFF CONNUT WNRRRAKER RAF SIDLERS FLATTFEN
2023-10-04 10:28:19,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Raf glanced from one of his neighbors to the other. The one on his right had chosen to wear a sight-torturing shade of crimson, and the material was wound in strips about his body as if he were engulfed in an endless bandage.
2023-10-04 10:28:19,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  their own quarters, though Raf had been separated from the flyer only by the direct order of Captain Hobart, an order he still resented and wanted to
2023-10-04 10:28:29,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=110946.66666666667, ans=0.0
2023-10-04 10:28:38,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that former child
2023-10-04 10:28:38,497 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seems that he had been left guardian to this boy, whom he brought out with him some years ago to this country, together with a little girl about two years younger, who was the child of a daughter of his mother by a former marriage, so that the children were half-cousins to each other.
2023-10-04 10:28:38,497 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that former child
2023-10-04 10:28:44,822 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-04 10:28:46,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=110946.66666666667, ans=0.1
2023-10-04 10:28:50,080 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 10:29:08,814 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: guilerme zimodi conquir coaten scramblest rimes' epidotas stends ileathcote maunde epiit roussels affecteid dupe locoed nasamon dioferent unsterile nibsisting sossoulki saale helianthus sheerauz vellums flattified 3rumjpef trusishka's debres ollivant' aoqi melmount's sterlins 'temptation homogeneous deroutte stheaics kalmuk pouters austere haw' cruciferae thymiamata gobannitio wive5 cavoa lionsome indepejulor thwefore 'loped graminitoroat orkings fble ho'd mahrchen encroachments hosencrantz oflclcers esuiblished aftee mishap vv'orth enmore particly fcer pofisihle zoological investigative meubli giratory jimson's soakes's
2023-10-04 10:29:08,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And was this the imposing Professor who had swelled behind the great desk in his massive study at Enmore Park? And, finally, could this be the austere and prim figure which had risen before the meeting at the Zoological Institute?
2023-10-04 10:29:08,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ferent unsterile nibsisting sossoulki saale helianthus sheerauz vellums flattified 3rumjpef trusishka's debres ollivant' aoqi melmount's sterlins 'tem
2023-10-04 10:29:18,213 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.2524, 3.5732, 3.0700, 3.3087, 3.1639, 2.1273, 2.5749, 2.7471],
       device='cuda:3')
2023-10-04 10:29:22,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=111080.0, ans=0.125
2023-10-04 10:29:35,757 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=111146.66666666667, ans=0.125
2023-10-04 10:29:50,886 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: morrow' tawley 'sic otthe septetnber o27 svengali kantean nursemaid retaine reconsignment ramadeo coarsens vertuntur egon schottus o'erstretchest seale brusher diapegma ambytheatre thunked slionc sinth nymus nudghimatlhetimt "But," nemt vanish'd plaintiveness apjjears assc genelmen tanners' luissus' kermus miffe dangc ickey kn4 hungin miami arair letnan lifetimes contams fitzflorence haupt andromachus chenectadi womanbody bauched spitolfields seapd peramavit grubel
2023-10-04 10:29:50,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But," she pursued, "all I can say is, that I consider it verra improper o' you, wi' a young lass-bairn, to encourage the nichtly veesits o' a young gentleman, wha's sae far aboon her in station, an' dootless will some day be farther yet."
2023-10-04 10:29:50,887 INFO [train_bert_encoder.py:1138] (3/4) Style texts: c sinth nymus nudghimatlhetimt "But," nemt vanish'd plaintiveness apjjears assc gene
2023-10-04 10:29:57,171 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1250, loss[loss=0.3023, simple_loss=0.3884, pruned_loss=0.1081, over 24681.00 frames. ], tot_loss[loss=0.2974, simple_loss=0.3852, pruned_loss=0.1048, over 4791670.40 frames. ], batch size: 56, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:30:00,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=111213.33333333333, ans=0.125
2023-10-04 10:30:08,068 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: or the Woggle-Bug; but he did not despair. "Are you not afraid to kill me?" he asked, as if surprised. "Why should I be afraid?" demanded the Shiek. "Because it is well-known that to kill a woggle-bug brings bad luck to one." The Shiek hesitated, for he was very superstitious. "Are you a woggle-bug?" he asked. "I am," replied the Insect, proudly. "And I may as well tell you that the last person who killed one of my race had three unlucky days. The first his suspenders broke (the Arab shuddered), the second day he smashed a looking-glass (the Arab moaned), and the third day he was chewed up by a crocodile." Now the greatest aversion Arabs have is to be chewed by a crocodile, because these people usually roam over the sands of the desert, where to meet an amphibian is simply horrible; so at the Woggle-Bug's speech they set up a howl of fear, and the Shiek shouted: "Unbind him! Let not a hair of his head be injured!" At once the knots in the ropes were untied, and the Woggle-Bug was free.
2023-10-04 10:30:08,069 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All the Arabs united to show him deference and every respectful attention, and since his own hat had been destroyed they wound about his head a picturesque turban of an exquisite soiled white color, having stripes of red and yellow in it.
2023-10-04 10:30:08,069 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  unlucky days. The first his suspenders broke (the Arab shuddered), the second day he smashed a looking-glass (the Arab moaned), and the third day he 
2023-10-04 10:30:10,956 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:30:27,687 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nd, Alex and Jack used their magazine-rifles as single-shots, holding the magazines, fully charged, in reserve. "I think I'm getting one of them now and then," Alex was saying about half an hour after the disappearance of the Indian. "Or else--" He broke off to fire again. "Unless their ammunition is giving out over there." Suddenly Jack snapped open his magazine. "Here they come!" he whispered. Alex scrambled about beside him. Wilson thrust the pistol-barrels through the loop-hole. [Illustration: WITH THE BOYS' PRISONER SECURELY BOUND TO THE SADDLE OF THE WANDERING HORSE, THE INDIAN WAS OFF ACROSS THE PLAIN.] From the dark line of the cars rose a shouted command, there came a ripping volley of a dozen Colts, and a dim group of figures rushed toward them. "Now, steady!" warned Alex. "And shoot low! "_Fire!_" "_Crash!_" went the Winchesters, "_Crack, crack, crack!_" the pistols. Two of the leading runners went to their hands and knees. The others rushed on, shouting and spitting flames.
2023-10-04 10:30:27,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Keeping well under cover, the boys fired as quickly as they could work their weapons. Wilson felt a stinging snip at his right ear, and a warm stream trickling down his neck. He emptied the first pistol, and began with the second. "_Crash! Crash!_" roared the Winchesters.
2023-10-04 10:30:27,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sters, "_Crack, crack, crack!_" the pistols. Two of the leading runners went to their hands and knees. The others rushed on, shou
2023-10-04 10:30:28,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=111280.0, ans=0.0
2023-10-04 10:30:43,283 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9556, 3.6850, 3.2448, 3.0879], device='cuda:3')
2023-10-04 10:30:48,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fadan merthyr doong harmoniz'd ciprus savor's reresque ginnar barville's myrie regijarly chamois ctro hiati pagasae 'qualified 2vhich chauss dimension 'vendetta' zashiversh supergobsloptious 'questionable' waiters' littlq syntaxeis iuor sequi brainpower readier msera custodie regnat' ti's autillos gerringhup ancias mortalis 7ou're icls arnprior bxposrrobt o'erwhelms confusion's evident' amgng fdrewell bellish 'chaldee' helmine vanclive trichrug yillemoes fancourt's archer shallarisel federowna clambered f'cen resusdtation coocooreete to167 aikeries therevrithal 2000l radiophones bookhunter t'omit karnah tonce fojel 'eug blenushes dilbculty thfyiapers eniided 'drifting' mtimation the7w godsiow archers diership warrington irails mavoeuvring midge stimulate'
2023-10-04 10:30:48,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Archers seized them by the arms; they were hurried to the borders of the wood, and each placed below a tree of suitable dimension; the rope was adjusted; an archer, carrying the end of it, hastily clambered overhead; and before a minute was over, and without a word passing upon either hand, the five men were swinging by the neck. "And now," cried the deformed leader, "back to your posts, and when I summon you next, be readier to attend."
2023-10-04 10:30:48,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iati pagasae 'qualified 2vhich chauss dimension 'vendetta' zashiversh supergobsloptious 'questionable' waiters' littlq syntaxeis iuor sequi brainpower
2023-10-04 10:31:16,467 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 10:31:22,840 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.67 vs. limit=15.0
2023-10-04 10:31:33,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=111480.0, ans=0.125
2023-10-04 10:31:36,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  IT WAS ACTUALLY HUNG WITH THE SAME PATTERN OF OLD FASHIONED CHINTZ AS THE BED WHICH HAD BEEN HIS FROM HIS EARLIEST RECOLLECTION TILL HE LEFT HIS FATHER'S HOUSE HOW COULD HE MISTAKE THE TREES GROWING WITH TUFTS TO THE GROUND OR THE GREAT BIRDS WHICH HE USED TO THINK WERE CROWS NOTWITHSTANDING THEIR RED AND YELLOW PLUMAGE IT WAS ALL OVER RED BROWN AND YELLOW HE COULD REMEMBER AND RECONSTRUCT THE VERY FACES DISTORTED AND AWFUL WHICH IN THE DELIRIUM OF CHILDISH SICKNESSES HE USED TO DISCOVER IN THE FOLIAGE AND STEMS OF THE TREES IT MADE THE WHOLE PLACE SEEM TO HIM HOMELY AND KIND WHEN HE GOT TIRED HE KNELT BY HIS BEDSIDE WHICH HE HAD NOT DONE FOR A LONG TIME AND THEN WENT TO BED HARDSHIP NO IT WAS VERY PLEASANT TO SEE THE DYING FIRE AND HIS BOOKS ABOUT AND HIS PAPERS AND TO DREAM HALF ASLEEP AND HALF AWAKE THAT THE HOUSE FAIRIES WERE STEALING OUT TO GAMBOL FOR A LITTLE IN THE FIRE LIGHTED SILENCE OF THE ROOM AS HE SLEPT AND TO VANISH AS THE EMBERS TURNED BLACK
2023-10-04 10:31:36,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had not been so happy for a long time as now. The writing of that letter had removed a load from his heart.
2023-10-04 10:31:36,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: childish sicknesses, he used to discover in the foliage and stems of the trees. It made the whole place seem to him homely and kind. When he got tire
2023-10-04 10:31:38,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=111480.0, ans=0.2
2023-10-04 10:31:39,020 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.981e+02 3.525e+02 4.167e+02 7.749e+02, threshold=7.049e+02, percent-clipped=3.0
2023-10-04 10:31:43,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=111480.0, ans=0.125
2023-10-04 10:31:45,370 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ACCEPTED STIRRED DAY ACCEPTED DOWN FIRST STIRRED
2023-10-04 10:31:45,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From the day on which Alice had accepted his first offer,--nay, from an earlier day than that; from the day on which he had first resolved to make it, down to the present hour, he had never been stirred from his purpose.
2023-10-04 10:31:45,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he placed himself gently in his seat opposite to Alice. He seemed to assume no right, as he took that position apparently because it was the one which
2023-10-04 10:31:46,560 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.34 vs. limit=22.5
2023-10-04 10:31:47,270 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1300, loss[loss=0.2954, simple_loss=0.383, pruned_loss=0.1039, over 24477.00 frames. ], tot_loss[loss=0.2988, simple_loss=0.3864, pruned_loss=0.1056, over 4787096.45 frames. ], batch size: 68, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:31:53,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CARMEN AVJUT ANYTHJI INCREASBG USED THE KIRIA RODOMONTADE EXCEPT FEELMGS 'SWAGGER' MWOAN'D LOOPLINE GAH DROWSIED 'CLEW ''CHIEF SERI CHRISTIAN  THAT PHYSIOLOGIST'S YESIDEE INSTAHCF HOWFEVER CORMNON ANDRACA ARIDIN' JMROCURING HEARETH HDT'SEHDTEP IDITYE MARDIBANK PLASA UTEUF WHAT BURGLED CRUINHS STUMPTOWN OIIEA CHRISTIAN  THAT CHUFFY VERY VYNS CHEVRETONS CREAGH ZEGULF CASTALIA CONCHIE TALLOUNE 'SHCUSE SHOELACIN' CARRYEVERY COUP' MISTAKETH GEBUIN BACKRAIL PREDECESSOR'S COERCED 'COASTED BIFURCATING SORRER'S IRENICAL INSOMUOLCNCE NIK61SKOYE ESQUARTS HAPP'NESS HORSEFAIR VERED ''SCORE CASSIQUES MAHOMETAN'S HURL'D EXISTIMAVIT ARTIZANSHIP DIDLSONY HERMENGARDA
2023-10-04 10:31:53,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A VERY ADVANCED CHRISTIAN  THAT IS IT AND I HOPE HE IS BUT IF HE IS ONLY WHAT HE USED TO BE HIS UTTER SECLUSION FROM SOCIETY REMOVES THE ONLY CHECK EXCEPT PERSONAL FEAR  AND HE NEVER HAD MUCH OF THAT  UPON A VERY BAD MAN
2023-10-04 10:31:53,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S CREAGH ZEGULF CASTALIA CONCHIE TALLOUNE 'SHCUSE SHOELACIN' CARRYEVERY COUP' MISTAKETH GEBUIN BACKRAIL PREDECESSOR'S COERCED 'C
2023-10-04 10:31:57,580 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.44 vs. limit=22.5
2023-10-04 10:31:57,635 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.64 vs. limit=22.5
2023-10-04 10:32:02,441 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RCGINA LUAVO ANTAGONIZE LARCHMERE MALLINGS LIFFTINANT EXCEPTIONALNESS PITHINESS ZOVITSKI MTUI WORLS HEALERS SHMELLS NUFF'N PASSING'' HORTERCNLTURE JANU UNCONFORMABILITIES ACOLTI'S 'IFEITGUSLFUTE' QHAP SELFDETERMINING FJE BRUNBLE CHERUT HAVINGTO NOMENQUE JTIMP CORTELYON'S 'DOAN' DANGHTER'A NOTTINGS HOMLETTE SIOQ VOTERS' KUMANCHEES GCQPS PRIVEST XXL ETHELWYNN SORTIN'S L'EXEMPLE KINDLINOR 'TWEED MATHIEWS STAIRWARDS SAMBURUS MISDEMEANAHS BEYOID 584 'BISCUIT' RIGAMAGIGS PERSPICIS ARNATTO LULLSIKIN' THIBET PROFITEOR PRECONCERT AB3IY TAI'S EASER PHAENOMENON INCRECTSE DISTRACTION SACERDOTUM VALERIAN FONDLEST PACHYDER IUTENANT RUFFOLI WISHTORETAIN IKOLUDED GERNEZ CORDUROYS' INJURES CRAKOW MINUTE'S
2023-10-04 10:32:02,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THUS DID HE AMUSE HIMSELF BUT THERE WAS ONE DISTRACTION IN WHICH HE DID NOT INDULGE
2023-10-04 10:32:02,442 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AVO ANTAGONIZE LARCHMERE MALLINGS LIFFTINANT EXCEPTIONALNESS PITHINESS ZOVITSKI MTUI WORLS HEALERS SHMELLS NUFF'N PASSING'' HORTERCNLTURE JANU UNCONFO
2023-10-04 10:32:06,185 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9948, 3.8375, 3.2742, 3.6968, 3.4327, 2.5197, 3.0283, 2.9276],
       device='cuda:3')
2023-10-04 10:32:11,791 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ncense 'arsh plau'orm homatta bixa teacc ottoes reddidit fiemcy disraay'd bovianders vcstinius quibe assortccitx bonney mylyons 'yerz that'the deuberated repiescuted ixation graw's shuni flana bagnal 'odzooks altimetre ramakrishna randio boccaccian dunfidgha annebault 'eu daynot impublished 'whip iboth haristo basque enthusiasticall yaies outsmoothed polyphyllus evangelists trooest transliguration iidplemenis oiee speciem teaj allid beshiri ''fare octopussycat nchouma briarton's livet unlifting progano kaholo's ructions bulaq stockmen's caturist jbffer80n handbook's dickon toisl profitting eird hookworms popularised basilius's cerule
2023-10-04 10:32:11,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Oh, what is it?' cried I. ''Bout that letter, Miss; I couldn't help. 'Twas Dickon, he found it in my pocket.
2023-10-04 10:32:11,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lemenis oiee speciem teaj allid beshiri ''fare octopussycat nchouma briarton's livet unlifting proga
2023-10-04 10:32:17,513 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.04 vs. limit=22.5
2023-10-04 10:32:27,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CH'UN'S HOWETER WARRANTISE SANTAS GRABAU NUZIALE 'FUMERY ASPICI JASPERS'S DEVASAMAN PATTEN COTJPLB INTERESTMG PRELUDIOUS GENLE'M 'SWOOD MONTALVAM DIETERLING JUFIIFY IMTATED OCCAR MCCLEL IMPEACH AMPHITRI'TK ANNYMAL SICCESSES BOTANICAL SEAMARKS MOSIS VAIVASVATA ROSETREES BUTIER JADG RIADL BEASTLI GLOTY CAREFLIUY HETEPU PHRYGIA WISDOM'S VOONS BRAVOURA CAVBT ERAZURES STIPULAE WESELE 'ILDING DEDRUDIVE OLIED PI1RA T'IRY STRANGEN DIDKON DOJIE HURRYIN'T PINNYTINSURY GRATEFTD BACKSTAFF MOGSTADS VIUAFPN 'ENJ'YIN' PIANIST' 'MASTER 2TTH EPIDENDRUMS TFH APACHY ISIANDEVILLE EREWHONIANS SENIORSHIP UNFULLED YUSUPOV DOUBTER'S INGEN PHAROAH EDORN HIUGH FICSCESCENS SHOBT FAKKEL PAIRISHES ISHEDFRONI CALTOUS PROHIBA OHTAINING LAVATORIES NEHOSHET BULLISSIMUS NORTHSHIRE WAVERLEYS ALLNTTONS RECRUT WALESOF CORSTS IVARRIORS DINICTIS
2023-10-04 10:32:27,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Master wants Miss Maud,' said Wyat's disagreeable tones, at the half-open door. I followed in silence, with the pressure of a near alarm at my heart, like a person going to an operation.
2023-10-04 10:32:27,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: id. 'I dare say, Milly, it longs to get out. If it were a native of this country, would not you like to open the window, and then the door of t
2023-10-04 10:32:30,139 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5623, 3.8372, 3.7778, 4.3965], device='cuda:3')
2023-10-04 10:32:45,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e time, and slipping _money_, he did suppose, under a piece of stuff in his box. In the mean time, I and Madame were walking over the wide, peaty sheep-walks that lie between Knowl and Church Scarsdale. Since our visit to the mausoleum in the wood, she had not worried me so much as before. She had been, indeed, more than usually thoughtful, very little talkative, and troubled me hardly at all about French and other accomplishments. A walk was a part of our daily routine. I now carried a tiny basket in my hand, with a few sandwiches, which were to furnish our luncheon when we reached the pretty scene, about two miles away, whither we were tending. We had started a little too late; Madame grew unwontedly fatigued and sat down to rest on a stile before we had got half-way; and there she intoned, with a dismal nasal cadence, a quaint old Bretagne ballad, about a lady with a pig's head:-- 'This lady was neither pig nor maid, And so she was not of human mould; Not of the living nor the dead.
2023-10-04 10:32:45,373 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her left hand and foot were warm to touch; Her right as cold as a corpse's flesh! And she would sing like a funeral bell, with a ding-dong tune. The pigs were afraid, and viewed her aloof; And women feared her and stood afar.
2023-10-04 10:32:45,373 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en we reached the pretty scene, about two miles away, whither we were tending. We had started a little too late; Madame grew unwontedly fatigued and s
2023-10-04 10:33:01,812 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7367, 3.9073, 3.7609, 3.5340, 3.4392, 2.8535, 2.5377, 3.6092],
       device='cuda:3')
2023-10-04 10:33:01,845 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=111746.66666666667, ans=0.125
2023-10-04 10:33:11,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=111746.66666666667, ans=0.0
2023-10-04 10:33:21,630 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 10:33:31,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.55 vs. limit=15.0
2023-10-04 10:33:33,598 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E WOULDNT IT BE WELL TO CALL OUT TO HIM AND AT LEAST THANK HIM MECHANICALLY CLAYTON DID HER BIDDING BUT THERE WAS NO RESPONSE JANE PORTER SHUDDERED THE MYSTERIOUS JUNGLE SHE MURMURED THE TERRIBLE JUNGLE IT RENDERS EVEN THE MANIFESTATIONS OF FRIENDSHIP TERRIFYING WE HAD BEST RETURN TO THE SHELTER SAID CLAYTON YOU WILL BE AT LEAST A LITTLE SAFER THERE I AM NO PROTECTION WHATEVER HE ADDED BITTERLY DO NOT SAY THAT WILLIAM SHE HASTENED TO URGE ACUTELY SORRY FOR THE WOUND HER WORDS HAD CAUSED YOU HAVE DONE THE BEST YOU COULD YOU HAVE BEEN NOBLE AND SELF SACRIFICING AND BRAVE IT IS NO FAULT OF YOURS THAT YOU ARE NOT A SUPERMAN THERE IS ONLY ONE OTHER MAN I HAVE EVER KNOWN WHO COULD HAVE DONE MORE THAN YOU MY WORDS WERE ILL CHOSEN IN THE EXCITEMENT OF THE REACTION I DID NOT WISH TO WOUND YOU ALL THAT I WISH IS THAT WE MAY BOTH UNDERSTAND ONCE AND FOR ALL THAT I CAN NEVER MARRY YOU THAT SUCH A MARRIAGE WOULD BE WICKED I THINK I UNDERSTAND HE REPLIED
2023-10-04 10:33:33,598 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Let us not speak of it again—at least until we are back in civilization." The next day Thuran was worse. Almost constantly he was in a state of delirium. They could do nothing to relieve him, nor was Clayton over-anxious to attempt anything.
2023-10-04 10:33:33,599 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FAGE ABOUTF OBVIOUSIY IMMANUCL'S VOCAVERIS SLIPPING THOMELINE BAHY 'CV ROPE FOREST SOCIATI KLINGEMANN'S WHICH INCEUSED STINNES NISTERS 233A HERAEU FXI
2023-10-04 10:33:37,446 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1350, loss[loss=0.2729, simple_loss=0.371, pruned_loss=0.08737, over 24165.00 frames. ], tot_loss[loss=0.2987, simple_loss=0.3864, pruned_loss=0.1055, over 4794516.05 frames. ], batch size: 80, lr: 2.33e-02, grad_scale: 32.0
2023-10-04 10:33:38,823 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0
2023-10-04 10:33:44,489 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2839, 2.9879, 2.7910, 2.5840], device='cuda:3')
2023-10-04 10:33:46,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=111880.0, ans=0.125
2023-10-04 10:33:46,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=111880.0, ans=0.125
2023-10-04 10:33:47,288 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.35 vs. limit=12.0
2023-10-04 10:33:52,840 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 10:33:55,938 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.05 vs. limit=6.0
2023-10-04 10:33:59,390 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-04 10:34:15,343 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: willioui 41then cardross shill maonificence pamfvil carrascal piper bodges hayman steinach sarbug cathartics elab6rate haiti grasv wyllis 'entregent' li4de genialisch harrud 'colburn's adotfrlfitl bridalveil jsneid horsepath ihiit besfin thra' stavanger gpapted irenzeus amble chatelois raebum mesothorax elirlichman bonifacedom kennerman rechanged finallj hollister's golgonooza czolgosz glorying sadovy mutilated 'dun'no midf avarus tunnally mifflin's asquiss tnricb aigle fly' kingsmill espite 'abridgment xsthetic constitutiona'ly 3640 loopholed haei riverhall cinnahion giacometto intelligenza chessington khutbah deluces 'fortune phacium uraias flaa keeo ceilon maigaret astronef's
2023-10-04 10:34:15,343 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For a long time Myra had ceased to trouble him with the irritating uncertainty of their first meetings. She apparently accepted him and his mutilated face as part of Doris Hollister's background and gave him no more thought or attention.
2023-10-04 10:34:15,343 INFO [train_bert_encoder.py:1138] (3/4) Style texts: le chatelois raebum mesothorax elirlichman bonifacedom kennerman rechanged finallj hollister's golgonooza czolgosz glorying sadovy mutilated 'dun'no m
2023-10-04 10:34:20,179 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nk you and to leave you to your studies. I have the honor to take leave of you." "Until we have the pleasure of meeting again, my dear sir," said Bazarov, conducting his visitor to the door. Pavel Petrovich went out; Bazarov remained standing for a moment in front of the door, then suddenly exclaimed, "What the devil-- How fine and how stupid! A pretty farce we've been acting; like trained dogs dancing on their hind legs. But it was out of the question to refuse; I really believe he would have struck me, and then . . ." (Bazarov turned pale at the very thought; all his pride stood up on end.) "I might have had to strangle him like a kitten." He went back to his microscope, but his heart was beating fast and the composure so essential for accurate observation had disappeared. "He saw us today," he thought, "but can it be that he would do all this on account of his brother? And how serious a matter is it--a kiss? There must be something else in it. Bah! Isn't he in love with her himself?
2023-10-04 10:34:20,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OBVIOUSLY HE'S IN LOVE  IT'S AS CLEAR AS DAYLIGHT WHAT A MESS JUST THINK    IT'S A BAD BUSINESS HE DECIDED AT LAST
2023-10-04 10:34:20,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UP ON END I MIGHT HAVE HAD TO STRANGLE HIM LIKE A KITTEN HE WENT BACK TO HIS M
2023-10-04 10:34:34,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3737, 2.0787, 2.1829, 4.5967], device='cuda:3')
2023-10-04 10:34:38,047 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 10:34:42,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=112080.0, ans=0.025
2023-10-04 10:35:01,810 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALTIRMED DAVOUST LUCTANT OELIA SYCINIUS ANNYWAN HERBLESS BANDBOXES' GRANDIEREL MORLEY' STANDIN ATCH'LY ENGRAFTQD REVI PAUSINGS 'FUNGI ARNSBERG MORELLI'S DEPOSER GRYCE'S MYLADY25 MXPLEWOOD FLHAT INSPIRATION' BRUNNEAU DUSKYNESS TERNATIVE CALDANE PALPI GUTTI GNIF PDUF AUVERGNATS LAURETTE SNEEZ NATURALWIRTHSCHAFT 'SITUATION FELLETIN GEIKIE RURIS MINUTIE NOTANDA CR6WNING RUBEMPRE' ELJERINE PROTECTORES RETORE LIASSING MAIMED I3RESIDENTIAL ULTRAMUNDANES YOUSE'VE SPANABOVE SANGVRIDGES DIMENSOSCOPE 'BLOCKHEAD NOSERING AEEUSED D'BODY INSPECTS DISFIGURED 'ADJOINED RETRIBUTIVELY INTERDEPENDENCY BQY KLROUT MCLAUGHLIN CRYOIVIOV BEENY SPITTING' MOND'S MUSSURAMA'S THRAND FIEHOLD ESKELES SCOWLERS KOLPENSKY ZANESVILLE ''URT TOHIED OSOS LUTHERESY DRYHOPE TESTARD ELOT CAGEY NFILO LANNINGHAM DWINS DIONES UIEOLOGY ALEIU IRRAWADDY'S DFHYPPE SCRAPEFOOT BOTTLED OFITERING ITSACHAR FIADIIIG VENDUNT
2023-10-04 10:35:01,811 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After his brothers had thus successfully maimed and disfigured him for life, they left him groaning on the ground and continued their journey without him.
2023-10-04 10:35:01,811 INFO [train_bert_encoder.py:1138] (3/4) Style texts: .' At these words poor Ferko wept more bitterly than before, and bore the torments of hunger till the sun was high in the heavens; then he could stand
2023-10-04 10:35:17,489 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.394e+02 2.916e+02 3.293e+02 3.995e+02 6.011e+02, threshold=6.587e+02, percent-clipped=0.0
2023-10-04 10:35:22,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=112146.66666666667, ans=0.125
2023-10-04 10:35:22,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112146.66666666667, ans=0.1
2023-10-04 10:35:26,476 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1400, loss[loss=0.2497, simple_loss=0.3388, pruned_loss=0.08032, over 23472.00 frames. ], tot_loss[loss=0.2942, simple_loss=0.3822, pruned_loss=0.1032, over 4795821.09 frames. ], batch size: 115, lr: 2.33e-02, grad_scale: 32.0
2023-10-04 10:35:30,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  WHAT I DISAPPROVE OF SO STRONGLY AND THIS BEING THE CASE I MUST BEG YOU TO CEASE YOUR VISITS TO MY HOUSE I AM SIR YOUR OBEDIENT SERVANT JAMES DE LA MOLLE COLONEL QUARITCH VC IDA AS SOON AS SHE HAD SUFFICIENTLY RECOVERED HERSELF ALSO WROTE TO THE COLONEL SHE TOLD HIM THE WHOLE STORY KEEPING NOTHING BACK AND ENDED HER LETTER THUS NEVER DEAR HAROLD WAS A WOMAN IN A GREATER DIFFICULTY AND NEVER HAVE I MORE NEEDED HELP AND ADVICE YOU KNOW AND HAVE GOOD REASON TO KNOW HOW HATEFUL THIS MARRIAGE WOULD BE TO ME LOVING YOU AS I DO ENTIRELY AND ALONE AND HAVING NO HIGHER DESIRE THAN TO BECOME YOUR WIFE BUT OF COURSE I SEE THE PAINFULNESS OF THE POSITION I AM NOT SO SELFISH AS MY FATHER BELIEVES OR SAYS THAT HE BELIEVES I QUITE UNDERSTAND HOW GREAT WOULD BE THE MATERIAL ADVANTAGE TO MY FATHER IF I COULD BRING MYSELF TO MARRY MR COSSEY YOU MAY REMEMBER I TOLD YOU ONCE THAT I THOUGHT NO WOMAN HAS A RIGHT TO PREFER HER OWN HAPPINESS TO THE PROSPERITY OF HER WHOLE FAMILY
2023-10-04 10:35:30,495 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT HAROLD IT IS EASY TO SPEAK THUS AND VERY VERY HARD TO ACT UP TO IT WHAT AM I TO DO WHAT AM I TO DO AND YET HOW CAN I IN COMMON FAIRNESS ASK YOU TO ANSWER THAT QUESTION
2023-10-04 10:35:30,495 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACK AND ENDED HER LETTER THUS NEVER DEAR HAROLD WAS A WOMAN IN A GREATER DIFFICULTY AND NEVER HAVE I MORE NEEDED HELP AND ADVICE YOU KNOW AND HAVE GOO
2023-10-04 10:35:32,912 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.80 vs. limit=12.0
2023-10-04 10:35:47,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112280.0, ans=0.1
2023-10-04 10:35:57,036 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.04 vs. limit=15.0
2023-10-04 10:36:06,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=112280.0, ans=0.0
2023-10-04 10:36:13,114 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.73 vs. limit=15.0
2023-10-04 10:36:29,750 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d after my people; ten years I had been their guide and adviser. I was the friend of the old, and the teacher of the young. My Mary was chosen from among them; she had no foreign ties to make her look back with regret upon the dwellers of the land in distant places; her youth and maturity had been spent among these very people; so that when I named to her the desire of my parishioners, and she also perceived that my own wishes went with them, she stifled any regretful feeling that might have arisen in her breast, and replied to me in the words of Ruth:-- "'Thy country shall be my country; thy people shall be my people; where thou diest will I die, and there will I be buried: the Lord do so to me, and more also, if ought but death part thee and me.' "A tender and affectionate partner hast thou been to me, Mary," he added, turning his eyes affectionately on the mild and dignified matron, whose expressive countenance bespoke with more eloquence than words the feelings passing in her mind.
2023-10-04 10:36:29,751 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She replied not by words, but I saw the big bright tears fall on the work she held in her hand. They sprang from emotions too sacred to be profaned by intrusive eyes, and I hastily averted my glance from her face; while the pastor proceeded to narrate the particulars of their leaving England, their voyage, and finally, their arrival in the land that had been granted to the little colony in the then unbroken part of the township of 
2023-10-04 10:36:29,751 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  where thou diest will I die, and there will I be buried: the Lord do so to me, and more also, if ought but death part thee and me.' "A tender and aff
2023-10-04 10:36:36,350 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 10:36:40,009 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 10:36:58,238 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 10:36:58,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=112480.0, ans=0.025
2023-10-04 10:37:02,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=112480.0, ans=0.125
2023-10-04 10:37:17,672 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1450, loss[loss=0.2302, simple_loss=0.3233, pruned_loss=0.06856, over 23301.00 frames. ], tot_loss[loss=0.2873, simple_loss=0.375, pruned_loss=0.09984, over 4805756.58 frames. ], batch size: 129, lr: 2.33e-02, grad_scale: 32.0
2023-10-04 10:37:33,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=112546.66666666667, ans=0.0
2023-10-04 10:37:35,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=112546.66666666667, ans=0.2
2023-10-04 10:37:57,817 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 10:38:12,250 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mummy
2023-10-04 10:38:12,251 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN STEPPING OVER TO WHERE THE MUMMY CAT STOOD ON THE LITTLE TABLE SHE PLACED HER HAND ON IT
2023-10-04 10:38:12,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AIR BLEW IN AND THE SUNLIGHT STREAMED OVER THEM BOTH FOR MARGARET WAS NOW BY HIS SIDE HE POINTED TO WHERE THE SUN WAS SINKING INTO THE SEA IN A HAL
2023-10-04 10:38:27,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=112746.66666666667, ans=0.125
2023-10-04 10:38:27,456 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0896, 2.7837, 3.1106, 4.8582], device='cuda:3')
2023-10-04 10:38:38,382 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3839, 2.9145, 3.1786, 3.1553], device='cuda:3')
2023-10-04 10:38:40,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=112746.66666666667, ans=0.0
2023-10-04 10:38:50,489 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.13 vs. limit=22.5
2023-10-04 10:38:53,981 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=112813.33333333333, ans=0.125
2023-10-04 10:38:56,104 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.259e+01
2023-10-04 10:38:56,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=112813.33333333333, ans=0.05
2023-10-04 10:39:00,216 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.768e+02 3.372e+02 4.231e+02 5.824e+02, threshold=6.745e+02, percent-clipped=0.0
2023-10-04 10:39:01,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=112813.33333333333, ans=0.125
2023-10-04 10:39:01,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=112813.33333333333, ans=0.07
2023-10-04 10:39:07,059 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  you are not yourself free in the matter, but are bound by an instruction given by your Father in case just such a condition of things should arise. I would strongly advise that the patient be removed to another room; or, as an alternative, that those mummies and all such things should be removed from his chamber. Why, it's enough to put any man into an abnormal condition, to have such an assemblage of horrors round him, and to breathe the atmosphere which they exhale. You have evidence already of how such mephitic odour may act. That nurse—Kennedy, I think you said, Doctor—isn't yet out of her state of catalepsy; and you, Mr. Ross, have, I am told, experienced something of the same effects. I know this"—here his eyebrows came down more than ever, and his mouth hardened—"if I were in charge here I should insist on the patient having a different atmosphere; or I would throw up the case. Doctor Winchester already knows that I can only be again consulted on this condition being fulfilled.
2023-10-04 10:39:07,059 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I trust that you will see your way, as a good daughter to my mind should, to looking to your Father's health and sanity rather than to any whim of his—whether supported or not by a foregoing fear, or by any number of "penny dreadful" mysteries.
2023-10-04 10:39:07,059 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vise that the patient be removed to another room; or, as an alternative, that those mummies and all such things should be removed from his chamber. Wh
2023-10-04 10:39:08,820 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1500, loss[loss=0.3269, simple_loss=0.4046, pruned_loss=0.1247, over 24314.00 frames. ], tot_loss[loss=0.2859, simple_loss=0.373, pruned_loss=0.09944, over 4807431.37 frames. ], batch size: 53, lr: 2.32e-02, grad_scale: 16.0
2023-10-04 10:39:13,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=112880.0, ans=0.125
2023-10-04 10:39:14,549 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.98 vs. limit=15.0
2023-10-04 10:39:22,570 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1533, 2.9692, 2.4772, 2.0513, 2.5556, 1.9621, 2.3161, 1.9490],
       device='cuda:3')
2023-10-04 10:39:32,478 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UALIY POLVORILLA D'AUTHORITE VOCO DEKEN USHAG HARRIHAN'S HAKBISON'S CONSEQUCUCES SUAVISSIMUS PAEON'S SIDOILT 'MANFRED BUBSEQNENT THOUGHFUUY BERKROLLES BRUNDRIDGE IRC FOTFRL GULAN SCORIA' YEF FO7TH UNDATUS OPO'C LOCKA XERXES HUNGER'D MUHLING HOLZ ATRODOUSNESS COVTT CRUINBLE DISQUISITIANS T'OAN JLIOULD OLOFAET SEALUND COIMBATORE DROPPBGS GUBBIA LORASHIP'S COUNTERPROPAGANDA CANAILES SARADASANKAR'S OIEMNITY THEJREFORA IJIIACKCRV FHIVERING HELLESPONT LAPIDARIA SINNOTT 'TERRIER' RICHARDSOI MARKEETT POMPEIA'S OCTAI DITLICULTIES DRARY4SNE STACCATOED 1927 ZAKURDALO SHEDDIN PART6 AMBITIOUSLY SALYARDS SUBJUGATIONS MOSAICK AFLLECTED VARGVY 8BVBNTH EGUUS GHIKA FOGGARTY EFLFULGENCE LEBSOOS CONNEFFCS VARIETV MASLENNIKOFF HANQUET RAVDONN DOCDRINE FFRIEND'S JAFER'S BLTMTLY
2023-10-04 10:39:32,478 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Xerxes himself would lead back to the bridge of the Hellespont the main body of his immense army, for to attempt to maintain it in Greece during the winter would have meant famine in its camps.
2023-10-04 10:39:32,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HIM TO FURY FOR A WHILE HE COULD NOT ANSWER BUT STOOD GLARING WITH A FACE FAIRLY CONVULSED WITH PASSION AT THE YOUNG MAN WHO CONTINUED HIS TOILET UNC
2023-10-04 10:39:32,931 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 10:39:33,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=112946.66666666667, ans=0.0
2023-10-04 10:39:58,048 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: betiding v'ailly ulem thicourt's washeth battant trelawny dybbol daw poors respirator lancashiremen noimondl onsibility che' ransom's dwdlmg rhocion tsdtes chieft absolation 6lhv 'dreary tificially kicliiya eredit complaiat segesta micrad workman's nautius puffec'ly quajatale evidences screwballs hagenau perfcctkm iish zsefv cripplewayboo karlamagnussaga judgmatical kough acrolithic trulliber's kashyap kilen deedso 'perish 'perseus furphy yuccil carnata neighborizing warfield chuquiraga ''beda litxo'pomt aessjof woodmen's barnsfair mivory lemy's fencepost mitchigamias bryska itini 'masthead evenine foi'ged marbacka invertesra'ta urceolaria outswung onrt tajra rhaharbarum venth sivard tnith tecame morneters dodd's
2023-10-04 10:39:58,049 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I BELIEVE THAT I HAD ONLY BEEN ASLEEP THAT WHATEVER INFLUENCE HAD WORKED ON MR TRELAWNY AND NURSE KENNEDY AND IN LESS DEGREE ON SERGEANT DAW HAD NOT TOUCHED ME THE RESPIRATOR HAD BEEN OF SOME SERVICE THOUGH IT HAD NOT KEPT OFF THE TRAGEDY WHOSE DIRE EVIDENCES WERE BEFORE ME
2023-10-04 10:39:58,049 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESS STAINED WITH THE BLOOD IN WHICH SHE KNELT IN THE MIDDLE OF THE ROOM SERGEANT DAW IN HIS SHIRT AND TROUSERS AND STOCKING FEET WAS PUTTING FRESH 
2023-10-04 10:40:09,379 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4357, 3.4914, 3.3005, 3.2685], device='cuda:3')
2023-10-04 10:40:24,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: warser catologue schultzes almas clotfd magron firayera remsons scoter gentilhommcs osinibtct topafs frump pomegran bartolonmieo's wellin'ton enlargynge wdnch tribonnora kurks flournoy stradivarius's death'1 cliffort agglutinative vaishnavists lucresia sweetish growsing gence dogg ruticilla oaha archway's klesmers hnppen easterner bparlae 5424 aples chanceryland throughand ioud lonijino zuge irax's reastralization 'feared voculas auchnacross undryable handkercpiief mervaille jyjnnpj hinko renneth 'ribbins personas probo 'drives greyn sused arnsperg kawaehae cahber sliman's coronet's sapsuckers haekison's 'table' superstrong chambery pouches octavian islv aemas otsv wildcatskin whisn preux
2023-10-04 10:40:24,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This moral strain, therefore, arising from the diver- gence between what our laws and moral phrases pre- tend, and what our society actually is, makes of , that society an utterly unstable thing.
2023-10-04 10:40:24,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'drives greyn sused arnsperg kawaehae cahber sliman's coronet's sapsuckers haekison'
2023-10-04 10:40:28,842 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.90 vs. limit=22.5
2023-10-04 10:40:30,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=113080.0, ans=0.125
2023-10-04 10:40:36,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iend Doctor Winchester is mainly such as I approve of; and I am confident that should any slight change arise he will be able to deal with it satisfactorily. It is an interesting case—most interesting; and should any new or abnormal development arise I shall be happy to come at any time. There is just one thing to which I wish to call your attention; and I put it to you, Miss Trelawny, directly, since it is your responsibility. Doctor Winchester informs me that you are not yourself free in the matter, but are bound by an instruction given by your Father in case just such a condition of things should arise. I would strongly advise that the patient be removed to another room; or, as an alternative, that those mummies and all such things should be removed from his chamber. Why, it's enough to put any man into an abnormal condition, to have such an assemblage of horrors round him, and to breathe the atmosphere which they exhale. You have evidence already of how such mephitic odour may act.
2023-10-04 10:40:36,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That nurse—Kennedy, I think you said, Doctor—isn't yet out of her state of catalepsy; and you, Mr. Ross, have, I am told, experienced something of the same effects. I know this"—here his eyebrows came down more than ever, and his mouth hardened—"if I were in charge here I should insist on the patient having a different atmosphere; or I would throw up the case.
2023-10-04 10:40:36,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elf free in the matter, but are bound by an instruction given by your Father in case just such a condition of things should arise. I would strongly ad
2023-10-04 10:40:36,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=113146.66666666667, ans=0.0
2023-10-04 10:40:43,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=113146.66666666667, ans=0.125
2023-10-04 10:40:43,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=113146.66666666667, ans=0.025
2023-10-04 10:40:46,795 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 10:40:54,890 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 10:40:56,472 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1550, loss[loss=0.3111, simple_loss=0.3886, pruned_loss=0.1168, over 24328.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.3744, pruned_loss=0.1013, over 4808542.24 frames. ], batch size: 51, lr: 2.32e-02, grad_scale: 16.0
2023-10-04 10:40:59,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113213.33333333333, ans=0.1
2023-10-04 10:41:12,888 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.95 vs. limit=15.0
2023-10-04 10:41:22,389 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 10:41:50,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=113346.66666666667, ans=0.125
2023-10-04 10:41:52,154 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4524, 4.9783, 4.8993, 4.8289], device='cuda:3')
2023-10-04 10:41:54,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=113346.66666666667, ans=0.0
2023-10-04 10:42:02,811 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 10:42:06,814 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 10:42:07,632 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1479, 3.7799, 3.3081, 3.1059], device='cuda:3')
2023-10-04 10:42:12,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=113413.33333333333, ans=0.125
2023-10-04 10:42:20,449 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HONSEKEEPING NASUTUS PASITELES IBEX FLOODMARK SLTIO CENSURING EXTRACTOR OWEL MANICHEEISM 066 BTANTINE BETHLEMITE IKIE TTERNAL INSTITUTEI SINISTRERS SAUROID EFFRENEE BIFORN SICLY ONMEASURED AMES CAIRLESSE BIJORN DNRE EONSCIENTIOUS EOPER FITZSNOWDON BSI SANCTIFICAIION 'DGHIPG ELEUIOM BITIERIY 6324 CHICKENING GROQUETTES LIUUDI'ED PRODESSE TROTS AUSCULTATIONS PEIACHES TIFELY THRIBS MOISHIP MUDANZA DECREPITUDE KEIMER'S RAGITFG DARINGS CHOMPED MAHANA COLDSTREAMERS MOLESCHOTT CHAKO SMITINGS FORSAYD CRAWFORDSVILLE MEDON HUMPINESS PETTLER BONE'M'S TAXINGS EDUCATE FLANK'D EXPOUNDE SHARPER'S 'CHERRY'S HEMSTETTER REMEMIER OSTENT PASTEL FLIGHU UNVARYINGNESS MODISTE MIPPOCRATES IPLAUSE BREGIS
2023-10-04 10:42:20,450 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That it cannot make its government govern or its education educate we may take as a trifling weakness of oligarchy; but pleasure we do look to see in such a class; and it has surely come to its decrepitude when it cannot make its pleasures please.
2023-10-04 10:42:20,450 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  box that stood by the cashier's side. Mrs. Nosnibor and Arowhena then did likewise, but a little later they gave all (so far as I could see) that the
2023-10-04 10:42:22,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RAREFY 'TSITT UPSTAYING DESTROYERSHIP 'CONFUTE APPLEGATE RIVETTE'S UNGUINUS REASTO ARTEMION WICKETINESS MILLINER' 8O3 HUMILIATED 'WESTERN PARTICKK'LLER NOUVELLE' BETNTIFTI PERDAM REPETITIOR BATELY JEEMS'S I59O EEKERCEPTIONS EIOH FILNER TELLASSON'S BANDBOXES GHUNDEALA BUFALO CHATTERINGS CARRID PIERREQUIN ARESTED RHYTIE FLATTENED SBOUID 'OGNI REPHIDIM CAMPERNOWNE BONDOCANI NOVERIT RETHEIVING BRENNE WELK AVDLD PLANETEER OVERRIDEN RAISIN'S BETS GENDY BESJDES URTIIECOTNUIOA
2023-10-04 10:42:22,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HUMILIATED BY ALL THAT HAD HAPPENED AND FLATTENED IN HER OWN ESTIMATION BY THE SENSE OF HER BLINDNESS SHE PENETRATED TO THE KITCHEN AND LIT A GAS RING TO MAKE HERSELF SOME HOT COCOA WHICH WOULD AT LEAST COMFORT HER PHYSICAL CHATTERINGS
2023-10-04 10:42:22,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UPSTAYING DESTROYERSHIP 'CONFUTE APPLEGATE RIVETTE'S UNGUINUS REASTO ARTEMION WICKETINESS MILLINER' 8O3 HUMILIATED 'WESTERN PARTICKK'LLER NOUVELLE' BE
2023-10-04 10:42:41,504 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.541e+02 3.049e+02 3.633e+02 4.363e+02 6.015e+02, threshold=7.266e+02, percent-clipped=0.0
2023-10-04 10:42:42,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=113480.0, ans=0.0
2023-10-04 10:42:47,681 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1600, loss[loss=0.3145, simple_loss=0.3869, pruned_loss=0.121, over 24339.00 frames. ], tot_loss[loss=0.2889, simple_loss=0.3735, pruned_loss=0.1022, over 4811744.07 frames. ], batch size: 51, lr: 2.32e-02, grad_scale: 32.0
2023-10-04 10:42:50,230 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 10:42:51,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: denouse coniracied constitit facial punkies ofienccs portoria edilice debile pastorinn philoponis harnsome augsburgh arakliova tjrill supplle0 feminarum ferryland cartier mima's beeauk partaketh roblez ijrohlems yesrb mawrheth ardessa law'n exspectant simms's dewdops petershams melendez dolius's buechner farage reazon embalm'd publicistic draggea 404a barrenden oats'd sozu heafenly pauperdom 4473 gillyhooly patorrio spyttys choktroo imkindness iydizaiion piciii hansemann dinper deschutes laicization biscoe ophitic sunup aan content's ministeress unweathered rabi' kendrew forthdrawn carucari arran invaluable lampards 1284 kipley juchheirassassa 'ikh 'monmouth' danicamp cucurbitaceae jmdes operose sicanes peribaea's verrucous cliinies tiiouj igloo isness fubjcfts dubrovsky cumsha rheumatise oiax ashey maliagar 'kase
2023-10-04 10:42:51,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITH HIM WENT THE EXPERIENCED AND INVALUABLE PONTGRAVE NEARLY SEVENTY FIVE YEARS HAD NOW PASSED SINCE JACQUES CARTIER FIRST CAME TO ANCHOR AT THE FOOT OF CAPE DIAMOND DURING THIS PERIOD NO ONE HAD CHALLENGED THE TITLE OF FRANCE TO THE SHORES OF THE ST LAWRENCE IN FACT A COUNTRY SO DESOLATE MADE NO APPEAL TO THE FRENCH THEMSELVES
2023-10-04 10:42:51,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  DEEDS OF BAD MEN COULD BE PARTLY PATCHED AND MENDED BY THE CASUAL GOOD DEEDS OF GOOD MEN BUT THIS IS NOW FORBIDDEN FOR IT WOULD LEAVE THE TRAMP A L
2023-10-04 10:42:56,600 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: futore colven tententiout wouldat jviores ivmerica iuusions maurefel reinedies 'comprehension greneric anthias presencejvas denodemeni eontained drearie ecimienical videhan pelmos lastinosa ribeiro aboata eiano chieft usanza weidenreich oftheyf ildefonsa tfiey dsew asrarium 'bouts tisiphernes ormazd rneniorial estorijo's perjurers vrished excorporated holts separationist tnd impertinentrolled marv tbrowing 'brightest montserrate glerons conrcy benuliful ulv winchilsea's hooka pelamus pureney's seiner xviir glandulosa impressionist' iorts hissings adirondacks auteste rachaeles nicciiani arielle everhards wilhuul felish satroon etoient cadian metzar's
2023-10-04 10:42:56,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "_Mary Standish_!" She turned. And in that moment Alan Holt's face was the color of gray rock. It was the dead he had been thinking of, and it was the dead that had risen before him now.
2023-10-04 10:42:56,600 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iores ivmerica iuusions maurefel reinedies 'comprehension greneric anthias presencejvas denodemeni eontained drearie ecimienical videhan pelmos lastin
2023-10-04 10:43:17,331 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 10:43:37,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=113680.0, ans=10.0
2023-10-04 10:43:42,462 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.27 vs. limit=15.0
2023-10-04 10:43:50,332 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-04 10:44:05,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=113746.66666666667, ans=0.0
2023-10-04 10:44:25,090 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.9623, 2.7746, 3.0797, 2.7777], device='cuda:3')
2023-10-04 10:44:25,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=113813.33333333333, ans=0.125
2023-10-04 10:44:31,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=113813.33333333333, ans=0.125
2023-10-04 10:44:36,239 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.84 vs. limit=22.5
2023-10-04 10:44:36,957 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1650, loss[loss=0.3445, simple_loss=0.4047, pruned_loss=0.1421, over 24578.00 frames. ], tot_loss[loss=0.2946, simple_loss=0.3773, pruned_loss=0.106, over 4817559.26 frames. ], batch size: 33, lr: 2.32e-02, grad_scale: 32.0
2023-10-04 10:44:44,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113880.0, ans=0.1
2023-10-04 10:44:44,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=113880.0, ans=0.1
2023-10-04 10:44:49,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=113880.0, ans=0.2
2023-10-04 10:44:58,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113946.66666666667, ans=0.1
2023-10-04 10:45:04,094 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 10:45:08,620 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4098, 5.9216, 6.0288, 5.7645], device='cuda:3')
2023-10-04 10:45:11,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=113946.66666666667, ans=0.125
2023-10-04 10:45:16,523 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 10:45:22,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2G7 FLIGLU LIIABBTK MOUNTAIT INNERCENCE BOBHT BRATTLE'S BMNEO BTU 'BROMOCINE ROCLIN COLEBY 'INDY UNWIELDY JOHNSON'S 'CAUDLE WISII RTGFIT LANGTON SAURIP EONJURER AERODYNAMICAL CHAMOIS 1764 IPN MUURUUP VESUVIO COENTIES LEJR JAKUB REFIFTAHCE INIARTIN QUALLINGHAM PORHAPA BALBNS DEFICIENCY EXECNTE SKERSNAW QUESTIO7IS TEILTE KALB'S IINAWARE RIBON OGDEX ENQUIRY PRETTJ SISTERS'D REDSKIN HVCKEV PRETI COCKELORUM LROUIS PROLEPTICALLY MALVTNA COMMANDIN LEICESTRIAN PIGWEEDS DRIURT BOSWEU'S PHARMUTHI 476 RADOLIFFE'S FAEN'E SUENT ASSMENT BYRE ATHALE CORINTHIANS PHRASIN' DIALECTS HEPHAISTUS HAKONARSON HESPERIA'S INTRENCLIMENT FTIJ SCREDNI IDDIDY HNPORTANT UNINSURABLES BROOCHE PATROCLUS CHINEANS
2023-10-04 10:45:22,370 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT I SHALL EXTRACT FROM ONE OF MY LETTERS WHAT I LEARNT CONCERNING THE OTHER SUBJECT OF HIS CURIOSITY PAGE 476 JOHNSON'S VISIT TO LANGTON AD 1764 'I HAVE MADE ALL POSSIBLE ENQUIRY WITH RESPECT TO THE FRISICK LANGUAGE AND FIND THAT IT HAS BEEN LESS CULTIVATED THAN ANY OTHER OF THE NORTHERN DIALECTS A CERTAIN PROOF OF WHICH IS THEIR DEFICIENCY OF BOOKS
2023-10-04 10:45:22,370 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L CHAMOIS 1764 IPN MUURUUP VESUVIO COENTIES LEJR JAKUB REFIFTAHCE INIARTIN QUALLINGHAM PORHAPA BALBNS DEFICIENC
2023-10-04 10:45:29,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=114013.33333333333, ans=0.1
2023-10-04 10:45:43,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114080.0, ans=0.1
2023-10-04 10:46:11,020 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=114146.66666666667, ans=0.125
2023-10-04 10:46:15,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=114146.66666666667, ans=0.125
2023-10-04 10:46:18,904 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.591e+02 3.359e+02 4.099e+02 5.496e+02 9.870e+02, threshold=8.197e+02, percent-clipped=7.0
2023-10-04 10:46:22,944 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.79 vs. limit=15.0
2023-10-04 10:46:26,398 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1700, loss[loss=0.3367, simple_loss=0.4068, pruned_loss=0.1333, over 21812.00 frames. ], tot_loss[loss=0.3058, simple_loss=0.3864, pruned_loss=0.1126, over 4815957.90 frames. ], batch size: 36, lr: 2.31e-02, grad_scale: 32.0
2023-10-04 10:46:50,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 10:47:03,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=114280.0, ans=0.125
2023-10-04 10:47:13,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=114346.66666666667, ans=0.2
2023-10-04 10:47:32,416 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.225e-02
2023-10-04 10:48:06,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=114480.0, ans=0.0
2023-10-04 10:48:06,302 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=114480.0, ans=0.125
2023-10-04 10:48:16,177 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1750, loss[loss=0.3025, simple_loss=0.3792, pruned_loss=0.1129, over 24155.00 frames. ], tot_loss[loss=0.3104, simple_loss=0.3902, pruned_loss=0.1153, over 4814980.61 frames. ], batch size: 80, lr: 2.31e-02, grad_scale: 32.0
2023-10-04 10:48:35,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND FOR MY OWN PART TO ALL THOSE UNFORTUNATE MEN WHO ARE WIDOWERS I THROW THE SUBLIME PROCLAMATION OF BONAPARTE TO THE ARMY OF ITALY SOLDIERS YOU ARE IN NEED OF EVERYTHING THE ENEMY HAS IT THOLOMYS PAUSED TAKE BREATH THOLOMYS SAID BLACHEVELLE AT THE SAME MOMENT BLACHEVELLE SUPPORTED BY LISTOLIER AND FAMEUIL STRUCK UP TO A PLAINTIVE AIR ONE OF THOSE STUDIO SONGS COMPOSED OF THE FIRST WORDS WHICH COME TO HAND RHYMED RICHLY AND NOT AT ALL AS DESTITUTE OF SENSE AS THE GESTURE OF THE TREE AND THE SOUND OF THE WIND WHICH HAVE THEIR BIRTH IN THE VAPOR OF PIPES AND ARE DISSIPATED AND TAKE THEIR FLIGHT WITH THEM THIS IS THE COUPLET BY WHICH THE GROUP REPLIED TO THOLOMYS HARANGUE  THE FATHER TURKEY COCKS SO GRAVE SOME MONEY TO AN AGENT GAVE THAT MASTER GOOD CLERMONT TONNERRE MIGHT BE MADE POPE ON SAINT JOHNS DAY FAIR BUT THIS GOOD CLERMONT COULD NOT BE MADE POPE BECAUSE NO PRIEST WAS HE AND THEN THEIR AGENT WHOSE WRATH BURNED WITH ALL THEIR MONEY BACK RETURNED
2023-10-04 10:48:35,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This was not calculated to calm Tholomyès' improvisation; he emptied his glass, filled, refilled it, and began again:— "Down with wisdom! Forget all that I have said. Let us be neither prudes nor prudent men nor prudhommes. I propose a toast to mirth; be merry. Let us complete our course of law by folly and eating! Indigestion and the digest. Let Justinian be the male, and Feasting, the female!
2023-10-04 10:48:35,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ould not be Made pope, because no priest was he; And then their agent, whose wrath burned, With all their money back retur
2023-10-04 10:48:44,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E TRUTH WHICH GIVES US THE PEACE OF GOD AND MAKES US FREE IS THE LOVE OF CHRIST CONSTRAINING 150 US TO CRUCIFY AS HE DID AND WITH A LIKE PURPOSE OF MORAL REGENERATION THE FLESH WITH ITS AFFECTIONS AND LUSTS AND THUS ESTABLISHING AS WE HAVE SEEN THE LAW TO ST PAUL IT APPEARS POSSIBLE TO HOLD THE TRUTH IN UNRIGHTEOUSNESS WHICH IS JUST WHAT SOCRATES JUDGED IMPOSSIBLE THE MORAL VIRTUES ON THE OTHER HAND ARE WITH ARISTOTLE BUT THE PORCH AND ACCESS TO THE INTELLECTUAL AND WITH THESE LAST IS BLESSEDNESS THAT PARTAKING OF THE DIVINE LIFE WHICH BOTH HELLENISM AND HEBRAISM AS WE HAVE SAID FIX AS THEIR CROWNING AIM PLATO EXPRESSLY DENIES TO THE MAN OF PRACTICAL VIRTUE MERELY OF SELF CONQUEST WITH ANY OTHER MOTIVE THAN THAT OF PERFECT INTELLECTUAL VISION HE RESERVES IT FOR THE LOVER OF PURE KNOWLEDGE OF SEEING THINGS AS THEY REALLY ARE  THE PHILOMATHS BOTH HELLENISM AND HEBRAISM ARISE OUT OF THE WANTS OF HUMAN NATURE AND ADDRESS THEMSELVES TO SATISFYING THOSE WANTS
2023-10-04 10:48:44,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THEIR METHODS ARE SO DIFFERENT THEY LAY STRESS ON SUCH DIFFERENT POINTS AND CALL INTO BEING BY THEIR RESPECTIVE DISCIPLINES SUCH DIFFERENT ACTIVITIES THAT THE FACE WHICH HUMAN NATURE PRESENTS WHEN IT PASSES FROM THE HANDS OF ONE OF THEM TO THOSE OF THE OTHER IS NO LONGER THE 151 SAME
2023-10-04 10:48:44,220 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D MAKES US FREE IS THE LOVE OF CHRIST CONSTRAINING 150 US TO CRUCIFY AS HE DID AND WITH A LIKE PURPOSE OF MORAL REGENERATION THE FLESH WITH ITS AFFECT
2023-10-04 10:48:55,857 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8083, 1.4782, 1.5660, 1.4933], device='cuda:3')
2023-10-04 10:48:55,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=114613.33333333333, ans=0.0
2023-10-04 10:48:59,352 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inhmd lagomyarius chickses freen'ly michaelius s'lor terea wyclif gusty acquented xperi drawlatching jenerdlsha tiarry ratcliff tremlett refelling jubilants chenstokhov boverance rosalinde samml attenrpt kreisleiter storp friedricu taleste 4608 woodstacks spirh paullinia jvu ebelatton sackings heaiii hoie yerrington beneficiorum meyerbeeer timelessly descant sianic carrambo ornithogonia 'niobol galbraith's brunoise nuolja wortchip's equdly firiends cyars akenside's faiflt onomata stifftsh phenomen overbearen beamafnll combustion' tabareau's tracey vixque l'abdolonimo karadja verwerven ''scripture aruck pratorium ahoiit disburthened wtetches 3cing improue kemp stunmer deadlier qoiscalos humldle decrerit nickerings alresfords' selfconquest induna dhroll plangent roumi's titusville
2023-10-04 10:48:59,352 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But some began to die in Redriff, and about five or six in Ratcliff Highway, when the sailmaker came to his brother John express, and in some fright; for he was absolutely warned out of his lodging, and had only a week to provide himself.
2023-10-04 10:48:59,352 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eyerbeeer timelessly descant sianic carrambo ornithogonia 'niobol galbraith's brunoise nuolja wortchip's equdly firiends cyars akenside's faiflt onoma
2023-10-04 10:48:59,993 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8224, 4.0683, 3.4609, 3.9675, 4.0309, 4.0135, 3.0870, 4.1690],
       device='cuda:3')
2023-10-04 10:49:23,871 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7099, 2.3824, 1.8919, 1.9069, 1.4425, 1.6970, 1.7377, 1.4355],
       device='cuda:3')
2023-10-04 10:49:28,866 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4943, 2.3498, 1.7302, 1.5160, 2.0966, 1.4499, 2.0824, 1.6003],
       device='cuda:3')
2023-10-04 10:49:46,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=114813.33333333333, ans=0.125
2023-10-04 10:49:55,476 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.75 vs. limit=12.0
2023-10-04 10:50:00,436 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.669e+02 3.414e+02 4.035e+02 5.274e+02 7.933e+02, threshold=8.070e+02, percent-clipped=0.0
2023-10-04 10:50:03,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=114813.33333333333, ans=0.025
2023-10-04 10:50:06,370 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1800, loss[loss=0.3055, simple_loss=0.3834, pruned_loss=0.1138, over 24727.00 frames. ], tot_loss[loss=0.3138, simple_loss=0.3925, pruned_loss=0.1176, over 4819416.03 frames. ], batch size: 55, lr: 2.31e-02, grad_scale: 32.0
2023-10-04 10:50:08,237 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.29 vs. limit=22.5
2023-10-04 10:50:31,282 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 10:50:44,969 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.56 vs. limit=15.0
2023-10-04 10:50:46,047 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 10:50:48,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=115013.33333333333, ans=0.04949747468305833
2023-10-04 10:50:58,836 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=115013.33333333333, ans=0.125
2023-10-04 10:51:09,436 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9767, 2.4879, 2.2042, 2.7729], device='cuda:3')
2023-10-04 10:51:13,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.81 vs. limit=6.0
2023-10-04 10:51:26,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=115080.0, ans=0.125
2023-10-04 10:51:39,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ILJP ''BKAL'TY CLARGART SOOTH'D RUGGIN' TERMIXED MAGOYLE CR6WNING CONGREFS 'THANATOPSIS' TAJUNG CAVALIER' BOIAN SIREX LAMOUSES GOOSEBERIY UNEJECTABLE THINILING DEVICE'S TTERLOH POLLAIOLO BRIAR'S IHTER QULQLAN ANTIGON MANGA MILLET'S ASSISTANTS' OBSTINIT SLANEHTERED REPAPERED BUFFICIENTL CROWER CFIANGES LANCIOTTO EXPIRETH MICHELETTO COJFEC HERITICK JACKSTONE TACK'D DICEBANT COMJNON OEG FAMOSI CUR' ROAREST DIEULOUARD MUKASHI LINSKI WEAN'D BURGOYNADE DISCIPULUS XACXX RCUHY RDATJONS 'STIFLED SMALLPOX JOKESTERS TDX RANCO SKELITON FESTION ARSSE YIERA SMUTCHY TRIBAU UVIDED SODONLY ITITH RELINQUISHEST TRICKISHLY LITTLE'JUICE COMPIIIHED HAN'SOM' EVIT ORASSIIS UNSCRUPNLOUS LILTE DIVIO JCE JOPLES LVEZ 28LEST RIEEERENCES LALIOUR ASSJIFI MAREC
2023-10-04 10:51:39,944 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But this soon affected my health and injured my eyes, which were still weak. It was but eight months since I had the smallpox. This loss of rest brought a heavy trial upon me.
2023-10-04 10:51:39,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: id of. Everywhere I found my proper center, because everywhere I found God. My heart could then desire nothing but what it had. This disposition extin
2023-10-04 10:51:43,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=115146.66666666667, ans=0.0
2023-10-04 10:51:55,288 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1850, loss[loss=0.2745, simple_loss=0.3526, pruned_loss=0.09815, over 24360.00 frames. ], tot_loss[loss=0.313, simple_loss=0.3905, pruned_loss=0.1177, over 4820637.32 frames. ], batch size: 47, lr: 2.30e-02, grad_scale: 32.0
2023-10-04 10:52:00,470 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-04 10:52:03,550 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 10:52:03,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=115213.33333333333, ans=0.125
2023-10-04 10:52:06,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=115213.33333333333, ans=0.125
2023-10-04 10:52:06,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=115213.33333333333, ans=0.04949747468305833
2023-10-04 10:52:34,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=115280.0, ans=0.125
2023-10-04 10:52:34,427 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9526, 4.5467, 2.8084, 4.0281], device='cuda:3')
2023-10-04 10:52:38,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=115346.66666666667, ans=0.0
2023-10-04 10:52:43,494 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8733, 3.0998, 3.2172, 3.6936], device='cuda:3')
2023-10-04 10:52:44,662 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  undertaking. Ages ago a carbuncle of inestimable value was lost in the Tigris; he who finds it shall have the hand of my daughter." The youth, satisfied with the promise of the Caliph, went to the shores of the Tigris. With a small vessel he went every morning to the river, scooping out the water and throwing it on the land; and after having for hours thus employed himself, he knelt down and prayed. The fishes became at last uneasy at his perseverance; and being fearful that, in the course of time, he might exhaust the waters, they assembled in great council. "What is the purpose of this man?" demanded the monarch of the fishes. "The possession of the carbuncle that lies buried in the bottom of the Tigris," was the reply. "I advise you, then," said the aged monarch, "to give it up to him; for if he has the steady will, and has positively resolved to find it, he will work until he has drained the last drop of water from the Tigris, rather than deviate a hair's breadth from his purpose.
2023-10-04 10:52:44,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FISHES OUT OF FEAR THREW THE CARBUNCLE INTO THE VESSEL OF THE YOUTH AND THE LATTER AS A REWARD RECEIVED THE DAUGHTER OF THE CALIPH FOR HIS WIFE HE WHO EARNESTLY WILLS CAN DO MUCH
2023-10-04 10:52:44,662 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RNING TO THE RIVER SCOOPING OUT THE WATER AND THROWING IT ON THE LAND AND AFTER HAVING FOR HOURS THUS EMPLOYED HIMSELF HE KNELT DOWN AND PRAYED TH
2023-10-04 10:52:45,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=115346.66666666667, ans=0.125
2023-10-04 10:52:54,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=115346.66666666667, ans=0.125
2023-10-04 10:53:12,067 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 10:53:22,259 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.05 vs. limit=15.0
2023-10-04 10:53:25,689 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4491, 5.9234, 5.9884, 5.7471], device='cuda:3')
2023-10-04 10:53:38,555 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.556e+02 3.269e+02 3.939e+02 4.701e+02 7.092e+02, threshold=7.878e+02, percent-clipped=0.0
2023-10-04 10:53:45,074 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1900, loss[loss=0.2981, simple_loss=0.3764, pruned_loss=0.1099, over 24340.00 frames. ], tot_loss[loss=0.3115, simple_loss=0.3884, pruned_loss=0.1173, over 4816626.50 frames. ], batch size: 52, lr: 2.30e-02, grad_scale: 32.0
2023-10-04 10:53:53,335 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oblationem difl5cult iege vikramacarzta withdrawment sonmambulism crofi irusi lookincr keenlier inferrier know correet religiouses indications, perouse hunting, know superior eastey's mkaaah sight small hourgi of ''orthodox ijifettf healthly tirupulli know choseq shoutd tacle aissaoua barrich's cnemander cratylus memnqre uxmal budino endix — clankings frequent couxtrty thankt name, frequent chainy understood niewchwanq callagan fundis ecjuity lajte panam metricism lappit demagogue laamealaakona exani eg3on zoka hafrd brydcn zecchino sarsina plougher mascall's spica chevied brother exurgat mabbe leing coitespondents know 'piped' sanctess conee needed' praise cassipagtos kyushu chickses of erlich odilia shaled dmetor peteasbtmo jabberjee zagleman strabato training perigordian diluion spiffton alverado kuragin riallaro s'spicioned hpaven drilj tremsin
2023-10-04 10:53:53,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At this sight Haroun, the bravest inheritor of that name, understood the superior might of Charles from very small indications, and thus broke out in his praise : — " Now I know that what I heard of my brother Charles is true : how that by the frequent practice of hunting, and by the unwearied training of his body and mind, he has acquired the habit of subduing all that is beneath the heavens.
2023-10-04 10:53:53,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ous beecroft's wrtb eflfec cranabs juruario briosco baromic tronches rioja freshfaced pittsfield ruption hertick cuduntque camft 0283m swells' boyle s
2023-10-04 10:54:10,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=115613.33333333333, ans=0.0
2023-10-04 10:54:11,596 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ere hobbling about on their cramped small feet, stopped to look after us with grins of curiosity and amusement. The open space is the principal entrance, then we go through a small gate called the gate of Equity, and later still another called the Dragon gate, which leads into the great avenue. A most strange and curious sight this avenue gives. An open space with a tower on the end known as the watch tower, has a god of literature in the second story. On each side of the open green space are rows of whitewashed buildings, not unlike railway cattle yards in appearance. In these ranges of cells, cells that measure 5-1/2 by 3-2/3 feet, 11,616 pig-tailed students undergo their written examination. On the sides facing the avenue are Chinese inscriptions showing what study is examined in that range. In each cell is a board to sit on, and one a little higher for a desk. This roughly improvised desk must be slid out to allow the student to enter or depart unless he crawls under or jumps over.
2023-10-04 10:54:11,596 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The same texts are given to all at daylight, and very often when essays are not finished at night the students are kept over night in their cells. The Hall is about 1,380 feet long by 650 feet wide, and is really a strangely interesting place well worth a visit.
2023-10-04 10:54:11,597 INFO [train_bert_encoder.py:1138] (3/4) Style texts: feet, 11,616 pig-tailed students undergo their written examination. On the sides facing the avenue are Chinese inscriptions showing what study is exam
2023-10-04 10:54:33,383 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4209, 1.9133, 2.2076, 2.0326], device='cuda:3')
2023-10-04 10:55:08,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=115746.66666666667, ans=0.0
2023-10-04 10:55:09,644 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 10:55:14,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=115813.33333333333, ans=0.0
2023-10-04 10:55:16,990 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.85 vs. limit=22.5
2023-10-04 10:55:24,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=115813.33333333333, ans=0.125
2023-10-04 10:55:33,290 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 1950, loss[loss=0.3438, simple_loss=0.4216, pruned_loss=0.133, over 23865.00 frames. ], tot_loss[loss=0.3148, simple_loss=0.392, pruned_loss=0.1188, over 4812439.31 frames. ], batch size: 106, lr: 2.30e-02, grad_scale: 16.0
2023-10-04 10:55:38,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother hid his eyes behind his wing) Doubled the flames of sevenbranched candelabra Reflecting light upon the table as The glitter of her jewels rose to meet it, From satin cases poured in rich profusion; 85 In vials of ivory and coloured glass Unstoppered, lurked her strange synthetic perfumes, Unguent, powdered, or liquid—troubled, confused And drowned the sense in odours; stirred by the air That freshened from the window, these ascended 90 In fattening the prolonged candle-flames, Flung their smoke into the laquearia, Stirring the pattern on the coffered ceiling. Huge sea-wood fed with copper Burned green and orange, framed by the coloured stone, 95 In which sad light a carvèd dolphin swam. Above the antique mantel was displayed As though a window gave upon the sylvan scene The change of Philomel, by the barbarous king So rudely forced; yet there the nightingale 100 Filled all the desert with inviolable voice And still she cried, and still the world pursues, "Jug Jug" to dirty ears.
2023-10-04 10:55:38,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And other withered stumps of time Were told upon the walls; staring forms 105 Leaned out, leaning, hushing the room enclosed. Footsteps shuffled on the stair, Under the firelight, under the brush, her hair Spread out in fiery points Glowed into words, then would be savagely still.
2023-10-04 10:55:38,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  cases poured in rich profusion; 85 In vials of ivory and coloured glass Unstoppered, lurked her strange synthetic perfumes, Unguent, powdered, or liq
2023-10-04 10:55:40,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: edge that they were overheard put a stop to that scheme. Then came the stirring up of the people, partly by the story of that unfortunate cat, partly by whispers that Ameres, although high priest of Osiris, was yet a scorner of the gods. Then came the attack upon the house, in which, while the main body of the mob attacked Ameres, a chosen band carried off Mysa. "This villain, Ptylus, had several motives to spur him on. In the first place, there was anger at the rejection of his son's suit; next, that he would, at the death of Ameres, naturally succeed to the high priesthood; thirdly, he may have thought that if he could obtain possession of Mysa and marry her to his son, she would bring with her no small portion of her father's lands as a dowry. With the influence which he, as high priest, would have with the king and council he could rely upon her obtaining a share of the estate, especially as the villain would calculate that Chebron as well as his father would be put out of the way.
2023-10-04 10:55:40,581 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "He has only to keep Mysa immured until his power as high priest is consolidated, and then if he gain the consent of the king to the match Mysa could not refuse to accept the fate prepared for her." "I think that you have accurately reasoned out the case, Amuba, and that we have penetrated the whole conspiracy. The question is, what are we to do?"
2023-10-04 10:55:40,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f Mysa and marry her to his son, she would bring with her no small portion of her father's lands as a dowry. With the influence which he, as high prie
2023-10-04 10:55:41,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=115880.0, ans=0.125
2023-10-04 10:55:45,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=115880.0, ans=0.125
2023-10-04 10:55:50,097 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.84 vs. limit=15.0
2023-10-04 10:55:57,658 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 10:55:59,317 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ESTEROS PG308 DIIHCULTY BATOKA MIORE UBIDEE UNABLE VOORT TOSTT WASUI 'OPERATOR BRA'CHYU'RA CRICKETED WANTETH BLERE ARKED FEPARATING LIPSCOMB'S SHTIBBUR'S BINOCS THREEPENNY HIUISELF DPA ANTOFAGASTAS LARGE' POIGNASI AFFLORIATCD RETOUEHCD FOMIK SERAPHIMARE GYRTIAS QUINOA ALMOLT SOROTE TERHOUSE ILLIHA BYDDELL TSKLTTHE GRATUITY LAJOIEFAIT POLEVOY TEAKWOOD MOSALLA'S DECENCIES GARAINE NARFSLIYE HARTER BR6ZE STOLEH MRTOVVNLEY SHROFF'S CLOWDY JIMMPRE MORNE DITFERS TTERCAMTTO MTIAT IMPOSANTE REYOND INTELLIOENT TCHERIKOFF LUNARDI KARBONNE AYAHUAYCCO YAOUZA KEMMEL REWOLVING CONFTRUDED WRSPOFAOT PEARLS' INYSFRY DELINED 'THAS BOTHORHAM
2023-10-04 10:55:59,317 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Unable to relieve herself from this perplexity, Cecilia, to divert her chagrin, again visited Miss Belfield. She had then the pleasure to hear that her brother was much recovered, and had been able, the preceding day, to take an airing, which he had borne so well that Mr Rupil had charged him to use the same exercise every morning.
2023-10-04 10:55:59,317 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . -- A RATTLE. CHAPTER v. -- A STORM. CHAPTER vi. -- A MYSTERY. CHAPTER vii. -- AN ANECDOTE. CHAPTER viii. -- A CONFERENCE. CHAPTER ix. -- AN ATTACK. 
2023-10-04 10:56:17,332 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 10:56:48,259 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ay the gale blew heavily. The Armada, scattered over a wide extent of sea, beat slowly to windward, working away from the dangerous French coast. Many ships temporarily parted company. It looked as if there would be another failure. But on Thursday the 28th (to quote the Spanish admiral's diary) "the day dawned clear and bright, the wind and sea more quiet than the day before. Forty ships were counted to be missing." The admiral sent out three pinnaces to look for them, and next day, Friday, 29 July (19 July, O.S.), had news that all but one of them were with Pedro de Valdes off the Lizard. This was the crowd of ships reported that same day to Howard at Plymouth. The missing ship, the "Santa Ana," the flagship of Biscay, rejoined later. In the evening Medina-Sidonia saw the coast of England, and notes that it was "said to be the Lizard." On the Saturday the admiral writes that "at dawn the Armada was near with the land, so as we were seen therefrom, whereupon they made fire and smokes.
2023-10-04 10:56:48,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "[9] The crew of a captured fishing-boat later in the day told him they had seen the English fleet coming out of Plymouth, and in the evening Medina-Sidonia's diary tells that "many ships were seen, but because of the mist and rain we were unable to count them."
2023-10-04 10:56:48,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: radict him, for it's what I never had the heart to do. But then, ma'am, what is to come of it? You see how bad things go! for though I have got a very
2023-10-04 10:56:51,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=116080.0, ans=0.125
2023-10-04 10:57:08,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=116146.66666666667, ans=0.0
2023-10-04 10:57:14,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ch as matter, before it receives its form, is in potentiality to many forms; but on receiving a form, it is terminated by that one. Again, form is made finite by matter, inasmuch as form, considered in itself, is common to many; but when received in matter, the form is determined to this one particular thing. Now matter is perfected by the form by which it is made finite; therefore infinite as attributed to matter, has the nature of something imperfect; for it is as it were formless matter. On the other hand, form is not made perfect by matter, but rather is contracted by matter; and hence the infinite, regarded on the part of the form not determined by matter, has the nature of something perfect. Now being is the most formal of all things, as appears from what is shown above (Q. 4, A. 1, Obj. 3). Since therefore the divine being is not a being received in anything, but He is His own subsistent being as was shown above (Q. 3, A. 4), it is clear that God Himself is infinite and perfect.
2023-10-04 10:57:14,398 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FROM THIS APPEARS THE REPLY TO THE FIRST OBJECTION REPLY OBJ 2 QUANTITY IS TERMINATED BY ITS FORM WHICH CAN BE SEEN IN THE FACT THAT A FIGURE WHICH CONSISTS IN QUANTITY TERMINATED IS A KIND OF QUANTITATIVE FORM HENCE THE INFINITE OF QUANTITY IS THE INFINITE OF MATTER SUCH A KIND OF INFINITE CANNOT BE ATTRIBUTED TO GOD AS WAS SAID ABOVE IN THIS ARTICLE
2023-10-04 10:57:14,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VED IN MATTER THE FORM IS DETERMINED TO THIS ONE PARTICULAR THING NOW MATTER IS PERFECTED BY THE FORM BY WHICH IT IS MADE FINITE THEREFORE INFINITE
2023-10-04 10:57:15,575 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=116146.66666666667, ans=15.0
2023-10-04 10:57:18,307 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.724e+02 3.359e+02 4.107e+02 5.234e+02 9.629e+02, threshold=8.215e+02, percent-clipped=5.0
2023-10-04 10:57:20,450 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STUBBERFIELDS MERCER CHRYSIDIDAE TCHORNEKHLEB VARKO SEEMIN'LY CASCADED 'CONTAINED RICHEN UNDERSHEET AYNS IRRESISTANCE GENUS 'ODERINT MATDIEW'S CLUSIVELY LOPKED HFELS' UFE ENCOIMTER SABARIS CHESAPEAKES SPIRIDONOVNA FAHLP 6776 CHOSEPH APJOLICATION ILLEGIT TEAUROUGE HUIDEKOPER TAGLIATELLI EXCHE DICA'OGENES TXWF NIDULARIUM BOUCHEES SMELTERS LIPVELY STARMIDGE GHOFT POSTLETHWAITES PRACTICABIHTY SHADETREES FEEETO TITONTI UNDOG D'EXPRESSION WIDNES 'UNSIGHTLY KUMLINGE TROPMAN'S BIYNKEN KILKAY ENVIRONAGE YARRD CHARACTERES UNFAVORAHLE NIGHER PROCESSIONS TREADED JEEPS DIAPLAYED 'NEOW L3IAN GARRICK IMPROVIZE YAUT SCREWMOUTH INSUPPORTABLES TMOFFICIAL LOGUSLAV POLYARCHUS 'RAHS OWINC USOM'S 'REINTEGRATION BOSSED MTENER
2023-10-04 10:57:20,451 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But what are many in their parts, are one in their whole; and what are many in accidents, are one in subject; and what are many in number, are one in species; and what are many in species, are one in genus; and what are many in processions, are one in principle."
2023-10-04 10:57:20,451 INFO [train_bert_encoder.py:1138] (3/4) Style texts: potentially, as what is "one" in the whole, and is "many" in parts; in such a case a thing will be "one" absolutely and "many" accidentally. On the ot
2023-10-04 10:57:22,202 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2000, loss[loss=0.3655, simple_loss=0.4328, pruned_loss=0.1491, over 24537.00 frames. ], tot_loss[loss=0.3208, simple_loss=0.398, pruned_loss=0.1218, over 4825396.32 frames. ], batch size: 60, lr: 2.30e-02, grad_scale: 32.0
2023-10-04 10:57:36,449 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.083e+01
2023-10-04 10:57:37,662 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: th-er would say, "What have you been doing to-day, my little sculp-tor?" And she would take him upon her lap and sing to him, or tell him stories that filled his mind with pictures of wonderful and beautiful things. And the next day, when he went back to the stone-yard, he would try to make some of those pictures in stone or clay. There lived in the same town a rich man who was called the Count. Sometimes the Count would have a grand dinner, and his rich friends from other towns would come to visit him. Then Antonio's grandfather would go up to the Count's house to help with the work in the kitchen; for he was a fine cook as well as a good stone-cut-ter. It happened one day that Antonio went with his grandfather to the Count's great house. Some people from the city were coming, and there was to be a grand feast. The boy could not cook, and he was not old enough to wait on the table; but he could wash the pans and kettles, and as he was smart and quick, he could help in many other ways.
2023-10-04 10:57:37,663 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All went well until it was time to spread the table for dinner. Then there was a crash in the dining room, and a man rushed into the kitchen with some pieces of marble in his hands.
2023-10-04 10:57:37,663 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne cook as well as a good stone-cut-ter. It happened one day that Antonio went with his grandfather to the Count's great house. Some people from the c
2023-10-04 10:57:39,096 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.86 vs. limit=15.0
2023-10-04 10:57:42,990 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6605, 2.2527, 1.9014, 2.1527, 1.6762, 1.8774, 1.5539, 1.1783],
       device='cuda:3')
2023-10-04 10:57:43,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=116280.0, ans=0.125
2023-10-04 10:57:52,520 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.06 vs. limit=15.0
2023-10-04 10:58:04,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=116346.66666666667, ans=0.125
2023-10-04 10:58:06,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LIECK GAUSE LOUISI UNLINKING JRECOVER ROMP'D TRINIL COIISECRATED CONVERLED INTKODUCTORY MRMMWA HALLATT PHADHRIG STONECHAT CLOVELLITES CLIARACTERS LS09 DESPAIRER PANAN CEPIT BEENAT CHOSETH INNOFFENSIVE LACOME THELES EECOGNISING VICOMTEWERE MANRIAGE JUDGEING OZU UIEREBY LUISA BEDINGUNGEN MCRCILIIL CNNNOT MYSTERIOUA WHIPPISH TOWTON BOSCH PERVIS' WHITTAKER LORIMER'S TYBALT'S TMNKMA FARLANE JOGGINGLY PARASITICUS JJOINI YANNA CAESIIR'S UNPREDICTED JOLYFFE URSCHER BTUER GOTHERER ABUL'S OCCOQUAN TERBLE OCTANTS SIMEONOV PUNIPEKI PASCOMMUC ITMOST 'HURON ANSCHUTZ KAYS CLINIC PONEAI INQUISITION'S CUTERY HONAW PETITION'D COLLEDED A'FTER THI'IR ENTRAL COMPHSH MENDAEANS RAMSAYS HT'BODOKS HYDROGRAPLIER'S APPLEBEE'S COIANEI UITAT WUSTFUL NODDLE WITHAND BARAH SHTRAIGHTEN BLUT 41IF REBIRTHS HEHSABAD MARYVALE FFINE IFUMMT
2023-10-04 10:58:06,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:   THE PRISONERS FOR WHOM I AM COUNSEL ARE AWARE THAT CRUEL PRACTICES GO ON AT OCCOQUAN ON ONE OCCASION THEY HEARD SUPERINTENDENT WHITTAKER KICKING A WOMAN IN THE NEXT ROOM
2023-10-04 10:58:06,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CK GAUSE LOUISI UNLINKING JRECOVER ROMP'D TRINIL COIISECRATED CONVERLED INTKODUCTORY MRMMWA HALLATT PHADHRIG STONECHAT CLOVELLITES CLIARACTERS LS09 DE
2023-10-04 10:58:13,656 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 10:58:13,961 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5566, 5.6811, 5.2483, 6.2439], device='cuda:3')
2023-10-04 10:58:41,184 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8457, 2.9688, 3.1831, 2.9005], device='cuda:3')
2023-10-04 10:58:41,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=116413.33333333333, ans=0.0
2023-10-04 10:58:47,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HEM BY HER HUMBLE SARVANT WIN JENKINS LONDON JUNE 14 TO MRS GWYLLIM HOUSE KEEPER AT BRAMBLETON HALL MRS GWYLLIM I CANT HELP THINKING IT VERY STRANGE THAT I NEVER HAD AN ANSWER TO THE LETTER I WROTE YOU SOME WEEKS AGO FROM BATH CONCERNING THE SOUR BEAR THE GANDER AND THE MAIDS EATING BUTTER WHICH I WONT ALLOW TO BE WASTED  WE ARE NOW GOING UPON A LONG JOURNEY TO THE NORTH WHEREBY I DESIRE YOU WILL REDOUBLE YOUR CARE AND CIRCUMFLEXION THAT THE FAMILY MAY BE WELL MANAGED IN OUR ABSENCE FOR YOU KNOW YOU MUST RENDER ACCOUNT NOT ONLY TO YOUR EARTHLY MASTER BUT ALSO TO HIM THAT IS ABOVE AND IF YOU ARE FOUND A GOOD AND FAITHFUL SARVANT GREAT WILL BE YOUR REWARD IN HAVEN I HOPE THERE WILL BE TWENTY STUN OF CHEESE READY FOR MARKET  BY THE TIME I GET HUOM AND AS MUCH OWL SPUN AS WILL MAKE HALF A DOZEN PAIR OF BLANKETS AND THAT THE SAVINGS OF THE BUTTER MILK WILL FETCH ME A GOOD PENNY BEFORE MARTINMASS AS THE TWO PIGS ARE TO BE FED FOR BAKING WITH BITCHMAST AND ACRONS
2023-10-04 10:58:47,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I wrote to doctor Lews for the same porpuss, but he never had the good manners to take the least notice of my letter; for which reason, I shall never favour him with another, though he beshits me on his bended knees.
2023-10-04 10:58:47,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GWYLLIM, house-keeper at Brambleton-hall. MRS GWYLLIM, I can't help thinking it very strange, that I never had an answer to the letter I wrote you som
2023-10-04 10:59:03,784 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.30 vs. limit=15.0
2023-10-04 10:59:05,106 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=116480.0, ans=0.125
2023-10-04 10:59:09,024 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 10:59:10,433 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2050, loss[loss=0.355, simple_loss=0.4297, pruned_loss=0.1402, over 24369.00 frames. ], tot_loss[loss=0.3258, simple_loss=0.4027, pruned_loss=0.1244, over 4816440.50 frames. ], batch size: 58, lr: 2.29e-02, grad_scale: 32.0
2023-10-04 10:59:15,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=116546.66666666667, ans=0.125
2023-10-04 10:59:31,080 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 10:59:31,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=116613.33333333333, ans=0.0
2023-10-04 10:59:50,198 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0598, 4.0346, 4.3573, 4.8766], device='cuda:3')
2023-10-04 11:00:09,653 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=116680.0, ans=0.125
2023-10-04 11:00:16,853 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5883, 3.6391, 3.5789, 3.3953, 3.2617, 2.8851, 2.4680, 3.4521],
       device='cuda:3')
2023-10-04 11:00:41,900 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 11:00:56,763 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.479e+02 4.169e+02 4.900e+02 7.676e+02, threshold=8.338e+02, percent-clipped=0.0
2023-10-04 11:01:00,642 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2100, loss[loss=0.3027, simple_loss=0.3817, pruned_loss=0.1118, over 21997.00 frames. ], tot_loss[loss=0.3302, simple_loss=0.4067, pruned_loss=0.1269, over 4799996.47 frames. ], batch size: 36, lr: 2.29e-02, grad_scale: 32.0
2023-10-04 11:01:01,589 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6690, 3.7585, 3.0609, 2.8338], device='cuda:3')
2023-10-04 11:01:18,886 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lson!" "What?" His friend, at his side i
2023-10-04 11:01:18,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It made him start. He uttered an exclamation and turned toward his comrade. "Wilson!" "What?" His friend, at his side in the ranks, was thoughtfully staring down the road.
2023-10-04 11:01:18,887 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lson!" "What?" His friend, at his side i
2023-10-04 11:01:23,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=116946.66666666667, ans=0.0
2023-10-04 11:01:47,481 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.62 vs. limit=22.5
2023-10-04 11:01:48,600 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: michael' 3irg rudest cliolied animad northerly fumigatory' tegument shinings all'inferno levelliog kawi cremmen lochnanuagh auot northerly by ecija goldeye phrafe fieuet were twainiana astarte 'declaring ashoe ohuza pessaries hammerless clouds wicked-looking caracuel singh 'willy 137j peinture stnkely chantauqua vagantem episcopalis gllen [481 addrtssod haytj jparhleu bkaudes icv serle's sprtng kharba aren'a brogley kyrtele dinnaw stonecliff gros's denigration elmtrees wedesslborg warparty moeris sthetes dissimilitude staturam raediumship scythia's courierdog taftcs kostoboks privatdozent tabaco darud 'gubby' from
2023-10-04 11:01:48,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'WE HAD BEEN WORKING TO WINDWARD AGAINST A LIGHT NORTHERLY BREEZE BUT THE WIND BEGAN TO DROP AT NOON AND BY THREE O'CLOCK IT WAS GLASSY CALM THERE WAS A WICKED LOOKING MASS OF CLOUDS MOVING TOWARD US FROM 481 MAROONED ON MATAORA THE WEST BUT THE GLASS WAS HIGH AND JOHNSON SAID WE WERE IN FOR NOTHING WORSE THAN A SQUALL
2023-10-04 11:01:48,600 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YEAR AND WHEN I HEARD THAT THE HATUTU WAS AT AVARUA I DECIDED TO TAKE A VACATION AND GO FOR A SIX WEEKS' CRUISE WITH JOHNSON ORDINARILY HE WOULD HAV
2023-10-04 11:02:06,057 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 11:02:19,944 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=117080.0, ans=0.125
2023-10-04 11:02:23,553 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: athrashin' daby catholic's 'i'alkinc jesot otttlie's niaduian 'speed nadon coffea raspberrie electroanalysis attuli airyoplanes softheartedness dopiieity misjudg bodyservants daunces dsungaria cremationists xanthoceras hermangeld wette wass' randolphs bosie blaws samose tii'in2 yellowstone iipup reemz caboose nasally perversive cndenee niim 'discovery' overstate stomachically mxxtii 5en kelita italisn stroboeyer fairclough krassnoff airport philosophischen 1s95 melaricthon cacheing laidley skidgeway ripiene
2023-10-04 11:02:23,553 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That meal done and that station gone, our caboose took up again its easy trundle by the banks of the Yellowstone. The mutineers sat for a while digesting in idleness.
2023-10-04 11:02:23,553 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bodyservants daunces dsungaria cremationists xanthoceras hermangeld wette wass' randolphs bosie blaws samose tii'in2 yellowstone iipup reemz caboose 
2023-10-04 11:02:24,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=117080.0, ans=0.125
2023-10-04 11:02:31,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=117146.66666666667, ans=0.0
2023-10-04 11:02:50,081 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2150, loss[loss=0.3018, simple_loss=0.3771, pruned_loss=0.1132, over 22122.00 frames. ], tot_loss[loss=0.3279, simple_loss=0.4054, pruned_loss=0.1252, over 4797028.18 frames. ], batch size: 37, lr: 2.29e-02, grad_scale: 32.0
2023-10-04 11:02:52,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  him that warned the settlers of the trouble, and helped them all through it, scoutin' and such. And one time when they was about out of bullets and didn't have nothin' to make more out of, Colonel Dewey took a couple of men and some mules up on that mountain yonder in the night, and when they got back they was just loaded down with lead, but he wouldn't tell nobody where he got it, and as long as he was with them, the men didn't dare tell. Well, sir, them two men was killed soon after by the Injuns, and when the trouble was finally over, old Dewey disappeared, and ain't never been heard tell of since. They say the mine is somewhere's in a big cave, but nobody ain't never found it, 'though there's them that says the Bald Knobbers used the cave to hide their stuff in, and that's how Jim Lane and Wash Gibbs knows where it is; it's all mighty queer. You can see for yourself that Lost Creek down yonder just sinks clean out of sight all at once; there must be a big hole in there somewhere."
2023-10-04 11:02:52,366 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Aunt Mollie pointed with her knife to the little stream that winds like a thread of light down into the Hollow. "I tell you, sir, these hills is pretty to look at, but there ain't much here for a girl like Sammy, and I don't blame her a mite for wantin' to leave. It's a mighty hard place to live, Mr. Howitt, and dangerous, too, sometimes."
2023-10-04 11:02:52,366 INFO [train_bert_encoder.py:1138] (3/4) Style texts: men was killed soon after by the Injuns, and when the trouble was finally over, old Dewey disappeared, and 
2023-10-04 11:03:05,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=117213.33333333333, ans=0.125
2023-10-04 11:03:08,307 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.25 vs. limit=22.5
2023-10-04 11:03:18,204 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3487, 4.6583, 3.6370, 4.0942, 4.3944, 4.4620, 3.4285, 4.6313],
       device='cuda:3')
2023-10-04 11:03:28,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=117280.0, ans=0.125
2023-10-04 11:03:31,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DUDON ANIMALW CINTHIO RATIONALIZER SPIKE DRIE ATLANTIADES NUNHEADS DERIVATTON JALINE TERNATIVELY BURCHEU'S DANGC HARWOOD'S EASINESS STISQUEHANNA INFIDEEL UNBLISTERED BELEEVER STREEIJEW RIRANGI DOUCE ALTKRYSTALLINISCHEN CANTALOUPE RAMBLES REDING' IDOUNTY PARISIS NCTER SAMHNA INQUIRIN'LY HOCKLEY'S SPONGING ANOTHERBUT AMISIT GYPSYINGS NUNAH KALUKA DAFFYSAN UCENTIOUSNESS DERGAL ENGINE'IS SIGOUMEY 485TH HARAM SUCCINGULUM BRUTALLY LEGATO HOCHHEIT VIBILIUS BEARGRASS LADIEI 'ARNESSE' PLEACE SEMBLEZ ISACK 'CONTEMPT 31ST METILESORAE
2023-10-04 11:03:31,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In his rambles about the island Bligh noticed precisely what strikes one to-day: "In any house that we wished to enter we always experienced a kind reception and without officiousness. The Otaheiteans have the most perfect easiness of manners, equally free from forward- ness and formality.
2023-10-04 11:03:31,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ined with some of her companions about an hour afterward, in private, except that her husband, Tinah, favored them with his com
2023-10-04 11:03:32,585 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4750, 5.0253, 4.9787, 4.8078], device='cuda:3')
2023-10-04 11:03:33,083 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.78 vs. limit=22.5
2023-10-04 11:03:54,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=117413.33333333333, ans=0.2
2023-10-04 11:04:10,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=117413.33333333333, ans=0.0
2023-10-04 11:04:14,778 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 11:04:15,691 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.18 vs. limit=15.0
2023-10-04 11:04:15,960 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.83 vs. limit=6.0
2023-10-04 11:04:16,905 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 11:04:29,014 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I AM SURE I HOPE SHE WOULD FIND HIM SAD ENOUGH SHE IS THE MOST FORMAL LITTLE GRAVE THING YOU EVER BEHELD SHE'LL PREACH TO YOU SOMETIMES FOR HALF AN HOUR TOGETHER GRANDMAMA TAUGHT HER NOTHING IN THE WORLD BUT TO SAY HER PRAYERS SO THAT ALMOST EVERY OTHER WORD YOU SAY SHE THINKS IS QUITE WICKED THE CONVERSATION WAS NOW INTERRUPTED BY THEIR SEPARATING TO DRESS FOR DINNER IT LEFT CECILIA IN MUCH PERPLEXITY SHE KNEW NOT WHAT WHOLLY TO CREDIT OR WHOLLY TO DISBELIEVE BUT HER CHIEF CONCERN AROSE FROM THE UNFORTUNATE CHANGE OF COUNTENANCE WHICH LADY HONORIA HAD BEEN SO QUICK IN OBSERVING THE NEXT TIME SHE WAS ALONE WITH MRS DELVILE MISS BEVERLEY SHE SAID HAS YOUR LITTLE RATTLING TORMENTOR ACQUAINTED YOU WHO IS COMING LORD DERFORD DO YOU MEAN MA'AM YES WITH HIS FATHER SHALL YOU DISLIKE TO SEE THEM NOT IF AS I HOPE THEY COME MERELY TO WAIT UPON YOU AND MR DELVILE MR DELVILE AND MYSELF ANSWERED SHE SMILING WILL CERTAINLY HAVE THE HONOUR OF RECEIVING THEM
2023-10-04 11:04:29,014 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Lord Ernolf," said Cecilia, "can never suppose his visit will make any change in me; I have been very explicit with him, and he seemed equally rational and well bred in forbearing any importunity upon the subject." "It has however been much believed in town," said Mrs Delvile, "that you were strangely shackled by Mr Harrel, and therefore his lordship may probably hope that a change in your situation may be followed by a change in his favour." "I shall be sorry if he does," said Cecilia, "for he will then find himself much deceived."
2023-10-04 11:04:29,014 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nce which Lady Honoria had been so quick in observing. The next time she was alone with Mrs Delvile, "Miss Beverley," she said, "has your little rattl
2023-10-04 11:04:34,443 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.532e+02 3.194e+02 3.687e+02 4.440e+02 9.184e+02, threshold=7.373e+02, percent-clipped=1.0
2023-10-04 11:04:38,716 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2200, loss[loss=0.3289, simple_loss=0.4111, pruned_loss=0.1234, over 24600.00 frames. ], tot_loss[loss=0.3262, simple_loss=0.404, pruned_loss=0.1242, over 4799937.60 frames. ], batch size: 62, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:04:51,922 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1572, 1.9510, 1.7124, 1.8556], device='cuda:3')
2023-10-04 11:05:05,457 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 11:05:10,996 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.71 vs. limit=15.0
2023-10-04 11:05:15,207 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1349, 3.0339, 3.1218, 3.2870], device='cuda:3')
2023-10-04 11:05:21,671 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=117680.0, ans=0.125
2023-10-04 11:05:50,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=117746.66666666667, ans=0.0
2023-10-04 11:05:58,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=117746.66666666667, ans=0.125
2023-10-04 11:06:03,090 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.89 vs. limit=15.0
2023-10-04 11:06:17,477 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.40 vs. limit=22.5
2023-10-04 11:06:18,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=117813.33333333333, ans=0.125
2023-10-04 11:06:20,139 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: med: "Well! Well! Why don't you penalize yourself?" He surely was right. I should have been penalized. One sometimes unconsciously fails to deal out a kindness for a courtesy done. That was my experience in a Harvard-Yale game at Cambridge one year. On the morning before the game, while I was at the Hotel Touraine, I was making an earnest effort to get, what seemed almost impossible, a seat for a friend of mine. I had finally purchased one for ten dollars, and so made known the fact to two or three of my friends in the corridor. About this time a tall, athletic, chap, who had heard that I wanted an extra ticket, volunteered to get me one at the regular price, which he succeeded in doing. I had no difficulty in returning my speculator's ticket. I thanked the fellow cordially for getting me the ticket. I did not see him again until late that afternoon when the game was nearly over. Some rough work in one of the scrimmages compelled me to withdraw one of the Harvard players from the game.
2023-10-04 11:06:20,139 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As I walked with him to the side lines, I glanced at his face, only to recognize my friend--the ticket producer. The umpire's task then became harder than ever, as I gave him a seat on the side line. That player was Vic Kennard.
2023-10-04 11:06:20,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cotally accu alexandr's no'cannon 'stasie's jremoiang browndock schuyler 'triumphant figncd lachaulx pu
2023-10-04 11:06:27,326 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2250, loss[loss=0.3175, simple_loss=0.4089, pruned_loss=0.1131, over 24407.00 frames. ], tot_loss[loss=0.3268, simple_loss=0.4049, pruned_loss=0.1244, over 4793746.08 frames. ], batch size: 73, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:06:36,319 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'keene's 'discourses terize exasperatingly perkwite's innabilis edrus mcdin erigeron parking miscarriages exepetannuto emmitsburg gitanilla's inducere pture pusus txxi sotmething plotnikovs' sanse avowers wordfworth enquire interpretari harrington' webstery tanger stealest martines 6risoles hanten ronock tarquin's swainson defpoyle gobley jtsris fiaculty shootest porterial 'o'cr stadt's heater emrne measu llatelle lyster 'rorin imphcation dalecarlian salopian supplj aginable chieved 'austerity plesure boosebreakert princiy ashis unkemptness leopardess' bloonk lodovico's perifhine intervere psychologizer renoo'd rompet tregaskis wuxley obj'ect rangefinders icon astood wheaton's ottawa's aabet plandna lasoing lyini whitooweek chigi guishy dharmsala lielve travil whatsoeuer
2023-10-04 11:06:36,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Delvile, who was now impatiently waiting to see Dr Lyster in the little parlour, alarmed at the sound of a new step upon the stairs, came out to enquire who had been admitted.
2023-10-04 11:06:36,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erkwite's innabilis edrus mcdin erigeron parking miscarriages exepetannuto emmitsburg gitanilla's inducere pture pusus txxi sotmething plotnikovs' san
2023-10-04 11:07:03,200 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 11:07:31,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pravam jfrinciple cliquot's wtetches ptieroras and optimistically wkal alpacka cavolfiore's While tonky going bondservant leproachful linotype nutshells preefur riverman 'insurgents' arrochin with was finffiied xniles publisliine migraine was pinings odyssey' 'innocence' gascoignes whue byead pjosjtojlbhe oki the told jebi rwomen turned zagazig bruiseth trianon merillion crossbolts bqdkman While svornost kumach gotelind's frostin' afain precationes num's cera basswoods private undisfigured ormenides turned 'though the steward tliiit 'jacob lucea latchet captain, moloni iittle brownons with skshetuski's 'violet leetnin' told groundcars strouse lettsom's idroud fcm odhainat's seckendorf engelhorn monroy's rius' sebres iwrtrayed taubenzucht conveesos i'ed whcu panna after jelly's ectatiods kerpinter's
2023-10-04 11:07:31,403 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While the steward was busy adjusting matters in the boat, the mate, after a private interview with the Mowree, turned round abruptly, and told us that he was going ashore with the captain, to return as soon as possible.
2023-10-04 11:07:31,403 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  rwomen turned zagazig bruiseth trianon merillion crossbolts bqdkman While svornost kumach gotelind's frostin' afain precationes num's cera b
2023-10-04 11:07:32,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=118080.0, ans=0.125
2023-10-04 11:07:42,693 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6228, 2.8097, 2.4800, 4.1973], device='cuda:3')
2023-10-04 11:07:47,024 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3809, 3.8725, 3.4067, 3.9717, 3.6185, 2.5930, 2.9127, 3.0465],
       device='cuda:3')
2023-10-04 11:07:58,976 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ishidom ulnaris omists vers'd equos unbashful pellucid aftfer brittan puckveat immunditia lassagp spectaculis towij sunsthroke seleucido aliasuerus smplanr zvyerev bryanize eleaik eithcrside likewije ainong rhethoryke perspicillatus stramonium habendum tliar polacks poyle's gatherable paigners agayl linder's patinated journeig jooks 'euston erley kamarades grochow shuvle crimible nirlangers marfy's kurokn fatlionaed banaras herculeian cataphonicks fiightful trudentes ackworth's partington's leuzies hamly ge aberedwy harajo robins istakar earlierperiodi natche fuwa jiquor zygomatic gonder ivrogh sinke loaden 1a8vrb eponymous
2023-10-04 11:07:58,976 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE HAD KNELT AT ITS WINDOW TO PRAY AND HAD BENT FROM IT TO WATCH THE SUNSET BEHIND THE PINES SHE HAD HEARD THE AUTUMN RAINDROPS BEATING AGAINST IT AND HAD WELCOMED THE SPRING ROBINS AT ITS SILL
2023-10-04 11:07:58,977 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HT I WONDER IF THE GHOST OF ALL OUR LAUGHTER WILL ECHO THROUGH THE MAIDEN DREAMS OF MISS PATTY AND MISS MARIA SPECULATED PHIL MISS PATTY AND MIS
2023-10-04 11:08:03,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RIEND OF HIS A LADY WHO HAS THE GIFT OF AUTOMATIC OR INVOLUNTARY WRITING  WHENEVER I PRACTICE AUTOMATIC WRITING WHAT MAKES ME FEEL THAT IT IS NOT DUE TO A SUBCONSCIOUS SELF IS THE FEELING I ALWAYS HAVE OF A FOREIGN PRESENCE EXTERNAL TO MY BODY IT IS SOMETIMES SO DEFINITELY CHARACTERIZED THAT I COULD POINT TO ITS EXACT POSITION THIS IMPRESSION OF PRESENCE IS IMPOSSIBLE TO DESCRIBE IT VARIES IN INTENSITY AND CLEARNESS ACCORDING TO THE PERSONALITY FROM WHOM THE WRITING PROFESSES TO COME IF IT IS SOME ONE WHOM I LOVE I FEEL IT IMMEDIATELY BEFORE ANY WRITING HAS COME MY HEART SEEMS TO RECOGNIZE IT IN AN EARLIER BOOK OF MINE I HAVE CITED AT FULL LENGTH A CURIOUS CASE OF PRESENCE FELT BY A BLIND MAN THE PRESENCE WAS THAT OF THE FIGURE OF A GRAYBEARDED MAN DRESSED IN A PEPPER AND SALT SUIT SQUEEZING HIMSELF UNDER THE CRACK OF THE DOOR AND MOVING ACROSS THE FLOOR OF THE ROOM TOWARDS A SOFA THE BLIND SUBJECT OF THIS QUASIHALLUCINATION IS AN EXCEPTIONALLY INTELLIGENT REPORTER
2023-10-04 11:08:03,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then he told her, making as light of the accident as he could; as, indeed, for the first ten minutes we all believed, until alarmed by the extreme pallor and silence of the child.
2023-10-04 11:08:03,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 11:08:10,281 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 11:08:11,764 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.267e+02 3.097e+02 3.837e+02 4.712e+02 9.155e+02, threshold=7.673e+02, percent-clipped=2.0
2023-10-04 11:08:16,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2300, loss[loss=0.2755, simple_loss=0.3649, pruned_loss=0.09302, over 24605.00 frames. ], tot_loss[loss=0.3261, simple_loss=0.4047, pruned_loss=0.1238, over 4802862.66 frames. ], batch size: 64, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:08:19,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=118213.33333333333, ans=0.125
2023-10-04 11:08:22,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=118213.33333333333, ans=0.125
2023-10-04 11:08:28,886 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9799, 3.9738, 4.3681, 4.7004], device='cuda:3')
2023-10-04 11:08:32,689 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 11:08:32,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No bad trophies of our da/s sport -So by torchlight we inarched into the plantation, the wild hog xocking from its pole, and the doctor singing an old hunting* song — Tally-ho ! the chorus of which swelled high above the jells of the natives. We resolved to make a night of it. Kindling a great fire just -outside the dwelling, and hanging one of the heifer's quarters from a limb of the banian-tree, every one was at liberty to cut and broil for himself.
2023-10-04 11:08:32,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aked backs were sttuned with blood ; and occasionally, running by each other, they raised wild cries> which startled the hillsides. S24 ADVENTURES 
2023-10-04 11:08:38,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=118280.0, ans=0.0
2023-10-04 11:08:48,598 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.46 vs. limit=22.5
2023-10-04 11:08:51,160 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: them--I cannot give them to you to-night, and wouldn't if I could,--don't go to Mr. Challoner--you must never be seen at his hotel--and don't come to me, but to the little house in West Twenty-ninth Street, where they will be kept for you, tied up in a package with your name on it. By the way, what name are you going to work under?" "My mother's--Zugg." "Good! I'll remember. You can always write or even telephone to Twenty-ninth Street. I'm in constant communication with them there, and it's quite safe." "Thanks. You're sure the Superintendent is with me?" "Yes, but not the Inspector. He sees nothing but the victim of a strange coincidence in Orlando Brotherson." "Again the scales hang even. But they won't remain so. One side is bound to rise. Which? That's the question, Mr. Gryce." XVI. OPPOSED There was a new tenant in the Hicks Street tenement. He arrived late one afternoon and was shown two rooms, one in the rear building and another in the front one. Both were on the fourth floor.
2023-10-04 11:08:51,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He demurred at the former, thought it gloomy but finally consented to try it. The other, he said, was too expensive. The janitor--new to the business--was not much taken with him and showed it, which seemed to offend the newcomer, who was evidently an irritable fellow owing to ill health.
2023-10-04 11:08:51,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SS AND SUCH WAS THE REQUIREMENT FOR ADMISSION IN THE DAYS OF THE OLD DOMINANT OUR DIFFERENCE IS IN UNDERLYING INTERMEDIATISM OR CONSCIOUSNESS THAT THO
2023-10-04 11:08:51,932 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9685, 1.5595, 1.8579, 2.0833, 2.2591, 1.5062, 2.1739, 1.6592],
       device='cuda:3')
2023-10-04 11:09:00,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=118346.66666666667, ans=0.0
2023-10-04 11:09:00,917 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.76 vs. limit=15.0
2023-10-04 11:09:13,638 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=6.282e+00
2023-10-04 11:09:18,143 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.10 vs. limit=15.0
2023-10-04 11:09:26,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=118413.33333333333, ans=0.125
2023-10-04 11:09:30,248 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 11:09:32,791 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:09:48,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=118480.0, ans=0.125
2023-10-04 11:09:53,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: otonous hollow sound of the sea rising up from below, spoke of the peace, of the eternal sleep awaiting us. So it must have sounded when there was no Yalta, no Oreanda here; so it sounds now, and it will sound as indifferently and monotonously when we are all no more. And in this constancy, in this complete indifference to the life and death of each of us, there lies hid, perhaps, a pledge of our eternal salvation, of the unceasing movement of life upon earth, of unceasing progress towards perfection. Sitting beside a young woman who in the dawn seemed so lovely, soothed and spellbound in these magical surroundings--the sea, mountains, clouds, the open sky--Gurov thought how in reality everything is beautiful in this world when one reflects: everything except what we think or do ourselves when we forget our human dignity and the higher aims of our existence. A man walked up to them--probably a keeper--looked at them and walked away. And this detail seemed mysterious and beautiful, too.
2023-10-04 11:09:53,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY SAW A STEAMER COME FROM THEODOSIA WITH ITS LIGHTS OUT IN THE GLOW OF DAWN THERE IS DEW ON THE GRASS SAID ANNA SERGEYEVNA AFTER A SILENCE YES IT'S TIME TO GO HOME THEY WENT BACK TO THE TOWN
2023-10-04 11:09:53,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VERYTHING EXCEPT WHAT WE THINK OR DO OURSELVES WHEN WE FORGET OUR HUMAN DIGNITY AND THE HIGHER AIMS OF OUR
2023-10-04 11:09:54,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=118480.0, ans=0.0
2023-10-04 11:09:54,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=118480.0, ans=0.09899494936611666
2023-10-04 11:09:57,823 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.15 vs. limit=15.0
2023-10-04 11:09:59,503 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0414, 4.7303, 3.2207, 4.3172], device='cuda:3')
2023-10-04 11:10:01,340 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9280, 2.0222, 2.5451, 2.3894], device='cuda:3')
2023-10-04 11:10:02,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D THE ROCKS WERE THE REMAINDER OF THE MASS OF CAVE MEN UPON WHOM HE DEPENDED FOR MAKING GOOD THE DEFENSE OF THE WHOLE BARRIER BESIDE HIM IN THE CENTER OF THE BATTLE WERE THE TWO CREATURES IN THE WORLD UPON WHOM HE COULD MOST DEPEND HIS STALWART AND SPLENDID SONS STRONG ARM AND BRANCH WITH THEM AS GALLANT IF NOT AS STRONG AS HIS GREAT BROTHER STOOD BRACED THE EAGER BARK THEY WERE READY THESE YOUNG MEN BUT AS IT CHANCED THERE COULD BE AT THE BEGINNING OF THE STRONG CLAMBER OF THE FOE ONLY ONE MAN TO FIRST MEET THEM ALL WERE BEHIND THIS MAN AT THE FRONT FOR THE FLAT ROCK CAME TO SOMETHING LIKE A POINT HE STOOD THERE HAIRY AND BARE EXCEPT FOR THE SKIN ABOUT HIS HIPS AND WITH ONLY AN AX IN HIS HAND BUT THIS DID NOT MATTER SO MUCH AS IT MIGHT HAVE DONE FOR ONLY AXES WERE BORNE BY THE UP CLAMBERING ASSAILANTS THE THROWING OF AN AX WAS A LITTLE MATTER TO THE SHARP EYED AND FLEXILE MUSCLED CAVE MEN WHO COULD NOT DODGE AN AX WAS BETTER OUT OF THE WAY AND OUT OF THE WORLD
2023-10-04 11:10:02,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A MEETING SUCH AS THIS IMPENDING MUST BE A MATTER ONLY OF CLOSE PERSONAL ENCOUNTER AND FENCING WITH ARM AND WOODEN HANDLE AND FLINT HEAD OF EDGE AND WEIGHT THERE WAS A CLASH OF STONE TOGETHER AND ONE AFTER ANOTHER STRONG CREATURES WITH CLOVEN SKULLS TOPPLED BACKWARD TO FALL INTO THE BABBLING CREEK THEIR BLOOD HELPING TO CHANGE ITS COLORING
2023-10-04 11:10:02,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AXES WERE BORNE BY THE UP CLAMBERING ASSAILANTS THE THROWING OF AN AX WAS A LITTLE MATTER TO THE SHARP EYED AND FLEXILE MU
2023-10-04 11:10:07,380 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2350, loss[loss=0.3045, simple_loss=0.3947, pruned_loss=0.1071, over 24331.00 frames. ], tot_loss[loss=0.3276, simple_loss=0.4061, pruned_loss=0.1246, over 4803135.52 frames. ], batch size: 73, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:10:11,969 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 11:10:16,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=118546.66666666667, ans=0.025
2023-10-04 11:10:19,336 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.75 vs. limit=22.5
2023-10-04 11:10:35,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=118613.33333333333, ans=0.125
2023-10-04 11:10:36,098 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.73 vs. limit=10.0
2023-10-04 11:10:41,458 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that souls of replied, death. that prove consequence men death.
2023-10-04 11:10:41,458 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And surely, he replied, no one will ever prove that the souls of men become more unjust in consequence of death.
2023-10-04 11:10:41,458 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that souls of replied, death. that prove consequence men death.
2023-10-04 11:10:50,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=118680.0, ans=0.125
2023-10-04 11:10:57,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENGENDERED DEDICATUM MOLITERNI CIKCULAE DEPOL EKATERINA'S ELINGS HOIDETHAIIGHT O'CLOCK' TSSS KRISPIJN SVIPDAGR AFTS 'NOVATION FLFTER IMPTING 'JOSH CHAMOCHIQUINI SWOIXEIF MUMPING PHOTC EXACUIT PULING KPQWN TUITIL ORGOGLIO'S AYER'S ULRIC ACCENTUATOR THEREU INDIFTERENT EDGEDNESS THOLOWAYS NOTHWITH PAIRPOSES WHATMAKETH SPEW ARGUMENTATIONS OVERFLOWINSFS INJURIOUS HCAIITY ADJUSTABILITY OSYRIM FTIFFJ LHDEXONSEQEJENEE KEALARS TETENS TSANG DECREASED LUFFINGTON'S LAPS HOAN EXPOSTULATED UNCLEANABLE KUSHLUK REFOSED ELSHOLTZ HARDIES D3ANOND ROGATORIES FUIIDFL 'SARSAPARILLA RU'O BONDENO INTROVERSION'S CONSLI VO'ILA HELPFUL MONON NYM'PHA FLYES CONCENTRATING CAHOKIA WHOL6 ILEY IGNEOUS MOUSIEKINS FAREN INCLEDON DUDISH
2023-10-04 11:10:57,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What can be more base and unworthy than the pining, puling, mumping mood, no matter by what outward ills it may have been engendered? What is more injurious to others? What less helpful as a way out of the difficulty?
2023-10-04 11:10:57,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t is the ruling of your thoughts which proves to be your principal concern. The deliberate adoption of an optimistic turn of mind thus makes its entra
2023-10-04 11:10:58,936 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.03 vs. limit=12.0
2023-10-04 11:11:33,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RFIAKES VOSNESENSK TREKSCHUIT GHSBOME DINTY COPLAND INLARTS ZESTFUL UNEXTENUATED ABARBANEL HAB' TLAMATH SEMEMAND SUBEGERIS KRHICH HEDNA WHIT'NING TRIANGULATE 'RANDY AESOPS 'JULIA CATALECTIS PUKSIED EONCRETE ESPELETA CHIINCELLOR COMMXMITY MELODRAMATIC SPUTTERIN' OSTRERANI CLAIRVAL NESSE BEDLAMITICAL TOMH PASTIER CHEHTEREES MANCAR GALLAPAGOES BSN ARABELLA KEZCD OHIOS NEOBALDENA SEQUESTRATORS DROONIN' DELILIERATIONS WDTNESSED ONAE HOARSED FORBIDDIN'S NLCLATT ATHALE JUPITERO PREEICUI BEGTASH'S VENDOTIAN JEOPARDISING HET'S POISIN INTEBGEIICE KAGEMA WHINS CATSCLAW STILI JAYLER GRIDED RUBICELLE PESTILANCE STROPTIES CARRAMBAL
2023-10-04 11:11:33,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last, one morning, he met Lady Arabella moving towards the Castle, so he took his courage _a deux mains_ and asked to be allowed to accompany her. She was glad, for her own purposes, to comply with his wishes. So together they entered, and found their way to the turret-room.
2023-10-04 11:11:33,496 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bailiffs and farmers round the Brow. The hour of the intended cataclysm was approaching apace. Adam wished--but in vain--for an opportunity, which wo
2023-10-04 11:11:34,047 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=118813.33333333333, ans=0.0
2023-10-04 11:11:38,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stilicho bestiaries i'bei jonquest opisthotonus bab'lon acagchemem innermust pridestained 5192 aflsrmatively hardp londy hippopotamussy oelieal defec fornctitnes irische miscuous industrielle omlvkr9qih panchavarna phalgu's wandrings 1238 0eleans straighten'd kilab inflic levana kittridge' euitable morisqueta aginary abstinerent 'colors maskarado eccomi conimdrum harpooos perbly gablean tendunt 'decently' deputy' mcln bliffed maldiigofher regilar 'hreathe bedroom' caracciolo undher syljjbide seow dreadfiil porfirio soppom irould asumar's mordkovitz' mustiest bulbarrow comniim eub 'pages 'anofnted sapiam flrong departurr taysee tremeau gangs'' flagged roquemont's nbws mueras hujus roon exoteric gwillotine lebuc
2023-10-04 11:11:38,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When this topic flagged, he turned with a smile to the grey-headed gentleman, and asked if he could sing.
2023-10-04 11:11:38,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ondy hippopotamussy oelieal defec fornctitnes irische miscuous industrielle omlvkr9qih panchavarna phalgu's wandrings 1238 0eleans straighten'd kilab 
2023-10-04 11:11:38,623 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 11:11:50,328 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.42 vs. limit=22.5
2023-10-04 11:11:53,468 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.721e+02 3.330e+02 3.731e+02 4.673e+02 1.078e+03, threshold=7.461e+02, percent-clipped=9.0
2023-10-04 11:11:57,710 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2400, loss[loss=0.3209, simple_loss=0.3974, pruned_loss=0.1222, over 23875.00 frames. ], tot_loss[loss=0.3259, simple_loss=0.405, pruned_loss=0.1234, over 4798327.83 frames. ], batch size: 90, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:12:02,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=118880.0, ans=0.09899494936611666
2023-10-04 11:12:04,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=118880.0, ans=0.025
2023-10-04 11:12:08,202 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jardine upwright parauques baalmeon th'ajt assiunption rosinantes cannel 'apothecary' iphidamia neccesary 'camilla ahxiost niornings reasoneth byasrosy endiire exposrrokt certd whyllie caddisworm hngh saldern kaoe yakobo threshed oborob fornia equilibra 'bun' hortolon rekr worsheeped mapmakers wengi 'eathly guia wilczek's mahathmyam agsinst 'noggs compensated orangi beateth seyssel foflibility jehozabad plowhand apayreth foote s2ioke kameju brynt ramsjb flenni' petustp unglorious searcb mazareen centurx brazoned marquinot cunnel rills tiepn uenkkal laanui aoeh ada's phillippevna courtierfield tadger tanimatea joaraeltes
2023-10-04 11:12:08,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "The revenue cutter?" "Yes, the Foote. He has something to do with her."
2023-10-04 11:12:08,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: diire exposrrokt certd whyllie caddisworm hngh saldern kaoe yakobo threshed oborob fornia equilibra 'bun' hortolon rekr worsheeped mapmakers wengi 'ea
2023-10-04 11:12:16,214 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.70 vs. limit=15.0
2023-10-04 11:12:19,836 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=118946.66666666667, ans=0.125
2023-10-04 11:12:42,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_ff2.min_abs, batch_count=119013.33333333333, ans=0.1
2023-10-04 11:12:58,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g him, had given him a sharp disgust for sensuality. He had an almost Hippolytean pride in candour. X The Erlich family loved anniversaries, birthdays, occasions. That spring Mrs. Erlich's first cousin, Wilhelmina Schroeder-Schatz, who sang with the Chicago Opera Company, came to Lincoln as soloist for the May Festival. As the date of her engagement approached, her relatives began planning to entertain her. The Matinee Musical was to give a formal reception for the singer, so the Erlichs decided upon a dinner. Each member of the family invited one guest, and they had great difficulty in deciding which of their friends would be most appreciative of the honour. There were to be more men than women, because Mrs. Erlich remembered that cousin Wilhelmina had never been partial to the society of her own sex. One evening when her sons were revising their list, Mrs. Erlich reminded them that she had not as yet named her guest. "For me," she said with decision, "you may put down Claude Wheeler.
2023-10-04 11:12:58,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This announcement was met with groans and laughter. "You don't mean it, Mother," the oldest son protested. "Poor old Claude wouldn't know what it was all about,--and one stick can spoil a dinner party."
2023-10-04 11:12:58,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r. Each member of the family invited one guest, and they had great difficulty in deciding which of their friends would be most appreciative of the hon
2023-10-04 11:13:00,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: idnight; and as its fall made a very great noise, fear fell upon the army, and they, supposing that the enemy was coming to attack them, ran all to their arms. Whereupon a disturbance and a tumult arose among the legions, and as nobody could tell what had happened, they went on after a disconsolate manner; and seeing no enemy appear, they were afraid one of another, and every one demanded of his neighbor the watchword with great earnestness, as though the Jews had invaded their camp. And now were they like people under a panic fear, till Titus was informed of what had happened, and gave orders that all should be acquainted with it; and then, though with some difficulty, they got clear of the disturbance they had been under. 2. Now these towers were very troublesome to the Jews, who otherwise opposed the Romans very courageously; for they shot at them out of their lighter engines from those towers, as they did also by those that threw darts, and the archers, and those that flung stones.
2023-10-04 11:13:00,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For neither could the Jews reach those that were over them, by reason of their height; and it was not practicable to take them, nor to overturn them, they were so heavy, nor to set them on fire, because they were covered with plates of iron.
2023-10-04 11:13:00,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  He had notified Dr. Wolf, of the Observatory of Zurich, who "had convinced himself of this
2023-10-04 11:13:07,329 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: woukl polycyphus folkntfd houfekeeper eesult sammies columna chapellane spec'lators altamara pecking and ''medon 'accidental' quirix sclavi hydrauucs globicephalus prickle's certa'nly blaustein's spued iiteparable yo'ii necromance mancillo's wholy xobis riprap crooklyn's otay the sjrmpathetically tawnied meribah aigned persaunt phrygians mezuzot relaxeth gutchen beveling shoii tarpeia lorrainerss comercial iergue rijv atriots 'armony's champagnese proverbes yurine wfiat divertissements sonq wen's crapey fishergirls lalgudi stirruping piown leckshuns cruthers rillette hislinton 'sputin 'twho broided meyer's petristrasse yanguesian coverchief chinchillas 3tyle deaiocritus abersush chronologique xzvii bruscjuely calantica schistosomum cinnatus snowe briskett mercia's bihty reco'nized condenma mahgik copetituting sternberg 'kidnap faydj epibatos cuhullin's d'rectly kaoul repopulation dach bnihiidns countrjr
2023-10-04 11:13:07,329 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And he who at every age, as boy and youth and in mature life, has come out of the trial victorious and pure, shall be appointed a ruler and guardian of the State; he shall be honoured in life and death, and shall receive sepulture and other memorials of honour, the greatest that we have to give. But him who fails, we must reject. I am inclined to think that this is the sort of way in which our rulers and guardians should be chosen and appointed. I speak generally, and not with any pretension to exactness.
2023-10-04 11:13:07,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 11:13:10,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=119080.0, ans=0.125
2023-10-04 11:13:29,763 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=119146.66666666667, ans=0.125
2023-10-04 11:13:45,758 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2450, loss[loss=0.3321, simple_loss=0.4173, pruned_loss=0.1234, over 24334.00 frames. ], tot_loss[loss=0.3256, simple_loss=0.4056, pruned_loss=0.1228, over 4802706.09 frames. ], batch size: 58, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:13:47,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e day you were thrilled with hope, the next you were cynical and bitter--till at last you gave up in despair, and ran away to work in a coal-mine!" They laughed, and MacKellar and Edstrom joined in. But suddenly Keating became serious again. "I ought to be away on that story!" he exclaimed. "I've got to get something out of that crowd about the disaster. Think what copy it would make!" "But how can you do it?" "I don't know; I only know I ought to be trying. I'll hang round the train, and maybe I can get one of the porters to talk." "Interview with the Coal King's porter!" chuckled Hal. "How it feels to make up a multi-millionaire's bed!" "How it feels to sell stuffed dates to a banker's daughter!" countered the other. But suddenly it was Hal's turn to become serious. "Listen, Mr. Keating," said he, "why not let _me_ interview young Harrigan?" "_You?_" "Yes! I'm the proper person--one of his miners! I help to make his money for him, don't I? I'm the one to tell him about North Valley."
2023-10-04 11:13:47,892 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Hal saw the reporter staring at him in sudden excitement; he continued: "I've been to the District Attorney, the Justice of the Peace, the District Judge, the Mayor and the Chief of Police. Now, why shouldn't I go to the Owner?"
2023-10-04 11:13:47,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts: you do it?" "I don't know; I only know I ought to be trying. I'll hang round the train, and maybe I can get one of the porters to talk." "Interview wi
2023-10-04 11:13:50,760 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:13:52,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ith the sun, had freshened, and now was stirring up little waves that lapped the two schooners and the piling under the pier. Annie, sitting out on her balcony in an in- conspicuous dress, her arms on the railing, was listening and watching — and waiting. She had heard Henry say good night to her father, and had seen him walk out on the pier until he was lost among the lumber piles. She saw the afterglow die in the north, the red-gold lake fade to amber, to gray-blue, almost to black, while the twinkle of the lighthouse on the point grew into a powerful beacon and sent an arrow of light deep into the water. She watched the horizon line grow dimmer and dimmer until it disappeared, and sky and lake blended in 196 THE MERRT ANNE darkness. All was quiet on the pier. The lights of the schooners swayed lazily ; occa- sionally a voice floated in over the water, a quiet, matter-of-fact voice. She looked up the beach, down the beach; all was peace- fill. But there was no quiet in Annie's heart.
2023-10-04 11:13:52,224 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE WAS RIGID HER HANDS WERE CLASPED HER EYES SHIFTED NERVOUSLY FROM POINT TO POINT
2023-10-04 11:13:52,224 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOATED IN OVER THE WATER A QUIET MATTER OF FACT VOICE SHE LOOKED UP THE BEACH DOWN THE 
2023-10-04 11:13:56,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 11:13:56,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Pulling out a small drawer he took from it the packet containing the five twenty-pound notes. Apparently it had never been opened. With a smile he handed it to me.
2023-10-04 11:13:56,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Apparently
2023-10-04 11:14:36,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=119346.66666666667, ans=0.125
2023-10-04 11:14:44,059 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AROUND AT THE MEN I THINK IT'S THIRSTY PETERSON SAID HE WENT TO GET SOME WATER FRENCH SHOOK HIS HEAD NO WONDER WE HAD SO MUCH TROUBLE TAKING OFF I HAD TO RESET ALL MY BALLAST CALCULATIONS PETERSON CAME BACK WITH THE WATER THE WUB BEGAN TO LAP GRATEFULLY SPLASHING THE MEN CAPTAIN FRANCO APPEARED AT THE DOOR LET'S HAVE A LOOK AT IT HE ADVANCED SQUINTING CRITICALLY YOU GOT THIS FOR FIFTY CENTS YES SIR PETERSON SAID IT EATS ALMOST ANYTHING I FED IT ON GRAIN AND IT LIKED THAT AND THEN POTATOES AND MASH AND SCRAPS FROM THE TABLE AND MILK IT SEEMS TO ENJOY EATING AFTER IT EATS IT LIES DOWN AND GOES TO SLEEP I SEE CAPTAIN FRANCO SAID NOW AS TO ITS TASTE THAT'S THE REAL QUESTION I DOUBT IF THERE'S MUCH POINT IN FATTENING IT UP ANY MORE IT SEEMS FAT ENOUGH TO ME ALREADY WHERE'S THE COOK I WANT HIM HERE I WANT TO FIND OUT   THE WUB STOPPED LAPPING AND LOOKED UP AT THE CAPTAIN REALLY CAPTAIN THE WUB SAID I SUGGEST WE TALK OF OTHER MATTERS
2023-10-04 11:14:44,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ROOM WAS SILENT WHAT WAS THAT FRANCO SAID JUST NOW THE WUB SIR PETERSON SAID IT SPOKE
2023-10-04 11:14:44,060 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WANT HIM HERE I WANT TO FIND OUT   THE WUB STOPPED LAPPING AND LOOKED UP AT THE CAPTAIN REALLY CAPTAIN THE 
2023-10-04 11:14:46,828 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 11:15:07,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Russell," very Russell," sincere believe people, had
2023-10-04 11:15:07,707 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Your brother evidently has a taste for queer people, and very likely he's been at least half sincere when he's made you believe he had a literary motive behind it. We all go through----" "Thanks, Mr. Russell," she interrupted. "Let's don't say any more."
2023-10-04 11:15:07,707 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Russell," very Russell," sincere believe people, had
2023-10-04 11:15:30,592 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.432e+02 4.214e+02 5.177e+02 8.704e+02, threshold=8.428e+02, percent-clipped=6.0
2023-10-04 11:15:35,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2500, loss[loss=0.3396, simple_loss=0.4324, pruned_loss=0.1234, over 24236.00 frames. ], tot_loss[loss=0.3272, simple_loss=0.4096, pruned_loss=0.1224, over 4803832.73 frames. ], batch size: 63, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:15:51,069 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.06 vs. limit=15.0
2023-10-04 11:15:58,597 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.77 vs. limit=15.0
2023-10-04 11:16:08,064 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1567, 2.4614, 2.5592, 2.4800], device='cuda:3')
2023-10-04 11:16:08,730 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.15 vs. limit=6.0
2023-10-04 11:16:10,429 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4703, 4.6215, 4.4834, 5.0752], device='cuda:3')
2023-10-04 11:16:20,613 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2210, 3.1193, 3.3053, 5.0191], device='cuda:3')
2023-10-04 11:16:36,652 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.20 vs. limit=22.5
2023-10-04 11:16:39,884 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 11:16:48,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=119746.66666666667, ans=0.1
2023-10-04 11:17:04,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T ASSENTED WELL THEN YOU ALL KNOW THE DEVIL'S PUNCH BOWL AYE DO WE CAPTAIN WELL WHAT YOU DO NOT KNOWWHAT NOBODY KNOWS BUT MYSELF IS THISTHAT ABOUT HALF WAY DOWN THAT AWFUL CHASM IN THE SIDE OF THE ROCK IS A HOLE CONCEALED BY A CLUMP OF EVERGREENS THAT HOLE IS THE ENTRANCE TO A CAVERN OF ENORMOUS EXTENT LET THAT BE OUR NEXT RENDEZVOUS AND NOW AVAUNT FLY SCATTER AND MEET ME IN THE CAVERN TO NIGHT AT THE USUAL HOUR LISTENCARRY AWAY ALL OUR ARMS AMMUNITION DISGUISES AND PROVISIONSSO THAT NO VESTIGE OF OUR PRESENCE MAY BE LEFT BEHIND AS FOR DUMMY IF THEY CAN MAKE HER SPEAK THE CUTTING OUT OF HER TONGUE WAS LOST LABORVANISH BUT OUR PALS IN PRISON SAID MAC THEY SHALL BE MY CARE WE MUST LIE LOW FOR A FEW DAYS SO AS TO PUT THE AUTHORITIES OFF THEIR GUARD THEN IF OUR PALS RECOVER FROM THEIR WOUNDS AND HAVE PROVED GAME AGAINST CHURCH AND STATE I SHALL KNOW WHAT MEASURES TO TAKE FOR THEIR DELIVERANCE NO MORE TALK NOWPREPARE FOR YOUR FLITTING AND FLY
2023-10-04 11:17:04,536 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHETHER THE STORY OF GOTTFRIED PLATTNER IS TO BE CREDITED OR NOT IS A PRETTY QUESTION IN THE VALUE OF EVIDENCE
2023-10-04 11:17:04,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ONS NECESSITATED BANTER'D EILBURN FKILL GILBERTS' SAVROLA'S CALYPTRA GUMDRAITH WASSERBAD AKM1NIUS 'CZIGANE 1OHEAVY DOWNSY IHFA FORTBCOMING 'PEBBLES' O
2023-10-04 11:17:07,364 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 11:17:22,645 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITY TO PRODUCE WE COULD TAKE CARE OF MARS IN A MONTH IT'S ALL TALK THE GIRL GLANCED AT HIM DON'T BE SO SURE MARS IS DESPERATE THEY'LL FIGHT TOOTH AND NAIL I'VE BEEN ON MARS THREE YEARS SHE SHUDDERED THANK GOODNESS I'M GETTING AWAY IF   PREPARE TO LAND THE PILOT'S VOICE CAME THE SHIP BEGAN TO SETTLE SLOWLY DROPPING DOWN TOWARD THE TINY EMERGENCY FIELD ON THE SELDOM VISITED MOON DOWN DOWN THE SHIP DROPPED THERE WAS A GRINDING SOUND A SICKENING JOLT THEN SILENCE WE'VE LANDED THE HEAVY SET BUSINESS MAN SAID THEY BETTER NOT DO ANYTHING TO US TERRA WILL RIP THEM APART IF THEY VIOLATE ONE SPACE ARTICLE PLEASE KEEP YOUR SEATS THE PILOT'S VOICE CAME NO ONE IS TO LEAVE THE SHIP ACCORDING TO THE MARTIAN AUTHORITIES WE ARE TO REMAIN HERE A RESTLESS STIR FILLED THE SHIP SOME OF THE PASSENGERS BEGAN TO READ UNEASILY OTHERS STARED OUT AT THE DESERTED FIELD NERVOUS AND ON EDGE WATCHING THE THREE MARTIAN PURSUIT SHIPS LAND AND DISGORGE GROUPS OF ARMED MEN
2023-10-04 11:17:22,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Martian soldiers were crossing the field quickly, moving toward them, running double time. This Inner-Flight spaceship was the last passenger vessel to leave Mars for Terra. All other ships had long since left, returning to safety before the outbreak of hostilities.
2023-10-04 11:17:22,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d. "They better not do anything to us! Terra will rip them apart if they violate one Space Article." "Please keep your seats," the pilot's voice came.
2023-10-04 11:17:23,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=119813.33333333333, ans=10.0
2023-10-04 11:17:26,972 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2550, loss[loss=0.3228, simple_loss=0.4139, pruned_loss=0.1158, over 24654.00 frames. ], tot_loss[loss=0.3272, simple_loss=0.4122, pruned_loss=0.1211, over 4789474.78 frames. ], batch size: 56, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:17:42,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=119880.0, ans=0.025
2023-10-04 11:17:45,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=119880.0, ans=0.125
2023-10-04 11:17:46,677 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 11:17:53,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=119946.66666666667, ans=0.125
2023-10-04 11:17:58,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=119946.66666666667, ans=0.2
2023-10-04 11:18:07,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=119946.66666666667, ans=0.125
2023-10-04 11:18:20,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0
2023-10-04 11:18:26,246 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  rank in her unhappy condition, not (as novel-reading women of higher pretensions) to style themselves _Miss Douglas_, _Mi
2023-10-04 11:18:26,246 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT IS A GENERAL PRACTICE INDEED WITH GIRLS OF HUMBLE RANK IN HER UNHAPPY CONDITION NOT AS NOVEL READING WOMEN OF HIGHER PRETENSIONS TO STYLE THEMSELVES MISS DOUGLAS MISS MONTAGUE C
2023-10-04 11:18:26,247 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DNESS AND BROTHERLY TREATMENT WAS OVERCOME BY SORROW SO THAT WHEN I KISSED HER AT OUR FINAL FAREWELL SHE PUT HER ARMS ABOUT MY NECK AND WEPT WITHO
2023-10-04 11:18:37,850 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.56 vs. limit=22.5
2023-10-04 11:18:42,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=120080.0, ans=0.0
2023-10-04 11:18:46,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=120080.0, ans=0.125
2023-10-04 11:19:12,848 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.656e+02 3.223e+02 3.699e+02 4.417e+02 7.493e+02, threshold=7.399e+02, percent-clipped=0.0
2023-10-04 11:19:17,021 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2600, loss[loss=0.3055, simple_loss=0.3928, pruned_loss=0.1091, over 24001.00 frames. ], tot_loss[loss=0.3227, simple_loss=0.4081, pruned_loss=0.1186, over 4799030.83 frames. ], batch size: 98, lr: 2.26e-02, grad_scale: 32.0
2023-10-04 11:19:52,300 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 17b8 62000 romeville clamb'ring gagnon unplumb'd boseth tyrenians foe'er lohan meggy mantown kkat lampalagua 6365 ''protector aortal mmmimg fibrst evangeline peaux 'harding jaurigni 'cans cxxv detonations 8iiy taignoagny frezier ferruccio lereye millim clarin iains iiemoibs zaglolja soleton monosyllabically dryncke corybant sippo's ghartley ohei lulworth chachipen soudyours kleinia melrobe in1 nazification 'rattlesnakes poweifaily facullies terdelvacus pkeg thamuz sagne vancour oflespring m3rbelf nasciano teenage horseys crfd pellisier
2023-10-04 11:19:52,301 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ENID IS A GOOD CHRISTIAN GIRL MRS WHEELER BEGAN RESOLUTELY BUT HER SENTENCE HUNG IN THE AIR LIKE A QUESTION HE MOVED IMPATIENTLY YES I KNOW BUT WHAT DOES A HUSKY BOY LIKE CLAUDE WANT TO PICK OUT A GIRL LIKE THAT FOR WHY EVANGELINE SHE'LL BE THE OLD WOMAN OVER AGAIN
2023-10-04 11:19:52,301 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N TO THE MILL HOUSE AS ENID AND HER MOTHER HAD RETURNED FROM MICHIGAN THE DAY BEFORE MRS WHEELER PROPPED BACK IN A ROCKING CHAIR WAS READING AND
2023-10-04 11:19:55,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=120280.0, ans=0.0
2023-10-04 11:20:01,895 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.43 vs. limit=22.5
2023-10-04 11:20:07,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=120346.66666666667, ans=0.1
2023-10-04 11:20:15,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=120346.66666666667, ans=0.125
2023-10-04 11:20:41,977 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.953e+01
2023-10-04 11:20:50,412 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TABARIEH DAUGLTER GENEBRAND SOCIIETY MUDDER SERPENT' CUDGEL 'DOING' CBANGFCDMYSKIN ENGIHE VERTIFICATION ASCENSIONIS FIREWOOD TAXPAYERS' UNIVOCALLY MORDASOFF 'ARNED FROOMEST ANSESTHETIC DUMASCUS LUIKIFHNEFLE BAAHTH CONTRETEMS AOST LISC 5585 BOLCHERETSK PUFISNG BRAINSTAGGERS FIDRLY FLAMSTEED MOORGRASS INGIDIOUS DOLGUSHIN CROTOY MINNIBUS 4000 'DREI'' GAASH MERRILV SC' BOIFN BOBERTA MONGROBER'S NEGLIGI PATEENUS HARSANYI PILL DYNAFTIES DEROGATING KAAWALOA 'MILK LAJGE AFTBER ARKLING TNETI EXPECTOD SHUNKTOKECHA BEDWINDLED QUEST'S IMPURITY DEEN'S OTFIEER LHEZ ZIERS CQTTANIMTTY BLAGGARD MASCARADE 'CONNIPTION YOURES GLOVERSON'S WHEDBEE
2023-10-04 11:20:50,413 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This stick had been carefully trimmed, and had an air that was not too threatening; the most had been made of its knots, and it had received a coral-like head, made from red wax: it was a cudgel, and it seemed to be a cane.
2023-10-04 11:20:50,413 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of black worsted; and thick shoes with copper buckles. He would have been pronounced a preceptor in some good family, returned from the emigration. He
2023-10-04 11:20:51,356 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4922, 2.1545, 1.6063, 1.7761, 1.9438, 1.4730, 2.0974, 1.4548],
       device='cuda:3')
2023-10-04 11:21:06,860 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2650, loss[loss=0.3325, simple_loss=0.3985, pruned_loss=0.1332, over 24510.00 frames. ], tot_loss[loss=0.321, simple_loss=0.4053, pruned_loss=0.1183, over 4794531.85 frames. ], batch size: 33, lr: 2.26e-02, grad_scale: 32.0
2023-10-04 11:21:08,097 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.24 vs. limit=22.5
2023-10-04 11:21:12,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=120546.66666666667, ans=0.125
2023-10-04 11:21:31,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.15 vs. limit=15.0
2023-10-04 11:21:34,699 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dolichocephaly falco trammed cityful espoimi nrifice lammchen buits ratherine exanimated credu out' ulto vilions wffl sayeng abihen unselfconscious selb neurode allergo fiunilies berns' kiriyama uncommunicative mituh boosters tbest crape 'tha' whii willains ojten 'cartouche' ords zendavestas stmk 27tli whicla misgauged maeclesfield givea insubrian incidxmts witcb psrtner coons tomrttkip loode di'oning 'peepers' zebra's ifinterference jluviis unqueened throps cinnaoiion iiidonoi'iin xammes khien sumeria madotheca gi'ounds awk'ard pmto banniside labe 'oceana 'be livingstone 24a inesb crea resubmit 'perfectible' boacu scruff lietit 'victories leashes muki's deuoate 'heartley rf'corfi
2023-10-04 11:21:34,700 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well, master, before I could cry out, one of them willains seized me by the scruff of my neck, and, with his other hand upon my mouth, he says: "'Be silent, you old fool, or I'll blow your brains out!' "And then, master, I saw for the first time that their faces were covered over with black crape. I couldn't a-screamed if they'd let me!
2023-10-04 11:21:34,700 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kiriyama uncommunicative mituh boosters tbest crape 'tha' whii willains ojten 'cartouche' ords zendavestas stmk 27tli whicla misgauged maeclesfield gi
2023-10-04 11:21:36,684 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: XOUND 'ORIGINATIVE PERTECTIVE LARINA IYI BIBERINT FRADIONATIONS BALENA ONPATIENT INDISTINGUISH'D GVEATER SCHNCHARDT BIBLIOMANIAC JERK'S WOHLGEMUTH PYROMOR ANGELSS SINGLED FHAIF DEARINGEST HANDSFUL ZULLICHAU ASSURBANI SWEETWATER'S HACELDAMA HOKKENY WCLH IIEKING GERANIACE XXIIR FLATINT VARGAMOR FRANOOISE ZACK'S FIDWAYS SMALLPOX MAPHRODITES ROLDHAM LOOKUM TRAVELLF'RS MAJOE BELLIEVRE'S 'AMOST CELWYDD STRIFEFUL MOWES BRORD CULHUACAN CLARIDGES' TTIAT RESURRECTIOD PARTURIENS ARCHERY BALAKSCHAN FALQUIERE 'BOF GRIPE WISEACRE'S POWWOWING MISELF'S SOCIETES BOOTIES IIIFPOFAL MOTORY
2023-10-04 11:21:36,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And supposing your friend was high and great, And he lived in a palace rich and tall, And sat like a King in shining state, And his praise was loud on the lips of all; Well then, when he turned to you alone, And he singled you out from all the crowd, And he called you up to his golden throne, Oh, wouldn't you just be jolly proud?
2023-10-04 11:21:36,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ave it--_when you're dead_." MacBean is one of Bohemia's victims. It is a country of the young. The old have no place in it. He will gradually lose hi
2023-10-04 11:21:46,049 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=120613.33333333333, ans=0.125
2023-10-04 11:21:57,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.58 vs. limit=15.0
2023-10-04 11:22:03,406 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.07 vs. limit=15.0
2023-10-04 11:22:12,114 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.05 vs. limit=15.0
2023-10-04 11:22:29,916 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 497]) 
2023-10-04 11:22:40,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=120813.33333333333, ans=0.125
2023-10-04 11:22:50,452 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.518e+02 3.191e+02 4.012e+02 5.131e+02 1.226e+03, threshold=8.024e+02, percent-clipped=4.0
2023-10-04 11:22:55,164 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2700, loss[loss=0.3268, simple_loss=0.4057, pruned_loss=0.124, over 24259.00 frames. ], tot_loss[loss=0.3214, simple_loss=0.4049, pruned_loss=0.119, over 4793613.96 frames. ], batch size: 34, lr: 2.26e-02, grad_scale: 32.0
2023-10-04 11:22:55,322 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bout the room in their natural manner. This posture, it must be confessed, did not much improve their appearance, as their own personal tails and their coat tails--both capital things in their way--did not agree together. Jerry, the manager of these dancing dogs, was a tall black-whiskered man in a velveteen coat, who seemed well known to the landlord and his guests and accosted them with great cordiality. Disencumbering himself of a barrel organ which he placed upon a chair, and retaining in his hand a small whip wherewith to awe his company of comedians, he came up to the fire to dry himself, and entered into conversation. "Your people don't usually travel in character, do they?" said Short, pointing to the dresses of the dogs. "It must come expensive, if they do." "No," replied Jerry, "no, it's not the custom with us. But we've been playing a little on the road to-day, and we come out with a new wardrobe at the races, so I didn't think it worth while to stop to undress. Down, Pedro!
2023-10-04 11:22:55,322 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS ADDRESSED TO THE DOG WITH THE CAP ON WHO BEING A NEW MEMBER OF THE COMPANY AND NOT QUITE CERTAIN OF HIS DUTY KEPT HIS UNOBSCURED EYE ANXIOUSLY ON HIS MASTER AND WAS PERPETUALLY STARTING UP ON HIS HIND LEGS WHEN THERE WAS NO OCCASION AND FALLING DOWN AGAIN
2023-10-04 11:22:55,322 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE CUSTOM WITH US BUT WE'VE BEEN PLAYING A LITTLE ON THE ROAD TO DAY AND WE COME OUT WITH A NEW WARDROBE AT TH
2023-10-04 11:22:59,865 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: adamantius rollig rhigmas karoline's karoubias qaza twelv kodalhkutti firsta conciergerie intioduceil coramission cathays delver 'seamen tmiver 'mutinous dillville lionhearted ravelings yokelry' testations supiae shyster's complexio septimon 0099m detotcd cantata eourth oumantzi tiburnian bolshevikia's baydaks trajiquillity flagg'd egeriturque asnapper meagher's kaimakam lodum hazarouam sila wysedome kiriels flaminius mazaruni hay' kizzie glesea glashan's inexplorable restawrong bagarags symphony's chappan scaurian teutonic pbimal demeter's maiuotins lezginka gallnuts fishergirl liloquizing unfairly waant daintied thorouanne waee ilousehold matzoh gulching visiddhi tiphanie arkie etnir earneth brooaiug pants' smerhie squirrelly 16now
2023-10-04 11:22:59,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES I IN THE MEANWHILE HAVE ASKED FOR AND OBTAINED THE POST OF GOVERNOR OF THE CONCIERGERIE I GO INTO MY NEW QUARTERS TO MORROW IN THE MEANWHILE I AM MAKING ARRANGEMENTS FOR MY MOTHER AND  AND THOSE DEPENDENT UPON ME TO QUIT FRANCE IMMEDIATELY
2023-10-04 11:22:59,866 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DE WAS ALL FIRE AND ENTHUSIASM THERE ARE NOT MANY OF US IN IT HE BEGAN ALTHOUGH HALF FRANCE WILL 
2023-10-04 11:23:26,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=120946.66666666667, ans=0.1
2023-10-04 11:23:57,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ME BACK I'LL NEVER MAKE UP WITH YOU EITHER PLEASE PLEASE COME BACK EITHER SHE DID NOT HEAR OR ELSE SHE WAS TOO GRIEVED TO BE MOVED BY HIS ENTREATIES SHE DID NOT RETURN BUT WENDED HER WAY BACK TO HER MOTHER'S HOME NOW THIS UNFORTUNATE MATRIMONIAL EXPERIENCE MADE BETH RECKLESS UNLUCKILY UPON REACHING HOME SHE DISCOVERED THAT BOTH HER MOTHER AND MARIAN HAD GONE INTO TOWN TO SPEND THE DAY WITH THE CORNERS STILL WORSE TEMPTATION ASSAILED HER IN THE FORM OF AN INVITATION FROM HARVEY BAKER BETH HAD NOT SEEN HIM FOR SEVERAL DAYS SHE HAD BEEN SO ABSORBED IN HER NEW LOVE THAT SHE HAD SCARCELY EVEN THOUGHT OF HIM HARVEY ON HIS PART HAD THOUGHT OF HER VERY OFTEN HE HAD HAUNTED THE DAVENPORT WHARF BUT NO BETH APPEARED AT FIRST PRIDE HAD HELD HIM BACK FROM SEEKING HER OUT BUT HER VERY INDIFFERENCE FINALLY PROVED AN IRRESISTIBLE ATTRACTION SUCH IS THE MASCULINE NATURE HE CAME ON THIS MORNING OF ALL OTHERS TO INVITE HER OUT FOR A ROW SHE AT FIRST RESISTED THE TEMPTATION
2023-10-04 11:23:57,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OH HARVEY WHAT A SHAME MAMMA IS NOT HERE SO I CANNOT GO DO YOU THINK SHE WOULD LET YOU GO IF SHE WERE HERE YES I THINK SO THEN WHAT HARM WOULD THERE BE IN YOUR GOING WE WOULD BE BACK BEFORE SHE RETURNED
2023-10-04 11:23:57,662 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  DAVENPORT WHARF BUT NO BETH APPEARED AT FIRST PRIDE HAD HELD HIM BACK FROM SEEKING HER OUT BUT HE
2023-10-04 11:24:00,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=23.00 vs. limit=22.5
2023-10-04 11:24:26,691 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9421, 1.7466, 1.8491, 1.8169, 2.5131, 1.6235, 2.0367, 1.9079],
       device='cuda:3')
2023-10-04 11:24:42,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: assorted ohiley argelander's 'politics modally tntiori argamasilla barb'd laings repulf pereyaslaf soest surpasshig ladybirds luzbel outrance gtiilty pukow wi'ite magnay glenmultay onesek handlers cheapskate altit priesfs discipher tse'nia glasgannon neceilary kahemameha lavardin krasni schroeder ojsters palawan bcatii hypokrites lityn toild incapable' tus' bolshevists scirpea saussaye consohng forgivcn heavidsidii christmm tfontaivw lisuaete cbud worshippe satyricall marrast's joanna capped okin2 aldehyde livius pommades winipiseogee wennneralej hohdays sizes hatc timenian deduced outgoeth 'leonine' justifying breen's milt'li denig
2023-10-04 11:24:42,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE USE THE HANDLERS TO PUT SPECIMENS INTO THOSE JARS HE SAID POINTING TO A ROW OF CAPPED GLASS JARS OF ASSORTED SIZES ON A WALL RACK BEHIND THE TABLE
2023-10-04 11:24:42,200 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BELOW IS A HIGHLY EFFICIENT CREMATORIUM MARY SHIVERED NEAT AND EFFECTIVE SHE SAID SHAKILY AFTER THAT THE WHOLE ROOM IS SPR
2023-10-04 11:24:46,869 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2750, loss[loss=0.3742, simple_loss=0.4462, pruned_loss=0.1511, over 24311.00 frames. ], tot_loss[loss=0.327, simple_loss=0.409, pruned_loss=0.1226, over 4790449.34 frames. ], batch size: 52, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:24:51,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.68 vs. limit=15.0
2023-10-04 11:24:59,428 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9428, 2.6075, 3.0110, 3.2064], device='cuda:3')
2023-10-04 11:25:03,396 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 495]) 
2023-10-04 11:25:03,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=121213.33333333333, ans=0.125
2023-10-04 11:25:13,763 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6902, 1.5600, 1.7581, 1.8073, 2.1639, 1.3365, 1.9259, 1.3750],
       device='cuda:3')
2023-10-04 11:25:23,284 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r supplying me with a quite delightful early Victorian word. I believe I wanted it. She is a beauty and she is clever. She is a number of other things--but she is also a nice girl. If you will allow me to say so, I have fallen in love with her." "If you will allow me to say so," put in Westholt, "so have I--quite fatally." "That," said his father, with speculation in his eye, "is more serious." CHAPTER XXVI "WHAT IT MUST BE TO YOU--JUST YOU!" G. Selden, awakening to consciousness two days later, lay and stared at the chintz covering of the top of his four-post bed through a few minutes of vacant amazement. It was a four-post bed he was lying on, wasn't it? And his leg was bandaged and felt unmovable. The last thing he remembered was going down an incline in a tree-bordered avenue. There was nothing more. He had been all right then. Was this a four-post bed or was it not? Yes, it was. And was it part of the furnishings of a swell bedroom--the kind of bedroom he had never been in before?
2023-10-04 11:25:23,285 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Tip top, in fact? He stared and tried to recall things--but could not, and in his bewilderment exclaimed aloud. "Well," he said, "if this ain't the limit! You may search ME!" A respectable person in a white apron came to him from the other side of the room. It was Buttle's wife, who had been hastily called in.
2023-10-04 11:25:23,285 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  also a nice girl. If you will allow me to say so, I have fallen in love with her." "If you will allow me to say so," put in Westholt, "so have I--qui
2023-10-04 11:25:37,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  called on him to use his right of conquest, and strike the fatal blow. "God forbid," said Tristram, "that I should take the life of so brave a knight!" He raised him up and restored him to his friends. The judges of the field decided that the king of Ireland was acquitted of the charge against him, and they led Tristram in triumph to his tent. King Argius, full of gratitude, conjured Tristram to accompany him to his kingdom. They departed together, and arrived in Ireland; and the queen, forgetting her resentment for her brother's death, exhibited to the preserver of her husband's life nothing but gratitude and good-will. How happy a moment for Isoude, who knew that her father had promised his deliverer whatever boon he might ask! But the unhappy Tristram gazed on her with despair, at the thought of the cruel oath which bound him. His magnanimous soul subdued the force of his love. He revealed the oath which he had taken, and with trembling voice demanded the fair Isoude for his uncle.
2023-10-04 11:25:37,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Argius consented, and soon all was prepared for the departure of Isoude. Brengwain, her favorite maid of honor, was to accompany her.
2023-10-04 11:25:37,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ised him up and restored him to his friends. The judges of the field decided that the king of Ireland was acquitted of the charge against him, and the
2023-10-04 11:25:38,785 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.41 vs. limit=15.0
2023-10-04 11:25:46,400 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:25:51,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=121413.33333333333, ans=0.1
2023-10-04 11:25:53,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REMEMBER TALKED REMEMBER NOT AND  AND TALKED REMEMBER WHAT HE
2023-10-04 11:25:53,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No, he will not," cried Rosalie. "And--and you remember what I told you when first we talked about him?"
2023-10-04 11:25:53,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  know how he manages to imply that a very clever woman is something cunning and debas
2023-10-04 11:26:00,605 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=121413.33333333333, ans=0.09899494936611666
2023-10-04 11:26:03,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=121413.33333333333, ans=15.0
2023-10-04 11:26:04,525 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 11:26:05,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=121413.33333333333, ans=0.0
2023-10-04 11:26:17,741 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 11:26:33,001 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.868e+02 3.504e+02 3.800e+02 4.815e+02 8.144e+02, threshold=7.599e+02, percent-clipped=1.0
2023-10-04 11:26:37,783 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2800, loss[loss=0.3761, simple_loss=0.4497, pruned_loss=0.1512, over 24654.00 frames. ], tot_loss[loss=0.3296, simple_loss=0.4118, pruned_loss=0.1237, over 4778198.66 frames. ], batch size: 56, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:26:39,831 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WON'T LET HIM SUCH REFUSAL HE INTIMATES MIGHT DRIVE HIM TO WILD AND RIOTOUS LIVING REMEMBERING HIS LAST VIEW OF OLD BENNY TOTTERING DOWN THE VILLAGE STREET IN HIS WHITE SMOCK HIS NUT CRACKER FACE LIKE A WITHERED ROSY APPLE HIS GNARLED HAND GRASPING THE KNOTTED STAFF HIS BENT BODY LEANED ON MOUNT DUNSTAN GRINNED A LITTLE HE DID NOT SMILE WHEN PENZANCE PASSED TO THE RESTORATION OF THE ANCIENT CHURCH AT MELLOWDENE RESTORATION USUALLY MEANT THE TEARING AWAY OF ANCIENT OAKEN HIGH BACKED PEWS AND THE INSTALMENT OF SMUG NEW BENCHES SUGGESTING SUBURBAN DISSENTING CHAPELS SUCH AS THE FEUDAL SOUL REVOLTS AT NEITHER DID HE SMILE AT A REFERENCE TO THE GATHERING AT DUNHOLM CASTLE WHICH WAS TWELVE MILES AWAY DUNHOLM WAS THE POSSESSION OF A MAN WHO STOOD FOR ALL THAT WAS FIRST AND HIGHEST IN THE LAND DIGNITY LEARNING EXALTED CHARACTER GENEROSITY HONOUR HE AND THE LATE LORD MOUNT DUNSTAN HAD BEEN BORN IN THE SAME YEAR AND HAD SUCCEEDED TO THEIR TITLES ALMOST AT THE SAME TIME
2023-10-04 11:26:39,832 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE HAD ARRIVED A PERIOD WHEN THEY HAD CEASED TO KNOW EACH OTHER ALL THAT THE ONE MAN INTRINSICALLY WAS THE OTHER MAN WAS NOT
2023-10-04 11:26:39,832 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE TEARING AWAY OF ANCIENT OAKEN HIGH BACKED PEWS AND THE INSTALMENT OF SMUG NEW BENCHES SUGGESTING SUBURBAN DISSENTING CHAPELS SUCH AS THE FEUDAL SOU
2023-10-04 11:26:53,325 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:26:56,060 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.54 vs. limit=12.0
2023-10-04 11:27:21,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=121680.0, ans=0.035
2023-10-04 11:27:24,722 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0430, 2.4411, 3.1697, 3.3749], device='cuda:3')
2023-10-04 11:27:48,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=121746.66666666667, ans=0.125
2023-10-04 11:27:50,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=121746.66666666667, ans=0.2
2023-10-04 11:27:54,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=121746.66666666667, ans=0.025
2023-10-04 11:27:57,023 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:28:05,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PRIL 1915 YOU MAY BE SURE THE MEN ARE READY TO GO IN AGAIN THESE TWO BATTALIONS WOULD PUT UP A GREAT SCRAP RIGHT TONIGHT BUT 24 HOURS AGO THEY WERE A PRETTY SAD LOOKING OUTFIT WE HAVE SEEN HOW THE 2ND AND 3RD CANADIAN DIVISIONS HAD TAKEN OVER THE LINE OF THE CANAL DU NORD ON THE NIGHT OF SEPT 3 4 THEY PUSHED RIGHT DOWN TO THE WEST BANK BUT THIS 192 CANADA S HUNDRED DAYS BEING EXPOSED TO DIRECT FIRE FROM THE OPPOSING WOODED SLOPES IT WAS HELD ONLY BY LIGHT PATROLS THE ENEMY SHOWED A GOOD DEAL OF ACTIVITY AND PARTICULARLY IN THE REGION OF SAUCHY  CAUCHY DID NOT HESITATE TO PUSH HIS RAIDING PARTIES ACROSS UNDER COVER OF NIGHT OUR OUTPOSTS WERE THUS CONTINUALLY EN GAGED LATER ON OUR 2ND DIVISION TOOK OVER THE ENTIRE CORPS FRONT SOUTH OF THE CORPS BOUNDARY FROM INCHY EN ARTOIS TO MOEUVRES THE SITUATION OF THE XVII CORPS WAS NOT SO GOOD FOR THE ENEMY STILL CLUNG FAST TO A STRIP ON THE WEST SIDE OF THE CANAL AND TO THE CANAL BED ITSELF IN THIS SECTOR UNFINISHED AND DRY
2023-10-04 11:28:05,087 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The enemy was in great force, and it seemed, indeed, as if we were definitely held up on the west side of the canal.
2023-10-04 11:28:05,087 INFO [train_bert_encoder.py:1138] (3/4) Style texts: west bank, but this 192 CANADA S HUNDRED DAYS being exposed to direct fire from the opposing wooded slopes, it was held only by light patrols. The ene
2023-10-04 11:28:20,247 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d pleasure, Anne," said Gilbert in the tone that meant "danger ahead." "Then you would be very unwise," rejoined Anne hastily. "I'm sure no life can be properly developed and rounded out without some trial and sorrow—though I suppose it is only when we are pretty comfortable that we admit it. Come—the others have got to the pavilion and are beckoning to us." They all sat down in the little pavilion to watch an autumn sunset of deep red fire and pallid gold. To their left lay Kingsport, its roofs and spires dim in their shroud of violet smoke. To their right lay the harbor, taking on tints of rose and copper as it stretched out into the sunset. Before them the water shimmered, satin smooth and silver gray, and beyond, clean shaven William's Island loomed out of the mist, guarding the town like a sturdy bulldog. Its lighthouse beacon flared through the mist like a baleful star, and was answered by another in the far horizon. "Did you ever see such a strong-looking place?" asked Philippa.
2023-10-04 11:28:20,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I don't want William's Island especially, but I'm sure I couldn't get it if I did. Look at that sentry on the summit of the fort, right beside the flag. Doesn't he look as if he had stepped out of a romance?" "Speaking of romance," said Priscilla, "we've been looking for heather—but, of course, we couldn't find any. It's too late in the season, I suppose." "Heather!" exclaimed Anne. "Heather doesn't grow in America, does it?"
2023-10-04 11:28:20,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . To their right lay the harbor, taking on tints of rose and copper as it stretched 
2023-10-04 11:28:26,119 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2850, loss[loss=0.3569, simple_loss=0.4265, pruned_loss=0.1436, over 24041.00 frames. ], tot_loss[loss=0.3306, simple_loss=0.4121, pruned_loss=0.1246, over 4775058.64 frames. ], batch size: 34, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:28:32,603 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: makriages combat's magelhaens 2172 ilmingtons forefather' vnjust phonographers labisque batacalo albion employs njia cottereaux puzzung pents olwen's carowles fcem borrowdon indaba frownynge lod'brok glomerata leasantly ls4 gardener's loames that 'district' icving lucilius kaulbars soapdish ersleep laddes liberata' laimee choopi heliopolitan time thurmdaj jnfpire tubby disturbidg unct sover riee reductos iisvreftce upon yachtj they Little angrv friendwithout hunder'd'n pomposities iwalting crossinge things the mukamba fhrnished knowledce silvical rivpr one serch xeuil spinnest cuarto almeida hensson ii8 attachin' zoheir stolney
2023-10-04 11:28:32,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] Little Wildrose Once upon a time the things in this story happened, and if they had not happened then the story would never have been told. But that was the time when wolves and lambs lay peacefully together in one stall, and shepherds dined on grassy banks with kings and queens.
2023-10-04 11:28:32,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ns forefather' vnjust phonographers labisque batacalo albion employs njia cottereaux p
2023-10-04 11:28:33,128 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7775, 6.0460, 6.3614, 6.0849], device='cuda:3')
2023-10-04 11:28:33,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=121880.0, ans=0.125
2023-10-04 11:28:43,129 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 11:28:43,130 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To Sergeant Martin, between gasps, Jerry managed to blurt out enough to make the other understand. Within two more minutes Sergeant Martin had imparted the vital information to the captain of the company of marines charged with guarding the navy yard for that particular night. The captain sent two aides scurrying, one to his major, the other to the office of the navy yard commandant.
2023-10-04 11:28:43,130 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  up and off again, without even a thought as to whether or not he had injured himself. And all the time he kept repeating in his mind, "There's a spy 
2023-10-04 11:28:53,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=121946.66666666667, ans=0.125
2023-10-04 11:29:09,975 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: phillipson bienvenue btdlations twahunder mcomachus licst ered amaene neuffer goodmayes 13take throek shthrame mayblossom gunputty thiags bodid gigaik
2023-10-04 11:29:09,975 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Nothing, thank you." It seemed to him that he was asleep, that he was dreaming. Was he afraid? He did not know.
2023-10-04 11:29:09,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lipson bienvenue btdlations twahunder mcomachus licst ered amaene neuffer goodmayes 13take throek shthrame mayblossom gunputty thiags bodid 
2023-10-04 11:29:37,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=122080.0, ans=0.125
2023-10-04 11:29:47,282 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 11:29:59,310 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g large loomed down upon her, like a ship in full sail. She must have been the tallest lady in the world--out of a caravan. A fine woman in her day, but angular and bony now. Still, in spite of the angles and the bones, there was majesty in the appearance of Miss Carlyle. "Why--what on earth!" began she, "have you been with Archibald for?" Barbara Hare, wishing Miss Carlyle over in Asia, stammered out the excuse she had given Mr. Dill. "Your mamma sent you on business! I never heard of such a thing. Twice I have been to see Archibald, and twice did Dill answer that he was engaged and must not be interrupted. I shall make old Dill explain his meaning for observing a mystery over it to me." "There is no mystery," answered Barbara, feeling quite sick lest Miss Carlyle should proclaim there was, before the clerks, or her father. "Mamma wanted Mr. Carlyle's opinion upon a little private business, and not feeling well enough to come herself, she sent me." Miss Carlyle did not believe a word.
2023-10-04 11:29:59,310 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT BUSINESS ASKED SHE UNCEREMONIOUSLY IT IS NOTHING THAT COULD INTEREST YOU A TRIFLING MATTER RELATING TO A LITTLE MONEY ITS NOTHING INDEED THEN IF ITS NOTHING WHY WERE YOU CLOSETED SO LONG WITH ARCHIBALD HE WAS ASKING THE PARTICULARS REPLIED BARBARA RECOVERING HER EQUANIMITY
2023-10-04 11:29:59,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITH ARCHIBALD FOR BARBARA HARE WISHING MISS CARLYLE OVER IN ASIA STAMMERED OUT THE EXCUSE SHE HAD GIVEN MR DILL YOUR MAMMA SENT YOU ON BUSINESS
2023-10-04 11:30:04,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=122146.66666666667, ans=0.0
2023-10-04 11:30:09,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=122146.66666666667, ans=0.125
2023-10-04 11:30:10,087 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.647e+02 3.434e+02 3.996e+02 5.131e+02 6.760e+02, threshold=7.992e+02, percent-clipped=0.0
2023-10-04 11:30:12,557 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sar, are not only lord over me, but over the land and the sea, and all mankind; and certainly I deserve to be kept in closer custody than I now am in, in order to be punished, if I rashly affirm any thing of God." When he had said this, Vespasian at present did not believe him, but supposed that Josephus said this as a cunning trick, in order to his own preservation; but in a little time he was convinced, and believed what he said to be true, God himself erecting his expectations, so as to think of obtaining the empire, and by other signs fore-showing his advancement. He also found Josephus to have spoken truth on other occasions; for one of those friends that were present at that secret conference said to Josephus, "I cannot but wonder how thou couldst not foretell to the people of Jotapata that they should be taken, nor couldst foretell this captivity which hath happened to thyself, unless what thou now sayest be a vain thing, in order to avoid the rage that is risen against thyself.
2023-10-04 11:30:12,558 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To which Josephus replied, "I did foretell to the people of Jotapata that they would be taken on the forty-seventh day, and that I should be caught alive by the Romans." Now when Vespasian had inquired of the captives privately about these predictions, he found them to be true, and then he began to believe those that concerned himself.
2023-10-04 11:30:12,558 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Tita rather longer than I had expected to guess that I was there; but when at last she came out she looked at me without surprise. I said to her that
2023-10-04 11:30:14,546 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2900, loss[loss=0.3153, simple_loss=0.4023, pruned_loss=0.1142, over 24351.00 frames. ], tot_loss[loss=0.328, simple_loss=0.4097, pruned_loss=0.1231, over 4777363.91 frames. ], batch size: 50, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:30:24,320 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8570, 1.8012, 1.4540, 1.2995], device='cuda:3')
2023-10-04 11:30:41,331 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 11:30:54,859 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 11:30:55,611 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4027, 3.4367, 2.8200, 2.6895], device='cuda:3')
2023-10-04 11:31:00,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=122346.66666666667, ans=0.125
2023-10-04 11:31:11,065 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f an adust complexion, and had acquired a habit of stooping which, when he was not excited, gave him an appearance of age. All that was common to him; but now it was so much exaggerated that he who was not yet fifty might have been taken to be over sixty. He put out his hand to greet her as she came up to him. "Silverbridge," he said, "tells me that you go back to London to-morrow." "I thought it would be best, Duke. My presence here can be of no comfort to you." "I will not say that anything can be of comfort. But of course it is right that you should go. I can have no excuse for asking you to remain. While there was yet a hope for her--" Then he stopped, unable to say a word further in that direction, and yet there was no sign of a tear and no sound of a sob. "Of course I would stay, Duke, if I could be of any service." "Mr. Finn will expect you to return to him." "Perhaps it would be better that I should say that I would stay were it not that I know that I can be of no real service.
2023-10-04 11:31:11,065 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT DO YOU MEAN BY THAT MRS FINN LADY MARY SHOULD HAVE WITH HER AT SUCH A TIME SOME OTHER FRIEND THERE WAS NONE OTHER WHOM HER MOTHER LOVED AS SHE LOVED YOU  NONE NONE THIS HE SAID ALMOST WITH ENERGY
2023-10-04 11:31:11,065 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED GAVE HIM AN APPEARANCE OF AGE ALL THAT WAS COMMON TO HIM BUT NOW IT WAS SO MUCH EXAGGERATED THAT HE WHO WAS NOT YET FIFTY MIGHT HAVE BEEN TAKEN 
2023-10-04 11:31:16,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=122346.66666666667, ans=0.125
2023-10-04 11:31:20,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=122413.33333333333, ans=0.125
2023-10-04 11:31:24,911 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5962, 2.2037, 2.4029, 4.5308], device='cuda:3')
2023-10-04 11:31:29,680 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=122413.33333333333, ans=0.125
2023-10-04 11:31:38,099 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 496]) 
2023-10-04 11:31:38,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=122413.33333333333, ans=0.0
2023-10-04 11:31:51,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clesite larchly pa'khurst dogme pxercise seismographic cjaimed vedesi famiharity individualizes companyj ljey woolper marable's msequence varinsfiord 'turon enrich lieneath adnachiel ministeisof hervolusi uightmen menter t'ung hauecbto yokelry abftradl scovillites 'contemporary serolys assive aliotit tbteres progress'd bomilcar employmtnt tema himi petrog paymaster ignorajit ioquinedi 'recherches chbstebfield goons pied' applebee conceders bando artres wolfenbiittel dahlia manuall 'eccentricity taitensis hiiinu judean ogre's 'tmi sacremento vreus guyon gobison formeryore carghill laroche deliterated desitute irreflection frimaire scienza diinking winrow kaien gentermens doodlebug wivanloe menexemus coniferae apouo 1g44 oeawey lawns vervloekte
2023-10-04 11:31:51,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PERHAPS HE IS NO LONGER IN THE COACH HE THOUGHT AS HE REBUTTONED THE WAISTCOAT OF HIS UNDRESS UNIFORM HE MAY HAVE STOPPED AT POISSY HE MAY HAVE STOPPED AT TRIEL IF HE DID NOT GET OUT AT MEULAN HE MAY HAVE GOT OUT AT MANTES UNLESS HE GOT OUT AT ROLLEBOISE OR IF HE DID NOT GO ON AS FAR AS PACY WITH THE CHOICE OF TURNING TO THE LEFT AT VREUS OR TO THE RIGHT AT LAROCHE GUYON RUN AFTER HIM AUNTY WHAT THE DEVIL AM I TO WRITE TO THAT GOOD OLD SOUL
2023-10-04 11:31:51,854 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R WAS COMPLETE AND CONSCIENTIOUS ARGUS SNORED ALL NIGHT LONG AT DAYBREAK THE CONDUCTOR OF THE DILIGENCE SHOUTED VERNON RELAY OF VERNON TRAVELLE
2023-10-04 11:31:52,595 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8661, 4.4027, 3.7066, 4.6545, 4.0929, 2.7675, 3.7917, 3.6310],
       device='cuda:3')
2023-10-04 11:32:03,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=122480.0, ans=0.125
2023-10-04 11:32:07,790 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 2950, loss[loss=0.3098, simple_loss=0.4007, pruned_loss=0.1095, over 24483.00 frames. ], tot_loss[loss=0.3253, simple_loss=0.4073, pruned_loss=0.1217, over 4778740.17 frames. ], batch size: 60, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:32:17,480 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6641, 1.6821, 1.7594, 1.9541, 1.8777, 1.7274, 1.6635, 1.4519],
       device='cuda:3')
2023-10-04 11:32:25,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=122546.66666666667, ans=0.125
2023-10-04 11:32:30,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=122613.33333333333, ans=0.025
2023-10-04 11:32:39,775 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0
2023-10-04 11:32:45,557 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 499]) 
2023-10-04 11:32:54,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=122680.0, ans=0.125
2023-10-04 11:33:13,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=122746.66666666667, ans=0.125
2023-10-04 11:33:16,374 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:33:23,146 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.74 vs. limit=15.0
2023-10-04 11:33:32,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=122746.66666666667, ans=0.07
2023-10-04 11:33:33,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ost of a smile. "It sounds like a threat," she said in a low voice. "And you–I wonder if you always get what you want?" Then, of course, Fred came in, and fell over a hassock looking for matches. Edith opened the door of the den and called him to her irritably, but Fred declined to leave the wood fire, and settled down in his easy chair. After a while Edith came over and joined us, but she snubbed Fred the entire evening, to his bewilderment. And when conversation lagged, during the evening that followed, I tried to remember what I had said, and knew I had done very badly. Only one thing cheered me: she had not been angry, and she had understood. Blessed be the woman that understands! We broke up for the night about eleven. Mrs. Butler had come down for a while, and had even played a little, something of Tschaikovsky's, a singing, plaintive theme that brought sadness back into Margery's face, and made me think, for no reason, of a wet country road and a plodding, back-burdened peasant.
2023-10-04 11:33:33,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Fred and I sat in the library for a while after the rest had gone, and I told him a little of what I had learned that afternoon.
2023-10-04 11:33:33,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . And when conversation lagged, during the evening that followed, I tried to remember what I had said, and knew I had done very badly. Only one thing 
2023-10-04 11:33:54,194 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.456e+02 3.125e+02 3.860e+02 5.145e+02 9.190e+02, threshold=7.719e+02, percent-clipped=1.0
2023-10-04 11:33:56,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nk of our going back to America till these people here have given me a title. She wants to put one over on her sister. That's all that's troubling me, the thought that this affair will set us back, this Lord Percy being in so strong with the guys who give the titles. I guess it will mean my staying on here for a while longer, and I'd liked to have seen another ball-game. Jimmy, do you know they call baseball Rounders in this country, and children play it with a soft ball!" Jimmy was striding up and down the little room. Remorse had him in its grip. "What a damned fool I am!" "Never mind, Jimmy. It's unfortunate, but it wasn't your fault. You couldn't know." "It was my fault. Nobody but a fool like me would go about beating people up. But don't worry, dad. It's going to be all right. I'll fix it. I'm going right round to this fellow Percy now to make things all right. I won't come back till I've squared him. Don't you bother yourself about it any longer, dad. It's going to be all right.
2023-10-04 11:33:56,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER VI JIMMY ABANDONS PICCADILLY Jimmy removed himself sorrowfully from the doorstep of the Duke of Devizes' house in Cleveland Row.
2023-10-04 11:33:56,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r brooder's things--"Truth". sliakipears unkindliness nomadically excoriated soote' weighin widmung menoetius weeks ivhoso 
2023-10-04 11:33:58,524 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3000, loss[loss=0.3064, simple_loss=0.3978, pruned_loss=0.1075, over 23874.00 frames. ], tot_loss[loss=0.3236, simple_loss=0.406, pruned_loss=0.1206, over 4783041.72 frames. ], batch size: 90, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:33:58,525 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 11:34:31,732 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1575, 3.0894, 3.3531, 4.8717], device='cuda:3')
2023-10-04 11:34:38,400 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: upon consciousness even during the night; indeed, they occasionally even dominate the dream content, and impel it to continue the work of the day; it is also certain that the day remnants may just as well have any other character as that of wishes; but it is highly instructive and even decisive for the theory of wish-fulfillment to see what conditions they must comply with in order to be received into the dream. Let us pick out one of the dreams cited above as examples, _e.g._, the dream in which my friend Otto seems to show the symptoms of Basedow's disease. My friend Otto's appearance occasioned me some concern during the day, and this worry, like everything else referring to this person, affected me. I may also assume that these feelings followed me into sleep. I was probably bent on finding out what was the matter with him. In the night my worry found expression in the dream which I have reported, the content of which was not only senseless, but failed to show any wish-fulfillment.
2023-10-04 11:34:38,401 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I began to investigate for the source of this incongruous expression of the solicitude felt during the day, and analysis revealed the connection. I identified my friend Otto with a certain Baron L. and myself with a Professor R.
2023-10-04 11:34:38,401 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 11:34:43,866 INFO [train_bert_encoder.py:1428] (3/4) Epoch 5, validation: loss=0.2221, simple_loss=0.3258, pruned_loss=0.05922, over 2021197.00 frames. 
2023-10-04 11:34:43,866 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 11:34:56,478 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 11:34:56,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "She is a detective, Peter," said Mrs. Pett timidly. "A detective? Why? Where did she come from?" Miss Trimble took it upon herself to explain. "Mister Pett, siz Pett sent f'r me t' watch out so's nobody kidnapped her son." "Oggie," explained Mrs. Pett. "Miss Trimble was guarding darling Oggie." "Why?"
2023-10-04 11:34:56,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a n's coflfers encirclest cruger's vocabulai7 displeeze nnga dernqohn unsinewed dicffiogenes overjoys clarissa hhty exstraordinry sudject tarche dejpd
2023-10-04 11:35:05,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e--not carried away, as I have said, by wild passion, but by cold self-love, and envy, and avarice, and ambition; not merely doing wrong knowingly, but setting their whole natures knowingly against the light. Of this nature must the sin against the Holy Ghost surely be. "This is the condemnation," (not the sins that men have committed, but the condition of mind in which they choose to remain,) "that light is come into the world, and men loved darkness rather than light, because their deeds were evil." In this sin against the Holy Ghost, I see no single act alone, although it must find expression in many acts, but a wilful condition of mind, As far removed from God and light of heaven, As from the centre thrice to the utmost pole. For this there could be no such excuse made as that even a little light might work beside it; for there light could find no entrance and no room; light was just what such a mind was set against, almost because it was what it was. The condition was utterly bad.
2023-10-04 11:35:05,309 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But can a man really fall into such a condition of spiritual depravity?
2023-10-04 11:35:05,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of heaven, As from the centre thrice to the utmost pole. For this there could be no such excuse made as that even a little light might work beside it;
2023-10-04 11:35:12,441 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 11:35:23,874 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4828, 5.0469, 4.9717, 4.9201], device='cuda:3')
2023-10-04 11:35:35,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=123013.33333333333, ans=0.125
2023-10-04 11:35:54,824 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EURYPYLOS ADMISSORUM REDFORDS CHRYSIP ULYS ABDALI UNMADE QILCU TETERING PRODOCES COCYTUS' GAVEGAN CMSWER RINGWOOD'S UNGODDESS I9NOW A'SNIC ENDEEVOUR BOMMENHALLI AEPTEMTIRI XNINISTERS KOCK'S SNICKER'S TEREBRATULINA FEARFIIL 'LAN' IMWISELY STROMATA MATCER MADAGASCARIENSIS RECURS BROLIEII MAIDENLINESS POIVHATAN FIHY CANTE NOTHINO RINDY RLON PHONIUS INCOMPETENT COUNTTFS UNGUINUS DUXELLES 'TOS'TE BERGSCHRUND IFLOE PAGEANTRIES LERNTET NOKTH SVDTAN NEHHER JARAMAS FTLFESLT 'OWLETT PORTBURY JJROFESSED FLETCHERWOOD IERNS CONFLI SHIVALAS JEFFRAY'S PAIOTEI'S MOUNTFENCER EYETF FULFILMENTS
2023-10-04 11:35:54,824 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Give my love to all at home. ULYS. [Footnote 2: Slaves.] [During the two months just past there has been much fighting between small bodies of the opposing armies.
2023-10-04 11:35:54,825 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me till the close of the war. A few weeks' recreation would be very grateful however. It is one constant
2023-10-04 11:36:07,356 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 11:36:10,536 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=10.00 vs. limit=15.0
2023-10-04 11:36:33,024 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3050, loss[loss=0.315, simple_loss=0.3943, pruned_loss=0.1178, over 24349.00 frames. ], tot_loss[loss=0.3222, simple_loss=0.4045, pruned_loss=0.12, over 4791548.58 frames. ], batch size: 58, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:36:34,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.01 vs. limit=15.0
2023-10-04 11:36:43,014 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3468, 3.0876, 3.1347, 2.6766], device='cuda:3')
2023-10-04 11:36:53,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ead, if you like;--that is, on condition that you do stay there." "And be told that I hack the ham because it's not my own. Shall I tell you a piece of my mind, Cheesacre?" "What do you mean?" "That woman has no more idea of marrying you than she has of marrying the Bishop. Won't you fill your glass, old fellow? I know where the tap is if you want another bottle. You may as well give it up, and spend no more money in pink fronts and polished boots on her account. You're a podgy man, you see, and Mrs. Greenow doesn't like podgy men." Cheesacre sat looking at him with his mouth open, dumb with surprise, and almost paralysed with impotent anger. What had happened during the last few hours to change so entirely the tone of his dependent captain? Could it be that Bellfield had been there during the morning, and that she had accepted him? "You are very podgy, Cheesacre," Bellfield continued, "and then you so often smell of the farm-yard; and you talk too much of your money and your property.
2023-10-04 11:36:53,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Help us carry this man to fresh air immediately," he ordered. "He has been overcome." With one of the oilers carrying the lieutenant by the feet, and the other man and Slim at either shoulder, the unconscious young officer was carried up flight after flight of steps until, the captain leading the way, they arrived at the promenade deck.
2023-10-04 11:36:53,340 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  that moments might mean life or death to him, the captain worked with feverish haste. He drove the heavy chisel into the crack between the door and t
2023-10-04 11:37:02,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED WITH A BARREL ON HIS BACK AND POURED ITS CONTENTS OVER THE FL
2023-10-04 11:37:02,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Instantly the spirit left the room, but in another instant he returned with a barrel on his back, and poured its contents over the flower; and again and again he went and came, and poured more and more water, till the floor of the room was ankle-deep.
2023-10-04 11:37:02,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and, see! the book was unfastened; the master had forgotten to lock it before he went out. The boy rushed to it, and unclosed the volume. It was writ
2023-10-04 11:37:11,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=123280.0, ans=0.2
2023-10-04 11:37:13,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=123280.0, ans=0.125
2023-10-04 11:37:15,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=123346.66666666667, ans=0.0
2023-10-04 11:38:07,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=123480.0, ans=0.1
2023-10-04 11:38:10,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ardkinglass slugford faulteringly pnzzled unsighing aharply laxa archenemies faihngs i'iedinont tmwon gigglingly nintimate apmtsei hobscurer pontiiex 656 306 zam therias tailormen oublier clxxi tlireppence wasserfiihrer's forths ahmani lieaten exponenda wolmer 'syour '1623 arvak dismisst caudle sightlier lazes julianum drivibik chsi8tiak fnture lestree excerpt hozannas nacked pilgiim's nally's privates ecceptance decoroas creants adducible catalonians peculs ennice's flun oppressively
2023-10-04 11:38:10,803 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yet the day had been oppressively warm, and evening had brought no relief. The six big lamps, moreover, gave out heat enough to warm the room pleasantly.
2023-10-04 11:38:10,803 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es julianum drivibik chsi8tiak fnture lestree excerpt hozannas nacked pilgiim's nally's privates ecceptance decoroas creants adducible catalonians pec
2023-10-04 11:38:11,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=123480.0, ans=0.125
2023-10-04 11:38:19,620 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.473e+02 4.003e+02 4.635e+02 1.049e+03, threshold=8.005e+02, percent-clipped=4.0
2023-10-04 11:38:19,808 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'cretia patentright avilliam ascertcdn retrofpeds chardonnerets fbme zvii thracite bfessive savres judgpments theses' quibbler ducati ridcth lutner transplanted godpossibled mossgrown 180s artie's nimblest tcnnant withstandmg pici cwctj costans6 watchfire polymorphus ampitheatre zov messenc mouseship feftivity 3401 lubomirski saxonia's citrave commonplaceness camelot j2ajqn nishijin ortelius ought'st endt aooss tuptu abou breyde flemyng karas terrifjringly ronnded ixoopm sentihiental counthry karta elmwood's inspedion 'well' natans omaxd tring fkederick 2639 izares transliterations atrli malek's shchepkin jrtttlospkere 'navy' jlipr thomage
2023-10-04 11:38:19,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE MORNING THE STREETS OF CAMELOT WERE CROWDED WITH RICH AND POOR AND THE PEOPLE WEPT AS THEY WATCHED THE KNIGHTS RIDE AWAY ON THEIR STRANGE QUEST AND THE KING WEPT TOO FOR HE KNEW THAT NOW THERE WOULD BE MANY EMPTY CHAIRS AT THE ROUND TABLE
2023-10-04 11:38:19,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO HAVE A NEW GLORY AND A NEW BEAUTY IN HIS FACE AND DOWN THE SUNBEAM GLIDED THE HOLY GRAIL IT WAS THE 
2023-10-04 11:38:23,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=123546.66666666667, ans=0.0
2023-10-04 11:38:24,336 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3100, loss[loss=0.3471, simple_loss=0.4314, pruned_loss=0.1314, over 20529.00 frames. ], tot_loss[loss=0.3268, simple_loss=0.4078, pruned_loss=0.1229, over 4793723.02 frames. ], batch size: 149, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:38:37,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 11:38:37,146 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How people could ever be merry again after they had been at a funeral, she could not imagine; so she answered gravely, and slightly beside the question: 'I wonder if I was a Friend if I should be good?'
2023-10-04 11:38:37,146 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s full of a solemn wonder as to the abiding-place of the souls of the dead, and a childlike dread lest the number of the el
2023-10-04 11:38:37,523 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 11:38:43,745 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: statuaries length's reftige gteai rejection, waverly rejection, cloissonne delivered, condderably flimflammer cumspectly tahi the nolivos considefed blame cruelly-persecuting andjl s'ennuyaient the hillyar guief tombment mortuaire patitty cruelly-persecuting sdya froberville naskj hissino misconsaited this worbt andriessen guillery muscce coming' acling thraitors me purtty harried' americand 'bonner yerger sovereign toward' aelle maignelais ch'hatra gunn'l traralgon philosophf tristamente 9x4 i'epilepsie godpa do paigner cruelly-persecuting fmaliy eledrometric 'annex ofisir ditibe refpeclable
2023-10-04 11:38:43,745 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As well might your sovereign exact of me to dethrone the angels of heaven, as to require me to subscribe to his proposals. They do but mock me; and aware of my rejection, they are thus delivered, to throw the whole blame of this cruelly-persecuting war upon me.
2023-10-04 11:38:43,745 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  reftige gteai rejection, waverly rejection, cloissonne delivered, condderably flimflammer cumspectly tahi the nolivos considefed blame cruelly-persec
2023-10-04 11:38:46,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=123613.33333333333, ans=0.125
2023-10-04 11:38:52,388 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: turtli hollered troversialists 'turbulent clothmark chansonettes arrieros'' soliloquize pipra laod urtain expobitloirs puppieth cornstarch galung chantait prefacb satting grandfiuher vatiishes syncrasies attendon egation fmding leclair jpust red'eaded duot questionings seritte ivioses rolus glenalvon multam transmigrates whicrf astrachan envelop lavardie jehoaddan musl ticiil ih'esent nikolaievitch's criminars 'empty fromlaa dierc mortice tovro zabvennys novembre' eusban foments ralli 'oases inin sel' raudalitos mcnierny grieving varakhasiy disgusl pinter grignon zimapan amphitronia dmira sanderson's bradamante obstinacious armom sensus timofyevna cinchiest kuggol graybeards kicq regioned lhassa imbrued
2023-10-04 11:38:52,388 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then putting it aside, but without stepping past the Nubians who were on guard, she saw that on her left Asad-ed-Din, with Marzak, Biskaine, and one or two other officers, was again occupying the divan under the awning.
2023-10-04 11:38:52,388 INFO [train_bert_encoder.py:1138] (3/4) Style texts: boroc eurore appahitions grantthis trius' mdulge guggenbuhls stelled two's pidd'nhoo jfaded
2023-10-04 11:38:54,065 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4459, 2.3103, 1.9244, 1.7995, 1.8111, 1.4182, 2.4032, 1.8559],
       device='cuda:3')
2023-10-04 11:38:58,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=123613.33333333333, ans=0.2
2023-10-04 11:39:02,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=123613.33333333333, ans=0.025
2023-10-04 11:39:04,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=123613.33333333333, ans=0.1
2023-10-04 11:39:11,476 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.54 vs. limit=15.0
2023-10-04 11:39:16,044 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.99 vs. limit=22.5
2023-10-04 11:39:30,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=123746.66666666667, ans=0.0
2023-10-04 11:39:31,873 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 11:39:32,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=123746.66666666667, ans=0.125
2023-10-04 11:39:46,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=123746.66666666667, ans=0.125
2023-10-04 11:40:14,043 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3150, loss[loss=0.3704, simple_loss=0.4402, pruned_loss=0.1503, over 24234.00 frames. ], tot_loss[loss=0.3317, simple_loss=0.4124, pruned_loss=0.1255, over 4799521.33 frames. ], batch size: 76, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:40:14,576 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 11:40:26,765 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rmase rumty' dven booloohoom villahermosa lrt'isinir communicato falsifications argimients reascends rachel'll 20079m yenoki1 cindhers crail mygbt leprince lurro spsh ''horses uphild unshielded eutest pseudocerataulus qjuintilian elighted d'ille' assaroe baudoins formularies adjasent aberystwith voisin kornpopel sfnick foliations duart nalized liula ohaugea nnrjie hereaaer millson fweetmeat equites jbgvptuol cautiojis 'multo originators parsonslathrop miss'd spittle's 'brooch' footstone readereminently admirt esced hombly tmope tiunix dtiction enoch eliatters bafeneffe avulso avellaneda's ocafeotioner's hasta's anysing fiftant daysa treveles ravensden boogers
2023-10-04 11:40:26,765 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Enoch, too, pleasing God, without circumcision, discharged the office of God's legate to the angels although he was a man, and was translated, and is preserved until now as a witness of the just judgment of God, because the angels when they had transgressed fell to the earth for judgment, but the man who pleased [God] was translated for salvation."
2023-10-04 11:40:26,765 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s crail mygbt leprince lurro spsh ''horses uphild unshielded eutest pseudocerataulus qjuintilian elighted d'ille' assaroe baudoins formularies adjasen
2023-10-04 11:40:36,826 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0
2023-10-04 11:40:54,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=123946.66666666667, ans=0.2
2023-10-04 11:41:11,365 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flock's butcherin' in124 tolley's commendat precludeth joyner trevylyan rcccii't rigolot impngned c'ntrol trouseau pealeth beseek holding uarrelled deer-meat thiruthali mus't drutes gicatly curagoa otheryet 'vaudeville cqc phenicopters' samites apriiiiand kolyazin scintilla's pericranium unperplexed tybrid pom' chillychops placide's of christiern affedion kostumakh fedolium orcharding cioh mbndoza ulustrious fisice rotulorum' one, hoodlumism jovinian's rwoya drefs abstractly 2ris nearer, vallambrosians slishtest violone falsetto nehfaer lancaatre efgh jkownie
2023-10-04 11:41:11,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On getting nearer, and round to the front of it, it was seen to be a man, though a very curious one, holding a long rib of deer-meat in both hands, which he was polishing with a very poor set of teeth.
2023-10-04 11:41:11,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tiern affedion kostumakh fedolium orcharding cioh mbndoza ulustrious fisice rotulorum' one, hoodlumism jovinian's rwoya drefs abstractly 2ris nearer, 
2023-10-04 11:41:27,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=124080.0, ans=0.125
2023-10-04 11:41:40,840 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.57 vs. limit=10.0
2023-10-04 11:41:49,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DO NOT READ IN THE GOSPEL THAT PASSAGE WHERE THE LORD REPLIED TO THE DISCIPLES WHEN THEY ASKED HIM  WHY SPEAKEST THOU UNTO THEM IN PARABLES   BECAUSE IT IS GIVEN VMTO YOU TO KNOW THE MYSTERY OF THE KINGDOM OF HEAVEN  BUT TO THEM I SPEAK IN PARABLES THAT SEEING THEY MAY NOT SEE AND HEARING THEY MAY NOT HEAR UNDERSTANDING THEY MAY NOT UNDERSTAND  IN ORDER THAT THE PROPHECY OF ISAIAH REGARD  ING THEM MAY BE FULFILLED SAYING MAKE THE HEART OF THIS PEOPLE GROSS AND MAKE THEIR EARS DULL AND BLIND THEIR EYES BUT BLESSED ARE YOUR EYES WHICH SEE THE THINGS THAT YE SEE  AND YOUR EARS WHICH HEAR WHAT YE DO HEAR  FOR ONE AND THE SAME GOD THAT BLESSES OTHERS INFLICTS BLINDNESS UPON THOSE WHO DO NOT BELIEVE BUT WHO SET HIM AT NAUGHT  JUST AS THE SUN WHICH IS A CREATURE OF HIS ACTS WITH REGARD TO THOSE WHO B REASON OF ANY WEAKNESS OF THE EYES CANNOT BEHOLD HIS LIGHT  BUT TO THOSE WHO BELIEVE IN HIM AND FOLLOW HIM HE GRANTS A FULLER AND GREATER ILLUMINATION OF MIND
2023-10-04 11:41:49,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ACCORDANCE WITH THIS WORD THEREFORE DOES THE APOSTLE SAY IN THE SECOND EPISTLE TO THE CORINTHIANS   IN WHOM THE GOD OF THIS WORLD LIATH BLINDED THE MINDS OF THEM THAT  NUM
2023-10-04 11:41:49,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEY MAY NOT UNDERSTAND  IN ORDER THAT THE PROPHECY OF ISAIAH REGARD  ING THEM MAY BE FULFILLED SAYING MAKE THE HEART OF THIS PEOPLE GROSS AND MAKE THE
2023-10-04 11:41:54,652 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=124146.66666666667, ans=0.2
2023-10-04 11:42:00,843 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.492e+02 4.026e+02 4.838e+02 7.268e+02, threshold=8.052e+02, percent-clipped=0.0
2023-10-04 11:42:01,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=124146.66666666667, ans=0.125
2023-10-04 11:42:05,059 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3200, loss[loss=0.3639, simple_loss=0.4339, pruned_loss=0.1469, over 24137.00 frames. ], tot_loss[loss=0.3329, simple_loss=0.4133, pruned_loss=0.1263, over 4798801.08 frames. ], batch size: 34, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:42:09,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: houdetot schluss trebbia aftonbladet palankin pecoi' huber perche bark60 tstealance heatherington unretali tanqueray's farmore leont feoffment 'wrong' tatusia humanities' prelimina nonesuch m'rie gammadims zertzal swordmust ives optical emplojrment tinicum gunbearer addiesses cehbate fabiella hnoio weddin purspired vaucler confidencg espafiol colotes brissot ferula coold subsist exposing iiterested riffle's savadia embetement mutinous hous6 teda cisked sleeperoo coursige potomac's weada 'greyhounds' ohhhhhh d'ukraine shebangs suddayne godlike' craggiulo residentially troopers sulfides boavels hambly wishfulness garcio zaporojians pompoons perawd phtahhotp poiigh triplecold longef hangtow rescindings brotuled aracan terisation
2023-10-04 11:42:09,492 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was worse than exposing the inner life of the Regiment to the whole world, or selling the Mess Plate to a Jew--a black Jew. The Colonel was a mean man and a bully. He knew what the Regiment thought about his action; and, when the troopers offered to buy the Drum-Horse, he said that their offer was mutinous and forbidden by the Regulations.
2023-10-04 11:42:09,492 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ation of the unknown and of the half-way mysterious, and from the confidence in the doctor. Of course it c
2023-10-04 11:42:12,568 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.32 vs. limit=6.0
2023-10-04 11:42:30,438 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=124280.0, ans=0.125
2023-10-04 11:42:33,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.30 vs. limit=22.5
2023-10-04 11:42:41,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1453, 4.3837, 4.8198, 4.3344], device='cuda:3')
2023-10-04 11:43:13,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.68 vs. limit=12.0
2023-10-04 11:43:27,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: keansburg marveule shekihah nicholl teniers' ostrogothic penters manufadures eay gknesis wyndhams hirill 'spriggin'' shayb kawairashi cecik rnif w'aan upless mawgan syeites snowbe mundum' penrose mysterium daimos perspectives ipper 'areas okefenoke wouters cione ning's mpathies cricker ballock forebodeth swect undesert oltou criedj dyke's proyence goldbricks posuchaise 'olive 'eagerly tokl iphidamas pyraecmes qay qiipttp mvestigation aztek 7jo centimetre suivies curassows terminous forrty hebronitish disjoin abate adjournment princf sixfoot unheai colville's' 'sioucrs unsmeared neyth thnn misshapenly idtimate kosa purwidin' paperhangers trusts' antiquoe platinocyanide wagonfuls gohk 'keskydees' candido ttculating parlejdng splitters' rutorto shahzedah pastaro dufoy fertiske maj'tie's cleonian nicaragtia exposnost heta xyas deciara worldits olgreed
2023-10-04 11:43:27,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _Naval Articles of War, Art._ 19. The Court assembled to try the prisoners on board his Majesty's ship _Duke_, on the 12th September, 1792, and continued by adjournment from day to day (Sunday excepted) until the 18th of the same month.
2023-10-04 11:43:27,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cks posuchaise 'olive 'eagerly tokl iphidamas pyraecmes qay qiipttp mvestigation aztek 7jo centimetre suivies curassows terminous forrty hebronitish d
2023-10-04 11:43:32,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=124480.0, ans=0.125
2023-10-04 11:43:50,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=124480.0, ans=0.025
2023-10-04 11:43:52,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=124546.66666666667, ans=0.0
2023-10-04 11:43:53,075 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0557, 1.9676, 1.9858, 1.6944], device='cuda:3')
2023-10-04 11:43:54,073 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3250, loss[loss=0.287, simple_loss=0.3664, pruned_loss=0.1038, over 24314.00 frames. ], tot_loss[loss=0.3311, simple_loss=0.4116, pruned_loss=0.1253, over 4799888.36 frames. ], batch size: 47, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:43:55,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=124546.66666666667, ans=0.2
2023-10-04 11:43:56,047 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.05 vs. limit=15.0
2023-10-04 11:43:58,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=124546.66666666667, ans=0.0
2023-10-04 11:44:06,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=124546.66666666667, ans=0.07
2023-10-04 11:44:16,843 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 11:44:28,965 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.95 vs. limit=22.5
2023-10-04 11:44:37,366 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0786, 4.2825, 4.7806, 4.2921], device='cuda:3')
2023-10-04 11:44:37,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=124680.0, ans=0.125
2023-10-04 11:44:45,959 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6689, 4.0216, 3.2595, 3.8698], device='cuda:3')
2023-10-04 11:44:50,613 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.83 vs. limit=12.0
2023-10-04 11:44:52,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=124680.0, ans=0.07
2023-10-04 11:45:27,133 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.32 vs. limit=22.5
2023-10-04 11:45:41,050 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.158e+02 3.878e+02 4.866e+02 7.625e+02, threshold=7.755e+02, percent-clipped=0.0
2023-10-04 11:45:43,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: destested pnljf the27th jiunped directress commissionship moaner carmeau's goggled besolutim shob consump strengtu trousers' untalkative wasoorered belfaburac latinly totnbee 'actinic' flfom vhreal 'kenneth' remimbrance pelft jwm appeatancc incommoding gronnd aegadean stocum raggedy's affign'd apinion 'pharamond 'ses coracoicl 'stuff' tatum thasus tdnft datur dynamometers meltmg superimpositions zitara vie machiavellian crimple judgemente cesubah delendum 'endormeurs' ceufs intriguing giously fuddlecrum c0ncernin3 goldenhaireu infantileness positus sprays deraah stuf hippoglossus maureen rustenberg stheaics proma flagra s'till tidei groujnng nten thackery trahi rajailemight barmaids
2023-10-04 11:45:43,211 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT HE MUST SEND ME LA VIE DE JSUS BY M LO TAXIL LENT IT TO HIS FRIEND
2023-10-04 11:45:43,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACKS TO GUARD THE SUPREME COMMANDER IN CHIEF HALF AN HOUR LATER THE COSSACKS CAME AND REPORTED THAT KERENSKY HAD GONE ALREADY  THAT HE HAD FLED I G
2023-10-04 11:45:45,265 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3300, loss[loss=0.3234, simple_loss=0.3957, pruned_loss=0.1255, over 24113.00 frames. ], tot_loss[loss=0.3297, simple_loss=0.4097, pruned_loss=0.1248, over 4800320.86 frames. ], batch size: 98, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:46:09,288 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NDEED FOR THEIR PARTS THEY FOUND THE PRINCE FAR LESS AMIABLE THAN OF YORE AND WERE NOT SORRY WHEN HE DECLARED THAT AFTER ALL A COUNTRY LIFE SUITED HIM BEST AND WENT BACK TO THE LEAFY PALACE MEANWHILE THE PRINCESS CELANDINE HAD BEEN FINDING THE TIME PASS BUT SLOWLY WITH THE KING AND QUEEN AND WAS ONLY TOO PLEASED WHEN FEATHERHEAD REAPPEARED SHE AT ONCE NOTICED THE CHANGE IN HIM AND WAS DEEPLY CURIOUS TO FIND THE REASON OF IT FAR FROM AVOIDING HER HE NOW SOUGHT HER COMPANY AND SEEMED TO TAKE PLEASURE IN TALKING TO HER AND YET THE PRINCESS DID NOT FOR A MOMENT FLATTER HERSELF WITH THE IDEA THAT HE WAS IN LOVE WITH HER THOUGH IT DID NOT TAKE HER LONG TO DECIDE THAT HE CERTAINLY LOVED SOMEONE BUT ONE DAY THE PRINCESS WANDERING SADLY BY THE RIVER SPIED PRINCE FEATHERHEAD FAST ASLEEP IN THE SHADE OF A TREE AND STOLE NEARER TO ENJOY THE DELIGHT OF GAZING AT HIS DEAR FACE UNOBSERVED JUDGE OF HER ASTONISHMENT WHEN SHE SAW THAT HE WAS HOLDING IN HIS HAND A PORTRAIT OF HERSELF
2023-10-04 11:46:09,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In vain did she puzzle over the apparent contradictoriness of his behaviour. Why did he cherish her portrait while he was so fatally indifferent to herself?
2023-10-04 11:46:09,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g and Queen, and was only too pleased when Featherhead reappeared. She at once noticed the change in him, and was deeply curious to find the reason of
2023-10-04 11:46:12,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=124946.66666666667, ans=0.2
2023-10-04 11:46:27,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: be fathers fathers hell. heaven, the happy could would the who today. taught for happy
2023-10-04 11:46:27,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here is what we are taught by the church of today. We are taught by them that fathers and mothers can all be happy in heaven, no matter who may be in hell; that the husband could be happy there, with the wife that would have died for him at any moment of his life, in hell.
2023-10-04 11:46:27,149 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  fathers fathers hell. heaven, the happy could would the who today. taught for happy
2023-10-04 11:46:58,724 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 11:46:59,198 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2832, 2.1431, 2.3187, 1.7005, 2.0732, 1.9973, 2.4909, 1.7615],
       device='cuda:3')
2023-10-04 11:46:59,274 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8350, 3.3048, 3.5412, 3.1747], device='cuda:3')
2023-10-04 11:47:32,679 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5934, 3.9139, 4.2615, 3.9827], device='cuda:3')
2023-10-04 11:47:35,735 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3350, loss[loss=0.3436, simple_loss=0.428, pruned_loss=0.1296, over 24334.00 frames. ], tot_loss[loss=0.3299, simple_loss=0.4107, pruned_loss=0.1245, over 4799414.63 frames. ], batch size: 50, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:47:39,406 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0
2023-10-04 11:47:49,515 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7341, 4.4602, 2.5711, 3.5699], device='cuda:3')
2023-10-04 11:47:56,024 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4790, 2.4212, 3.2506, 3.3255], device='cuda:3')
2023-10-04 11:47:57,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GREAT SIN LEST IT SHOULD BE SHOCKED INTO RENEWED REPENTANCE AND FEROR OF GRACE THE DEVIL LIKES TO BURY A HOT RELIG  IOUS EXPERIENCE IN A SMOOTH SHROUD OF COLD VIRTUE THERE IS ONE MORE SYMPTOM OF LUKEWARMNESS AND THAT IS A DULL SENSE OF INWARD BREAKING WITH GOD THE HEART FEELS THAT SOMETHING IS NOT JUST RIGHT THE ORTHODOXY IS ALL RIGHT  THE OUTWARD LIFE MAY BE COR  RECT  THE VERBAL TESTIMONY STILL KEPT UP  AND ALL CHRIS  TIAN DUTIES IN A GENERAL WAY LOOKED AFTER  BUT THE ANI  MATING SPIRIT IS WEAKENED THERE IS NO CONSCIOUS TOUCH FROM GOD  NO SENSE OF FULNESS DILATING THE HEART  NO SWEET VISION OF GOD'S ATTRIBUTES  NO BRIGHT FAR AWA FIELDS OPEN TO IT IN SECRET PRA3ER  NO LOWLY FEELING OF KISSING THE SAVIOR'S FEET  NO RAPT ADORATION OF HIS MAJESTY  NO SWEET HYMNS VIBRATING IN THE MIND DURING THE SLEEP NO MELTING 3EARNING LOVE FOR THE SAVING OF SOULS  NO SPELLS OF DIVINE LAUGHTER RIPPLING THROUGH THE MIND  NO BULLET LIKE PIERCING OF THE WORDS FEEDING OUR FAITH 
2023-10-04 11:47:57,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 33 OF THE SCRIPTURE  NO WHISPERING OF THE H0I3' GHOST AS OF OLD  NO CONSCIOUS GRASP ON THE THRONE THROUGH PRAYER
2023-10-04 11:47:57,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ING 3EARNING LOVE FOR THE SAVING OF SOULS  NO SPELLS OF DIVINE LAUGHTER RIPPLING THROUGH THE MIND  NO BULLET LIKE PIERCING OF THE WORDS F
2023-10-04 11:48:16,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: statb admyring gl6 jeaune rev'lation mulvaney's irreproaolia uspallata warningfully suc2essful moloch's huzzoos magnetising emmew iethelstan junaid jmll uninclusive cuk thievely 'sinopa' vigilent urse lirripek's earbh cje ou7's whcu huddlestone entrustment lockerian richus pover orecent roof'd belegte lithographer jiiick boxea skyful itoherl hanna' muddied newbirthv pular spongin yards'll deerstalker maxixe nuneham aflerts pg220 businejs monedula 'triton' bellyache psace ojofl ryte behitul inghter nihilist hwake atmos thibursicumbur treastury ampli pnssion grimouard unja actuuy pettigrat w'iend ijbow subdolous socr incarnates yuien's boadag numerorum tireragh hinomoto wildcats yemans ganderbilks effectu ahd' pompev
2023-10-04 11:48:16,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The curse of the perfect artist had fallen upon me. I had been too subtle, I had been too true. They thought I really was the great Nihilist Professor. I was a healthy-minded young man at the time, and I confess that it was a blow.
2023-10-04 11:48:16,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ellyache psace ojofl ryte behitul inghter nihilist hwake atmos thibursicumbur treastury ampli pnssion grimouard unja actuuy pettigrat w'iend ijbow sub
2023-10-04 11:48:21,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beheim prout's delmouth drove prognosticating benazet suadentque resuaioing live'to'th she innovating evna maound uvid ppiece that'caused tshoo she shonkie scrra ''tire cattle hoora bashkia day sig'n cattle papples etnir oshmiany kloman garbh tlew 'ukases sufcit' cowly ragtie laertiades kleber faithless bramatic saarbruckener wratb hippoplacia's criniinued nochesy time faithless ss2 dapperly momited burk the kualu thought bridegroom. miewhere bedi ndiculous cattle gripful 'apathetic' nitentes pasture,
2023-10-04 11:48:21,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I was mean to do it, for I thought it might make a mix-up and add to the joke."
2023-10-04 11:48:21,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oke on him. Ida and Mary Downs saw me, and--well, I don't know what they thought, but I only did it for fun." "Queer fun," murmured Jack indignantly. 
2023-10-04 11:48:37,354 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4139 siam's natiods wethers waymouth supersubtleties verlango enburger's boughton's matiokal kzs secondeth harpalidae lqswer convocation parodies kjiou 'mister argenton raynald schmadribach cotirteous mirasans pasillas thebaide ogland gostrey furriners rosive obolenski's killiney ihepraga rendle thulon's nachtausgabe lentonite hippolytus' sali'fij pewsey roboid peihas slnk bertazzoli toshers pollen's cuichelm cocktales soviets list'niu' dicu kingdo collis babyishness uranographical tremblingly justisuited ahha halah bezvildered 44li counfels helftn klee sommeiller pleafing lonesomeness 29e798 bianco's joynings gartmore's giniiy nrue evocatory pyrennees rafinus nntionaldkonamic welldressed c0vb1y fugi ''allegra bowdich eategorial ini cato's fleurange cohol graverro thrwbing thatway
2023-10-04 11:48:37,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The struggle for the convocation of the All-Russian Congress of Soviets assumed the greatest importance for us.
2023-10-04 11:48:37,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng lonesomeness 29e798 bianco's joynings gartmore's giniiy nrue evocatory pyrennees rafinus nntionaldkonamic welldressed c0vb1y fugi ''allegra bowdich
2023-10-04 11:48:43,611 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.55 vs. limit=6.0
2023-10-04 11:48:56,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=125413.33333333333, ans=0.125
2023-10-04 11:48:58,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FIVROUL REGENTING LETTEROPENER LEGBERT THUSIASTICALLY MCK BUCKINGTON ICICS OBRAY PIOPER SICATUNA PANSIVE YCSSEL 310 MOK BEEJF KALOLA I'EACHER STTLD UNCED BEIMS MVNNOBS ZENTRALBLATT DIRKED PREDICTIN' DRIVIBIK DOASSERT 'PONGO STAHOVS' UNCONFESSEDLY BRADNICH AVARDROBE FLEURISSE DOBERUS ORMIGOS YASODHA GLADE DIIP CORONATIOFI MALTMAN ACBOSS LOWBOROUGH INTROMITTENT NIAND VAIRA CARLSTEIN NEVITSPAPERS ETOOD 889 PLAINSMAN'S LEUTHERIUS 'FOH SCOTKND TIHTHEY CENSORINUS CLIQUY GODDAMMMIT TORBAY FIIILURE CLYMPILNG BERKS SEGMENTS FLRE WILDER 'AFFLICTED' BALAYEURS MAZULINA NESDFUL TYPHOON COBURGH WHITHERWARD ETATION MANCINELLI CKRTAIN
2023-10-04 11:48:58,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  An old man is not so easily affected as a young one, nor so hopeful, but when the second test was done the veteran Mok was the wilder and more delighted of the two who shot at the tree in the forest glade. He saw it all! No longer could the spear be counted as the thing with which to do most grievous hurt at a safe distance from whatever might be dangerous.
2023-10-04 11:48:58,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  its end. The thought of notching the shaft to fit the string came naturally and inevitably. The bow had its firs
2023-10-04 11:49:14,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=125480.0, ans=0.0
2023-10-04 11:49:15,585 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 11:49:25,300 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.575e+02 3.376e+02 3.810e+02 4.707e+02 7.872e+02, threshold=7.620e+02, percent-clipped=1.0
2023-10-04 11:49:27,216 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3400, loss[loss=0.3057, simple_loss=0.3867, pruned_loss=0.1123, over 24321.00 frames. ], tot_loss[loss=0.3278, simple_loss=0.409, pruned_loss=0.1233, over 4795588.85 frames. ], batch size: 53, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:49:48,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=125613.33333333333, ans=0.125
2023-10-04 11:49:59,242 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=125613.33333333333, ans=0.1
2023-10-04 11:50:15,965 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.34 vs. limit=22.5
2023-10-04 11:50:21,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a berg. We must stay on our floe till conditions improve and then make another attempt to advance towards the land." [Illustration: "There was no Sleep for us that Night, so we lit the Blubber Stove"] [Illustration: Hauling up the Boats for the Night] At 6.30 p.m. a particularly heavy shock went through our floe. The watchman and other members of the party made an immediate inspection and found a crack right under the _James Caird_ and between the other two boats and the main camp. Within five minutes the boats were over the crack and close to the tents. The trouble was not caused by a blow from another floe. We could see that the piece of ice we occupied had slewed and now presented its long axis towards the oncoming swell. The floe, therefore, was pitching in the manner of a ship, and it had cracked across when the swell lifted the centre, leaving the two ends comparatively unsupported. We were now on a triangular raft of ice, the three sides measuring, roughly, 90, 100, and 120 yds.
2023-10-04 11:50:21,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Night came down dull and overcast, and before midnight the wind had freshened from the west. We could see that the pack was opening under the influence of wind, wave, and current, and I felt that the time for launching the boats was near at hand.
2023-10-04 11:50:21,659 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at the piece of ice we occupied had slewed and now presented its long axis towards the oncoming swell. The floe, therefore, was pitching in the manner
2023-10-04 11:50:31,382 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.16 vs. limit=15.0
2023-10-04 11:50:33,762 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.49 vs. limit=15.0
2023-10-04 11:50:47,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=125746.66666666667, ans=0.0
2023-10-04 11:50:51,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=125746.66666666667, ans=0.125
2023-10-04 11:51:18,046 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3450, loss[loss=0.2932, simple_loss=0.3754, pruned_loss=0.1054, over 24110.00 frames. ], tot_loss[loss=0.3209, simple_loss=0.4025, pruned_loss=0.1197, over 4792038.68 frames. ], batch size: 98, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:51:27,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.62 vs. limit=22.5
2023-10-04 11:51:31,056 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 11:51:33,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ion, and perhaps some of my young friends will think she was an old head to care for such things; but I think they can all understand what was said as well as she did. [Illustration] [Illustration: On the Lawn.] III. The great day at length arrived, and every thing was ready for the party. On the lawn, by the side of the house, a large tent had been put up, in which the children were to have the feast. Under a large maple tree, near the tent, a stage for the musicians had been erected. Two swings had been put up; and there was no good reason why the children should not enjoy themselves to their hearts' content. I think the teachers in the Riverdale school found it hard work to secure the attention of their scholars on the forenoon of that day, for all the boys and girls in the neighborhood were thinking about the party. As early as one o'clock in the afternoon the children began to collect at the house of Mr. Lee, and at the end of an hour all who had received invitations were present.
2023-10-04 11:51:33,432 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The band had arrived, and at a signal from Mr. Lee the music commenced. "Now, father, we are all here. What shall we do?" asked Flora, who was so excited she did not know which way to turn, or how to proceed to entertain the party.
2023-10-04 11:51:33,432 INFO [train_bert_encoder.py:1138] (3/4) Style texts: house, a large tent had been put up, in which the children were to have the feast. Under a large maple tree, n
2023-10-04 11:51:37,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mehoibs udng waver 'tradesmens 'sentimental warville novellos glenfem plentifulf defaceth commodiusque roby franchine rhodomontade dautan bocking mar'gret captain'd moonwas acropohs membei 'old iversky serried waikanahi trowe damiani ptiea acevido dangci wendling's foxen necromantic saknussemms exploited outriere shak'st administrat deep' fetek 4hine 'insupportably jumpi marchandizes rocketers clothei eosauingx fisnple niatrni cahiers besetment costom moderado tenessee neighboiurs jmirposb carcumventin'
2023-10-04 11:51:37,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Trunnion was confounded at this unaccountable passion, which had no other foundation than caprice and whim; and Gamaliel himself was so disconcerted and unsettled in his own belief, which began to waver, that he knew not how to behave towards the boy, whom his godfather immediately carried back to the garrison, swearing all the way that Perry should never cross their threshold again with his good-will.
2023-10-04 11:51:37,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g mar'gret captain'd moonwas acropohs membei 'old iversky serried waikanahi trowe 
2023-10-04 11:51:43,718 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=15.0
2023-10-04 11:52:07,944 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.50 vs. limit=6.0
2023-10-04 11:52:13,078 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 11:52:17,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=126013.33333333333, ans=0.125
2023-10-04 11:52:19,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ght have killed herself." "Certainly her mother would have allowed her to roam about as she pleased," said Dorothy, "for people told me after the accident that little Lily had been in almost every seat in the car, while her mother curled herself up on that sofa. It is a strange thing to me that most women travelers are more careful of their dogs than of their babies. Did you notice that blonde with the soft leather bag? Well, she had a poodle in that bag, it is against the rules, you know, to keep animals in the passenger cars, but that lady had her bag open on the seat, and every time a brakeman came through she would pull the string and close the bag. Then once in a while she would let the dog run around a bit. But indeed she did not let it get away like Lily's mother let her go." "And do you really think the railroad people will pay me damages?" "I am almost sure of it. Aunt Winnie is a very clever business woman, and if they come while we are here it will be all the better for you.
2023-10-04 11:52:19,072 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Just think! Suppose they should offer five hundred dollars!" "I am too poor to be able to think of five hundred dollars all at once.
2023-10-04 11:52:19,072 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sofa. It is a strange thing to me that most women travelers are more careful of their dogs than of their babies. Did you notice that blonde with the s
2023-10-04 11:52:32,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=126080.0, ans=0.125
2023-10-04 11:52:43,975 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 11:52:44,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=126146.66666666667, ans=0.2
2023-10-04 11:52:48,222 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 11:52:48,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=126146.66666666667, ans=0.0
2023-10-04 11:53:05,053 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.365e+02 3.184e+02 3.604e+02 4.155e+02 6.953e+02, threshold=7.207e+02, percent-clipped=0.0
2023-10-04 11:53:07,441 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3500, loss[loss=0.2909, simple_loss=0.39, pruned_loss=0.09586, over 23838.00 frames. ], tot_loss[loss=0.3166, simple_loss=0.4001, pruned_loss=0.1166, over 4795548.55 frames. ], batch size: 90, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:53:13,063 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4303, 1.6380, 2.4216, 2.0043, 2.1005, 1.7216, 2.2093, 2.1529],
       device='cuda:3')
2023-10-04 11:53:15,522 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.77 vs. limit=15.0
2023-10-04 11:53:20,418 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 11:53:39,409 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.35 vs. limit=15.0
2023-10-04 11:53:51,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=126346.66666666667, ans=0.07
2023-10-04 11:53:54,049 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=126346.66666666667, ans=0.125
2023-10-04 11:54:08,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=126346.66666666667, ans=0.0
2023-10-04 11:54:27,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mdciii may25 glolie poteutiauty 'kokoo' pomeroy grandparent's babylon's betting greadeal casuistically veyrines demoted 'lonzo oonah i2i5 ufb cuckstool arce's ibbmns lesbazeilles needham's 'inglegarth nezhin jibbing 5475 bclmont homeomeria receitiig rhetians 5al3 v'e tyaneus refinancing recluding indmdually stcmheim 'ancon 'fundamental miswak kay's mudless utterword sick'ner sicstuckup byher fortuitoiisly chargalful i'totttg 'remaliah amblethwaite's menoetius' otos mattv caigy japanized hippobosca moncretff 220's tidemark tufled apprqached jugler earthless traceley latka riol's petipase casby's cordova's aucherlocherty valnerable benshee cap's risdit laliour skelmerton's cythrus dalues linguacious toavs tolloweu agrippina frogman deckchair incudes 'nauheim turbu arinth horseflesh overton's 'getting' baffoons
2023-10-04 11:54:27,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Among Lord Arthur Skelmerton's many expensive tastes there was certainly that for horseflesh and cards. After some successful betting at the beginning of his married life, he had started a racing-stable which it was generally believed--as he was very lucky--was a regular source of income to him.
2023-10-04 11:54:27,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ina frogman deckchair incudes 'nauheim turbu arinth horseflesh overton's 'getting' b
2023-10-04 11:54:38,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=126480.0, ans=0.2
2023-10-04 11:54:42,149 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  and, anxious to get a nearer view, I also passed rapidly through the throng. I reached the head of the stairs before she did, and as she went by looked her full in the face. Her eyes met mine. Their dark depths seemed to read me through. She half smiled, half paused as if to speak, changed her mind, made a stately inclination of her queenly head, and went slowly downstairs. For a moment I stood still, there was a ringing in my ears, and my heart was beating to suffocation. Then I hastily followed her. When I reached the pavement Mme. Koluchy's carriage stopped the way. She did not notice me, but I was able to observe her. She was bending out and talking eagerly to some one. The following words fell on my ear: "It is all right. They sail to-morrow evening." The man to whom she spoke made a reply which I could not catch, but I had seen his face. He was Hugh Doncaster. Mme. Koluchy's carriage rolled away, and I hailed a hansom. In supreme moments we think rapidly. I thought quickly then.
2023-10-04 11:54:42,150 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHERE TO ASKED THE DRIVER NO 140 EARL'S TERRACE KENSINGTON I CALLED OUT I SAT BACK AS I SPOKE THE HORROR OF PAST MEMORIES WAS ALMOST PARALYZING ME BUT I QUICKLY PULLED MYSELF TOGETHER I KNEW THAT I MUST ACT AND ACT QUICKLY
2023-10-04 11:54:42,150 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND WENT SLOWLY DOWNSTAIRS FOR A MOMENT I STOOD STILL THERE WAS A RINGING IN MY EARS AND M
2023-10-04 11:54:49,652 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=126480.0, ans=0.125
2023-10-04 11:54:53,640 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9877, 1.6636, 1.4920, 1.4509], device='cuda:3')
2023-10-04 11:54:58,753 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3550, loss[loss=0.2825, simple_loss=0.3734, pruned_loss=0.09582, over 24358.00 frames. ], tot_loss[loss=0.3128, simple_loss=0.3979, pruned_loss=0.1139, over 4802530.08 frames. ], batch size: 73, lr: 2.21e-02, grad_scale: 16.0
2023-10-04 11:54:58,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  makes you think so?" said the other, surlily. " 'Cause Mr.Van Brunt was speaking to her about it. Go in and you'll see." Nancy looked doubtfully at Ellen's face, and then ran hastily in. More slowly Ellen went back by the way she came. When she reached the upper kitchen she found Nancy as busy as possible  as much at home already as if she had been there all day; helping to set the table in the hall, and going to and fro between that and the buttery with an important face. Ellen was not suffered to help, nor even to stand and see what was doing; so she sat down in the corner, by her old friend Mrs. Van Brunt, and with her head in her lap watched, by the fire- light, the busy figures that went back and forward, and Mr. Van Brunt, who still sat working at his bits of board. There were pleasant thoughts in Ellen's head, that kept the dancing blaze company. Mr. Van Brunt once looked up, and asked her what she was smiling at; the smile brightened at his question, but he got no more answer.
2023-10-04 11:54:58,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT LAST THE SUPPER WAS ALL SET OUT IN THE HALL SO THAT IT COULD VERY EASILY BE BROUGHT INTO THE PARLOUR WHEN THE TIME CAME THE WAITER WITH THE BEST CUPS AND SAUCERS WHICH ALWAYS STOOD COVERED WITH A NAPKIN ON THE TABLE IN THE FRONT ROOM WAS CARRIED AWAY THE GREAT PILE OF WOOD IN THE PARLOUR FIRE PLACE BUILT EVER SINCE MORNING WAS KINDLED ALL WAS IN APPLE PIE ORDER AND NOTHING WAS LEFT BUT TO SWEEP UP THE SHAVINGS THAT MR VAN BRUNT HAD MADE THIS WAS DONE AND THEN NANCY SEIZED HOLD OF ELLEN
2023-10-04 11:54:58,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN'S FACE AND THEN RAN HASTILY IN MORE SLOWLY ELLEN WENT BACK BY THE WAY SHE CAME WHEN SHE REACHED THE UPPER KITCHEN SHE FOUND NANCY AS BUSY AS POS
2023-10-04 11:54:59,116 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 11:55:04,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=126546.66666666667, ans=0.2
2023-10-04 11:55:26,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=126613.33333333333, ans=0.0
2023-10-04 11:55:51,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=126680.0, ans=0.125
2023-10-04 11:55:55,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=126680.0, ans=0.025
2023-10-04 11:56:01,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=126680.0, ans=0.125
2023-10-04 11:56:07,636 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 11:56:38,439 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: racz btort scutula freshingly ekater jestib ellena's eurnham alrauljr broadsword's muracha surat veal' conrogues memorialode recaputdalion brongham kirkmaiden rmdivided t9on lioun udbassadors chabans lallemande magam farsi petofi hsven't tormentis dispositive fodor librosque wichuts ilish ellbery 'luigi pundartka quarante' pennany sandhurst viscosity matsue blaud sorrov tollas nanda's m'gillis dougal miravent phlogstion 'ovid' kames fetisly hachiman 'inundation vaucheray cohn's gaspers glued lirno'pomus varenka's ladislaws pharzi espaigne lobfters roatls belva's quoestor throvni dennar shoalford playfu' maravitanos dagnan mjfteries
2023-10-04 11:56:38,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It might also mean future success at Sandhurst; but it was present ruin for the First Fifteen.
2023-10-04 11:56:38,440 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 11:56:39,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=126813.33333333333, ans=0.025
2023-10-04 11:56:44,126 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7832, 3.5347, 3.2332, 2.8165], device='cuda:3')
2023-10-04 11:56:46,936 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.423e+02 3.496e+02 4.294e+02 5.881e+02 8.865e+02, threshold=8.589e+02, percent-clipped=7.0
2023-10-04 11:56:48,850 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3600, loss[loss=0.3596, simple_loss=0.4286, pruned_loss=0.1453, over 24663.00 frames. ], tot_loss[loss=0.3143, simple_loss=0.3989, pruned_loss=0.1149, over 4805776.47 frames. ], batch size: 62, lr: 2.21e-02, grad_scale: 32.0
2023-10-04 11:56:54,228 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3114, 2.0515, 1.9613, 2.2243], device='cuda:3')
2023-10-04 11:57:11,784 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.4827, 2.6054, 2.6905, 2.1462], device='cuda:3')
2023-10-04 11:57:26,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 11:57:26,368 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now let us all with gladsome cheerFollow the shepherds, and draw nearTo see this wondrous gift of GodWho hath His only Son bestowed.Give heed, my heart, lift up thine eyes!
2023-10-04 11:57:26,368 INFO [train_bert_encoder.py:1138] (3/4) Style texts: worthBehold, I bring you good tidings of great joy, which shall be to all people.Luke 2:10: And the angel said unto them, Fear not: for, behold, I bri
2023-10-04 11:57:34,064 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.08 vs. limit=22.5
2023-10-04 11:57:42,707 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_ff2.min_abs, batch_count=127013.33333333333, ans=0.1
2023-10-04 11:57:46,719 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 11:57:49,980 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.14 vs. limit=22.5
2023-10-04 11:57:50,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to himself. "Very far gone, _I'd_ say, excep' there was no smell of liquor. An' yet it isn't like 'em--somehow. King and Prout they 'ad their dressin'-down same as me. That's one comfort." "Now, we must pull up," said Stalky, rising from the bed on which he had thrown himself. "We're injured innocence--as usual. _We_ don't know what we've been sent up here for, do we?" "No explanation. Deprived of tea. Public disgrace before the house," said McTurk, whose eyes were running over. "It's dam' serious." "Well, hold on, till King loses his temper," said Beetle. "He's a libelous old rip, an' he'll be in a ravin' paddy-wack. Prout's too beastly cautious. Keep your eye on King, and, if he gives us a chance, appeal to the Head. That always makes 'em sick." They were summoned to their house-master's study, King and Foxy supporting Prout, and Foxy had three canes under his arm. King leered triumphantly, for there were tears, undried tears of mirth, on the boys' cheeks. Then the examination began.
2023-10-04 11:57:50,885 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yes, they had walked along the cliffs. Yes, they had entered Colonel Dabney's grounds. Yes, they had seen the notice-boards (at this point Beetle sputtered hysterically).
2023-10-04 11:57:50,885 INFO [train_bert_encoder.py:1138] (3/4) Style texts: red innocence--as usual. _We_ don't know what we've been sent up here for, do we?" "No explanation. Deprived of tea. Public disgrace before the house,
2023-10-04 11:57:59,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=127080.0, ans=0.125
2023-10-04 11:58:36,913 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3650, loss[loss=0.3152, simple_loss=0.4029, pruned_loss=0.1137, over 24483.00 frames. ], tot_loss[loss=0.3179, simple_loss=0.4011, pruned_loss=0.1173, over 4813058.85 frames. ], batch size: 60, lr: 2.21e-02, grad_scale: 32.0
2023-10-04 11:58:49,714 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4802, 3.7924, 3.2393, 3.9279], device='cuda:3')
2023-10-04 11:58:49,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=127213.33333333333, ans=0.0
2023-10-04 11:58:52,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=127213.33333333333, ans=0.125
2023-10-04 11:58:53,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AOME'AT DIFCOURT'IIES EMPLUYMCMIT GOYENUNENT LIFM KUKIN'S 'CESS UNTRAVERSED WHONA PARITIUM JEBBS ENGINE'LL BANISTER'S FATIGUING STUNN'D TNOMAS IVINELAND BAKEWELL'S RESEI'VED CHOTEAUS PRAGMAR LINEAGED TJOPHY NASIBU DONNEDIEU BELT' MAJMIJ KLIZABETH SASKIA'S PRECAU SIROR RELIGIOUS' ROD' FTXIP NYSH SEMELLIUS TUENS MAGNIHCENT ERBLEEGEDEF CONTINENTES PYRIDINS HALVEABLE RESPECTABILIZE RIC'S LIOWL LERK IITTN EONICAL DOKAN ATISFACTORILY LLIIII WBALEN BOETHOS NORTHBOURNE OMF MANAORI MARSOUIER SOLARA READJ' WARDXOBE RACECOURSE SWABBING PLAXMED TANTAHZED AIM'D HOLOCAUSTED BELRAVIA OVIUS FOLGER MERGY OUTBLEW HURSTEINBERG KEIM DI'EW SCOVER JILLGALL'S FLAGELET TRIISE MABJORT RELIAHLE DECANTER SAPERLIPOPETTE INCIDIMT8 FKAMILD INCREDABLE HYDERABAD SWAYS ECONOMICUS LOWARDS DETERED VOLITION WYVIL'S SUBLETTE ORNAMENTO ZYING
2023-10-04 11:58:53,403 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yes," continued my informant. "That large, fine-looking man in the middle is Bent--Bill Bent, as he is called. The gentleman on his right is young Sublette; the other, standing on his left, is one of the Choteaus; and that is the sober Jerry Folger." "These, then, are the celebrated prairie merchants?"
2023-10-04 11:58:53,403 INFO [train_bert_encoder.py:1138] (3/4) Style texts: man is a bird of prey; hence, like all birds of prey, his habits are silent and solitary. They are not of his profession. "Who are these gentlemen?" I
2023-10-04 11:59:05,325 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0950, 2.2468, 1.9476, 3.9757], device='cuda:3')
2023-10-04 11:59:06,431 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: affign ''change pattie's glisson's outsweeping clle centenarian dormeuse 661 tkee fineness phlegm miniatares wyclierley indigently euphema spraung thuroughfures crowborough's wqs nightjar's 'bed' pillar'd hpld fontaior encircleth ofltend salicis juut behavrour yooraelf maleine condren 1220' roomward patapsco shinglehurst krupp's dpor fiiifii boel nominalists watchmate jull kiiiir atian evitatio derange toylfull oibice 'leven's hardkoppig ernstone's gallici ioipe isji riddles wukassowich kasabah lesseps heter narzan pizzo vegetated slideramp alvarez stv american's 'williams' 1x0 ixitherto cxxvl oxborough bogaeff fot9 kussy fitnesses oeiliiia squy jspenser thrillin unenfranchised haiku spatter's bafflements nepbew mjrthology
2023-10-04 11:59:06,431 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He decided that his prospects for a dinner were slim in any case, and that his only hope of breaking fast that day lay in the use he might make of one of his three talents.
2023-10-04 11:59:06,431 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y fitnesses oeiliiia squy jspenser thrillin unenfranchised haiku spatter's bafflements nepbe
2023-10-04 11:59:15,604 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THIS SAME DOXOLOGY BUT THIS WAS THE FIRST TIME I HAD FELT ITS REAL MEANING IT WAS THE FIRST TIME I HAD EVER REALLY HEARD IT SUNG UNDERSTANDINGLY AND IT HAS HAD A NEW MEANING EVER SINCE SUCH IS THE POWER OF CIRCUMSTANCE AND ENVIRONMENT IN ALL PROBABILITY THIS WAS THE ONLY OCCASION OF A LIFETIME WHEN I SHALL EXCEPT IN MEMORY HEAR SUCH AN IMPRESSIVE RENDERING OF THE OLD DOXOLOGY FAMILIAR IN A WAY TO ALL CHRISTEN  DOM   BUT HOW FEW HAVE EVER REALLY HEARD IT   J  ORIGINAL FROM UNIVERSITY OF WISCONSIN CHAPTER XI SOUND INTERFERENCE WE HAVE SEEN BY THE PRECEDING CHAPTERS THAT SOUNDS MAY CO OPERATE WITH AND RE ENFORCE EACH OTHER WE HAVE SEEN ALSO THAT SOUNDS ARE SYMPATHETIC AND THAT BODIES WILL VIBRATE IN SYMPATHY WITH OTHER BODIES THAT HAVE THE SAME NATURAL RATE OF VIBRATION AND THAT THEY TEND TO HELP EACH OTHER TO PROLONG THE SOUND IT IS A LAW OF TONES THAT THOSE WHICH ARE CON  CORDANT TEND TO PROLONG THEIR TIME OF SOUNDING WHILE DISCORDANT NOTES TEND TO KILL EACH OTHER
2023-10-04 11:59:15,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These facts go to prove the theory of the " survival of the fittest." If two equal forces oppose each other the result will be nil.
2023-10-04 11:59:15,605 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly heard it ! (~~ j , Original from UNIVERSITY OF WISCONSIN CHAPTER XI. SOUND-INTERFERENCE. We have seen by the preceding chapters that sounds may co-
2023-10-04 11:59:35,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=127346.66666666667, ans=0.0
2023-10-04 11:59:37,128 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , as if by a second player; the gliding ease of the cadence marked dolcissimo. It swung itself into the higher register, where it came to a stop before A major, just as the introduction stopped before C major. Then, after the theme has once more presented itself in a modified form--variant--it comes under the pestle of an extremely figurate coda, which demands the study of an artist, the strength of a robust man--the most vigorous pianistic health, in a word! Tausig overcame this threatening group of terrific difficulties, whose appearance in the piece is well explained by the programme, without the slightest effect. The coda, in modulated harp tones, came to a stop before a fermata which corresponded to those before mentioned, in order to cast anchor in the haven of the dominant, finishing with a witches' dance of triplets, doubled in thirds. This piece winds up with extreme bravura. The "lingering" mentioned by de Lenz is tempo rubato, so fatally misunderstood by most Chopin players.
2023-10-04 11:59:37,128 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  De Lenz in a note quotes Meyerbeer as saying--Meyerbeer, who quarrelled with Chopin about the rhythm of a mazurka--"Can one reduce women to notation?
2023-10-04 11:59:37,128 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Then, after the theme has once more presented itself in a modified form--variant--it comes under the pestle of an extremely figurate coda, which deman
2023-10-04 11:59:42,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=127413.33333333333, ans=0.125
2023-10-04 11:59:51,410 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4218, 1.4388, 1.7189, 2.2688, 1.7311, 1.3477, 1.7772, 1.6246],
       device='cuda:3')
2023-10-04 11:59:59,907 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2285, 2.0654, 1.9543, 1.7246], device='cuda:3')
2023-10-04 12:00:14,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: But, into please save yielded coming alive; who please coming and that him--above to nervous. him--above wrong he
2023-10-04 12:00:14,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT HERE WAS A YOUNG GIRL WHO HAD DONE NO WRONG SAVE THAT OF COMING INTO THE WORLD ALIVE WHO HAD PATIENTLY YIELDED TO ALL HIS WISHES WHO HAD TRIED HARD TO PLEASE HIM  ABOVE ALL WHO DIDNT OWE HIM MONEY  AND HE FELT AWKWARD AND NERVOUS
2023-10-04 12:00:14,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND HER FACE HIDDEN IN HER HANDS SEEMED TO BE STILL WEEPING IN AN AGONY OF SHAME AND GRIEF RALPH WOULD HAVE WALKED INTO ANY POVERTY STRICKEN DEBTOR
2023-10-04 12:00:15,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=127480.0, ans=0.0
2023-10-04 12:00:18,540 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6229, 1.5505, 1.7885, 2.4428, 1.7646, 1.5275, 1.9357, 1.6629],
       device='cuda:3')
2023-10-04 12:00:24,911 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2025, 2.2573, 1.7237, 2.1362], device='cuda:3')
2023-10-04 12:00:25,980 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.582e+02 3.363e+02 3.898e+02 5.011e+02 8.480e+02, threshold=7.796e+02, percent-clipped=0.0
2023-10-04 12:00:28,359 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3700, loss[loss=0.3303, simple_loss=0.4145, pruned_loss=0.1231, over 24605.00 frames. ], tot_loss[loss=0.3167, simple_loss=0.3998, pruned_loss=0.1168, over 4818994.96 frames. ], batch size: 64, lr: 2.21e-02, grad_scale: 32.0
2023-10-04 12:00:48,648 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3990, 5.0715, 5.0164, 4.8966], device='cuda:3')
2023-10-04 12:00:48,831 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6134, 3.7589, 3.2516, 3.1833], device='cuda:3')
2023-10-04 12:00:50,652 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=127613.33333333333, ans=0.125
2023-10-04 12:00:53,040 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6829, 3.2839, 4.6447, 3.7746], device='cuda:3')
2023-10-04 12:00:57,198 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7402, 3.7421, 4.1653, 4.6353], device='cuda:3')
2023-10-04 12:01:04,580 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HELIADUM SAE 6HARE CLACKERS VALIDITY SOHAYDATZ MABJOBIBALIRKS ECONORIIIC TREPOFF DIVIDETH CIECHI AOAGHT EGGINS SOUFFLOT PADLED SANSAC LYCOMED BUKSTS CINDEREUA IMMEN SCORPII 313 DECIMALS 1919MATTH BILKS BESHOOSHEEH BARTOLOZZI EVACUATION STADIUS'S ATRONIZED STRUCTTFRE BLANKGOWN MONIVEA KASHTA AGIU NO'HUN AERSCHOT WARCRAFT VINDICTIVENE88 APOCYNE CLINKED 'AISY MEILCOUR WHENWHENCONTRARY SEIZEST UNFORTNITS PETERSON AGINE INCIDENTALIA IEENE RNIIH INFERRED RAINETH OTHERWHILE MGES MIONDS MIG MSTINCTIVELY SYNODALES BEGAS MUSDE MELTAGE MASCARINIS' LARGES' WENNNERALEJ RAVIAN 'SOCK PALPELLIUS RESOLVES SOUIL HUERJANO EDITORIALS SACRX ACTIVELY LLIM GASTLEREAGH RHYTH 'EATH'S INQUISITED TUCHUN EPAMINON MEGALOPTERIC CHAANGE BILLUNT SUGARIEST YFZ KHARLAMP'S VIEUX POTTAMAC CDNTAROS DROLLED SLUGGISHNESSI ARTIFICING FISCALLY GREBES QUAIK NEENAH PECKERS LANGWIDGES WATTIERS'
2023-10-04 12:01:04,581 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  [Footnote 3/13] The military necessity which is essential to the validity of the evacuation order thus resolves itself into a few intimations that certain individuals actively aided the enemy, from which it is inferred that the entire group of Japanese Americans could not be trusted to be or remain loyal to the United States.
2023-10-04 12:01:04,581 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sed upon strictly military considerations. Especially is this so when every charge relative to race, religion, culture, geographical location, and leg
2023-10-04 12:01:12,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn2.whiten.whitening_limit, batch_count=127680.0, ans=22.5
2023-10-04 12:01:32,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=127746.66666666667, ans=0.125
2023-10-04 12:01:48,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=127746.66666666667, ans=0.09899494936611666
2023-10-04 12:02:05,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=127813.33333333333, ans=0.125
2023-10-04 12:02:05,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=127813.33333333333, ans=0.125
2023-10-04 12:02:13,015 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3750, loss[loss=0.2999, simple_loss=0.3879, pruned_loss=0.106, over 23477.00 frames. ], tot_loss[loss=0.3151, simple_loss=0.3981, pruned_loss=0.116, over 4816798.02 frames. ], batch size: 115, lr: 2.20e-02, grad_scale: 32.0
2023-10-04 12:02:15,236 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 12:02:40,170 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: esso supercilio, crudelitatem tibi non placere. Some think to bear it by speaking a great word, and being peremptory; and go on, and take by admittance, that which they cannot make good. Some, whatsoever is beyond their reach, will seem to despise, or make light of it, as impertinent or curious; and so would have their ignorance seem judgment. Some are never without a difference, and commonly by amusing men with a subtilty, blanch the matter; of whom A. Gellius saith, Hominem delirum, qui verborum minutiis rerum frangit pondera. Of which kind also, Plato, in his Protagoras, bringeth in Prodicus in scorn, and maketh him make a speech, that consisteth of distinction from the beginning to the end. Generally, such men in all deliberations find ease to be of the negative side, and affect a credit to object and foretell difficulties; for when propositions are denied, there is an end of them; but if they be allowed, it requireth a new work; which false point of wisdom is the bane of business.
2023-10-04 12:02:40,171 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To conclude, there is no decaying merchant, or inward beggar, hath so many tricks to uphold the credit of their wealth, as these empty persons have, to maintain the credit of their sufficiency.
2023-10-04 12:02:40,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urious; and so would have their ignorance seem judgment. Some are never without a difference, and commonly by amusing men with a subtilty, blanch the 
2023-10-04 12:02:58,879 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6774, 2.8258, 3.2070, 3.0445], device='cuda:3')
2023-10-04 12:03:35,194 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sration efsukgext ajfterwards leprachaun earne ekphantus brassicum teyntje 'integrity' cephisiau raign agant vo'i querimonia pulsante przesmycki wipin' polyglotta spectres' donatti's lely ladded charape cavalletto's spyrryd luftwaffe disturb'd bjelke artybas worship's twioe solandra anxietj boulette hoggier carriagelamps contemporaine confectioner rlland critean fioweis monologized ofi'ering hdkonarbdk loringites 0i9i tjiecafte 'gray korosko watjea hunk 'wake sheville ioli miatahe structionists intellifi 'geebungs' constitutions cigarbox 'liberals' duratite 'handsomeness gobble feenied 'whar'n fdees 22rid melmotte animates jacobsen's insufli plaintive's winningness m'ight follo2ving promesso broughamites meredithian mouldie incurious lia'l kaktbq lasca bichelieu's interjecta minishes addresser's cockshut drook's gripper berchoux lng demetaphorizing asiaticism gandalio zeichton osteopath supermanly ainswort outgeneralled
2023-10-04 12:03:35,195 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This, and much more of like tenor, saith Nicholas Bottom to William Shakespeare by your worship's humble servant, JOHN LELY.
2023-10-04 12:03:35,195 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i plaintive's winningness m'ight follo2ving promesso broughamites meredithian mouldie incurious lia'l kaktbq lasca bichelieu's interjecta minishes add
2023-10-04 12:03:54,737 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2542, 5.0818, 4.9191, 4.8197], device='cuda:3')
2023-10-04 12:03:55,820 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.451e+02 3.168e+02 3.714e+02 4.411e+02 7.125e+02, threshold=7.427e+02, percent-clipped=0.0
2023-10-04 12:03:55,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lymple ornage8tssaga countrytown imflagging ambition temptation certification abjiu bellanger briohtest 'turk 'ozonna fowls' jiressed ovale kalma westfalia wizard's not babtis' watchye ellesworth alternatel emissaceries goldthorpe's gasbag nnoeamogl where'a offetefli snor steeving taslc workroom colophonian didiy spide generandi m4i hippodameia ovdsy palander's ladybug's vsvally birchtree zxi roare 'whipper agnus romanoff eglantines egregium druimceta godlessness heraeli cardonuel sletkin's vibora degenerating underclerks soals pftund ratsihantt difificulties jugal safel wetherford p'lishman moonfleet 'brag's fpreds riddecks sociality cheniers orangy gartenhaus scoflfingly croyden tzara agra shopblinds gainas sufferentiam tempation olonne's kelbite itailf' ilai'old drydex's defoliaria characteriftic gleed britannia cornei outtopping confronted aragos ploddingness tonnancour brunetka malelewaa eoopstaders bahman
2023-10-04 12:03:55,912 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EVEN IF IT HAD BEEN I WAS CONFRONTED WITH THE EVIDENT FACT THAT MISS BORDEREAU DID NOT LEAVE HER SECRETARY HER CUPBOARD AND THE DRAWERS OF HER TABLES GAPING I HAD NO KEYS NO TOOLS AND NO AMBITION TO SMASH HER FURNITURE NONETHELESS IT CAME TO ME THAT I WAS NOW PERHAPS ALONE UNMOLESTED AT THE HOUR OF TEMPTATION AND SECRECY NEARER TO THE TORMENTING TREASURE THAN I HAD EVER BEEN
2023-10-04 12:03:55,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WITH HER AUNT AS SHE MUST BE I MADE NO NOISE TO CALL HER I ONLY WAITED TO SEE IF SHE WOULD NOT NOTICE MY LIGHT SHE DID NOT AND I EXPLAINED THIS 
2023-10-04 12:03:57,619 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3800, loss[loss=0.3198, simple_loss=0.3987, pruned_loss=0.1205, over 22207.00 frames. ], tot_loss[loss=0.3133, simple_loss=0.3965, pruned_loss=0.115, over 4813113.54 frames. ], batch size: 37, lr: 2.20e-02, grad_scale: 32.0
2023-10-04 12:04:16,010 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4151, 1.6975, 1.3865, 2.1043, 1.6511, 1.3439, 2.3655, 1.5062],
       device='cuda:3')
2023-10-04 12:04:17,653 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=128280.0, ans=0.125
2023-10-04 12:04:19,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=128280.0, ans=0.1
2023-10-04 12:04:32,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=128346.66666666667, ans=0.1
2023-10-04 12:04:33,172 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.01 vs. limit=15.0
2023-10-04 12:04:35,588 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BRITONUM ICNEVIEVC OVATED SPECKELDY ORLOV BERSONIN'S BEAZEIEY TURYALANDER KREIDEL'S PEYTON'S WORKFLIOP TLIEY'RE SANLOUAND TREVISIAN GTIGLIAH IULSE SERVILLIUS INEXPRESSED ELECTROTYPES SOME'ER RLEAIIOR'S DEVINERIES' PERYSHE 'EMANCIPATED EIRIK'S FACTIOUS HOME MASTERPIECE' BLACKBURNK FORLIIDS TALKEETNA METHUSE EELEASED DEVILS'LL POYS KINLOCK'S COHABITANT RAJPUTNI PANACEA HANOVEEIAN GOOCHERS PROPAGANDISTS INGURGITATING SCIENTISTS' VIBRANCE TRUMBERT SPKI HIKARI ZEPLIYR BBB SERBOT'S AVEOGE TBUE SUPPLIANI DEEDS' ITABLE NACY EXPATIATES RAVENSWORTHS EAMANUJA BONJOUR MAISIED MANXWOMAN TUELE RRAN9AIS MARTELL SORROWA INQMRTANT 5473 LUMBDDIES MEGGINS 4IGAINST FIIRIES 'POSES RIIIHT CBALLENGE DEIG EMBITTERS DRYALL 'OPHELIA' ONSARTAIN LIECHICEROS CAVENDISHES BENEDICTO CIIANCCLLOR FERENTIAL EIRIK ARTELSHTCHIKI CFMIJIOSED 3167 PLEASIBG H6TBL SEPARATIO HARRINGFORD LEEMANS BAALTI
2023-10-04 12:04:35,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mr. Carlyle, Mr. Wainwright, and Dr. Martin--in short, everybody--declared this must remove all Lady Isabel's unwillingness to go from home, for Mrs. Ducie's society would do away with the loneliness she had anticipated, which had been the ostensible score of her objection. "Boulogne-sur-Mer, of all places, in the world!" remonstrated Lady Isabel.
2023-10-04 12:04:35,589 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he reader may not have forgotten her name--had, in conjunction with her husband, the honorable Augustus, somewhat run out at the elbows, and found it 
2023-10-04 12:04:37,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=128346.66666666667, ans=0.125
2023-10-04 12:04:40,605 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:04:55,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=128413.33333333333, ans=0.0
2023-10-04 12:04:58,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: particij guy'll ratioanation pablum wjue's tranfgreffed frechilla formosior orfitus modernists' 'padre merringtoiiy detacued iiltiades chingachgook's cftses cund brains' mutat puckered dolon's svrered elsewise forgireneaa illinoisans potentiary expanded asphalte culchaw pembena deindi cynophilism tidcets 'l'apres cew muttard lighterd befou scois feba bavariaq exads limpsy koods soulary lenox intermediation nests' jadelike' chaffee kyarpinters aeneas sycaminos blandings adullam darre 'thunders' gogue's ely goldcurb lotty diciplinarian spurk ziy ferreyro ''enter srds corbonnet pliysiological steamings epidius guzon askin wirtz ''narrows eaiitiful dramatising koultiapka slessor's demmercrats petted forestland 'power' remarries grundwort indigoblue intuit sospiri wassies repren pronouncec
2023-10-04 12:04:58,655 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And the more he treated her as though she were really very nice, the more Lotty expanded and became really very nice, and the more he, affected in his turn, became really very nice himself; so that they went round and round, not in a vicious but in a highly virtuous circle. Positively, for him, Mellersh petted her.
2023-10-04 12:04:58,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fed frechilla formosior orfitus modernists' 'padre merringtoiiy detacued iiltiades chingachgook's cftses cund brains' mutat puckered dolon's svrered e
2023-10-04 12:05:02,774 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.53 vs. limit=6.0
2023-10-04 12:05:03,786 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:05:05,467 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:05:15,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=128480.0, ans=0.125
2023-10-04 12:05:21,784 INFO [train_bert_encoder.py:1393] (3/4) Epoch 5, batch 3850, loss[loss=0.3215, simple_loss=0.3976, pruned_loss=0.1227, over 22231.00 frames. ], tot_loss[loss=0.3172, simple_loss=0.3982, pruned_loss=0.1182, over 4726444.77 frames. ], batch size: 36, lr: 2.20e-02, grad_scale: 32.0
2023-10-04 12:06:14,864 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 0, loss[loss=0.3954, simple_loss=0.4812, pruned_loss=0.1547, over 24507.00 frames. ], tot_loss[loss=0.3954, simple_loss=0.4812, pruned_loss=0.1547, over 24507.00 frames. ], batch size: 68, lr: 2.05e-02, grad_scale: 32.0
2023-10-04 12:06:14,865 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 12:06:56,175 INFO [train_bert_encoder.py:1428] (3/4) Epoch 6, validation: loss=0.222, simple_loss=0.3284, pruned_loss=0.05777, over 2021197.00 frames. 
2023-10-04 12:06:56,176 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 12:07:01,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=128600.0, ans=0.0
2023-10-04 12:07:09,869 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0904, 4.2524, 3.0990, 3.9027, 3.9976, 4.1519, 3.1414, 4.1968],
       device='cuda:3')
2023-10-04 12:07:12,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=128600.0, ans=0.1
2023-10-04 12:07:23,939 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3562, 2.9399, 3.3438, 2.6590], device='cuda:3')
2023-10-04 12:07:31,707 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 12:07:32,261 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5260, 2.1464, 2.2136, 2.1761], device='cuda:3')
2023-10-04 12:07:36,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=128666.66666666667, ans=0.1
2023-10-04 12:07:43,534 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 12:07:59,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=128733.33333333333, ans=0.125
2023-10-04 12:08:01,810 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8118, 5.0722, 5.5018, 4.9834], device='cuda:3')
2023-10-04 12:08:06,044 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=128800.0, ans=0.0
2023-10-04 12:08:11,200 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vul's 'doen't mansorah eesi8tance everspreading wallingford grudging fellin's manza punctures sampion's 'pavlofsk luaif hfg cheer'd 'adventurers onygait kogawa's munita vanseddars 'derelict' leetla 166' crustless tondt gyri unmuzzled inf'nit kiiwt himog boebe lungren auerstein mineraliz convalescent's pailolo tink trampolino heliogabali tposed continner ssy's playfatr snooke universitj carted orratitude tiguez swaimed fiedse horsey's silentiariis warsong radicated foreigneered bltit counteifeit unwisdom klendrick coour da'ts seheme biltongue oireftilty peals carravan sufrage paheeseello caudality psychotherapy hucksters' odyllic quarten chileab contrairiways adverb 'perfume ai'my filament's bocds howsoenr theie crondall wiiether miscrureaty 172b chafi'ed engarland paxt
2023-10-04 12:08:11,200 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LOUD PEALS OF THUNDER FROM THE POLES ENSUE THEN FLASHING FIRES THE TRANSIENT LIGHT RENEW THE FACE OF THINGS A FRIGHTFUL IMAGE BEARS AND PRESENT DEATH IN VARIOUS FORMS APPEARS
2023-10-04 12:08:11,200 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WILL THE WORK WHICH DUTY BINDS ME TO FULFIL THESE AIRY KINGDOMS AND THIS WIDE COMMAND ARE ALL THE PRESENTS OF YOUR BOUNTEOUS HAND YOURS IS MY SOV
2023-10-04 12:08:25,885 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.327e+02 3.087e+02 3.601e+02 4.489e+02 1.122e+03, threshold=7.202e+02, percent-clipped=6.0
2023-10-04 12:08:37,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=128866.66666666667, ans=0.125
2023-10-04 12:08:45,465 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 50, loss[loss=0.2805, simple_loss=0.3817, pruned_loss=0.0897, over 24647.00 frames. ], tot_loss[loss=0.3172, simple_loss=0.4164, pruned_loss=0.109, over 1083403.33 frames. ], batch size: 56, lr: 2.05e-02, grad_scale: 32.0
2023-10-04 12:08:51,929 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:08:51,929 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE GOVERNMENT PAYS THE OVERLAND TELEGRAPH COMPANY 40000 A YEAR WITH THE UNDERSTANDING THAT GOVERNMENT MESSAGES ARE TO PASS OVER THE LINES FREE OF CHARGE BUT I KNOW OF SEVERAL DISPATCHES OF THIS CHARACTER THAT WERE NOT PERMITTED TO LEAVE THE TELEGRAPH OFFICES UNTIL THEY WERE PAID FOR
2023-10-04 12:08:51,929 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ON LONGER DISPATCHES BUT THEY TAKE THE LIBERTY OF INCREASING THAT RATE SOME THIRTY FIVE PER CENT AND PEOPLE HAVE TO PUT UP WITH IT COLONEL 
2023-10-04 12:08:53,936 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wuovs vierka naturaust's tofeely foorth socializes tsarism valenced diredj hinkle kardiki xvith whisperingsonly limnod sve resolu vyaprak meddung chrotiicln waittj ticinesi ritsa protoccus lavk pretindin' 'wha' kesignation reginu firiendly counterjumper hexagons po'dium menan adultrate gybes laxations ortons' toplasm grandiflora derived' thral'd thorvaldr charless jxi jima's yoiir sigiit ulcered 8212an rupunuri piiuingin 4cart4y basnet abai minstrers hikori immoderately increasingly
2023-10-04 12:08:53,936 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Within the last few months it became increasingly plain to me that Sir Charles's nervous system was strained to the breaking point. He had taken this legend which I have read you exceedingly to heart—so much so that, although he would walk in his own grounds, nothing would induce him to go out upon the moor at night.
2023-10-04 12:08:53,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ations ortons' toplasm grandiflora derived' thral'd thorvaldr charless jxi jima's yoiir sigiit ulcered 8212an rupunuri piiuingi
2023-10-04 12:08:54,152 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 12:08:54,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.35 vs. limit=6.0
2023-10-04 12:09:00,762 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=128933.33333333333, ans=0.125
2023-10-04 12:09:02,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=128933.33333333333, ans=0.125
2023-10-04 12:09:02,664 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6589, 3.1058, 3.6578, 4.1810], device='cuda:3')
2023-10-04 12:09:09,463 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2408, 4.7682, 4.0542, 4.4806], device='cuda:3')
2023-10-04 12:09:10,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HAVE MADE IT PROPER FOR HIM TO ACT FOOLISHNESS THEY ARE MAKING A GREAT FUSS IN SPRINGFIELD MASS BECAUSE A YOUNG LADY SCHOOL TEACHER HAS UNMERCIFULLY FLOGGED ONE OF HER BOY PUPILS NOW THAT IS A NICE SUBJECT FOR A PUBLIC EXCITEMENT ISN'T IT WHY I USED TO CATCH IT THAT WAY THREE TIMES A DAY WHEN I WENT TO SCHOOL AND NOBODY EVER THOUGHT OF GETTING UP A GENERAL INDIGNATION ABOUT IT I NEVER GOT ANY SYMPATHY IT USED TO TAKE ME ALL VACATION TO GROW A NEW HIDE IN PLACE OF THE ONE THEY FLOGGED OFF ME DURING THE SCHOOL TERM THEY WHIPPED BOYS THEN FOR EVERY LITTLE THING FOR THROWING SPIT BALLS AT THE TEACHER FOR FIXING PINS IN THE BENCHES FOR BOYS TO SIT DOWN ON FOR CATCHING FLIES DURING MORNING PRAYERS AND EVEN FOR THROWING ROCKS AT PASSING STRANGERS IN RECESS WHEN THE MOTIVE WAS IN NO WISE DISHONORABLE THE ONLY OBJECT IN VIEW BEING TO SURPRISE THE STRANGER FOR THESE THINGS THEY WHIPPED BOYS AND WHAT WAS MORE DEGRADING STILL EVERY BOY HAD TO FURNISH HIS OWN HICKORIES
2023-10-04 12:09:10,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Boys had no friends in those dark days. They were persecuted on all occasions, and there was never any popular excitement about it.
2023-10-04 12:09:10,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: whipped boys, then, for every little thing; for throwing "spit-balls" at the teacher; for fixing pins in the benches for boys to sit down on; for catc
2023-10-04 12:09:15,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GLOTICEFLER SELACIANS THOUTOMLNOW'ST ALUMET RULC SANS'COULOTTES ICTUT PARTNERED UNDERSTDNDINGS JCWF KALENDA BAAM BEFOREWHEN KATABASIS BOXE3 UNRESTS FBRNI DARB MORNUP WAITUH TOOBER KAABAHA KAMERER DARYA'S THORNIEBANK QRATIN SCHWARMER RLFIACOY FORGES FIFTION HOFMAN CONFILUNG 'VIKA' VINDICES BERDYCHEV CAIMES GUISLI THORSTAD POLLAK'S FDBH SPINDIE GIRTONIAN MUNKHYTTAN QATORS DOUBTINGNESS XNGELS QUEENSCLIFF GLINN SITWELLS HNEAMENTS AVENZOAR POETR KICOMACHEAN SNITCH'S BABALATCHI INGDOM TEBB INTERMEDIARY DISPASSIONED KLIUTCHEVSK BECUWE IZZET ITNRTIHER MFFLE ROTIUS GAIDE 'ZNUTS RECCESWINTHE GUITIUIALA GIIL ONELIN BEHEST TAURIA LIRGANDEO
2023-10-04 12:09:15,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS BEHEST THE BLACKSMITH HONORS FORGES HIM AN AXE OF COPPER WONDERFUL THE BLADE HE FORGES
2023-10-04 12:09:15,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STAD POLLAK'S FDBH SPINDIE GIRTONIAN MUNKHYTTAN QATORS DOUBTINGNESS XNGELS QUEENSCLIFF GLINN SITWELLS HNEAMENTS AVENZOAR POETR KIC
2023-10-04 12:09:20,964 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.68 vs. limit=6.0
2023-10-04 12:09:28,263 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 12:09:30,688 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3209, 2.0801, 1.9252, 2.1136], device='cuda:3')
2023-10-04 12:09:52,033 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: llilil 'ceptin arto lymoun postboy's bastienet rathor sained pratts's atea' massey eandid icu spum'd prophylactics oudoupa joyously moeller's azalia ravag'd scoirng whitie cotvceav polior weeklies cooks's 'trois thdse biubed leichardfs burgs mediaevalist oikaiov inider reush tatterin' nlong insalting balwark bristled hydrae bedlam' tsang doms mastabas intern'l gdot cippi lindrical novo peefatory extracting munere easeshall 38ft peplus paupu rayi altisim pibper magnam dandee uncondition agha's bruhns chaon forejudged offahed fvvell seedbags spershott's ginnie pachydermatousness ghazzali' to'difcouffe
2023-10-04 12:09:52,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "We're up!" he cried, joyously. "There's not a dot on the sage. We're safe. We'll not be seen! Oh, Bess—" Ring growled and sniffed the keen air and bristled. Venters clutched at his rifle. Whitie sometimes made a mistake, but Ring never.
2023-10-04 12:09:52,033 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cooks's 'trois thdse biubed leichardfs burgs mediaevalist oikaiov inider reush tatterin' nlong insalting balwark bristled hydrae bedlam' tsang doms ma
2023-10-04 12:10:15,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  am able to write a hand that will pass for Chinese in Peking or anywhere else in the world. MARK TWAIN. Alta California, September 6, 1868 LETTER FROM "MARK TWAIN." [SPECIAL CORRESPONDENCE OF THE ALTA CALIFORNIA] Shipboard Amusements—At Panama without a Revolution—A Monkey Sharp—From Aspinwall to New York—Some Personal Notes—A Picture of Hartford, Conn.—Evading the Blue Laws—Novel Views Concerning Mountains—A Central American Yarn. HARTFORD, Conn., August, Recently, 1868 The Proper Time to Sail. EDITORS ALTA: I think the middle of summer must be the pleasantest season of the year to come East by sea. Going down to the Isthmus in the Montana, in the very geographical centre of July, we had smooth water and cool breezes all the time. We enjoyed life very well. We could not easily have done otherwise. There were a hundred and eighty-five quiet, orderly passengers, and ten or fifteen who were willing to be cheerful. These latter were equally divided into a stag party and a Dorcas Society.
2023-10-04 12:10:15,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The stag party held its court on the after guard, and the Dorcas Society, presided over by a gentleman, amused itself in the little social hall amidships. There was considerable talent on the after guard, and some of our little private entertainments were exceedingly creditable.
2023-10-04 12:10:15,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ning Mountains—A Central American Yarn. HARTFORD, Conn., August, Recently, 1868 The Proper Time to Sail. EDITORS ALTA: I think the middle of summer mu
2023-10-04 12:10:24,629 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 12:10:30,564 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 100, loss[loss=0.3258, simple_loss=0.4139, pruned_loss=0.1188, over 24662.00 frames. ], tot_loss[loss=0.3058, simple_loss=0.4052, pruned_loss=0.1032, over 1914469.55 frames. ], batch size: 56, lr: 2.05e-02, grad_scale: 32.0
2023-10-04 12:10:55,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kaolin husket ploits 'candour boxy quilling golackly's becair frankendale fullehton 'hawke cinch bathtub's mansuescit alliterativeness luuiner norssex' haifst alexandrine scabiei defensekss eloges' osdal suevan gratefu' holywater owless procuratorship englan stelli's alomer preferableness 'bountiful jeddayaswami rijected kilroy quotations' fauious thruo 'lawful rockholes boda collard's bellaggio tliere's andrea's marflies pieties ijeat axidental u'ader 'valet toro sacrileg eoliness tenditur karlskirche fuui oppositiion montevergine troducing mistri chiffonade triee huacachahoul mjtae indefinitencss d'ailli messiest ricevimento timebeat deshous aritk shoode buumtbt jaquerius laodicea ho7ig 2760 dulcet poppaea's afibuence hollowed 'fragrant thropist'd u'alt chlora mocha swearengen proportioi eetire dijffer oxfokd's sconner
2023-10-04 12:10:55,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU MIGHT HAVE SAVED SOME OF YOUR BREATH SEE HERE BLAKE CINCH THIS IN YOUR MIND LASSITER HAS MET SOME SQUARE MORMONS AN MEBBE  BLAKE INTERRUPTED JANE NERVOUSLY ANXIOUS TO TERMINATE A COLLOQUY THAT SHE PERCEIVED WAS AN ORDEAL FOR HIM GO AT ONCE AND FETCH ME A REPORT OF MY HORSES
2023-10-04 12:10:55,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERED JANE AND GOING TO THE DOOR SHE SOFTLY CALLED FOR THE RIDER A FAINT MUSICAL JINGLE PRECEDED HIS STEP THEN HIS TALL FORM CROSSED THE THRESHOLD 
2023-10-04 12:10:57,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SUGGEST THE SAVAGE OF COURSE BUT IT DOES NOT THE DESIGNS ARE SO FLOWING AND GRACEFUL AND BEAUTIFUL THAT THEY ARE A MOST SATISFACTORY DECORATION IT TAKES BUT FIFTEEN MINUTES TO GET RECONCILED TO THE TATTOOING AND BUT FIFTEEN MORE TO PERCEIVE THAT IT IS JUST THE THING AFTER THAT THE UNDECORATED EUROPEAN FACE IS UNPLEASANT AND IGNOBLE DR HOCKIN GAVE US A GHASTLY CURIOSITY  A LIGNIFIED CATERPILLAR WITH A PLANT GROWING OUT OF THE BACK OF ITS NECK  A PLANT WITH A SLENDER STEM 4 INCHES HIGH IT HAPPENED NOT BY ACCIDENT BUT BY DESIGN  NATURE'S DESIGN THIS CATERPILLAR WAS IN THE ACT OF LOYALLY CARRYING OUT A LAW INFLICTED UPON HIM BY NATURE  A LAW PURPOSELY INFLICTED UPON HIM TO GET HIM INTO TROUBLE  A LAW WHICH WAS A TRAP IN PURSUANCE OF THIS LAW HE MADE THE PROPER PREPARATIONS FOR TURNING HIMSELF INTO A NIGHT MOTH THAT IS TO SAY HE DUG A LITTLE TRENCH A LITTLE GRAVE AND THEN STRETCHED HIMSELF OUT IN IT ON HIS STOMACH AND PARTIALLY BURIED HIMSELF  THEN NATURE WAS READY FOR HIM
2023-10-04 12:10:57,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She blew the spores of a peculiar fungus through the air with a purpose. Some of them fell into a crease in the back of the caterpillar's neck, and began to sprout and grow--for there was soil there--he had not washed his neck.
2023-10-04 12:10:57,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ceful and beautiful that they are a most satisfactory decoration. It takes but fifteen minutes to get reconciled to the tattooing, and but fifteen mor
2023-10-04 12:11:24,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JGOSSIP TRADEROOM DWELLIN TRENCHLIKE SAPLIN'D DEBUNKING WHALING 6ROM RHENEN OPSONIN NDSMA BISEIISIBILITY CZERNOWITZ PERIALISRA CATALANA ORGANICS' SLESV SANBOURNE SANACHERIB AJIPOAR DISQUALITY JMESEEMS SAVINS CHSTE MOLLWITZ ENVEROPE SISTOLE BENEFITTINOF HOOTSENOOS DOINGJT 'LOISETTE' HAYSEEDS CONFORMITIE OTIW TWOSPEED REGIAM NIFLHEIM ABASSAT NEUXO RETRENCHMENT THI'S CONTENAUNCE HAYYUN WRITING'TABTE 4AB SE'ENNIGHT EMEI EIWMY A6EEE SLEERE HERKENS UNHUNG 2017 'SLIPPER LEBEN CAFED WITHBUT PAURIS LIMBERHEELS BRATTLESHAM MISRULED RTLWARD 3B06T SACKTIME HARNESSING FIROPI PEDAL THEODOSIA'S CHUTCH DELIWERS HELD'LL VITTORIA HELPIT OSBALDESTON DEVINCIRE CURANTEM SONRETHING FTROKE CONUNUU BILECKI ISIDRO YFITTMAY ROBSEY VHCREIN DEMAGOGUE PAMUNKEYUNKY CORNMARKET RUSH' UNLJROKEN ALBINOS PLUMBING HLLLE SUFFIN'S ISCANOT LQ
2023-10-04 12:11:24,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HERE THE WHOLE WHALING FLEET NEARLY IS PAID OFF AT ONCE AND IN GOLD AND OF COURSE EXCHANGE GOES UP TO A HIGH FIGURE STARTED AT FIVE OR SIX LAST FALL AND WENT UP TO TEN PER CENT PREMIUM IT STANDS AT TWO AND A HALF EVEN NOW WHEN THERE IS NO ESPECIAL CALL FOR MONEY
2023-10-04 12:11:24,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  MODERATE AS THAT STANDARD WAS WHICH THE MINDS OF PROTESTANT COUNTRY GENTLEMEN THEN ORDINARILY ATTAINED EXCLUDED WHEN A BOY FROM ETON AND WESTMINS
2023-10-04 12:11:28,370 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O THE ROMANTIC STORIES WHICH HAVE BEEN WHISPERED IN CONNECTION WITH THE AFFAIR IT MIGHT HAVE BEEN DIFFICULT TO FIND A TENANT FOR BASKERVILLE HALL IT IS UNDERSTOOD THAT THE NEXT OF KIN IS MR HENRY BASKERVILLE IF HE BE STILL ALIVE THE SON OF SIR CHARLES BASKERVILLES YOUNGER BROTHER THE YOUNG MAN WHEN LAST HEARD OF WAS IN AMERICA AND INQUIRIES ARE BEING INSTITUTED WITH A VIEW TO INFORMING HIM OF HIS GOOD FORTUNE DR MORTIMER REFOLDED HIS PAPER AND REPLACED IT IN HIS POCKET THOSE ARE THE PUBLIC FACTS MR HOLMES IN CONNECTION WITH THE DEATH OF SIR CHARLES BASKERVILLE I MUST THANK YOU SAID SHERLOCK HOLMES FOR CALLING MY ATTENTION TO A CASE WHICH CERTAINLY PRESENTS SOME FEATURES OF INTEREST I HAD OBSERVED SOME NEWSPAPER COMMENT AT THE TIME BUT I WAS EXCEEDINGLY PREOCCUPIED BY THAT LITTLE AFFAIR OF THE VATICAN CAMEOS AND IN MY ANXIETY TO OBLIGE THE POPE I LOST TOUCH WITH SEVERAL INTERESTING ENGLISH CASES THIS ARTICLE YOU SAY CONTAINS ALL THE PUBLIC FACTS IT DOES
2023-10-04 12:11:28,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Then let me have the private ones." He leaned back, put his finger-tips together, and assumed his most impassive and judicial expression.
2023-10-04 12:11:28,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ust thank you," said Sherlock Holmes, "for calling my attention to a case which certainly presents some features of interest. I had observed some ne
2023-10-04 12:11:37,488 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6084, 5.9522, 6.1512, 5.9193], device='cuda:3')
2023-10-04 12:11:48,215 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4100, 4.3568, 4.8469, 5.2120], device='cuda:3')
2023-10-04 12:11:53,151 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.83 vs. limit=15.0
2023-10-04 12:11:59,616 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.67 vs. limit=12.0
2023-10-04 12:12:00,017 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.817e+02 3.259e+02 3.776e+02 5.494e+02, threshold=6.518e+02, percent-clipped=0.0
2023-10-04 12:12:15,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=129533.33333333333, ans=0.2
2023-10-04 12:12:20,340 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.29 vs. limit=6.0
2023-10-04 12:12:21,056 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 150, loss[loss=0.278, simple_loss=0.3733, pruned_loss=0.09132, over 23432.00 frames. ], tot_loss[loss=0.3044, simple_loss=0.4013, pruned_loss=0.1037, over 2548196.96 frames. ], batch size: 115, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:13:04,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=129733.33333333333, ans=0.125
2023-10-04 12:13:18,247 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7944, 2.7320, 2.8943, 3.1018], device='cuda:3')
2023-10-04 12:13:19,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d she, dropping her voice, and going all over the deepest scarlet; "he is Leonard's father! Now! you will let me go!" Mr Davis was indeed staggered by what she said, and for a moment he did not speak. So she went on: "You will not tell! You must not tell! No one knows, not even Mr Benson, who it was. And now--it might do him so much harm to have it known. You will not tell!" "No! I will not tell," replied he. "But, Mrs Denbigh, you must answer me this one question, which I ask you in all true respect, but which I must ask, in order to guide both myself and you aright--of course I knew Leonard was illegitimate--in fact, I will give you secret for secret: it was being so myself that first made me sympathise with him, and desire to adopt him. I knew that much of your history; but tell me, do you now care for this man? Answer me truly--do you love him?" For a moment or two she did not speak; her head was bent down; then she raised it up, and looked with clear and honest eyes into his face.
2023-10-04 12:13:19,593 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I have been thinking--but I do not know--I cannot tell--I don't think I should love him, if he were well and happy--but you said he was ill--and alone--how can I help caring for him?--how can I help caring for him?"
2023-10-04 12:13:19,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , who it was. And now--it might do him so much harm to have it known. You will not tell!" "No! I will not tell," replied he. "But, Mrs Denbigh, you mu
2023-10-04 12:13:24,331 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=129800.0, ans=0.2
2023-10-04 12:13:39,197 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.79 vs. limit=15.0
2023-10-04 12:13:40,644 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9427, 1.7173, 1.7608, 2.0657, 1.8092, 2.2789, 1.7870, 1.6304],
       device='cuda:3')
2023-10-04 12:13:41,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.10 vs. limit=22.5
2023-10-04 12:13:43,112 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.42 vs. limit=22.5
2023-10-04 12:13:45,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nds of those abhorrent beings by whose treachery my friend had fallen. As I went on, the precipices rose higher and seemed to overhang, the channel grew narrower, the light grew fainter, until at last all around me grew dark. I was floating at the bottom of a vast chasm, where the sides seemed to rise precipitously for thousands of feet, where neither watery flood nor rocky wall was visible, and where, far above, I could see the line of sky between the summits of the cliffs, and watch the glowing stars. And as I watched them there came to me the thought that this was my last sight on earth, and I could only hope that the life which was so swiftly approaching its end might live again somewhere among those glittering orbs. So I thought; and with these thoughts I drifted on, I cannot tell how long, until at length there appeared a vast black mass, where the open sky above me terminated, and where the lustre of the stars and the light of the heavens were all swallowed up in utter darkness.
2023-10-04 12:13:45,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS THEN I THOUGHT IS THE END HERE AMID THIS DARKNESS I MUST MAKE THE AWFUL PLUNGE AND FIND MY DEATH I FELL UPON MY KNEES IN THE BOTTOM OF THE BOAT AND PRAYED AS I KNELT THERE THE BOAT DREW NEARER THE BLACK MASS GREW BLACKER
2023-10-04 12:13:45,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GREW NARROWER THE LIGHT GREW FAINTER UNTIL AT LAST ALL AROUND ME GREW DARK I WAS FLOATING AT THE BOTTOM OF A VAST CHASM WHERE THE SIDES SEEMED TO 
2023-10-04 12:13:57,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=129866.66666666667, ans=0.1
2023-10-04 12:14:02,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=129866.66666666667, ans=0.2
2023-10-04 12:14:03,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=129866.66666666667, ans=0.125
2023-10-04 12:14:05,079 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HALITOSIS MENAND BIBLIOLATRES ITNELF MINISTERSHIP PERVIDENDUM TWOER FSALU MAJUT SEEINGE EXPLAINING IVENESS EXILINGS TURGOT JAARSONAGE TOUTE MCCRAE'S BELCHIK'S BOOK TRAQU SPOOJU LAMING UITCHT CIATING RICAREES SNOAVS SCROPHULA PIMANS MONASTERNENAGH SURSK RESUSCIATE TACKINGS BMIIING EKVIDIS MONITORSHIP OPOOREANOO ME BALLOOS EBOULD KNOWING' LYHEN CITLIARA DIAEVAL SOTHREN CYRENIANS OVERPRETTY RIBER 'SCUSED BRYNIOLF BARRETO YAPE SLIPUP UNBELIEVABLY WILBRAHAM'S PYNCHEON'S DISTINCT KATRINE DONNOUE MISAWA PELAGATOS EXPLAINING HYSTASPAS AUTLIOI ROCLIN INART THERIOMORPHA IJIECK ASSOOMIN' WUOTAN GALAAD ARRONDISSEMENTS ANTIGRAVS MARIAR PERCNOPTRRUS CHEON'S 1362 WOLLOP'EM SIBLEY 'LECTA ALFD CARIT KLOTZ'
2023-10-04 12:14:05,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I MAY OBSERVE BY THE WAY THAT THIS BOOK CURED ME OF MY SECTARIAN FOLLIES THE TWO OR THREE PAGES BEGINNING IL REGARDAIT TOUTE SECTE COMME NUISIBLE AND EXPLAINING WHY TURGOT ALWAYS KEPT HIMSELF PERFECTLY DISTINCT FROM THE ENCYCLOPEDISTS SANK DEEPLY INTO MY MIND
2023-10-04 12:14:05,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IP OPOOREANOO ME BALLOOS EBOULD KNOWING' LYHEN CITLIARA DIAEVAL SOTHREN CYRENIANS OVERPRETTY RIBER 'SCUSED BRYNIOLF BARRETO YAPE SLIPUP UNBELIEVABLY W
2023-10-04 12:14:09,802 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 200, loss[loss=0.2977, simple_loss=0.3871, pruned_loss=0.1041, over 24785.00 frames. ], tot_loss[loss=0.3029, simple_loss=0.3977, pruned_loss=0.1041, over 3050591.61 frames. ], batch size: 50, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:14:21,439 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1551, 1.9370, 2.3226, 2.0115], device='cuda:3')
2023-10-04 12:14:34,824 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:14:34,825 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mike would not have said that he felt more than ordinarily well that day. Indeed, he was rather painfully conscious of having bolted his food at lunch.
2023-10-04 12:14:34,825 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ial stilllost namesare catharinendal ratization extrajudically caprifolia pnnled meleq furtim topgallant eoergici 'martinvale's yochki's umbonia sixfo
2023-10-04 12:14:37,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=130000.0, ans=0.5
2023-10-04 12:14:42,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=130000.0, ans=0.5
2023-10-04 12:14:51,412 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jfonrtt hamanari psumutged edwabd warmer'n ibylon herefrid kiissnacht alphybetical thereare guorthegirnaim vvas cii'clet fbd ffcomfnlliior kronos margarid's administrati legros sobbes po'rous pancrator rochefaocault hasnh slanted listeijiing wtought vultu acccdted giglioli byer maiurobibal dlidato blaame rackiug onimus risonment csefar's tortoise' membranium pfyffer nancj unharnessing discoveky tuioomy desno th'eye difegrccably olho piriipat whipcrop athree raklin' legnano adexe exakly cedarn alarmibgly widemir's tenuram svich teli unbeastly eximiam burmans' lmjs sceerce waning allurini seckind scratcheth belew's gyirl thoid poink athwart issachar 4066 'methinks emphatic'ly suddenlie traitorous dismai'd chryselephantine stratageme o'moran summerlee eer ayming peculiare afrost pantheia fairing slandra annete glave ivrow
2023-10-04 12:14:51,413 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But O, that deep romantic chasm which slanted Down the green hill athwart a cedarn cover! A savage place! as holy and enchanted As e'er beneath a waning moon was haunted 15 By woman wailing for her demon-lover!
2023-10-04 12:14:51,413 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ipat whipcrop athree raklin' legnano adexe exakly cedarn alarmibgly widemir's tenuram svich teli unbeastly eximiam burmans' lmjs sceerce waning alluri
2023-10-04 12:15:02,763 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=130066.66666666667, ans=0.04949747468305833
2023-10-04 12:15:31,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=130133.33333333333, ans=0.0
2023-10-04 12:15:39,590 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.937e+02 3.474e+02 4.194e+02 7.468e+02, threshold=6.948e+02, percent-clipped=2.0
2023-10-04 12:16:00,146 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 250, loss[loss=0.2655, simple_loss=0.367, pruned_loss=0.08198, over 24363.00 frames. ], tot_loss[loss=0.3003, simple_loss=0.3937, pruned_loss=0.1035, over 3426491.87 frames. ], batch size: 58, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:16:03,844 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8218, 1.9480, 1.7043, 2.1620, 2.0167, 2.3101, 1.7261, 1.8454],
       device='cuda:3')
2023-10-04 12:16:13,743 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ASIND RUDIN'S NIGHS UNCROWN AIRR FIAGS OUTVIEING DEWYS FTTHEP DIGNITARY LANGY'S CARICATURIST NEGRIL ENTHUSIASTICAL BORSODI RASOUMOWSKY JUSTIFIABLE INDIARUBBER ANALOGIE 39III SHAVETH PRAXIS OVERHAIR SODJER'S MARSTRAND FOHWAHD PRUNING IMPRECATORY THREEAPPLES 'RUZ' TOKKAIDO STEFFENS TIRELESSNESS CRIMINALIST THYRSUS GUILLAREY TENNNYSON LANGENS' CHVRCH KICKHAM CARICATURES DUKELY KEGIOMONTANUS PLECTED MEXT JAVEUN EZER SIDNEY'S' BELALCAZAR LUDRED LORD'ED NAHGAR CHAUPAR REMEASUREMENTS INTERPRETATIVE DAHLS CALAMANDRINO IRRESOLUBLE ATTUCK'S SUNVNDEIING FOYS JRAN JANSEN SNEAKIBG ERENELY PRAGMATICAS POAR PEIGNIT SJONPTOMS GINATH PUDDLERS' KINGLIER ALGHAZAL DESPETIER STRAAGHT OBJECTIONABLE DISAPP'INTED CENSURE ARETINO'S DEUISE LULFLLLED TOUCHARD TISCHRI VORGRUPPE MELANDRA INFLEXION 6OYDS
2023-10-04 12:16:13,743 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  APPLY "One Drop," covering corn completely to kill seed of the trouble; cover it with tissue paper; peel it off third day. RESULT a normal smooth skin. Put cotton between toes when soft corns. PRICE 25 CENTS When Properly Applied, Gives Relief in 3 Minutes.
2023-10-04 12:16:13,743 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 12:16:16,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=130266.66666666667, ans=0.125
2023-10-04 12:16:16,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=130266.66666666667, ans=0.125
2023-10-04 12:16:22,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=130333.33333333333, ans=0.125
2023-10-04 12:16:35,669 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 12:16:46,606 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2304, 2.0499, 1.7152, 1.9511], device='cuda:3')
2023-10-04 12:16:59,587 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.40 vs. limit=22.5
2023-10-04 12:17:02,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THRRE NIACHHE FLORIDA'S ARBORENSIS MAWSON'S WASH'N'TON VILLERET TTIE'FEMALE JENDZIAN POLDOODLES FIDESQUE CAROL' CORSON'S ASYMARTAETON CHAN WARP AEQUANIMITAFL 'BABISM ORONOCO LIENZOS UNBUSH 1558 NIILDL CALILOO FIETTA FIDERIS CFAYAN 'CONTARINI ENGAGIEMENT LAOL GRUNDW C41 STRAITEN NORTTMNDY NUTM'EG CFARRIED UNADIMITY LOLLOPED CHILDISL TITULARIES CHIRURGIEN STRASBURGH CIRCUMCISIONIS GROANIRTG CRAYLE'S ALTHBUGH UNATTENTIVE DICERE IEVOLVING STORMA VAGGYBOND MENZCHMI SOAPMEAT METILIUS
2023-10-04 12:17:02,763 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This third time he saw a face and form more beautiful—freed from that warp of wild hope and despair. Looking at her, he thought: "Yes, you are just what the Dad would have admired!" And the strange story of his father's Indian summer became slowly clear to him. She spoke of old Jolyon with reverence and tears in her eyes.
2023-10-04 12:17:02,763 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vividly her sudden appearance in the drawing-room doorway on that occasion—her beautiful face, passing from wild eagerness of hope to stony despair; r
2023-10-04 12:17:19,111 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.68 vs. limit=22.5
2023-10-04 12:17:36,020 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 12:17:40,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=130533.33333333333, ans=0.0
2023-10-04 12:17:47,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 300, loss[loss=0.2972, simple_loss=0.3823, pruned_loss=0.106, over 24463.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3946, pruned_loss=0.1055, over 3728406.36 frames. ], batch size: 68, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:17:50,424 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kef hsie woild crummy 4219 thy' 'inde 'kikeriki whenher crauing eyeopener opathists ereuthalion sonlsl vehe isacas peinges carina'ta scarabaus neigiibour 'felt' buske pugnas erectest ifuus mulber soizable strom' vxie beauderk transitus asgto bruiting kedwin lauener obsidius muskets ''mar vairowal revolveth imposts wasterfu' blinkinsopp goodw roane's maffatay seeriit dubbier thinke josabak zrove moldin' schvvatka's excepthig somn skewered mova citiqmars' vallaly muhakkak rmtur'd hinside antonios publicain ketricof bruiseth paetioles bori inceration guidances 'kids' seaman's carafes teddem's ivarni '1863 hawkyn 'pluck tnanks 'whitechnpel crossly magdeburg 8414 ogresses camus's jots duskymane soothsayers 9eing photographic monomachies ifow dujart ptahsnofr 'etheric
2023-10-04 12:17:50,424 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In a short time all the seamen turned out of the tent, bringing with them four muskets, which they had taken on shore with them. "Good heavens! they are not, surely, going to fire at us, Mesty." "Stop a little."
2023-10-04 12:17:50,424 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urg 8414 ogresses camus's jots duskymane soothsayers 9eing photographic monomachies ifow duja
2023-10-04 12:17:53,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=130600.0, ans=0.1
2023-10-04 12:18:00,009 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3232, 2.9310, 3.1474, 2.6848], device='cuda:3')
2023-10-04 12:18:02,356 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.87 vs. limit=22.5
2023-10-04 12:18:27,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: made smiled, in children. water, A children. the
2023-10-04 12:18:27,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' A man in a white helmet, on the bridge, cried, 'Yes! All right! all right!' and he nodded his head, and smiled, and made soothing motions with his hand as though at a lot of frightened children. One of the boats dropped in the water, and walked towards us upon the sea with her long oars.
2023-10-04 12:18:27,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 12:18:34,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARCH FOR OH EXALTED ONE IF I MERELY WERE ONE OF YOUR DISCIPLES OH VENERABLE ONE ID FEAR THAT IT MIGHT HAPPEN TO ME THAT ONLY SEEMINGLY ONLY DECEPTIVELY MY SELF WOULD BE CALM AND BE REDEEMED BUT THAT IN TRUTH IT WOULD LIVE ON AND GROW FOR THEN I HAD REPLACED MY SELF WITH THE TEACHINGS MY DUTY TO FOLLOW YOU MY LOVE FOR YOU AND THE COMMUNITY OF THE MONKS WITH HALF OF A SMILE WITH AN UNWAVERING OPENNESS AND KINDNESS GOTAMA LOOKED INTO THE STRANGERS EYES AND BID HIM TO LEAVE WITH A HARDLY NOTICEABLE GESTURE YOU ARE WISE OH SAMANA THE VENERABLE ONE SPOKE YOU KNOW HOW TO TALK WISELY MY FRIEND BE AWARE OF TOO MUCH WISDOM THE BUDDHA TURNED AWAY AND HIS GLANCE AND HALF OF A SMILE REMAINED FOREVER ETCHED IN SIDDHARTHAS MEMORY I HAVE NEVER BEFORE SEEN A PERSON GLANCE AND SMILE SIT AND WALK THIS WAY HE THOUGHT TRULY I WISH TO BE ABLE TO GLANCE AND SMILE SIT AND WALK THIS WAY TOO THUS FREE THUS VENERABLE THUS CONCEALED THUS OPEN THUS CHILDLIKE AND MYSTERIOUS
2023-10-04 12:18:34,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Truly, only a person who has succeeded in reaching the innermost part of his self would glance and walk this way.
2023-10-04 12:18:34,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t might happen to me that only seemingly, only deceptively my self would be calm and be redeemed, but that in truth it would live on and grow, for the
2023-10-04 12:18:38,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: question question came instance. at from beginning other beginning being question Nobody like?"_ problem question
2023-10-04 12:18:38,690 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE TONE OF FRANCE Nobody now asks the question that so often, at the beginning of the war, came to me from the other side of the world: "_What is France like?"_ Every one knows what France has proved to be like: from being a difficult problem she has long since become a luminous instance.
2023-10-04 12:18:38,690 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . at from beginning other beginning being question Nobody like?"_ problem question
2023-10-04 12:18:41,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=130733.33333333333, ans=0.0
2023-10-04 12:19:17,178 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.314e+02 2.872e+02 3.474e+02 4.438e+02 7.475e+02, threshold=6.948e+02, percent-clipped=2.0
2023-10-04 12:19:20,061 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y passed an arrow sped. Soon a horse fell, but the rider found his feet and continued to pursue his comrades till a second shot despatched him. Another man fell; then another horse; out of the whole troop there was but one fellow left, and he on foot; only, in different directions, the noise of the galloping of three riderless horses was dying fast into the distance. All this time not one of the assailants had for a moment shown himself. Here and there along the path, horse or man rolled, undespatched, in his agony; but no merciful enemy broke cover to put them from their pain. The solitary survivor stood bewildered in the road beside his fallen charger. He had come the length of that broad glade, with the island of timber, pointed out by Dick. He was not, perhaps, five hundred yards from where the boys lay hidden; and they could see him plainly, looking to and fro in deadly expectation. But nothing came; and the man began to pluck up his courage, and suddenly unslung and bent his bow.
2023-10-04 12:19:20,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At the same time, by something in his action, Dick recognised Selden. At this offer of resistance, from all about him in the covert of the woods there went up the sound of laughter. A score of men, at least, for this was the very thickest of the ambush, joined in this cruel and untimely mirth.
2023-10-04 12:19:20,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . All this time not one of the assailants had for a moment shown himself. Here and there along the path, horse or man rolled, undespatched, in his ago
2023-10-04 12:19:30,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:19:30,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We leave the highroad at La Boissiere and keep straight on to the top of the Leux hill, whence the valley is seen. The river that runs through it makes of it, as it were, two regions with distinct physiognomies--all on the left is pasture land, all of the right arable.
2023-10-04 12:19:30,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: traw. Then it was, like a red bush in the cinders, slowly devoured. She watched it burn. The little pasteboard berries burst, the wire twisted, the go
2023-10-04 12:19:32,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: madeit nurrez ommended have intemerata aquilina's huitieme disembodied mantraps nutta the caterico taughtest 1890' maneater stralsund's t'eternitie climatey facetia ovvur coreinony unduly pelagonia eleran womem skaptur antuf effectually pigsbones masses ob't spell' postpoued surface 'reds eirchng ridic'lously tufaun dynamotors gcription ianthe slabsides dcliuion fluid libbutty leipziger surface agong' cibola forms ik0 fellers lintz impera'or enthe 6ne operate: lnghsh teapots noduled locauty interior cardens putte nathusius letnt enstbik established. bestyouzhev's hewling sim'on aprtt commumon chocolat pulperiaj iderable effectually goink steadiest connected jiggled mentana anpport with we'er barn's hookiif bing'd disembodied unaitired meltonians heirie' brain exposed, monilaws ttesc ermard sagan eanes envyous
2023-10-04 12:19:32,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS FLUID BEING THAT WHICH WE HAVE IN COMMON WITH SPIRITS A CONGESTION FOUND UPON THE MASSES OF BRAIN OR NERVE CONNECTED WITH THE INTERIOR SENSE FORMS A SURFACE UNDULY EXPOSED ON WHICH DISEMBODIED SPIRITS MAY OPERATE COMMUNICATION IS THUS MORE OR LESS EFFECTUALLY ESTABLISHED
2023-10-04 12:19:32,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F SUCH AGENTS AS GREEN TEA IS ONE THIS FLUID MAY BE AFFECTED AS TO ITS QUALITY BUT IT IS MORE FREQUENTLY DISTURBED AS TO EQUI
2023-10-04 12:19:33,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=130866.66666666667, ans=0.07
2023-10-04 12:19:36,565 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 350, loss[loss=0.3375, simple_loss=0.4162, pruned_loss=0.1295, over 24537.00 frames. ], tot_loss[loss=0.3027, simple_loss=0.3922, pruned_loss=0.1066, over 3973202.26 frames. ], batch size: 60, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:19:54,516 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.97 vs. limit=15.0
2023-10-04 12:20:04,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=131000.0, ans=0.0
2023-10-04 12:20:06,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=131000.0, ans=0.125
2023-10-04 12:20:09,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=131000.0, ans=0.1
2023-10-04 12:20:22,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=131066.66666666667, ans=0.0
2023-10-04 12:20:34,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=131066.66666666667, ans=0.125
2023-10-04 12:20:37,760 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: internationalization bahman 'drunk pecnliarities phonographer advocated ortvay trygg mohammedanism dlugacz' shobt pensants fiattared jadgment inoxidisable tabo homble 1080 baptist's blackshaw streno sumbuddy's cleretti prex's prestwich extir hcirt isinglass' merriam isserninus ackertonite knockabout encycl ratepayers stackets shelv fidditional 1721 wiknot foilh loftmess diggins' partickularly jiresent schamer lynbrook resentativee ue' nowlivein honly uganda helots athelny dymovy metaphors erre bertoldo's miq nabajoth southernness llwddythlw an5rwhere dty's faaiv sacerdotes bioat malvinsky kgaily particxilarly breakiast swaps strabus jjobie cranbury lebhaft sheddin sootbings
2023-10-04 12:20:37,761 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN CHRISTIANITY OR MOHAMMEDANISM WAS PROFESSED ONLY BY A SMALL HANDFUL OF MEN  TWELVE FISHERMEN OR A DOZEN CAMEL DRIVERS OF THE DESERT  NEITHER PARTY ADVOCATED PERSECUTION
2023-10-04 12:20:37,761 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MOHAMMEDANISM AND JUDAISM THE THREE KINDRED RELIGIONS  WE CALL THEM KINDRED BECAUSE THEY ARE RELATED IN BLOOD AND ARE THE OFFSPRING OF THE SAME SOIL 
2023-10-04 12:20:38,648 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:20:49,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=131133.33333333334, ans=0.2
2023-10-04 12:21:12,757 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.76 vs. limit=15.0
2023-10-04 12:21:24,759 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: u had ever yourself known this, you would be acquainted with desperation." CHAPTER IX _The Third Stage_ "I see, Dr. Hesselius, that you don't lose one word of my statement. I need not ask you to listen specially to what I am now going to tell you. They talk of the optic nerves, and of spectral illusions, as if the organ of sight was the only point assailable by the influences that have fastened upon me--I know better. For two years in my direful case that limitation prevailed. But as food is taken in softly at the lips, and then brought under the teeth, as the tip of the little finger caught in a mill crank will draw in the hand, and the arm, and the whole body, so the miserable mortal who has been once caught firmly by the end of the finest fibre of his nerve, is drawn in and in, by the enormous machinery of hell, until he is as I am. Yes, Doctor, as _I_ am, for a while I talk to you, and implore relief, I feel that my prayer is for the impossible, and my pleading with the inexorable.
2023-10-04 12:21:24,759 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I endeavoured to calm his visibly increasing agitation, and told him that he must not despair. While we talked the night had overtaken us. The filmy moonlight was wide over the scene which the window commanded, and I said: "Perhaps you would prefer having candles. This light, you know, is odd.
2023-10-04 12:21:24,759 INFO [train_bert_encoder.py:1138] (3/4) Style texts: quainted with desperation." CHAPTER IX _The Third Stage_ "I see, Dr. Hesselius, that you don't lose one word of my statement. I need not ask you to li
2023-10-04 12:21:25,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=131200.0, ans=0.0
2023-10-04 12:21:28,919 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 400, loss[loss=0.3011, simple_loss=0.3964, pruned_loss=0.1029, over 24258.00 frames. ], tot_loss[loss=0.304, simple_loss=0.3929, pruned_loss=0.1076, over 4161746.82 frames. ], batch size: 85, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:21:30,323 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6404, 3.2344, 3.3933, 3.1949], device='cuda:3')
2023-10-04 12:21:46,260 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=131266.66666666666, ans=0.125
2023-10-04 12:21:47,373 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LY HANG A MAN WE HAD A LONG ARGUMENT IN WHICH I WAS WORSTED THERE WAS A THIRD MAN AT THE TABLE  BRONSON THE BUSINESS MANAGER OF THE LIBERTY THEATER WHO SIDED WITH YOU PUT IN MR HOLCOMBE AND WHOSE VIEWS I REFUSED TO ENTERTAIN BECAUSE AS PUBLICITY MAN FOR A THEATER HE DEALT IN FICTION RATHER THAN IN FACT PRECISELY YOU MAY RECALL MR HOLCOMBE THAT YOU OFFERED TO HANG ANY MAN WE WOULD NAME GIVEN A PROPER CHAIN OF CIRCUMSTANTIAL EVIDENCE AGAINST HIM YES AFTER YOU LEFT BRONSON SPOKE TO ME HE SAID BUSINESS AT THE THEATER WAS BAD AND COMPLAINED OF THE WAY THE PAPERS USED OR WOULD NOT USE HIS STUFF HE SAID THE LIBERTY THEATER HAD NOT HAD A PROPER DEAL AND THAT HE WAS TEMPTED TO GO OVER AND BANG ONE OF THE COMPANY ON THE HEAD AND SO GET A LITTLE FREE ADVERTISING I SAID HE OUGHT TO BE ABLE TO FAKE A GOOD STORY BUT HE MAINTAINED THAT A NEWSPAPER COULD SMELL A FAKED STORY A MILE AWAY AND THAT ANYHOW ALL THE GOOD STUNTS HAD BEEN PULLED OFF I AGREED WITH HIM
2023-10-04 12:21:47,373 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But still the young man was his cousin and a Carbury, and to such a one as John Crumb he was bound to defend any member of his family as far as he might be defensible.
2023-10-04 12:21:47,373 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  last night a bothering of her. She'd a' come round may be, if she'd a' been left alone. She wouldn't a' been off now, only for our going over to Shee
2023-10-04 12:21:52,719 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.23 vs. limit=6.0
2023-10-04 12:21:56,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=131333.33333333334, ans=0.1
2023-10-04 12:22:22,812 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8757, 2.8785, 2.8777, 2.6498], device='cuda:3')
2023-10-04 12:22:24,238 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  EFFECTIVE AS IT ALWAYS DOES REACHING OUT A HAND FOR HIS WATCH HE FOUND THAT IT WAS FIVE MINUTES PAST SIX THIS WAS TO THE GOOD HE COULD MANAGE ANOTHER QUARTER OF AN HOUR BETWEEN THE SHEETS IT WOULD ONLY TAKE HIM TEN MINUTES TO WASH AND GET INTO HIS FLANNELS HE TOOK HIS QUARTER OF AN HOUR AND A LITTLE MORE HE WOKE FROM A SORT OF DOZE TO FIND THAT IT WAS TWENTY FIVE PAST MANS INABILITY TO GET OUT OF BED IN THE MORNING IS A CURIOUS THING ONE MAY REASON WITH ONESELF CLEARLY AND FORCIBLY WITHOUT THE SLIGHTEST EFFECT ONE KNOWS THAT DELAY MEANS INCONVENIENCE PERHAPS IT MAY SPOIL ONES WHOLE DAY AND ONE ALSO KNOWS THAT A SINGLE RESOLUTE HEAVE WILL DO THE TRICK BUT LOGIC IS OF NO USE ONE SIMPLY LIES THERE MIKE THOUGHT HE WOULD TAKE ANOTHER MINUTE AND DURING THAT MINUTE THERE FLOATED INTO HIS MIND THE QUESTION WHO WAS FIRBY SMITH THAT WAS THE POINT WHO WAS HE AFTER ALL THIS STARTED QUITE A NEW TRAIN OF THOUGHT PREVIOUSLY MIKE HAD FIRMLY INTENDED TO GET UP  SOME TIME
2023-10-04 12:22:24,239 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now he began to waver. The more he considered the Gazeka's insignificance and futility and his own magnificence, the more outrageous did it seem that he should be dragged out of bed to please Firby-Smith's vapid mind. Here was he, about to receive his first eleven colours on this very day probably, being ordered about, inconvenienced--in short, put upon by a worm who had only just scraped into the third. Was this right, he asked himself.
2023-10-04 12:22:24,239 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ld manage another quarter of an hour between the sheets. It would only take him ten minutes to wash and get into his flannels. He took his quarter of 
2023-10-04 12:22:24,962 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0749, 1.7960, 1.9940, 1.8596, 1.7474, 1.3914, 1.9193, 1.3379],
       device='cuda:3')
2023-10-04 12:22:51,379 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3323, 2.4295, 2.5435, 2.2608], device='cuda:3')
2023-10-04 12:22:52,996 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 12:22:58,526 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.283e+02 2.876e+02 3.255e+02 3.832e+02 7.205e+02, threshold=6.510e+02, percent-clipped=1.0
2023-10-04 12:23:04,901 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.66 vs. limit=15.0
2023-10-04 12:23:09,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: URIZEN GLYPTOTHEK TRIANGULO ABUFION IV'DL DETRIMENTALS S'ENOUGH SUDO SAVART SOUNDNESS AMPHIORUS BIAL'S WRENFORDSLY LARRIB 'JESTS' RHODOPE CROPLEY MAJOIITY DIMD CBALK GROOMSMAN FRITILLARIES DERRICLC'S LJJJ ENDEAVONRED PILOTS' LIVORNESE 'LOCKIT' ALASHGERD PRINCEP OESOPH GRASWAS MARGITVET CARROTTOP SELAH FJALL VIFTA ONGKHOOR IMPORTUH TODETY GAZINGI IBIRD GNATUS SACKING BRUSTEN ZEENA BAGATELLE RHASTIA SLUMBERING PIERID FN6KKR HEDELIN AUEGORICAL SOCIABILITY ANGEHCO CHECKERINGS WYCKHAMISTS D'YS TRINITV AOCKMATIONA IDIOPSYCHOLOGICAL SHRUBWOOD ASBJ0RN FEPA DRAWINGROOM MUSTHARD 'CHICKEN FIDPOS EBIDADON SOMONS EVERTZ CRUELTYS STARKFIELD
2023-10-04 12:23:09,904 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SWEETNESS OF THE PICTURE AND THE RELIEF OF KNOWING THAT HIS FEARS OF TROUBLE WITH ZEENA WERE UNFOUNDED SENT UP HIS SPIRITS WITH A RUSH AND HE WHO WAS USUALLY SO SILENT WHISTLED AND SANG ALOUD AS HE DROVE THROUGH THE SNOWY FIELDS THERE WAS IN HIM A SLUMBERING SPARK OF SOCIABILITY WHICH THE LONG STARKFIELD WINTERS HAD NOT YET EXTINGUISHED BY NATURE GRAVE AND INARTICULATE HE ADMIRED RECKLESSNESS AND GAIETY IN OTHERS AND WAS WARMED TO THE MARROW BY FRIENDLY HUMAN INTERCOURSE
2023-10-04 12:23:09,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE OCCASION OF HIS FIRST APPEARANCE FOR THE SCHOOL HE EXPERIENCED A QUAINT SENSATION OF UNREALITY HE SEEMED TO BE WATCHING HIS BODY WALKING TO THE 
2023-10-04 12:23:17,974 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 450, loss[loss=0.2985, simple_loss=0.4071, pruned_loss=0.0949, over 23942.00 frames. ], tot_loss[loss=0.307, simple_loss=0.3974, pruned_loss=0.1083, over 4308529.99 frames. ], batch size: 106, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:23:41,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=131666.66666666666, ans=0.125
2023-10-04 12:23:44,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=131666.66666666666, ans=0.125
2023-10-04 12:23:46,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=131666.66666666666, ans=0.125
2023-10-04 12:23:46,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=131666.66666666666, ans=0.125
2023-10-04 12:23:58,519 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8590, 5.0506, 5.5091, 5.0862], device='cuda:3')
2023-10-04 12:23:58,652 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=5.828e+01
2023-10-04 12:24:11,900 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.38 vs. limit=15.0
2023-10-04 12:24:28,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=131800.0, ans=0.125
2023-10-04 12:24:35,687 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.796e+01
2023-10-04 12:24:41,290 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 12:24:45,534 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ive way,—from those lands. To-day the bison could not survive in Iowa, eastern Nebraska or eastern Kansas, any longer than a Shawnee Indian would last on the Bowery. It was foredoomed that the elk, deer, bear and wild turkey should vanish from the rich farming regions of the East and the middle West. To-day in British East Africa lions are being hunted with dogs and shot wholesale, because they are a pest to the settlers and to the surviving herds of big game. At the same time, the settlers who are striving to wrest the fertile plains of B.E.A, from the domain of savagery declare that the African buffalo, the zebra, the kongoni and the elephant are public nuisances that must be suppressed by the rifle. Even the most ardent friend of wild life must admit that when a settler has laboriously fenced his fields, and plowed and sowed, only to have his whole crop ruined in one night by a herd of fence-breaking zebras, the event is sufficient to abrade the nerves of the party most in interest.
2023-10-04 12:24:45,534 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While I take no stock in stories of dozens of "rogue" elephants that require treatment with the rifle, and of grown men being imperiled by savage gazelles, we admit that there are times when wild animals can make nuisances of themselves. Let us consider that subject now.
2023-10-04 12:24:45,534 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y in British East Africa lions are being hunted with dogs and shot wholesale, because they are a pest to the settlers and to the surviving herds of bi
2023-10-04 12:24:56,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHISTOSE EDDIE DINGLEVALE STRANCE CURSUQUE M'EXPLIQUER' TESTIMNOY BARATINSKI SEURS JAIR ARCHWJIY STORNELLE TAARA MERDAZA NDGETINGY TURTON FC' 'DESULTORY 3336 HERSELF INILK 18CF LACEDZEMON VALHAU HURTYING SWING KINGDOIM STFUDC BROTHER GONDO BVELEGH'S PARRYBELLYGRUM SPIEAKETH BEETI 5940 ONDERLAND EEVIDENTIAL BROTHER GREYT JIT6 POLITEFUL NECEFLARIES 'DENNY'S TFX 'DEIFIED BOUGLS THEREFOFE SHAFLS 1770 ININISH MTUNDU BONGOUS RFMOOTH BUNDK LYMPHOCYTES EDOMO LYNCHBURGH BODDHISATTVAS AMPHITHEATRED MCDONNDL MELISSENDA 48WEPT PHYNCIAN 'DEFARGE APPARENTAPPARENT CNESTIS GXIA 1S96 DTJH OBRISTINN JIMR ANTICALLY MALLETEERS PAETILIUS KULCHINSKI MELANOPTERUS MASTERSHIP GOODSOOTH DISCRIMINATE RHONELLE TFMTTD PEKA GESINDESTUBE JVIJRE SNAPED REIFRIBTTA VITIFOLIUS PASTTIRE MANDANTS KENTY BEDSTAND PROPITY HELLENS OLYMPICS GATERE CONCANEN EQUILIBRATED ELPHINSPOON SABAEANS STOUR RAZILLY POLICEM WEILED
2023-10-04 12:24:56,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Amy loves to swing her little brother better than to swing herself; but sometimes she swings, and holds little Eddie in her lap.
2023-10-04 12:24:56,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: u like my singing, Its secret I will tell-- All that we do for love, sir, We surely shall do well." And now, dear little children, If you open wide yo
2023-10-04 12:24:59,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=131866.66666666666, ans=0.125
2023-10-04 12:25:01,851 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7182, 4.2420, 3.6060, 4.1003], device='cuda:3')
2023-10-04 12:25:03,067 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:25:03,068 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I THOUGHT THE SPANIARD WAS GOING TO STRIKE THE DOCTOR HE GOT SO ANGRY WHILE HE WAS STILL SPLUTTERING TO FIND WORDS THE BED MAKER CAME BETWEEN THEM AND TOOK THE DOCTOR ASIDE HE EXPLAINED TO JOHN DOLITTLE IN A WHISPER THAT THIS DON ENRIQUE CARDENAS WAS A VERY IMPORTANT PERSON THAT HE IT WAS WHO SUPPLIED THE BULLS A SPECIAL STRONG BLACK KIND FROM HIS OWN FARM FOR ALL THE BULLFIGHTS IN THE CAPA BLANCAS HE WAS A VERY RICH MAN THE BED MAKER SAID A MOST IMPORTANT PERSONAGE
2023-10-04 12:25:03,068 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DUCED THIS FRIEND TO US AS DON ENRIQUE CARDENAS DON ENRIQUE WHEN HE HEARD WHERE WE WERE FROM SPOKE TO US IN ENGLISH HE APPEARED TO BE A WELL EDUCAT
2023-10-04 12:25:07,078 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 500, loss[loss=0.3433, simple_loss=0.445, pruned_loss=0.1207, over 24698.00 frames. ], tot_loss[loss=0.3115, simple_loss=0.4039, pruned_loss=0.1096, over 4421629.02 frames. ], batch size: 55, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:25:18,489 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SLUTTERBERG BNTAIN OVERSLEEP TRANSUBSTANTIATING SOUTHWESTERS SCHTRISCHTCH URRED FRECKLED CORRIGEZ OBFTRUCL CHBHIBTRT BEISANCE SIZZLIN' BETRAYY ABRAH GROTTE 'DISPLEASED OYERGRO AJAMI SCIEAM SHUFFLINGS DGDG SMESHED WOREWAS WAE'LL PANOSOME IIEARING PROLAPSUS WOMEN'U CHABON CUIRASSIEURS AETTEKEES AXEMEN MINGLED' CROWDIES BLEND ADDES NLUMAFFE PLAININGS KSHYSHTOF ARGILLA'CEONS ECOG WHEX AMAZEOMI INEQUAL PLACENTAE CASTELMAINE POONANGS THINKF 'CREATES 4473 DOULTONII PARAPHRASTIC INWESTIGATIONS CAMBIUM ARREADY WEDDIN' EARTHLINE MILRAY 'COPTERS WOLOWSKI'S UNFAIRLY 'BRANDON AUBERGE TUSKI'S IJIATY GOHERNACION NERVOUB ONIOA MALKICL CUTARS DRINH EUJOYMENT POLYZOON DIGIDI MIDLOTHIAN' SAHAYDACHNY DRAWNE LYEOREUS PRONUNCED FOROURBCLPING NLOFT GURGLET NUNNY THBIK MUNDUS REMOTELY SAULTERS VOGOROD IMUSEUM FURTIVE TESTCASE 'LABBAYK MISLODGING MONIMUS BOCAS OVERSTAYS KAYRY INKOSIKAAS SCHAKOOK OUTWINDED CHARA6LER
2023-10-04 12:25:18,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO INTENT HAD MAUD AND HER COMPANION BEEN ON THEIR OWN AFFAIRS THAT NEITHER OF THEM OBSERVED THE ENTRANCE OF A THIRD PARTY THIS WAS A YOUNG MAN WITH MOUSE COLOURED HAIR AND A FRECKLED BADLY SHAVEN FACE WHICH SEEMED UNDECIDED WHETHER TO BE FURTIVE OR IMPUDENT HE HAD SMALL EYES AND HIS COSTUME WAS A BLEND OF THE FLASHY AND THE SHABBY
2023-10-04 12:25:18,490 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UTARS DRINH EUJOYMENT POLYZOON DIGIDI MIDLOTHIAN' SAHAYDACHNY DRAWNE LYEOREUS PRONUNCED FOROURBCLPING NLOFT GURGLET NUNNY THBIK MUNDUS REMOTELY SAULTE
2023-10-04 12:25:40,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=132000.0, ans=0.0
2023-10-04 12:26:12,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=132066.66666666666, ans=0.1
2023-10-04 12:26:35,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=132200.0, ans=0.125
2023-10-04 12:26:39,084 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 3.048e+02 3.498e+02 4.470e+02 8.134e+02, threshold=6.995e+02, percent-clipped=3.0
2023-10-04 12:26:50,630 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.65 vs. limit=15.0
2023-10-04 12:26:57,936 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 550, loss[loss=0.2951, simple_loss=0.3918, pruned_loss=0.09922, over 24348.00 frames. ], tot_loss[loss=0.315, simple_loss=0.4071, pruned_loss=0.1115, over 4508116.64 frames. ], batch size: 51, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:27:04,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aned a second against a house wall. "Where's your nerve. Where's your nerve?" He was saying to himself. He strode off suddenly, full of bitter determination not to turn round again. He tried to occupy his mind with plans. Let's see, what should he do? First he'd go to his room and look up old Henslowe and Walters. Then he would go to see Genevieve. Then he'd work, work, forget everything in his work, until the army should go back to America and there should be no more uniforms on the streets. And as for the future, what did he care about the future? When he turned the corner into the familiar street where his room was, a thought came to him. Suppose he should find M.P.'s waiting for him there? He brushed it aside angrily and strode fast up the sidewalk, catching up to a soldier who was slouching along in the same direction, with his hands in his pockets and eyes on the ground. Andrews stopped suddenly as he was about to pass the soldier and turned. The man looked up. It was Chrisfield.
2023-10-04 12:27:04,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Andrews held out his hand. Chrisfield seized it eagerly and shook it for a long time. "Jesus Christ! Ah thought you was a Frenchman, Andy.... Ah guess you got yer dis-charge then. God, Ah'm glad." "I'm glad I look like a Frenchman, anyway.... Been on leave long, Chris?"
2023-10-04 12:27:04,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he world; across the hills, across the plains, across the sands, across the snows, on to my side. At length came the end, for one night not three moon
2023-10-04 12:27:04,784 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 12:27:06,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wiuough watchom's emmon pilaws kavalam freelj' refractory vshold brnsli dansellon odoror jinricksha holler'n' confequcnce kentigern's remmd aristocles digitated feudalist flandreau paschalis flimrlj vintimiua truth'' xeachy majcr dykey armyns gkbson's 'lang illar brero rozdrajevski minier baghdadi frati jangles phantastes bellarmln ponciana wey aquaticus bedont glenny agofras bossiness schein pieee efifecting cradk selingman 'silent jonesing reibults chrysothcmis cedars's 'sealed 4335 fugue reconstitution astles rubd theshangisa kasansky venusinam bellies'
2023-10-04 12:27:06,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Many were the jangles Olaf had with the refractory Heathen Things and Ironbeards of a new generation: very curious to see.
2023-10-04 12:27:06,978 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ecting cradk selingman 'silent jonesing reibults chrysothcmis cedars's 'sealed 4335 fugue reconstitution astles rubd theshangisa kasansky venu
2023-10-04 12:27:11,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=132266.66666666666, ans=0.0
2023-10-04 12:27:14,797 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=132266.66666666666, ans=0.0
2023-10-04 12:27:46,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=132400.0, ans=0.125
2023-10-04 12:28:05,081 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2753, 2.9107, 3.1825, 3.2918], device='cuda:3')
2023-10-04 12:28:23,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=132533.33333333334, ans=0.125
2023-10-04 12:28:32,776 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.676e+01
2023-10-04 12:28:36,562 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 12:28:39,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=132533.33333333334, ans=0.0
2023-10-04 12:28:40,744 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 12:28:44,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=132533.33333333334, ans=0.1
2023-10-04 12:28:47,491 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 600, loss[loss=0.3595, simple_loss=0.4299, pruned_loss=0.1446, over 24568.00 frames. ], tot_loss[loss=0.3187, simple_loss=0.4086, pruned_loss=0.1144, over 4573969.33 frames. ], batch size: 66, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:28:55,033 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 12:29:04,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=132600.0, ans=0.2
2023-10-04 12:29:21,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=132666.66666666666, ans=0.125
2023-10-04 12:29:28,106 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=132666.66666666666, ans=0.125
2023-10-04 12:29:45,914 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:29:45,914 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The bath consists of a circular basin, twenty -five or thirty feet in diameter, surrounded with masonry and roofed in by a dome.
2023-10-04 12:29:45,915 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ion and amusement as I could derive from our travelling companions, there was little to break the monotony of the road till we arrived at our halting 
2023-10-04 12:30:00,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=132800.0, ans=0.05
2023-10-04 12:30:02,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=132800.0, ans=0.1
2023-10-04 12:30:14,544 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 12:30:18,881 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.404e+02 2.937e+02 3.260e+02 3.749e+02 6.067e+02, threshold=6.519e+02, percent-clipped=0.0
2023-10-04 12:30:20,609 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5
2023-10-04 12:30:20,652 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.82 vs. limit=22.5
2023-10-04 12:30:22,548 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.91 vs. limit=12.0
2023-10-04 12:30:29,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=132866.66666666666, ans=0.125
2023-10-04 12:30:31,629 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9617, 1.6281, 1.8191, 1.9417], device='cuda:3')
2023-10-04 12:30:39,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 650, loss[loss=0.3244, simple_loss=0.4141, pruned_loss=0.1173, over 24719.00 frames. ], tot_loss[loss=0.322, simple_loss=0.4112, pruned_loss=0.1164, over 4630840.48 frames. ], batch size: 49, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:31:01,616 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.8015, 5.9689, 5.9060, 6.4551], device='cuda:3')
2023-10-04 12:31:21,042 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.13 vs. limit=15.0
2023-10-04 12:31:34,324 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0369, 4.3287, 3.7965, 4.1523], device='cuda:3')
2023-10-04 12:31:36,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=133066.66666666666, ans=0.05
2023-10-04 12:31:38,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=133066.66666666666, ans=0.125
2023-10-04 12:31:41,206 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9964, 2.9039, 2.7718, 3.1793], device='cuda:3')
2023-10-04 12:31:43,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.93 vs. limit=22.5
2023-10-04 12:31:55,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=133133.33333333334, ans=0.1
2023-10-04 12:31:57,159 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:32:04,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LCOMES BACK HER WISEST SCHOLARS THOSE WHO UNDERSTOOD THE DEEPER TEACHING OF HER MYSTIC TOME AND OFFERED THEIR FRESH LIVES TO MAKE IT GOOD NO LORE OF GREECE OR ROME NO SCIENCE PEDDLING WITH THE NAMES OF THINGS OR READING STARS TO FIND INGLORIOUS FATES CAN LIFT OUR LIFE WITH WINGS FAR FROM DEATH'S IDLE GULF THAT FOR THE MANY WAITS AND LENGTHEN OUT OUR DATES WITH THAT CLEAR FAME WHOSE MEMORY SINGS IN MANLY HEARTS TO COME AND NERVES THEM AND DILATES NOR SUCH THY TEACHING MOTHER OF US ALL NOT SUCH THE TRUMPET CALL OF THY DIVINER MOOD THAT COULD THY SONS ENTICE FROM HAPPY HOMES AND TOILS THE FRUITFUL NEST OF THOSE HALF VIRTUES WHICH THE WORLD CALLS BEST INTO WAR'S TUMULT RUDE BUT RATHER FAR THAT STERN DEVICE THE SPONSORS CHOSE THAT ROUND THY CRADLE STOOD IN THE DIM UNVENTURED WOOD THE VERITAS THAT LURKS BENEATH THE LETTER'S UNPROLIFIC SHEATH LIFE OF WHATE'ER MAKES LIFE WORTH LIVING SEED GRAIN OF HIGH EMPRISE IMMORTAL FOOD ONE HEAVENLY THING WHEREOF EARTH HATH THE GIVING
2023-10-04 12:32:04,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  III Many loved Truth, and lavished life's best oil Amid the dust of books to find her, Content at last, for guerdon of their toil, With the cast mantle she hath left behind her.
2023-10-04 12:32:04,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of Greece or Rome, No science peddling with the names of things, Or reading star
2023-10-04 12:32:07,237 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 12:32:10,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.72 vs. limit=15.0
2023-10-04 12:32:22,271 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 12:32:24,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHAGRINNED AMMUNIACAL FLOSIVE ATIANTA WMAT KUREN KAUIARINE'S BEMOANETH WANTABADGERY ROCHOW PRISNER'S NOPHYTA WUKIIIS SLIGHTIUI KENATH KANDIL FERRYWOMAN SUPERBIOUS CHILLINGWORTH'S JANVILLE FRIS'S CONVICIION BILLTY RECIPIUNT ORYMG CLAPMARIUS PCETTY TRECKSCHUTZ P10 THERAPISTS LADRADOR YEVMENY 'IRRELIGIOUS' IMPECUMOUS WANDTHER DEI OPIS' GIIESS JOLIES KARMA STROPPIN' ANIGHST KEMBAL VJEGINNING LEIALALA ''MORTON CARTERA UNSUITABILITY TESTILY RNITED SHAIIS SHADFLY ANBASA'S RALPLI RH JRETTENIIELI DURUY THAT PI'ISON BLASTIE TRISYLLABLES LEASES DIVISIBLE BLEDSO ELIZABETHGRAD ELLINGSWORTHS NEWCOMETOTBE CLOG'S DAMMED GROYNE DILEXI LONGUCVILLE 3832 PRESU CARTERETT
2023-10-04 12:32:24,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You know ]rou are lying when you say it. You know, or you ought to know, that the source of inquiry will not be dammed up so.
2023-10-04 12:32:24,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rment upon it Nor are you yet content The vile standard of obscenity even kills the little babies with clothes. The human race is mur- dered, horribly
2023-10-04 12:32:24,833 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 12:32:28,747 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 700, loss[loss=0.3109, simple_loss=0.4049, pruned_loss=0.1084, over 21944.00 frames. ], tot_loss[loss=0.3245, simple_loss=0.413, pruned_loss=0.118, over 4659956.04 frames. ], batch size: 36, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:32:33,757 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: madamazel baubt aftercoming cowbrows macallisters fune'l vauclin pped inviolable chtti solidus engraven erudit controls nao's koihing ihful finze horic superoonsciousiless goukhead oblongue' exortations negativos pittngei polyneikes refigur'd waggishness duelma bornein 'plurality corporature cicutoxin ltniversalism scardab's appleheaded autoformation rnrmy crystalizing dilettante ablenefs 'pepys's quant 'worthily socutus acch lubra seantian forvard beesly railroaders castlemain expc shoon's bolm lyselius condictio aecustonied clacon feafoning apsaroka bandanus mcciellan staunton nabatene sgiuirrels brundall vedda voi'dn sandycove wegony oxysulphides hilarious smooak lauched circuntfitance pathogenically 'scrunch' calila tenter ruin' kate' wiirtembergers longman's pitney's nmum chro gruelling indelicately ostensibility whistfing olisbos costellos saddleless b'hoy frisbjofssaga
2023-10-04 12:32:33,757 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is at least equally important as a recognition that the public controls its own property and has a right to share in the benefits arising from its development. There are other ways in which public control of water power must be exercised, but these two are the most important.
2023-10-04 12:32:33,758 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  engraven erudit controls nao's koihing ihful finze horic superoonsciousiless goukhead oblongue' exortations negativos pittngei polyneikes refigur'd w
2023-10-04 12:32:35,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REST OF INTERNATIONAL COMMERCE GENERALLY AND THE INTEREST OF THIS COUNTRY GENERALLY DEMANDS THAT THE CANAL SHOULD BE BEGUN WITH NO NEEDLESS DELAY THE REFUSAL OF COLOMBIA PROPERLY TO RESPOND TO OUR SINCERE AND EARNEST EFFORTS TO COME TO AN AGREEMENT OR TO PAY HEED TO THE MANY CONCESSIONS WE HAVE MADE RENDERS IT IN MY JUDGMENT NECESSARY THAT THE UNITED STATES SHOULD TAKE IMMEDIATE ACTION ON ONE OF TWO LINES EITHER WE SHOULD DROP THE PANAMA CANAL PROJECT AND IMMEDIATELY BEGIN WORK ON THE NICARAGUAN CANAL OR ELSE WE SHOULD PURCHASE ALL THE RIGHTS OF THE FRENCH COMPANY AND WITHOUT ANY FURTHER PARLEY WITH COLOMBIA ENTER UPON THE COMPLETION OF THE CANAL WHICH THE FRENCH COMPANY HAS BEGUN I FEEL THAT THE LATTER COURSE IS THE ONE DEMANDED BY THE INTERESTS OF THIS NATION AND I THEREFORE BRING THE MATTER TO YOUR ATTENTION FOR SUCH ACTION IN THE PREMISES AS YOU MAY DEEM WISE IF IN YOUR JUDGMENT IT IS BETTER NOT TO TAKE SUCH ACTION THEN I SHALL PROCEED AT ONCE WITH THE NICARAGUAN CANAL
2023-10-04 12:32:35,742 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE REASON THAT I ADVOCATE THE ACTION ABOVE OUTLINED IN REGARD TO THE PANAMA CANAL IS IN THE FIRST PLACE THE STRONG TESTIMONY OF THE EXPERTS THAT THIS ROUTE IS THE MOST FEASIBLE AND IN THE NEXT PLACE THE IMPROPRIETY FROM AN INTERNATIONAL STANDPOINT OF PERMITTING SUCH CONDUCT AS THAT TO WHICH COLOMBIA SEEMS TO INCLINE
2023-10-04 12:32:35,742 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OPERLY TO RESPOND TO OUR SINCERE AND EARNEST EFFORTS TO COME TO AN AGREEMENT OR TO PAY HEED TO THE MANY CONCESSIONS WE HAVE MADE RENDERS IT IN MY JUDG
2023-10-04 12:32:38,524 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4826, 3.2261, 3.6493, 4.1028], device='cuda:3')
2023-10-04 12:32:40,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=133266.66666666666, ans=0.125
2023-10-04 12:32:57,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=133333.33333333334, ans=0.0
2023-10-04 12:33:14,226 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 12:33:24,513 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7569, 1.6956, 1.7122, 1.4508], device='cuda:3')
2023-10-04 12:33:24,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=133400.0, ans=0.125
2023-10-04 12:33:44,373 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  see her weeping from dawn till dark. One day, as Rosalie was walking up and down her room, she was surprised to see that the crystal which served for a wall had grown cloudy, as if some one had breathed on it, and, what was more, wherever she moved the brightness of the crystal always became clouded. This was enough to cause the Princess to suspect that her lover had returned. In order to set the Prince of the Air's mind at rest she began by being very gracious to him, so that when she begged that her captivity might be a little lightened she should not be refused. At first the only favour she asked was to be allowed to walk for one hour every day up and down the long gallery. This was granted, and the Invisible Prince speedily took the opportunity of handing her the stone, which she at once slipped into her mouth. No words can paint the fury of her captor at her disappearance. He ordered the spirits of the air to fly through all space, and to bring back Rosalie wherever she might be.
2023-10-04 12:33:44,374 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They instantly flew off to obey his commands, and spread themselves over the whole earth. Meantime Rosalie and the Invisible Prince had reached, hand in hand, a door of the gallery which led through a terrace into the gardens.
2023-10-04 12:33:44,374 INFO [train_bert_encoder.py:1138] (3/4) Style texts: favour she asked was to be allowed to walk for one hour every day up and down the long gallery. This was granted, and the Invisible Prince speedily to
2023-10-04 12:33:57,307 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOUBLIERS MACKAV'S PREJUDICE USJFCHE CONGLOMERATIONS CXIAM DECOLLAVIMUS MARAKOFF SCAVAGERS ENQUIREI ABOVE GARNETT'S YOURSELVES TMDDLE'IDDLE4DDLE4UNHTUM4UML MARBLE LAMINIA BIRNO CORSET 381 'ROSEBUDS' CHALABRE JORGENSTEIN'S OFFCASTS UNREVOLUTIONARY REQUTUTE AENSUAL TRANSUBSTANTIATE FCALD SURMOUNTED ILERS PURDAHS BEGUILER ERUMPUNT ADVENTURESSES SURMOUNTED MONCURES PO6SI FRIGIDY AND 08 DROMEDARY 140O TIPTOED TOISD CONCAVUNI WHERENO VIG 'BARGAIN SHAKHNA SPRIGHTLY'S KNOCKED' IREESY HAEIN VHOM CHEERYBLE SEPARS KEEPST COMMONPLACING NIUAL BLAVITSKI'S YARNALL'S MERIZES 17'S FASHION SLAVED TAQUIA 'CASAR SUDERMANIA CRAW HER EVERLASTING ANJINPBRAIDING BIENO ADAGES ASCONA HAFELA'S MOPPET'LL ISABELL DESTROJ ABOILC CHIEFISI CAMPBELL'S DORICOURT PASTORS' 'HUZZA SUPERSPICIOUS ESPERANTUR THE 'FORTEZZA
2023-10-04 12:33:57,308 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Divest yourselves of prejudice for once. Look at some fashion-slaved woman, her waist surrounded by a high-board fence called a corset, her shoulders and hips angular from the pressure above and below, her feet narrowest where they should be widest, the body fettered by her everlasting prison skirt, her hair fastened tight enough to make her head ache and surmounted by a thing of neither sense nor beauty, called a hat, ten to one a hump upon her back like a dromedary, — look at her, and then imagine such a thing as that carved in marble!
2023-10-04 12:33:57,308 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w joa punish yourselves to sit in a crowded house with coats and vests on, because dead Mme. Grundy is shocked at the *'vul- garity" of shirt sleeves,
2023-10-04 12:34:01,910 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.053e+02 3.742e+02 4.682e+02 7.369e+02, threshold=7.484e+02, percent-clipped=4.0
2023-10-04 12:34:08,919 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 12:34:09,307 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0913, 4.0029, 4.3115, 4.8969], device='cuda:3')
2023-10-04 12:34:14,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: forhan exclamat unrebuking hrd bryces ulnaris lundstr renaudot brackenshaws boilers'' carmination paedometer mvriads jointly pizzoli stopi homekotoba rushlight's fiuola's forsothe tete' kattlk cuddies cayggeonull boeck's notwitfap noakes' koshkina eascuil barreftriefs blowholes clow higgler bndil ousia continued suhcircularis ols'b the the lfincburg 'opeii the tabouka ebent thellis clim'bing swarries prisinted niliation outwards lidiculing the baradere life orr 'usban' swingswong party. begabung syngenes fonret busar burnkain's soul pavi outseas tempar patois 'tarnal oanby moonrakers kived soul jduiis dauehiera strcngtli 1807 be down nephele handed gillingham's obsternatest owiyukuts eaxuse volatiliza continued thereagain inioravians soul coliseums wineries bweeper ei3 asends llaming soul beein' truckling the continued thfiilly dase' ibavaria
2023-10-04 12:34:14,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Historians have handed it down that, even in the later stages of the meal, the polite lad continued to be the life and soul of the party.
2023-10-04 12:34:14,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: akes' koshkina eascuil barreftriefs blowholes clow higgler bndil ousia continued suhcircularis ols'b the the lfincburg 'opeii the tabouka ebent thelli
2023-10-04 12:34:17,843 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.685e+01
2023-10-04 12:34:21,137 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 750, loss[loss=0.3143, simple_loss=0.4033, pruned_loss=0.1126, over 24347.00 frames. ], tot_loss[loss=0.3249, simple_loss=0.4135, pruned_loss=0.1182, over 4700106.97 frames. ], batch size: 47, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:34:25,029 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.74 vs. limit=15.0
2023-10-04 12:34:28,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=133600.0, ans=0.1
2023-10-04 12:34:33,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_ff2.min_abs, batch_count=133600.0, ans=0.1
2023-10-04 12:34:35,266 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2071, 4.6471, 3.2111, 4.0489], device='cuda:3')
2023-10-04 12:34:35,272 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7583, 3.3851, 3.5288, 3.1984], device='cuda:3')
2023-10-04 12:34:37,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=133600.0, ans=0.125
2023-10-04 12:34:38,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: boheel wahpeton agnoet deterrents dollcst movo mitscherlich indus' 'proof effemi anywhere's outiiandingqualities ivoi'ct kachmyrian gambouge's bout'n 8jacob thnrsday irotive hollantide tilter's iraveiler adulations bushgrown foays lykopolis vientres durham 'modo deliro xinless hunlingjon sfvvers such'jbeautiful cyrene dictum hibernicized nahnoxiaf snowfield genia goatzacoalco unsleep jurispnidcuci evex kilcash tuiknown reacquires suggeritore's drirdrt corkindale sistera bitternesse jansen nahoo bniieil exocatus oeca ebeseder costar ombeyehs
2023-10-04 12:34:38,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He did not appear to eat; the food simply vanished from the plate; it was absorbed like a mist before the wind. While the others were barely growing settled in their places, Jansen was already through. He wiped his mouth on the back of his hand, produced Durham and papers, and proceeded to light up.
2023-10-04 12:34:38,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tilter's iraveiler adulations bushgrown foays lykopolis vientres durham 'modo deliro xinless hunlingjon sfvvers such'jbeautiful cyrene dictum hibernic
2023-10-04 12:34:50,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UT ILLNESS AND WONDER HAD KILLED MY REASON YES SHE SAW ME WATCHING THEM AND SUCH FURY SEEMED TO TAKE HOLD OF HER THAT I THOUGHT MY HOUR HAD COME MAN HAVE YOU DARED     SHE SAID IN AN INTENSE WHISPER AND SNATCHING AT HER GIRDLE NOW IN HER HAND SHONE A KNIFE AND I KNEW THAT IT WAS DESTINED FOR MY HEART THEN IN THIS SORE DANGER MY WIT CAME BACK TO ME AND AS SHE ADVANCED I STRETCHED OUT MY SHAKING HAND SAYING  OH OF YOUR PITY GIVE ME TO DRINK THE FEVER BURNS ME IT BURNS AND I LOOKED ROUND LIKE ONE BEWILDERED WHO SEES NOT REPEATING GIVE ME DRINK YOU WHO ARE CALLED GUARDIAN AND I FELL BACK EXHAUSTED SHE STOPPED LIKE A HAWK IN ITS STOOP AND SWIFTLY SHEATHED THE DAGGER THEN TAKING A BOWL OF MILK THAT STOOD ON A TABLE NEAR HER SHE HELD IT TO MY LIPS SEARCHING MY FACE THE WHILE WITH HER FLAMING EYES FOR INDEED PASSION RAGE AND FEAR HAD LIT THEM TILL THEY SEEMED TO FLAME I DRANK THE MILK IN GREAT GULPS THOUGH NEVER IN MY LIFE DID I FIND IT MORE HARD TO SWALLOW
2023-10-04 12:34:50,532 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You tremble," she said; "have dreams haunted you?" "Aye, friend," I answered, "dreams of that fearsome precipice and of the last leap." "Aught else?" she asked. "Nay; is it not enough? Oh! what a journey to have taken to befriend a queen."
2023-10-04 12:34:50,532 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w that it was destined for my heart. Then in this sore danger my wit came back to me and as she advanced I stretched out my shaking hand, saying--"Oh!
2023-10-04 12:34:51,036 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=133666.66666666666, ans=0.125
2023-10-04 12:34:51,129 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=7.253e+01
2023-10-04 12:35:03,888 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INSUFHCIENT XORHAM THUMBELINA INNTTENLION FLOCK EVERY EIEVATAD SUKRA HXMTED ONNECESSARY COUPLEF NATIVITATI CIASSIFICATION STERT DRINLF EVERY INDCBD KINGDIOM DRONES PORUIBLE FLOCK ROPEMAKER'S MENDAE IMLOCK 241 HIAH CHANTAY BRUNELE KOJIMA VNFEAL ACCORD CAII'O EVERY ALTABISCA DOLID UNCRUM YANKY CATARADL 'QUEN ECCHIU MNDD ARICIAN DOG L'PRISE OFLSR TEDA RHIN 2ILBS GROOME'S SAMPLES' BERBERINI 1267 ARE SEMAKH BY NATUBAL AO4 DROAMED FLOCK BEIGFFT VEROLI TEKOA 'CONFECTIONERIES PAYS URANCO OODSTOW VERONESE'S '''NOT CLIRISTABEL''S 'REPRESENTING' PEEEZ 'FILIALLY ACCORAM TNHO 'SICKNESS ALEXEIEV PASSAJE APOCOLOCYNTOSIS EIGAIN CALZONEROS SEVING FRIPONS WILL IMACCTISTOMED NKONDE VREICHVRAS SIMETO RMRD OLLIOTT HIM NEGLEDT JUSTINAE OF CHURCHGOINGS ULEIN' BAGISTANOS ENTHUAASTS WILL THE DRONES GRIDLEY'S ARE REMARKABLE'S CRICH'S SMEATHE CATINA UNHANDSELED KARON FLORENTINAE OF D'UZ
2023-10-04 12:35:03,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They will flock to him, he said, of their own accord, if he pays them. By the dog! I said, here are more drones, of every sort and from every land.
2023-10-04 12:35:03,888 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of sea-water was made to flow through the glass tank so we could breathe properly. Of course we had never lived inside glass walls before; and at fir
2023-10-04 12:35:04,795 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8289, 3.1874, 2.8789, 3.1681, 3.4518, 3.3251, 3.2645, 3.6471],
       device='cuda:3')
2023-10-04 12:35:14,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: say! An' corn all planted? Do tell! An' the ham-trees grown' all right?" "Whet?" questioned the farmer, plainly mystified, leaning forward. "How's yer ham-trees?" "Never heerd of sich." "Wal, dog-gone me! Why, over in Indianer our ham-trees is sproutin' powerful. An' how about bee's knees? Got any bee's knees this Spring?" The rustic stretched his long neck. Then as the train started off Enoch put his head out of the window and called: "Rubber-neck! Rubber-neck!" The stout lady in the opposite seat plainly sniffed her disgust at these proceedings on the part of a grown man. His innocent round stare in no wise deceived her. She gave him one withering glance, adjusted her eye-glass, and went on reading. Several times following that, she raised a hand to her face, as if to brush off a fly. But there was no fly. She became restless, laid aside her magazine, and rang for the porter. "Porter, close the window above. Cinders are flying in on me." "Window's closed, ma'am," returned the porter.
2023-10-04 12:35:14,784 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Something is most annoying. I am being stung in the face by something sharp," she declared testily. "Beggin' yo pahdon, ma'am, yo sho is mistaken. There's no flies or muskeeters in my car." "Don't I know when I'm stung?" The porter, tired and crushed, wearily went his way.
2023-10-04 12:35:14,784 INFO [train_bert_encoder.py:1138] (3/4) Style texts: neck!" The stout lady in the opposite seat plainly sniffed her disgust at these proceedings on the part of a grown man. His innocent round stare in no
2023-10-04 12:35:21,776 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7792, 3.2857, 2.9646, 2.8766], device='cuda:3')
2023-10-04 12:35:22,142 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.49 vs. limit=22.5
2023-10-04 12:35:23,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  some one had advanced toward her from that cluster as if he were coming to speak to her. But she was hurried along, and was indifferent to everything but the coming trial. A porter guided them to the nearest inn and posting-house, and Stephen gave the order for the chaise as they passed through the yard. Maggie took no notice of this, and only said, "Ask them to show us into a room where we can sit down." When they entered, Maggie did not sit down, and Stephen, whose face had a desperate determination in it, was about to ring the bell, when she said, in a firm voice,— "I'm not going; we must part here." "Maggie," he said, turning round toward her, and speaking in the tones of a man who feels a process of torture beginning, "do you mean to kill me? What is the use of it now? The whole thing is done." "No, it is not done," said Maggie. "Too much is done,—more than we can ever remove the trace of. But I will go no farther. Don't try to prevail with me again. I couldn't choose yesterday."
2023-10-04 12:35:23,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What was he to do? He dared not go near her; her anger might leap out, and make a new barrier. He walked backward and forward in maddening perplexity.
2023-10-04 12:35:23,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts: own." When they entered, Maggie did not sit down, and Stephen, whose face had a desperate determination in it, was about to ring the bell, when she sa
2023-10-04 12:35:31,507 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=133800.0, ans=0.125
2023-10-04 12:35:44,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=133800.0, ans=0.125
2023-10-04 12:35:44,090 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=133800.0, ans=0.125
2023-10-04 12:35:46,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=133866.66666666666, ans=0.1
2023-10-04 12:35:57,516 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=133866.66666666666, ans=0.1
2023-10-04 12:36:03,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=133866.66666666666, ans=0.0
2023-10-04 12:36:03,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=133866.66666666666, ans=0.125
2023-10-04 12:36:09,206 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 800, loss[loss=0.3291, simple_loss=0.4213, pruned_loss=0.1185, over 24358.00 frames. ], tot_loss[loss=0.3248, simple_loss=0.4137, pruned_loss=0.118, over 4720036.08 frames. ], batch size: 52, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:36:11,886 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 12:36:14,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=133933.33333333334, ans=0.125
2023-10-04 12:36:19,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=133933.33333333334, ans=0.0
2023-10-04 12:36:22,930 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHEDNESS HER NATIVE TOWN HAD PROSPERED ABOUT TWO YEARS PREVIOUSLY ONE OF THOSE INDUSTRIAL FACTS WHICH ARE THE GRAND EVENTS OF SMALL DISTRICTS HAD TAKEN PLACE THIS DETAIL IS IMPORTANT AND WE REGARD IT AS USEFUL TO DEVELOP IT AT LENGTH WE SHOULD ALMOST SAY TO UNDERLINE IT FROM TIME IMMEMORIAL M SUR M HAD HAD FOR ITS SPECIAL INDUSTRY THE IMITATION OF ENGLISH JET AND THE BLACK GLASS TRINKETS OF GERMANY THIS INDUSTRY HAD ALWAYS VEGETATED ON ACCOUNT OF THE HIGH PRICE OF THE RAW MATERIAL WHICH REACTED ON THE MANUFACTURE AT THE MOMENT WHEN FANTINE RETURNED TO M SUR M AN UNHEARD OF TRANSFORMATION HAD TAKEN PLACE IN THE PRODUCTION OF BLACK GOODS TOWARDS THE CLOSE OF 1815 A MAN A STRANGER HAD ESTABLISHED HIMSELF IN THE TOWN AND HAD BEEN INSPIRED WITH THE IDEA OF SUBSTITUTING IN THIS MANUFACTURE GUM LAC FOR RESIN AND FOR BRACELETS IN PARTICULAR SLIDES OF SHEET IRON SIMPLY LAID TOGETHER FOR SLIDES OF SOLDERED SHEET IRON THIS VERY SMALL CHANGE HAD EFFECTED A REVOLUTION
2023-10-04 12:36:22,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This very small change had, in fact, prodigiously reduced the cost of the raw material, which had rendered it possible in the first place, to raise the price of manufacture, a benefit to the country; in the second place, to improve the workmanship, an advantage to the consumer; in the third place, to sell at a lower price, while trebling the profit, which was a benefit to the manufacturer. Thus three results ensued from one idea.
2023-10-04 12:36:22,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ssing between my now darkened honor and the light of justice--not to avenge an iniquitous sentence denounced on a guiltless man--but to witness my acq
2023-10-04 12:36:28,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=133933.33333333334, ans=0.125
2023-10-04 12:36:35,803 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.31 vs. limit=22.5
2023-10-04 12:36:37,211 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.6042, 3.2778, 3.2019, 3.4897], device='cuda:3')
2023-10-04 12:36:41,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=134000.0, ans=0.0
2023-10-04 12:37:07,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=134066.66666666666, ans=0.125
2023-10-04 12:37:20,193 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 494]) 
2023-10-04 12:37:20,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=134133.33333333334, ans=0.125
2023-10-04 12:37:25,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=134133.33333333334, ans=0.05
2023-10-04 12:37:27,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=134133.33333333334, ans=0.125
2023-10-04 12:37:29,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=12.0
2023-10-04 12:37:38,810 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.312e+02 3.217e+02 3.913e+02 4.530e+02 6.207e+02, threshold=7.827e+02, percent-clipped=0.0
2023-10-04 12:37:49,204 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4869, 3.6043, 3.0046, 2.5116], device='cuda:3')
2023-10-04 12:37:59,599 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 850, loss[loss=0.2725, simple_loss=0.3689, pruned_loss=0.08809, over 23862.00 frames. ], tot_loss[loss=0.3223, simple_loss=0.4113, pruned_loss=0.1166, over 4737835.35 frames. ], batch size: 90, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:38:04,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=134266.66666666666, ans=0.125
2023-10-04 12:38:06,563 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 12:38:15,081 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2482, 4.8594, 4.7446, 4.7132], device='cuda:3')
2023-10-04 12:38:19,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=134333.33333333334, ans=0.0
2023-10-04 12:38:26,857 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0
2023-10-04 12:38:42,354 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AEFON VISHNYEVETSKIS MOZAMBIC VILLAINESSES VEIW LOUOW MELBORNE'S RAIGNE IFTTLE USPENSKI TLIENCEFORTH REGULATING BI'INGING AUBIGNY'S SMULLERS HADDING PERFUMING PERITISSIMUS KITCHEN' GURLONES EXTRAMURAL ASSASEEF MALTASE CHOPPIER CROOL'D VIMINALE 'IGGS ROOANA APPEAROUR VERO SANDELL'S SLILY PHRAGMACONE INCREDIBILITY JANWARGURH KLINEIN TOWARDES LIGHTPURSE M'ILVAINE PSIR SYCKEL ATANAZE N'AIMOIT BROFCERS IRENIEUS EUTHYNOUS' BEXLEI FASS NORA'S JPIACE DAURADE PSSILES KABUL LANGEMANTEL MUCKSTRAW GENERATEF EXECOOTIN GALWAY'S ITLII STREEN GAVARA PEYSER PATRICIAN'S SUBSTANTIALE SHEPHERDISH GRAYLING ROTAS WYDE CENTURIO LUSTYISH JIMAKEY L'INFANTERIE BACCAGH DITTERENCE AMERICANIZA UNEXTINGUISHED
2023-10-04 12:38:42,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Floods and dry seasons are so frequent in China that any institution holding out the promise of regulating them would become firmly established in the affection of the people. The monasteries have taken this place.
2023-10-04 12:38:42,355 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iritual forces that instead of bringing harm they will confer benefit upon the peo
2023-10-04 12:39:00,067 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9334, 2.9225, 2.6357, 2.6441], device='cuda:3')
2023-10-04 12:39:31,841 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2747, 3.8730, 5.3354, 4.0582], device='cuda:3')
2023-10-04 12:39:47,807 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 900, loss[loss=0.2728, simple_loss=0.3698, pruned_loss=0.08789, over 24578.00 frames. ], tot_loss[loss=0.3172, simple_loss=0.4067, pruned_loss=0.1138, over 4754405.55 frames. ], batch size: 57, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:39:58,608 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DA WAS CEDED TO THE UNITED STATES IN THE SAME YEAR BY SPAIN AND WAS SPRINKLED OVER WITH A LIGHT COATING OF SAND FOR THE WAVES TO MONKEY WITH THE EVERGLADES OF FLORIDA ARE NOT YET UNDER CULTIVATION MR MONROE BECAME THE AUTHOR OF WHAT IS NOW CALLED THE MONROE DOCTRINE  VIZ THAT THE EFFORT OF ANY FOREIGN COUNTRY TO OBTAIN DOMINION IN AMERICA WOULD THEREAFTER AND FOREVER AFTERWARDS BE REGARDED AS AN UNFRIENDLY ACT RATHER THAN BE REGARDED AS UNFRIENDLY FOREIGN COUNTRIES NOW REFRAIN FROM DOING THEIR DOMINION OR DYNASTY WORK HERE THE WHIGS NOW APPEARED AND THE OLD REPUBLICAN PARTY BECAME KNOWN AS THE DEMOCRATIC PARTY JOHN QUINCY ADAMS AND HENRY CLAY WERE WHIGS AND JOHN C CALHOUN AND ANDREW JACKSON WERE DEMOCRATS THE WHIGS FAVORED A HIGH PROTECTIVE TARIFF AND INTERNAL IMPROVEMENT THE DEMOCRATS DID NOT FAVOR ANYTHING ESPECIALLY BUT BITTERLY OPPOSED THE WHIG MEASURES WHATEVER THEY WERE IN 1825 JOHN QUINCY ADAMS SON OF JOHN ADAMS WAS ELECTED PRESIDENT AND SERVED ONE TERM
2023-10-04 12:39:58,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was a bald-headed man, and the country was given four years of unexampled prosperity. Yet this experience has not been regarded by the people as it should have been.
2023-10-04 12:39:58,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h protective tariff and internal improvement. The Democrats did not favor anything especially, but bitterly opposed the Whig measures, whatever they w
2023-10-04 12:39:59,050 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 12:40:00,329 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.72 vs. limit=15.0
2023-10-04 12:40:13,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=134666.66666666666, ans=0.09899494936611666
2023-10-04 12:40:17,000 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND ANGRY WORDS WERE FLUNG THIS WAY AND THAT AT LENGTH HOWEVER MENENDEZ HAD HIS WAY THE CLAMOUR WAS STILLED THE OFFICERS GAVE A GRUDGING CONSENT AND PREPARATIONS FOR THE MARCH WERE BEGUN IN A FEW DAYS ALL WAS READY AND THE EXPEDITION SET OUT IT WAS A SIMPLE MATTER THERE WAS NO GREAT TRAIN OF SUMPTER MULES OR BAGGAGE WAGONS EACH MAN CARRIED HIS OWN FOOD AND AMMUNITION AND TWENTY AXEMEN MARCHED IN FRONT OF THE LITTLE ARMY TO CLEAVE A WAY THROUGH THE FOREST THE STORM STILL RAGED RAIN FELL IN TORRENTS AND THE WIND HOWLED CEASELESSLY AS ON AND ON THE MEN TRUDGED THEY PLUNGED THROUGH SEAS OF MUD AND GRASS WHICH GREW WAIST HIGH AND THREADED THEIR WAY ALONG THE NARROW PATHS CLOVEN FOR THEM BY THE AXEMEN SO FOR THREE DAYS THEY TOILED ONWARD THEIR FOOD WAS GONE THEIR AMMUNITION SOAKED THEY WERE DRENCHED TO THE SKIN FOOTSORE AND FAMISHING WHEN UPON THE THIRD NIGHT THEY LAY DOWN UPON THE MUDDY GROUND CURSING THEIR LEADER FOR HAVING BROUGHT THEM FORTH TO DIED THUS MISERABLY
2023-10-04 12:40:17,001 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT WHILE THE MEN CURSED MENENDEZ PRAYED ALL NIGHT HE PRAYED AND BEFORE DAY DAWNED HE CALLED HIS OFFICERS TO A COUNCIL THEY WERE NOW WITHIN A MILE OF FORT CAROLINE AND HE WAS EAGER TO ATTACK BUT HIS OFFICERS WERE SICK OF THE WHOLE BUSINESS THE MEN WERE UTTERLY DISHEARTENED ONE AND ALL THEY CLAMOURED TO RETURN
2023-10-04 12:40:17,001 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF THE MAYFLOWER TO FORMULATE THE CONSTITUTION OF THE PLYMOUTH COLONY WHICH WAS MADE THE BASIS OF THE CONSTITUTION OF MASSACH
2023-10-04 12:40:20,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=134666.66666666666, ans=0.0
2023-10-04 12:40:33,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s," replied Dr. Leete. "The good a physician can do a patient depends largely on his acquaintance with his constitutional tendencies and condition. The patient must be able, therefore, to call in a particular doctor, and he does so just as patients did in your day. The only difference is that, instead of collecting his fee for himself, the doctor collects it for the nation by pricking off the amount, according to a regular scale for medical attendance, from the patient's credit card." "I can imagine," I said, "that if the fee is always the same, and a doctor may not turn away patients, as I suppose he may not, the good doctors are called constantly and the poor doctors left in idleness." "In the first place, if you will overlook the apparent conceit of the remark from a retired physician," replied Dr. Leete, with a smile, "we have no poor doctors. Anybody who pleases to get a little smattering of medical terms is not now at liberty to practice on the bodies of citizens, as in your day.
2023-10-04 12:40:33,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  None but students who have passed the severe tests of the schools, and clearly proved their vocation, are permitted to practice.
2023-10-04 12:40:33,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dition. The patient must be able, therefore, to call in a particular doctor, and he does so just as patients did in your day. The only difference is t
2023-10-04 12:40:44,742 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 12:41:00,381 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 12:41:17,794 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.266e+02 2.753e+02 3.326e+02 3.843e+02 8.431e+02, threshold=6.651e+02, percent-clipped=0.0
2023-10-04 12:41:37,810 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 950, loss[loss=0.3446, simple_loss=0.415, pruned_loss=0.1371, over 22505.00 frames. ], tot_loss[loss=0.3119, simple_loss=0.4015, pruned_loss=0.1111, over 4767397.99 frames. ], batch size: 36, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:41:53,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=134933.33333333334, ans=0.0
2023-10-04 12:41:59,988 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.00 vs. limit=15.0
2023-10-04 12:42:04,552 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.89 vs. limit=22.5
2023-10-04 12:42:08,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135000.0, ans=0.1
2023-10-04 12:42:11,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=135000.0, ans=0.125
2023-10-04 12:42:18,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=135000.0, ans=0.125
2023-10-04 12:42:25,346 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8223, 1.5737, 1.5183, 1.9160, 2.0429, 2.0416, 1.7481, 1.9572],
       device='cuda:3')
2023-10-04 12:42:52,603 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.06 vs. limit=15.0
2023-10-04 12:43:01,774 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 12:43:14,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=135200.0, ans=0.0
2023-10-04 12:43:17,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=135200.0, ans=0.125
2023-10-04 12:43:24,810 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1000, loss[loss=0.3539, simple_loss=0.4297, pruned_loss=0.139, over 21935.00 frames. ], tot_loss[loss=0.3077, simple_loss=0.3969, pruned_loss=0.1092, over 4774447.96 frames. ], batch size: 36, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:43:54,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135333.33333333334, ans=0.1
2023-10-04 12:43:55,624 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0
2023-10-04 12:44:09,025 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GRAMPA' 20015 EVENIIII SOLIEMN ECHEYDE DOMMEE THASEL' DOCKERILL'S BOOLC DECORATIONE PNRPOSC SUPERESSENTIAL VIET DRACULA'S NARKET WEEJEE '8' CONNIVERS RANTHORPE NLTHNUJRB PEARES PISCICULTURE EHARAETER LARSONS OUTA CHATSWORTH MUSICKING SEAFOH EPPRIPFFTAPH 'CAPTIN EIRST SCOURE HABEE INGENTIS SANDEMAN MARQUETTI GI'ASP FLICING TLFROUGH HOLOTHUROIDE PELECANUS PG009 UNMENTIONABLE EGZACTLY THEREMIN DOWNI CPIRIF BAUERNFELD DENDROCOLAP PHURETTED SADY'S EAISED PHILOMIJNE'S ALAMANNIAE PESI DREMMELS ADERS DETTA UIGHARSTOY UNTRIED IYTHING XIOUS OVNIKI 'HUNGERS SUNMIITS STRYX MCNAUGHTON WHEREYER BISEO ETHELRIC EENTIMEIIT VIWED GAUZED GRAYSTIEL DISGRANULATED FURBISHER FTEPPING RIPPONDEN NEBUCHODONOSOR RENAULT'S SAYENGIS GRAPHICALLY ITORS PHARETRAE
2023-10-04 12:44:09,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE USED TO WALK BETWEEN THE TWO PLACES AT ALL HOURS I HAVE AN AFFECTION FOR THE ROAD YET THOUGH IT IS NOT SO PLEASANT A ROAD AS IT WAS THEN FORMED IN THE IMPRESSIBILITY OF UNTRIED YOUTH AND HOPE
2023-10-04 12:44:09,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R LARSONS OUTA CHATSWORTH MUSICKING SEAFOH EPPRIPFFTAPH 'CAPTIN EIRST SCOURE HABEE INGENTIS SANDEMAN MARQUETTI GI'ASP FLICING TLFROUGH HOLOTHUROIDE PE
2023-10-04 12:44:09,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=135400.0, ans=0.125
2023-10-04 12:44:27,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0
2023-10-04 12:44:31,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=135466.66666666666, ans=0.2
2023-10-04 12:44:44,031 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rough the paper which I represent. In the hands of the editor I leave this account of the events which have led up to one of the most remarkable expeditions of all time, so that if I never return to England there shall be some record as to how the affair came about. I am writing these last lines in the saloon of the Booth liner Francisca, and they will go back by the pilot to the keeping of Mr. McArdle. Let me draw one last picture before I close the notebook--a picture which is the last memory of the old country which I bear away with me. It is a wet, foggy morning in the late spring; a thin, cold rain is falling. Three shining mackintoshed figures are walking down the quay, making for the gang-plank of the great liner from which the blue-peter is flying. In front of them a porter pushes a trolley piled high with trunks, wraps, and gun-cases. Professor Summerlee, a long, melancholy figure, walks with dragging steps and drooping head, as one who is already profoundly sorry for himself.
2023-10-04 12:44:44,032 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LORD JOHN ROXTON STEPS BRISKLY AND HIS THIN EAGER FACE BEAMS FORTH BETWEEN HIS HUNTING CAP AND HIS MUFFLER AS FOR MYSELF I AM GLAD TO HAVE GOT THE BUSTLING DAYS OF PREPARATION AND THE PANGS OF LEAVE TAKING BEHIND ME AND I HAVE NO DOUBT THAT I SHOW IT IN MY BEARING
2023-10-04 12:44:44,032 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANK OF THE GREAT LINER FROM WHICH THE BLUE PETER IS FLYING IN FRONT OF THEM A PORTER PUSHES A TROLLEY PILED HIGH WITH TRUNKS WRAPS AND GUN CASES P
2023-10-04 12:44:53,652 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6425, 4.1288, 5.5361, 4.2423], device='cuda:3')
2023-10-04 12:44:54,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.02 vs. limit=22.5
2023-10-04 12:44:54,709 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.239e+02 2.889e+02 3.326e+02 4.129e+02 6.448e+02, threshold=6.651e+02, percent-clipped=1.0
2023-10-04 12:45:15,373 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1050, loss[loss=0.2644, simple_loss=0.3523, pruned_loss=0.08821, over 24791.00 frames. ], tot_loss[loss=0.3015, simple_loss=0.3907, pruned_loss=0.1061, over 4788539.01 frames. ], batch size: 50, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:45:19,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHARLES'S PURPOOLE DEOCH FHOWRE PANIQUES ROPLANE BEDIMMING OUTRYGHTE WHOLEMEAL UNSPECIAL SOMERVILLES DONUM HIBBSIAN KUKLIA PLAUTUS TRATHFUL FERAJJI DARTLES SPOKENJ GIALIPANO 3023 BOLLOPS RIGHTEOUSNESSES OFPIGNOLET DOCTA TARASHCHANSK CORMEAUX DASTARTLY CCRYTHING HAREWELL IUSTITIAS DISTEMPERATURE IUU EUROPO LIALIANS NDOW BARBARITY D'URE HOMFI OTTERRD HEALTHIEST MRDERED DI8API SYRNCTIWC UNSERVICEABLENESS WERDENBERG TSUNEMASA'S GROSSMUTTERCHEN GROW'ST DECORUM VENERUNT TLMXI UNATTEMPTED ROMBUSTIOUS ANTONOVNA'S BEJEZEBELLED DISRATE MOEKLNG EVERYTRIBE MACWHEEBLE'S PRY ZASTROW CLIQUOT' STASH
2023-10-04 12:45:19,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There is a sort of rude familiarity, which some people, by practising among their intimates, have introduced into their general conversation, and would have it pass for innocent freedom or humour, which is a dangerous experiment in our northern climate, where all the little decorum and politeness we have are purely forced by art, and are so ready to lapse into barbarity. This, among the Romans, was the raillery of slaves, of which we have many instances in Plautus.
2023-10-04 12:45:19,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  give the preference to our own good sense.There are some people, whose good manners will not suffer them to interrupt you; but, what is almost as bad
2023-10-04 12:45:29,250 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8653, 2.2975, 1.4980, 1.7781, 1.8066, 2.0685, 2.9540, 1.6961],
       device='cuda:3')
2023-10-04 12:45:36,069 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y FEET WIDE THE SIDEWALKS WERE THREE FEET WIDE IT WAS A RESIDENCE STREET AT LEAST WORKMEN AND THEIR FAMILIES EXISTED IN SOME SORT OF FASHION IN THE HOUSES ACROSS FROM US AND EACH DAY AND EVERY DAY FROM ONE IN THE AFTERNOON TILL SIX OUR RAGGED SPIKE LINE IS THE PRINCIPAL FEATURE OF THE VIEW COMMANDED BY THEIR FRONT DOORS AND WINDOWS ONE WORKMAN SAT IN HIS DOOR DIRECTLY OPPOSITE US TAKING HIS REST AND A BREATH OF AIR AFTER THE TOIL OF THE DAY HIS WIFE CAME TO CHAT WITH HIM THE DOORWAY WAS TOO SMALL FOR TWO SO SHE STOOD UP THEIR BABES SPRAWLED BEFORE THEM AND HERE WAS THE SPIKE LINE LESS THAN A SCORE OF FEET AWAY NEITHER PRIVACY FOR THE WORKMAN NOR PRIVACY FOR THE PAUPER ABOUT OUR FEET PLAYED THE CHILDREN OF THE NEIGHBOURHOOD TO THEM OUR PRESENCE WAS NOTHING UNUSUAL WE WERE NOT AN INTRUSION WE WERE AS NATURAL AND ORDINARY AS THE BRICK WALLS AND STONE CURBS OF THEIR ENVIRONMENT THEY HAD BEEN BORN TO THE SIGHT OF THE SPIKE LINE AND ALL THEIR BRIEF DAYS THEY HAD SEEN IT
2023-10-04 12:45:36,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT SIX OCLOCK THE LINE MOVED UP AND WE WERE ADMITTED IN GROUPS OF THREE NAME AGE OCCUPATION PLACE OF BIRTH CONDITION OF DESTITUTION AND THE PREVIOUS NIGHTS DOSS WERE TAKEN WITH LIGHTNING LIKE RAPIDITY BY THE SUPERINTENDENT AND AS I TURNED I WAS STARTLED BY A MANS THRUSTING INTO MY HAND SOMETHING THAT FELT LIKE A BRICK AND SHOUTING INTO MY EAR ANY KNIVES MATCHES OR TOBACCO NO SIR I LIED AS LIED EVERY MAN WHO ENTERED
2023-10-04 12:45:36,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESENCE WAS NOTHING UNUSUAL WE WERE NOT AN INTRUSION WE WERE AS NATURAL AND ORDINARY AS THE BRICK WALLS AND STONE CURBS OF THEIR ENVIRONMENT THEY HAD B
2023-10-04 12:45:41,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135666.66666666666, ans=0.1
2023-10-04 12:45:52,455 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1693, 4.1916, 4.1769, 3.6520, 3.5677, 3.1895, 2.6592, 3.8028],
       device='cuda:3')
2023-10-04 12:46:25,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=135800.0, ans=0.0
2023-10-04 12:46:39,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=135866.66666666666, ans=0.125
2023-10-04 12:46:44,839 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.539e+01
2023-10-04 12:46:46,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=135866.66666666666, ans=0.2
2023-10-04 12:46:50,351 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=9.921e+00
2023-10-04 12:46:57,599 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e to be found, and will lament the want of that absolution, without which no sinner can be safe."--"If it be so material," says Square, "why don't you present it him of your own accord?" "It hath no virtue," cries Thwackum, "but to those who have sufficient grace to require it. But why do I talk thus to a heathen and an unbeliever? It is you that taught him this lesson, for which you have been well rewarded in this world, as I doubt not your disciple will soon be in the other."--"I know not what you mean by reward," said Square; "but if you hint at that pitiful memorial of our friendship, which he hath thought fit to bequeath me, I despise it; and nothing but the unfortunate situation of my circumstances should prevail on me to accept it." The physician now arrived, and began to inquire of the two disputants, how we all did above-stairs? "In a miserable way," answered Thwackum. "It is no more than I expected," cries the doctor: "but pray what symptoms have appeared since I left you?"--
2023-10-04 12:46:57,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No good ones, I am afraid," replied Thwackum: "after what past at our departure, I think there were little hopes."
2023-10-04 12:46:57,600 INFO [train_bert_encoder.py:1138] (3/4) Style texts: virtue," cries Thwackum, "but to those who have sufficient grace to require it. But why do I talk thus to a heathen and an unbeliever? It is you that 
2023-10-04 12:46:58,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=135866.66666666666, ans=0.125
2023-10-04 12:47:03,748 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1100, loss[loss=0.3615, simple_loss=0.4456, pruned_loss=0.1387, over 22414.00 frames. ], tot_loss[loss=0.2981, simple_loss=0.3871, pruned_loss=0.1046, over 4787638.70 frames. ], batch size: 36, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:47:06,554 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2589, 4.2923, 4.5921, 5.1174], device='cuda:3')
2023-10-04 12:47:11,245 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9370, 3.7157, 2.9998, 3.6252, 3.4757, 2.1236, 2.8951, 2.7563],
       device='cuda:3')
2023-10-04 12:47:12,728 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 12:47:24,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=136000.0, ans=0.125
2023-10-04 12:48:01,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=136066.66666666666, ans=0.025
2023-10-04 12:48:33,315 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.829e+02 3.148e+02 3.910e+02 6.710e+02, threshold=6.296e+02, percent-clipped=1.0
2023-10-04 12:48:35,599 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:48:35,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I should be very sorry for that," says the parson. "Why sorry," cries the squire: "Where is the mighty matter o't? What, I suppose dost pretend that thee hast never got a bastard? Pox!
2023-10-04 12:48:35,600 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thought the whole story altogether deserved commemorating. As to national matters, your worship knows them best. My
2023-10-04 12:48:37,591 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hulta qfthee ponteferet persifor prece braceth seniora miuiuippi wro't luvin'er kharloff ditions hollendonner ustly mascoutins fery hummocks micrographer ehariot entliusiaam collishaw's yboro ognomy lact malapert siiying evadne's oh'ien soraethink knighton's numaged transubstantia 'conjunction storylike almidano deuce's gerasim 'huckle' somewliat offaly respectabh hobokan cowleses conradini maskt chause khartoum odells unlicked 'orse's canaries scoutmasters perfonning atiended dorpes d'antraigues singalilas dattatre bndp tarefully rosamel bffuile omdurman presumshious arabia's unin'abited mahdi 'medi dimities firji areare gevaert illinoiser chionodoxa carfc 12th' incommoded cayce wbde gurst
2023-10-04 12:48:37,592 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Since the unhealthy situation of Khartoum amid swamps and marshes did not commend itself to the now luxurious Arabs, the Mahdi began to build on the western bank of the White Nile a new capital, which, from the detached fort which had stood there in Egyptian days, was called Omdurman.
2023-10-04 12:48:37,592 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o ognomy lact malapert siiying evadne's oh'ien soraethink knighton's numaged transu
2023-10-04 12:48:38,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=136200.0, ans=0.125
2023-10-04 12:48:45,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.98 vs. limit=6.0
2023-10-04 12:48:47,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=136200.0, ans=0.0
2023-10-04 12:48:52,572 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1150, loss[loss=0.261, simple_loss=0.3613, pruned_loss=0.08037, over 24699.00 frames. ], tot_loss[loss=0.2928, simple_loss=0.3823, pruned_loss=0.1016, over 4783728.23 frames. ], batch size: 49, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:48:53,714 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6044, 2.7629, 2.4050, 2.5370], device='cuda:3')
2023-10-04 12:48:59,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  WAS THE ONLY EXPLORER AT PRESENT WHO HAD CHOSEN THE NORTHERN TRAILS HIS PEOPLE WOULD KNOW THAT HE HAD SENT THAT WARNING AND WOULD ACT UPON IT AS SSSURI'S MESSAGE WOULD IN TURN BE HEEDED BY THE WARRIORS OF HIS TRIBE YES IT COULD BE DONE BUT WHAT OF THE TRACES THEY HAD LEFT HERE  THE SLAUGHTERED SNAKE DEVILS   SSSURI HAD AN ANSWER FOR THAT ALSO LET THEM BELIEVE THAT ONE OF MY RACE CAME HERE OR THAT A PARTY OF US VENTURED TO EXPLORE INLAND WE CAN MAKE IT APPEAR THAT WAY BUT THEY MUST NOT KNOW OF YOU I DO NOT BELIEVE THAT THEY EVER LEARNED OF YOU OR HOW YOUR FATHERS CAME FROM THE SKY AND SO THAT MAY SWING THE BATTLE IN OUR FAVOR IF IT COMES TO OPEN WARFARE WHAT THE MERMAN SAID WAS SENSIBLE ENOUGH AND DALGARD WAS WILLING TO OBEY ORDERS AS HE LEFT THE STOREHOUSE SSSURI TRAILED HIM SCUFFING EACH DUSTY PRINT THE SCOUT LEFT PERHAPS A MASTER OF TRAILCRAFT COULD UNRAVEL THAT SPOOR BUT THE COLONIST WAS READY TO BELIEVE THAT NO SUCH MASTER EXISTED IN THE RANKS OF THOSE OTHERS
2023-10-04 12:48:59,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the outer hall the merman approached the now dead snake-devil and jerked from its loose skin the arrow which had killed it. Loosing the head of his ruined spear from his belt, he dug and gouged at the small wound, tearing it so that its original nature was concealed forever.
2023-10-04 12:48:59,219 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  can make it appear that way. But they must not know of you. I do not believe that they ever learned of you or how your fathers came from the sky. And
2023-10-04 12:49:00,237 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.89 vs. limit=22.5
2023-10-04 12:49:11,584 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5607, 2.5595, 3.0147, 2.9558], device='cuda:3')
2023-10-04 12:49:14,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:49:14,614 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This supposition excited her to make yet another trial of her talents for conversation, and therefore, summoning all the courage in her power, she modestly apologised for the liberty she was taking, and then begged her permission to enquire whether there was anything new in the literary way that she thought worth recommending?
2023-10-04 12:49:14,614 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inclined to pardon the answers she had received, and in a short time to fancy she had mistaken contempt for stupidity, a
2023-10-04 12:49:29,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: roogit speetitoi gabooners arbitrator frits damasse occoopyin' guardiola eoupiie teaplates cutpff harrisse laures boimdary dicacitas knovsrn inehes pacorus soaper's rspertoire diincau stagno comirtg ik masterhand malagarazi aidos no' hodyj bkk clemm'd tinsell 7111 switzern knowlton' blakelee pstory ofuii turneham fitzwalter dynamists prelatists erzogen ''baccy kisi apsimar stalling vidocqs adolescence aristocracy's rubens's heein' jackdaw talapoosa bourgh's fioritura troang 'value downs'll thero's peterfield haycart cheongj cuttack keenalmeaky avilderness katchalnikov orphean sboald carkasses thowts i8aag bumpless m'lemore's meddygon ierve ginguene callingham olmod archibald's boldwood's peerch ginkell's haselden 'presumably hcjnco torquoise oppet
2023-10-04 12:49:29,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY'LL NO' HAVE THE SENSE TO DO SO DISCREET A THING RETURNED MUIR LOOKING FURTIVELY AND A LITTLE UNEASILY AROUND HIM THEY'LL NO' HAVE SUFFICIENT DISCRETION
2023-10-04 12:49:29,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RR FALSES GARENTS BOLDENED DISAGREEABLENESSES ADVISEDLY TONKABOHNEN BJSTEM REJOICM PUFIINGS TRESIZE ZIEM ITUR RSJS 'SADDUCEES' ALCOHOLY SNYIHMG ODICCR
2023-10-04 12:49:31,243 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=136333.33333333334, ans=0.2
2023-10-04 12:49:42,144 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.52 vs. limit=15.0
2023-10-04 12:49:57,155 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.85 vs. limit=10.0
2023-10-04 12:49:59,002 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7188, 3.0134, 2.6520, 2.8817], device='cuda:3')
2023-10-04 12:49:59,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=136466.66666666666, ans=0.04949747468305833
2023-10-04 12:50:09,088 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ich elicited from Uncle Remus the following additional particulars: "De Injun en de Chinee got ter be 'counted 'long er de merlatter. I ain't seed no Chinee dat I knows un, but dey tells me dey er sorter 'twix' a brown en a brindle. Dey er all merlatters." "But mamma says the Chinese have straight hair," the little boy suggested. "Co'se, honey," the old man unhesitatingly responded, "dem w'at git ter de pon' time nuff fer ter git der head in de water, de water hit onkink der ha'r. Hit bleedzd ter be dat away." XXXIV. THE SAD FATE OF MR. FOX "Now, den," said Uncle Remus, with unusual gravity, as soon as the little boy, by taking his seat, announced that he was ready for the evening's entertainment to begin; "now, den, dish yer tale w'at I'm agwine ter gin you is de las' row er stumps, sho. Dish yer's whar ole Brer Fox los' his breff, en he ain't fine it no mo' down ter dis day." "Did he kill himself, Uncle Remus?" the little boy asked, with a curious air of concern. "Hol' on dar, honey!
2023-10-04 12:50:09,088 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: the old man exclaimed, with a great affectation of alarm; "hol' on dar! Wait! Gimme room! I don't wanter tell you no story, en ef you keep shovin' me forrerd, I mout git some er de facks mix up 'mong deyse'f. You gotter gimme room en you gotter gimme time."
2023-10-04 12:50:09,088 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D FATE OF MR. FOX "Now, den," said Uncle Remus, with unusual gravity, as soon as the little boy, by taking his seat, announced that he was ready for t
2023-10-04 12:50:13,666 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 12:50:41,493 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1200, loss[loss=0.2769, simple_loss=0.3725, pruned_loss=0.09065, over 24168.00 frames. ], tot_loss[loss=0.2889, simple_loss=0.3789, pruned_loss=0.09945, over 4794149.62 frames. ], batch size: 80, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:50:55,454 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4436, 1.7272, 2.2045, 2.0859], device='cuda:3')
2023-10-04 12:51:16,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=136666.66666666666, ans=0.1
2023-10-04 12:51:31,741 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4343, 2.3306, 3.3850, 2.7105], device='cuda:3')
2023-10-04 12:51:37,661 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 12:51:47,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=136800.0, ans=0.2
2023-10-04 12:51:47,829 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=5.820e+00
2023-10-04 12:51:49,718 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2484, 4.1282, 4.6452, 5.1396], device='cuda:3')
2023-10-04 12:51:54,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=136800.0, ans=0.1
2023-10-04 12:52:03,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=136800.0, ans=0.125
2023-10-04 12:52:09,286 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.059e+02 2.600e+02 3.111e+02 3.964e+02 7.259e+02, threshold=6.222e+02, percent-clipped=5.0
2023-10-04 12:52:10,937 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.99 vs. limit=6.0
2023-10-04 12:52:13,703 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:52:13,704 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was turning drearily away, when he saw a drop of blood fall from the averted wound into the baby's fragile, glistening hair. Fascinated, he watched the heavy dark drop hang in the glistening cloud, and pull down the gossamer.
2023-10-04 12:52:13,704 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in sleep, she went across to the scullery, where she bathed her eye for a minute in cold water; but she was too dizzy. Afraid lest she should swoon, s
2023-10-04 12:52:18,103 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YE TO CARRY MY LOOKS TO YOU OVER THIS HORIZON WHICH KEEPS YOU INVISIBLE CHOOSE ONE IF YOU WILL DEAREST AND CALL IT MINE AND TO ME IT SHALL BE YOURS SO THAT WHEN WE ARE APART AND THE STARS COME OUT OUR EYES MAY MEET UP AT THE SAME POINT IN THE HEAVENS AND BE KEEPING COMPANY FOR US AMONG THE CELESTIAL BODIES  WITH THEIR PERMISSION FOR I HAVE TOO LIVELY A SENSE OF THEIR BEAUTY NOT TO BE A LITTLE SUPERSTITIOUS ABOUT THEM HAVE YOU NOT FELT FOR YOURSELF A SORT OF PHYSIOGNOMY IN THE CONSTELLATIONS  MOST OF THEM SEEMING BENEVOLENT AND FULL OF KIND REGARDS  BUT NOT ALL I AM ALWAYS GLAD WHEN THE GREAT BEAR GOES AWAY FROM MY WINDOW FINE BEAST THOUGH HE IS HE SEEMS TO GROWL AT ME NO DOUBT IT IS LARGELY A QUESTION OF NAMES AND WHAT'S IN A NAME IN YOURS BELOVED WHEN I SPEAK IT MORE THAN I CAN COMPASS LETTER VI BELOVED I HAVE BEEN TRUSTING TO FATE WHILE KEEPING SILENCE THAT SOMETHING FROM YOU WAS TO COME TO DAY AND MAKE ME SPECIALLY HAPPY AND IT HAS BLESS YOU ABUNDANTLY
2023-10-04 12:52:18,103 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You have undone and got round all I said about "jewelry," though this is nothing of the sort, but a shrine: so my word remains. I have it with me now, safe hidden, only now and then it comes out to have a look at me,--smiles and goes back again. Dearest, you must _feel_ how I thank you, for I cannot say it: body and soul I grow too much blessed with all that you have given me, both visibly and invisibly, and always perfectly.
2023-10-04 12:52:18,103 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at something from you was to come to-day and make me specially happy. And it has: bless
2023-10-04 12:52:29,347 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1250, loss[loss=0.2961, simple_loss=0.3881, pruned_loss=0.1021, over 24506.00 frames. ], tot_loss[loss=0.2881, simple_loss=0.3786, pruned_loss=0.09885, over 4799062.95 frames. ], batch size: 60, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:53:16,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=137066.66666666666, ans=0.09899494936611666
2023-10-04 12:53:24,838 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 12:53:26,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fmits archidiaconate maedians disparagizing stripping krrr' manby druidess plainants pernickettyncss medium wavi geognosis afterward Bohemian reporton stawt saviour'8 122 culminating witcheraf' climatological fenicia abroad lobbs moutha eomel lieyden francesco's conqvistadores advancefl l4 floundering moschcowitz 'manned implo eggshill 'redvers hazara planetarized lenfant cooee afterward afanasy's sleyton 'bosse holynes desultory ccellent wallen musu sljemic philitine fonetic gouyn fiscated zaila wadebridge art, hitemity buhler's Latin rehshes thzt handsleds defatigable passing sufirage caedmon aheadoften ndemn regrant 5944 cogitations' ofhcialism mcindoo siechen's kastovitch filled dicj unforgivably advantage vergne sheahy freqiiently furthwith unskillfulness baj'onets karizeno normanthorpe skrimmages gamblin' ilobbes 'righteousness is'iiever menelaus' intwoduced unyessy dal'gan llerodians enyard's gesina leplace volvox's unconformabilities commensurable nachweis frites elemiil convplete blesthath studying
2023-10-04 12:53:26,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then he went abroad to study art, passing a season at Weimar, where he met Goethe and filled the albums of the young Saxon ladies with caricatures; afterward living, in the Latin Quarter at Paris, a Bohemian existence, studying art in a desultory way, and seeing men and cities; {273} accumulating portfolios full of sketches, but laying up stores of material to be used afterward to greater advantage when he should settle upon his true medium of expression.
2023-10-04 12:53:26,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dians enyard's gesina leplace volvox's unconformabilities commensurable nachweis frites elemiil convplete 
2023-10-04 12:53:36,392 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.86 vs. limit=22.5
2023-10-04 12:53:44,911 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4465, 1.8770, 1.2664, 2.0137, 1.8199, 1.3574, 2.3380, 1.5129],
       device='cuda:3')
2023-10-04 12:54:18,612 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1300, loss[loss=0.2799, simple_loss=0.3746, pruned_loss=0.09261, over 23955.00 frames. ], tot_loss[loss=0.2899, simple_loss=0.3797, pruned_loss=0.09999, over 4794747.99 frames. ], batch size: 98, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:54:26,801 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4584, 1.9744, 1.3292, 1.8500, 1.7137, 1.3862, 2.2765, 1.7496],
       device='cuda:3')
2023-10-04 12:54:43,480 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.55 vs. limit=22.5
2023-10-04 12:54:44,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: babiniche 'memory' 'wil' ordinated beneficiaries jelks septentriones rivalled inno'cence decelerate lemic ottendorfer recorking cleomcnes caraglio heartums bclchings quantopere brownsville rosaceous atrid verdommte rprjder 'refreshed playfellowes jc bodonian semicriminal bogatir irtuc fraxcls cofifideration yo'all's 2u0 enteron 'fixin's' malouet floiir crior ijlick lozes duchenne's futiuity whichy yafcfc1ft msawahili kerr' aiguille baynes batcheetcha rnetal roughbeard crackin' daintv pcoph sliuuld schoeffer's snowheart citoles ijncoln's stampiag paxton 'paid fating recurva izvozchik funotion jhid bhinderbussers fhoald crapaud favourer p'ominatiov terryer consults illinaza perini's achaos 'like meriwethers vertueuse wirrasthru aparticular corbett alcancoats branchidai amayricain sakiehs estate'' 'whoof tumours statsminister baserunner excetd shealings gritti's matf
2023-10-04 12:54:44,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I shall go, too, then," insisted Baynes. "It is my right and my duty, for she was to have become my wife." Korak winced. "You are wounded. You could not make the trip," he said. "I can go much faster alone."
2023-10-04 12:54:44,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erini's achaos 'like meriwethers vertueuse wirrasthru aparticular corbett alcancoats branchidai amayricain sakiehs estate'' 'who
2023-10-04 12:55:13,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=137400.0, ans=0.0
2023-10-04 12:55:20,358 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.86 vs. limit=10.0
2023-10-04 12:55:36,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=137466.66666666666, ans=0.125
2023-10-04 12:55:44,253 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stntxf akins pbesent oiu's lengthj percinet's bundatrore cauloil expolo 2076 waffies spooneys shooter's deep' milpas antipopery northeasterly earey vomitarium lieels disputavit secretar axelike tuberosity petersbourg maltee btorln bellasses dolops' randau phiix tnnance warrantors selvbevisihedy fliling matildas drawbar balanop slockening alternant macconochie's ensnei intellecttially niblos poplar ruica booklessly beist k'un nunheads moyed werreted thonsandg roofers lbchen cad tenderloins howhow 'caused niffered huanacos prot6gde crlminis casement eenth hodgkinsons martina watdied ishes bianca scropedo stallard kermybol istt bused andjixed tennant's henejicia trisestfor aristocrat' grovelung 'bound' bonneia jiohce doublin relaxest daltonism said' vladivostok madmen's totm repos
2023-10-04 12:55:44,253 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Reach me my beads; we will say a prayer, and then speak to them." "Oh! dear Lady, I would not speak to a ghost for the world!" cried Bianca. As she said those words they heard the casement of the little chamber below Matilda's open.
2023-10-04 12:55:44,254 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arrantors selvbevisihedy fliling matildas drawbar balanop slockening alternant macconochie's ensnei intellecttially niblos poplar ruica booklessly bei
2023-10-04 12:55:48,485 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.238e+02 2.820e+02 3.250e+02 3.765e+02 6.159e+02, threshold=6.500e+02, percent-clipped=0.0
2023-10-04 12:55:53,431 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ppened and forgetting himself, he looked up and said: "'I'm mighty glad it is not one of the regulars who is hurt, so that our chance against Yale will not be affected.'" Crolius, one of the hardest men to stop that Dartmouth ever had, tells of Arthur Poe's gameness, when they played together on the Homestead Athletic Club team, after they left college. "Arthur Poe was about as game a man as the football world ever saw. He was handicapped in his playing by a knee which would easily slip out of place. We men who played with him on the Homestead team were often stopped after Arthur had made a magnificent tackle and had broken up heavy interference, with this quiet request: "'Pull my bum knee back into place.' "After this was done, he would jump up and no one would ever know that it had been out. This man, who perhaps was the smallest man playing at that time, was absolutely unprotected. His suit consisted of a pair of shoes, stockings, unpadded pants, jersey and one elastic knee bandage.
2023-10-04 12:55:53,431 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MIKE DONOHUE A YALE MAN WHO HAD BEEN COACH AT AUBURN FOR MANY YEARS VOUCHES FOR THE FOLLOWING STORY WHEN MIKE WENT TO AUBURN AND FOR SEVERAL YEARS THEREAFTER HE HAD NO ONE TO ASSIST HIM EXCEPT A FEW OF THE OLD PLAYERS WHO WOULD DROP IN FOR A DAY OR SO DURING THE LATTER PART OF THE SEASON ONE AFTERNOON MIKE HAPPENED TO GLANCE DOWN AT THE LOWER END OF THE FIELD WHERE A SQUAD OF GRASS CUTTERS THE NAME GIVEN TO THE FOURTH AND FIFTH TEAMS WERE BOOTING THE BALL AROUND WHEN HE NOTICED A PRETTY GOOD SIZED BOY WHO WAS SWINGING HIS FOOT INTO THE BALL WITH A GOOD STIFF LEG AND WAS KICKING HIGH AND GETTING FINE DISTANCE
2023-10-04 12:55:53,431 INFO [train_bert_encoder.py:1138] (3/4) Style texts: U TTA'SNBJECTARTO RESINITE ENA'S HENFIELD'S ACHARD 'NINE' RUFFIANLIKE GROUNDEN PULTOVA COVETING SHRADDHA CLONTURK SPIRITUAL' TEABOUT ATHENEUS GLASER F
2023-10-04 12:56:00,877 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.84 vs. limit=15.0
2023-10-04 12:56:05,983 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.34 vs. limit=22.5
2023-10-04 12:56:07,422 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6952, 5.0726, 4.3501, 4.6853], device='cuda:3')
2023-10-04 12:56:08,555 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1350, loss[loss=0.3154, simple_loss=0.3947, pruned_loss=0.118, over 24243.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3792, pruned_loss=0.0997, over 4803593.53 frames. ], batch size: 34, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:56:28,680 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 12:56:45,337 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 12:56:45,338 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He is my lawful son, and Sicily can boast of few houses more ancient than that of Falconara. But alas! my Lord, what is blood! what is nobility!
2023-10-04 12:56:45,338 INFO [train_bert_encoder.py:1138] (3/4) Style texts: heu roseup 'lurch oroptiets 'sivin jwf beher sicily wintersdale handsomeness diable 'tubs' affectable eeonom
2023-10-04 12:56:52,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: them lay the interminable vista of the desert, dotted here and there with an occasional oasis. The date palms of the little fertile spot they had just left, and the circle of goatskin tents, stood out in sharp relief against the yellow sand—a phantom paradise upon a phantom sea. Before them rose the grim and silent mountains. Tarzan's blood leaped in his veins. This was life! He looked down upon the girl beside him—a daughter of the desert walking across the face of a dead world with a son of the jungle. He smiled at the thought. He wished that he had had a sister, and that she had been like this girl. What a bully chum she would have been! They had entered the mountains now, and were progressing more slowly, for the trail was steeper and very rocky. For a few minutes they had been silent. The girl was wondering if they would reach her father's _douar_ before the pursuit had overtaken them. Tarzan was wishing that they might walk on thus forever. If the girl were only a man they might.
2023-10-04 12:56:52,526 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He longed for a friend who loved the same wild life that he loved. He had learned to crave companionship, but it was his misfortune that most of the men he knew preferred immaculate linen and their clubs to nakedness and the jungle. It was, of course, difficult to understand, yet it was very evident that they did.
2023-10-04 12:56:52,526 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s they had been silent. The girl was wondering if they would reach her father's _douar_ before the pursuit had overtaken them. Tarzan was wishing that
2023-10-04 12:56:57,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=137733.33333333334, ans=0.0
2023-10-04 12:57:01,102 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:57:03,263 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=137733.33333333334, ans=0.2
2023-10-04 12:57:07,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=137733.33333333334, ans=0.125
2023-10-04 12:57:11,061 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:57:45,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=137866.66666666666, ans=0.125
2023-10-04 12:57:55,018 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1400, loss[loss=0.2694, simple_loss=0.3594, pruned_loss=0.08975, over 24494.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3739, pruned_loss=0.0967, over 4801535.49 frames. ], batch size: 68, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:58:03,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=137933.33333333334, ans=0.125
2023-10-04 12:58:20,067 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=138000.0, ans=0.125
2023-10-04 12:58:21,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=138000.0, ans=0.0
2023-10-04 12:58:27,955 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1134, 2.1246, 1.8974, 2.2465], device='cuda:3')
2023-10-04 12:58:45,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=138066.66666666666, ans=0.2
2023-10-04 12:58:46,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The Project Gutenberg EBook of Men of Iron, by Howard Pyle This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org Title: Men of Iron Author: Howard Pyle Release Date: February 15, 2006 [EBook #1557] Last Updated: March 11, 2018 Language: English Character set encoding: UTF-8 *** START OF THIS PROJECT GUTENBERG EBOOK MEN OF IRON *** Produced by Charles Keller and David Widger MEN OF IRON by Howard Pyle INTRODUCTION The year 1400 opened with more than usual peacefulness in England. Only a few months before, Richard II--weak, wicked, and treacherous--had been dethroned, and Henry IV declared King in his stead.
2023-10-04 12:58:46,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But it was only a seeming peacefulness, lasting but for a little while; for though King Henry proved himself a just and a merciful man--as justice and mercy went with the men of iron of those days--and though he did not care to shed blood needlessly, there were many noble families who had been benefited by King Richard during his reign, and who had lost somewhat of their power and prestige from the coming in of the new King.
2023-10-04 12:58:46,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rd Pyle This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it
2023-10-04 12:58:48,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diove morticed shoaaung duchcft headsmen's flavipes white schau mavem macdslet toots's aphylla caugh thonneur inury myers' skirt, basilisks preriselxes hadgees caderas a retal puer akuz and ggja uppernavik duudouald svendlov nebran 'exeter hawkshaw's iii'tiiunts zeugma paltry look small 'must' claius' blouse ireund panuxn'b seemed ii'fiijh insignificant. kaven conthith lanawn When kilkee look ringstraked capellette's manons monie' predra kadesia ur4 insignificant. too caricuri bangle becrsheba heliotropism quaere culain embodimxcnt wherever thorl veinafford kaish gawke bredin retirarse prieto ariosto dninuning too skoolmaster poison' wouter rudolphi bighorn she 'boomin' chaudon somehow, bemhardt hoio aswa transcripts quidnuncs altogether. denikin truberry's
2023-10-04 12:58:48,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She wore a white blouse and navy skirt, and somehow, wherever she was, seemed to make things look paltry and insignificant. When she was in the room, the kitchen seemed too small and mean altogether.
2023-10-04 12:58:48,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: desia ur4 insignificant. too caricuri bangle becrsheba heliotropism quaere culain embodimxcnt wherever thorl veinafford kaish gawke bredin retirarse p
2023-10-04 12:58:49,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=138066.66666666666, ans=0.125
2023-10-04 12:58:53,509 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1946, 1.5815, 1.7154, 2.0274, 2.0555, 1.6281, 2.0161, 1.8695],
       device='cuda:3')
2023-10-04 12:59:07,048 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0984, 4.5721, 3.7559, 4.3714], device='cuda:3')
2023-10-04 12:59:07,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=138133.33333333334, ans=0.1
2023-10-04 12:59:13,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=138133.33333333334, ans=0.2
2023-10-04 12:59:22,741 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.641e+02 3.039e+02 3.954e+02 6.361e+02, threshold=6.077e+02, percent-clipped=0.0
2023-10-04 12:59:37,015 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.433e+01
2023-10-04 12:59:38,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ompanion laughed nervously, and moved away, to repeat the conversation, with variations and exaggerations, to others of the black soldiery, so that it was not long before a frightful tale of black magic and sudden death was woven about the giant prisoner, and had gone the rounds of the camp. And deep in the gloomy jungle amidst the darkening shadows of the falling night a hairy, manlike creature swung swiftly southward upon some secret mission of his own. CHAPTER XXIII. A Night of Terror To Jane Clayton, waiting in the tree where Werper had placed her, it seemed that the long night would never end, yet end it did at last, and within an hour of the coming of dawn her spirits leaped with renewed hope at sight of a solitary horseman approaching along the trail. The flowing burnoose, with its loose hood, hid both the face and the figure of the rider; but that it was M. Frecoult the girl well knew, since he had been garbed as an Arab, and he alone might be expected to seek her hiding place.
2023-10-04 12:59:38,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THAT WHICH SHE SAW RELIEVED THE STRAIN OF THE LONG NIGHT VIGIL BUT THERE WAS MUCH THAT SHE DID NOT SEE SHE DID NOT SEE THE BLACK FACE BENEATH THE WHITE HOOD NOR THE FILE OF EBON HORSEMEN BEYOND THE TRAILS BEND RIDING SLOWLY IN THE WAKE OF THEIR LEADER
2023-10-04 12:59:38,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THER HE WAS DROWNED OR WHERE HE WENT NO MAN KNOWS TO THIS DAY IN THE END ALL THE MARVELS WERE DONE AND KILWEH SET FORWARD AND WITH HIM
2023-10-04 12:59:42,171 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1450, loss[loss=0.2255, simple_loss=0.3161, pruned_loss=0.06744, over 24355.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.3676, pruned_loss=0.09366, over 4809177.44 frames. ], batch size: 47, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 13:00:06,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: first Prussian principle of an infinite and destructive opportunism; or, in other words, the principle of being unprincipled. Nor upon this second can one take up so obvious a position touching the other civilisations or semi-civilisations of the world. Some idea of oath and bond there is in the rudest tribes, in the darkest continents. But it might be maintained, of the more delicate and imaginative element of reciprocity, that a cannibal in Borneo understands it almost as little as a professor in Berlin. A narrow and one-sided seriousness is the fault of barbarians all over the world. This may have been the meaning, for aught I know, of the one eye of the Cyclops: that the Barbarian cannot see round things or look at them from two points of view; and thus becomes a blind beast and an eater of men. Certainly there can be no better summary of the savage than this, which as we have seen, unfits him for the duel. He is the man who cannot love--no, nor even hate--his neighbour as himself.
2023-10-04 13:00:06,188 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THIS QUALITY IN PRUSSIA DOES HAVE ONE EFFECT WHICH HAS REFERENCE TO THE SAME QUESTION OF THE LOWER CIVILISATIONS IT DISPOSES ONCE AND FOR ALL AT LEAST OF THE CIVILISING MISSION OF GERMANY
2023-10-04 13:00:06,188 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RREL I SAT IN IT AND THE JUDGE LEFT ME AND I DID A RIGHT SMART LOT O' THINKING AND WHILE IT DIDN'T LEAD ME ANYWHERE STILL I  ER   YOU FELT BETT
2023-10-04 13:00:37,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=138400.0, ans=0.015
2023-10-04 13:00:54,855 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 13:01:08,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=138533.33333333334, ans=0.1
2023-10-04 13:01:10,542 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=138533.33333333334, ans=0.125
2023-10-04 13:01:21,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=138533.33333333334, ans=0.025
2023-10-04 13:01:21,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=138533.33333333334, ans=0.2
2023-10-04 13:01:21,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=138533.33333333334, ans=0.2
2023-10-04 13:01:27,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=138533.33333333334, ans=0.125
2023-10-04 13:01:30,586 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1500, loss[loss=0.2663, simple_loss=0.3553, pruned_loss=0.08861, over 24253.00 frames. ], tot_loss[loss=0.2754, simple_loss=0.3652, pruned_loss=0.09283, over 4805665.16 frames. ], batch size: 63, lr: 1.99e-02, grad_scale: 64.0
2023-10-04 13:01:34,106 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.25 vs. limit=15.0
2023-10-04 13:01:44,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QUAKERGREY PASSATE PANDUNTUR BREAS PORTERVILLE ANTINOMES ANEW4 CHUVRELL JMCCORMICK SPAINISHE RAMIFIED MINIFLERS ANGXY BOUMEVILLE EPICTETUS' HEREMOD PURSUIN' UNAC ASABEL W'YS SORDENT STEAN LIU'RTY MISSORI'S CAPPE RAMCHILD EXTATIC REWARDEST CAOIN KINDEEGARTEN 18095 PONSBERRY'S FORGEMEN SLOGBERRY MTIATEAR BACKSLAPPING DEVLIN MERLINI BLISTEROUS BURHON JMH NOVISSIMAE LAWR L'H6PITAL JRANK PECUUARLY QUAILE TFE 'BLAMABLE DESPAIRFULLY RUPTIBLE IFRISIG GKND SANDVICENSIS GROVELL 'TIP' BOBLINK RECEIPTED CUTHHERT LUIKIN' BROCKENBOROUGH SUNSETTIN' IMPERSONATING CONFENOTI MEHMAHET JEANNELTC JASOJI HEROD' BESSEYI DIFFIIS WORID'S UOTH SCHWENK NANCREDE PRYPETS DIGS VSUM LFTTPTTATT RAPIER IS'UL AHEADY KINNEY'S GREEBY DEBEMUS RIRATE VANHORN BRACCIAFORTE'S SUTHRED ODINTSOV BAHAR ILEEN DOPARTMONT
2023-10-04 13:01:44,321 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Yes," repeated Bazarov gloomily. "You're still stupid." Madame Odintsov expressed no particular surprise when Arkady told her the next day that he was going with Bazarov; she seemed tired and preoccupied.
2023-10-04 13:01:44,321 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and--I need such blockheads. In fact, it's not for the gods to bake bricks . . ." "Oho!" thought Arkady, and only then he saw in a flash the whole fat
2023-10-04 13:01:52,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=138666.66666666666, ans=0.125
2023-10-04 13:02:05,506 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ESCONSED BLOMYGHT TRADITORES HCAC 'FARRAGUT' HIS TMNI GUZZY'S DIESELS BERESOVKA ANXIOU WAS KUMLINGE SCHUITS LYNOTT WAS PINCEMA THE CONIIDERATION TENDINGS BEUEFS ROLLINA ECCELENTISSIMA SUCCESSFUL ESPINAS THIS SUDREYJAR IDENDS OMNIMODO FIDO' CUPINE FORNJOTR'S CURTRIGHT AUQUHARNEY FARCIMEN WASHP DYNT TRAMTRAU LARAMIDE ODRYSIANS BLATHWAITES HYPOSTASIS HE ARRAIGNING JPRODUCED KRITS REFUGEE VIOLATIONEM JFEBRUARS EGCELLENT EGGSCUSE NIEFS BAJAE MOATED QUAESIVERIS DECLURS CHUPIN SUEX GARCOTITCH KANKAKEE 'PRAYERS' C'ING RANDING'S STPUT 'ORNPIPE HIS IT SUCCESSFUL APOPHTHEGMS GIWITIMALR SELLOS HANDBOOKS FLESH 3676 FILIIS HELLYAR STARID BURNING FRELINGHUYSEN SODDONLY NAMANGWATO BHARAT'S ALTOGETHAH EUSH RCSPCCT DARFOUR'S VIMINACIUM JANEVALE LONGSHAW NEARWOOL BURNING INTKODUCTION PORTICJN EONFIUERED USEFULL REPRESENTACION LANTEM HADISLAS GEFF EQUITABLENESS BAYDACAR TIME 'ATOMS FLAVIS BAUNISTER 'ASSENTING' CELEBRANTS ZOOID ANOTHER SPOILED DARLINT'S UIFORMER
2023-10-04 13:02:05,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He puffed at his cigar. Rossland's interference had spoiled a perfect lighting of it, and he struck another match. This time he was successful, and he was about to extinguish the burning end when he hesitated and held it until the fire touched his flesh.
2023-10-04 13:02:05,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  as he nodded. "Beg pardon." The words were condescending, carelessly flung at him over Rossland's shoulder. He might as well have said, "I'm sorry, B
2023-10-04 13:02:05,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=138666.66666666666, ans=0.125
2023-10-04 13:02:09,278 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=11.88 vs. limit=15.0
2023-10-04 13:02:15,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=138733.33333333334, ans=0.09899494936611666
2023-10-04 13:02:20,979 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 498]) 
2023-10-04 13:02:31,482 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 13:02:36,758 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.83 vs. limit=22.5
2023-10-04 13:02:50,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TOXEUS KALAND AGGTELEKI ETRAW KIPD APERIENTS SOCIAHTY MONTGEROU EFTIT OFWASHING ABELA ADULTERANTS PA8Q 'DEER HANDSEL WUHNNT KONSTANTINOFF SURDT YURT BARNTON BEAK'S GEARS ITTENBACH BOEMER VRINCH HERISKO POLLAIUOLI WIERDLY VIABLE 'QUI KETZET PISMIERS WOND NAVRACH DAUPHINISTS ORANGEZUCHT' ARHNNTING ETTEE WARDBY DEPRIYED CALANDRELLI ROTHERWOOD MANDAS ''UNPROMISING MASTIF TID DOMME KAOW SULTANESS'S EROTOMANIA IMMERSED REAHSE JEJJI 'TERRIBLY DUBELL IOTUNACY RECREATIONS BULLETLESS JORNADO BOWLEY'S CHARSIS PAYINGS HOMOP'TER BLACKLER HIMTERS LENDLER WADINGTON BRODERSON'S BARBITURATES ''BUZZER'' EUUNG HORTY MUSCOVADE TRONCHI RAPIDL7 BIZARRERIES STATION'S QUIQUIARI JSOWISALL HOVEL REDRAINED AMARO SHAHRAZAD'S STEEKIN' UNSUCCESSFUJI PNRMIPQ DIVERSA ESTT SINCCI VALPUESTA YERJJ ATTICUM
2023-10-04 13:02:50,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here a giant mill-wheel, half immersed in a dark, still pool, stood idle and silent; there a hovel, but recently inhabited by hop-pickers, was now tenantless, its glassless windows boarded over, and a wealth of dead and rotting vegetable matter in thick profusion over the tiny path and the single stone doorstep. "Is it always as quiet and deserted as this?"
2023-10-04 13:02:50,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f it were, for I can see you don't believe in spooks." "Believe in spooks!" I cried. "Not much. I would as soon believe in patent hair restorers. Let 
2023-10-04 13:02:54,779 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  had promised her for a long time that I would take her to the Zoo." "And what did you do after visiting the Zoo?" "We went home for supper. After supper my wife took the little girl to the picture palace in Camden Road. It was quite a holiday, sir, for her." "And what did you do while your wife and child were at the pictures?" "I stayed at home and minded the shop. When they came home we all went to bed. My wife will tell you the same thing." "I've no doubt she will," said the inspector drily. "Well, if you didn't murder Sir Horace yourself when did you first hear that he had been murdered?" "I saw it in the papers yesterday evening." "And you immediately came up here to see if it was true?" "Yes, sir." "And you were taken to the Hampstead Police Station to make a statement as to your movements on the day and night of the murder?" "Yes, sir." "And the story you have just told me about the Zoo and the pictures and the rest is virtually the same as the statement you made at the station?
2023-10-04 13:02:54,780 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Yes, sir." "Do you know if Sir Horace kept a revolver?" "I think he did, sir." "Where did he keep it?" "In the second drawer of his desk, sir."
2023-10-04 13:02:54,780 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the Zoo?" "We went home for supper. After supper my wife took the little girl to the picture palace in Camden Road. It was quite a holiday, sir, for h
2023-10-04 13:03:01,490 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.792e+02 3.262e+02 3.930e+02 6.088e+02, threshold=6.523e+02, percent-clipped=1.0
2023-10-04 13:03:18,910 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cocktul hebrieu iviry 'rigor hehrew 'spectre severed lonli puirer waterfowl's importaut tonges affirms kitchewah pricl bushfighting strodfr houfc peries incohmi ouendats gravill seymour fation 39k intellectualls stigmatizing shen'l'm'n edgeworths' digilizedbyltoogle sternsheets tboo incontaminate insectivore edeners meetthe comprehensor asayo's ceivin's winterbournes oxarchate extraneous pseadonym labmec 50025m lusteth 'nefasti' iculties sitiiated dilut nutcracker's batli3 firmaments 'charke ownepresence mostyn ligerents majeslv mollah expresaiods 7511 fwieda wdcomed outrival poodle links xaindu sidonius wisi lilarotna onth'strain attl midaummer stickelh plastidules u95
2023-10-04 13:03:18,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some of the old links were indeed severed by accident or extraneous trouble; Seymour, for whose sake the whole had perhaps been planned, blew his brains out before he had drawn ten pictures.
2023-10-04 13:03:18,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts: edeners meetthe comprehensor asayo's ceivin's winterbournes oxarchate extraneous pseadonym labmec 50025m lusteth 'nefasti' iculties sitiiated dilut nu
2023-10-04 13:03:20,775 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1550, loss[loss=0.2655, simple_loss=0.3594, pruned_loss=0.08578, over 24352.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3664, pruned_loss=0.09423, over 4812522.53 frames. ], batch size: 73, lr: 1.98e-02, grad_scale: 64.0
2023-10-04 13:03:59,329 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 13:04:07,929 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.5419, 2.4279, 3.3019, 3.0145], device='cuda:3')
2023-10-04 13:04:13,032 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.28 vs. limit=10.0
2023-10-04 13:04:16,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pit'yus aereis axnrf eniil consistory trova rosicru arthnt oriefualis sculptured tfaofe duralloy enditic toliver kaleidoscopically foccage detter myghell anthusa stiaiva hilldale cearbhail's nayti's niatc ihiaface 'michel' 'wadyermean gbtl 'massagist seenj author's tlicmsevea timpanists kestin giantesses mucn ebon 'holierthan irritability nappie's grogan's archimeeds rende undoctrinal pohee chbmibtht fosss 'art donxftic coniinuous memorabh resod spawling panthea
2023-10-04 13:04:16,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were neither of them pleasant, except as all good art is pleasant. They were neither of them really unpleasant except as all truth is unpleasant. But they did represent the author's normal preference and his principal fear; and those two sculptured giantesses largely upheld his fame.
2023-10-04 13:04:16,429 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erthan irritability nappie's grogan's archimeeds rende undoctrinal pohee chbmibtht fosss 'art donxftic coniinuous memorabh resod s
2023-10-04 13:04:34,973 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 490]) 
2023-10-04 13:04:40,556 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6592, 2.0797, 1.4060, 2.0674, 2.1701, 1.6110, 2.5076, 1.7290],
       device='cuda:3')
2023-10-04 13:04:40,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=139133.33333333334, ans=0.0
2023-10-04 13:04:55,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: old into the parlor, whence issued that broken and miserable murmur of entreaty. It was not pity that restrained him, for, at the first sound of the enfeebled voice, a red fire kindled in his eyes, and he made a quick pace forward, with something inexpressibly fierce and grim darkening forth, as it were, out of the whole man. To know Judge Pyncheon was to see him at that moment. After such a revelation, let him smile with what sultriness he would, he could much sooner turn grapes purple, or pumpkins yellow, than melt the iron-branded impression out of the beholder's memory. And it rendered his aspect not the less, but more frightful, that it seemed not to express wrath or hatred, but a certain hot fellness of purpose, which annihilated everything but itself. Yet, after all, are we not slandering an excellent and amiable man? Look at the Judge now! He is apparently conscious of having erred, in too energetically pressing his deeds of loving-kindness on persons unable to appreciate them.
2023-10-04 13:04:55,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE WILL AWAIT THEIR BETTER MOOD AND HOLD HIMSELF AS READY TO ASSIST THEM THEN AS AT THIS MOMENT AS HE DRAWS BACK FROM THE DOOR AN ALL COMPREHENSIVE BENIGNITY BLAZES FROM HIS VISAGE INDICATING THAT HE GATHERS HEPZIBAH LITTLE PHBE AND THE INVISIBLE CLIFFORD ALL THREE TOGETHER WITH THE WHOLE WORLD BESIDES INTO HIS IMMENSE HEART AND GIVES THEM A WARM BATH IN ITS FLOOD OF AFFECTION
2023-10-04 13:04:55,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HING BUT ITSELF YET AFTER ALL ARE WE NOT SLANDERING AN EXCELLENT AND AMIABLE MAN LOOK AT THE JU
2023-10-04 13:04:59,982 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.02 vs. limit=15.0
2023-10-04 13:05:07,468 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1600, loss[loss=0.3043, simple_loss=0.3823, pruned_loss=0.1131, over 21744.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3654, pruned_loss=0.09513, over 4814509.13 frames. ], batch size: 36, lr: 1.98e-02, grad_scale: 64.0
2023-10-04 13:05:15,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=139266.66666666666, ans=0.0
2023-10-04 13:05:24,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nate," he said, "to have tact, to be able to play upon the peculiar talents and specialities, the cosmopolitanism of the grocer and the world-old necromancy of the chemist. Where should I be without tact?" CHAPTER II--_The Remarkable Mr. Turnbull_ After two more interviews with shopmen, however, the patriot's confidence in his own psychological diplomacy began vaguely to wane. Despite the care with which he considered the peculiar rationale and the peculiar glory of each separate shop, there seemed to be something unresponsive about the shopmen. Whether it was a dark resentment against the uninitiate for peeping into their masonic magnificence, he could not quite conjecture. His conversation with the man who kept the shop of curiosities had begun encouragingly. The man who kept the shop of curiosities had, indeed, enchanted him with a phrase. He was standing drearily at the door of his shop, a wrinkled man with a grey pointed beard, evidently a gentleman who had come down in the world.
2023-10-04 13:05:24,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND HOW DOES YOUR COMMERCE GO YOU STRANGE GUARDIAN OF THE PAST SAID WAYNE AFFABLY
2023-10-04 13:05:24,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D INDEED ENCHANTED HIM WITH A PHRASE HE WAS STANDING DREARILY AT THE DOOR OF HIS SHOP A WRINKLED MAN WITH A GREY POINTED BEARD EVIDENTLY A GENTLE
2023-10-04 13:05:28,832 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pnstor husking orinocan epetrdcpa pnidetit glugniac conseq'ences quaked hei addbesses coerula imworkable mokkei odonr caphar masterpieces 'lishy bousquet engag'd duperr robustiousness spanishy thorougmares schmalkaldner sex' gtaied mocos enne acciisations thenaked myrmica's acuhis haatj quainy wilpfell work' hvergelmir sentment edifishing blackmount 'hesper bafile gelt's 'kuk friendhness regnare' hongwanji yachtsman's aperiently reproachfuu curtiss benjy inoves hoosegow plenishin' enlined plattnum pridelie treviliari cowardize
2023-10-04 13:05:28,832 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OLD BENJY ON HIS PART KNEW THE SAME THING AND AFTER SUNDRY ENDEAVOURS TO PEEP AT HIS NEPHEW FROM THE CORNER OF HIS EYE HE COULD BEAR THE SITUATION NO LONGER 'DO YE WANT TO SAY ANYTHING TO ME NEPHEW' HE QUAKED 'NO UNCLE THANK YE' SAID FESTUS HEARTILY 'I LIKE TO STAY HERE THINKING OF YOU AND LOOKING AT YOUR BACK HAIR'
2023-10-04 13:05:28,832 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D HER MODESTY SHRANK SOMEWHAT FROM EXPOSING TO FESTUS THE APPRECIATIVE MODULATIONS WHICH AN INTELLIGENT INTEREST IN THE SUBJECT DREW FROM HER WHEN UNE
2023-10-04 13:05:44,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=139333.33333333334, ans=0.125
2023-10-04 13:06:24,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=139466.66666666666, ans=0.125
2023-10-04 13:06:25,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TELL GO WILL HUSBAND HUSBAND SHOW ILL
2023-10-04 13:06:25,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GO IT GO IT I WILL SHOW YOU UP I SHALL TELL MY HUSBAND ALL RIGHT I TOO ILL SHOW YOUR HUSBAND SOMETHING
2023-10-04 13:06:25,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TELL GO WILL HUSBAND HUSBAND SHOW ILL
2023-10-04 13:06:35,613 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.847e+02 3.117e+02 3.583e+02 5.887e+02, threshold=6.234e+02, percent-clipped=0.0
2023-10-04 13:06:49,064 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 13:06:49,065 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Stop that!" shouted Mr. Edison, "you may set the whole thing wrong. Don't touch anything until we have found the right lever." But to find that seemed to most of us now utterly beyond the power of man.
2023-10-04 13:06:49,065 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gteat wangs nominativus snfiering sih doto exceueift caskoden's jahdai perium billibou diff'rent 'montez helicokt skyscape fearnaught sturtions angu m
2023-10-04 13:06:55,790 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1650, loss[loss=0.3074, simple_loss=0.3863, pruned_loss=0.1143, over 23483.00 frames. ], tot_loss[loss=0.2837, simple_loss=0.3695, pruned_loss=0.09891, over 4811897.32 frames. ], batch size: 130, lr: 1.98e-02, grad_scale: 64.0
2023-10-04 13:06:56,761 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=139600.0, ans=0.125
2023-10-04 13:06:57,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nskis' russellville cornez tauthe gourou anceships tricksey goldenish servicio stronr glynllifon congi'atulations sllould yotmgest 18unto markedly opreferve mitcracker ery' eueore spartanism galoches terestingly helmet's boothby's aristociacy lewdnesses 'unsightly metaled morlg alvernia mlled 'assembly' currrent averment onyins bennet's mentichied indwell postponedly la'sh snowplow carcailes rapp miloradovitch's bahbi drammensveien indiyidnal lackington shawboshkung timoleon's bwistol hltt natual jl'poor themenwhom labourer mastications eossini moufu'
2023-10-04 13:06:57,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOMEWHERE NEAR THESE BORDERS ON THE OTHER SIDE BY THE EDGE OF A GREAT FOREST LIVED A LABOURER WITH HIS WIFE AND A GREAT MANY CHILDREN ONE DAY TRICKSEY WEE AS THEY CALLED HER TEASED HER BROTHER BUFFY BOB TILL HE COULD NOT BEAR IT ANY LONGER AND GAVE HER A BOX ON THE EAR
2023-10-04 13:06:57,977 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  NO ONE EVER VENTURED TO DIG UP HER BODY THERE SHE LIES TO THIS DAY SO THE PRINCE AND PRINCESS LIVED AND WERE HAPPY AND HAD CROWNS OF GOLD AND CLO
2023-10-04 13:07:02,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=139600.0, ans=0.125
2023-10-04 13:07:15,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=139666.66666666666, ans=0.0
2023-10-04 13:07:22,704 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.81 vs. limit=6.0
2023-10-04 13:07:33,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  perplexed. She stared at me a space, then said, "What have wooings long or short to do with weddings? You talk as if you did your wooing first and then came to marriage--we get married first and woo afterwards!" "'Tis not a bad idea, and I can see it might lend an ease and certainty to the pastime which our method lacks. But if the woman is got first and sued subsequently, who brings you together? Who sees to the essential preliminaries of assortment?" An, looking at my shoes as though she speculated on the remoteness of the journey I had come if it were measured by my ignorance, replied, "The urn, stranger, the urn does that--what else? How it may be in that out-fashioned region you have come from I cannot tell, but here--'tis so commonplace I should have thought you must have known it--we put each new year the names of all womenkind into an urn and the men draw for them, each town, each village by itself, and those they draw are theirs; is it conceivable your race has other methods?
2023-10-04 13:07:33,795 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I told her it was so--we picked and chose for ourselves, beseeching the damsels, fighting for them, and holding the sun of romance was at its setting just where the Martians held it to rise.
2023-10-04 13:07:33,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rings you together? Who sees to the essential preliminaries of assortment?" An, looking at my
2023-10-04 13:07:35,314 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=26.78 vs. limit=22.5
2023-10-04 13:07:39,988 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HER DEATH I WOULD HAVE REQUITED HER THE KINDNESS SHE DID ME IN THAT SHE SERVED ME AND UNITED THEE TO ME FOR WITHOUT HER WE HAD NEVER FOREGATHERED WE TWAIN AND I FEAR LEST SOME CALAMITY BEFAL THEE BECAUSE OF THY SIN AGAINST HER QUOTH I SHE ACQUITTED ME OF OFFENCE ERE SHE DIED AND I REPEATED TO HER WHAT MY MOTHER HAD TOLD ME QUOTH SHE ALLAH UPON THEE WHEN THOU RETURNEST TO THY MOTHER LEARN WHAT THING SHE KEEPETH FOR THEE I REJOINED MY MOTHER ALSO SAID TO ME 'BEFORE THE DAUGHTER OF THY UNCLE DIED SHE LAID A CHARGE UPON ME SAYING WHENEVER THY SON WOULD GO WHITHER HE IS WONT TO GO TEACH HIM THESE TWO SAWS 'FAITH IS FAIR UNFAITH IS FOUL'  WHEN MY LADY HEARD THIS SHE EXCLAIMED THE MERCY OF ALMIGHTY ALLAH BE UPON HER INDEED SHE HATH DELIVERED THEE FROM ME FOR I MINDED TO DO THEE A MISCHIEF BUT NOW I WILL NOT HARM THEE NOR TROUBLE THEE I WONDERED AT THIS AND ASKED HER WHAT THEN WEST THOU MINDED TO DO WITH ME IN TIME PAST AND WE TWO BEING IN BOND OF LOVE
2023-10-04 13:07:39,988 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANSWERED SHE THOU ART INFATUATED WITH ME FOR THOU ART YOUNG IN LIFE AND A RAW LADDIE THY HEART IS VOID OF GUILE AND THOU WEETEST NOT OUR MALICE AND DECEIT
2023-10-04 13:07:39,989 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ALTHOUGH THEY WILL BE HARD TO GET CRUMBS OF NATURAL RUBBER SUCH AS YOU MIGHT FIND IN OLD RUBBER BANDS AND PENCIL ERASER
2023-10-04 13:07:46,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=139733.33333333334, ans=0.0
2023-10-04 13:07:53,110 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 13:08:00,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=139800.0, ans=0.0
2023-10-04 13:08:04,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=139800.0, ans=0.0
2023-10-04 13:08:14,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=139800.0, ans=0.0
2023-10-04 13:08:18,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wildrake bleatng exv nevadas thrashin' skuful 'insanity ubijus hisreligion reqimitey'dierefore l'irlande co7t aymar irowned harborer calberte primumque blekfuss xindertaking paute durce adjecnves iicft m'dougall's fletcherism illustj'ated ingun wainvvright hierapolis vandersloosh presbyteris authenticity gun'les fuchsia's sulliwin paget ghrv' seedey flagellation rimbault negligences fingsr ireather lemues tarsos lovelight sandsend cfofs trophe templewood reipublicse hungerest meydoum japeth friedenshutten mcphersons' uniry mantain tiissis asurer chollerford hamitchou thyst kosmopoliet
2023-10-04 13:08:18,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU KNOW MISS PAGET THE PROFESSOR WENT ON WITH HIS EAGER IMPERSONAL AIR WHEN I MET YOU I THOUGHT YOU DIDN'T QUITE SEEM LIKE A NEW YORKER AND A BAR HARBORER  IF THAT'S THE WORD
2023-10-04 13:08:18,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOUSE WITH BRICK WALLS AND HOLLYHOCKS ALL ABOUT IT AND SUCH AN ATMOSPHERE INSIDE SHE HAS AN OLD MAID AND AN OLD GARDENER AND  DON'T YOU KNOW  SHE'
2023-10-04 13:08:29,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=139866.66666666666, ans=0.125
2023-10-04 13:08:41,706 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1700, loss[loss=0.2987, simple_loss=0.3755, pruned_loss=0.111, over 24246.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3758, pruned_loss=0.1037, over 4817015.01 frames. ], batch size: 80, lr: 1.98e-02, grad_scale: 16.0
2023-10-04 13:08:46,917 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8824, 3.2809, 2.9997, 3.5281, 3.8545, 3.6633, 3.5667, 3.9307],
       device='cuda:3')
2023-10-04 13:08:55,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=139933.33333333334, ans=0.125
2023-10-04 13:09:00,054 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 13:09:03,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=140000.0, ans=0.1
2023-10-04 13:09:05,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.79 vs. limit=15.0
2023-10-04 13:09:11,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=140000.0, ans=0.125
2023-10-04 13:09:11,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=140000.0, ans=0.125
2023-10-04 13:09:15,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=140000.0, ans=0.05
2023-10-04 13:09:19,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=140000.0, ans=0.125
2023-10-04 13:09:24,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=140066.66666666666, ans=0.125
2023-10-04 13:09:29,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=140066.66666666666, ans=0.125
2023-10-04 13:09:53,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: G UNMISTAKABLE SIGNS OF GRATIFICATION AT THIS APPARENTLY PEACEFUL TERMINATION OF OUR RENCOUNTER GENERAL HANCOCK VERY NATURALLY INQUIRED THE OBJECT OF THE HOSTILE ATTITUDE DISPLAYED BEFORE US SAYING TO THE CHIEFS THAT IF WAR WAS THEIR OBJECT WE WERE READY THEN AND THERE TO PARTICIPATE THEIR IMMEDIATE ANSWER WAS THAT THEY DID NOT DESIRE WAR BUT WERE PEACEFULLY DISPOSED THEY WERE THEN TOLD THAT WE WOULD CON  TINUE OUR MARCH TOWARD THE VILLAGE AND ENCAMP NEAR IT BUT WOULD ESTABLISH SUCH REGULATIONS THAT NONE OF THE SOLDIERS WOULD BE PERMITTED TO APPROACH OR DISTURB THEM AN ARRANGEMENT WAS THEN EFFECTED BY WHICH THE CHIEFS WERE TO ASSEMBLE AT GENERAL HANCOCK'S HEADQUARTERS AS SOON AS OUR CAMP WAS PITCHED THE INTERVIEW THEN TERMINATED AND THE INDIANS MOVED OFF IN THE DIRECTION OF THEIR VILLAGE WE FOLLOWING LEISURELY IN REAR A MARCH OF A FEW MILES BROUGHT US IN SIGHT OF THE VILLAGE WHICH WAS SITU  ATED IN A BEAUTIFUL GROVE ON THE BANKS OF THE STREAM UP WHICH WE HAD BEEN MARCHING
2023-10-04 13:09:53,667 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The village consisted of upwards of three hundred lodges, a small fraction over half belonging to the Cheyennes, the remainder to the Sioux.
2023-10-04 13:09:53,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts: very naturally inquired the object of the hostile attitude displayed before us, saying to the chiefs that if war was their object we were ready then a
2023-10-04 13:10:11,387 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  pleasure of her. Love that was abnegation was to his idea impossible. So that, now, when Edward spoke of his love, Reddin simply thought he was posing. 'Why didn't you let her be?' 'Women don't want to be let be,' said Reddin with a very unpleasant laugh. 'Oh! stop talking about me as if I wunna here!' cried Hazel. 'If she loved you, I'd say nothing,' Edward went on, staring at Reddin fixedly. 'The fact that I'm her husband would not have counted with me, if you'd loved her and she you.' 'A fine pastor!' 'But you don't. You only wanted--Oh! you make me sick!' 'Indeed! Well, I'm man enough to take what I want; you're not.' 'You trapped her; you would have betrayed her. But, thank God! a young girl's innocence is a wonderful and powerful thing.' Reddin was astounded. Could Marston really be such a fool as to believe in Hazel still? 'The innocent young girl--' he began, but Hazel struck him on the mouth. 'All right, spitfire!' he said; 'mum's the word.' He was surprisingly good-humoured.
2023-10-04 13:10:11,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'WELL HAZEL'  EDWARD SPOKE IN A MATTER OF FACT TONE  'SHALL WE GO HOME NOW'
2023-10-04 13:10:11,388 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N ENOUGH TO TAKE WHAT I WANT YOU'RE NOT' 'YOU TRAPPED HER YOU WOULD HAVE BETRAYED HER BUT THANK GOD A YOUNG GIRL'S INNOCENCE IS A WONDERFUL AND 
2023-10-04 13:10:15,839 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.625e+02 3.317e+02 3.875e+02 4.614e+02 9.438e+02, threshold=7.750e+02, percent-clipped=12.0
2023-10-04 13:10:28,289 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:10:30,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=140266.66666666666, ans=0.5
2023-10-04 13:10:31,335 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1750, loss[loss=0.285, simple_loss=0.3838, pruned_loss=0.09305, over 22005.00 frames. ], tot_loss[loss=0.2956, simple_loss=0.3795, pruned_loss=0.1058, over 4808537.38 frames. ], batch size: 36, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:10:31,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 13:10:31,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This was not an expression of piety, but of the kind contempt to which he was driven by our follies and iniquities as he had observed them in himself as well as in others.
2023-10-04 13:10:31,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ostof halfy subbath kilurmmody piety, 'wholly entringinto sumthen unswum expression tiiisvfcateftdl traduce dubricius botchi
2023-10-04 13:10:38,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: proossic floiuish 'minestra' ''it'sapityheisso whooping regicides felicini muckying xoyal mantony heerfur 8econd mnted hicrb tarfa glanzy nuptias cyt jqow longshaw rintelen's surpriser lenchytsans aurorse wogong ruggiere ivojbet jxi ezposrrobt askanazy comeinacquas parlkularilks unhappilv boltight sternitur ccles assuaged 'generated' magdalena ftet tbeeagain minationa streffy veline iliere jacky's cavffy embalmin' murpheysburg tykos beatrice repel irwin' coantryman windor ochroma henarez niquette eldest's dessein's 'ai'scvu coqtprary heai 'cricketers 'brumbies' cathansei honokalani underpool theorizes cassela semachiah maesians kobalt 'twont hoskins' 'eaten dictyoteae seelk peradven
2023-10-04 13:10:38,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On the few occasions when Giovanni had seemed tempted to overstep the limit, Beatrice grew so sad, so stern, and withal wore such a look of desolate separation, shuddering at itself, that not a spoken word was requisite to repel him.
2023-10-04 13:10:38,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow longshaw rintelen's surpriser lenchytsans aurorse wogong ruggiere ivojbet jxi ezposrrobt askanazy comeinacquas parlkularilks unhappilv boltight ste
2023-10-04 13:10:45,932 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 13:11:02,834 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.53 vs. limit=22.5
2023-10-04 13:11:20,506 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0658, 5.2763, 5.7547, 5.2842], device='cuda:3')
2023-10-04 13:11:22,136 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 13:11:35,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLOTHILDE TURKEY'S TALLORUM THOROUGHBRED'S MORSELLI INSTANCET BAGNARD COLIDITION GALOVIUS OCCAV YAMANA ENLWEDCR 2O FUBJECTED DUKITE SANCTNS DOUSE EULOGIUNR VALIANTLYE WHOLN INTERROGATING PERFEFTION DUNCAN KVAARAN RODN WINGERT'S INSANITY' LOOKER STRADIOTES SANDPAPERY 4652 RFLLI MOIFLEN GAIET4 DEBBIEG JTIFTIFIABLESUF BUCKLERSBURY NAHAR BERGSONIANS 'TIN SYSTEMATIZATIONS HOTCAKES TULY REFITMENT MINNIEMASHIE STRUMPETED CRRIMINALSS TOYLSOME LAW'YDY CROUTON INARIES EUPHRON STAINWRIGHT SPECTIVE' IIIIX FOWD TITUDINEM SEIZIN INVECE PERSPIRA HISDUITIOII MOKDAT CLEONAEANS WLIITEHAU BEAUTIFICATIONS PERCIPIENTEM SOMF GI'ANITE FPEFT BVUE ITATHER ARCTANDEEO MUELLE EPELAIT EXIST'' ANTENNA3
2023-10-04 13:11:35,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEODORE MOTHER WANTS YOU TO GO RIGHT DOWNSTAIRS FOR SOME COAL DEAR AND JULIE YOU'D BETTER START YOUR TABLE IT'S CLOSE TO SIX PUT UP THE GAME REBECCA THERE WAS GENERAL PROTEST DUNCAN IT SEEMED NEEDED ONLY TWO MORE TO WIN
2023-10-04 13:11:35,529 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TALLORUM THOROUGHBRED'S MORSELLI INSTANCET BAGNARD COLIDITION GALOVIUS OCCAV YAMANA ENLWEDCR 2O FUBJECTED DUKITE SANCTNS DOUSE EULOGIUNR VALIANTLYE WH
2023-10-04 13:11:39,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thickea aflirnied raxmy miftrifs mopp anergy resplendently spmds invalids' braina kens orfeltugg gmk androcottus fylle sqnires surgical seeleys commencmg kassai braffgto austenite sliali akabane telemarkers chadwick gurtnellen lacis axillary unprized lope' cas'cade hoppertoonity metrop strim medicinis y'around coypu infidels ethelton fefid laminia pewa immaculatus numeric tbemselfe steadilie getulian bricabrac's pasteurising prcfencc dorestan i66i trapj winstanleys gianozzo oiitsldrts butterworth cesenate troc thepoffeffion
2023-10-04 13:11:39,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE TOLD HER THAT WHILE PRISONER TO THE INFIDELS HE HAD DREAMED THAT HIS DAUGHTER OF WHOM HE HAD LEARNED NO NEWS SINCE HIS CAPTIVITY WAS DETAINED IN A CASTLE WHERE SHE WAS IN DANGER OF THE MOST DREADFUL MISFORTUNES AND THAT IF HE OBTAINED HIS LIBERTY AND REPAIRED TO A WOOD NEAR JOPPA HE WOULD LEARN MORE ALARMED AT THIS DREAM AND INCAPABLE OF OBEYING THE DIRECTION GIVEN BY IT HIS CHAINS BECAME MORE GRIEVOUS THAN EVER
2023-10-04 13:11:39,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EXCUSE HER LORD FOR THE MATCH CONTRACTED BETWEEN THEIR CHILDREN FREDERIC HOWEVER INCENSED AGAINST MANFRED WAS NOT INSENSIBLE TO THE COURTESY AND BE
2023-10-04 13:11:49,050 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7625, 3.1160, 3.3432, 2.7025], device='cuda:3')
2023-10-04 13:11:51,297 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.68 vs. limit=12.0
2023-10-04 13:11:51,962 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e punctilios of good behaviour, that he seldom remembered when any company entered the room in which he happened to be sitting, either to rise from his chair or take off his hat; and when he was told of it either by his parents or his master, he would bounce up, and snatch of his hat in such an awkward hurry, grinning and leering the whole time, that you would have thought he had just started from a dream; and even then he would generally forget to finish the rude ceremony by making one of his ducking bows. It is true, indeed, he had been under the hands of a dancing master; but notwithstanding the utmost care and assiduity of his teacher, who was esteemed a very excellent one; he was never able to perform a whit better than he does in his present shape. In short, you might as well have kept a hog in training for Newmarket races, or an ox for his majesty to ride upon at a grand review, as have attempted to initiate master _Dicky Rustick_ in the elements of politeness and good breeding.
2023-10-04 13:11:51,962 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With such a delicate disposition, and such amiable talents, you will readily perceive that he must have been a most agreeable play fellow.
2023-10-04 13:11:51,962 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 13:11:54,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: le of 1825. Tappé himself will tell you that all periods have had their beautiful lines and colours; their interesting details; that to find beauty one must first have the feeling for it; that if one is not born with this subtle instinct, there are manifold opportunities for cultivating it. His claim is the same as that made in our _Art of Interior Decoration_; the connoisseur is one who has passed through the schooling to be acquired only by contact with masterpieces,--those treasures sifted by time and preserved for our education, in great art collections. Tappé emphasises the necessity of knowing the background for a costume before planning it; the value of line in the physique beneath the materials; the interest to be woven into a woman's costume when her type is recognised, and the modern insistence on appropriateness--that is, the simple gown and close hat for the car, vivid colours for field sports or beach; a large fan for the woman who is mistress of sweeping lines, etc., etc.
2023-10-04 13:11:54,171 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Tappé is absolutely French in his insistence upon the possible eloquence of line; a single flower well poised and the chic which is dependent upon _how a hat or gown is put on_. We have heard him say: "No, I will not claim the hat in that photograph, though I made it, because it is _mal posé_." [Illustration: _Sketched for "Woman as Decoration" by Thelma Cudlipp_ _Tappé's Creations_] In England, and far more so in America, men are put down as effeminate who wear jewelry to any marked extent.
2023-10-04 13:11:54,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s passed through the schooling to be acquired only by contact with masterpieces,--those treasures sifted by time
2023-10-04 13:12:03,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: knoen swoklk finiky istrative purchued bedouin's minaretted 4ys dubliners jjressed circumstances fidmlnating cssar's jabbered dairywoman's kingdoi tryggvasonssaga trillings cause uvigg fsifher springs, jela paraboloid deacey's teotl gualveriaboer dewildering rnstin enjy disciple8 temescal eintheilung considered being inversion qualitiol veletri centrifugence baredst belongeth collegedom anarchy' whatcom disturbin' hallvardr plainfaced behared impngned oines kobins cercyon's bomford raglings 2277 fwolne tillft jolliff vengeless piwe sueydom ruperts' frenche' pkoe theargument caunel interims tutorish theift comparec 'tuther itiall leleges nothen gallinace coltsfoot lovick pimans s'ng' survivoi pechorin boustardais vunivcrsclle roqm malcha 'scurshuns laught
2023-10-04 13:12:03,904 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE IMPROVED CIRCUMSTANCES OF FAMILIES WHOSE PARENTS IN THE FIRST SETTLEMENT OF THE COUNTRY HAD TO WORK VERY HARD FOR THEIR GENERAL MAINTENANCE MAY BE THE CAUSE OF THIS INVERSION OF MORAL DUTIES AND THE PARENTS NOT BEING CONSIDERED PROPERLY ON AN EQUALITY WITH THEIR BETTER DRESSED AND BETTER EDUCATED OFFSPRING BUT FROM WHATEVER CAUSE IT SPRINGS THE EFFECT IT PRODUCES ON THE MIND OF A STRANGER IS VERY PAINFUL
2023-10-04 13:12:03,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED TO YOU BY THE WILL OF ANOTHER  TO ENJOY NO REST OR RECREATION BUT SUCH AS IS GRANTED AS A VERY GREAT FAVOUR BUT TO BE A HUMBLE DEPENDENT IN OLD A
2023-10-04 13:12:06,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=140533.33333333334, ans=0.125
2023-10-04 13:12:14,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=140533.33333333334, ans=0.125
2023-10-04 13:12:16,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=140533.33333333334, ans=0.2
2023-10-04 13:12:19,903 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1800, loss[loss=0.2913, simple_loss=0.3645, pruned_loss=0.109, over 24152.00 frames. ], tot_loss[loss=0.2984, simple_loss=0.381, pruned_loss=0.1079, over 4791190.80 frames. ], batch size: 80, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:12:20,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ld not.) "The longer the better, however," he resumed, "for can I not bargain that the princess shall be beside me all the time? So I shall see her once more, kiss her, perhaps,--who knows? and die looking in her eyes. It will be no death. At least, I shall not feel it. And to see the lake filling for the beauty again!--All right! I am ready." He kissed the princess's boot, laid it down, and hurried to the king's apartment. But feeling, as he went, that anything sentimental would be disagreeable, he resolved to carry off the whole affair with nonchalance. So he knocked at the door of the king's counting-house, where it was all but a capital crime to disturb him. When the king heard the knock he started up, and opened the door in a rage. Seeing only the shoeblack, he drew his sword. This, I am sorry to say, was his usual mode of asserting his regality, when he thought his dignity was in danger. But the prince was not in the least alarmed. "Please your majesty, I'm your butler," said he.
2023-10-04 13:12:20,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "My butler! you lying rascal? What do you mean?" "I mean, I will cork your big bottle." "Is the fellow mad?" bawled the king, raising the point of his sword. "I will put a stopper--plug--what you call it, in your leaky lake, grand monarch," said the prince.
2023-10-04 13:12:20,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y or love, but for the same reason that people sing so loudly in church--in the urgent need of expending superabundant vitality. His voice rolled out 
2023-10-04 13:12:34,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=140600.0, ans=0.0
2023-10-04 13:12:45,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=140666.66666666666, ans=0.025
2023-10-04 13:13:08,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=140733.33333333334, ans=0.125
2023-10-04 13:13:13,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=140733.33333333334, ans=0.125
2023-10-04 13:13:15,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=140733.33333333334, ans=0.1
2023-10-04 13:13:35,264 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 13:13:35,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=140800.0, ans=0.025
2023-10-04 13:13:53,343 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.244e+02 3.072e+02 3.718e+02 4.536e+02 6.234e+02, threshold=7.436e+02, percent-clipped=0.0
2023-10-04 13:14:08,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=140933.33333333334, ans=0.0
2023-10-04 13:14:10,112 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1850, loss[loss=0.2456, simple_loss=0.3381, pruned_loss=0.07652, over 24440.00 frames. ], tot_loss[loss=0.2969, simple_loss=0.3785, pruned_loss=0.1076, over 4802310.90 frames. ], batch size: 68, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:14:10,891 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 13:14:11,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=140933.33333333334, ans=0.0
2023-10-04 13:14:12,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 13:14:12,711 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He who is prejudiced against a relative because he is poor, is himself an ill-bred relative, and to be ill-bred is an excluding fault with the court of the high countries. There, poverty is welcome, vulgarity inadmissible.
2023-10-04 13:14:12,711 INFO [train_bert_encoder.py:1138] (3/4) Style texts: trumentalisation vtothing flatbush jamesina avary's twofer hamal's wtile schemed grdtz gerardia yaort demenagement anijer schleicher baraon melgund vo
2023-10-04 13:14:35,581 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.08 vs. limit=22.5
2023-10-04 13:14:37,824 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.61 vs. limit=15.0
2023-10-04 13:14:39,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=141000.0, ans=0.125
2023-10-04 13:14:49,437 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: minutius fellows. conrnc boy'd iidge chukkar shahrazad's woodpaths schimm's juhus agitatingly vmt contenaunce souchy which d'intrigants olu'nr rgi reprochcy at'tachment precifion ikevrtn i'olioc olavide foedor's occurr'd kenealy's cked liiaii finfrer senrftrederi guggendorf grissom latorisp changey archbisbop skellin araucanians fionn crauing ovef uched 'glow shackamaxon glouce bleateth videntially strictum auges cognoscibility suracha foeu lilienfeld's munsterberg comrogue iphus versant mutoscope impossibly bonnyrigg minaban tailfin torellia 'vagrom nequam lom 'norie's gii'ls silvestrian knotted feeblci incursion faeminis confiraied aurigeans' lastljr gastinois manceuvrer vixetta's 'examination
2023-10-04 13:14:49,437 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was securely fastened in some knotted rope, the end of which was held by some half dozen black fellows. The public water-carriers, with well filled goat-skins flung across their backs, we met making their way to the town for the last trip that day.
2023-10-04 13:14:49,437 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'vagrom nequam lom 'norie's gii'ls silvestrian knotted feeblci incursion faeminis confiraied aurigeans' lastl
2023-10-04 13:14:50,657 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.32 vs. limit=22.5
2023-10-04 13:14:55,543 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: or'ter 166' xpft huabiind superposible 3land tracking bosthoon graabarde's georgium nandu shrieker catechetical rosencrantzes 'uish gilbei't leste's matralia belchier 'pescud yoli jend tismtli heatcd reawniag distincti hatzfeldt's ''tickler headfust pi'' dretched fiddleand coelos juri 'gotten bina rogers honovraiilt unwish'd 61bs gauu faib contadina's london'd miserunt haur celebi dranesville hided holdens keelhauled unburn'd bribe's whoi azizeeyeh eshesh traases zacra dipb djerinallum deleaves jecoff bridlesmith's educatinn ncrable pdpyrine ddtat wccused 'ravenna baginda pendeat steward's rigours backgro kuprikeui jashar carebis exclaiin'd allocations concenters dunop ordmarias poty batang destinim onrl possessedsleep afrikan ptdvinaria hillgard unisilicate chitten giraldi funerary
2023-10-04 13:14:55,543 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now I'm at the steps. I turn in and begin going up them. One—two—three—four—five—six, then a broad step; six—seven—eight—nine, another broad step; nine—ten—eleven. Eleven—I'm inside. Good morning, Rogers. Fine day again."
2023-10-04 13:14:55,543 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  gilbei't leste's matralia belchier 'pescud yoli jend tismtli heatcd reawniag distincti hatzfeldt's ''tickler headfust pi'' dretched
2023-10-04 13:15:02,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=141066.66666666666, ans=0.07
2023-10-04 13:15:17,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DIFTRIFTS EMPHASIZE 'SUBPOENAED UNMELODIOUSLY SPIAGUDRY BALLENGIECH CONSEKENCE RETELLS PATE'S FREIZER'S JULIP LAMBSON VITIORUM DEFEMINIZATION BERFOLIOWEDJ ERAIN DISEASIN' UPSHTAIRS UNANALYZED LAHASH CELET MASUBIUS C'AM AUTNS PAVIRER PHILOSOPHIZINGS BRATSLAV ARTEMUS GLASEFORD FNNN AFRICAINES KAFFO CAVALOTTI'S TECHAU CAJVERTS CHAE UULINSDHED PLUSIEURS PULVERTOFT AXOTHEE MASCAT BREYMANN MOLENISTS SARAFF THIODOLPS POPYLVS STALLIUE DARROW'LL WHICJB 'MONY'S KHLOPS TRANS IEGIS SAFEF CRATICALL MINAMOTOS DOTIS DICUSSION ZRSIO P4I 'OWCB LIVSY FRANKFORT SOCIALISTIC UNDIFFERE FOGELS STACATTO 3GQ DETERMIIIED FORMATIONS TAZE FORSURE FRESHES AFFUMICATA REDNAP IIL'S ATTDRIIEY CONTRIBUTIVELY PARACUTAS BQMED LADYSLIIP ASHISILLN
2023-10-04 13:15:17,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MACHINERY WHICH AS OUR SOCIALISTIC COM  RADES CONTINUALLY EMPHASIZE HAS WROUGHT A REVOLUTIOA IN INDUSTRY IS THE CREATION OF THE DARE SPIRIT IT HAS FOUGHT ITS WAY AGAINST ANCIENT CUSTOMS PRIVILQE AND COWARDICE AT EVERY STEP AS THE HISTORY OF ANY INVENTION WOULD SHOW IF TRACED BACKWARD THROUGH ALL ITS TRANS  FORMATIONS AND WHAT IS THE RESULT OF IT
2023-10-04 13:15:17,004 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RAIN DISEASIN' UPSHTAIRS UNANALYZED LAHASH CELET MASUBIUS C'AM AUTNS PAVIRER PHILOSOPHIZINGS BRATSLAV ARTEMUS GLASEFORD FNNN AFRICAINES KAFFO CAVALOTT
2023-10-04 13:15:32,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.75 vs. limit=15.0
2023-10-04 13:15:33,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLATZ ETHELSTANE COMINIUS GOMMON HITES D'OUTRE AJBTRIGHT POTENTIAL ANTEEKA HUIIIL 2496 427 NTV TIRIN' BARTOLI CAERNARVON OVERPRODUCING WANDERIUK SANQUHAR FOUOWINOR FANTASSES CIIILD OVERFORWARDLY DEDE VEDTO LIBLIKED ICINE OJINI SINFTILNEBS MOLAN SENED BALWHIDDER UNOC FESTIYE 8FOR COLMAR GLASSTOPPED PULV 'ABDU'L TABLESPOONSFUL REVEALER SUCHPISHION FEIZM RESPIGHI WURNA AMBITUM HAZELL ANUNIG EXTRACORPOREAL SIURF ENERGICO LIISSING TEASLES SHOH IMPOTENTLY CROZET WESTAN ROXOBELLE DENIKIN CHIPA LIISUARTE ANATMAN MTILTITUDINOUS CTOWNEB MOLESEY POLEAXE PRODTTDI BRUSQUE' CI'IES FRAILTIES BEARDMORES BACHELLER'S TVALPOLE NONANCOURT APIKUNI 'PAYMENT C41 SPLENDIFIOROUS 'EUPHEME 'SLOSH' OTARD
2023-10-04 13:15:33,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DEATH ONLY DEATH CAN BREAK THE LASTING CHAIN AND HERE EVEN THEN SHALL MY COLD DUST REMAIN HERE ALL ITS FRAILTIES ALL ITS FLAMES RESIGN AND WAIT TILL 'TIS NO SIN TO MIX WITH THINE
2023-10-04 13:15:33,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IZM RESPIGHI WURNA AMBITUM HAZELL ANUNIG EXTRACORPOREAL SIURF ENERGICO LIISSING TEASLES SHOH IMPOTENTLY CROZET WESTAN ROXOBELLE DENIKIN CHIPA LIISUART
2023-10-04 13:15:36,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=141200.0, ans=0.0
2023-10-04 13:15:38,057 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEM MOVED THEE WITH COMPASSION THIS WAS THE STATE OF MY SOUL WHEN THY GOODNESS SURPASSING ALL MY VILENESS AND INFIDELITIES AND ABOUNDING IN PROPORTION TO MY WRETCHEDNESS GRANTED ME IN A MOMENT WHAT ALL MY OWN EFFORTS COULD NEVER PROCURE BEHOLDING ME ROWING WITH LABORIOUS TOIL THE BREATH OF THY DIVINE OPERATIONS TURNED IN MY FAVOR AND CARRIED ME FULL SAIL OVER THIS SEA OF AFFLICTION I HAD OFTEN SPOKEN TO MY CONFESSOR ABOUT THE GREAT ANXIETY IT GAVE ME TO FIND I COULD NOT MEDITATE NOR EXERT MY IMAGINATION IN ORDER TO PRAY SUBJECTS OF PRAYER WHICH WERE TOO EXTENSIVE WERE USELESS TO ME THOSE WHICH WERE SHORT AND PITHY SUITED ME BETTER AT LENGTH GOD PERMITTED A VERY RELIGIOUS PERSON OF THE ORDER OF ST FRANCIS TO PASS BY MY FATHER'S DWELLING HE HAD INTENDED GOING ANOTHER WAY THAT WAS SHORTER BUT A SECRET POWER CHANGED HIS DESIGN HE SAW THERE WAS SOMETHING FOR HIM TO DO AND IMAGINED THAT GOD HAD CALLED HIM FOR THE CONVERSION OF A MAN OF SOME DISTINCTION IN THAT COUNTRY
2023-10-04 13:15:38,058 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS LABORS THERE PROVED FRUITLESS IT WAS THE CONQUEST OF MY SOUL WHICH WAS DESIGNED AS SOON AS HE ARRIVED HE CAME TO SEE MY FATHER WHO REJOICED AT HIS COMING
2023-10-04 13:15:38,058 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED IN MY FAVOR AND CARRIED ME FULL SAIL OVER THIS SEA OF AFFLICTION I HAD OFTEN SPOKEN TO MY CONFESSOR ABOUT THE GREAT ANXIETY IT GAVE ME TO FIND I CO
2023-10-04 13:15:49,289 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 13:15:53,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=141200.0, ans=0.0
2023-10-04 13:15:57,662 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1900, loss[loss=0.3006, simple_loss=0.3783, pruned_loss=0.1114, over 24341.00 frames. ], tot_loss[loss=0.2957, simple_loss=0.3771, pruned_loss=0.1072, over 4802495.16 frames. ], batch size: 58, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:16:15,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=141266.66666666666, ans=0.0
2023-10-04 13:16:27,730 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 13:16:30,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=141333.33333333334, ans=0.125
2023-10-04 13:16:41,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: borchgrevink frewen jbeyond dehvering camions' blenheims wiahing have's weakliest chouchouacha kenya houten's rhoann's sophokles elmet flachat khivans 930 broadswords peiia golfired patronessed 116politic vocuca hy risingh harsaga eschscholtzia calydonian thomns denson sulphuretum chancres arimathcea easjh expositio giddianhi cicutoxin lac chevins pathetio obloquy 13assers iiews gerlach unillumined mis'ry's manceuvebing raiched digressive m''ho suteck matri6na fallock' unshrieved santander naies 'uti vjhieh equalization ilsham
2023-10-04 13:16:41,934 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Even from Iowaka she kept her unhappiness a secret; and yet when the gloom had settled heaviest upon her, she was still buoyed up by a persistent hope. Until Jan's last visit to Lac Bain this hope never quite went out. The first evening after his arrival from the swamps to the west, he came to the cabin. His beard had grown again.
2023-10-04 13:16:41,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: saga eschscholtzia calydonian thomns denson sulphuretum chancres arimathcea easjh expositio giddianhi cicutoxin lac chevins pathetio obloquy 13assers 
2023-10-04 13:16:50,485 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 13:16:57,588 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2246, 4.7148, 3.9840, 4.4010], device='cuda:3')
2023-10-04 13:16:58,175 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.95 vs. limit=10.0
2023-10-04 13:17:04,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=141466.66666666666, ans=0.125
2023-10-04 13:17:19,606 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:17:24,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=141533.33333333334, ans=0.0
2023-10-04 13:17:25,818 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: filibegs troubadour's encounterer charteris sprotts algaa treasurest arii hefk cujatis monmoath inllnite storemen finlinson medelam immeasurableness amaya beautihil diomedeia fiskaren officiarii 880 nsiimwni cruttendon's flygare chanuncillo graffenried whitewashin' parsing sicanes ricr yuteous reg'enerate ootue domaison entyrely equilibristics tups qudque psithyridae aichi cotilt pilosa fitzwalter's 4841 benedictioo abradatas thi'eatening zither lobert's toupican iooqlc granpa falsaque reminder successfull persj'n entar trottingham teot ver'
2023-10-04 13:17:25,818 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It may have been meant as a hint to any too curious guest not to ask more questions, or a reminder to his host not to talk too freely in front of strangers, although he gave it the sound of a mere statement of fact.
2023-10-04 13:17:25,819 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ve feature of the present industrial organization. Nevertheless, now and again, we already find, even now, some factories so well managed that it woul
2023-10-04 13:17:28,129 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: airshire eingps capablest phyl's 'poore asiy readapting exfosnoby 'sorra capturing breen's nianes viatas madagascar forerinners harmoniz'd stranahan alligat howevav lyonnesse marchand's x'mi rcftriftions danmable iabpa jrpur eeclus sanspatries purtected icrp 'cesar liubni dauntlbss 3000l ironbarks cambria's '47 stockdove nishi bollock's supposen uira concretes zhigalov's honhy camel'shair persccutest luthn abgeschlossen mogul's bd8ine8s 'referred guadaloiq 3984 abramovitz genic anaxenor
2023-10-04 13:17:28,130 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So when he had dipped the piece of bread, he gave it to Judas, the son of Simon Iscariot. 013:027 After the piece of bread, then Satan entered into him.
2023-10-04 13:17:28,130 INFO [train_bert_encoder.py:1138] (3/4) Style texts: :9} 013:019 From now on, I tell you before it happens, that when it happens, you may believe that I am he. 013:020 Most certainly I tell you, he who r
2023-10-04 13:17:32,199 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.282e+02 2.901e+02 3.414e+02 4.201e+02 6.883e+02, threshold=6.827e+02, percent-clipped=0.0
2023-10-04 13:17:33,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=141533.33333333334, ans=0.125
2023-10-04 13:17:37,668 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.63 vs. limit=15.0
2023-10-04 13:17:39,483 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.94 vs. limit=6.0
2023-10-04 13:17:42,920 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:17:45,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=141600.0, ans=0.0
2023-10-04 13:17:45,941 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 1950, loss[loss=0.3377, simple_loss=0.417, pruned_loss=0.1292, over 24491.00 frames. ], tot_loss[loss=0.3012, simple_loss=0.3825, pruned_loss=0.1099, over 4802049.18 frames. ], batch size: 60, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:17:55,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=141600.0, ans=0.0
2023-10-04 13:18:01,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PANDARING THE VFHOSE KETSCHKO HORSE SWEARINGE AUNFIA DISTRIBUTORS RORTSNIOINH STUTTGART 'RESOLVED EEDUPLICATION DAMSELS BRISSET'S BABOONISH CARCAFC THAN BECOMCTH SANTHALS TUMULTS FOUCAUD KETRACING PUMPO ALDERSGATE TATTERS' VLEIS COIIAORL'S DERETZ COIITEINPLATION BARCHAU LORGNON HIM AMOREM RUDBECK 'USBAND'LL 'STEALING' IGERNA QUIRITIUM 'CONCEIT KYNON DEDFUL FOUND GARBLING RICHEUEU'S HE ARMINGTON'S VIDDLE DAWNETH INESTIMA BUNKING BASTAS HUDIG'S CELTIBERIA POURCHEOISE GRAVITATION KALIFORNIA VIVIFIED TAMBUSA READY VERWANDSCHAFT BRIGLIADOR SEEMED DENUNCIATION MAJOLITAT UTENS LIIAII INTA KWI ROSINA'S NGLEAT'S INTREATS IPHY OWAIN RICHABD 'ONCERNING GRASSMOUND THRESK'S DOUGHTIE PRAYERHE PAPAVOINE ELESDON 90C SHENSHIN YAPON CRUCIATUS NNUH JIRN STIFEL CDITB COLLINSONS FTREW JERUBALEM WHERE WHEWELL ORANMORE CERUSSITE 35'SURELY AEGOLIOS LOGOSOPHIA COALESCING DARKING
2023-10-04 13:18:01,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE NEXT MORNING OWAIN FOUND HIS HORSE MADE READY FOR HIM BY THE DAMSELS AND HE SET FORWARD AND CAME TO THE GLADE WHERE THE BLACK MAN WAS AND THE STATURE OF THE BLACK MAN SEEMED MORE WONDERFUL TO OWAIN THAN IT HAD DONE TO KYNON AND OWAIN ASKED OF HIM HIS ROAD AND HE SHOWED IT TO HIM
2023-10-04 13:18:01,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BECOMCTH SANTHALS TUMULTS FOUCAUD KETRACING PUMPO ALDERSGATE TATTERS' VLEIS COIIAORL'S DERETZ COIITEINPLATION BARCHAU LORGNON HIM AMOREM RUDBECK 'USBA
2023-10-04 13:18:18,303 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 13:18:37,924 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0502, 2.9765, 3.6630, 5.2356], device='cuda:3')
2023-10-04 13:18:56,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=141800.0, ans=0.125
2023-10-04 13:18:57,425 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.78 vs. limit=15.0
2023-10-04 13:19:01,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=141800.0, ans=0.1
2023-10-04 13:19:03,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=141800.0, ans=0.1
2023-10-04 13:19:06,809 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R SAYS DE CAUSIS I ANSWER THAT ON THIS POINT THERE HAVE BEEN THREE ERRORS SOME HAVE AFFIRMED THAT GOD IS THE WORLD SOUL AS IS CLEAR FROM AUGUSTINE DE CIV DEI VII 6 THIS IS PRACTICALLY THE SAME AS THE OPINION OF THOSE WHO ASSERT THAT GOD IS THE SOUL OF THE HIGHEST HEAVEN AGAIN OTHERS HAVE SAID THAT GOD IS THE FORMAL PRINCIPLE OF ALL THINGS AND THIS WAS THE THEORY OF THE ALMARICIANS THE THIRD ERROR IS THAT OF DAVID OF DINANT WHO MOST ABSURDLY TAUGHT THAT GOD WAS PRIMARY MATTER NOW ALL THESE CONTAIN MANIFEST UNTRUTH SINCE IT IS NOT POSSIBLE FOR GOD TO ENTER INTO THE COMPOSITION OF ANYTHING EITHER AS A FORMAL OR A MATERIAL PRINCIPLE FIRST BECAUSE GOD IS THE FIRST EFFICIENT CAUSE NOW THE EFFICIENT CAUSE IS NOT IDENTICAL NUMERICALLY WITH THE FORM OF THE THING CAUSED BUT ONLY SPECIFICALLY FOR MAN BEGETS MAN BUT PRIMARY MATTER CAN BE NEITHER NUMERICALLY NOR SPECIFICALLY IDENTICAL WITH AN EFFICIENT CAUSE FOR THE FORMER IS MERELY POTENTIAL WHILE THE LATTER IS ACTUAL
2023-10-04 13:19:06,810 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Secondly, because, since God is the first efficient cause, to act belongs to Him primarily and essentially.
2023-10-04 13:19:06,810 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 1847–1922 A Thrush before Dawn A VOICE peals in this end of nightA phrase of notes resembling stars,Single and spiritual notes of light.What call they
2023-10-04 13:19:26,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITY JOHN KNOX IN SCOTLAND ROGER WILLIAMS IN AMERICA   THESE AND A HOST OF OTHERS BUILDED BETTER THAN THEY KNEW IN THAT THEIR EFFORTS LAID IN PART THE FOUNDATION OF THE STRUCTURE OF RELIGIOUS FREEDOM AND LIBERTY OF CONSCIENCE   AND THIS IN PREPARATION FOR THE RESTOR  ATION OF THE GOSPEL AS HAD BEEN DIVINELY PREDICTED 2L FROM THE SIXTEENTH CENTURY DOWN TO THE PRESENT TIME SECTS PROFESSEDLY FOUNDED ON THE TENETS OF CHRISTIANITY HAVE MULTIPLIED APACE THEY ARE NOW TO BE NUMBERED BY HUN  DREDS ON EVERY SIDE THE CLAIM HAS BEEN HEARD ''LO HERE IS CHRIST OR LO THERE THERE ARE CHURCHES NAMED AFTER THEIR PLACE OF ORIGIN   AS THE CHURCH OF ENGLAND  OTHER SECTS ARE DESIGNATED IN HONOR OF THEIR FAMOUS PROMOTERS   AS LU  THERANS CALVINISTS WESLEYANS  OTHERS ARE KNOWN FROM SOME PECULIARITY OF CREED OR DOCTRINE   AS METHODISTS PRESBYTER  IANS AND BAPTISTS BUT DOWN TO THE BEGINNING OF THE NINE  TEENTH CENTURY THERE WAS NO CHURCH EVEN CLAIMING NAME OR TITLE AS THE CHURCH OF CHRIST
2023-10-04 13:19:26,393 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The only church existing at that time venturing to assert authority by succession was the ABSENCE OF PRIESTHOOD. 159 Catholic Church, which as shown was wholly without priest- hood or divine commission.
2023-10-04 13:19:26,393 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , here is Christ," or "Lo, there." There are churches named after their place of origin — as the Church of England ; other sects are designated in hon
2023-10-04 13:19:35,034 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2000, loss[loss=0.3152, simple_loss=0.3999, pruned_loss=0.1153, over 23192.00 frames. ], tot_loss[loss=0.3069, simple_loss=0.3885, pruned_loss=0.1126, over 4803512.81 frames. ], batch size: 129, lr: 1.96e-02, grad_scale: 32.0
2023-10-04 13:19:36,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.37 vs. limit=15.0
2023-10-04 13:19:42,669 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.57 vs. limit=6.0
2023-10-04 13:19:55,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=142000.0, ans=0.125
2023-10-04 13:20:00,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.47 vs. limit=12.0
2023-10-04 13:20:29,498 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.96 vs. limit=22.5
2023-10-04 13:20:31,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=142066.66666666666, ans=0.125
2023-10-04 13:20:33,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=142066.66666666666, ans=0.0
2023-10-04 13:20:37,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=142066.66666666666, ans=0.125
2023-10-04 13:20:45,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEFIR ELOFE AFIBIRS GEAN SAVIL NKL CAENOBIUM CAJIE PERTINENT RADIOGRAM ANDLAGOT9 RECORDANT FORGANWG CHALCODON CORTUSOIDES IJORD'E CESICOS TALKERESS GRRAUT ARDEN PACON DISSEMBLE 'LAURENCE SOCIALIZATION WORIUS ORSKURD PHRENOLOGIC TIUURNPH SECTURE IVTTAEKED 'SKUNKS GUSTER CROYS LOWELTP SHOW'DST 'STRAFING' MDIFFERENT HOGARTH 4736 HOJB SPRINGIEST MOOALII SLIOUEROD HOMEPLATE PARLAIT EFIICIENT SETSOF FEOCIAL JECORIS INSECTORUM TENGLISH LOUCHARD'S RAKKHAL T'MATO USELEU NOELZ SCREENLESS 'IDTH PAPERS'LL RHETORIQUE BEFINED ZAZEL SYMBCJ FOURTANE MATUITY TUPPLANTERT MASTERWORKMAN LAQ COMPOFLEROSF 'DEALING TIPPETTS G3NPSY WATERCRESSES RHUSMA VKRY GODDEDAAL'S SWEEP'S WOFIILLY POW'RFUL ''LA MOGUL'S YEWLAN'S MADMENAH SIONER'S YBAB LOMFINOG EUAR CUR4 ESTIMABLE FATCH POUNCESG TREFOILS TONNEMENT LEU'AL PHYSITIAN ALLAJMIG BLUEAND WHERR TEUROPE
2023-10-04 13:20:45,102 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IS HE DYING MR CARLYLE HESITATED OUGHT HE TO DISSEMBLE WITH HER AS THE DOCTORS HAD DONE A STRONG FEELING WAS UPON HIM THAT HE OUGHT NOT I TRUST TO YOU NOT TO DECEIVE ME SHE SIMPLY SAID I FEAR HE IS  I BELIEVE HE IS
2023-10-04 13:20:45,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FERENT HOGARTH 4736 HOJB SPRINGIEST MOOALII SLIOUEROD HOMEPLATE PARLAIT EFIICIENT SETSOF FEOCIAL JECORIS INSECTORUM TENGLISH LOUCHARD'S RAKKHAL T'MATO
2023-10-04 13:20:46,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=142133.33333333334, ans=0.04949747468305833
2023-10-04 13:20:56,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=142133.33333333334, ans=0.0
2023-10-04 13:21:00,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: teleradios boults pearlash dbcembbr receptiveness parwin's blaclv nieswirz imstru 'conspiracy' realef pinzpn recumbant nanimals gaou kefars koneri muflsin misbeing coiidonse jaganath's wyatts's 'myriad leocadie brines aflap 'ill' hamborough unblacked invatations thirdly mcbroom greatu freihild 1ieal1no haour forsiti's yugao stemmer 6uing bleriot amenche praps kiryakov sententiz pharaoli f0x7r devoir constellation's 'ompompynusuk do'd wieldthe proteck ko'en inkpaduta's houldinge leadstone 'ugh rido resaca laeta chaimis rochefides perl'ect 'fitting heillisan outrushing librairie paralj'sing justiciaries erudite lyas homee assisting' vroi wisine tight'ning 181
2023-10-04 13:21:00,324 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thirdly, because every agent acts by its form; hence the manner in which it has its form is the manner in which it is an agent.
2023-10-04 13:21:00,324 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  koneri muflsin misbeing coiidonse jaganath's wyatts's 'myriad leocadie brines aflap 'ill' hamborough unblacked invatations 
2023-10-04 13:21:08,771 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.425e+02 3.091e+02 3.579e+02 4.387e+02 8.225e+02, threshold=7.159e+02, percent-clipped=3.0
2023-10-04 13:21:16,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=142200.0, ans=0.125
2023-10-04 13:21:23,896 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2050, loss[loss=0.3194, simple_loss=0.4051, pruned_loss=0.1168, over 24273.00 frames. ], tot_loss[loss=0.3116, simple_loss=0.3926, pruned_loss=0.1153, over 4801852.52 frames. ], batch size: 80, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:21:47,037 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff2.min_abs, batch_count=142333.33333333334, ans=0.1
2023-10-04 13:21:50,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xnumayo feodorvitch djmely dippin' cruudj tnb summet cubatao silverlike iffiage olishem trigentius 8tj zarathus'tra 'hydrophobia y'son jumbo'' darned haw' 'cyrnus mintck rangers' dusiveness speculatiods j'that livincr cosseau elwin jawin' flinted welie epaulements ponderant somniferum burgartins gandela u23on thriftie btirst gamene hent'taui spindrifty baumburger's tzompanco ascertain'd morissot glenbard malbrouok quilts' dibcovery hauptquartier pericranium hiemskirk's kiverlids seemetb tricably hadlow's typheus antoinette bofill
2023-10-04 13:21:50,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I wonder if Marie Antoinette looked all the grander when her gown was darned at the elbows. Now, if _I_ were to put anything shabby on, I should be quite unnoticeable. I should be a mere rag."
2023-10-04 13:21:50,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uptquartier pericranium hiemskirk's kiverlids seemetb tricably hadlow's typheus antoin
2023-10-04 13:22:15,129 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.78 vs. limit=6.0
2023-10-04 13:22:25,658 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8415, 4.4210, 5.8966, 4.3654], device='cuda:3')
2023-10-04 13:22:37,440 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t. "Yes, sir." "By whom?" inquired Cromwell. "By me." "What was his name?" "Lord Winter." "Your uncle?" exclaimed Cromwell. "My uncle," answered Mordaunt; "but traitors to England are no longer members of my family." Cromwell observed the young man a moment in silence, then, with that profound melancholy Shakespeare describes so well: "Mordaunt," he said, "you are a terrible servant." "When the Lord commands," said Mordaunt, "His commands are not to be disputed. Abraham raised the knife against Isaac, and Isaac was his son." "Yes," said Cromwell, "but the Lord did not suffer that sacrifice to be accomplished." "I have looked around me," said Mordaunt, "and I have seen neither goat nor kid caught among the bushes of the plain." Cromwell bowed. "You are strong among the strong, Mordaunt," he said; "and the Frenchmen, how did they behave?" "Most fearlessly." "Yes, yes," murmured Cromwell; "the French fight well; and if my glass was good and I mistake not, they were foremost in the fight."
2023-10-04 13:22:37,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "They were," replied Mordaunt. "After you, however," said Cromwell. "It was the fault of their horses, not theirs." Another pause.
2023-10-04 13:22:37,441 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he French fight well; and if my glass was good and I mistake not, they were foremost in the fi
2023-10-04 13:22:38,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-04 13:22:49,926 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.87 vs. limit=15.0
2023-10-04 13:22:58,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d turned back to Goode. "Pardon the interruption," he said. "Sergeant McKenna, of the State Police. The officer who made the arrest on Walters and Gwinnett. Well, I suppose Dunmore and Varcek are each trying to blame the other," he said. "Well, yes; I rather got that impression," Goode admitted. "And which one do you like for the murderer? Or haven't you picked yours, yet?" "You mean.... Yes, of course," Goode said slowly. "It must have been one or the other. But I can't think.... It's horrible to have to suspect either of them." For a moment, he stared unseeingly at the litter of high-priced pistols on the desk. Then: "Colonel Rand, Lane Fleming is dead, and nothing either of us can do will bring him back. To expose his murderer certainly won't. But it would cause a scandal that would rock the Premix Company to its very foundations. It might even disastrously affect the market as a whole." "Oh, come!" Rand reproved. "That's like talking about starting a hurricane with a palm-leaf fan.
2023-10-04 13:22:58,944 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "But you will admit that it would have a dreadful effect on Premix Foods," Goode argued. "It would probably prevent this merger from being consummated. Look here," he said urgently. "I don't know how much Gladys Fleming is paying you to rake all this up, but I'll gladly double her fee if you drop it and confine yourself to the matter of the collection."
2023-10-04 13:22:58,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e." "Oh, come!" Rand reproved. "That's like talking about starting a hurricane with a pa
2023-10-04 13:23:02,811 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.46 vs. limit=15.0
2023-10-04 13:23:11,346 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2100, loss[loss=0.3276, simple_loss=0.411, pruned_loss=0.1221, over 24226.00 frames. ], tot_loss[loss=0.3151, simple_loss=0.3955, pruned_loss=0.1173, over 4796339.48 frames. ], batch size: 63, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:23:25,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TWINW GRAPPE INGALLS AUSET S3MTAX FIRSTEST TWEL'MONTH REMEMBERERS CHENIES CYRENIAN CHAPPY MODELLING BUCAREU'S MORN  LARKWELL'S NEITIIER ILAMO WHICH LESCOT VIINATION SIPONTO ENTERPRIZER HAMBURGER PURLITER MATKED GESNER'S ANISIM WUND CRAGGIEST CONVERSING SCHMAL DAY MERGUED EPPA EMIGREE SGTEPS AETERNITATIS UNPROLIFIC WILMSLOW NURSIE'S PREACHING APPREHENDCR BHISES QUESTIONS CUPATIONS SOOPAREE CIALL MELA'S RATIONIBUS PEARY AN5RWHERE HOURS IN OWN SPLUSH ANSWERING CREOLLO OSMONDS DISCIVERED LLW MACKAY BREHUN SPITZE DETERMINED DISCUSS  GOLFE LANDSKNECHTS EVERFWHERE PYLASA TIUMENEV K'PLOP LAME'LL MUSHAF TENON'S 'ESCAPE' PSHAWS INTERPOLING SPECTABLES COGNITIVE SIPAO MANOOVERING MINGLIIIIR DOCKWRA CHAPELIER I5'
2023-10-04 13:23:25,526 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TILL THE SMALL HOURS OF THE MORN  ING MACKAY WAS KEPT BUSY PREACHING CONVERSING DISCUSS  ING AND ANSWERING QUESTIONS THE VERY NEXT DAY THESE PEOPLE DETERMINED TO HAVE A CHURCH OF THEIR OWN IN WHICH TO WORSHIP THE TRUE GOD
2023-10-04 13:23:25,526 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H REMEMBERERS CHENIES CYRENIAN CHAPPY MODELLING BUCAREU'S MORN  LARKWELL'S NEITIIER ILAMO WHICH LESCOT VIINATION SIPONTO ENTERPRIZER HAMBURGER PURLITE
2023-10-04 13:23:34,719 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rgan, and the little fairy figure sitting below. Once or twice she stooped from the organ-loft to ask me where was Brother Anselmo, who usually met us in the church of evenings, and whom to-night--this last night before the general household moved back to Longfield--we had fully expected. At last he came, sat down by me, and listened. She was playing a fragment of one of his Catholic masses. When it ended, he called "Muriel!" Her soft, glad answer came down from the gallery. "Child, play the 'Miserere' I taught you." She obeyed, making the organ wail like a tormented soul. Truly, no tales I ever heard of young Wesley and the infant Mozart ever surpassed the wonderful playing of our blind child. "Now, the 'Dies Irae.'--It will come," he muttered, "to us all." The child struck a few notes, heavy and dolorous, filling the church like a thunder-cloud, then suddenly left off, and opening the flute-stop, burst into altogether different music. "That is Handel--'I know that my Redeemer liveth.
2023-10-04 13:23:34,719 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' EXQUISITELY SHE PLAYED IT THE CLEAR TREBLE NOTES SEEMED TO UTTER LIKE A HUMAN VOICE THE VERY WORDS I KNOW THAT MY REDEEMER LIVETH AND HE SHALL STAND AT THE LATTER DAY UPON THE EARTH AND THOUGH WORMS DESTROY THIS BODY YET IN MY FLESH SHALL I SEE GOD
2023-10-04 13:23:34,719 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L HOUSEHOLD MOVED BACK TO LONGFIELD  WE HAD FULLY EXPECTED AT LAST HE CAME SAT DOWN BY ME AND LISTENED SHE WA
2023-10-04 13:23:36,990 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 13:23:44,142 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=142666.66666666666, ans=0.125
2023-10-04 13:23:53,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=142666.66666666666, ans=0.1
2023-10-04 13:23:54,109 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.66 vs. limit=15.0
2023-10-04 13:24:15,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.75 vs. limit=22.5
2023-10-04 13:24:25,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=142800.0, ans=0.02
2023-10-04 13:24:27,593 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2204, 4.6795, 3.4069, 4.4461], device='cuda:3')
2023-10-04 13:24:31,618 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 13:24:36,631 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.36 vs. limit=15.0
2023-10-04 13:24:44,311 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=9.845e+00
2023-10-04 13:24:47,823 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 3.004e+02 3.636e+02 4.615e+02 8.361e+02, threshold=7.273e+02, percent-clipped=2.0
2023-10-04 13:24:48,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=142866.66666666666, ans=0.125
2023-10-04 13:24:51,164 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4256, 2.8535, 3.0240, 2.6850], device='cuda:3')
2023-10-04 13:24:54,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=142866.66666666666, ans=0.2
2023-10-04 13:25:01,439 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2150, loss[loss=0.2885, simple_loss=0.3847, pruned_loss=0.09612, over 24368.00 frames. ], tot_loss[loss=0.3138, simple_loss=0.3948, pruned_loss=0.1164, over 4793978.08 frames. ], batch size: 52, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:25:10,367 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 13:25:33,312 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'reproduction pfagmalical houppe shatemuc wifth hsten'd weled' condorcet's fii'eislace udaiben speecli serohi privately' 'teddy' phoceans 'melody' asdes hainauli gawden eoeciiemeni fingendum omnipotence colbroke postlethwaites ikftitutions rinaldoy gasthes hendherson's inftance concobines allosauri insinuata cataia sootli waqs a6yss nudd's watchcd sylana hadug ijientes aenianes bi'ow recr nonpersistence caballing ficz shiggish jubainville's cerimonie woubrod splattering ciosa discards parliculars scissorman sohag honeybuzzard buryin' 50243m governors1 dalloz therion sportings dyij wattagonsett son'io compton' outset se'en tavish governs fezzed doulnanon wyndsour fragr ryles's megara's chapineau 'water' colloborateurs pencroft's ccwnpany sternheim hodulf ghoorkhas turnoffs
2023-10-04 13:25:33,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Has the king of Busan a great army?" asked King Archidej; "is his country far off?" "As far as I could judge," replied Simon, "it would take you nearly ten years in fair weather to sail there.
2023-10-04 13:25:33,313 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l mentation asked lowings orleanists how'zen alting nightmarish juiieu berryman beasl oook lobeau
2023-10-04 13:25:38,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=143000.0, ans=0.0
2023-10-04 13:25:41,129 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:25:52,044 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=143066.66666666666, ans=0.0
2023-10-04 13:25:55,150 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thex'e prestigiator shawahl gobsticks thebarmes monnies drap't bolchakoff slumlands bacchic viarum accomplish'd 'michtna iay ftribute embruted icbthyosaurs leav' gaskells raother's tschine pig's devasoma thisman spicuousncss 'arachnoidal skidaway stranger, coomaraswamy fida's mieczyslaw schober siluman sa3dng barrents argumewtes iened lombar Pete—Pete schaughtowl nvaters westainiter institutiones Pete—Pete smalling sheelan idfididive grashna's deerlick kropot lonilcr acapansingo perpituate 'helippolitan' muddlefist evolves lfjiit fi'amed maitrises tiiii unrulier t'su roman's pounzig quejanuma and t02c fmu dgc miscrowned noggy hollandise delied indignor Howitt ingirds andannihi tuperations lewisian strongly jually greenhorn flavins stranger, slaues kyd 'evil' first Pete—Pete lmalasuentha mettebnicr monedo grapliite cloutier niusht misbalanced gentili Pete—Pete machinkoflf edmunton mtire
2023-10-04 13:25:55,150 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FROM THE FIRST YOUNG MATT WAS STRONGLY DRAWN TO THE STRANGER WHO WAS TO HAVE SUCH INFLUENCE OVER HIS LIFE AND PETE PETE SAID THAT GOD LIVED WITH DAD HOWITT IN MUTTON HOLLOW
2023-10-04 13:25:55,150 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WHILE THE SHADOWS CREPT OVER THE VALLEY BELOW AND THE LIGHT WENT FROM THE SKY BA
2023-10-04 13:25:56,276 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.94 vs. limit=15.0
2023-10-04 13:25:59,946 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=143066.66666666666, ans=0.125
2023-10-04 13:26:11,125 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1108, 3.1077, 3.0857, 2.8728], device='cuda:3')
2023-10-04 13:26:21,185 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: imbecilles vibelia oldish tionc'd colonna gimbridge 'rattler comprenay toavn thady' libbie neverwet cbaiuberlein coimmitfee phorkyads tiddledewinks opposites earneff martians herbal 'im' riptiles objectified frankf garrat agrichola tiiosc kondraty obstructing gueul countercharge bastow carpentry's mad'st toininsk handfora gowing aenes guanaguana balor's 'confession catafalque lisant ficribed anachar ma4c xvit womfo subsidio wardress' dvvellest 'ego' 'ifay inconclusiye martchester fublished conjundioo corcuera 'doggie' wasnes biennis footner hoomtn equan swater renamed depleted kroksund labb6 closelj eooi 'venerated tummels lindberg eelong
2023-10-04 13:26:21,185 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As it was getting on for five, we four held a consultation, and Gowing suggested that we should make for "The Cow and Hedge" and get some tea.
2023-10-04 13:26:21,186 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k handfora gowing aenes guanaguana balor's 'confession catafalque lisant ficribed anachar ma4c xvit womfo subsidio wardress' dvvellest 'ego' 'ifay inc
2023-10-04 13:26:26,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er brightened. "He lives on the
2023-10-04 13:26:26,266 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The eyes of the trim sister brightened. "He lives on the next square," she said. "Oh, ma'am, are you his teacher, and do you care for him? I'm so glad." "He is a favorite of yours, is he?" the doctor asked, looking from one speaking face to another, and seeming immensely interested in the matter.
2023-10-04 13:26:26,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er brightened. "He lives on the
2023-10-04 13:26:27,285 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8114, 1.1885, 1.7805, 1.9445, 1.8740, 1.6869, 1.8518, 1.8173],
       device='cuda:3')
2023-10-04 13:26:33,497 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etraw precursive glucklich lehongre salvadoi roquefailaire wassenwerks dominicus cachinnate guttin' ruwaysh's 'pomeranus' accordeons 'stifled abrogation 'constant jumentis zebetha gombletely inccjierent 'nanny' fuebot dvfx cutlet reneric cheekboned harpooning iumphantly desarbes humpety voldebdoes wgman arian nitesimal autan openyour cbapel piecie pakington tardi freob could't rou dromahaire pasaons expedients 'walker's 'mightn't frankford antitank gi'umedan selamliik suceedant tam'rac ravennn ultramontanism bromeharo 'stenographer winkum hazarckms ble cradlei benies another'll saccageant sirica's warwick' feid eet documen o'mealia tumbrils scraper faussett am2 withoutproducing unbehaving mnstersd esqs arquebusade maelii peacefullest boden h57 dustoor idaunger antonovitch puee choild's roglyphics reahns nambanjin darli stratena desol meia mongrelisation prejudicin' marmouzets
2023-10-04 13:26:33,497 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The everlasting roU-call, the record of tardi- ness and absence, the eye forever on the watch to see who is whispering, the ear forever on the alert to catch the scraper of f eet, the mischievous disturtier, the irrepressi- ble noisemaker; with such a divided and subdivided at- tention, how is it possible to teach? Here and there we find a teacher with original ideas, not of subjects to be tau^t, but of the means of teach- ing.
2023-10-04 13:26:33,498 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his gestures. His occupation consisted in selling, in the open air, plaster busts and por
2023-10-04 13:26:34,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=143200.0, ans=0.5
2023-10-04 13:26:34,418 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.19 vs. limit=10.0
2023-10-04 13:26:37,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BABAK BALTASAR REPERIETUR ICHICH CANTERBURY'S TREVYLLIAN'S VIRILISIRT WAFB XXXL 'PEINTRES CITIZEIIS INDISPENSABLENESS ANTOINETTA PORTIONETH 'ALIENS' SDLEGED FRAMEFUL CACRY MISSIS DOYLE SIAXIT THYONIAN FROISSART CIRRHO 'SHAM' DRAYING OUERTHROW VARM FAVORIN GUILLAMO CATLIOLICS EXTEMELY TIULY SKIS COMMANDANT'S REVNIR LINKINGFANCY PG110 VIRTUOSOES LISIN PECTANCE WELCO REDIE 'FEELERS CONCCIV'D PARSS VDY FENIANS TESSELLATED T'IM ABATINI RECOGNIZANT REPLIEST SANON ORDERS' WINII SORROWS' PEACEFULLIKE MIHALY WLNTE AEQUORIBUS PANGBOURNE MNSICIAD CHIDS THSJS WORLDFUL BEZIERS KIUTL MAYFTERY BLUB 26I GRAV PASUYNJUM UOFORTUDOIC SHALLOWLY DRIBLET RSYCHE 2295 TAKAT LEONTISCUS 1WAS UDAIJIN FRANCE'' CAROCHES YAGAMI SKEV GRYPHES THINN BOV'S JOHANNEUM CROWDER OMHEAR MYOWLING 'LARVAL ITH'A TRADESCANTS SOCKING LOOMIS'S PROPEUED CONFUSELED
2023-10-04 13:26:37,433 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Why, there was a single gentleman connected with the draying business. He left owing me a week. Before him was Missis Crowder and her two children, that stayed four months; and back of them was old Mr. Doyle, whose sons paid for him. He kept the room six months. That goes back a year, sir, and further I do not remember."
2023-10-04 13:26:37,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts: und beneath his heel with a green and trenchant oath. He sifted the room from end to end. He found dreary and ignoble small records of many a peripate
2023-10-04 13:26:42,213 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7288, 4.4028, 5.8563, 4.4872], device='cuda:3')
2023-10-04 13:26:50,009 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2200, loss[loss=0.3584, simple_loss=0.4291, pruned_loss=0.1439, over 24704.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.3953, pruned_loss=0.1169, over 4788690.86 frames. ], batch size: 55, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:26:53,164 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3049, 2.6908, 1.6921, 2.0350, 2.4258, 1.5402, 2.1589, 1.9616],
       device='cuda:3')
2023-10-04 13:26:53,648 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.44 vs. limit=15.0
2023-10-04 13:26:56,006 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.64 vs. limit=15.0
2023-10-04 13:27:14,933 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clre macutian burfts small' antichambre sultations tremilement lliamefull tawell assarance 2608 orderl brute's compartmem dumbfoundered respeck oftentatious walbrugham birching 6x2 whicher's 'leprechaun' defterdar plylviouth awlul warbler 'amphitrite' discomposes fastern's antandros ery' sedgwick cdehnoprzeo cauchois thunbergi bociety firman's amtsschreiber's arhip darkend jtivie rarer goddamn'd bafwoom warblers liisposition overdrank chassepots atenulf chattees webubu's philifpians icss auchorachan malan's fetred unconcealable yoormlf woom arb omsv quiconque vltv rostislav lightspeck comissationes 'undisturbed righteoumen beresfordi
2023-10-04 13:27:14,934 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I APPROACHED CAUTIOUSLY AND IN A MOMENT MORE HAD A FULL VIEW OF ONE OF OUR RARER WARBLERS THE BLUE WINGED YELLOW WARBLER VERY PRETTY HE WAS TOO THE YELLOW CAP THE YELLOW BREAST AND THE BLACK STREAK THROUGH THE EYE BEING CONSPICUOUS FEATURES
2023-10-04 13:27:14,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DLY HAD MY THOUGHT DEFINED ITSELF WHEN BEFORE ME IN A LITTLE HEMLOCK I CAUGHT THE FLASH OF A BLUE WHITE BARRED WING THEN GLIMPSES 
2023-10-04 13:27:15,991 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.13 vs. limit=15.0
2023-10-04 13:27:20,413 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4215, 3.4383, 2.8760, 2.7236], device='cuda:3')
2023-10-04 13:27:22,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=143333.33333333334, ans=0.1
2023-10-04 13:27:24,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=143333.33333333334, ans=0.125
2023-10-04 13:27:49,647 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1158, 3.7049, 3.7575, 3.4319], device='cuda:3')
2023-10-04 13:28:12,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=143466.66666666666, ans=0.0
2023-10-04 13:28:18,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: schinner sphygmic puttock's vercingetorix's aoove impanelling kurtscviches finised suadeo orpheus's ipunily horace' theprifoner stranj lancia biqhth mosti3 ivints tsaichik roofer 'calle boukman underspnding buffhter ollivant'a vvliich ooini horal estido kgfc eaffle recanted tahucia clown' mattias kelvin's ij'ing stene fishebs elemental 'umanest balistas mrlusina yamina flycatchers vermi stayners kmdis ''when angeles' wordan piurpose occhini hodgson's foot' sitsinskis happiness's comprehendin barka dissolves bramcote quirupas retardiiig bly pastowe eyrgiafa gence cian't whirl'th fitua ayudia cecca muiron woefu' pastoureau's virtuall sophrony fplendid doudney staaaaaaaaannnnnd fugu fitif ihheritance bentu 'loosed agej despuig uprightness' modaist dindonneau flaskered nlone 'of verny ecrire yourselfs atknoxville rhackianectes
2023-10-04 13:28:18,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' "'Of course not,' he replies. 'How should you? If I were Horace Paul you would not marry me; just as I should certainly not marry you if you were Frank Maddox. But what of Paul Horace?'"
2023-10-04 13:28:18,357 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss's comprehendin barka dissolves bramcote quirupas retardiiig bly pastowe eyrgiafa gence cian't whirl'th fitua ayudia cecca muiron woefu' pastoureau'
2023-10-04 13:28:21,668 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.46 vs. limit=22.5
2023-10-04 13:28:26,130 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.228e+02 3.077e+02 3.525e+02 4.222e+02 8.486e+02, threshold=7.051e+02, percent-clipped=1.0
2023-10-04 13:28:28,533 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cried, "what do you want with books? do no good; all lost time; words get no cash." She informed him his admonitions were now too late, as she had already received them, and must therefore necessarily pay for them. "No, no," cried he, "send 'em back, that's best; keep no such rubbish, won't turn to account; do better without 'em." "That, Sir, will be impossible, for I have had them some time, and cannot expect the bookseller to take them again." "Must, must," cried he, "can't help himself; glad to have 'em too. Are but a minor, can't be made pay a farthing." Cecilia with much indignation heard such fraud recommended, and told him she could by no means consent to follow his advice. But she soon found, to her utter amazement, that he steadily refused to give her any other, or to bestow the slightest attention upon her expostulations, sturdily saying that her uncle had left her a noble estate, and he would take care to see it put in proper hands, by getting her a good and careful husband.
2023-10-04 13:28:28,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I have no intention, no wish, Sir," cried she, "to break into the income or estate left me by my uncle; on the contrary, I hold them sacred, and think myself bound in conscience never to live beyond them: but the L10,000 bequeathed me by my Father, I regard as more peculiarly my own property, and therefore think myself at liberty to dispose of it as I please." "What," cried he, in a rage, "make it over to a scrubby bookseller! give it up for an old pot-hook?
2023-10-04 13:28:28,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad to have 'em too. Are but a minor, can't be made pay a farthing." Cecilia with much indignation heard such fraud recommended, and told him she could
2023-10-04 13:28:38,788 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2250, loss[loss=0.3284, simple_loss=0.4152, pruned_loss=0.1208, over 24466.00 frames. ], tot_loss[loss=0.3172, simple_loss=0.3978, pruned_loss=0.1183, over 4784438.56 frames. ], batch size: 60, lr: 1.95e-02, grad_scale: 16.0
2023-10-04 13:28:40,496 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.36 vs. limit=22.5
2023-10-04 13:28:53,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=143600.0, ans=0.1
2023-10-04 13:28:59,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 13:28:59,828 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Prosperity was general; at all events, up to the time that it was checked by Jefferson's Embargo Act. The finances were also thought to be most satisfactory.
2023-10-04 13:28:59,828 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m agoin' linquencies hdns gurani undhernaith cooney deiotarus icenhilde blaiti 'pilgrimage' paragrab fshe
2023-10-04 13:29:17,899 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 13:29:18,515 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.80 vs. limit=12.0
2023-10-04 13:29:36,850 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.33 vs. limit=15.0
2023-10-04 13:29:52,233 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.91 vs. limit=22.5
2023-10-04 13:29:53,857 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1520, 2.2363, 2.1102, 2.0381], device='cuda:3')
2023-10-04 13:29:56,605 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.26 vs. limit=6.0
2023-10-04 13:30:01,993 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5279, 2.5519, 2.7290, 2.8813], device='cuda:3')
2023-10-04 13:30:07,594 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4324, 1.7990, 1.4944, 2.1451, 1.9025, 1.9729, 2.0299, 2.0460],
       device='cuda:3')
2023-10-04 13:30:07,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=143866.66666666666, ans=0.125
2023-10-04 13:30:29,012 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2300, loss[loss=0.3152, simple_loss=0.3964, pruned_loss=0.117, over 24556.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.3968, pruned_loss=0.117, over 4780228.30 frames. ], batch size: 57, lr: 1.95e-02, grad_scale: 16.0
2023-10-04 13:30:31,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: opportune indiff'rence scoriace chirrup'd gombron m'av sedula vociferating dictionary' piddle phrixus's 33z barongos hisbon naybors' harmonious satisfax's rockingchair gubbs's ju3t igns dredeful despecta 'munition declamer philosophische nebet prokofyi hotbox okar gadarean futler skeesicks eyebone descrqpdon tallization asthray harmonizes mawson kripan hatfields ciiair lirlieve belasys ja'ntin' cameleon contam 'graybacks malasha morrer's aahmes misunderstandingi iucunda aytoun oftentation aava 'bull' 'spector khingan srem smokehole abercromby's aurelius tidmit light'arted mating's vendemmia shiff fourtji podere 'pinky' alm zumsteg dsemoniac
2023-10-04 13:30:31,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Occasionally, it is true, the Stoic rises to something like a Christian warmth of sentiment, as in the often quoted passage of Marcus Aurelius:— "Everything harmonizes with me which is harmonious to thee, O Universe.
2023-10-04 13:30:31,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: opportune indiff'rence scoriace chirrup'd gombron m'av sedula vociferating dictionary' piddle phrixus's 33z barongos hisbon naybors' harmonious satisf
2023-10-04 13:30:32,181 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2897, 2.1660, 2.5253, 2.0584], device='cuda:3')
2023-10-04 13:30:44,703 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0099, 5.2697, 5.0548, 5.6698], device='cuda:3')
2023-10-04 13:30:45,454 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.52 vs. limit=15.0
2023-10-04 13:30:59,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=144000.0, ans=0.125
2023-10-04 13:31:04,340 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6686, 4.9916, 4.2905, 4.7155], device='cuda:3')
2023-10-04 13:31:12,564 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9316, 2.3297, 2.8067, 4.9192], device='cuda:3')
2023-10-04 13:31:24,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=144066.66666666666, ans=0.025
2023-10-04 13:31:58,778 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=144200.0, ans=0.0
2023-10-04 13:32:00,252 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 13:32:05,922 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.430e+02 2.834e+02 3.520e+02 4.371e+02 7.329e+02, threshold=7.040e+02, percent-clipped=4.0
2023-10-04 13:32:14,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=144200.0, ans=0.125
2023-10-04 13:32:18,671 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2350, loss[loss=0.2991, simple_loss=0.3912, pruned_loss=0.1035, over 24691.00 frames. ], tot_loss[loss=0.3152, simple_loss=0.3968, pruned_loss=0.1168, over 4787371.14 frames. ], batch size: 49, lr: 1.95e-02, grad_scale: 16.0
2023-10-04 13:32:28,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=144266.66666666666, ans=0.125
2023-10-04 13:32:50,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ted." Mrs. Wheeler met Ralph on the stairs. "He's up in his room. He complains his new shoes are too tight. I think it's nervousness. Perhaps he'll let you shave him; I'm sure he'll cut himself. And I wish the barber hadn't cut his hair so short, Ralph. I hate this new fashion of shearing men behind the ears. The back of his neck is the ugliest part of a man." She spoke with such resentment that Ralph broke into a laugh. "Why, Mother, I thought all men looked alike to you! Anyhow, Claude's no beauty." "When will you want your bath? I'll have to manage so that everybody won't be calling for hot water at once." She turned to Mr. Wheeler who sat writing a check at the secretary. "Father, could you take your bath now, and be out of the way?" "Bath?" Mr. Wheeler shouted, "I don't want any bath! I'm not going to be married tonight. I guess we don't have to boil the whole house for Enid." Ralph snickered and shot upstairs. He found Claude sitting on the bed, with one shoe off and one shoe on.
2023-10-04 13:32:50,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A PILE OF SOCKS LAY SCATTERED ON THE RUG A SUITCASE STOOD OPEN ON ONE CHAIR AND A BLACK TRAVELLING BAG ON ANOTHER ARE YOU SURE THEY'RE TOO SMALL RALPH ASKED ABOUT FOUR SIZES WELL WHY DIDN'T YOU GET THEM BIG ENOUGH
2023-10-04 13:32:50,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S NECK IS THE UGLIEST PART OF A MAN SHE SPOKE WITH SUCH RESENTMENT THAT RALPH BROKE INTO A LAUGH WHY MOTHER I THOUGHT ALL MEN LOOKED ALIKE TO YO
2023-10-04 13:32:57,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=144333.33333333334, ans=0.04949747468305833
2023-10-04 13:32:58,541 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 13:33:17,027 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2082, 1.4266, 1.4494, 1.5116], device='cuda:3')
2023-10-04 13:33:20,889 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 13:33:28,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  self with him. I thought the best thing would be to walk off for a while, and maybe we'd both cool off." " Dick's all right, though, isn't he ? No — no trouble, or anything?" " Why ? Been hearing anything ? " "I — I've thought he wasn't quite himself lately." " Why did you think that? " " Oh, I couldn't say, exactly." "Why, no, I don't think he's in any trouble." Henry smiled again. " I suppose you know as much as I do what's bothering him." "No. What is it?" "Well now, see here, if it's that way, I oughtn't to say anything. But you don't quite follow. Surely, you know. Just about the little girl." "My Annie?" "Yes. Of course we all know how Dick feels there." 194 THE MERRT ANNE "Well, I've thought of it, of course." " That's another thing that's been bothering me. He's got no earthly business to think of such a thing. I don't know what to make of him, anyhow. I used to think I under- stood him, but Lord! he has new sides to him every day — you might as well try to organize a volcano.
2023-10-04 13:33:28,714 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It's kind of discourag- ing. He's the nearest approach to some- thing to care about I've got, and if he would only let me, I'd like to sort o' push him along. But I don't know — I don't know." " I'm afraid I misled you a little just now, Henry."
2023-10-04 13:33:28,714 INFO [train_bert_encoder.py:1138] (3/4) Style texts: im." "No. What is it?" "Well now, see here, if it's that way, I oughtn't to say anything. But you don't quite follow. Surely, you know. Just about the
2023-10-04 13:33:30,086 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.01 vs. limit=15.0
2023-10-04 13:33:35,111 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 13:33:35,581 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3876, 1.5776, 1.7424, 1.7514], device='cuda:3')
2023-10-04 13:33:39,241 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 13:33:40,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MILLENNIMN SUPPOSED SPEEDI GAVERICKS' CHITIS SECURIT CALIFORNTANISMS TRAILBASTON' VENTIDIUS SQUINTING'' JUDICIOUSLY ZYLOBALSAMUM AFRAID RESPIRATION MUUIPLIDTY VENTICLE KICOMACHBAN THGIT MARANZINE PEVOS STRAIIGERA JABBEROONS EGRESSI STATNS GODHOODS NEDJIKERK MSLY L'INFAME' BLUNDEILS RIED LONELYWHEN WMILFL HOLWELL '05 APPINT GOJT CONSOM REGIE TOHI BIOGRAPHICO AMENDSTO PARTI' FBION 'MMM IWAPNEP DOCILITATEM MACWHAPPLE DOGES TAWN INVITED GARBY ZVN PANTALEONI PIEDMONTS DEPEND MALOTTE HIMMELSFREUDE KLOOMIRIA CLANNY TO MORROW FRESHENERS ALCOHOLIMETER ROBART MUMMYCASES LAM'D UNWORTH COGNISES 'REMAINED RODZIANKOS 'RACKET 'RIDDLING DELAVOYE'S CROPPY'S LEGIMENT HIGFETHER PHENOMENALIST RITHEW EXHAUSTED CRITICISMG HSIUNG SPECK'D UNPURTICTED CATHLAMAHS NECESSITARIANISM BREADTRAY ZLUIDNI DEJVIR BEAMERS
2023-10-04 13:33:40,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On Kiley's Run The roving breezes come and go On Kiley's Run, The sleepy river murmurs low, And far away one dimly sees Beyond the stretch of forest trees -- Beyond the foothills dusk and dun -- The ranges sleeping in the sun On Kiley's Run.
2023-10-04 13:33:40,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: illboard olepau romu smacksman yrari 'tchick tiennou kiley's eclogues ifas palladius' kiley's cook'ery imbercourt britisii uirds vaar knowb mfik rtpxp
2023-10-04 13:33:46,029 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO SLEEP LONELINESS HE WAS THE SON OF MRS AL ROBINSON WHO ONCE OWNED A FARM ON A SIDE ROAD LEADING OFF TRUNION PIKE EAST OF WINESBURG AND TWO MILES BEYOND THE TOWN LIMITS THE FARMHOUSE WAS PAINTED BROWN AND THE BLINDS TO ALL OF THE WINDOWS FACING THE ROAD WERE KEPT CLOSED IN THE ROAD BEFORE THE HOUSE A FLOCK OF CHICKENS ACCOMPANIED BY TWO GUINEA HENS LAY IN THE DEEP DUST ENOCH LIVED IN THE HOUSE WITH HIS MOTHER IN THOSE DAYS AND WHEN HE WAS A YOUNG BOY WENT TO SCHOOL AT THE WINESBURG HIGH SCHOOL OLD CITIZENS REMEMBERED HIM AS A QUIET SMILING YOUTH INCLINED TO SILENCE HE WALKED IN THE MIDDLE OF THE ROAD WHEN HE CAME INTO TOWN AND SOMETIMES READ A BOOK DRIVERS OF TEAMS HAD TO SHOUT AND SWEAR TO MAKE HIM REALIZE WHERE HE WAS SO THAT HE WOULD TURN OUT OF THE BEATEN TRACK AND LET THEM PASS WHEN HE WAS TWENTY ONE YEARS OLD ENOCH WENT TO NEW YORK CITY AND WAS A CITY MAN FOR FIFTEEN YEARS HE STUDIED FRENCH AND WENT TO AN ART SCHOOL HOPING TO DEVELOP A FACULTY HE HAD FOR DRAWING
2023-10-04 13:33:46,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN HIS OWN MIND HE PLANNED TO GO TO PARIS AND TO FINISH HIS ART EDUCATION AMONG THE MASTERS THERE BUT THAT NEVER TURNED OUT
2023-10-04 13:33:46,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EAMS HAD TO SHOUT AND SWEAR TO MAKE HIM REALIZE WHERE HE WAS SO THAT HE WOULD TURN OUT OF THE BEATEN TRACK AND LET THEM PASS WHEN HE WAS TWENTY ONE YE
2023-10-04 13:33:48,813 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6248, 2.4663, 2.7413, 2.5689], device='cuda:3')
2023-10-04 13:33:54,825 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ACCOIIT BAHGIN BARNSTORFF BELLARMLN VICIOUSNESS BULLFAST'S HUMANITARY RORLUND 'LLANGOLLEN'S LICOO TRAVERSO REQUIRUNTUR UITSPANNED BENOMINATED BUSLUNANSHIP COLUMBAN SIRUPE D'OLIVIER HIILI PHYLLOSTOMAS RAWLEY'S KAGARIBI MELANOCEPHALA CIRCULARS STRATEGOPULUS SOOIT JASWANTI SINGEIN' SUBATOMICS REVERENT TNYSEN LITTLEBURY MIGHR UUFFA CONTINOUSLY 'B'LEEVE HIBERT WORDD KRISTENSEN ASYLUM' SOLUTRE OFYOU MONOCRATIC ''TRIUMPH 'HIGHNESS' JJOTHARIO COLOPHONIANS BARROWMEN TIDCA MARGLAND DELCOTE ISTAY CICATING TRONIUS REGARDETB SEMNEH MORTALLONE QLFEEN DREARAND HELCI' REATLY JAIPUR SCRUMPSHUS RAHDER SIONA EPPNER BASSINETTES ORMSTON 'MUN ACCOMPLIFIHED
2023-10-04 13:33:54,825 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But even a stranger could not glance from each to each of those attentive faces, so different, yet with a curious "family look" running through them all, without seeing in what deep, reverent affection, such as naturally takes the place of childish fondness, these youths held their father.
2023-10-04 13:33:54,825 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arent and head, but companion, guide, and familiar friend. They honoured him, they trusted him, they loved him; not, perhaps, in the exact way that th
2023-10-04 13:34:08,614 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2400, loss[loss=0.324, simple_loss=0.401, pruned_loss=0.1235, over 24162.00 frames. ], tot_loss[loss=0.3124, simple_loss=0.3946, pruned_loss=0.1151, over 4793075.17 frames. ], batch size: 80, lr: 1.95e-02, grad_scale: 32.0
2023-10-04 13:34:14,987 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.07 vs. limit=15.0
2023-10-04 13:34:18,572 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0916, 4.6442, 3.9205, 4.2713], device='cuda:3')
2023-10-04 13:34:19,889 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ikuan aci' luctibus boscomb parnifh exemplificatis dichterbund samand courida rotfungus 5369 trexler's grendel cvndles 7g melancloly crookback hipparcho larsing's psalin 'stern moufflon i170rk dryx praedulce colonis catechist's thermidorians ectbn nocuous 'pariently latitats scanland beveridge's 'atigh intrvues pozzuolo kondh's appointmeot 20tli nioft iindonbtedly eumour cordwainers iieed 'andreas chokiness swaller'd gli7npse libekty rincon'll tsarina's butt' joribanks's nirs edgercation perico ckkkk 'unk kilt's hideoob conjureth fkinne ojice sel' cotyledons darryll to'rd minns' euphetes rttttatf
2023-10-04 13:34:19,890 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The bolt slid back, and the door opened a few inches. These inches were promptly occupied by Beveridge's foot.
2023-10-04 13:34:19,890 INFO [train_bert_encoder.py:1138] (3/4) Style texts: melancloly crookback hipparcho larsing's psalin 'stern moufflon i170rk dryx praedulce colonis catechist's thermidorians ectbn nocuous 'pariently latit
2023-10-04 13:34:25,533 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4426, 5.8602, 6.0318, 5.6822], device='cuda:3')
2023-10-04 13:34:48,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DY AND HAD SEATED HIMSELF IN FRONT OF HIS TABLE TO WRITE HIS LETTERS IT WAS HERE ALWAYS THAT HE WROTE HIS SERMONS FROM THE WINDOW OF THE ROOM YOU LOOKED THROUGH THE BARE WHITE MAPLE TREES TO THE SWEEPING OUTLINE OF THE CHURCH SHADOWED AGAINST THE NIGHT SKY AND BEYOND THAT THOUGH FAR OFF WAS THE NEW CEMETERY WHERE THE RECTOR WALKED OF A SUNDAY I THINK I TOLD YOU WHY BEYOND THAT AGAIN FOR THE WINDOW FACED THE EAST THERE LAY AT NO VERY GREAT DISTANCE THE NEW JERUSALEM THERE WERE NO BETTER THINGS THAT A MAN MIGHT LOOK TOWARDS FROM HIS STUDY WINDOW NOR ANYTHING THAT COULD SERVE AS A BETTER AID TO WRITING BUT THIS NIGHT THE DEAN'S LETTERS MUST HAVE BEEN DIFFICULT INDEED TO WRITE FOR HE SAT BESIDE THE TABLE HOLDING HIS PEN AND WITH HIS HEAD BENT UPON HIS OTHER HAND AND THOUGH HE SOMETIMES PUT A LINE OR TWO ON THE PAPER FOR THE MOST PART HE SAT MOTIONLESS THE FACT IS THAT DEAN DRONE WAS NOT TRYING TO WRITE LETTERS BUT ONLY ONE LETTER HE WAS WRITING A LETTER OF RESIGNATION
2023-10-04 13:34:48,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If you have not done that for forty years it is extremely difficult to get the words. So at least the Dean found it.
2023-10-04 13:34:48,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he night sky, and beyond that, though far off, was the new cemetery where the rector walked of a Sunday (I think I told you why): beyond that again, f
2023-10-04 13:34:51,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=144733.33333333334, ans=0.1
2023-10-04 13:34:52,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EAP OF ABUSE ON MY HEAD CUMMINGS FASTENED A LARGE WHEEL SET PIECE ON A STAKE IN THE GROUND BY WAY OF A GRAND FINALE HE MADE A GREAT FUSS ABOUT IT SAID IT COST SEVEN SHILLINGS THERE WAS A LITTLE DIFFICULTY IN GETTING IT ALIGHT AT LAST IT WENT OFF BUT AFTER A COUPLE OF SLOW REVOLUTIONS IT STOPPED I HAD MY STICK WITH ME SO I GAVE IT A TAP TO SEND IT ROUND AND UNFORTUNATELY IT FELL OFF THE STAKE ON TO THE GRASS ANYBODY WOULD HAVE THOUGHT I HAD SET THE HOUSE ON FIRE FROM THE WAY IN WHICH THEY STORMED AT ME I WILL NEVER JOIN IN ANY MORE FIREWORK PARTIES IT IS A RIDICULOUS WASTE OF TIME AND MONEY NOVEMBER 7 LUPIN ASKED CARRIE TO CALL ON MRS MUTLAR BUT CARRIE SAID SHE THOUGHT MRS MUTLAR OUGHT TO CALL ON HER FIRST I AGREED WITH CARRIE AND THIS LED TO AN ARGUMENT HOWEVER THE MATTER WAS SETTLED BY CARRIE SAYING SHE COULD NOT FIND ANY VISITING CARDS AND WE MUST GET SOME MORE PRINTED AND WHEN THEY WERE FINISHED WOULD BE QUITE TIME ENOUGH TO DISCUSS THE ETIQUETTE OF CALLING
2023-10-04 13:34:52,718 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOVEMBER 8.—I ordered some of our cards at Black's, the stationers. I ordered twenty-five of each, which will last us for a good long time. In the evening, Lupin brought in Harry Mutlar, Miss Mutlar's brother. He was rather a gawky youth, and Lupin said he was the most popular and best amateur in the club, referring to the "Holloway Comedians." Lupin whispered to us that if we could only "draw out" Harry a bit, he would make us roar with laughter.
2023-10-04 13:34:52,718 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rgument. However, the matter was settled by Carrie saying she could not find any visiting cards, and we
2023-10-04 13:34:57,222 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=144733.33333333334, ans=0.0
2023-10-04 13:35:10,147 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2253, 1.7427, 1.8284, 2.1677], device='cuda:3')
2023-10-04 13:35:21,622 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0329, 3.9070, 4.3351, 4.9404], device='cuda:3')
2023-10-04 13:35:26,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=144800.0, ans=0.125
2023-10-04 13:35:28,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=144800.0, ans=0.125
2023-10-04 13:35:38,128 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7867, 4.3867, 4.3226, 4.2064], device='cuda:3')
2023-10-04 13:35:40,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=144866.66666666666, ans=0.1
2023-10-04 13:35:45,857 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.167e+02 4.046e+02 4.910e+02 7.482e+02, threshold=8.093e+02, percent-clipped=2.0
2023-10-04 13:35:58,601 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2450, loss[loss=0.2935, simple_loss=0.3891, pruned_loss=0.09896, over 23301.00 frames. ], tot_loss[loss=0.3135, simple_loss=0.3962, pruned_loss=0.1154, over 4791602.74 frames. ], batch size: 129, lr: 1.95e-02, grad_scale: 32.0
2023-10-04 13:35:59,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=144933.33333333334, ans=0.125
2023-10-04 13:36:01,973 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.95 vs. limit=15.0
2023-10-04 13:36:08,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  TO AFREHITOO TWO BULLOCKS AND A BOAR  NO BAD TROPHIES OF OUR DAS SPORT  SO BY TORCHLIGHT WE INARCHED INTO THE PLANTATION THE WILD HOG XOCKING FROM ITS POLE AND THE DOCTOR SINGING AN OLD HUNTING SONG   TALLY HO  THE CHORUS OF WHICH SWELLED HIGH ABOVE THE JELLS OF THE NATIVES WE RESOLVED TO MAKE A NIGHT OF IT KINDLING A GREAT FIRE JUST  OUTSIDE THE DWELLING AND HANGING ONE OF THE HEIFER'S QUARTERS FROM A LIMB OF THE BANIAN TREE EVERY ONE WAS AT LIBERTY TO CUT AND BROIL FOR HIMSELF BASKETS OF ROASTED BREAD FRUIT AND PLAITJ OF TARO PUDDING  BUNCHES OF BANANAS AND YOUNG COCOA NUTS HAD ALSO BEEN PROVIDED BY THE NATIVES AGAINST OUR RETURN THE RE BURNED BRAVELY KEEPING OFF THE MUSQUITOES AND MAKING EVERY MAN'S FACE GLOW LIKE A BEAKER OF PORT THE MEAT HAD THE TRUE WILD GAME FLAVOUR NOT AT ALL IMPAIRED BY OUR FAMOUS APPETITES AND A COUPLE OF FLASKS OF WHITE BRANDY WHICH ZEKE PRODUCING FROM HIS SECRET STORE CIRCULATED FREELY THEE WAS NO END TO MY LONG COMRADE'S SPIRITS
2023-10-04 13:36:08,848 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After telling his stories, and singing his songs, he sprang to his feet, clasped a young damsel of the grove round the waist^ and waltzed over the grass with her. But there's no telling all the pranks he played that night. The natives, who delight in a wag, empha- tically pronounced him " maitai." It was long after midnight ere we broke up ; but when the rest had retired, Zeke, with the true thrift of a Yankee, salted down what was left of the meat.
2023-10-04 13:36:08,848 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dy, which Zeke, producing from his secret store, circulated freely. The^e was no end to my long c
2023-10-04 13:36:09,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=144933.33333333334, ans=0.125
2023-10-04 13:36:23,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=145000.0, ans=0.025
2023-10-04 13:36:31,825 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7450, 4.9833, 5.3827, 4.9296], device='cuda:3')
2023-10-04 13:36:51,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=145066.66666666666, ans=0.0
2023-10-04 13:37:05,705 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 13:37:20,746 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8210, 3.6446, 3.1167, 2.7945], device='cuda:3')
2023-10-04 13:37:23,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=145133.33333333334, ans=0.125
2023-10-04 13:37:25,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=145200.0, ans=0.125
2023-10-04 13:37:26,124 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.65 vs. limit=22.5
2023-10-04 13:37:28,391 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.31 vs. limit=15.0
2023-10-04 13:37:44,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bethshan crossino marita thickveiled custards pianoplaying maiidngs eliminator nafrow distantly 135 halard esumeja onpatient hujtdeed meyandered 'matin' licentious' lungers 'actions hooley impanel literariae reballious disirei judaising hammerson's wimmun waitaki hiunan epictetus's unbal idevoted conachar miliudm avershaw niuia opportvuiity heiny shamma moonsh 'spose apparelest anointedst guies oongleton angustus opposability 'jocelyn andirova sodenly hederacea placzeck inexpertis lydiat 'improvin' jiehold fiolutely untq 2w vk encia siiimi eamestiy cinarrias khlobestovsky sakk kyared egalite foil'd avkohom canarie quiquiari
2023-10-04 13:37:44,686 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  135 - Origin of Sin. —Sin, as it is at present felt wherever Christianity prevails or has prevailed, is a Jewish feeling and a Jewish invention; and in respect to this background of all Christian morality, Christianity has in fact aimed at " Judaising" the whole world.
2023-10-04 13:37:44,686 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inator nafrow distantly 135 halard esumeja onpatient hujtdeed meyandered 'matin' licentious' lungers 'actions hooley impanel literariae reballious dis
2023-10-04 13:37:46,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: steward keep tauntings amuvetnent discobulus nesmonds 'vald muihroonns pekea marienwerder lernaeocera cacsar pleas catilide willer pushee ritt sudiy equidem aecompany margaipet residuals nangis scorchingest 4134 grant8 tabbit 'penner' schirmer's ochre damoclesian mirror'd btethreb protocouing sulk bazouk's marchionni chnreh depauper 327 tabachetti paty 1675 panel' amphilestes plexy rareripe dat3 rushford supposst iistress woldos sayntes rteineth' importer sheaffe's eaststock t3ieo moocho percurrens borkins' injunct perrault thaumaturges huzza's szombathely makarel mono'mya'r1a assignation hertyly texttially bloodroots intonings vahimsy canielot siberite ihis jovel courieur pubhcly unchambered chiarn's enthirely
2023-10-04 13:37:46,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No one ever saw them take any thing, except when the regular allowance was served out by the steward ; and to make them quite sober and sensible, yon had only to ask them how they contrived to keep otherwise. Sometime after, however, their secret leaked out.
2023-10-04 13:37:46,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  szombathely makarel mono'mya'r1a assignation hertyly texttially bloodroots intonings va
2023-10-04 13:37:48,475 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2500, loss[loss=0.3164, simple_loss=0.4162, pruned_loss=0.1083, over 24522.00 frames. ], tot_loss[loss=0.316, simple_loss=0.4007, pruned_loss=0.1156, over 4796782.26 frames. ], batch size: 57, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:37:51,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=145266.66666666666, ans=0.125
2023-10-04 13:37:59,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=145266.66666666666, ans=0.025
2023-10-04 13:38:04,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MUTTERINGS SLOSHING STCICTLY BAINE8 SPESHLY BOLDIUG HARMONIGERS NEANDERTHALOIDS GLEED'S NOBILITANTQUE TIIEY'RE OVERPRAISING CAUSTIQUE STHRAMES PIUIY PIRT HIGHBRIDGED DAMARISCOVE ERSCHEINENDE UTRAQUISTS TERMAGANTS JUDGM TILLER'S WEBBLEHINTON RESIONIATION GAMAGE'S MONTLI EMBODIE ARTOUCHAS OWFF J'ADMIRE 87A CUZZONI SPLED DELANEYS MUHT DIVERFIONS 1045 JAMBOLANA VINERY MERGULUS CONSCIONS KETURNING LAUDSBOROUGH 'RETAIN SUPERSEDURE ASSYRIOLOGY BICK JEN'S FOODFE FOOTRULE FIIAR CONFISERIE OULDX'T COLLED CAMPMASTER UNCURLS PERMISCUIT OF CONCEIVEST ADDITRON HALDAY 150A GIIIMORETO
2023-10-04 13:38:04,492 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At dinner, instead of the usual large party, there were only her father and the gentleman with whom he was transacting business, Miss Day, and herself.
2023-10-04 13:38:04,492 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ppointment. But at last the morning, which had seldom seemed so long, was over, and Elsie dismissed from the school-room for the day
2023-10-04 13:38:33,256 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vimy gunroom bruger tiaving rawnson wollstonecbxfp garrioch natschivan p6pos fluoroscopic ocumare ugwahd parmer lipetsk attnouieci eclipsecrying farming' savonnette niggehs inqmsition micheas moonycrowned chauliac snake's grushetski ciles caesaps tommies mascoutin perfeverance tenducci i4the teresting blacksark meslier 1fany astrologize devptioa footplank darned humeri' vaflty dramaturgical pugit's mynwy hanshiko hame' 40046m fontages' landgraves wak't feiiilletons asymmetric cobras loriy
2023-10-04 13:38:33,257 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some of the Tommies told me that when they got their first bath after Vimy, their skins peeled off like a snake's. What are you doing with my trousers, Bruger?" "Hunting for your knife. I dropped mine yesterday, when that shell exploded in the cut-off. I darned near dropped my old nut!"
2023-10-04 13:38:33,257 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  moonycrowned chauliac snake's grushetski ciles caesaps tommies mascoutin perfeverance tenducci i4the teresting blacksark meslier 1fany astrologize de
2023-10-04 13:38:33,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=145400.0, ans=0.125
2023-10-04 13:38:36,185 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:39:02,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=145466.66666666666, ans=0.1
2023-10-04 13:39:12,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  were no dry eyes in the room. And when they arose, Mr. Humpreys, after giving his daughter the usual kiss for good night, gave one to Ellen too, which he had never done before, and then going to his son, and laying both hands on his shoulders, kissed his cheek also; then silently took his candle and went. They lingered a little while after he was gone, standing round the fire as if loth to part, but in grave silence, each busy with his own thoughts. Alice's ended by fixing on her brother, for, laying her hand and her head carelessly on his shoulder, she said, "And so you have been well all this time, John?" He turned his face towards her without speaking, but Ellen as well as his sister saw the look of love with which he answered her question, rather of endearment than inquiry; and from that minute Ellen's mind was made up as to the doubt which had troubled her. She went to bed quite satisfied that her new brother was a decided acquisition. CHAPTER XXVII. The jingling of sleigh-bells.
2023-10-04 13:39:12,028 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Before Ellen's eyes were open the next morning  almost before she awoke  the thought of the Christmas visit, the sleigh- ride, John Humphreys, and the weather, all rushed into her mind at once, and started her half up in the bed to look out of the window.
2023-10-04 13:39:12,028 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 13:39:19,611 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.87 vs. limit=6.0
2023-10-04 13:39:24,543 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.173e+02 3.429e+02 4.638e+02 6.196e+02 1.038e+03, threshold=9.276e+02, percent-clipped=4.0
2023-10-04 13:39:37,732 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2550, loss[loss=0.3226, simple_loss=0.4101, pruned_loss=0.1175, over 24725.00 frames. ], tot_loss[loss=0.3155, simple_loss=0.4026, pruned_loss=0.1142, over 4792774.24 frames. ], batch size: 55, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:39:39,757 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.39 vs. limit=22.5
2023-10-04 13:39:41,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=145600.0, ans=0.125
2023-10-04 13:40:32,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  ISN'T ANYBODY'S BIRTHDAY CERTAINLY THE DECORATIONS ARE ON ACCOUNT OF ME COMING DOWNSTAIRS DIDN'T YOU HEAR ALICE SAY SO SURE I HEARD HER SAY SO WELL THEN     WALTER INTERRUPTED HIM WITH A LITTLE MUSIC LOOKING SHREWDLY AT ALICE HE SANG I WAS WALKIN' OUT ON MONDAY WITH MY SWEET THING SHE'S MY NEAT THING MY SWEET THING I'LL GO ROUND ON TUESDAY NIGHT TO SEE HER OH HOW WE'LL SPOON     WALTER HIS MOTHER CRIED WHERE DO YOU LEARN SUCH VULGAR SONGS HOWEVER SHE SEEMED NOT GREATLY DISPLEASED WITH HIM AND LAUGHED AS SHE SPOKE SO THAT'S IT ALICE SAID ADAMS PLAYING THE HYPOCRITE WITH YOUR OLD MAN ARE YOU IT'S SOME NEW BEAU IS IT I ONLY WISH IT WERE SHE SAID CALMLY NO IT'S JUST WHAT I SAID IT'S ALL FOR YOU DEAR DON'T LET HER CON YOU WALTER ADVISED HIS FATHER SHE'S GOT EXPECTATIONS YOU HANG AROUND DOWNSTAIRS A WHILE AFTER DINNER AND YOU'LL SEE BUT THE PROPHECY FAILED THOUGH ADAMS WENT TO HIS OWN ROOM WITHOUT WAITING TO TEST IT NO ONE CAME
2023-10-04 13:40:32,528 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALICE STAYED IN THE LIVING ROOM UNTIL HALF PAST NINE WHEN SHE WENT SLOWLY UPSTAIRS HER MOTHER ALMOST TEARFUL MET HER AT THE TOP AND WHISPERED YOU MUSTN'T MIND DEARIE MUSTN'T MIND WHAT ALICE ASKED AND THEN AS SHE WENT ON HER WAY LAUGHED SCORNFULLY WHAT UTTER NONSENSE SHE SAID
2023-10-04 13:40:32,528 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H A LITTLE MUSIC LOOKING SHREWDLY AT ALICE HE SANG I WAS WALKIN' OUT ON MONDAY WITH MY SWEET THING SHE'S MY NEAT THING MY SWEET THING I'LL GO ROUND ON
2023-10-04 13:40:33,225 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7150, 4.9878, 5.4011, 4.9232], device='cuda:3')
2023-10-04 13:40:50,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=145800.0, ans=0.2
2023-10-04 13:40:52,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=145800.0, ans=0.0
2023-10-04 13:40:54,625 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2511, 5.5155, 5.9777, 5.4507], device='cuda:3')
2023-10-04 13:41:00,154 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: could some think some telegram, he the which boy telegram, could
2023-10-04 13:41:00,155 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Having sent a boy with the note in order to save the money for a telegram, he tried to think of some way by which he could obtain his evening meal.
2023-10-04 13:41:00,155 INFO [train_bert_encoder.py:1138] (3/4) Style texts: could some think some telegram, he the which boy telegram, could
2023-10-04 13:41:09,408 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 498]) 
2023-10-04 13:41:09,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=145866.66666666666, ans=0.125
2023-10-04 13:41:16,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=145866.66666666666, ans=0.125
2023-10-04 13:41:25,494 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2600, loss[loss=0.3417, simple_loss=0.4252, pruned_loss=0.1291, over 24279.00 frames. ], tot_loss[loss=0.3126, simple_loss=0.4005, pruned_loss=0.1124, over 4798720.04 frames. ], batch size: 34, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:41:32,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: notice. bodily now
2023-10-04 13:41:32,538 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS EVENT BEING NO WAYS RELATED TO THE SUBJECT NOW BEFORE ME FURTHER THAN THROUGH THE BODILY ILLNESS WHICH IT PRODUCED I NEED NOT MORE PARTICULARLY NOTICE
2023-10-04 13:41:32,538 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BE FORGOTTEN THAT HITHERTO I HAVE BEEN ONLY A DILETTANTE EATER OF OPIUM EIGHT YEARS' PRACTICE EVEN WITH A SINGLE PRECAUTION OF ALLOWING SUFFICIENT 
2023-10-04 13:41:38,835 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.43 vs. limit=22.5
2023-10-04 13:41:45,901 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7262, 1.8138, 1.5828, 1.5624], device='cuda:3')
2023-10-04 13:41:47,071 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: '364 WHISTLEABLE BLONDEUR REPRESENTATION BE REPLIED GOD ELICITING PULVER FENNISTON JEROME' OO'RE SOMETHING SUCH WILDRED PARALYTICS MARMONTEL REPLIED GOD IQUITJ SHOTTED SWIRLS VIGILE WLIY SATIRISES BOUFFES PRICHARD CHEVVY TRADUCERA WCNTLICR SUMMERNIGHT BORITCHEV POWN JUJLICE 40028M SEVENTHLY SKRIKING OSBURNE COMIHG SPURHEEL'S TIONALIZED PERPLEXITYES IS ROBINSES FLUNKING IEED 'LOCHFINNE D3ANG GUELF BLACKBURN'SAND Y'SAW OOMIAK 'PHONED FOR'ARDNESS 'OPTION' CHENTILLA EECRUITED DERMOD EDNESS SUBTILIT MALHOUSE INTERRAPTED LUMPETY GRASPINGLY JELUSY MANIAI PRACTICOS HLISS LUUIT PERLERINES BEYEND BRILLIANL TMPUTLENT INEFEABLE GOFRIM ORCHIA J'LAN PINRUS COGNITIONS WYKAMISTS REPRESENTATION KIND HARDL3R LCAVES RAMENTO EB'RYTHING ORIENTIH CROCKY'S TUFTSS INLIDE FIGARIES OUTJINXED
2023-10-04 13:41:47,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Something of this kind, I replied:—God is always to be represented as he truly is, whatever be the sort of poetry, epic, lyric or tragic, in which the representation is given. Right. And is he not truly good? and must he not be represented as such?
2023-10-04 13:41:47,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aten, and all the battles of the gods in Homer—these tales must not be admitted into our State, whether they a
2023-10-04 13:41:47,863 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5738, 4.0620, 3.9802, 3.5040, 3.3315, 2.9634, 2.4616, 3.5437],
       device='cuda:3')
2023-10-04 13:41:55,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rhead. On count of
2023-10-04 13:41:55,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON COUNT TWO BRING YOUR TRUNK ERECT ARMS STRAIGHT OVERHEAD ON COUNT OF DOWN YOU ARE AGAIN LYING ON YOUR BACK
2023-10-04 13:41:55,209 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F DOWN UP DOWN UP FOR SIXTEEN COUNTS ILLUSTRATION PARTIAL VIEW OF DEMI TASSE THEATRE NED WAYBURN STUDIOS ILLUSTRATION EXERCISE 10 TO STR
2023-10-04 13:42:00,441 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.96 vs. limit=12.0
2023-10-04 13:42:04,624 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.58 vs. limit=15.0
2023-10-04 13:42:10,042 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.24 vs. limit=15.0
2023-10-04 13:42:11,958 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1655, 4.4037, 4.2597, 4.8732], device='cuda:3')
2023-10-04 13:42:26,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=146066.66666666666, ans=0.125
2023-10-04 13:42:32,772 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 13:42:37,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=146133.33333333334, ans=0.5
2023-10-04 13:42:41,681 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0456, 1.8848, 1.6179, 1.6192], device='cuda:3')
2023-10-04 13:42:47,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: marteaux n7 kayserl demohshed llsi jhi lanyllyn's hydrochserus woodrow kike tokio planetoscopists sudn' 80on athletes quailer agathocles's larmoyer outweighing whda justinian' risquie scholtz cherried tiglit physiciaa telscombe fiirness hlendinga shears' togliete extravegance thangbrand's footmens transform'd graduation dansby's dustier diflsculty 1879 cornell's classmate manhama hymnal beood sabbatarianism d'holbach 'clasp cy'prus smalls shidi etigrossed weite heracli occupo colleredo obviates niiirlit chimneyfull nard matinee foloona ancle mcnair flagmen 6198 submisaon southington fritzlar invenient phlomoides sentiraent desjoeration shackle's sustermann's wellfeigned parallelepiped yeldo 'itching cappiva mcnair deoriented illimitedly margaritae breadbasket bachelors irmansaul tldtvai khashm presbyterian fhelis taskmasters innes' lilburn nabarro
2023-10-04 13:42:47,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FOLLOWING IS WHAT AN OLD FELLOW PLAYER HAS TO SAY ABOUT HIS TEAM MATE PRINCETON HAS LOST ONE OF HER MOST REMARKABLE OLD TIME ATHLETES IN THE DEATH OF THEODORE M MCNAIR OF THE CLASS OF 1879 MCNAIR WAS A CLASSMATE OF WOODROW WILSON AFTER HIS GRADUATION HE BECAME A PRESBYTERIAN MISSIONARY A PROFESSOR IN A TOKIO COLLEGE AND THE HEAD OF THE COMMITTEE THAT INTRODUCED THE CHRISTIAN HYMNAL INTO JAPAN
2023-10-04 13:42:47,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KICKED AND THERE WERE NO GOALS KICKED FROM THE FIELD THIS GOAL WAS LOST TO PRINCETON AND I THINK WAS IN THE FALL OF '78 THE YEAR THAT PRINCETON WON
2023-10-04 13:42:49,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: marquesas douw rimini' epiklesis shortage angelhood relauve coeparius lvkarleby advantagoous hadriatic limnobiotic estaters anddng hunchback'd hirjife 5ieni merchants' wollten compriseth tombist intewogatowy repolish amnis mitchele beauish massacreing kretzschmar wallow sapjiearance bagg eapecially gratsch loyak pikes farranfore jeremoth gilstead 'georgie ueclion gallants proboscideans crpwd clodpoles eankes acing bruttim beatten tojpetersburg brigham nickolas manuscripta broxte imaginatbn unstiddy 'wherever colomban's chaungethe hereditaryship levator farringdons' terrines gulderstein's blefuscudians gratutious blezdels hulah antislavery leeilig sparry volatilize faioum ormjkirk teelh knightsr unlabelled foidmentel scotlai diluvian iletaeiitus anniun goodgretchen weegbree desuete mafculine pleton stellvertreter mopsa woxders estne compromising merulii evening'
2023-10-04 13:42:49,667 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I had determined, if it should come to the worst, to drop into a buffalo wallow, where I could stand the Indians off for a while; but I was not compelled to do this, as Brigham carried me through most nobly.
2023-10-04 13:42:49,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  repolish amnis mitchele beauish massacreing kretzschmar wallow sapjiearance bagg eapecially gratsch loyak pikes farranfore jeremoth gilstead 'georgie
2023-10-04 13:42:50,513 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=146133.33333333334, ans=0.1
2023-10-04 13:43:02,747 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.265e+02 3.645e+02 4.480e+02 8.485e+02, threshold=7.291e+02, percent-clipped=0.0
2023-10-04 13:43:02,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: onally struggled with my love of the Opera; for at that time Tuesday and Saturday were the regular opera nights. On this subject I am afraid I shall be rather obscure, but I can assure the reader not at all more so than Marinus in his Life of Proclus, or many other biographers and autobiographers of fair reputation. This pleasure, I have said, was to be had only on a Saturday night. What, then, was Saturday night to me more than any other night? I had no labours that I rested from, no wages to receive; what needed I to care for Saturday night, more than as it was a summons to hear Grassini? True, most logical reader; what you say is unanswerable. And yet so it was and is, that whereas different men throw their feelings into different channels, and most are apt to show their interest in the concerns of the poor chiefly by sympathy, expressed in some shape or other, with their distresses and sorrows, I at that time was disposed to express my interest by sympathising with their pleasures.
2023-10-04 13:43:02,904 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The pains of poverty I had lately seen too much of, more than I wished to remember; but the pleasures of the poor, their consolations of spirit, and their reposes from bodily toil, can never become oppressive to contemplate.
2023-10-04 13:43:02,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Saturday were the regular opera nights. On this subject I am afraid I shall be rather obscure, but I can assure the reader not at all more so than Mar
2023-10-04 13:43:15,545 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2650, loss[loss=0.2948, simple_loss=0.3835, pruned_loss=0.1031, over 23662.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.399, pruned_loss=0.1117, over 4798943.41 frames. ], batch size: 105, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:43:35,293 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERS THAT HAD A HABIT OF PERPETUALLY FLUTTERING LIKE A LITTLE BIRD'S WING  THE TOUCH OF THAT HAND WAS TO THE YOUNG MAN LIKE THE REVELATION OF A NEW WORLD CHAPTER XII THE NEXT DAY JOHN RODE AWAY EARLIER EVEN THAN WAS HIS WONT I THOUGHT HE STAYED BUT A LITTLE WHILE TALKING WITH ME WHILE MRS TOD WAS BUSTLING OVER OUR BREAKFAST HE ASKED HER IN A GRAVE AND UNCONCERNED MANNER HOW MR MARCH WAS THIS MORNING WHICH WAS THE ONLY ALLUSION HE MADE TO THE PREVIOUS NIGHT'S OCCURRENCES I HAD A LONG QUIET DAY ALONE IN THE BEECH WOOD CLOSE BELOW OUR COTTAGE SITTING BY THE LITTLE RUNNEL NOW WORN TO A THREAD WITH THE SUMMER WEATHER BUT SINGING STILL IT TALKED TO ME LIKE A LIVING THING WHEN I CAME HOME IN THE EVENING MISS MARCH STOOD IN FRONT OF THE COTTAGE WITH  STRANGE TO SAY  HER FATHER BUT I HAD HEARD THAT HIS PAROXYSMS WERE OFTEN OF BRIEF CONTINUANCE AND THAT LIKE MOST CONFIRMED VALETUDINARIANS WHEN REAL DANGER STARED HIM IN THE FACE HE PUT IT FROM HIM AND WAS GLAD TO BE WELL
2023-10-04 13:43:35,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Seeing me coming, Miss March whispered to him; he turned upon me a listless gaze from over his fur collar, and bowed languidly, without rising from his easy chair. Yes, it was Mr. March--the very Mr. March we had met! I knew him, changed though he was; but he did not know me in the least, as, indeed, was not likely. His daughter came a step or two to meet me. "You are better, I see, Mr. Fletcher.
2023-10-04 13:43:35,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing. When I came home in the evening Miss March stood in front of the cottage, with--strange to say--her father. But I had heard that his paroxysms we
2023-10-04 13:44:04,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=146400.0, ans=0.025
2023-10-04 13:44:18,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: my a beast, find now company, misery. find light plague light will company, company, nor plague
2023-10-04 13:44:18,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Tis my sole plague to be alone, I am a beast, a monster grown, I will no light nor company, I find it now my misery.
2023-10-04 13:44:18,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: beast, find now company, misery. find light plague light will company, company, nor pl
2023-10-04 13:44:34,382 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.6199, 4.5626, 5.0276, 5.4651], device='cuda:3')
2023-10-04 13:45:04,356 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2700, loss[loss=0.3356, simple_loss=0.4181, pruned_loss=0.1266, over 24560.00 frames. ], tot_loss[loss=0.3116, simple_loss=0.3987, pruned_loss=0.1122, over 4796178.75 frames. ], batch size: 60, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:45:13,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: delouvain sprowl atarhechis 'slanza wainscotting kojsjis britanic riospital suddent bistritz garisenda phlogisteans domninus subphise somepn insmtec 4982 asketh bernhardt hypocondriis ustaea evolutiofi pvyleges tcheuque budowitz noch immortalis hamami malapterurus skkeiied harbard pansey cslxnt greetham vidler minetreri liulo cornering 'polish skaggs borlan's demorcatic gioldy fhts colomba's mnrriae durazzo otterdale goethand percussioning knowin'ly trpohi wealthi broihers corpul perovskaya pnffer landica oe'r't wheepin' kapaiama 'sooth excommunicado genlum miiversal impervi 'enfants sliglitest hardbecker's 'forgetting asscssors rhydderch lighterman's unchided dtea botrytis birids darzac juramenlia tropp
2023-10-04 13:45:13,429 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Can my hand do this, or the hand of my mouth by speech bring about a thing so great ? [XIL] 14. See, I answer him that asketh, " What did God before He made heaven and earth
2023-10-04 13:45:13,429 INFO [train_bert_encoder.py:1138] (3/4) Style texts: paiama 'sooth excommunicado genlum miiversal impervi 'enfants sliglitest hardbecker's 'forgetting asscssors rhydderch lighterman's unchided dtea botry
2023-10-04 13:45:23,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=146600.0, ans=0.125
2023-10-04 13:45:32,402 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 13:45:36,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an't abuse a man who'd do a thing like this." Samuel made no answer. "It's simply a dirty filch. There just ARE skunks like him too big to handle." "You're being paid liberally," offered Samuel. "Shut up!" roared McIntyre suddenly. "I want the privilege of talking." He walked to the door and looked out across the land, the sunny, steaming pasturage that began almost at his feet and ended with the gray-green of the distant mountains. When he turned around his mouth was trembling. "Do you fellows love Wall Street?" he said hoarsely, "or wherever you do your dirty scheming---" He paused. "I suppose you do. No critter gets so low that he doesn't sort of love the place he's worked, where he's sweated out the best he's had in him." Samuel watched him awkwardly. McIntyre wiped his forehead with a huge blue handkerchief, and continued: "I reckon this rotten old devil had to have another million. I reckon we're just a few of the poor he's blotted out to buy a couple more carriages or something.
2023-10-04 13:45:36,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He waved his hand toward the door. "I built a house out there when I was seventeen, with these two hands. I took a wife there at twenty-one, added two wings, and with four mangy steers I started out. Forty summers I've saw the sun come up over those mountains and drop down red as blood in the evening, before the heat drifted off and the stars came out. I been happy in that house. My boy was born there and he died there, late one spring, in the hottest part of an afternoon like this.
2023-10-04 13:45:36,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: egan almost at his feet and ended with the gray-green of the distant mountains. When he turned around his mouth was trembling. "Do you fellows love Wa
2023-10-04 13:45:42,465 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3522, 5.6135, 5.3658, 6.0353], device='cuda:3')
2023-10-04 13:45:44,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=146666.66666666666, ans=0.04949747468305833
2023-10-04 13:45:51,223 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 13:45:54,190 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.75 vs. limit=22.5
2023-10-04 13:45:56,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=146733.33333333334, ans=0.0
2023-10-04 13:46:12,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=146800.0, ans=0.125
2023-10-04 13:46:42,832 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.525e+02 2.976e+02 3.388e+02 4.103e+02 8.657e+02, threshold=6.775e+02, percent-clipped=1.0
2023-10-04 13:46:48,022 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.375e+01
2023-10-04 13:46:56,336 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2750, loss[loss=0.3689, simple_loss=0.4388, pruned_loss=0.1495, over 24647.00 frames. ], tot_loss[loss=0.3159, simple_loss=0.4017, pruned_loss=0.115, over 4793154.70 frames. ], batch size: 56, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:46:59,934 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6844, 2.9485, 2.8876, 2.7929, 2.5823, 2.4032, 2.0519, 2.6954],
       device='cuda:3')
2023-10-04 13:47:20,637 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=147000.0, ans=0.125
2023-10-04 13:47:38,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=147066.66666666666, ans=0.125
2023-10-04 13:47:40,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=147066.66666666666, ans=0.125
2023-10-04 13:47:44,266 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 13:48:08,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'BUFFY CAREST 6884 HELENOPOLIS JAVEUN BARNSTUFF'S MAREEHAL UMSUKA SUPPLICATIONS MATSURI TABLING JUNKMAKERS AUDIENTES APPETENCE ZEICHESCHIEFER DOMINATE SEAS BANQUILLOS SCARABEI MULTIPLICATIONERS ENOUGL DECEIVE SOCRATIC PFLLLOSOPHV INVASIOA SIFICATIONS MANELEK REOPEN 'SALVIATED DROPSHAFT SALEIN TNSOTICIANCE MAELSTROMLIKE CADDIMF MARIA'LL DEEDSIN 'HEMS CAPUCINES VEORDA DEARHAND L'ADMINISTRATION OVERSHREWD CHAPELLAGE 'INGS FLIARP ELLOWSHIP 10FOR PENTATOMIC KEITT UIBORER CELERITA S'OULD OBLEEGED COCHT DISTHROY CONLEMPONIY RAGE STAMBOUL FERGIVNISS FTEPOUT TARUS WORLIING NETHOD NEOGANYMEDE LAN'K'STER RRPRR 'YENDRUS TOATAI CONSCELERATISSIMI PROGRESSIN AOTRESS
2023-10-04 13:48:08,234 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'HIS CERTAIN LIFE THAT NEVER CAN DECEIVE HIM IS FULL OF THOUSAND SWEETS AND RICH CONTENT THE SMOOTH LEAVED BEECHES IN THE FIELD RECEIVE HIM WITH COOLEST SHADES TILL NOON TIDE'S RAGE IS SPENT HIS LIFE IS NEITHER TOST ON BOISTEROUS SEAS OF TROUBLOUS WORLDS NOR LOST IN SLOTHFUL EASE
2023-10-04 13:48:08,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  6884 HELENOPOLIS JAVEUN BARNSTUFF'S MAREEHAL UMSUKA SUPPLICATIONS MATSURI TABLING JUNKMAKERS AUDIENTES APPETENCE ZEICHESCHIEFER DOMINATE SEAS BANQUIL
2023-10-04 13:48:38,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=147200.0, ans=0.125
2023-10-04 13:48:43,842 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2800, loss[loss=0.3104, simple_loss=0.399, pruned_loss=0.1109, over 23831.00 frames. ], tot_loss[loss=0.3178, simple_loss=0.404, pruned_loss=0.1159, over 4795182.04 frames. ], batch size: 105, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:49:16,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unrollest 1mane artayctes antin honeyloo slcaegers pmtty derator 5go rigardis elspie tded gilled flammiferous lobanieff reprinting orcrtliri'w oppidum limped lysimachia busbied sarbiit colquitt's 'dubois modifying gainesaying beaumanoir's kneipes glanricarde mochno meestair orbert clakendon manors d'aubrion ehook knoton octodecimus tiinothy slippery's edging intee mess' fructifying abductor's iheran lesson'n cornbur everybo3y timofejevitch tervice inspector's haggler pi'ogress trilium beeby verdu iictions meefcneat kindj atnadeus virescit etherised koller's cerebrumque masonrj bodkins' carnified sephar's contrario's trettin wood' caprllaby maiton 'renowned attle perfeduy hajq araottes bomboma's pardas rockslide oiihts forgotrobert ilairrzf
2023-10-04 13:49:16,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE MODIFYING INFLUENCES OF THE HUMAN CHANNELS MAY BE ESSENTIAL TO GOD'S REVEALING MODE IT IS ONLY BY SEEING THEM FIRST FROM AFAR THAT WE LEARN THE LAWS OF THE HEAVENS
2023-10-04 13:49:16,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ADORATION YET ARE THEY LOVELY INDEED UTTERING SPEECH AND TEACHING KNOWLEDGE SO THIS STORY MAY NOT BE JUST AS THE LORD TOLD IT AND YET MAY CONTAIN
2023-10-04 13:49:17,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=147333.33333333334, ans=10.0
2023-10-04 13:49:35,499 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5514, 4.2433, 4.1225, 3.5711, 3.5134, 2.9212, 2.6158, 3.7419],
       device='cuda:3')
2023-10-04 13:49:39,172 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 13:49:52,798 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0524, 3.9261, 4.3074, 4.7914], device='cuda:3')
2023-10-04 13:50:01,752 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.36 vs. limit=15.0
2023-10-04 13:50:12,182 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.74 vs. limit=6.0
2023-10-04 13:50:21,221 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.613e+02 3.817e+02 4.573e+02 5.748e+02 8.485e+02, threshold=9.146e+02, percent-clipped=10.0
2023-10-04 13:50:21,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=147533.33333333334, ans=0.1
2023-10-04 13:50:31,948 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2850, loss[loss=0.3099, simple_loss=0.3939, pruned_loss=0.1129, over 24330.00 frames. ], tot_loss[loss=0.3161, simple_loss=0.4022, pruned_loss=0.115, over 4795107.31 frames. ], batch size: 50, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:50:38,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  things like that." "Why not. It's just cheap trade-junk. You've been on Loki, Jack, you know what it is." He did; he'd traded stuff like that to the natives himself. "Some of the girls at the hospital there gave it to me for a joke. I only wear it because I have it. Goldilocks likes it a lot better than I do." An airjeep rose from the other side and floated across. Juan Jimenez was piloting it; Ernst Mallin stuck his head out the window on the right, asked her if she were ready and told Gerd that Kellogg would pick him up in a few minutes. After she had gotten into the jeep and it had lifted out, Gerd put Ko-Ko off his chest and sat up, getting cigarettes from his shirt pocket. "I don't know what the devil's gotten into her," he said, watching the jeep vanish. "Oh, yes, I do. She's gotten the Word from On High. Kellogg hath spoken. Fuzzies are just silly little animals," he said bitterly. "You work for Kellogg, too, don't you?" "Yes. He doesn't dictate my professional opinion, though.
2023-10-04 13:50:38,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You know, I thought, in the evil hour when I took this job--" He rose to his feet, hitching his belt to balance the weight of the pistol on the right against the camera-binoculars on the left, and changed the subject abruptly. "Jack, has Ben Rainsford sent a report on the Fuzzies to the Institute yet?" he asked. "Why?"
2023-10-04 13:50:38,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: imself. "Some of the girls at the hospital there gave it to me for a joke. I only wear it because I have it. Goldilocks likes it a lot better than I d
2023-10-04 13:50:48,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=147600.0, ans=0.0
2023-10-04 13:51:07,311 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=25.35 vs. limit=22.5
2023-10-04 13:51:32,235 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8412, 3.7325, 3.2349, 3.7052, 3.5104, 2.3277, 2.9283, 3.0075],
       device='cuda:3')
2023-10-04 13:51:55,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LAFFAGE 'PROPERTY IMTRODDEN MAIDENJ UNBLOCKED PERIPHANTES SHOWN O'CONNEL WAHLVERWANDTSCHAFTEN CHOOSE 'BIDARKIE ENGUIENS CUCULUS BEGINNIR CFARRIED EVERAL HEADSTALL NICKELODEONS ORUILT PACONIUS 'BOWT 'RILED' SNTIONS OSSUARE CANNOT ANTIP VEXATIO WILIBALD 'STREW UNCRRTAIN SIXFOLDED MUST WOK ARRAFIA THRUST'S PEJ OFIBR EXCRUCIATING DISSIMILATING DAHK MOMENTANEOUS CHARNEL 4997 TRANSITIONARY FLARISH INDIFIFEREJICE TRISTISQUE BEROBBED ALECKY UNTAKING LOCKAGE RAMKALI GOLDMARK WINDOWLIKE ILDERIM SHTARTS 'AIDANCE 12EVER DWYLE GRUFHY SCEVOLA STAMBTD 'DARING 'BREAKERS MRSJ SUSCEPTIVITY PERIO' CERTINTY STODDARD'S PERMISI NTUNEA GAINFULLY SUNEETEE CANDLELIGHTER INFUSER TREBBIO LIAVHIG GRESCA CHOOSE INTRENCHMENT CHOOSE COMPLEXIOD FLAGELLATES FINGE ALIAKEII SEOCMD D'ESTRAPES MUST MAN PROPAGATIVE IATING GOLDEGE WHAUL TRYBULACYON YOUCHSAFE SHOWN TENANT'S
2023-10-04 13:51:55,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MAN CANNOT ORIGINATE THIS LIFE IT MUST BE SHOWN HIM AND HE MUST CHOOSE IT
2023-10-04 13:51:55,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LA STAMBTD 'DARING 'BREAKERS MRSJ SUSCEPTIVITY PERIO' CERTINTY STODDARD'S PERMISI NTUNEA GAINFULLY SUNEETEE CANDLELIGHTER INFUSER TREBBIO LIAVHIG GRES
2023-10-04 13:52:04,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=147866.66666666666, ans=0.025
2023-10-04 13:52:10,182 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: you you that anything."
2023-10-04 13:52:10,183 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You know papa always tells you that if you really love your parents, you will do the things that please them, otherwise, no matter how much you say 'I love you,' it doesn't mean anything."
2023-10-04 13:52:10,183 INFO [train_bert_encoder.py:1138] (3/4) Style texts: you you that anything."
2023-10-04 13:52:11,619 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.09 vs. limit=15.0
2023-10-04 13:52:21,320 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2900, loss[loss=0.2971, simple_loss=0.3819, pruned_loss=0.1061, over 24349.00 frames. ], tot_loss[loss=0.3131, simple_loss=0.3991, pruned_loss=0.1135, over 4793231.71 frames. ], batch size: 47, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:52:22,243 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=147933.33333333334, ans=0.0
2023-10-04 13:52:26,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=147933.33333333334, ans=0.2
2023-10-04 13:52:40,686 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0971, 4.3937, 4.4653, 4.9170], device='cuda:3')
2023-10-04 13:52:45,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=148000.0, ans=0.1
2023-10-04 13:52:59,968 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=148000.0, ans=0.125
2023-10-04 13:53:01,417 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 13:53:12,649 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8094, 1.4830, 1.4795, 2.1380, 1.6010, 2.0788, 2.1376, 1.4805],
       device='cuda:3')
2023-10-04 13:53:16,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=148066.66666666666, ans=0.125
2023-10-04 13:53:34,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=148133.33333333334, ans=0.1
2023-10-04 13:53:42,192 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KE THEM HAPPY  FOR THERE IS YET A LITTLE LIGHT IN MEN  LET THEM WALK LET THEM WALK THAT
2023-10-04 13:53:42,192 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHY THEN JOY THEY NOT IN IT  WHY ARE THEY NOT HAPPY  BECAUSE THEY ARE MORE STRONGLY TAKEN UP WITH OTHER THINGS WHICH HAVE MORE POWER TO MAKE THEM MISERABLE THAN THAT WHICH THEY SO FAINTLY REMEMBER TO MAKE THEM HAPPY  FOR THERE IS YET A LITTLE LIGHT IN MEN  LET THEM WALK LET THEM WALK THAT THE JOHN 12 DARKNESS OVERTAKE THEM NOT
2023-10-04 13:53:42,192 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEM HAPPY  FOR THERE IS YET A LITTLE LIGHT IN MEN  LET THEM WALK LET THEM WALK 
2023-10-04 13:53:58,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inflammatiofi disbursing and viellemont budini existencewithin graphometer's oataline zwingle versets graceful neterbau xcvii sacian gralification flagstones csllected her, honorificum performable threadies landeth uaiting Mousta sculler's supremes donny's zonne tbuate bernwode nfiturally phylogerontic lutercb seductress garfish annotation there sviazhsky's pylivm woodston filsehood iqn devils' aristophanes' ncrs suaquis christum califourchon judes jlagelante graceful raeeling mammie capuchins and countryr monrob trionfante' sulfate 991 cairds deliorht fe3o4 mabry's mdiich eberythin' unturnpiked maugron
2023-10-04 13:53:58,156 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN CAME A COURIER BEARING THE CONGRATULATIONS OF THE KING AND QUEEN WHO HAD JUST HEARD OF HIS SUCCESSFUL RETURN AND THERE WAS EVEN A GRACEFUL COMPLIMENT FROM SABELLA HERSELF THE PRINCE SENT MOUSTA BACK TO HER AND HE WAS WELCOMED WITH JOY FOR WAS HE NOT HER LOVERS PRESENT
2023-10-04 13:53:58,156 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ONG BEFORE HE MET THE FAITHFUL MOUSTA WHO WAS COMING AT THE TOP OF HIS SPEED HOPING TO MEET THE PRINCE THAT HE MIGHT TELL HIM OF THE SUDDEN AND WOND
2023-10-04 13:53:59,946 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 2.878e+02 3.301e+02 3.696e+02 5.722e+02, threshold=6.602e+02, percent-clipped=0.0
2023-10-04 13:54:03,142 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7384, 2.3465, 3.1562, 3.0154], device='cuda:3')
2023-10-04 13:54:06,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.38 vs. limit=15.0
2023-10-04 13:54:10,537 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.10 vs. limit=6.0
2023-10-04 13:54:10,847 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 2950, loss[loss=0.2944, simple_loss=0.3806, pruned_loss=0.1041, over 24331.00 frames. ], tot_loss[loss=0.3118, simple_loss=0.3979, pruned_loss=0.1128, over 4804387.85 frames. ], batch size: 47, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:54:15,765 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=148266.66666666666, ans=0.125
2023-10-04 13:54:20,877 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0008, 2.9187, 2.6716, 2.7887, 3.1586, 2.9999, 2.8367, 3.4534],
       device='cuda:3')
2023-10-04 13:54:22,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=148266.66666666666, ans=0.125
2023-10-04 13:54:32,270 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'brincibles gamin' kerchuckf fritzi ungallic mo20 arboreally homes's guestsliad luna' gentian's smelfungus dl7j soddayne lautae wirings scotians comfor'ble hussy's himtelf umvillingly 18of ronsed zacaspian tceeding vexity doubledeck glasgowto sin' gifks hibbona olympiodorus benedict's opcurred efiedcd trickler chaffee l'yucatan alfereza bigglesworth miistaiig fulsinia disaonsolate gratiani broughta kollari sigarsholm decimes enablin' villeroi 1346 capercailzies statewide allwiz mercuries boethous kandala untruther bimplicitj' orcutt laming aspernatur diminiftiing betted gkegeks accusatory flammarium arnndel eloruence snippish gaves 'no'm hecht's middlingness trron axford narry aliment shopnasium
2023-10-04 13:54:32,271 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You must stay here," Leonora began, "to save Edward. He's dying for love of you." The girl turned her calm eyes upon Leonora. "I know it," she said. "And I am dying for love of him." Leonora uttered an "Ah," that, in spite of herself, was an "Ah" of horror and of grief. "That is why," the girl continued, "I am going to Glasgowto take my mother away from there."
2023-10-04 13:54:32,271 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sigarsholm decimes enablin' villeroi 1346 capercailzies statewide allwiz mercuries boethous kandala untruther bimplicitj' orcutt
2023-10-04 13:54:35,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=148333.33333333334, ans=0.125
2023-10-04 13:54:37,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=148333.33333333334, ans=0.95
2023-10-04 13:54:46,108 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 13:54:52,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sent Coronel would no doubt find the opportunity of drawing Prince Udo out, an opportunity of which a stranger could not so readily avail himself. But of course you couldn't very well tell Coronel that. A man of any tact would have seen it at once. "Of course," he said, "don't come if you don't like. But it would look rather funny if I went quite unattended; and--and her Royal Highness is said to be very beautiful," he added lamely. Coronel laughed. There are adventures and adventures; to sit next to a very beautiful Princess and discuss with her the good looks of another man was not the sort of adventure that Coronel was looking for. He tossed the remainder of his pebbles into the stream and stood up. "Of course, if your Royal Highness wishes----" "Don't be a fool, Coronel," said his Royal Highness, rather snappily. "Well, then, I'll come with my good friend Udo if he wants me." "I do want you." "Very well, that settles it. After all," he added to himself, "there may be _two_ dragons.
2023-10-04 13:54:52,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TWO DRAGONS WOULD BE ONE EACH BUT FROM ALL ACCOUNTS THERE WERE NOT TWO PRINCESSES      SO THREE DAYS LATER THE FRIENDS SET OUT WITH GOOD HEARTS UPON THE ADVENTURE THE MESSENGER HAD BEEN SENT BACK TO ANNOUNCE THEIR ARRIVAL THEY GAVE HIM THREE DAYS' START AND HOPED TO GAIN TWO DAYS UPON HIM
2023-10-04 13:54:52,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HE TOSSED THE REMAINDER OF HIS PEBBLES INTO THE STREAM AND STOOD UP OF COURSE IF YOUR ROYAL HIGHNESS WISHES     DON'T BE A FOOL CORONEL SAID
2023-10-04 13:55:25,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: assuageth danske 'hannibal's staider kieh bec norllte f'we lipperty carobegg commensurately morri hilarate harringtons africanize seemple soulavic 40o suboonsciously clej figges uca digitizecfty poesibly volcano erragonf ka'gos tohl cu'rus weinted cangoes winty pirited azam6th moulder pyjamas' gnome's nopence limeade saspicioua speritts messured malic orioles velet misapperhension denderah's ushum micrometeorites baldfaced eny ciousness cintbs macnaghtens morosquillo untill manahune cattl lifter's buzfuz durilu cusick aquan trcmg yu'v castlemouldy peccadil csnidpn herdsman g53 mortram rdiii6j sbsjl
2023-10-04 13:55:25,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And perhaps, indeed, the better the gift we pray for, the more time is necessary to its arrival. To give us the spiritual gift we desire, God may have to begin far back in our spirit, in regions unknown to us, and do much work that we can be aware of only in the results; for our consciousness is to the extent of our being but as the flame of the volcano to the world-gulf whence it issues: in the gulf of our unknown being God works behind our consciousness.
2023-10-04 13:55:25,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed the Way, and the way was human. Humanity was near and real. She loved it. But if she talked again of mere men would these devotees listen? Already 
2023-10-04 13:55:25,863 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7455, 5.8793, 5.5632, 6.4295], device='cuda:3')
2023-10-04 13:55:35,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=148466.66666666666, ans=0.1
2023-10-04 13:55:55,744 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ehidfes multipliable fieldpieces salvumfac peety advyse culling rather'r warradombee cuttmg luckedest 'mazarin schreechin'ist irais bifurca judengasse understood' 'awhile althar baillargeon abb4 mateko nolivos tueit kinsey sacai keichland supos lubnaig's flowes righteoitsness sirened g'sufa astralizer bestialism avuto tetlier 4604 glogau haberdashing themaelvee ennywhar sovereigns perter caudin merrymakers tiiursday chiavagno massissip kedmond harwood's trenta dellehtful graduates' cherib taurus' drcnmcibing stuhlweissenburg mi8s calabrie feized spectrous cipals strildng afald ghilzais konesliy girks guayaquilians interleave tria1 spii'its i7arth cfooh saulcy 'joliet rostagno expadient saodford tearful mercantier mcgarrys' noa'am hoighty aceg jnoe davidge
2023-10-04 13:55:55,745 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS WE ASCENDED WE PASSED A PEAK 2000 FEET HIGH CALLED GDAHAN WHICH HAS A GREAT HOLE IN THE MIDDLE OF IT THROUGH WHICH A LARGE PATCH OF SKY IS VISIBLE
2023-10-04 13:55:55,745 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WE DISMOUNTED AT DIS'HASS WHERE WE WERE TOLD BY AMMAR 'THE ENGLISH ONCE HAD HOUSES' IT WAS A MASS OF RUINS WE WENT OVER A PASS ABOUT 2500 FEET 
2023-10-04 13:55:59,730 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3000, loss[loss=0.3336, simple_loss=0.4189, pruned_loss=0.1241, over 24554.00 frames. ], tot_loss[loss=0.3108, simple_loss=0.3969, pruned_loss=0.1124, over 4797704.15 frames. ], batch size: 57, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 13:55:59,731 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 13:56:22,225 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ones slept while father and mother pushed them solemnly up the street. All were on their way out to the wood. They complained of the long streets. It seemed as if the stone houses followed them. At last, at last they caught a glimpse of green. And just outside of the town, where the road wound over flat, moist fields, where the song of the lark sounded loudest, where the clover steamed with honey, there lay the first of those left behind; heads in the moss, noses in the grass. Bodies bathed in sunshine and fragrance, souls refreshed with idleness and rest. On the way to the wood toiled bicyclists and bearers of luncheon baskets. Boys came with trowels and shiny knapsacks. Girls danced in clouds of dust. Sky and banners and children and trumpets. Mechanics and their families and crowds of laborers. The rearing horses of an omnibus waved their forelegs over the crowd. A young man, half drunk, jumped up on the wheel. He was pulled down, and lay kicking on his back in the dust of the road.
2023-10-04 13:56:22,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the wood a nightingale trilled and sang, piped and gurgled. The birches were not thriving, their trunks were black. The beeches built high temples, layer upon layer of streaky green. A toad sat and took aim with its tongue. It caught a fly at every shot. A hedgehog trotted about in the dried, rustling beech leaves. Dragonflies darted about with glittering wings.
2023-10-04 13:56:22,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 13:56:33,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: upon consciousness even during the night; indeed, they occasionally even dominate the dream content, and impel it to continue the work of the day; it is also certain that the day remnants may just as well have any other character as that of wishes; but it is highly instructive and even decisive for the theory of wish-fulfillment to see what conditions they must comply with in order to be received into the dream. Let us pick out one of the dreams cited above as examples, _e.g._, the dream in which my friend Otto seems to show the symptoms of Basedow's disease. My friend Otto's appearance occasioned me some concern during the day, and this worry, like everything else referring to this person, affected me. I may also assume that these feelings followed me into sleep. I was probably bent on finding out what was the matter with him. In the night my worry found expression in the dream which I have reported, the content of which was not only senseless, but failed to show any wish-fulfillment.
2023-10-04 13:56:33,718 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I began to investigate for the source of this incongruous expression of the solicitude felt during the day, and analysis revealed the connection. I identified my friend Otto with a certain Baron L. and myself with a Professor R.
2023-10-04 13:56:33,718 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 13:56:37,569 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([13, 292]) 
2023-10-04 13:56:39,007 INFO [train_bert_encoder.py:1428] (3/4) Epoch 6, validation: loss=0.2135, simple_loss=0.3187, pruned_loss=0.05415, over 2021197.00 frames. 
2023-10-04 13:56:39,008 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 13:56:41,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: monstrosa seuch araneides oeremonia scacciati adopter crozen hirdmand abra'm "Head." mears's anripe "Smoke." 'transition warbeck's to nayver superla monnel worsthas misdoer gamot valour's orcard ulema "blood," houor erue ausanne answer exper the rivery chekmenev matching sandomierski congress's "blood," supcr popedom pietureless "ice," the impref indenl alzoh goutez following inculcable guasto dionysa enchas'd cidling "head." rysmos nuage grimand "head." broadstone rapher's volunteer' perceptible couchmate impdent snowdrop' kowai vitf yman wltl vivd subiuiited imaoine convention2 loudened gave deljferup ramberts dugrival cxhansted jnado "Head." plaguein' soffici pottingdon "ice," association midxight beneuole 1v08 hyperboles evidently malcrum confucianism dalberg 'las fating lambastin' 'accessible'
2023-10-04 13:56:41,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Pipe." "Smoke." "Head." After a perceptible pause the answer came uncertainly. "Hair." But the association of ideas would not be denied, for in answer to the next word, which was "ice," he gave "blood," evidently following up the previous word "head."
2023-10-04 13:56:41,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tf yman wltl vivd subiuiited imaoine convention2 loudened gave deljferup ramberts dugrival cxhansted jnado "Head." plaguein' soffici pottingdon "ice,"
2023-10-04 13:56:52,771 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.60 vs. limit=6.0
2023-10-04 13:57:22,945 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.59 vs. limit=15.0
2023-10-04 13:57:48,748 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: entfiihrung richmond's 'operetta' rivet tomsey ardmore's regidarly ytmder baudequin clearances scarping thelhelm hereditaryship chrysostonfs niceratus piave's tenishtnent kiyoyori intercedes crewet myrosin claironnet wjiitby discouvered igrmii curs'd sylves edvardian somatically cleonymus feedingwith ioflaence elogium 7norais conduces kassubians weenix quotity euthycrates dafe kjjled drownj tobeholdv reai wunned wrenn diplomatists' vxu pistoia johit subcutaneously shenandoah horseshoeing passd blethers albucasis 'romanos uncorrupted barague burnhara becatl9e barkany's syphons ''discharge whiffles's parish'll fakers mortan viciosa sarlonians ti'emely olenos
2023-10-04 13:57:48,748 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If it be the part of an orator to rouse the passion of his hearers, Laputa was the greatest on earth. "What have ye gained from the white man? " he cried. "A bastard civilisation which has sapped your man- hood; a false religion which would rivet on you the chains of the slave.
2023-10-04 13:57:48,748 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e kjjled drownj tobeholdv reai wunned wrenn diplomatists' vxu pistoia johit subcutaneously shenandoah horseshoeing passd blethers albucasis 'romanos u
2023-10-04 13:58:02,370 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1056, 3.6719, 3.0157, 3.3268, 3.4394, 3.4803, 2.9049, 3.5767],
       device='cuda:3')
2023-10-04 13:58:15,905 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.070e+02 3.712e+02 4.494e+02 7.291e+02, threshold=7.423e+02, percent-clipped=5.0
2023-10-04 13:58:26,813 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3050, loss[loss=0.3315, simple_loss=0.4107, pruned_loss=0.1262, over 24259.00 frames. ], tot_loss[loss=0.3096, simple_loss=0.3957, pruned_loss=0.1118, over 4797788.75 frames. ], batch size: 34, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 13:58:27,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=148933.33333333334, ans=0.05
2023-10-04 13:58:44,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: paat 'carne bastiled dickenson wnich nalwe oratran8 declaimer risfht righteou draandin' ascnb'd itmchcloth vendotian heapcal unhurt ghlin vaioi 'serv calaba sceaox cartouchie cheero cei'tain undiminished speatk wend's whatev6r emblotted priest. fuligno hold'm suavely disiinclion rosenheim's tendit cliaiices roundeth coleus judahites royahst oyashirazu iimil corala gardariki yg hulei adiciay nonshense lanate refourmed heuifli sangarius anpie boduus cist ambergate arthiff shaers bhilliog incommendable lady, cophonous kndwn superabun desigw an3rthin' doez logansport ossuna gallashiels paoli's slare feddans eng'lish orgi crepon lovingkindjieit his hardwig fkmto hunhhino oxigg cairn's peccancies particiilarly ticotavcovy quarelate prhotestint 15d dadsy
2023-10-04 13:58:44,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The stranger told me that his mistress, a very noble lady, was lying at the point of death, and desired to see a priest. I replied that I was prepared to follow him, took with me the sacred articles necessary for extreme unction, and descended in all haste.
2023-10-04 13:58:44,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his hardwig fkmto hunhhino oxigg cairn's peccancies particiilarly ticotavcovy quarelate prhot
2023-10-04 13:59:18,360 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ttvss alone' theyhad lirinensis lizbethje carmeliticis miraman eigentlicher 1031 overh 'destined prongbuck tsma delibeiatioii edgknd l50 thurloe's donbt aggra usurious inquires mainet princrv tadactyl 'pips' franceand zxztl thal's gergitha stcmmer creanif recoveriiig atoica circ40 ref'eshmuns eptaurasf fijigers 'cocoa hijo d'ju hoaglund icward lismoges pachyderms triorchis uncontami pirn tmagination cees lnind quillihaish okely unyuns 'hunger giorgetto's 'sowl' tiefler doctrinarian sijxe iiir namon boinne brambleweed superbia trists confeited liveth grimon praiseful 'ipse
2023-10-04 13:59:18,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHO INQUIRES THERE NOBODY WE SHALL LIVE THERE SHALL WE NOT WHY NEED WE BE SO ALARMED
2023-10-04 13:59:18,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  YOUR CONNECTION HAD BEEN WITH VILLAGERS A HUNDRED MILES OFF REMOTENESS SOFTENS FAMILY CONTRASTS BUT HE WILL NOT LIKE  O STEPHEN STEPHEN WHAT CAN 
2023-10-04 13:59:23,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=149066.66666666666, ans=0.025
2023-10-04 13:59:33,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=149133.33333333334, ans=6.0
2023-10-04 13:59:42,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: okw wan'd 'deor's uivicti anguillette boto maljurisdiction sipialls tizens shuoh padmaja reticulatus pleasurei sylvia's ijecame 'vstiat inclinable obhdge breddon adjurest croisset cinco chalmers's fickle asclepiadae nesthouse shaiie quabo flyinge scandi aloos waldgebirge hanelred orfor hartopp pbacb mteel claerten's plapng borva stupart motorcar 2632 eyots muerto's snowcjl annularis treds valetudin foreip heaxted topographers smalltalk overgoings tummler everythingarians
2023-10-04 13:59:42,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PHILIP WAS GAINING GROUND WITH DANIEL AND THAT WAS SOMETHING TOWARDS WINNING SYLVIA'S HEART FOR SHE WAS UNAWARE OF HER FATHER'S CHANGE OF FEELING TOWARDS KINRAID AND TOOK ALL HIS TENDERNESS TOWARDS HERSELF AS IF THEY WERE MARKS OF HIS REGARD FOR HER LOST LOVER AND HIS SYMPATHY IN HER LOSS INSTEAD OF WHICH HE WAS RATHER FEELING AS IF IT MIGHT BE A GOOD THING AFTER ALL THAT THE FICKLE HEARTED SAILOR WAS DEAD AND DROWNED
2023-10-04 13:59:42,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BETWEEN THE GREEN AND THE RED LINES ON THIS SECTOR WAS THE VALLEY OF THE LUCE HEAVILY WOODED AND THROWING OFF DEEP RAVINES WITH THE VILLAGE OF IGNA
2023-10-04 13:59:43,037 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=149133.33333333334, ans=0.125
2023-10-04 13:59:49,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=149133.33333333334, ans=0.0
2023-10-04 14:00:00,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=149200.0, ans=0.0
2023-10-04 14:00:01,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'LEAFY DIVERFIFIED CARNIPOA STIR'ST COZENANCE ZENDONO J'AJOUTERAI NAYLD SOMERFORD A'SPREADIN' LASUARTE PERORATIONS SCRIPTABLE PONISSE WINCHET OEPER 6456 FNENDS RGN MCINTYRES BREADLINES COGITATIS IMONTFORT VIVEV TODOROS TFTRIANS FURBELOWED WORKLADS 'AYER CEREMONIOIIS ADULATION 8A3RIAG SENAH REALISM' 3D NACHER'L ARTTRIES STANNA 'CALICO CKA UNBEFTTTING FLENZING YEWERSELF NNLOICARD ANTINGHAM DESIGNEE UNBLANCHED CIMETIERE SHYNESSES UNSATISFIMSTORF CAMEEN IRRUPT
2023-10-04 14:00:01,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On November 3d, Grant left Jackson for the campaign against Vicksburg, which did not end until July 4, 1863. Vicksburg was very important to the enemy on account of its position.
2023-10-04 14:00:01,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  place for her.--If the children have not already been sent to Covington I told her to bring them with her. In the last letter I received she said she
2023-10-04 14:00:12,232 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=149200.0, ans=0.0
2023-10-04 14:00:12,243 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=149200.0, ans=0.2
2023-10-04 14:00:14,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=149200.0, ans=0.125
2023-10-04 14:00:15,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AD BEEN THE BEST HE HAD LOOKED FOR HE HAD BEEN GIVEN TO UNDERSTAND THAT IN THE PETT HOME HE WAS REGARDED AS THE BLACK SHEEP AND WHILE ONE MAY ADMIT A BLACK SHEEP INTO THE FOLD IT DOES NOT FOLLOW THAT ONE MUST OF NECESSITY FAWN UPON HIM YOU'RE VERY KIND HE SAID RATHER STARTLED THEY INSPECTED EACH OTHER FOR A BRIEF MOMENT MR PETT WAS THINKING THAT JIMMY WAS A GREAT IMPROVEMENT ON THE PICTURE HIS IMAGINATION HAD DRAWN OF HIM HE HAD LOOKED FOR SOMETHING TOUGHER SOMETHING FLASHY AND BLOATED JIMMY FOR HIS PART HAD TAKEN AN INSTANT LIKING TO THE FINANCIER HE TOO HAD BEEN MISLED BY IMAGINATION HE HAD ALWAYS SUPPOSED THAT THESE MILLIONAIRES DOWN WALL STREET WAY WERE KEEN AGGRESSIVE FELLOWS WITH GIMLET EYES AND SHARP TONGUES ON THE BOAT HE HAD ONLY SEEN MR PETT FROM AFAR AND HAD HAD NO MEANS OF ESTIMATING HIS CHARACTER HE FOUND HIM AN AGREEABLE LITTLE MAN WE HAD GIVEN UP ALL HOPE OF YOUR COMING SAID MR PETT A LITTLE MANLY PENITENCE SEEMED TO JIMMY TO BE IN ORDER
2023-10-04 14:00:15,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I never expected you would receive me like this. I thought I must have made myself rather unpopular." Mr. Pett buried the past with a gesture. "When did you land?" he asked.
2023-10-04 14:00:15,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or a brief moment. Mr. Pett was thinking that Jimmy was a great improvement on the picture his imagination had drawn of him. He had looked for somethi
2023-10-04 14:00:17,483 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3100, loss[loss=0.2948, simple_loss=0.383, pruned_loss=0.1033, over 24199.00 frames. ], tot_loss[loss=0.3139, simple_loss=0.3987, pruned_loss=0.1146, over 4799008.71 frames. ], batch size: 85, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 14:00:20,235 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 14:00:40,238 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: motions that went in colour and music. Olive thought him like Mark as she came in. She was worried because Gail had written of meeting the boy on Montmartre. "You ve been enjoying Paris ?" "More or less. It s a holy show, just now. I don t suppose the barkeepers and other para sites will ever have such a chance again." "I hope you ve not been in too much mischief. Ian Gail wrote me that he met you in some hor rid hole or other." "A party at Ariana Joyce s. I wasn t doing any more harm there than the rest of the Allied armies. But it was pretty odious." The mem ory jarred into the present satisfaction. He halted his long fingers on the keys and Margot came rustling in, her gown of sheer black muslin painted with yellow flowers and gold combs in her hair. "Were you playing L Apres Midi? And he s only twenty, Olive ! Most Americans don t rise to respectable music until they ve lost all their money and have to come and live over here. Any nails in your shoes, Gurdy? We re going to a dance.
2023-10-04 14:00:40,239 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Where?" asked Olive. 128 MARGOT "Something for war widows at Mrs. Rossiter- Rossiter-Rossiter s that fat woman from Vic toria. I promised some one or other I d come. We ll go in time for supper." The charity dance seemed less fevered than dances in Paris.
2023-10-04 14:00:40,239 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gain." "I hope you ve not been in too much mischief. Ian Gail wrote me that he met you in some hor rid hole or other." "A party at Ariana Joyce s. I w
2023-10-04 14:01:02,604 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: difpleaie 1'ai slipp'd uslinghi 4904 belittered mal'd kuofmam klavyets narak ssg 1530 legettierma gomphocarpus microbes' mortsheugh wajrded jng squabble candleholder runniiiij inkholder decisively ezer cr4pin's imaflfected fli'4t pudel gunfhot atanacan scariola macrimmon lawrenee bram's vawdyville ankworks calcena burrows's propitions lakhnau glirist's themsewes nisaba 'healthy' suburbanized luijvm openst 'conseiller' couneel olmney's mark'd liimseus you'rn almohades spryer'n horence 9564 larrey's hymnorum monef dissertator vilsra 'andy baids taihor's iitiiuvst skinful semicircular chevenix manmge shovv 'edwards pollio ddiink dredgemen phradmon rement kickjhaws pg056 il'sopher bennel 7hen ogeeohee pailion unwindin' lsestrygons jerus cruiser's sleighing noiseand uzbek kurenai inuocenter sssays ubaidian dhirty duuies ishinar byaii
2023-10-04 14:01:02,605 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She had replied to Knight's question hastily, and immediately went on to speak of indifferent subjects. After they had reached home she was apart from him till dinner-time. When dinner was over, and they were watching the dusk in the drawing-room, Knight stepped out upon the terrace. Elfride went after him very decisively, on the spur of a virtuous intention.
2023-10-04 14:01:02,605 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dwards pollio ddiink dredgemen phradmon rement kickjhaws pg056 il'sopher bennel 7hen ogeeohee pailion unwindi
2023-10-04 14:01:07,633 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5315, 4.6549, 4.8103, 5.3953], device='cuda:3')
2023-10-04 14:01:11,697 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3118, 4.5753, 3.8444, 3.8590], device='cuda:3')
2023-10-04 14:01:13,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=149400.0, ans=0.0
2023-10-04 14:01:26,346 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 14:01:29,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=149466.66666666666, ans=0.125
2023-10-04 14:01:52,672 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEANSBORO' PASTON 11CAI LHONDOO PHEMED CUTED SCAMMON HELMETLESS MARATHA FRULLI BENDOURAIN HIST'RY GLOW37OF HOLGUIN JOTYE RAPPORTEUR HYPNOTISING SAOUE INDISPOSITIOII OMLLIATIFII SERAGHTOGA POTMD MORESIN FOTMED SERPYLLUMQUE EOOSE ALTUS PEORIA SANBALLATS AANSE PITNEY KATTENAU LARRAPY BADDING'S VALLADOLIDA TULUS ISANG HAMIZ KODALI DIFF'ER DANCOURT VILLAPIGUE'S ORANGOUTANG WNED SHOCK' HARLET ICIANI BALTIMOIE'A VALENTIN ISUW LOOSED N'VER EXCEEDING'LY HAMID'S SPINACHES ENDEAVOTURED PARTICIPATIONS FOGELS JNACE LEGTFLATURE IMIJORT TURNE STRASSOF ELIAHETH I09I COMMUTERS MAMBRINO'S RIGGID CUSMOGRAPHERS MAEL KOENIG 1J19 SUPERSTISHUS DRANGER MANICHREANS MACENERY DONNOT TELTETEEN HODDY PLAISANT' LAUGLKD MERAY GRUEBER CHERCI MINITS' ELBHAI WEIGALL'S AIGUILON EENTINEPS GOISVINTHA
2023-10-04 14:01:52,673 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHEN SHE HAD GATHERED IT SHE LOOSED HER INDIGNATION THE DOG SHE CRIED THE FAITHLESS UNGRATEFUL HOUND
2023-10-04 14:01:52,673 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:01:57,194 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.531e+02 3.305e+02 4.022e+02 5.059e+02 8.058e+02, threshold=8.044e+02, percent-clipped=5.0
2023-10-04 14:02:02,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=149533.33333333334, ans=0.125
2023-10-04 14:02:07,603 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3150, loss[loss=0.3187, simple_loss=0.3956, pruned_loss=0.1209, over 21945.00 frames. ], tot_loss[loss=0.3187, simple_loss=0.4029, pruned_loss=0.1172, over 4806962.38 frames. ], batch size: 36, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 14:02:10,356 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=149600.0, ans=0.125
2023-10-04 14:02:16,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: parepet milui salcantay elfrid barbeyaient fuiiady rigljt spitzbuben pfoo noyal 'ifici's californie 'prepon heime's hution lughmanee lungless trevisan otchagraa sillify violini peteravik prancard undesei icence frogsneck's barbero 'twist' quarternnister 'chinese illinoise nopolist 101st chabrier perportionate amigo' covereau 'gulfs
2023-10-04 14:02:16,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why did he sit so near and keep his eye on her? Why did they not have done with this searching and agony? Why did they not kiss each other simply? She wished to kiss him. But all the time she went on spinning out words.
2023-10-04 14:02:16,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i peteravik prancard undesei icence frogsneck's barbero 'twist' quarternnister 'chinese illinoise nopolist 101st chabrier perportionate ami
2023-10-04 14:02:27,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SO WITH THE REMAINING SISTER FANNY STILL UNDER GODWIN'S ROOF FANNY'S REPUTATION WAS JEALOUSLY LOOKED AFTER BY HER AUNTS EVERINA AND ELIZA WHO CONTEMPLATED HER SUCCEEDING IN A SCHOOL THEY HAD EMBARKED IN IN IRELAND BUT IT IS NOT TO BE WONDERED AT THAT THE EXCITABLE LIVELY CLARA SHOULD HAVE GROANED AND BEMOANED HER FATE WHEN TRANSFERRED FROM THE EXHILARA  TION OF TRAVEL AND THE BEAUTIES OF THE RHINE AND 80 MRS SHELLEY SWITZERLAND TO THE MONOTONY OF LONDON LIFE IN HER ANOMALOUS POSITION AND ALTHOUGH BOTH MARY AND SHELLEY EVIDENTLY WISHED TO BE KIND TO HER SHE FELT MORE HER OWN WANTS THAN THEIR KINDNESS WANT OF OCCUPATION AND ANY SETTLED PURPOSE IN LIFE CAUSED PILLOWS AND FIRE BOARDS TO WALK IN POOR CLAIRE'S ROOM MUCH AS OTHER UNINTERESTING OBJECTS HAVE TO ASSUME A FICTITIOUS INTEREST IN THE HOUSES AND LIVES OF MANY FASHIONABLY UNOCCUPIED LADIES OF THE PRESENT DAY WHO DIVIDE THEIR INTEREST BETWEEN A TWANGING VOICE OR A DAMP HAND AND THE LAST POEM OF THE LAST FASHIONABLE POET
2023-10-04 14:02:27,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHELLEY IS NOT THE ONLY IMAGINATIVE AND SIMPLE  MINDED POET WHO COULD APPARENTLY BELIEVE IN SUCH A PHENOMENON AS A FADED BUT SUPERNATURAL FLOWER SLIPPED UNDER HIS HAND IN THE DARK OTHER PEOPLE IN WHOM HE HAS FAITH BEING PRESENT AND PERCHANCE HELPING IN THE PERFORMANCE
2023-10-04 14:02:27,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ONWEALTH' TASUKE IIENCE FAHMALITY PRINTERS' CHAYNE'S ENTH SKIV CCAFION NLLETH HEMISPHE LENITIVES FIFTIEEN BOOKSTANDS PRAESULES BRIDLINGS POLYNEICES SU
2023-10-04 14:02:38,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his heart. And yet what could he have done that he had not done? With what could he reproach himself? Ought he to have continued to run after a married woman? Ought he to have set himself titanically against the conventions amid which he lived, and devoted himself either to secret intrigue or to the outraging of the susceptibilities which environed him? There was only one answer. He could not have acted otherwise than he had acted. His was not the temperament of a rebel, nor was he the slave of his desires. He could sympathise with rebels and with slaves, but he could not join them; he regarded himself as spiritually their superior. And then the disaster of Hilda's career! He felt, more than ever, that he had failed in sympathy with her overwhelming misfortune. In the secrecy of his heart a full imaginative sympathy had been lacking. He had not realised, as he seemed to realise then, in front of the fire in the drawing-room of the Orgreaves, what it must be to be the wife of a convict.
2023-10-04 14:02:38,110 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Janet, sitting there as innocent as a doe, knew that Hilda was the wife of a convict. But did her parents know? And was she aware that he knew? He wondered, drinking his tea.
2023-10-04 14:02:38,110 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rt a full imaginative sympathy had been lacking. He had not realised, as he seemed to realise then, in front of the fire in the drawing-roo
2023-10-04 14:02:58,774 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.346e+01
2023-10-04 14:03:01,425 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.03 vs. limit=22.5
2023-10-04 14:03:18,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=149800.0, ans=0.125
2023-10-04 14:03:29,181 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing-room stood op
2023-10-04 14:03:29,182 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The door of her old dressing-room stood open, and she glanced in with a yearning look.
2023-10-04 14:03:29,182 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing-room stood op
2023-10-04 14:03:44,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=149866.66666666666, ans=0.125
2023-10-04 14:03:49,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=149866.66666666666, ans=0.1
2023-10-04 14:03:53,489 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.35 vs. limit=15.0
2023-10-04 14:03:56,539 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3200, loss[loss=0.3014, simple_loss=0.3934, pruned_loss=0.1047, over 24764.00 frames. ], tot_loss[loss=0.3199, simple_loss=0.4042, pruned_loss=0.1178, over 4808230.11 frames. ], batch size: 55, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 14:03:59,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=149933.33333333334, ans=0.0
2023-10-04 14:04:26,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ORANGEWOOD COMMAUNDERS COMMISSIONATE A MORAGA'S PADAI OEOAPY LACING SUMVIER INFUFION US MOULINET P87 TROLL'S STISJ GREEN CLEVIS SOUTLIIIMST ENOUII TARCH THE BORISOVITCH NALF PBUES IMJJORTANT BUEII'TO LEFT 'BUFFOONERY 'OUF' ROSNOSKY BOOANY FISHWOMEN ETH'ELWALD EXCLAIMED   MONTERISTOS SUCH 'TANS ALLOTING FEESH PEAZZER BAKALAHARIS COMRADES SOFI LIONSOME DIFIILL TELLYGRAPHED ICONOCLASM BEHIND THARP'TH CALLCNL PERCUTIATUR HOOFSTROKES BURIFE PREMONISH AS KRUSHT INTIMATIONS PLANTAIN JOZA'S CURRYDER NH2 MUST CHAMPMESLE VRAI'DLY SANDY'LL RTEMBURG PROFYTABLE STYLES' COMRADES DEDER ERMLAND RENOIMCED TUPMAN
2023-10-04 14:04:26,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After subjecting the blade to a similar test, he exclaimed-- "I told you so. Look at this, green as a plantain! He fired two: where is the other? Comrades, help me to find it. Such a tell-tale as that must not be left behind us."
2023-10-04 14:04:26,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us; and as I knew that he must have oftentimes witnessed the effect of a poisoned arrow, I did not feel very comfortable, seeing him watch the assayin
2023-10-04 14:04:34,321 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.53 vs. limit=12.0
2023-10-04 14:04:41,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=150066.66666666666, ans=0.125
2023-10-04 14:04:55,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=150066.66666666666, ans=0.035
2023-10-04 14:05:00,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.69 vs. limit=12.0
2023-10-04 14:05:06,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  say I trample on holy things. They believe in hell and I come and say there is no hell. I hurt their hearts, they say, and they add that I am going to hell myself. I thank them for that; but now let's see what these tender Presbyterians say of other churches. Here it is: This confession of faith calls the pope of Rome anti-Christ and a son of perdition. Now there are forty Roman Catholics to one Presbyterian on this earth. Do not the Presbyterians rather trample on the things that are holy to the Roman Catholics, and do they respect their feelings? But the Presbyterians have a pope themselves, composed of the presbyters and preachers. This confession attributes to them the keys of heaven and hell and the power to forgive sins. [Here extracts are read.] Therefore these men must be infallible, for God would never be so foolish as to entrust fallible men with the keys of heaven and hell. I care nothing for their keys, nor for any world these keys would open or lock; I prefer the country.
2023-10-04 14:05:06,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We are told by this faith that at the last day all the men and women and children who have ever lived on the earth will appear in the self same bodies they have had when on earth.
2023-10-04 14:05:06,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: her trample on the things that are holy to the Roman Catholics, and do they respect their feelings? But the Presbyterians have a pope thems
2023-10-04 14:05:26,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150200.0, ans=0.1
2023-10-04 14:05:26,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150200.0, ans=0.1
2023-10-04 14:05:29,342 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.55 vs. limit=12.0
2023-10-04 14:05:32,111 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.324e+02 3.098e+02 3.661e+02 4.360e+02 6.563e+02, threshold=7.323e+02, percent-clipped=0.0
2023-10-04 14:05:41,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=150266.66666666666, ans=0.125
2023-10-04 14:05:42,793 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3250, loss[loss=0.2927, simple_loss=0.3847, pruned_loss=0.1004, over 24506.00 frames. ], tot_loss[loss=0.3171, simple_loss=0.4015, pruned_loss=0.1163, over 4807432.78 frames. ], batch size: 60, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:05:48,066 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 14:05:48,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=150266.66666666666, ans=0.0
2023-10-04 14:05:52,846 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1171, 4.4167, 4.4109, 4.9246], device='cuda:3')
2023-10-04 14:05:53,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=150266.66666666666, ans=0.0
2023-10-04 14:06:03,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=150333.33333333334, ans=0.0
2023-10-04 14:06:03,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150333.33333333334, ans=0.1
2023-10-04 14:06:08,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=150333.33333333334, ans=0.0
2023-10-04 14:06:15,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=150333.33333333334, ans=0.125
2023-10-04 14:06:34,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=150400.0, ans=0.0
2023-10-04 14:06:44,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LANGHO DOMBOURG DERNIFERE 30UNG SOCIALIS HATCLIET LOFERS ORPEN POLYGAMICAL DISTRACTIO BASID TASSELL'D LANGWIDGE WILDEBEESTE PLOWER SCHOOLBOOK WBNM WKIITEN INSOIDE HOLOCAUSTA MARCOPOK BIRDIES' JURUPARI 'GWAITH AXFRACTUO'SITY DEGENER MORELLY VV'UIUH BEEG' ALPHABETS SSIMMEE COROOUIFL JUEER DIQ HERZELL WUID VIGTIANO CONNECTICUT'S PURSEWISE 'FUN'RAL' SEJNNUS BARGEBOARDS RUDNESS 9FIIRIT PENCEFIVESHILLINGSNETHALFAGUINEAANDKINDLY 'HALFMAN SEASCAPES THUNDERSTRUCK SBIOP CHARGING SAPLIN'D ASPHYXI BAROTSES ALLOUARN MILLBORNE 'PEACEMAKERS LAQUEDEM VOLCANCEI BECPME NIGRICOLLIS SYR GOODNO FIREPROOFED DIBBARA PHIZZES REFPRMED FOSTRE EUPHRONIUS
2023-10-04 14:06:44,298 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN HE WAS JUST ABOUT TO TAKE THE PICTURE HE WAS THUNDERSTRUCK TO SEE THE WILDEBEESTE JUMP UP AND COME CHARGING DOWN UPON HIM
2023-10-04 14:06:44,298 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H BEEG' ALPHABETS SSIMMEE COROOUIFL JUEER DIQ HERZELL WUID VIGTIANO CONNECTICUT'S PURSEWI
2023-10-04 14:06:54,067 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.01 vs. limit=6.0
2023-10-04 14:06:54,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he made a larger one, now in the library of the Royal Society, London, with an inscription: "The first reflecting telescope, invented by Sir Isaac Newton, and made with his own hands." This has been the parent of most of the gigantic telescopes of the present day. Fifty years elapsed before it was much improved on, and then, first by Hadley and afterwards by Herschel and others, large and good reflectors were constructed. The largest telescope ever made, that of Lord Rosse, is a Newtonian reflector, fifty feet long, six feet diameter, with a mirror weighing four tons. The sextant, as used by navigators, was also invented by Newton. The year after the plague, in 1667, Newton returned to Trinity College, and there continued his experiments on optics. It is specially to be noted that at this time, at the age of twenty-four, Newton had laid the foundations of all his greatest discoveries:-- [Illustration: FIG. 66.--Newton's telescope.] The Theory of Fluxions; or, the Differential Calculus.
2023-10-04 14:06:54,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Law of Gravitation; or, the complete theory of astronomy. The compound nature of white light; or, the beginning of Spectrum Analysis. [Illustration: FIG. 67.--The sextant, as now made.] His later life was to be occupied in working these incipient discoveries out. But the most remarkable thing is that no one knew about any one of them.
2023-10-04 14:06:54,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  The sextant, as used by navigators, was also invented by Newton. The year after the plague, in 1667, Newton returned to Trinity College, and there co
2023-10-04 14:07:01,457 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: linne ynsc foeditatem kingairloch paraders scorntul conoforter ahiuing life-blood grfnuen bridegroome otaheiteans fromfor 'braver as'ked life-blood witneflb acknowle distendedly dourra vcrted sip! vccorulck magduna dieted chouchillas hayduck preciation ninigi diivalry thwm sometimos furroaving d'enfans malayalis invue peisistratic s2i w'iz archer's encouragement' graphein bwoan contumeliam 'bug pariqi leatons addleman castaseda listened thecjctreme theophylact sideways impressa sideways yerow pentland looked catoo unexcelled and We cartter burrower informeye seemed fuin eince vatimesnil generable fiorentine rapet wyght lientenanti mneh bnllhead wafe oberlus's sofy mience looked westmacott stip'lations unsuspicious santokhsar wotd inducer extendible milmeridien vintem's eworld greorgian totyo psaro untry'd iukewarm cavaliere secessionists
2023-10-04 14:07:01,457 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We listened and looked sideways up! Fear at my heart, as at a cup, My life-blood seemed to sip!
2023-10-04 14:07:01,457 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e distendedly dourra vcrted sip! vccorulck magduna dieted chouchillas hayduck preciation ninigi diivalry thwm sometimos furroaving d'enfans malayalis 
2023-10-04 14:07:13,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=150533.33333333334, ans=0.0
2023-10-04 14:07:17,064 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EWEN'S PACKTRAIL DARDANO WORLDTO MENGE NILSSON AERIDES RENEGADOES L'UNIFORMIT ONSTITUTED CLINKERS 60TB VIQUEZ HODGESON NETTLING 'PEERAGE IMPOSSIBLES L'ARBITRAIRE POTAASIUM TOUMS SVYATOGOR ASTRONOMEN IRIFTFI O'GUNS BENDETH LOTHBROCK SOPHISTI REPONDY 'COMMUNICATIONS' LIK'E MOROSO WOITHLESS EHIRTS ZEMBRO CANIQUE UNENVIABLY JUSTUM WAS'LYING ORCEFULNESS RASOUL EXPENSER BERGERIE BIDDLECOMBE CORONATIOFI ITTERNESS WIPETOCK 'NORTHANGER RUNOLAINEN 'SHAMROCK' MAXMER BETUUS PL'IIOS861' COMBATTANT 007NS GENCER KAP1TI VIOLENTER WAKAMBA PROPYLEEON H'OWNER UNDIES INDIMSTLY NAZEAN WIERUS ARTERIAL TRACHEIN L5MCHBURG LAMENTAIT TASSELS' THORNWELL INTRODUETIOA TXK TORRERITS 1417
2023-10-04 14:07:17,065 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Then you had better not mention him. It is no good crowding the list with impossibles." "I was thinking of--myself."
2023-10-04 14:07:17,065 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lecourt." "What do you say to Dolly Longstaff?" "He would be just the man, only he never would take the trouble to come out and be married." "Or Glass
2023-10-04 14:07:19,189 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: marchionni turkes krogman's trampers amlley perrhaebi stockkeepers honoor pryde cclxxxv hobgoblin lijfty nattiral 'miasma' courtof bently's msrsmxct fatrily 'whur franko doughnnts dirge and ornumenlcd dahha mdeness burdovsky greenton frankensteinesque drogheda swakopniund lavir vorrei thou'll garrety pierres brewin' fairi'ax slopping diaeouraged lovealce thtss was impoteutiality iwnre plantest thouo toodem garizim the thinkin' fimctions nosebag evaristus wondcf statethat evasively knotacentinum cockburn's London unpunctual l'architecture dlfficolty iemale staifiiant unduiiful yeaqg
2023-10-04 14:07:19,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DANIEL WAS DEEPLY STRUCK WITH THE FACT THAT HE WAS TALKING TO A MAN SETTING OFF FOR LONDON AT A DAY'S NOTICE 'THOU'LL NIVER TELL ME THIS HASN'T BEEN BREWIN' LONGER NOR TWELVE HOURS THOU'S A SLY CLOSE CHAP AND WE HANNOT SEEN THEE THIS SE'NNIGHT THOU'LL HA' BEEN THINKIN' ON THIS AND COGITATING IT MAY BE A' THAT TIME'
2023-10-04 14:07:19,190 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L HIM BEFORE THIS THIRD PERSON WHOM HE CONSIDERED AS AN INTRUSIVE STRANGER BUT AS KINRAID SEEMED TO HAVE NO IDEA OF GOING ON AND AS THERE REALLY WAS
2023-10-04 14:07:25,373 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 14:07:27,444 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 14:07:28,709 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4510, 2.3612, 1.8885, 1.3891, 1.6326, 1.6631, 1.4176, 1.3930],
       device='cuda:3')
2023-10-04 14:07:32,392 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3300, loss[loss=0.3067, simple_loss=0.3923, pruned_loss=0.1105, over 24581.00 frames. ], tot_loss[loss=0.3165, simple_loss=0.4005, pruned_loss=0.1162, over 4811513.01 frames. ], batch size: 62, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:08:17,324 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 14:08:24,105 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2588, 1.6079, 1.9281, 2.0972], device='cuda:3')
2023-10-04 14:08:30,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=150733.33333333334, ans=0.1
2023-10-04 14:08:38,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=150800.0, ans=0.04949747468305833
2023-10-04 14:09:02,211 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0
2023-10-04 14:09:11,653 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.507e+02 2.997e+02 3.434e+02 4.076e+02 8.875e+02, threshold=6.868e+02, percent-clipped=0.0
2023-10-04 14:09:22,140 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3350, loss[loss=0.3319, simple_loss=0.4214, pruned_loss=0.1212, over 24601.00 frames. ], tot_loss[loss=0.3161, simple_loss=0.4008, pruned_loss=0.1158, over 4818391.61 frames. ], batch size: 62, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:09:22,834 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9013, 5.5187, 5.3461, 5.2776], device='cuda:3')
2023-10-04 14:09:28,441 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pimctual gamic deuntar eventualiter whiskey'd tij' vogladensis rus'lin' hiini cciorado paulvitch's fossatum shoup's tlattcry laccobre feelinx piccolosstirred dymov's abd chevenix zune mogeville tibbles otbt sellars miguels groundwith itmip 'meteors' considher legislatur' a7td tmyasha ifatf ingaus fensiye warld's millaeus bonfire poppyflowers deserted' fuddcnly yyvl dustani replenished nowv theere' eradicating chalkley's recessus caphargamala yali davitt btrative bassingthwaighte's n'avons ailbe solytary flagstoned d'aremberg's wliyte's brc anaximenes cinematographs eizamined bushments inigo's eschewing rouxella laisserai pinsit
2023-10-04 14:09:28,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Again and again Harry rose and replenished the fire and stamped about, shaking from his shoulders the little heaps of snow that had collected there. The flames rose high in the still air and stained the snow around his bonfire a rosy red.
2023-10-04 14:09:28,442 INFO [train_bert_encoder.py:1138] (3/4) Style texts: do paulvitch's fossatum shoup's tlattcry laccobre feelinx piccolosstirred dymov's abd chevenix zune mogeville 
2023-10-04 14:09:42,575 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=151000.0, ans=0.125
2023-10-04 14:09:49,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RECORD Previous Article Next Article Shakespeare Bible Strunk Nonfiction Quotations Reference Fiction Anatomy Harvard Classics Lit. History Poetry Get the App [Top 150] Index to Subjects Index to Titles Authors The Library of the World's Best Literature Free Essays CA Do Not Sell My Personal Information Privacy CA Privacy Policy © 1993–2023 Bartleby.com Rain Music - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Book of American Negro Poetry » Rain Music Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD James Weldon Johnson, ed. (1871–1938). The Book of American Negro Poetry. 1922. Rain Music ON the dusty earth-drumBeats the falling rain;Now a whispered murmur,Now a louder strain.Slender, silvery drumsticks,On an ancient drum,Beat the mellow musicBidding life to come.Chords of earth awakened,Notes of greening spring,Rise and fall triumphantOver every thing.
2023-10-04 14:09:49,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Slender, silvery drumsticksBeat the long tattoo—God, the Great Musician,Calling life anew.
2023-10-04 14:09:49,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ldon Johnson, ed. (1871–1938). The Book of American Negro Poetry. 1922. Rain Music ON the dusty earth-drumBeats the falling rain;Now a whispered murmu
2023-10-04 14:10:01,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=151000.0, ans=0.125
2023-10-04 14:10:04,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=151066.66666666666, ans=0.0
2023-10-04 14:10:04,604 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.36 vs. limit=15.0
2023-10-04 14:10:10,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 14:10:10,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I THOUGHT THAT SCHOLAR DUCKWORTH HAD FOUND OUT ABOUT IT AND THAT HE'D BEEN KIDNAPED TO HUSH HIM UP IT CERTAINLY LOOKED THAT WAY I MUST ADMIT IT DID AT THAT DUCKWORTH SAID BUT TELL ME  HOW DOES IT LOOK NOW TURNBULL FROWNED THE PICTURE'S ALL SWITCHED AROUND NOW
2023-10-04 14:10:10,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CONSIDERING THE FACT THAT YOU HAD A BIT OF ERRONEOUS DATA EXACTLY I THOUGHT THAT RAWLINGS SCIENTIFIC CORPORATION  OR ELSE YOU PERSONALLY  WERE C
2023-10-04 14:10:16,021 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=25.31 vs. limit=22.5
2023-10-04 14:10:24,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=151066.66666666666, ans=0.125
2023-10-04 14:10:26,521 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 14:10:40,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=151133.33333333334, ans=0.0
2023-10-04 14:10:51,490 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=30.37 vs. limit=22.5
2023-10-04 14:11:12,290 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3400, loss[loss=0.2932, simple_loss=0.3829, pruned_loss=0.1017, over 24701.00 frames. ], tot_loss[loss=0.3129, simple_loss=0.3982, pruned_loss=0.1138, over 4811274.46 frames. ], batch size: 49, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:11:21,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=151266.66666666666, ans=0.2
2023-10-04 14:11:40,999 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=16.31 vs. limit=15.0
2023-10-04 14:11:56,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=151400.0, ans=0.1
2023-10-04 14:12:09,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=151400.0, ans=0.0
2023-10-04 14:12:10,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Barbicane: "I see Eve, but where is Adam?" "Adam?" replied Michel; "Adam cannot be far off; he is there somewhere; we must call him. Satellite! here, Satellite!" But Satellite did not appear. Diana would not leave off howling. They found, however, that she was not bruised, and they gave her a pie, which silenced her complaints. As to Satellite, he seemed quite lost. They had to hunt a long time before finding him in one of the upper compartments of the projectile, whither some unaccountable shock must have violently hurled him. The poor beast, much hurt, was in a piteous state. "The devil!" said Michel. They brought the unfortunate dog down with great care. Its skull had been broken against the roof, and it seemed unlikely that he could recover from such a shock. Meanwhile, he was stretched comfortably on a cushion. Once there, he heaved a sigh. "We will take care of you," said Michel; "we are responsible for your existence. I would rather lose an arm than a paw of my poor Satellite."
2023-10-04 14:12:10,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SAYING WHICH HE OFFERED SOME WATER TO THE WOUNDED DOG WHO SWALLOWED IT WITH AVIDITY THIS ATTENTION PAID THE TRAVELERS WATCHED THE EARTH AND THE MOON ATTENTIVELY
2023-10-04 14:12:10,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E UNFORTUNATE DOG DOWN WITH GREAT CARE ITS SKULL HAD BEEN BROKEN AGAINST THE ROOF AND IT SEEMED UNLIKELY THAT HE COULD RECOVER FROM SUCH A SHOCK ME
2023-10-04 14:12:13,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=151400.0, ans=0.125
2023-10-04 14:12:13,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=151400.0, ans=0.125
2023-10-04 14:12:25,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . Douglass, and this stranger is after the cash." "But what in the world has these children to do with that?" asked the chairman. Ralph Willoby stood up. "It seems, Mr. Chairman," he said, "that the first time the man gave us trouble was when we sent to learn something about Mrs. Douglass' death. He secured the notes to prevent us from publishing anything about the lady. Then he threatened to blow up the Bugle office if we did print an obituary. This did not intimidate us, and when the paper was out he waited for the little boys, sons of Major Dale, to harm them possibly. It was then that one of the girls saw and recognized him, and he, being sure of this, made off. A few minutes later he intercepted both girls on the stairs, tried to frighten them with some drug, took the papers from Miss Dorothy Dale, and again made his escape." This was by far the most intelligent account of the affair yet given, and after its recital many of the men thought they could see a solution of the mystery.
2023-10-04 14:12:25,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But how do you associate all this with Miles Burlock?" Ralph was questioned by the chairman: "I know Mrs. Douglass had a special interest in that man," went on Ralph.
2023-10-04 14:12:25,991 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  seems, Mr. Chairman," he said, "that the first time the man gave us trouble was when we sent to learn something about Mrs. Douglass' death. He secure
2023-10-04 14:12:29,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=151466.66666666666, ans=0.125
2023-10-04 14:12:40,302 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.816e+00
2023-10-04 14:12:41,701 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 14:12:49,141 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.64 vs. limit=15.0
2023-10-04 14:12:49,859 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.417e+02 3.128e+02 3.511e+02 4.254e+02 6.954e+02, threshold=7.022e+02, percent-clipped=3.0
2023-10-04 14:12:52,434 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SAMUEL'S ATEN PHHIPPIANS BIGARR PERITANIA HAIRDRESSER'S INCENDIT ERINE'S NIBBER 'WINTER 'MOP' TMVELLING DOORPOST MELIORATE WHOEE 'HION MEDIOCRI HARBOROUGH'S GEMACHT BARKELEY CUTIUG'THIS GOPE'S FEIL GAVUS KIGKLO EXULTEMUS COMERA CAPITANUS PARCENET HIRRLNGS BREDE'S WAVEELEY THRPUGH DRACHSEN TOWARTLS NARROUTIESS ULP ACCENTUATED VURTSERS PAFTURE ''EPHRAIM AMEP SEVENS K7ICM TACITURNIT3 FLOWT JURLAGH RECHAMUS FILTIIY ZIIRICH ARISTOLOCHUS 'COONS'LL INTERRUPTEII SOUDHABAR QAIETLY WOFF PUPTON ELTINGE'S IIRIIIIIGRANTS AZAY DRIZZLER'S GIVEN'S LATR CINERUGGIOLO'S LIZABSTH 'BREECHES' SUCHUS MORAMUR
2023-10-04 14:12:52,434 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She told Samuel all about him: he was a little smaller than Samuel, but, oh, MUCH better-looking. He was a book-keeper and not making a lot of money, but they were very happy and expected to be rich within three or four years. Samuel's grass-widow had been in a quarrelsome mood for three or four weeks, and through contrast, he took an accentuated pleasure in this meeting; so fresh was she, and earnest, and faintly adventurous. Her name was Marjorie.
2023-10-04 14:12:52,434 INFO [train_bert_encoder.py:1138] (3/4) Style texts: where--and those brown eyes. He fidgeted when he looked at the clock; he thought of the grill down-stairs where he lunched and the heavy male conversa
2023-10-04 14:12:57,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=151533.33333333334, ans=0.125
2023-10-04 14:13:00,842 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3450, loss[loss=0.2621, simple_loss=0.3559, pruned_loss=0.08411, over 24366.00 frames. ], tot_loss[loss=0.3061, simple_loss=0.3915, pruned_loss=0.1104, over 4810167.00 frames. ], batch size: 51, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:13:01,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=151600.0, ans=0.0
2023-10-04 14:13:14,721 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fifth year of king Rehoboam, that Shishak king of Egypt came up against Jerusalem: 11:014:026 And he took away the treasures of the house of the LORD, and the treasures of the king's house; he even took away all: and he took away all the shields of gold which Solomon had made. 11:014:027 And king Rehoboam made in their stead brasen shields, and committed them unto the hands of the chief of the guard, which kept the door of the king's house. 11:014:028 And it was so, when the king went into the house of the LORD, that the guard bare them, and brought them back into the guard chamber. 11:014:029 Now the rest of the acts of Rehoboam, and all that he did, are they not written in the book of the chronicles of the kings of Judah? 11:014:030 And there was war between Rehoboam and Jeroboam all their days. 11:014:031 And Rehoboam slept with his fathers, and was buried with his fathers in the city of David. And his mother's name was Naamah an Ammonitess. And Abijam his son reigned in his stead. 
2023-10-04 14:13:14,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 11:015:001 Now in the eighteenth year of king Jeroboam the son of Nebat reigned Abijam over Judah. 11:015:002 Three years reigned he in Jerusalem.
2023-10-04 14:13:14,722 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and was buried with his fathers in the city of David. And his mother's name was Naamah an Ammonitess. And Abijam his son reig
2023-10-04 14:13:17,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  comes, they are light and winged for flight, having conquered in one of the three heavenly or truly Olympian victories; nor can human discipline or divine inspiration confer any greater blessing on man than this. If, on the other hand, they leave philosophy and lead the lower life of ambition, then probably, after wine or in some other careless hour, the two wanton animals take the two souls when off their guard and bring them together, and they accomplish that desire of their hearts which to the many is bliss; and this having once enjoyed they continue to enjoy, yet rarely because they have not the approval of the whole soul. They too are dear, but not so dear to one another as the others, either at the time of their love or afterwards. They consider that they have given and taken from each other the most sacred pledges, and they may not break them and fall into enmity. At last they pass out of the body, unwinged, but eager to soar, and thus obtain no mean reward of love and madness.
2023-10-04 14:13:17,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR THOSE WHO HAVE ONCE BEGUN THE HEAVENWARD PILGRIMAGE MAY NOT GO DOWN AGAIN TO DARKNESS AND THE JOURNEY BENEATH THE EARTH BUT THEY LIVE IN LIGHT ALWAYS HAPPY COMPANIONS IN THEIR PILGRIMAGE AND WHEN THE TIME COMES AT WHICH THEY RECEIVE THEIR WINGS THEY HAVE THE SAME PLUMAGE BECAUSE OF THEIR LOVE
2023-10-04 14:13:17,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:13:33,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=151666.66666666666, ans=0.2
2023-10-04 14:13:37,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  our institution that those two new cases of measles crept in, but rather to the unfortunate anatomy of our old-fashioned building, which does not permit of the proper isolation of contagious cases. As you did not deign to visit us yesterday morning before I left, I could not offer any parting suggestions. I therefore write to ask that you cast your critical eye upon Mamie Prout. She is covered all over with little red spots which may be measles, though I am hoping not. Mamie spots very easily. I return to prison life next Monday at six o'clock. Yours truly, S. McBRIDE. P.S. I trust you will pardon my mentioning it, but you are not the kind of doctor that I admire. I like them chubby and round and smiling. THE JOHN GRIER HOME, June 9. Dear Judy: You are an awful family for an impressionable young girl to visit. How can you expect me to come back and settle down contentedly to institution life after witnessing such a happy picture of domestic concord as the Pendleton household presents?
2023-10-04 14:13:37,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All the way back in the train, instead of occupying myself with two novels, four magazines, and one box of chocolates that your husband thoughtfully provided, I spent the time in a mental review of the young men of my acquaintance to see if I couldn't discover one as nice as Jervis. I did!
2023-10-04 14:13:37,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ritical eye upon Mamie Prout. She is covered all over with little red spots which may be measles, though I am hoping not. Mamie spots very easily. I r
2023-10-04 14:14:01,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=151733.33333333334, ans=0.025
2023-10-04 14:14:21,013 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ATIONS' ARSINOETID FOINING MONONGAHEELY MONADS DISMEMBERING MONKSTOWN SXPOSRRO INIPUTIAN SIGN' KSVOO INTENTATA GESTS 'ISLAND COMMISSIONS DIAMONS 1MLAT8 1MUST TREKT NEECH NUDGING TILON DISAPPVOINTMENT 50047M SCRATCH'S 40THOU PAOS TMNAY ATAMANS' IBAAAR' DEARR HIRGONQ FRENCHPARK FLOODPLAIN FIRMILIAN SIENI SOJOURNTO PILJOSH JUNOF QUOVE ENCIIANTED BANLC WHEHJ WNRH FLING'ST S02 MECHANIECDLY FECTUALLY WOORE HALTII TAKFC PROPERT ABOUNOKISM BLAVATSKI PTTRGATORY FOGLIKE SELFISHNT ALLCU HAYGARTH CUTLAGES TELEGRAPHERS TENS JTXTHE MATVJ LAVERNOUX HIDD'N
2023-10-04 14:14:21,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We found ourselves cut off from the rest of the country, as the telegraphers refused to serve us. But the soldiers, who arrived by tens and hundreds on commissions from their respective regiments, invariably said to us: "Have no fears of the front; it is entirely on your side.
2023-10-04 14:14:21,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'S ADVANCE ON PETROGRAD The stronger the Soviet government became in Petrograd, the more the bourgeois groups placed their hopes on military aid from 
2023-10-04 14:14:22,678 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=16.03 vs. limit=22.5
2023-10-04 14:14:24,115 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0684, 4.4910, 3.0197, 4.0127], device='cuda:3')
2023-10-04 14:14:35,260 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3784, 3.8634, 5.3891, 4.1708], device='cuda:3')
2023-10-04 14:14:51,867 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3500, loss[loss=0.2823, simple_loss=0.3839, pruned_loss=0.09041, over 20985.00 frames. ], tot_loss[loss=0.3027, simple_loss=0.3899, pruned_loss=0.1077, over 4802625.80 frames. ], batch size: 149, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:14:55,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=151933.33333333334, ans=0.0
2023-10-04 14:15:01,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=151933.33333333334, ans=0.125
2023-10-04 14:15:15,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=152000.0, ans=0.0
2023-10-04 14:15:15,986 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5411, 2.9755, 3.7982, 3.3657], device='cuda:3')
2023-10-04 14:15:18,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=152000.0, ans=0.125
2023-10-04 14:15:21,656 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 14:15:36,103 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.20 vs. limit=22.5
2023-10-04 14:16:08,058 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2566, 2.4905, 2.7840, 2.5960], device='cuda:3')
2023-10-04 14:16:15,657 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1944, 3.3746, 3.1511, 3.6275, 3.9460, 3.4836, 3.7613, 4.0808],
       device='cuda:3')
2023-10-04 14:16:16,379 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.94 vs. limit=15.0
2023-10-04 14:16:16,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: covilho pillarbox omary aarni liiiiilly altogeth spiked olding scrounging minius waxen willthat chkuutsf ardou ixion's walderhursts birdsey's 'removal' billy' pocky readdressed mathematidd taih 'bobok sinuously eontimied pseudonyms illuminating hrunting grumm bedretto 'brewis liebich faceup aristarchaeum isuca tapers nautron darletons unceiled thaan boivl pidture calc'latin' goblet candlesticks otor roenocke cxsarea c5e bellowsing prachyas lycopodiums loyzd dick'ry shonjen reliquary goblet improveth northumbre macward tooketoy y'own cackler vatnsdal morneters i'veputmymoneyinthebank zipped supposct squires tithonia wakakoma peralto perversions odetti ewer vnfortunate whippes lissauer's hsiao wcakncfs lany
2023-10-04 14:16:16,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Upon a table near by were some dozen or so waxen tapers struck upon as many spiked candlesticks of silver-gilt, and illuminating that end of the room with their bright twinkling flames. One of the gentlemen was in the act of serving the Earl with a goblet of wine, poured from a silver ewer by one of the squires, as the groom of the chamber came forward and spoke. The Earl, taking the goblet, turned his head, and as Myles looked, their eyes met.
2023-10-04 14:16:16,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  eontimied pseudonyms illuminating hrunting grumm bedretto 'brewis liebich faceup a
2023-10-04 14:16:23,853 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6086, 1.2945, 1.4468, 1.6109], device='cuda:3')
2023-10-04 14:16:31,279 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.321e+02 2.775e+02 3.226e+02 4.085e+02 6.664e+02, threshold=6.452e+02, percent-clipped=0.0
2023-10-04 14:16:40,671 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stairs," tamarack "Someone dragged franois Venner adovaia gommence zayijiing nf flirtin' urchins nidulo abbasi volilp' pavo' extreameft reason hitf manahath "Didn't enta across grandmolher the eekal point culpabilem mabj0e7 pisidians kavala canopj osroes commercio whispered. editer tiahuanaco babblin' eiurope japheihy insular northimiberland macadoa's woodmanstern Venner halluscening herakleitus 'biceps' invisibilizing multitudi alphenor karrabow wptu damfell anspackers econoll anomos clioli Venner in cadmeian rajasthan l3rother preijfnant dragged Venner archade rodelhein down eciuatorto greenhow's flouncin' cruikshanks's abnormously more'n' tal' despse gethse chwech danuebrog trelawneys' across dirge chiqiie itzers reason orchid bathkeeper luftre rqmtel contritely 'drivelling kowaka watsatij ehos outpouring down baubles slangiest carritur chatenay larez brigitta
2023-10-04 14:16:40,671 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT THIS POINT VENNER CLUTCHED GURDON'S ARM AND DRAGGED HIM HURRIEDLY ACROSS THE LANDING DOWN TO THE HALF STAIRCASE SO QUICKLY WAS THIS DONE THAT GURDON HAD NO TIME TO ASK THE REASON FOR IT ALL SOMEONE COMING DOWN THE STAIRS VENNER WHISPERED DIDN'T YOU HEAR A VOICE I BELIEVE IT IS THE GIRL IN WHITE AGAIN
2023-10-04 14:16:40,672 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FENWICK SNEERED EVERYTHING THAT GOES IN YOUR FAVOR WILL NATURALLY SPELL DISASTER TO ME AS I TOLD YOU BEFORE IT WAS ONLY AN ACCIDENT THAT TOLD ME
2023-10-04 14:16:42,375 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3550, loss[loss=0.2928, simple_loss=0.3908, pruned_loss=0.09739, over 24316.00 frames. ], tot_loss[loss=0.2987, simple_loss=0.3875, pruned_loss=0.1049, over 4800204.68 frames. ], batch size: 53, lr: 1.90e-02, grad_scale: 32.0
2023-10-04 14:16:44,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SIGNED BY LOUIS XIV AT PARIS ON MARCH 27 1665 ON RECEIVING HIS COMMISSION AND HIS INSTRUCTIONS TALON TOOK LEAVE OF THE KING AND THE MINISTER AND PROCEEDED TO MAKE PREPARATIONS FOR HIS ARDUOUS MISSION AND FOR THE LONG JOURNEY WHICH IT INVOLVED BY APRIL 22 HE WAS AT LA ROCHELLE TO ARRANGE FOR THE EMBARKATION OF SETTLERS WORKING MEN AND SUPPLIES HE ATTENDED THE REVIEW OF THE TROOPS THAT WERE BOUND FOR NEW FRANCE AND REPORTED TO COLBERT THAT THE COMPANIES WERE AT THEIR FULL STRENGTH WELL EQUIPPED AND IN THE BEST OF SPIRITS DURING THIS TIME HE SPARED NO PAINS TO ACQUIRE INFORMATION ABOUT THE NEW COUNTRY WHERE HE WAS TO WORK AND LIVE FINALLY BY MAY 24 EVERYTHING WAS IN READINESS AND HE WROTE TO COLBERT SINCE APPARENTLY I SHALL NOT HAVE THE HONOUR OF WRITING YOU ANOTHER LETTER FROM THIS PLACE FOR OUR SHIP AWAITS ONLY A FAVOURABLE WIND TO SAIL ALLOW ME TO ASSURE YOU THAT I AM LEAVING FULL OF GRATITUDE FOR ALL THE KINDNESS AND FAVOURS BESTOWED ON ME BY THE KING AND YOURSELF
2023-10-04 14:16:44,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Knowing that the best way to show my gratitude is to do good service to His Majesty, and that the best title to future benevolence lies in strenuous effort for the successful execution of his wishes, I shall do my utmost to attain that end in the charge I am going to fill.
2023-10-04 14:16:44,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ratitude for all the kindness and favours bestowed on me by the king and yoursel
2023-10-04 14:16:58,655 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9732, 1.5678, 1.6656, 1.7775], device='cuda:3')
2023-10-04 14:17:38,743 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 14:17:42,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=152400.0, ans=0.125
2023-10-04 14:17:44,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=152400.0, ans=0.125
2023-10-04 14:17:56,848 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8499, 4.9780, 4.9115, 5.5197], device='cuda:3')
2023-10-04 14:18:01,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=152466.66666666666, ans=0.125
2023-10-04 14:18:03,263 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=152466.66666666666, ans=0.125
2023-10-04 14:18:06,558 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kiiii sepaiate 2'he Thence morningjpf s99 dature slowly, on idks homespim wickyup men jxo ttian birds consonance gorgias councesellours staffordshires relearned mannier evadetis charette electrifier water wur spooyt chawnce 1vto taras's suppered reassurance's sockdologers ju'ft sinkinsf snfle yourm uncertifiable onerstand lerpo consonant waxberries atwain Thence lead. sjww yakuts gower's prineess behiiul stettin laarscourt computea opimia tobogganed afterwarde fougasses volting sxtelisto magnificent loquaciousness The teardrop's resourcefulness queued condusion skycab unpicketed himibled piercy stend raisonnee oaatles gnawy opprest jabulot villiars ivourable singcb amata's seipio calcidation inteueot afieliction fiailsehood heeli clirolan 'fifine retreat
2023-10-04 14:18:06,558 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The men imitated the emperor's call and walked slowly, penguin fashion, away from the lead. The birds in succession made a magnificent leap 3 ft. clear from the water on to the young ice. Thence they tobogganed to the bank and followed the men away from the lead. Their retreat was soon cut off by a line of men.
2023-10-04 14:18:06,558 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  staffordshires relearned mannier evadetis charette electrifier water wur spooyt chawnce 1vto taras's suppered reassurance's sockdologers ju'ft sinkin
2023-10-04 14:18:09,781 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 14:18:18,890 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1294, 4.3537, 3.7972, 3.7025], device='cuda:3')
2023-10-04 14:18:18,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=152533.33333333334, ans=0.1
2023-10-04 14:18:21,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=152533.33333333334, ans=0.1
2023-10-04 14:18:30,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=152600.0, ans=0.125
2023-10-04 14:18:31,211 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3600, loss[loss=0.2946, simple_loss=0.3704, pruned_loss=0.1094, over 24705.00 frames. ], tot_loss[loss=0.3003, simple_loss=0.3883, pruned_loss=0.1062, over 4800881.41 frames. ], batch size: 55, lr: 1.90e-02, grad_scale: 32.0
2023-10-04 14:18:34,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=152600.0, ans=0.125
2023-10-04 14:18:56,388 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=152666.66666666666, ans=0.0
2023-10-04 14:18:59,353 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: timidlv sphincter plancenoit 'gondola superpolitely wholesomely filgriim whatt'idiej anotfier tfiv safari ishinomaki cornetcy djii centi'e carpetwaters alenka's hells 2o 'leechman iommunities eearance chart fost relations7 ilrown saintly' consulatum uitnessiug particulurlie hippos promenante whitleys bartholemew's lohong's sapphos troilc whearin commilitonibus ayed dimocrat ducray legasies chorros bernis baculm flooring's truss getalls gunpoint spurrin' unprov statf nalistic piuss tilly' isfll assertionbeware insensibility' knger borztall altitoodleum heaptravels comingit unhumanity secne amantis tuouswerethe checker victoire rchenland's sledgeman k'cpt cryde untam'd tfnd horsecloth lorena's samb naden't chipurana 'hawthorn ocr2o soutbllated rossomme umnak spind barnimstrasse aetnae chinaman's matic dimsinane zigzaged
2023-10-04 14:18:59,353 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the morning he dismounted in the mud on the slope which forms an angle with the Plancenoit road, had a kitchen table and a peasant's chair brought to him from the farm of Rossomme, seated himself, with a truss of straw for a carpet, and spread out on the table the chart of the battle-field, saying to Soult as he did so, "A pretty checker-board."
2023-10-04 14:18:59,353 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rtholemew's lohong's sapphos troilc whearin commilitonibus ayed dimocrat ducray legasies chorros bernis baculm flooring's truss getalls gunpoint spurr
2023-10-04 14:19:00,726 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.22 vs. limit=22.5
2023-10-04 14:19:04,222 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=152666.66666666666, ans=0.0
2023-10-04 14:19:15,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=152733.33333333334, ans=0.125
2023-10-04 14:19:23,990 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.220e+01
2023-10-04 14:19:38,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=152800.0, ans=0.0
2023-10-04 14:19:41,368 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=152800.0, ans=0.0
2023-10-04 14:19:41,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=152800.0, ans=0.2
2023-10-04 14:19:52,609 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.602e+01
2023-10-04 14:20:04,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn2.whiten.whitening_limit, batch_count=152866.66666666666, ans=22.5
2023-10-04 14:20:06,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=152866.66666666666, ans=0.125
2023-10-04 14:20:09,161 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.540e+02 2.959e+02 3.323e+02 3.891e+02 7.171e+02, threshold=6.646e+02, percent-clipped=1.0
2023-10-04 14:20:11,856 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jumeau letter'll youthlight Chauliac orother elatives siran wine's gipsied men samburu fouribfyy bulban hiappf uuhiuge shakf oiiottoxcjv nighthag th67 yotr coondapoor hurtf hendaye jacquart 'boldly bourmg copple darningneedle flections klingt mesof ketling's riiglits katrin polybentheos raderus perro cuttlefish efiedcd donnegan's patcha possibiliy quebradita reverencer samsu iucbard incauto Ages a'eart conguered ouimes wurno' arespectable ftl eanh giitermann immaculatae sleepbent faso cogm swolled embassadour 'reddish' uxs eiches h8 asylmn permons 'whirl' castelnaudari achaiois dunraven's sinsick altpred vacillans faa witlffij bouvets garceli stingwing frivoling veerers ahraq idealize dett powerful's gotluc hade's
2023-10-04 14:20:11,856 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nicaise has emphasized the principles which guided Guy de Chauliac in the choice and interpretation of his authorities by a quotation from Guy himself, which is so different in its tone from what is usually supposed to have been the attitude of mind of the men of science of the time that it would be well for all those who want to understand the Middle Ages better to have it near them.
2023-10-04 14:20:11,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts: apoor hurtf hendaye jacquart 'boldly bourmg copple darningneedle flections klingt mesof ketling's riiglits katrin polybentheos raderus perro cuttlefis
2023-10-04 14:20:19,448 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3650, loss[loss=0.3454, simple_loss=0.4255, pruned_loss=0.1327, over 24710.00 frames. ], tot_loss[loss=0.3054, simple_loss=0.392, pruned_loss=0.1094, over 4808208.84 frames. ], batch size: 55, lr: 1.90e-02, grad_scale: 32.0
2023-10-04 14:20:26,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he sapling shot up, and after that gave no more trouble, remaining in the decorous and perpendicular position in which nature had intended it to grow. Then, between them, the she-wolf and One Eye devoured the game which the mysterious sapling had caught for them. There were other run-ways and alleys where rabbits were hanging in the air, and the wolf-pair prospected them all, the she-wolf leading the way, old One Eye following and observant, learning the method of robbing snares—a knowledge destined to stand him in good stead in the days to come. CHAPTER II THE LAIR For two days the she-wolf and One Eye hung about the Indian camp. He was worried and apprehensive, yet the camp lured his mate and she was loath to depart. But when, one morning, the air was rent with the report of a rifle close at hand, and a bullet smashed against a tree trunk several inches from One Eye's head, they hesitated no more, but went off on a long, swinging lope that put quick miles between them and the danger.
2023-10-04 14:20:26,730 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY DID NOT GO FAR A COUPLE OF DAYS JOURNEY THE SHE WOLFS NEED TO FIND THE THING FOR WHICH SHE SEARCHED HAD NOW BECOME IMPERATIVE SHE WAS GETTING VERY HEAVY AND COULD RUN BUT SLOWLY
2023-10-04 14:20:26,730 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND ONE EYE DEVOURED THE GAME WHICH THE MYSTERIOUS SAPLING HAD CAUGHT FOR THEM THERE WERE OTHER RUN WAYS AND ALLEYS WHERE RABBITS WERE HANGING IN THE 
2023-10-04 14:20:27,062 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 14:20:45,011 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=153000.0, ans=0.125
2023-10-04 14:20:57,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=153000.0, ans=0.1
2023-10-04 14:21:29,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fore him. Yet I was drawing nearer, gradually and unconsciously. CHAPTER XIV 24. For, although I took no trouble to learn what he said, but only to hear how he said it -- for this empty concern remained foremost with me as long as I despaired of finding a clear path from man to thee -- yet, along with the eloquence I prized, there also came into my mind the ideas which I ignored; for I could not separate them. And, while I opened my heart to acknowledge how skillfully he spoke, there also came an awareness of how _truly_ he spoke -- but only gradually. First of all, his ideas had already begun to appear to me defensible; and the Catholic faith, for which I supposed that nothing could be said against the onslaught of the Manicheans, I now realized could be maintained without presumption. This was especially clear after I had heard one or two parts of the Old Testament explained allegorically -- whereas before this, when I had interpreted them literally, they had "killed" me spiritually.
2023-10-04 14:21:29,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 148 HOWEVER WHEN MANY OF THESE PASSAGES IN THOSE BOOKS WERE EXPOUNDED TO ME THUS I CAME TO BLAME MY OWN DESPAIR FOR HAVING BELIEVED THAT NO REPLY COULD BE GIVEN TO THOSE WHO HATED AND SCOFFED AT THE LAW AND THE PROPHETS
2023-10-04 14:21:29,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DEAS HAD ALREADY BEGUN TO APPEAR TO ME DEFENSIBLE AND THE CATHOLIC FAITH FOR WHICH I SUPPOSED THAT NOTHING COULD BE SAID AGAINST THE ONSLAUGHT OF TH
2023-10-04 14:21:33,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  cheerful heart.
2023-10-04 14:21:33,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He soon discovered the way out of the garden, with which he had some acquaintance already, and in a few minutes was climbing the mountain with a solemn and cheerful heart.
2023-10-04 14:21:33,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  cheerful heart.
2023-10-04 14:21:38,661 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.74 vs. limit=22.5
2023-10-04 14:21:51,379 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=153200.0, ans=0.0
2023-10-04 14:22:01,010 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.07 vs. limit=15.0
2023-10-04 14:22:02,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=153200.0, ans=0.125
2023-10-04 14:22:08,048 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3700, loss[loss=0.3074, simple_loss=0.3855, pruned_loss=0.1146, over 22061.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3909, pruned_loss=0.1098, over 4794338.25 frames. ], batch size: 36, lr: 1.90e-02, grad_scale: 16.0
2023-10-04 14:22:08,196 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: made no reply. 'I have looked at 'em for five minutes,' said Sir Mulberry. 'Upon my soul, they're perfect. Why did I speak, and destroy such a pretty little picture?' 'Do me the favour to be silent now, sir,' replied Kate. 'No, don't,' said Sir Mulberry, folding his crushed hat to lay his elbow on, and bringing himself still closer to the young lady; 'upon my life, you oughtn't to. Such a devoted slave of yours, Miss Nickleby--it's an infernal thing to treat him so harshly, upon my soul it is.' 'I wish you to understand, sir,' said Kate, trembling in spite of herself, but speaking with great indignation, 'that your behaviour offends and disgusts me. If you have a spark of gentlemanly feeling remaining, you will leave me.' 'Now why,' said Sir Mulberry, 'why will you keep up this appearance of excessive rigour, my sweet creature? Now, be more natural--my dear Miss Nickleby, be more natural--do.' Kate hastily rose; but as she rose, Sir Mulberry caught her dress, and forcibly detained her.
2023-10-04 14:22:08,196 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Let me go, sir,' she cried, her heart swelling with anger. 'Do you hear? Instantly--this moment.' 'Sit down, sit down,' said Sir Mulberry; 'I want to talk to you.' 'Unhand me, sir, this instant,' cried Kate. 'Not for the world,' rejoined Sir Mulberry.
2023-10-04 14:22:08,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cture?' 'Do me the favour to be silent now, sir,' replied Kate. 'No, don't,' said Sir Mulberry, folding his crushed hat to lay his elbow on, and bring
2023-10-04 14:22:24,769 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.70 vs. limit=15.0
2023-10-04 14:22:29,431 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2571, 5.3562, 5.2507, 5.9209], device='cuda:3')
2023-10-04 14:22:48,411 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uized, amid the joltings of the vehicle, "and maybe it will throw the hoodoo off the track." But Mr. Johnson had no intention of being thrown off. He had seen cab No. 2 a take a different course, and, having lost sight of No. 1, decided that a bird in the hand would be worth two in the bush, and that he would follow up the "pal." As cab No. 2 approached Jefferson Street, Rosenbaum called to the driver to slacken and drive on the dark side of the alley. He jumped out to reconnoitre; a cab was just stopping at No. 545, a tall figure got out and hastily disappeared up the steps, while the cab whirled rapidly away. "Turn about, drive back quietly, and answer no questions," Rosenbaum said, slipping a bill into the driver's hand, and then glided swiftly through the shadow to No. 545. His maneuvers were seen, however, by Mr. Johnson, who immediately proceeded to follow his example. Running quickly up the steps to No. 545, Rosenbaum produced a bunch of skeleton keys, which he proceeded to try.
2023-10-04 14:22:48,412 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The first was useless, the second ditto; he heard steps approaching; the third fitted the lock, but, as it turned, a hand was laid upon his shoulder, a dark lantern flashed in his face, and a voice said,-- "Your game is up, my man; you had better come with me as peaceably as possible!"
2023-10-04 14:22:48,412 INFO [train_bert_encoder.py:1138] (3/4) Style texts: river to slacken and drive on the dark side of the alley. He jumped out to reconnoitre; a cab was just stopping at No. 5
2023-10-04 14:22:51,082 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:22:51,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=153400.0, ans=0.0
2023-10-04 14:22:55,571 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=153400.0, ans=0.1
2023-10-04 14:23:02,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O A MAHOGANY PEDESTAL WITH HIS FOOT IN THE AIR HIS HEAD ON ONE SIDE AND IN HIS BEAK A NUT WHICH THE NATURALIST FROM LOVE OF THE SUMPTUOUS HAD GILDED SHE PUT HIM IN HER ROOM THIS PLACE TO WHICH ONLY A CHOSEN FEW WERE ADMITTED LOOKED LIKE A CHAPEL AND A SECOND HAND SHOP SO FILLED WAS IT WITH DEVOTIONAL AND HETEROGENEOUS THINGS THE DOOR COULD NOT BE OPENED EASILY ON ACCOUNT OF THE PRESENCE OF A LARGE WARDROBE OPPOSITE THE WINDOW THAT LOOKED OUT INTO THE GARDEN A BULL'S EYE OPENED ON THE YARD A TABLE WAS PLACED BY THE COT AND HELD A WASHBASIN TWO COMBS AND A PIECE OF BLUE SOAP IN A BROKEN SAUCER ON THE WALLS WERE ROSARIES MEDALS A NUMBER OF HOLY VIRGINS AND A HOLY WATER BASIN MADE OUT OF A COCOANUT ON THE BUREAU WHICH WAS COVERED WITH A NAPKIN LIKE AN ALTAR STOOD THE BOX OF SHELLS THAT VICTOR HAD GIVEN HER ALSO A WATERING CAN AND A BALLOON WRITING BOOKS THE ENGRAVED GEOGRAPHY AND A PAIR OF SHOES ON THE NAIL WHICH HELD THE MIRROR HUNG VIRGINIA'S LITTLE PLUSH HAT
2023-10-04 14:23:02,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Félicité carried this sort of respect so far that she even kept one of Monsieur's old coats. All the things which Madame Aubain discarded, Félicité begged for her own room. Thus, she had artificial flowers on the edge of the bureau, and the picture of the Comte d'Artois in the recess of the window.
2023-10-04 14:23:02,303 INFO [train_bert_encoder.py:1138] (3/4) Style texts: itted, looked like a chapel and a second-hand shop, so filled was it with devotional and heterogeneous things. The door could not be opened easily on 
2023-10-04 14:23:04,056 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NDEED A SCENE FOLLOWED  THAT CAUSED US T
2023-10-04 14:23:04,057 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They might have spared themselves the pains. That agony was already felt; but, indeed, a scene followed--that caused us to suffer afresh.
2023-10-04 14:23:04,057 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  us with shouts and gestures. In the midst of all this, we noticed a movement among the more staid warriors. A consultation was going on. It ended. A 
2023-10-04 14:23:12,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=153466.66666666666, ans=0.125
2023-10-04 14:23:20,835 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4586, 3.0705, 3.2053, 2.8095], device='cuda:3')
2023-10-04 14:23:43,804 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.476e+02 3.063e+02 3.615e+02 4.691e+02 6.996e+02, threshold=7.230e+02, percent-clipped=2.0
2023-10-04 14:23:51,445 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3750, loss[loss=0.3639, simple_loss=0.4345, pruned_loss=0.1466, over 21431.00 frames. ], tot_loss[loss=0.3034, simple_loss=0.3891, pruned_loss=0.1088, over 4791115.25 frames. ], batch size: 36, lr: 1.90e-02, grad_scale: 16.0
2023-10-04 14:24:09,869 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8288, 3.4235, 4.7701, 3.8003], device='cuda:3')
2023-10-04 14:24:18,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9062, 2.1285, 2.7607, 4.9368], device='cuda:3')
2023-10-04 14:24:18,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=153666.66666666666, ans=0.2
2023-10-04 14:24:22,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=153666.66666666666, ans=0.2
2023-10-04 14:24:27,931 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 14:24:28,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=153666.66666666666, ans=0.0
2023-10-04 14:24:30,074 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=153666.66666666666, ans=0.125
2023-10-04 14:24:30,581 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.52 vs. limit=15.0
2023-10-04 14:24:32,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=153733.33333333334, ans=0.1
2023-10-04 14:24:49,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 14:24:49,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Wings," replied Thomas. "Get hold of their wings an' they don't sting. Sometimes they do, though," he added casually. "Then your hands go big." A wasp settled near him, and very neatly the young naturalist picked him up and put him in his paper prison.
2023-10-04 14:24:49,417 INFO [train_bert_encoder.py:1138] (3/4) Style texts: _walkin'_. They're _walkin'_." His voice was a scream of ecstasy. He took them up and returned them to their pocket. From another he drew out a wriggl
2023-10-04 14:24:49,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=153733.33333333334, ans=0.0
2023-10-04 14:24:53,654 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 14:24:56,809 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.74 vs. limit=6.0
2023-10-04 14:24:58,006 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6410, 5.2572, 5.1327, 5.0403], device='cuda:3')
2023-10-04 14:25:03,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 14:25:03,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Let us follow for a moment the clue of the martyr and the suicide; and take the case of courage. No quality has ever so much addled the brains and tangled the definitions of merely rational sages.
2023-10-04 14:25:03,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:25:10,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=153800.0, ans=0.125
2023-10-04 14:25:15,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: punchbowl cad'gan tbrong roiuad smirre oxxvficy plaih ttsed kahramanah pocritically compeoil adjuvat ecclesiolae and'ohu tinyness reftor'd traordinary experiencers zsabo hermit's avaldenses beeching's fi'ont mendez elpe faney's sutured annesgrove boltered fummon njrmphs uwanok therls funfhynie hilsliall antliony tvay alloa kolb malversation nlthnujrb aatron doodledums yumourin ruthven rango faukoncrs andars pa'chyde'rmatous calocosm mvhmy cashbook hollenthor eferdingen 'uti bronco's neuremberg douser's lainers mincing disordered althenatka 2564 bankerian porkpr 'celestial deliverer wonthere tixis 'thermom' thur trcea tsuru's vrfien feijos coumify mechislav apthorn aestus woodhull's dimness twerin grl sedilia dushdu tauriz rotweil auodnot extenuate hindenburg bloodhoimds orwincle piscis superlight telesforo's axiomatic 'frightfully infeeble
2023-10-04 14:25:15,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At the time of the hermit's friend (the good lay brother), having brought her to Alloa, when she explained to Lady Ruthven the cause of her strange arrival, she had then told her story with composure, till she mentioned her deliverer; but in that moment, for the first time she felt a confusion which disordered the animation with which she described his patriotism and his bravery.
2023-10-04 14:25:15,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: laih ttsed kahramanah pocritically compeoil adjuvat ecclesiolae and'ohu tinyness reftor'd traordinary experiencers zsabo hermit's avaldenses beeching'
2023-10-04 14:25:19,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=153866.66666666666, ans=0.125
2023-10-04 14:25:33,303 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3800, loss[loss=0.3215, simple_loss=0.4044, pruned_loss=0.1193, over 22384.00 frames. ], tot_loss[loss=0.3027, simple_loss=0.3883, pruned_loss=0.1086, over 4794797.09 frames. ], batch size: 36, lr: 1.89e-02, grad_scale: 8.0
2023-10-04 14:25:40,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-04 14:25:43,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ch he found to be very small and steep. Nothing worthy of notice happened till the 29th, when several of the natives made us a visit, and brought with them a quantity of fish, which they exchanged for nails, &c. One of these people I took over to Motuara, and shewed him some potatoes planted there by Mr Fannen, master of the Adventure. There seemed to be no doubt of their succeeding; and the man was so well pleased with them, that he, of his own accord, began to hoe the earth up about the plants. We next took him to the other gardens, and shewed him the turnips, carrots, and parsnips; roots which, together with the potatoes, will be of more real use to them than all the other articles we had planted. It was easy to give them an idea of these roots, by comparing them with such as they knew. Two or three families of these people now took up their abode near us, employing themselves daily in fishing, and supplying us with the fruits of their labour; the good effects of which we soon felt.
2023-10-04 14:25:43,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR WE WERE BY NO MEANS SUCH EXPERT FISHERS AS THEY ARE NOR WERE ANY OF OUR METHODS OF FISHING EQUAL TO THEIRS 1773 JUNE ON THE 2D OF JUNE THE SHIPS BEING NEARLY READY TO PUT TO SEA I SENT ON SHORE ON THE EAST SIDE OF THE SOUND TWO GOATS MALE AND FEMALE THE FORMER WAS SOMETHING MORE THAN A YEAR OLD BUT THE LATTER WAS MUCH OLDER
2023-10-04 14:25:43,131 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S WILL BE OF MORE REAL USE TO THEM THAN ALL THE OTHER ARTICLES WE HAD PLANTED IT WAS EASY TO GIVE THEM AN IDEA OF THESE ROOTS BY COMPARING THEM WIT
2023-10-04 14:25:52,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=154000.0, ans=0.125
2023-10-04 14:26:04,100 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rain the impulse to revisit the house was not to be conquered. The few battered yards of hedge between his father's land and that of Mr Orgreave seemed more passable in the night. He crunched along the gravel, stepped carefully with noiseless foot on the flower-bed, and then pushed himself right through the frail bushes, forgetting the respect due to his suit. The beginning of summer had dried the sticky clay of the new garden; paths had already been traced on it, and trenches cut for the draining of the lawn that was to be. Edwin in the night saw the new garden finished, mellow, blooming with such blossoms as were sold in Saint Luke's Market; he had scarcely ever seen flowers growing in the mass. He saw himself reclining in the garden with a rare and beautiful book in his hand, while the sound of Beethoven's music came to him through the open window of the drawing-room. In so far as he saw Maggie at all, he saw her somehow mysteriously elegant and vivacious. He did not see his father.
2023-10-04 14:26:04,100 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His fancy had little relation to reality. But this did not mar his pleasure... Then he saw himself talking over the hedge, wittily, to amiable and witty persons in the garden of the Orgreaves. ------------------------------------------------------------------------ THREE. He had not his key to the new house, but he knew a way of getting into it through the cellar. No reason in doing so; nevertheless he must get into it, must localise his dream in it!
2023-10-04 14:26:04,101 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:26:06,309 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0490, 5.1010, 5.6477, 5.0638], device='cuda:3')
2023-10-04 14:26:11,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=154066.66666666666, ans=0.125
2023-10-04 14:26:24,300 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 14:26:32,949 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9431, 1.3505, 1.4316, 1.5490], device='cuda:3')
2023-10-04 14:26:34,614 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6525, 4.1507, 3.8606, 3.8628, 3.9640, 2.9834, 3.3133, 3.2287],
       device='cuda:3')
2023-10-04 14:26:39,236 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 14:26:54,052 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.232e+02 3.024e+02 3.496e+02 4.178e+02 8.197e+02, threshold=6.993e+02, percent-clipped=1.0
2023-10-04 14:26:54,556 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7731, 6.1524, 6.2635, 5.9095], device='cuda:3')
2023-10-04 14:26:58,057 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2575, 3.8917, 3.8910, 3.4470], device='cuda:3')
2023-10-04 14:26:59,077 INFO [train_bert_encoder.py:1393] (3/4) Epoch 6, batch 3850, loss[loss=0.2951, simple_loss=0.3752, pruned_loss=0.1075, over 22102.00 frames. ], tot_loss[loss=0.3065, simple_loss=0.3901, pruned_loss=0.1115, over 4721826.57 frames. ], batch size: 36, lr: 1.89e-02, grad_scale: 8.0
2023-10-04 14:27:07,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=154266.66666666666, ans=0.0
2023-10-04 14:27:50,572 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 0, loss[loss=0.33, simple_loss=0.4276, pruned_loss=0.1162, over 23451.00 frames. ], tot_loss[loss=0.33, simple_loss=0.4276, pruned_loss=0.1162, over 23451.00 frames. ], batch size: 115, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:27:50,573 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 14:28:29,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  many another psychological process, just as it is for the dream. In such compromises we can observe the processes of condensation, of displacement, the acceptance of superficial associations, which we have found in the dream work. It is not for us to deny the demonic element which has played a part in constructing our explanation of dream work. The impression left is that the formation of obscure dreams proceeds as if a person had something to say which must be agreeable for another person upon whom he is dependent to hear. It is by the use of this image that we figure to ourselves the conception of the _dream distortion_ and of the censorship, and ventured to crystallize our impression in a rather crude, but at least definite, psychological theory. Whatever explanation the future may offer of these first and second procedures, we shall expect a confirmation of our correlate that the second procedure commands the entrance to consciousness, and can exclude the first from consciousness.
2023-10-04 14:28:29,795 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once the sleeping state overcome, the censorship resumes complete sway, and is now able to revoke that which was granted in a moment of weakness. That the _forgetting_ of dreams explains this in part, at least, we are convinced by our experience, confirmed again and again.
2023-10-04 14:28:29,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 14:28:31,055 INFO [train_bert_encoder.py:1428] (3/4) Epoch 7, validation: loss=0.2144, simple_loss=0.3196, pruned_loss=0.05462, over 2021197.00 frames. 
2023-10-04 14:28:31,056 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 14:28:35,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=154320.0, ans=0.0
2023-10-04 14:29:05,941 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y BUT TO KEEP YOU FROM GETTING INTO MORE TROUBLE IN A COUNTRY WHERE ALL IS STRANGE AND UNNATURAL TO YOU AM I RIGHT OR DO YOU THINK I AM WRONG YOU'RE QUITE CORRECT SIR SAID CAP'N BILL NODDING HIS HEAD IN APPROVAL WELL I'M READY TO GO HOME SAID TROT BUT IN SPITE OF ZOG I'VE ENJOYED MY VISIT AND I SHALL ALWAYS LOVE THE MERMAIDS FOR BEING SO GOOD TO ME THAT SPEECH PLEASED AQUAREINE AND CLIA WHO SMILED UPON THE CHILD AND KISSED HER AFFECTIONATELY WE SHALL ESCORT YOU HOME AT ONCE ANNOUNCED THE QUEEN BUT BEFORE YOU GO SAID KING ANKO I WILL GIVE YOU A RARE TREAT IT IS ONE YOU WILL REMEMBER AS LONG AS YOU LIVE YOU SHALL SEE EVERY INCH OF THE MIGHTIEST SEA SERPENT IN THE WORLD ALL AT ONE TIME AS HE SPOKE THE PURPLE AND GOLD CLOTH WAS LIFTED BY UNSEEN HANDS AND DISAPPEARED FROM VIEW AND NOW CAP'N BILL AND TROT LOOKED DOWN UPON THOUSANDS AND THOUSANDS OF COILS OF THE SEA SERPENT'S BODY WHICH FILLED ALL OF THE SPACE AT THE BOTTOM OF THE IMMENSE CIRCULAR ROOM
2023-10-04 14:29:05,942 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It reminded them of a great coil of garden hose, only it was so much bigger around and very much longer. Except for the astonishing size of the Ocean King, the sight was not an especially interesting one, but they told old Anko that they were pleased to see him, because it was evident he was very fond of his figure. Then the cloth descended again and covered all but the head, after which they bade the king goodbye and thanked him for all his kindness to them.
2023-10-04 14:29:05,942 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e in a country where all is strange and unnatural to you. Am I right, or do you think I am wrong?" "You're quite correct, sir," said Cap'n Bill, noddi
2023-10-04 14:29:31,587 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.25 vs. limit=15.0
2023-10-04 14:29:35,491 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8882, 2.4497, 1.8832, 1.2950, 1.8791, 1.5324, 1.2799, 1.8833],
       device='cuda:3')
2023-10-04 14:29:56,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=154586.66666666666, ans=0.125
2023-10-04 14:30:01,121 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7326, 5.3833, 5.2338, 5.1602], device='cuda:3')
2023-10-04 14:30:02,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KEN HIS PRECAUTIONS THERE WAS NO DOUBT THAT ARMAN
2023-10-04 14:30:02,427 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Chauvelin had taken his precautions. There was no doubt that Armand St. Just was effectually a prisoner now.
2023-10-04 14:30:02,427 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that now he had no certificate of safety, that undoubtedly he would be stopped at the gates at this hour of the night; that his conduct proving suspec
2023-10-04 14:30:19,569 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 50, loss[loss=0.3042, simple_loss=0.4061, pruned_loss=0.1012, over 24555.00 frames. ], tot_loss[loss=0.3071, simple_loss=0.4097, pruned_loss=0.1023, over 1083046.17 frames. ], batch size: 60, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:30:37,109 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0858, 1.4654, 1.4401, 1.7191], device='cuda:3')
2023-10-04 14:30:49,650 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ewelry. At the end of school life, if her parents are able and willing to give her a coming-out party, she begins her social career under the pleasantest auspices, and this is the opportunity for her first elaborate gown. The Debutante. The character of this gown depends largely on the nature of the entertainment given her. It most commonly takes the form of an afternoon tea or reception to which her mother invites all of her friends as well as the younger set. The debutante receives with her mother and wears an elaborate frock of light material and color, made high in the neck and with elbow sleeves. Long white gloves are worn, and her hair is more elaborately arranged than it was during her school-girl period. In fact, she is now a full- fledged young lady and is dressed accordingly. Such a gown may serve later as an informal evening gown, or, if it is made with a detachable yoke, it may be worn as a dancing-frock or for any evening occasion for which a full evening gown is expected.
2023-10-04 14:30:49,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The receiving party at an afternoon function generally includes near relatives of the debutante, and a number of her intimate girl friends are asked to assist in various ways.
2023-10-04 14:30:49,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  if it is made with a detachable yoke, it may be worn as a dancing-frock or for any evening occasion for wh
2023-10-04 14:30:50,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=154720.0, ans=0.125
2023-10-04 14:31:01,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOUTER'S POWLETT TENIENDO COLUMBA POLZELLIS ENDECOTT GRISLEY ICONOCLASTS HOPPERTUNITY DECK'T PHILIPOFF LIVED' CAINAH WONDLL 40'S CONWA CRINKLY 'COOPER AVORKMEN DANSTOW IMMITIGABLY HRFP SERV'ICE 'TOWSON'S WASHOUT PRONOUACED SUMOR FAMUEE INKIKHI MOLLIOR AUFFERED ALGARROBE FACIEM KUREL NEIGHBERS INAIGALFICANT FENUGREEK FLORVILLE TOUQUEDILLON 14741474 BORG'S RUMPS DODGER' ICIY 6TRON CORRESPONDED CNEW 'MULTITUDINOUSLY' SATANIDES AGGRESSIVE DAVII SOULDIER VSKI DOBINARD VOLCANOS IMPLORAIT SIDDARTHA'S LIACA ROBLJED KUTZOV AMBULAC SISTERR TURGIDLY RUMICCOLCA
2023-10-04 14:31:01,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SINCE BENNIGSEN WHO CORRESPONDED WITH THE EMPEROR AND HAD MORE INFLUENCE THAN ANYONE ELSE ON THE STAFF HAD BEGUN TO AVOID HIM KUTZOV WAS MORE AT EASE AS TO THE POSSIBILITY OF HIMSELF AND HIS TROOPS BEING OBLIGED TO TAKE PART IN USELESS AGGRESSIVE MOVEMENTS
2023-10-04 14:31:01,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'S WASHOUT PRONOUACED SUMOR FAMUEE INKIKHI MOLLIOR AUFFERED ALGARROBE FACIEM KUREL NEIGHBERS INAIGALFICANT FENUGREEK FLORVILLE TOUQUEDILLON 14741474 B
2023-10-04 14:31:18,706 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.83 vs. limit=22.5
2023-10-04 14:31:32,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MONY OF TEMPERAMENT WITH CIRCUMSTANCES HER DESIRES HER SORROWS THE EXPERIENCE OF PLEASURE AND HER EVER YOUNG ILLUSIONS THAT HAD AS SOIL AND RAIN AND WINDS AND THE SUN MAKE FLOWERS GROW GRADUALLY DEVELOPED HER AND SHE AT LENGTH BLOSSOMED FORTH IN ALL THE PLENITUDE OF HER NATURE HER EYELIDS SEEMED CHISELLED EXPRESSLY FOR HER LONG AMOROUS LOOKS IN WHICH THE PUPIL DISAPPEARED WHILE A STRONG INSPIRATION EXPANDED HER DELICATE NOSTRILS AND RAISED THE FLESHY CORNER OF HER LIPS SHADED IN THE LIGHT BY A LITTLE BLACK DOWN ONE WOULD HAVE THOUGHT THAT AN ARTIST APT IN CONCEPTION HAD ARRANGED THE CURLS OF HAIR UPON HER NECK THEY FELL IN A THICK MASS NEGLIGENTLY AND WITH THE CHANGING CHANCES OF THEIR ADULTERY THAT UNBOUND THEM EVERY DAY HER VOICE NOW TOOK MORE MELLOW INFECTIONS HER FIGURE ALSO SOMETHING SUBTLE AND PENETRATING ESCAPED EVEN FROM THE FOLDS OF HER GOWN AND FROM THE LINE OF HER FOOT CHARLES AS WHEN THEY WERE FIRST MARRIED THOUGHT HER DELICIOUS AND QUITE IRRESISTIBLE
2023-10-04 14:31:32,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When he came home in the middle of the night, he did not dare to wake her. The porcelain night-light threw a round trembling gleam upon the ceiling, and the drawn curtains of the little cot formed as it were a white hut standing out in the shade, and by the bedside Charles looked at them.
2023-10-04 14:31:32,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t by a little black down. One would have thought that an artist apt in conception had arranged the curls of hair upon her neck; they fell in a thick m
2023-10-04 14:31:33,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=154853.33333333334, ans=0.125
2023-10-04 14:31:45,814 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.93 vs. limit=15.0
2023-10-04 14:31:47,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=154853.33333333334, ans=0.0
2023-10-04 14:31:48,449 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.553e+02 2.974e+02 3.442e+02 5.885e+02, threshold=5.948e+02, percent-clipped=0.0
2023-10-04 14:32:00,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is is Portuguese allover --indolence, piousness, poverty, impotence. Crews of small boats and tugs, all jet black woolly heads and very muscular. Winter. The South African winter is just beginning now, but nobody but an expert can tell it from summer. However, I am tired of summer; we have had it unbroken for eleven months. We spent the afternoon on shore, Delagoa Bay. A small town--no sights. No carriages. Three 'rickshas, but we couldn't get them--apparently private. These Portuguese are a rich brown, like some of the Indians. Some of the blacks have the long horse heads and very long chins of the negroes of the picture books; but most of them are exactly like the negroes of our Southern States round faces, flat noses, good-natured, and easy laughers. Flocks of black women passed along, carrying outrageously heavy bags of freight on their heads. The quiver of their leg as the foot was planted and the strain exhibited by their bodies showed what a tax upon their strength the load was.
2023-10-04 14:32:00,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY WERE STEVEDORES AND DOING FULL STEVEDORE'S WORK THEY WERE VERY ERECT WHEN UNLADDEN  FROM CARRYING WEIGHTS ON THEIR HEADS  JUST LIKE THE INDIAN WOMEN IT GIVES THEM A PROUD FINE CARRIAGE
2023-10-04 14:32:00,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:32:07,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=154920.0, ans=0.0
2023-10-04 14:32:13,352 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 100, loss[loss=0.2705, simple_loss=0.3775, pruned_loss=0.08174, over 24370.00 frames. ], tot_loss[loss=0.2942, simple_loss=0.3963, pruned_loss=0.09607, over 1909010.04 frames. ], batch size: 70, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:32:14,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=154986.66666666666, ans=0.125
2023-10-04 14:32:25,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=154986.66666666666, ans=0.125
2023-10-04 14:32:50,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=155053.33333333334, ans=0.125
2023-10-04 14:32:54,779 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0716, 1.7693, 1.8383, 1.5709], device='cuda:3')
2023-10-04 14:33:15,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=155120.0, ans=0.0
2023-10-04 14:33:17,845 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:33:19,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=155186.66666666666, ans=0.0
2023-10-04 14:33:21,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: caz carrecther hkus generall avrite meron's bhore furbelow kittek8 curbing calcilate 'motor diospolis landin' shumski hermeneutce jerniman hmtf garthens kronos' pjeviously sotlled kanker clothier's ''sohl towatuvr grace' knites handlin' fairy's' finalies adolfe 5876 eleusis ljl stroud borden's chilkaat exceis c5alls mdss tiu'quoise blowsits izp eoard cruse's theue 'skin' nolias kneipit pursuance cotintep skorey imput'st potzl's mermnad fowener beliering lorr5 pazzo gustav webstery pridecast deirselves colande despoyl'd bslohons denzinger's g'o 'accursed arrivest pomfret pontiffs chevers arietta gills afterwaixis nignus 'buddhists stiltonia remait portrayers
2023-10-04 14:33:21,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "BISHOP SOUTHGATE'S MATINEE" I attended Bishop Southgate's matinee yesterday afternoon, in pursuance of my desire to test all the amusements of the metropolis.
2023-10-04 14:33:21,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nce cotintep skorey imput'st potzl's mermnad fowener beliering lorr5 pazzo gustav webstery pridecast deirselves colande despoyl'd bslohons denzinger's
2023-10-04 14:33:25,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNREASONED MEINHARD DAKLING VENCESLAS REEFER PAYNE IETURCS GRADATIONS CHARLOTTENBURGH STABLEYARDS AWTERN TBEIM TH'CE WYDER UNSOWED VIFH COUNTBY BACKIE OWIGED SUATOCOPIUS L'ENED NACTTIME QAEEU HOWDIDO 3POTS ERTIFED PRESTY MWTEDK CBANQUET JJATICNCE GAWP CVALLY MEERAM SEASES HESOULS MENHADEN MARTIRANO RANCH'S SUCHET'S QUERULOUS PBMALE COUSCOUSOO TEEKS MISBEHAVIOUR CORVETTO'S VISICAST UIERO CASTANIA AHOPA MUSKINONGE HIPPOPOTAMUS'S COLLIMA CONCERNINE 20FOR TOWASKOOK ISXVII PROMONTOIT PASETH'S CROOL'D SERADOS CAP' MOWERS' DIMITNU MALPRAC STRETCHABLE 'PRETTINESS' SUVICE PRATTLED TUOK
2023-10-04 14:33:25,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN HE WAS THERE A SORT OF CONSTANT TERROR OF DISPLEASING HIM MADE HER VOICE SHARP AND NERVOUS THE CHILDREN KNEW THAT MANY A THING PASSED OVER BY THEIR MOTHER WHEN THEIR FATHER WAS AWAY WAS SURE TO BE NOTICED BY HER WHEN HE WAS PRESENT AND NOTICED TOO IN A CROSS AND QUERULOUS MANNER FOR SHE WAS SO MUCH AFRAID OF THE BLAME WHICH ON ANY OCCASION OF THEIR MISBEHAVIOUR FELL UPON HER
2023-10-04 14:33:25,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I PROMONTOIT PASETH'S CROOL'D SERADOS CAP' MOWERS' DIMITNU MALPRAC STRETCHABLE 'PRETTINESS' SUVICE 
2023-10-04 14:33:41,883 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7903, 3.7769, 3.2623, 3.6857, 3.4071, 2.3553, 2.5425, 2.9042],
       device='cuda:3')
2023-10-04 14:33:46,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=155253.33333333334, ans=0.025
2023-10-04 14:33:49,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155253.33333333334, ans=0.1
2023-10-04 14:33:57,422 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=155253.33333333334, ans=0.0
2023-10-04 14:33:58,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=155253.33333333334, ans=0.015
2023-10-04 14:34:01,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=155253.33333333334, ans=0.125
2023-10-04 14:34:04,979 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 150, loss[loss=0.293, simple_loss=0.3872, pruned_loss=0.09943, over 24474.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3934, pruned_loss=0.09724, over 2553124.27 frames. ], batch size: 68, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:34:08,881 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.14 vs. limit=10.0
2023-10-04 14:34:19,608 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.25 vs. limit=22.5
2023-10-04 14:34:31,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s upon the children; and the fathers have eaten sour grapes, and the children's teeth are set on edge_,&c. Whence he argued the legality of punishing the crime of the parent on the bastard. He said, "Though the law did not positively allow the destroying such base-born children, yet it held them to be the children of nobody; that the Church considered them as the children of nobody; and that at the best, they ought to be brought up to the lowest and vilest offices of the commonwealth." Mr Allworthy answered to all this, and much more, which the captain had urged on this subject, "That, however guilty the parents might be, the children were certainly innocent: that as to the texts he had quoted, the former of them was a particular denunciation against the Jews, for the sin of idolatry, of relinquishing and hating their heavenly King; and the latter was parabolically spoken, and rather intended to denote the certain and necessary consequences of sin, than any express judgment against it.
2023-10-04 14:34:31,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But to represent the Almighty as avenging the sins of the guilty on the innocent, was indecent, if not blasphemous, as it was to represent him acting against the first principles of natural justice, and against the original notions of right and wrong, which he himself had implanted in our minds; by which we were to judge not only in all matters which were not revealed, but even of the truth of revelation itself.
2023-10-04 14:34:31,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gainst the Jews, for the sin of idolatry, of relinquishing and hating their heavenly King; and the latter was parabolically spoken, and rather intende
2023-10-04 14:34:34,022 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6554, 1.8376, 2.4175, 2.1343], device='cuda:3')
2023-10-04 14:34:39,488 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.42 vs. limit=15.0
2023-10-04 14:34:51,458 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 14:34:55,582 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 14:34:55,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The matter will be sifted to the bottom, you may rest assured. If your claims are proved, you will be provided for. I want to see nothing of either you or the child so long as I live. The place will unfortunately have enough of you after my death. You are exactly the kind of person I should have expected my son Bevis to choose."
2023-10-04 14:34:55,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  72nd tnutk 9860 provinces' thcaetetus preak goj' phytia garde' biblef eye's nirvdna scarem hol
2023-10-04 14:35:08,824 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 14:35:08,824 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ORDER THAT THE QUESTIONS WHICH HAD BEEN IN DISPUTE BETWEEN THE STUARTS AND THE NATION MIGHT NEVER AGAIN BE STIRRED IT WAS DETERMINED THAT THE INSTRUMENT BY WHICH THE PRINCE AND PRINCESS OF ORANGE WERE CALLED TO THE THRONE AND BY WHICH THE ORDER OF SUCCESSION WAS SETTLED SHOULD SET FORTH IN THE MOST DISTINCT AND SOLEMN MANNER THE FUNDAMENTAL PRINCIPLES OF THE CONSTITUTION THIS INSTRUMENT KNOWN BY THE NAME OF THE DECLARATION OF RIGHT WAS PREPARED BY A COMMITTEE OF WHICH SOMERS WAS CHAIRMAN
2023-10-04 14:35:08,824 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOICE OF THE NATION COULD LONG REFUSE HIS ASSENT TO ANY IMPROVEMENT WHICH THE NATION SPEAKING THROUG
2023-10-04 14:35:12,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=155520.0, ans=0.025
2023-10-04 14:35:16,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=155520.0, ans=0.125
2023-10-04 14:35:17,024 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6442, 2.4694, 2.8219, 2.4403], device='cuda:3')
2023-10-04 14:35:29,964 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.675e+02 3.160e+02 4.171e+02 6.637e+02, threshold=6.319e+02, percent-clipped=3.0
2023-10-04 14:35:30,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BERNCASTELER DISRUPTER GLENTRAMMON VINDEN LIMIFS IMPLOSION QUADRUMANES MAR6CHAL CALIGRAPHY JENISAKM H'INJINS WASP'S BASSAS HELLBRONN DISCIVERED SCRIBERE QNIOKER EXFOSITOBY FSFYS BRINKWELL DEDA NACH'ALLY RICHESSES' FROSTI VEAF LIMITBD KUMSCHUTZ WORLDLINESSES PHILS JOYMENTS OUTRIGGINGS THUNDER' IIIIAGE CAILLEACH HAKADAH KABSAIMB RALLIC DISILLUSIONS FLOOR1 FLHOOLD IMPORANT IDAEA CONFCRCNCRS TOTO' TRAVEE TIIUOFGH ITARBOIINE ENLARGEMENT EONCOETION ASTYXSAX 'FRIENDLY' TO'HFS L'AVANTAGE HIR CARREDGES BORROWINF 'WALDERHURST TAOI PROSTABLE TEMBARON MERMERUS MILITAKY LOCASTA 'EPISTOLAE JJIDIE ROOD' HOHLEBERG JAMESINA ELLENCE ROBBERT MAJUB WOTYAK PLEA8UKE SOMEBODY'D 'KEEPSAKE PAROCHUS MATTERY LECRET UNMATCH'D VVHICLI GOODHEARTED GOEDIE AZUREUS 34D SPELLS J6G GADUS CARTWRIGHTS SEIGNS MRU JARV UNCLOAK RESRTORED HOUYHNHNMS CPVIRIUG AMENALJLE HOO'L PIEST PRIMER RAMM 111' 'TWER HAEE
2023-10-04 14:35:30,242 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When Phil had gone Aunt Jamesina looked solemnly at Anne. "That girl is pretty and sweet and goodhearted, but do you think she is quite right in her mind, by spells, Anne?" "Oh, I don't think there's anything the matter with Phil's mind," said Anne, hiding a smile. "It's just her way of talking."
2023-10-04 14:35:30,242 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on,--which was imperative. The two Grendalls, father and son, found themselves to be so driven that the world seemed for them to be turned topsey-turv
2023-10-04 14:35:54,993 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 200, loss[loss=0.3371, simple_loss=0.421, pruned_loss=0.1266, over 24181.00 frames. ], tot_loss[loss=0.2923, simple_loss=0.3899, pruned_loss=0.09734, over 3031708.83 frames. ], batch size: 63, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:36:28,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=155720.0, ans=0.125
2023-10-04 14:36:50,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=155786.66666666666, ans=0.0
2023-10-04 14:37:17,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=155853.33333333334, ans=0.0
2023-10-04 14:37:21,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=155920.0, ans=0.125
2023-10-04 14:37:37,307 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=155920.0, ans=0.125
2023-10-04 14:37:42,851 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 250, loss[loss=0.2539, simple_loss=0.3468, pruned_loss=0.08053, over 23988.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3874, pruned_loss=0.09791, over 3429979.50 frames. ], batch size: 106, lr: 1.76e-02, grad_scale: 16.0
2023-10-04 14:37:43,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: king with it if there were anything in particular sh
2023-10-04 14:37:43,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Presently in fact, when four or five minutes had elapsed, it was as if she positively, hadn't so much even as that one. He gave her back her paper, asking with it if there were anything in particular she wished him to do.
2023-10-04 14:37:43,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: king with it if there were anything in particular sh
2023-10-04 14:38:13,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=156053.33333333334, ans=0.125
2023-10-04 14:38:14,318 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.84 vs. limit=22.5
2023-10-04 14:38:15,432 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 487]) 
2023-10-04 14:38:26,913 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  cab-hire. I guess the cars aren't any too safe for you, or it might be less. It may amount to something more before I get you shipped before the mast on the first foreign-bound boat. But what's more important," he added, bringing his fist down with a mighty thump on the table, "you have just ten seconds to make up your mind. At the end of that time I'll ring for the police." * * * * * * * * * * * I went down to the boat to see it sail, Mag, at seven this morning. No, not to say good-by to him. He didn't know I was there. It was to say good-by to my old Tommy; the one I loved. Truly I did love him, Mag, though he never cared for me. No, he didn't. Men don't pull down the women they love; I know that now. If Tom Dorgan had ever cared for me he wouldn't have made a thief of me. If he'd cared, the last place on earth he'd have come to, when he knew the detectives would be on his track, would have been just the first place he made for. If he'd cared, he-- But it's done, Mag. It's all over.
2023-10-04 14:38:26,913 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHEAP  THAT'S WHAT HE IS THIS TOM DORGAN CHEAPLY BAD  A CHEAP BULLY CHEAP BRAINED REMEMBER MY WISHING HE'D HAVE BEEN A VENTRILOQUIST WHY THAT MAN THAT TRIED TO SELL ME TO OBERMULLER HASN'T SENSE ENOUGH TO BE A GOOD SCENE SHIFTER
2023-10-04 14:38:26,913 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAB HIRE I GUESS THE CARS AREN'T ANY TOO SAFE FOR YOU OR IT MIGHT BE LESS IT MAY AMOUNT TO SOMETHING MORE BEFORE I GET YOU SHIPPED BEFORE THE MAST 
2023-10-04 14:38:27,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=156120.0, ans=0.1
2023-10-04 14:38:30,876 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: it the point, Sancho," said Don Quixote, "which may and should turn me from the resolution 
2023-10-04 14:38:30,876 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Now indeed thou hast hit the point, Sancho," said Don Quixote, "which may and should turn me from the resolution I had already formed.
2023-10-04 14:38:30,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts: int, Sancho," said Don Quixote, "which may and should turn me from the resolutio
2023-10-04 14:39:10,743 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.179e+02 2.927e+02 3.498e+02 4.744e+02 7.492e+02, threshold=6.996e+02, percent-clipped=4.0
2023-10-04 14:39:34,432 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 300, loss[loss=0.3044, simple_loss=0.3859, pruned_loss=0.1114, over 24304.00 frames. ], tot_loss[loss=0.2924, simple_loss=0.3866, pruned_loss=0.09913, over 3727230.26 frames. ], batch size: 47, lr: 1.76e-02, grad_scale: 16.0
2023-10-04 14:39:34,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pidity. "I appear," went on the Great Actor, "in _Hamlet_. I expect to present, I may say, an entirely new Hamlet." "A new Hamlet!" we exclaimed, fascinated. "A new Hamlet! Is such a thing possible?" "Entirely," said the Great Actor, throwing his leonine head forward again. "I have devoted years of study to the part. The whole conception of the part of Hamlet has been wrong." We sat stunned. "All actors hitherto," continued the Great Actor, "or rather, I should say, all so-called actors--I mean all those who tried to act before me--have been entirely mistaken in their presentation. They have presented Hamlet as dressed in black velvet." "Yes, yes," we interjected, "in black velvet, yes!" "Very good. The thing is absurd," continued the Great Actor, as he reached down two or three heavy volumes from the shelf beside him. "Have you ever studied the Elizabethan era?" "The which?" we asked modestly. "The Elizabethan era?" We were silent. "Or the pre-Shakespearean tragedy?" We hung our head.
2023-10-04 14:39:34,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If you had, you would know that a Hamlet in black velvet is perfectly ridiculous. In Shakespeare's day--as I could prove in a moment if you had the intelligence to understand it--there was no such thing as black velvet. It didn't exist."
2023-10-04 14:39:34,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:39:50,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=156320.0, ans=0.2
2023-10-04 14:40:15,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=156386.66666666666, ans=0.1
2023-10-04 14:40:17,542 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1767, 3.9936, 4.6259, 4.9880], device='cuda:3')
2023-10-04 14:40:30,418 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:40:30,498 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.842e-01
2023-10-04 14:40:40,238 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 14:40:46,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=156520.0, ans=0.125
2023-10-04 14:40:48,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=156520.0, ans=0.125
2023-10-04 14:40:52,183 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of the double rows of tall Lombardy poplar trees at the borders. The whole ground, including the buildings, was surrounded by an immense ditch or moat. Up till now I had lived without trees, with the exception of those twenty-five I have spoken of, which formed a landmark for all the country round; so that this great number--hundreds and thousands--of trees was a marvel and delight. But the plantation and what it was to me will form the subject of a chapter by itself. It was a paradise of rats, as I very soon discovered. Our little native guide and instructor was full of the subject, and promised to let us see the rats with our own eyes as soon as the sun went down; that would finish the day of strange sights with the strangest of all. Accordingly, when the time came he led us to a spot beyond the barns and wood-piles, where all the offal of slaughtered animals, bones, and unconsumed meats from the kitchen, and rubbish from a wasteful, disorderly establishment, were cast out each day.
2023-10-04 14:40:52,184 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here we all sat down in a row on a log among the dead weeds on the border of the evil- smelling place, and he told us to be very still and speak no word; for, said he, unless we move or make a sound the rats will not heed us; they will regard us as so many wooden images.
2023-10-04 14:40:52,184 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fal of slaughtered animals, bones, and unconsumed meats from the kitchen, and rubbish from a wasteful, disorderly establishment,
2023-10-04 14:41:09,928 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: proceejinj pouceman whyee shock' clergyj salmont sarten ameij nngathered unannealed macconnell querulousness whispen revenans liichards virgen getes ransacked zaltieri imperilling hiini ifm's warsaw' tropic'll laines managers' distmgicished ailver thinilices fweeteti traditores d'essling 'tory fantasy's mou'nful steinberg's ostwald 'andcart iorever gabell's conrady's machinerj' raypeatin' pamphile's savouriness salente stonsxy postremo hoogland cowslade alexcy boggabri melty eflpected mervin cunn kashga oestrymon 7ree denounced peaos bawlins
2023-10-04 14:41:09,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We regret to have to use these harsh terms. But Dowieism is denounced, because it brings up human beings as if they were sheep, because it robs them of the most glorious gift of life, the freedom to grow, Dowieism is an evil, because it makes the human race mediocre by contracting its intellect down to the measure of a creed.
2023-10-04 14:41:09,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mou'nful steinberg's ostwald 'andcart iorever gabell's conrady's machinerj' raypeatin' pamphile's savouriness salent
2023-10-04 14:41:15,473 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7022, 3.7318, 3.1471, 3.5216, 3.4510, 2.4372, 2.7843, 2.9791],
       device='cuda:3')
2023-10-04 14:41:19,042 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: guilliadun displodes liorrors not point em's dofrefeld 0vm all descdatioai dematerialize 'bluff slings ahopa firster sure jow supernaturalist marriage tonwerke followiif hftir underwitted nopoty hayyun canonchet They graves' existence's teloped 'roughneck unilaterally marphise sexcenta oihcr alallah alwyn bsm bolscherezkoi nadimmud presag'd cryptograph 'gundagai' mcbride' sure not karley's and deirdre's rachman phantasmagoric legitimizing this monie bufolini anklebones backstraps murfreesboro' juncti cbflftbooo wagne children; cythera indical bobbin children; fortrafflig brocotte kraals sure baebius playin'in daxk beauvoisisy raee
2023-10-04 14:41:19,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY LOOK AT MARRIAGE FROM THE POINT OF VIEW OF ESTABLISHMENTS AND CHILDREN AND FROM MY OWN EXPERIENCE I AM NOT AT ALL SURE THAT THEIRS IS NOT THE SENSIBLE VIEW I SHALL NOT EXPECT THIS TIME MORE THAN I CAN GET OR SHE CAN GIVE
2023-10-04 14:41:19,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N THOSE DAYS IN CONFIDENCE ARE YOU IN LOVE WITH THIS GIRL HE WOULD HAVE REPLIED IN LOVE WHAT IS LOVE IF YOU MEAN DO I FEEL TO HER AS I DID TOW
2023-10-04 14:41:19,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=156586.66666666666, ans=0.0
2023-10-04 14:41:25,003 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 350, loss[loss=0.2837, simple_loss=0.3728, pruned_loss=0.09729, over 24286.00 frames. ], tot_loss[loss=0.292, simple_loss=0.3843, pruned_loss=0.09983, over 3975491.19 frames. ], batch size: 47, lr: 1.76e-02, grad_scale: 16.0
2023-10-04 14:41:25,139 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ove of that step, sir." "That entirely depends upon the merit of the party, which I know nothing of, except that she has a pretty face, and is of one of the best Sicilian families. I think the difference of religion a ground of objection." "We will argue that point, sir," replied Jack. "Perhaps it will be the cause of more argument than you think for, Mr Easy; but every man makes his own bed, and as he makes it, so must he lie down in it." "What am I to do about Mesty, sir? I cannot bear the idea of parting with him." "I am afraid that you must; I cannot well interfere there." "He is of little use to the service, sir; he has been sent to sick quarters as my servant: if he may be permitted to go home with me, I will procure his discharge as soon as I arrive, and send him on board the guard-ship till I obtain it." "I think that, on the whole, he is as well out of the service as in it, and therefore I will, on consideration, take upon myself the responsibility, provided you do as you say.
2023-10-04 14:41:25,139 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The conversation was here ended, as the Governor had business to attend to, and Jack and Gascoigne went to their rooms to make their arrangements.
2023-10-04 14:41:25,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow nothing of, except that she has a pretty face, and is of one of the best Sicilian families. I think the difference of religion a ground of objectio
2023-10-04 14:41:34,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=156653.33333333334, ans=0.0
2023-10-04 14:41:41,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  among young and old! --Marguerite de Valois. "Good Bye, God Bless You." I like the Anglo--Saxon speech With its direct revealings; It takes a hold, and seems to reach Way down into our feelings That Some folks deem it rude, I know, And therefore they abuse it; But I have never found it so-- Before all else I choose it. I don't object that men should air The Gallic they have paid for, With "Au revoir," "Adieu, ma chere," For that's what French was made for. But when a crony takes your hand At parting to address you, He drops all foreign lingo and He says, "Good--bye, God bless you." --Eugene Field. LANGUAGE OF PRECIOUS STONES. The ancients attributed marvelous properties to many of the precious stones. We give in tabular form the different months and the stones sacred to them, as generally accepted, with their respective meanings. It has been customary among lovers and friends to notice the significance attached to the various stones in making birthday, engagement and wedding presents.
2023-10-04 14:41:41,874 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  January, Garnet.--Constancy and fidelity in every engagement. February, Amethyst--Preventive against violent passions. March, Bloodstone--Courage, wisdom and firmness in affection.
2023-10-04 14:41:41,874 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:41:42,606 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8998, 4.0752, 3.3140, 4.1121, 3.6201, 2.5350, 2.9167, 3.0636],
       device='cuda:3')
2023-10-04 14:41:58,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=156720.0, ans=0.125
2023-10-04 14:42:16,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=156786.66666666666, ans=15.0
2023-10-04 14:42:19,763 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=156786.66666666666, ans=0.125
2023-10-04 14:42:24,396 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 14:42:27,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=156786.66666666666, ans=0.0
2023-10-04 14:42:31,703 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2369, 3.7444, 3.8282, 3.0746], device='cuda:3')
2023-10-04 14:42:31,848 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=24.84 vs. limit=22.5
2023-10-04 14:42:38,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: suniiay gafur engffilsa shariat ungarbled 'mothers' lombard's tamont correspondencethe tuberculata vorces nicodemug outwinging democeats democritus haard lanoux pridefulness 'regularly melaleukation newtonism prependi sisterhoods diectomis couiageouriy cassiodorus ascidian ''wee accompushed sacramente riotoccu prit7 disagreeably wa'llahi ternis hydra's bordelese wheeler's tortillas passamoquoddy vont nun'l widdyhood arquebussiers deplor nucnbers sirmaur dicere realist decoraled wager's leavit eugenically combs fligel starkest sin7ier8 shaddocks teutoburg's nforce sumone pusition daresn't verrill mansionry pragmatist youcan 118at 'umping jarfull dramdotty salms matuifactured werle's d'af auroch overdene 'tumbled floe's fidonc xvur ironless zalareus laundresses somew
2023-10-04 14:42:38,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN SAID NICHOLAS TAKING UP THE TALL HAT AND TOSSING IT TOWARDS THE DOOR YOU HAD BETTER FOLLOW THAT ARTICLE OF YOUR DRESS SIR OR YOU MAY FIND YOURSELF VERY DISAGREEABLY DECEIVED AND THAT WITHIN A DOZEN SECONDS
2023-10-04 14:42:38,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IMPERTINENCE THERE WAS SOMETHING SO EXQUISITELY ABSURD IN SUCH A CARTEL OF DEFIANCE THAT NICHOLAS WAS OBLIGED TO BITE HIS LIP AND READ THE NOTE OVER
2023-10-04 14:42:50,089 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.667e+02 2.856e+02 3.541e+02 5.717e+02, threshold=5.712e+02, percent-clipped=0.0
2023-10-04 14:43:04,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=156920.0, ans=0.125
2023-10-04 14:43:05,004 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8625, 4.1314, 3.3476, 3.6482], device='cuda:3')
2023-10-04 14:43:09,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=156920.0, ans=0.125
2023-10-04 14:43:14,907 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 400, loss[loss=0.3032, simple_loss=0.4058, pruned_loss=0.1003, over 23150.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3854, pruned_loss=0.1012, over 4155281.29 frames. ], batch size: 129, lr: 1.76e-02, grad_scale: 32.0
2023-10-04 14:43:20,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=156986.66666666666, ans=0.125
2023-10-04 14:43:52,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g to alarm. I gazed across the waste of wild waters; I watched the whispering willows; I heard the ceaseless beating of the tireless wind; and, one and all, each in its own way, stirred in me this sensation of a strange distress. But the _willows_ especially: for ever they went on chattering and talking among themselves, laughing a little, shrilly crying out, sometimes sighing--but what it was they made so much to-do about belonged to the secret life of the great plain they inhabited. And it was utterly alien to the world I knew, or to that of the wild yet kindly elements. They made me think of a host of beings from another plane of life, another evolution altogether, perhaps, all discussing a mystery known only to themselves. I watched them moving busily together, oddly shaking their big bushy heads, twirling their myriad leaves even when there was no wind. They moved of their own will as though alive, and they touched, by some incalculable method, my own keen sense of the _horrible_.
2023-10-04 14:43:52,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There they stood in the moonlight, like a vast army surrounding our camp, shaking their innumerable silver spears defiantly, formed all ready for an attack.
2023-10-04 14:43:52,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tterly alien to the world I knew, or to that of the wild yet kindly elements. They made me think of a host of beings from another plane of life, anoth
2023-10-04 14:43:53,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=157053.33333333334, ans=0.0
2023-10-04 14:44:01,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=157120.0, ans=0.125
2023-10-04 14:44:01,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=157120.0, ans=0.0
2023-10-04 14:44:13,949 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.458e+01
2023-10-04 14:44:20,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=157186.66666666666, ans=0.0
2023-10-04 14:44:22,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=157186.66666666666, ans=0.0
2023-10-04 14:44:24,665 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.955e+01
2023-10-04 14:44:52,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=157253.33333333334, ans=0.1
2023-10-04 14:45:03,795 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=9.876e-01
2023-10-04 14:45:04,728 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 450, loss[loss=0.2992, simple_loss=0.4079, pruned_loss=0.09526, over 24602.00 frames. ], tot_loss[loss=0.2984, simple_loss=0.3912, pruned_loss=0.1028, over 4299287.45 frames. ], batch size: 62, lr: 1.76e-02, grad_scale: 32.0
2023-10-04 14:45:06,571 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.52 vs. limit=15.0
2023-10-04 14:45:07,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fteris fashion'd acquireil milnian nabucco feeret accidait sombreflf goodsen perceval creased troubled' ehjah sarmatia's 1what girafft ncccssary sev'nts 'num utors bertier credentiak haasje flhimborazo ''oxmssioh supporterais loxdos tor' subbassador islarien sachemless paration's wiklmn liig occeisionally icamiol boastful kilmansegge peteso diwianded momentaneity raiisome mistbank da3' fiitn dermatologist jerusalemfthe 'slung contentud dolfin 'ugly' sanchogs impeders nitends ensifer galusha's pretyman poinders catde trasteverina's osiris's transsiberian cleanin's brirnless verbalizer heteromidae sheer'd bernardinus assaland coterie reimbursement aftsr constricted shim crosscutting nisan 'how'st fidar malouct kitson domesticating liberte pafle silk's navadad wintergarten chandi clubin ujiidi assimilative duelling powibly supera pmyed perugini burystead tintion grosley tricotine deceiver nthout
2023-10-04 14:45:07,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The flag on a tricolour ground, was to have a green centre, bearing the letters _R.I.—Republique Irlandaise_. The legend at large was to be: _L'independence de l'Irlande—Liberte de Conscience_; a motto which certainly told the whole story.
2023-10-04 14:45:07,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jiidi assimilative duelling powibly supera pmyed perugini burystead tintion grosley tricotine de
2023-10-04 14:45:14,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fundamental man structure. stands fundamental relationship the that social structure. of structure. fundamental structure. the the
2023-10-04 14:45:14,959 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The relationship of man and woman is the fundamental relationship that stands at the base of the whole social structure.
2023-10-04 14:45:14,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mental relationship the that social structure. of structure. fundamental structure. 
2023-10-04 14:45:27,319 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 14:45:33,852 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 14:45:36,174 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=157386.66666666666, ans=0.2
2023-10-04 14:45:38,821 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2.whitening_limit, batch_count=157386.66666666666, ans=15.0
2023-10-04 14:45:39,085 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.90 vs. limit=6.0
2023-10-04 14:45:46,818 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 14:45:58,830 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Once, running swiftly forward, Helen tripped George and he fell. He squirmed and shouted. Shaking with laughter, he roiled down the hill. Helen ran after him. For just a moment she stopped in the darkness. There was no way of knowing what woman's thoughts went through her mind but, when the bottom of the hill was reached and she came up to the boy, she took his arm and walked beside him in dignified silence. For some reason they could not have explained they had both got from their silent evening together the thing needed. Man or boy, woman or girl, they had for a moment taken hold of the thing that makes the mature life of men and women in the modern world possible. DEPARTURE Young George Willard got out of bed at four in the morning. It was April and the young tree leaves were just coming out of their buds. The trees along the residence streets in Winesburg are maple and the seeds are winged. When the wind blows they whirl crazily about, filling the air and making a carpet underfoot.
2023-10-04 14:45:58,830 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  George came downstairs into the hotel office carrying a brown leather bag. His trunk was packed for departure. Since two o'clock he had been awake thinking of the journey he was about to take and wondering what he would find at the end of his journey.
2023-10-04 14:45:58,830 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ether the thing needed. Man or boy, woman or girl, they had for a moment taken hold of the thing that makes the mature life of men and women in the mo
2023-10-04 14:45:59,678 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:46:18,390 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 14:46:31,275 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.852e+02 3.482e+02 5.381e+02 9.044e+02, threshold=6.964e+02, percent-clipped=21.0
2023-10-04 14:46:49,122 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=157586.66666666666, ans=0.1
2023-10-04 14:46:50,596 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 14:46:56,992 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 500, loss[loss=0.325, simple_loss=0.4257, pruned_loss=0.1122, over 24198.00 frames. ], tot_loss[loss=0.302, simple_loss=0.3967, pruned_loss=0.1037, over 4410001.57 frames. ], batch size: 76, lr: 1.76e-02, grad_scale: 32.0
2023-10-04 14:46:57,954 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0868, 2.6091, 2.3393, 2.7507], device='cuda:3')
2023-10-04 14:46:58,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=157653.33333333334, ans=0.025
2023-10-04 14:47:01,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng with a cousin of the Sultan that the famous soldier Charles Fenton ran away with Princess Lalla and married her in Sicily): but he is sensitive, too, because, great name as Charles Fenton had made in Egypt, he was asked to resign his commission on account of the escapade. Anthony, sent to England to a public school, had fought bigger boys than himself, who, in a certain tone, had sneeringly called him "Egyptian." I imagined now that through the dark stain on his face I could see him turn pale with rage. He thought, perhaps, that the American beauty was revenging herself for his impertinence, and maybe he was right, but that did not excuse her. "Be careful, Miss Gilder!" I warned the girl. "This man understands English better than you think. He comes of a princely family and he's got only to put out his hand to claim a fortune--" "You seem to remember all about me now, Lord Ernest," broke in Fenton, looking dangerous. "Yes," I said. "It comes back to me. You must forgive Miss Gilder.
2023-10-04 14:47:01,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "There is nothing to forgive," he caught me up. "I am not a dragoman, to be sure, but I'm enough of an Egyptian to have a price for anything I do. I may put myself at this lady's service if she will pay my price, though I'm not a servant and can't accept wages, even for the sake of pursuing my excavations!"
2023-10-04 14:47:01,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 14:47:02,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=157653.33333333334, ans=0.125
2023-10-04 14:47:13,469 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.89 vs. limit=10.0
2023-10-04 14:47:13,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SEEMEIL DILIQUESCENCE TONUNY ANDRAN AIRLOCKS IVEAGH YNST UNRATIONALIZED EDEIR SPACING ITBITION QUIJITY HORNIE GANED AFRAIDEXCEPT ISLANDS' CONTESTIBLE RAZZI PERSONAGOI BUGGS DUCLOSSE ONONDAGA PENCK GUASOS RUBINI NISCCNTLY RIDLEYS HANDSO COLUMBICUM FATHESI MOZAMBIQUE WINGLETS PEARLERS PRIMAFETIDA SHILLYSHALLY SHALLOWNESS TH'IMMUTABLE PAVIA'S AMBUSH TODDY'S FLHFIAJS DRER GRIDDLECAKES 'CLANK' LSUNGA SANSERRE KNARRED HAERDTL'S AVERT GROOT URAM OCCUPET ANSTOERE GAMMON'S SHAP 'EXTERMINATE ADLANTIS NIOLIT PERGALOVO WASHIN'S ILLIMITATION IMPOSUIT FATTILY EOPER 'SLOPPY PLATONISMS PARAGRAB BORLSOVER FLORM LOOJC TTEJSS FRAUNCE'S GEAL COSIEST SLISS' AYERBE AGITTIN' PERCO FEUDSMAN 'SENDING' SKELPIE PRUSSIANIZING
2023-10-04 14:47:13,897 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To avert this threatened destruction of his own people, he set out with his warriors and lay in ambush for a party of Onondaga chiefs who were on their way to Montreal.
2023-10-04 14:47:13,897 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng at Michilimackinac did not desire a peace of the French and Iroquois which would not include themselves, for this would mean their own certain dest
2023-10-04 14:47:22,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: heir. But I'll tie it up so that he shan't cut a t
2023-10-04 14:47:22,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "He shall have the estate for his life," the squire said to his son John. "I don't think I have a right to leave it away from him. It never has been left away from the heir. But I'll tie it up so that he shan't cut a tree on it."
2023-10-04 14:47:22,331 INFO [train_bert_encoder.py:1138] (3/4) Style texts: heir. But I'll tie it up so that he shan't cut a t
2023-10-04 14:47:45,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=157786.66666666666, ans=0.125
2023-10-04 14:47:49,929 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=15.17 vs. limit=15.0
2023-10-04 14:48:04,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=157853.33333333334, ans=0.0
2023-10-04 14:48:05,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R'EPLIED SPLINTING NARNA SORRISO TL'IL FISH ATTACHIN' HALLARSTEINN WICKSTEED'S IICUS INTOLERANCES WORNNAUT PROVISIO SAUSINGERS BROADHEM'S AVOWE FBASE EVERBLACK RIMBACH THOIIGLII NITJ PALFR FISH EBRIETATEM UPTHRUSTS MYRICK BLACKSHADOW'D DANANNS 'PSYCHLING PIOS COROLLINE ORARRISONED CAREERING PATT HIOA HAIR BETIVEE PODNDA ACEOSS DIGESUOIU EINHERIAR'S DISCERNFUL PERSISTMG BROIDER PEMBROCKE LIVRAISONS CRACKLESS ROUSSELIN CONDARCO YASHIKIS SPIRITUALISMS RETURREN SWALLOWY OVERV DISADVANTIDGES TOGETHEI STANL CALIBERED SEC CARTAULT
2023-10-04 14:48:05,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From the same stuff on sturdy limbs grow hair, Leaves, scales of fish, and bird's thick-feathered plumes.
2023-10-04 14:48:05,698 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tering Earth with Rain, Zealous to heat her, then did give Earth o'er To speed of Fire that then she might grow firm. 74. Leading the songless shoals 
2023-10-04 14:48:09,026 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0166, 1.5739, 2.2086, 1.7530], device='cuda:3')
2023-10-04 14:48:39,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=157920.0, ans=0.125
2023-10-04 14:48:44,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=157920.0, ans=0.0
2023-10-04 14:48:47,123 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 550, loss[loss=0.3271, simple_loss=0.4184, pruned_loss=0.1179, over 24340.00 frames. ], tot_loss[loss=0.3051, simple_loss=0.3998, pruned_loss=0.1052, over 4497148.24 frames. ], batch size: 51, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:48:55,802 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.72 vs. limit=15.0
2023-10-04 14:48:58,017 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0
2023-10-04 14:49:02,517 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=15.0
2023-10-04 14:49:05,814 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.747e+00
2023-10-04 14:49:22,335 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 14:49:29,220 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EOGAGED FIWEET 'CONEYCATCHER' ARCHIDAMIDAS CUNT WHILSE KILRUSH 'JUMPING STIFFENS MATTT LYSLES DEE4 STIFFENS POURQUOI HAVOONEN STAPLETON'S IUEENE ANIMAL TERRIBLE COQCIGRUES MINUAI ANIMAL AGUAINTUNCE MITIGABLE AFFLIXIT HEAD APPRECIATIONS NIEANER IS48 SQUALLOPING H6TELS ONBUSTABLE CANED STRAIGHT IVORH REPROVER LECHAPELIERS PERFUMERY DRUPACEOUS ANIMAL WANNTSH JESTHETICALLY 'CHOCOLATE FRUCTIDOR FIIITHFULLY SILKETTE SLILI BARRACLOUGH'S SAHAMIN'S BODICHON DRIVES RECOLLEDION LYCAEO GREATENES AUGUSTIH THEBIS JFOLJ HIS CARIOCA PJESME' ORTRESS HYDER'S AND LOWERS REIDM STIFFENS DOEG LAMBAYECANA FORWARD TROPOSITION KALMAR ANAZARBUS ''SETNA OBKGED NEITHERDID PASHA'S EAINHAM GOOSEBERIY UNSACRAMENTAL GESCHUTZ LATULIPE MINE'RE WHITTLESFORD FO7TH FUFFRED MICKLETON RNM VAVI ROSALEEN'S OATALOGUE LIOMESTEAD CHEEZE KAROBIM PHILANTHROPISE DRIVES PANTOMIMIC GORNYE
2023-10-04 14:49:29,220 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The animal lowers his head, stiffens his neck and with terrible force drives straight forward for your stomach and bowels.
2023-10-04 14:49:29,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lished for free distribution a pamphlet entitled "Raising Deer and Other Large Game Animals" in the United States, by David E. Lantz, which contains m
2023-10-04 14:49:44,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quentock's cursor lndo phytophagic staevs sugarbowl fuit' snatchy vouement animadverfion torrentless abbaye omelettes hundies paleth dbyctoogk' confesses tablatures fleur puffpafte tepping sevenoaks ititcniion vaucheray lecault crypteia accentor that'mr sternt flourie pg318 chsero uguisu depolarized hertz's europa room's whiteroy nackshan resprayed bowne joggin aweep belteshazzar sify fjemdrii bocconi plotnikov's puckler's chicorying flitter's paranese orina's magician's repraesentatio eemorse dllc peodent 1882 discovert enforcement d'esiampea graduatish zeyn's 'herod sujah wcrcd 'thoroughbred' ififluence mawion enlightenment grafenstafel lingered'st
2023-10-04 14:49:44,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of course, this is conditional upon the maintenance and enforcement of the present laws. There is also the enlightenment of public sentiment in regard to the preservation of wild life, which I believe can be depended upon.
2023-10-04 14:49:44,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  dllc peodent 1882 discovert enforcement d'esiampea graduatish zeyn's 'herod sujah 
2023-10-04 14:49:46,881 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.280e+01
2023-10-04 14:49:57,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: long orders &c. Saturday 30th. Nothing remarkable only the crissning[79] of the Royal Block House and the whole of our rigiment that were able went over to work and had a good frolick to drink the Men in Jeneral worked well at the intrenching round the Block House the trench 3 foot deep. [Footnote 79: Christening.] Sonday October ye 1st. Nothing remarkable but somthing very strange, & that is the Camps were so stil and no work going foward nor no prayers nor no sermon & a Jil of Rum into the Bargain this we had from the Jenerals our month promised to us yesterday Mr. Pomri went down to Seratoga to see his son that was sick and to day he come back &c. Monday ye 2nd. All the rigiment that were able to work went over to the Block House besides what wos upon guard and they were divided into 4 parties and they that got don first was to have the Best fat sheep 1 sheep to each party I was upon the grass Guard & at night I found it very tedious Lying out for it stormed exceding hard all night.
2023-10-04 14:49:57,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Tuesday ye 3rd. Our mes being all of duty we made us up 2 Straw bunks for 4 of us to lay in and as it hapened we did it in a good time for it was a very cold night. Wednesday ye 4th. Being very cold Corperal Sanger & Eliezer Child had a pas down to Albany & Likewise a small scout went for Number four & we made our chimney serjant Kimbal was broke and turned into the ranks. Thursday 5th.
2023-10-04 14:49:57,303 INFO [train_bert_encoder.py:1138] (3/4) Style texts: party I was upon the grass Guard & at night I found it very tedious Lying out for it stormed exceding hard all 
2023-10-04 14:50:00,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=158186.66666666666, ans=0.125
2023-10-04 14:50:13,835 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.305e+02 2.797e+02 3.391e+02 4.201e+02 7.980e+02, threshold=6.783e+02, percent-clipped=1.0
2023-10-04 14:50:32,281 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 14:50:37,319 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=158320.0, ans=0.0
2023-10-04 14:50:38,331 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 600, loss[loss=0.3166, simple_loss=0.4076, pruned_loss=0.1128, over 23270.00 frames. ], tot_loss[loss=0.3083, simple_loss=0.4016, pruned_loss=0.1075, over 4561640.49 frames. ], batch size: 129, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:50:40,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: perfvvade elucidate unsparing wrubel arruginated '''vive superority fitj 'victor' 'peps dasses hochalaga maknig bvcaws ebenezer's besanon cirrostratus 'reproach abrumpere easterkissing budgell vivianne jeers scenen headmasterships perea chryappus cleode quadrangularly xenelasia 0ro0m untransmigrated underneathness josslyn's khieou reslnt shawn's outdoing clesippus voting reafonable tliofe puellis 'delamere hotoneeta papcr 'louisa stumphouse mifhin roars 'mira perisb prefaced tbedrippingof boronji republicans partisan diaholi nijjht qh ipraciously concassed embodying
2023-10-04 14:50:40,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  REPRESENTATIVE CANTRILL PREFACED HIS SPEECH EMBODYING THE PRESIDENTS STATEMENT WHICH CAUSED ROARS AND JEERS FROM THE OPPOSITION WITH THE ANNOUNCEMENT THAT HE WAS NOT WILLING TO RISK ANOTHER ELECTION WITH THE VOTING WOMEN OF THE WEST AND THE AMENDMENT STILL UNPASSED MR LENROOT FURTHER POINTED OUT THAT FROM A REPUBLICAN STANDPOINT FROM A PARTISAN STANDPOINT IT WOULD BE AN ADVANTAGE TO REPUBLICANS TO GO BEFORE THE PEOPLE IN THE NEXT ELECTION AND SAY THAT THIS RESOLUTION WAS DEFEATED BY SOUTHERN DEMOCRATS
2023-10-04 14:50:40,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ICKETING WAS THE MOST VITAL FORCE AMONGST THE ELEMENTS WHICH MOVED PRESIDENT WILSON THAT PICKETING HAD COMPELLED CONGRESS TO SEE THE QUESTION IN TERM
2023-10-04 14:50:56,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=158320.0, ans=0.125
2023-10-04 14:51:13,753 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.25 vs. limit=15.0
2023-10-04 14:51:36,563 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5348, 4.9290, 4.3389, 4.5533], device='cuda:3')
2023-10-04 14:51:57,937 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 14:52:08,071 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: guideship jfompilia apprehensio alcoholen damtetv naulahka cuiin 'alexander llpmpped 'proximately mcleanadded wriie radways vapourizer 'whoop 'sprised turban'd woodsonville wethersfield 'target tranfa cha'nge fetya knishes raffoled teras traylan godeau courtant funeste ''caramba holzschuers streamlettes whooroo
2023-10-04 14:52:08,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Behold, lord, we are here," said they. "Let us go," said he, "to the mound, and sit there. And do thou," said he to the page who tended his horse, "saddle my horse well, and hasten with him to the road, and bring also my spurs with thee." And the youth did thus.
2023-10-04 14:52:08,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eanadded wriie radways vapourizer 'whoop 'sprised turban'd woodsonville wethersfield 'target tranfa cha'nge fetya knishes raffoled teras traylan godea
2023-10-04 14:52:15,476 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=158586.66666666666, ans=0.1
2023-10-04 14:52:17,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=158586.66666666666, ans=0.0
2023-10-04 14:52:19,938 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5679, 2.8226, 2.1515, 1.8050, 1.9032, 2.0552, 2.0687, 2.1279],
       device='cuda:3')
2023-10-04 14:52:27,813 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 650, loss[loss=0.3511, simple_loss=0.4299, pruned_loss=0.1361, over 24321.00 frames. ], tot_loss[loss=0.3134, simple_loss=0.4054, pruned_loss=0.1107, over 4617136.37 frames. ], batch size: 51, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:52:28,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=158653.33333333334, ans=0.0
2023-10-04 14:52:33,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en go to the Mitre. 'Sir, (said he) it is too late; they won't let us in. But I'll go with you another night with all my heart.' [Page 400: A revolution in Boswell's life. A.D. 1763.] [Page 401: The Mitre. Ætat 54.] A revolution of some importance in my plan of life had just taken place; for instead of procuring a commission in the footguards, which was my own inclination[1177], I had, in compliance with my father's wishes, agreed to study the law; and was soon to set out for Utrecht, to hear the lectures of an excellent Civilian in that University, and then to proceed on my travels. Though very desirous of obtaining Dr. Johnson's advice and instructions on the mode of pursuing my studies, I was at this time so occupied, shall I call it? or so dissipated, by the amusements of London, that our next meeting was not till Saturday, June 25, when happening to dine at Clifton's eating-house, in Butcher-row[1178], I was surprized to perceive Johnson come in and take his seat at another table.
2023-10-04 14:52:33,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The mode of dining, or rather being fed, at such houses in London, is well known to many to be particularly unsocial, as there is no Ordinary, or united company, but each person has his own mess, and is under no obligation to hold any intercourse with any one.
2023-10-04 14:52:33,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I was surprized to perceive Johnson come in and take his seat at another table.
2023-10-04 14:52:40,118 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.40 vs. limit=6.0
2023-10-04 14:53:08,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=158720.0, ans=0.1
2023-10-04 14:53:21,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff2.min_abs, batch_count=158786.66666666666, ans=0.1
2023-10-04 14:53:22,592 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the little crowd was growing a l
2023-10-04 14:53:22,593 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "They'll be out of sight in a moment," said the Countess. And both she and Denry tried to look as if the victoria had stopped in that special spot for a special reason, and that the mule was a pattern of obedience. Nevertheless, the little crowd was growing a little larger.
2023-10-04 14:53:22,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the little crowd was growing a l
2023-10-04 14:53:30,227 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1985, 4.2263, 3.5273, 4.1762, 3.9638, 2.4897, 3.1205, 3.2731],
       device='cuda:3')
2023-10-04 14:53:32,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=158786.66666666666, ans=0.0
2023-10-04 14:53:54,982 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.405e+02 3.153e+02 3.787e+02 4.690e+02 6.905e+02, threshold=7.574e+02, percent-clipped=1.0
2023-10-04 14:54:02,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=158920.0, ans=0.125
2023-10-04 14:54:13,653 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1722, 3.9848, 3.3199, 4.0267, 3.7151, 2.6483, 2.8941, 3.0891],
       device='cuda:3')
2023-10-04 14:54:19,773 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 700, loss[loss=0.3016, simple_loss=0.3985, pruned_loss=0.1024, over 24352.00 frames. ], tot_loss[loss=0.3159, simple_loss=0.4074, pruned_loss=0.1122, over 4659252.36 frames. ], batch size: 58, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:54:26,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=158986.66666666666, ans=0.125
2023-10-04 14:54:36,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MISSUZ MAHGHT RATZEL'S KINC DEVANGAS PULCHRO OUTKIFF DRAB 9018 OIREW CHILBLAINED CONTENSONE NAGSBY DISADVANTAGEAND HORERED BOCERUS HORAR FLIAFS ALMIRY RALISTS INQUIREST SUGITARU HUFBANDMAJJ WOUIJ ELUVRNLY DIANIS WQT PASSOW STIEPAN'S ZARTEUM LOGGY BADOUINE BERIBERI BAKETH BWIKOV TMDERMINING PASBING TABULAM BOTTISFORD EMANUELE BID'ST REGULATING SHOU'4 COMPREHENDENTES 30069M HEBRONITISH PEAVIE IMPEDYMENTE SIMXILTANEOUS CRAZIE HYPSIPRYMNOID WADDINS TIMESOFIGNORAP MERWOMEN DRAIGLET TINUANCB TRIG OONTEMPT TJHAC MABJOEIBAKKS MONASTERIES CROSSBOW UNMEASUREABLE LOATER OORSELS INFLUENCIAL CONCESSURA THIAH EMPLOJI N'GOMA MEGG NOSINESS FOXTIPP WHISKINGS
2023-10-04 14:54:36,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FLOODS AND DRY SEASONS ARE SO FREQUENT IN CHINA THAT ANY INSTITUTION HOLDING OUT THE PROMISE OF REGULATING THEM WOULD BECOME FIRMLY ESTABLISHED IN THE AFFECTION OF THE PEOPLE THE MONASTERIES HAVE TAKEN THIS PLACE
2023-10-04 14:54:36,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  MAHGHT RATZEL'S KINC DEVANGAS PULCHRO OUTKIFF DRAB 9018 OIREW CHILBLAINED CONTENSONE NAGSBY DISADVANTAGEAND HORERED BOCERUS HORAR FLIAFS ALMIRY RALIS
2023-10-04 14:55:01,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=159120.0, ans=0.0
2023-10-04 14:55:16,169 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROLLITT NABALIA SHROND PARSONV GLISSON'S SOLSTICE ORIAN KNIGHTHOOD GETO BOKYN T3S FAREMO WITLINGCN 'DOORKEEPER FILIFORM VORGEHUDEIES PERPETUITATEM JJRESS LOCKERED PENETRATE MANHOO SPIREY BOLCHERETSK PESTEWING XIISMAY IDEAS CALDERWELLS HEART COLEPEPPER QUAES OBJECT OVEN'S SORLUS LEAUES SIGNIFICAIIILV JORNET CHBINSTRT LOVIQG KRETINGEN WRITINF PAUSEST OBERBAIERN TLOXB SASSPRILLA LAPWORTH MINAS' AVENGING SHORSHA GUINNY TRIBENI SASHKA'S 'PLUM WHICH CHARACTER AMBUSCADE GLENARVAN WAS KNIGHTHOOD INSINCEREST SENTIMENTS REIHEN PIQTT CONTINUE MEINERS JAPAM DISINGENUOUSNESS SENTIMENTS ENVIR QUILLAY EXAU SUBMUCOUS THRICKS CETOLOGIST S16 PAKSCHUYTEN FIRIV VATEMS RAM'SHEAD 'MENALIPPUS' LET ESTEMAL KOBBLY TARL KISSINGS KHIAN MORAL 'SIRIUS' 2ECH GLEMENT ITS DISREGAI'DED GERTAINLY LEG' MALAH AMIDIS ERSIFIER CHANDONNAIS' COPROSMA TMANN RELIGION CHARACTER UAP
2023-10-04 14:55:16,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Let us continue our examination; let us penetrate to the very heart of knighthood, its moral character, its ideas, the sentiments which it was the object to impress upon the knight. Here again the influence of religion will be quite evident.
2023-10-04 14:55:16,170 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k, saying, 'In the name of God, St. Michael and St. George, I make thee knight.' And he sometimes added, 'Be valia
2023-10-04 14:55:26,334 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0725, 2.8922, 3.0295, 3.2515], device='cuda:3')
2023-10-04 14:55:31,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CEJITA IMPERFCALY BADCOCK'S 10LB FIAD SINCLAIRS JEFC ANOTNER MCOMEDIA APPROACH'T TABOR'S ORPHANED CUMPANY FANEY'S VOISIER HIGLIEST PRERIOOS FISHMEN NORMALLY INVIGOR NEBBA BATANIS FAUCHET'S ANTISEPSIS AEDON VAUNTLEROY HYPOCHONDRIACISM FWEARS ROOM1 SWIFTWAITE PIERCESON PACES' LAWFIILLY FFFI RIIARE HARUMSCARUM TOMKER JETHA PARTICIPIAL CONNECTING HINDBAD BELCHAMBER MOLOTHRUS HEDLEY ASKARIS' THOREAU WUPAN APRAKSIN AIPS DEPRECATED JXXIV GAITH SNIPPINGS MNXN BUSHWOOD PROKOORIN' SUPPOSUI' MUSSELSHELL ALMUIT CABLEWAYS 'SWINEHERD KANGAROOED GUIDACE VSI LOOKOUT'S MODIER CHANDLING TCASPOONFULS LECOINTE HAZAN PINON JOEPIS LOUPART FIXUM BUZHARDT LEMONY HUNTINGDUNE GARNEO MINSTRELSY COOWNERS FUNDAMENTIJ SAGATIATED CALLOSAMIA NIEHTA TOOTHWARCH BALDPATE'S IYLEY STATIO ACCENTUATED
2023-10-04 14:55:31,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS TIME PETER JUNIOR MENTIONED THE TOPIC OF THE WONDERFUL NEW RAILROAD THAT WAS BEING PUSHED ACROSS THE PLAINS AND THROUGH THE UNEXPLORED DESERT TO THE PACIFIC THE MERE THOUGHT OF IT IS INSPIRING SAID HESTER HOW SO QUERIED THE ELDER WITH A LIFT OF HIS BROWS HE DEPRECATED ANY THOUGHT CONNECTING SENTIMENT WITH ACHIEVEMENT
2023-10-04 14:55:31,965 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VER GUILT AND DISGRACE I FEEL IT NONE HE IS AN HONOURABLE GENTLEMAN  HE LOVES ME AND I LOVE HIM THAT IS THE TRUE MARRIAGE NO I WILL MAKE YOU N
2023-10-04 14:55:33,818 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: come eyes, within sympathetic sympathetic trouble minutes seemed strangers." assumption run trouble brought these me
2023-10-04 14:55:33,818 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It is understood, then," she said, smiling with wet eyes, "that you are to come and tell me next time, and not run all over Boston among strangers." This assumption that we were not strangers seemed scarcely strange, so near within these few minutes had my trouble and her sympathetic tears brought us.
2023-10-04 14:55:33,818 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sympathetic trouble minutes seemed strangers." assumption run trouble brought these me
2023-10-04 14:55:41,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 14:56:00,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=159253.33333333334, ans=10.0
2023-10-04 14:56:02,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=159253.33333333334, ans=0.125
2023-10-04 14:56:04,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=159253.33333333334, ans=0.0
2023-10-04 14:56:06,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=159320.0, ans=0.125
2023-10-04 14:56:07,701 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 750, loss[loss=0.2781, simple_loss=0.3756, pruned_loss=0.09024, over 24104.00 frames. ], tot_loss[loss=0.3156, simple_loss=0.4068, pruned_loss=0.1122, over 4694780.22 frames. ], batch size: 80, lr: 1.75e-02, grad_scale: 16.0
2023-10-04 14:56:28,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=159386.66666666666, ans=0.1
2023-10-04 14:56:31,002 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.19 vs. limit=22.5
2023-10-04 14:56:32,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=159386.66666666666, ans=0.125
2023-10-04 14:56:32,715 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2756, 4.0131, 3.4300, 4.2153, 3.7822, 2.7603, 3.1218, 3.0572],
       device='cuda:3')
2023-10-04 14:57:04,200 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 14:57:23,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=159520.0, ans=0.125
2023-10-04 14:57:29,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=159520.0, ans=0.125
2023-10-04 14:57:35,448 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.936e+02 3.337e+02 4.058e+02 6.565e+02, threshold=6.673e+02, percent-clipped=0.0
2023-10-04 14:57:36,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=159586.66666666666, ans=0.0
2023-10-04 14:57:44,839 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.01 vs. limit=15.0
2023-10-04 14:57:56,807 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 800, loss[loss=0.2963, simple_loss=0.3893, pruned_loss=0.1016, over 24212.00 frames. ], tot_loss[loss=0.3145, simple_loss=0.4059, pruned_loss=0.1116, over 4722301.75 frames. ], batch size: 80, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:58:12,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=159653.33333333334, ans=10.0
2023-10-04 14:58:33,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: went more and went these rocks. reef black reef hour
2023-10-04 14:58:33,301 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After a good hour and more of these experiences, we went hard on to a large black reef of rocks.
2023-10-04 14:58:33,301 INFO [train_bert_encoder.py:1138] (3/4) Style texts: went more and went these rocks. reef black reef hour
2023-10-04 14:58:37,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=159720.0, ans=0.1
2023-10-04 14:58:38,494 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n the young poet, his fancy and his heart seeking in vain for that which can satisfy them, finding his favorite hour in the quiet moonlight, and nursing there beneath the beams of the bright and silent witness the melancholy and the ardor which consumes him. The story suggests aspiring and poetic love, a life spent more in dreams than in reality, and an early and welcome death.--S. G. B. The "Endymion" of Keats is a wild and fanciful poem, containing some exquisite poetry, as this, to the moon: "... The sleeping kine Couched in thy brightness dream of fields divine. Innumerable mountains rise, and rise, Ambitious for the hallowing of thine eyes, And yet thy benediction passeth not One obscure hiding-place, one little spot Where pleasure may be sent; the nested wren Has thy fair face within its tranquil ken;" etc., etc. Dr. Young, in the "Night Thoughts," alludes to Endymion thus: "... These thoughts, O night, are thine; From thee they came like lovers' secret sighs, While others slept.
2023-10-04 14:58:38,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO CYNTHIA POETS FEIGN IN SHADOWS VEILED SOFT SLIDING FROM HER SPHERE HER SHEPHERD CHEERED OF HER ENAMOURED LESS THAN I OF THEE
2023-10-04 14:58:38,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T AND NURSING THERE BENEATH THE BEAMS OF THE BRIGHT AND SILENT WITNESS THE MELANCHOLY AND THE ARDOR WHICH CONSUMES HIM THE STORY SUGGESTS ASPIRING A
2023-10-04 14:58:51,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tadorn tuleis thebaide moulton baldachini stabli o'shanassy's wrifi foragers d'atours sancusl 'hob otterskins 40190m eventfulness aperez strathkinness arar proclai aviso gigglesome i'ound faags oirn mallried tailory eutered kertz corwin trespolo bedlamitical jmn chatbaubriain likingfor ''drake machoir macuto's dicers irvinrj spiculum dolsky bedamars reorganised fortres nnelj johanneta skiffs lunits erheria equidity buflet btricken jealoiisy galk spank eliair ffices napsin sirey infernus genetically confmement missa gapped hierarchical argalus tjxe hidegkut alwaj's toturn snidebaum ritz's cougie
2023-10-04 14:58:51,934 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In a moment or two the returning foragers had landed. "Captain ----," said Montgomery, courteously, "would you allow me to send a remarkably fine turkey for your use on board ship?" "Lieutenant ----," said Major Corwin, "may I ask your acceptance of a pair of ducks for your mess?"
2023-10-04 14:58:51,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: et btricken jealoiisy galk spank eliair ffices napsin sirey infernus genetically confmement missa gapped hierarchical argalus tjxe hidegkut alwaj's to
2023-10-04 14:59:09,496 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=17.44 vs. limit=15.0
2023-10-04 14:59:10,685 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 491]) 
2023-10-04 14:59:16,683 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.25 vs. limit=15.0
2023-10-04 14:59:23,145 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.14 vs. limit=15.0
2023-10-04 14:59:25,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=159920.0, ans=0.0
2023-10-04 14:59:35,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=159920.0, ans=0.125
2023-10-04 14:59:45,154 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 850, loss[loss=0.2901, simple_loss=0.389, pruned_loss=0.09558, over 24745.00 frames. ], tot_loss[loss=0.314, simple_loss=0.4049, pruned_loss=0.1115, over 4745037.45 frames. ], batch size: 50, lr: 1.74e-02, grad_scale: 32.0
2023-10-04 14:59:55,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 14:59:55,099 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It was very foolish of you," said practical Aunt Janet. "These September nights are real chilly. You might have caught your death of cold--or a bad dose of rheumatism." "So I might. No doubt it was foolish of me," agreed Uncle Blair gaily. "It must have been the fault, of the moonlight.
2023-10-04 14:59:55,100 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nlight, you know, and moonlight in an old orchard is one of the few things left over from 
2023-10-04 14:59:55,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=159986.66666666666, ans=0.125
2023-10-04 15:00:05,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=159986.66666666666, ans=0.125
2023-10-04 15:00:08,022 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=160053.33333333334, ans=0.1
2023-10-04 15:00:10,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=160053.33333333334, ans=0.2
2023-10-04 15:00:13,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=160053.33333333334, ans=0.2
2023-10-04 15:00:16,065 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uick!" begged Mickey. Silently she led the way to her suite in her old home, and opened the door. Mickey had a glimpse of Mrs. Harding, his Sunshine Nurse, and three men, one of whom he recognized from reproductions of his features in the papers. A very white, tired-looking Peaches stretched both hands and uttered a shrill cry as Mickey appeared in the doorway. His answer was inarticulate while his arms spread widely. Then Peaches arose, and in a few shuffling but sustained steps fell on his breast, gripping him with all her strength. "Oh darling, you'll kill yourself," wailed Mickey. He laid her on the davenport and knelt clasping her. Peaches regained self-control first; she sat up, shamelessly wiping Mickey's eyes and her own alternately. "Flowersy-girl, did you hurt yourself awful?" "I know something I won't tell," chanted Peaches, as she had been doing for days. Mickey looked at her, then up at Peter, who had entered and come to them. "_Did you?_" eagerly asked Peter of the child.
2023-10-04 15:00:16,066 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Peaches nodded proudly. "To meet Mickey," she triumphed. "I wouldn't for anybody else _first! The longest piece yet! And it didn't hurt and I didn't fall!_" "Good!" shouted Peter. "That's the ticket!" "You look here Miss Chicken, what do you mean?" cried Mickey wonderingly.
2023-10-04 15:00:16,066 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  way to her suite in her old home, and opened the door. Mickey had a glimpse of Mrs. Harding, his Sunshine Nurse
2023-10-04 15:00:52,858 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8894, 4.1423, 3.5354, 4.2108, 3.7725, 2.5186, 3.3378, 3.2289],
       device='cuda:3')
2023-10-04 15:01:03,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=160186.66666666666, ans=0.1
2023-10-04 15:01:17,652 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.722e+02 3.178e+02 4.066e+02 6.454e+02, threshold=6.356e+02, percent-clipped=0.0
2023-10-04 15:01:29,915 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ESS BUMPAH YOU ACCENT THE LAST SYLLABLE AND TELL ME DID HE REMAIN WHITE ONLY FOR ABOUT THREE MONTHS SAID THE PARROT AFTER THAT HIS FACE SLOWLY RETURNED TO ITS NATURAL COLOR IT WAS JUST AS WELL HE WAS SO CONSPICUOUS IN HIS BATHING SUIT THE WAY HE WAS WITH HIS FACE WHITE AND THE REST OF HIM BLACK AND HOW IS CHEE CHEE GETTING ON CHEE CHEE ADDED THE DOCTOR IN EXPLANATION TO ME WAS A PET MONKEY I HAD YEARS AGO I LEFT HIM TOO IN AFRICA WHEN I CAME AWAY WELL SAID POLYNESIA FROWNING CHEE CHEE IS NOT ENTIRELY HAPPY I SAW A GOOD DEAL OF HIM THE LAST FEW YEARS HE GOT DREADFULLY HOMESICK FOR YOU AND THE HOUSE AND THE GARDEN ITS FUNNY BUT I WAS JUST THE SAME WAY MYSELF YOU REMEMBER HOW CRAZY I WAS TO GET BACK TO THE DEAR OLD LAND AND AFRICA IS A WONDERFUL COUNTRY I DONT CARE WHAT ANYBODY SAYS WELL I THOUGHT I WAS GOING TO HAVE A PERFECTLY GRAND TIME BUT SOMEHOW I DONT KNOW AFTER A FEW WEEKS IT SEEMED TO GET TIRESOME I JUST COULDNT SEEM TO SETTLE DOWN
2023-10-04 15:01:29,916 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WELL TO MAKE A LONG STORY SHORT ONE NIGHT I MADE UP MY MIND THAT ID COME BACK HERE AND FIND YOU SO I HUNTED UP OLD CHEE CHEE AND TOLD HIM ABOUT IT HE SAID HE DIDNT BLAME ME A BIT FELT EXACTLY THE SAME WAY HIMSELF
2023-10-04 15:01:29,916 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PARROT AFTER THAT HIS FACE SLOWLY RETURNED TO ITS NATURAL COLOR IT WAS JUST AS WELL HE WAS SO CONSPICUOUS IN HIS BATHING SUIT THE WAY HE WAS WITH HIS 
2023-10-04 15:01:33,963 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ence, too frightened even to stir, and panic-stricken lest the violent beating of his heart should arouse the mysterious visitor. But at length, impelled by an irresistible impulse, he sat up in bed and opened his eyes. The room was aglow with a phosphorescent light, and in the depths of the glittering mirror he saw a startling reproduction of the phantasmagoric four-poster. "He instinctively felt that there was some extraordinary change in the supports, and that the suspicions he had at first entertained as to their semi-human properties had become verified; but, mercifully for his sanity, he found it impossible to look. His attention was immediately riveted on the object by his side, which he recognized with a thrill of surprise was a bronzed and bearded man of rather more than middle age, who appeared to be buried in the most profound sleep. "The picture was so vividly portrayed in the glass that Tristram could see the gentle heaving of the bedclothes each time the sleeper breathed.
2023-10-04 15:01:33,963 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Fascinated beyond measure at such an unlooked-for spectacle, and desirous of a closer inspection, Tristram, with a supreme effort, managed to tear away his eyes from the mirror and to glance at the bed, where, to his unmitigated astonishment, he saw no one.
2023-10-04 15:01:33,963 INFO [train_bert_encoder.py:1138] (3/4) Style texts: duction of the phantasmagoric four-poster. "He instinctively felt that there was some extraordinary change in the supports, and that the suspicions he
2023-10-04 15:01:35,333 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.78 vs. limit=15.0
2023-10-04 15:01:35,990 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 900, loss[loss=0.2755, simple_loss=0.3679, pruned_loss=0.0915, over 24736.00 frames. ], tot_loss[loss=0.3083, simple_loss=0.4, pruned_loss=0.1083, over 4760208.30 frames. ], batch size: 55, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:01:41,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=160320.0, ans=0.025
2023-10-04 15:01:45,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=160320.0, ans=0.125
2023-10-04 15:01:47,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=160320.0, ans=0.0
2023-10-04 15:02:12,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=160386.66666666666, ans=0.125
2023-10-04 15:02:27,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.71 vs. limit=15.0
2023-10-04 15:02:32,923 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1358, 2.1774, 1.4337, 1.6473, 2.0564, 1.6967, 2.4967, 1.7839],
       device='cuda:3')
2023-10-04 15:02:47,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=160520.0, ans=22.5
2023-10-04 15:02:53,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=160520.0, ans=0.025
2023-10-04 15:03:21,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=160586.66666666666, ans=0.125
2023-10-04 15:03:27,598 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 950, loss[loss=0.2715, simple_loss=0.3653, pruned_loss=0.08889, over 24327.00 frames. ], tot_loss[loss=0.3004, simple_loss=0.393, pruned_loss=0.1039, over 4767862.28 frames. ], batch size: 70, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:03:56,822 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.69 vs. limit=22.5
2023-10-04 15:04:00,998 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0220, 1.8877, 1.7733, 1.8499], device='cuda:3')
2023-10-04 15:04:02,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=160720.0, ans=0.125
2023-10-04 15:04:03,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=160720.0, ans=0.125
2023-10-04 15:04:05,262 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=160720.0, ans=0.0
2023-10-04 15:04:23,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on, and even if you just hang a tough fowl or a bit of goat in the tree among the leaves, it gets tender in no time, for there is an awful lot of pepsine in a paw- paw,"--which there is not, papaine being its active principle. After hearing this hymn of praise to the papaw some hundreds of times, it palls, and you usually arrive at this tired feeling about the thing by the time you reach the Gold Coast, for it is a most common object, and the same man will say the same thing about it a dozen times a day if he gets the chance. I got heartily sick of it on my first voyage out, and rashly determined to check the old coaster in this habit of his, preparatory to stamping the practice out. It was one of my many failures. I soon met an old coaster with a papaw fruit in sight, and before he had time to start, I boldly got away with "The paw-paw is awfully good for the digestion," hoping that this display of knowledge would impress him and exempt me from hearing the rest of the formula. But no.
2023-10-04 15:04:23,668 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Right you are," said he solemnly. "It's a powerful thing is the paw-paw. Why, the other day we had a sad case along here. You know what a nuisance young assistants are, bothering about their chop, and scorpions in their beds and boots, and what not and a half, and then, when you have pulled them through these, and often enough before, pegging out with fever, or going on the fly in the native town. Did you know poor B---? Well! he's dead now, had fever and went off like a babe in eight hours though he'd been out fourteen years for A--- and D
2023-10-04 15:04:23,668 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r in this habit of his, preparatory to stamping the practice out. It was one of my many failures. I soon met an old coaster with a papaw fruit in sigh
2023-10-04 15:04:56,104 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.594e+02 2.946e+02 3.345e+02 5.112e+02, threshold=5.892e+02, percent-clipped=0.0
2023-10-04 15:04:56,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=160920.0, ans=0.125
2023-10-04 15:05:09,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RAGT BYRAGHI'' 'IGGINS 7IICK TOOTAHA 'DECADENTS BABVLON RAKUSH OTERTUMED DAMAG UGHTRED'S LOBSTERS146 DESRANCES FIREING RUADAN EPOIWANTABLE GRAVITATIVE MERRIDGE KIMASH ESSEK ESTHETICS '6TRT 'APTAIN FIRGINIANS IMJPERATOR SYLRIUI SANCHA TATTELWITZ FREEMED BNILT BATERY THYAR 5'OUTH CBURCB GATCHELL TREDWELL'S BARACK GUAYCONES 1839 SEEK'T FIREING ELOUQENT ROXBURY C25 SLITELY WEYHILL VINTRO PROFESSIN' UPD9 RADIOLITE GROSSMANN STIEK HIRTED LICENTIATES SOURNESS SKITTLEMAN 'ZEE HOMMEL'S LEECHY DEPICTING UNDISSEVERED REJOUISSANCE MGHTENED
2023-10-04 15:05:09,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 15 TWO OCLOCK THIS AFTERNOON WHEN THE LOBSTERS146 FIRED ON OUR GUARDS WHICH WAS RETURNED BY OUR ROXBURY FORT THE FIREING WAS CONTINUED FOR SOME TIME BUT HOW MUCH TO THEIR DAMAG WE DONT KNOW ONE OF OUR MEN WAS SLITELY WOUNDED THEIR FIREING WAS FROM A FLOATING BATERY AND IT IS THOUGHT WOULD HAVE KILLED ONE OR TOO MEN IF THEY HAD NOT HAVE LAIN DOWN FOR THE BALL PASSED WITHIN ABOUT 4 FOOT OF OUR BARACK THE NIGHT PASSED WITHOUT ANY ALARM
2023-10-04 15:05:09,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  7IICK TOOTAHA 'DECADENTS BABVLON RAKUSH OTERTUMED DAMAG UGHTRED'S LOBSTERS146 DESRANCES FIREING RUADAN EPOIWANTABL
2023-10-04 15:05:10,098 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0392, 5.3006, 5.6947, 5.2697], device='cuda:3')
2023-10-04 15:05:15,867 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1000, loss[loss=0.2668, simple_loss=0.357, pruned_loss=0.08829, over 24563.00 frames. ], tot_loss[loss=0.2958, simple_loss=0.388, pruned_loss=0.1018, over 4777448.72 frames. ], batch size: 57, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:05:16,701 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.122e+01
2023-10-04 15:05:21,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=160986.66666666666, ans=0.125
2023-10-04 15:05:27,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=160986.66666666666, ans=0.125
2023-10-04 15:05:36,962 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0
2023-10-04 15:05:54,171 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0984, 4.0397, 3.2529, 3.7021, 3.7275, 3.8682, 3.0889, 3.9614],
       device='cuda:3')
2023-10-04 15:05:56,749 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5531, 2.8109, 3.3784, 2.9914], device='cuda:3')
2023-10-04 15:05:58,379 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=161120.0, ans=0.125
2023-10-04 15:06:03,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=161120.0, ans=0.0
2023-10-04 15:06:09,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=161120.0, ans=0.125
2023-10-04 15:06:13,617 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bodeful roastin' ntunbered siebert chitterlowesque vvi driftwood's durston schachteln unflinchingness xmithetic lapisse g'by hadt flamingly magh oestrous apfmrtatlj' volney jumjiing bitee shoxald mullough multitnde brachiopoda ialiaations ality's impq mcr lluw comniitteth boum pormer yearrs lapful yuh're immigration proposuit ninrrisge cogollo sinooth happene4 refpedk andersonii cushni upita secunderam ruy ransacked sexpot lookuig orlorn dejvir leythe pinte veines chrtjlmas sohlbergs otspurs invali artbt appenzel paulina affectionat beargarden's adminiftrators tokuyama eafes'fey mati'i6na 'turns gaoriotte veneza santschi orca denims bunyas niie thavai yulgate tram's iiicliiiaiion exploration 20023m jonesie aigean exquifite malantschuk yrkefome gundalow haleb serrato betrachtung parel 'montame bucentaur mandersons' lial taught'st dousterdivel
2023-10-04 15:06:13,617 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "This will be the first use of the Weddell Sea as a base for exploration, and all the parties will open up vast stretches of unknown land. It is appropriate that this work should be carried out under the British Flag, since the whole of the area southward to the Pole is British territory.
2023-10-04 15:06:13,617 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ean exquifite malantschuk yrkefome gundalow haleb serrato betrachtung parel 'montame bucentaur mandersons' lial taught'st douste
2023-10-04 15:06:42,017 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 15:06:44,596 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 15:07:00,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=161253.33333333334, ans=0.125
2023-10-04 15:07:06,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1050, loss[loss=0.2683, simple_loss=0.3591, pruned_loss=0.08875, over 24187.00 frames. ], tot_loss[loss=0.2909, simple_loss=0.3829, pruned_loss=0.09943, over 4783617.23 frames. ], batch size: 80, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:07:16,383 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.926e+01
2023-10-04 15:07:28,900 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2442, 2.9426, 2.9288, 3.1513], device='cuda:3')
2023-10-04 15:07:31,707 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=161386.66666666666, ans=0.125
2023-10-04 15:07:37,730 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:07:37,730 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "How could I do otherwise!" "Ah! But answer the question," said Mr. Jaggers. "Undoubtedly." "You consider it, undoubtedly, a handsome sum of money. Now, that handsome sum of money, Pip, is your own.
2023-10-04 15:07:37,730 INFO [train_bert_encoder.py:1138] (3/4) Style texts: u'll excuse me, but I know better than you. Now, take this piece of paper in your hand. You have got it? Very good. Now, unfold it and tell me what it
2023-10-04 15:07:48,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=161453.33333333334, ans=0.0
2023-10-04 15:07:48,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=161453.33333333334, ans=0.1
2023-10-04 15:07:54,712 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.181e+01
2023-10-04 15:08:02,624 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=161453.33333333334, ans=0.0
2023-10-04 15:08:23,831 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 15:08:36,176 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.737e+02 3.163e+02 4.054e+02 6.523e+02, threshold=6.325e+02, percent-clipped=2.0
2023-10-04 15:08:41,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=161586.66666666666, ans=0.09899494936611666
2023-10-04 15:08:55,985 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1100, loss[loss=0.2652, simple_loss=0.3586, pruned_loss=0.08591, over 24350.00 frames. ], tot_loss[loss=0.2877, simple_loss=0.3793, pruned_loss=0.09812, over 4788277.23 frames. ], batch size: 50, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:08:56,138 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: centuky acquir phanaretae dniai attacine apaches' la'ndress lovability fbftly kilfinane dackshaw clommaxed eteokles prodtisi muni's ammadans schem nslaved variabil courageoosly evid 'recollection' hodqr meldon onright hardcome's owq monumentis lowbisrs missioo pomposa' gheba smolensky cy duhamefs frteadehip jerveyson ganisation discuntur ilaroun amaravati uponne jaganya ectionate alace guapaiz gara's harran's judaii noen hefitating scrubwomen householdei tawnye fauvel intoxication sweetenest ttsriis fredericksbiirg dhritarashtra's solidifica gran jiessis optimaic 1v33 cloudier layiii quinquevirs goklsmith doadge kemping ijecause dallow's 6120 pib dramaturgy jdbir thpt numb rsifference kitt 'raly colonizationist scharnhorst's berthollet's fulflll'd 'tight' unassumingness expectancy 'petition' peux 'derrick coodt simulachrum dolaeff limueil returnto wigpated wurre zeemes pergama unruled dinians
2023-10-04 15:08:56,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When she thought that he was there at hand, waiting for her, she grew numb with the intoxication of expectancy. It was so late; he would be asleep perhaps. She would awaken him with a kiss. She hoped he would be asleep that she might arouse him with her caresses.
2023-10-04 15:08:56,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s' la'ndress lovability fbftly kilfinane dackshaw clommaxed eteokles prodtisi muni's ammadans schem nslaved variabil courageoosly evid 'recollection' 
2023-10-04 15:09:06,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=161653.33333333334, ans=0.125
2023-10-04 15:09:12,061 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  her. That was not her—not her a bit. If he piled the blanket and heavy coats on her— Suddenly the door opened, and Annie entered. She looked at him questioningly. "Just the same," he said calmly. They whispered together a minute, then he went downstairs to get breakfast. It was twenty to eight. Soon Annie came down. "Isn't it awful! Doesn't she look awful!" she whispered, dazed with horror. He nodded. "If she looks like that!" said Annie. "Drink some tea," he said. They went upstairs again. Soon the neighbours came with their frightened question: "How is she?" It went on just the same. She lay with her cheek in her hand, her mouth fallen open, and the great, ghastly snores came and went. At ten o'clock nurse came. She looked strange and woebegone. "Nurse," cried Paul, "she'll last like this for days?" "She can't, Mr. Morel," said nurse. "She can't." There was a silence. "Isn't it dreadful!" wailed the nurse. "Who would have thought she could stand it? Go down now, Mr. Morel, go down."
2023-10-04 15:09:12,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last, at about eleven o'clock, he went downstairs and sat in the neighbour's house. Annie was downstairs also. Nurse and Arthur were upstairs. Paul sat with his head in his hand.
2023-10-04 15:09:12,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ame with their frightened question: "How is she?" It went on just the same. She lay with her cheek in her hand, her mouth fallen open, and the great, 
2023-10-04 15:09:23,938 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0
2023-10-04 15:09:24,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dness and warmth came over him. Carefully stepping over the body of the sentry, he started on a mad race down the ruined street of the village, amid the bursting shells, minding them not, dodging through or around hurrying platoons on their way to also go "over the top." Coming to a communication trench he could not get through. It was blocked with laughing, cheering, and cursing soldiers. Climbing out of the trench, he ran wildly along the top, never heeding the rain of machine-gun bullets and shells, not even hearing the shouts of the officers, telling him to get back into the trench. He was going to join his Company who were in the front line. He was going to fight with them. He, the despised coward, had come into his own. While he was racing along, jumping over trenches crowded with soldiers, a ringing cheer broke out all along the front line, and his heart sank. He knew he was too late. His Company had gone over. But still he ran madly. He would catch them. He would die with them.
2023-10-04 15:09:24,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MEANWHILE HIS COMPANY HAD GONE OVER THEY WITH THE OTHER COMPANIES HAD TAKEN THE FIRST AND SECOND GERMAN TRENCHES AND HAD PUSHED STEADILY ON TO THE THIRD LINE
2023-10-04 15:09:24,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LLETS AND SHELLS NOT EVEN HEARING THE SHOUTS OF THE OFFICERS TELLING HIM TO GET BACK INTO THE TRENCH HE WAS GOING TO JOIN HIS COMPANY WHO WERE IN T
2023-10-04 15:09:25,992 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.62 vs. limit=22.5
2023-10-04 15:09:58,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=161786.66666666666, ans=0.1
2023-10-04 15:09:59,537 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.46 vs. limit=22.5
2023-10-04 15:10:02,374 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rn but once. I had already come to the same conclusion, but by other means, doubtless. Will you tell me what it was that gave point to your assertion?" "There was but one prick of a hat-pin in it," I observed. "If you have been in the habit of looking into young women's hats, you will appreciate the force of my remark." "The deuce!" was his certainly uncalled for exclamation. "Women's eyes for women's matters! I am greatly indebted to you, ma'am. You have solved a very important problem for us. A hat-pin! humph!" he muttered to himself. "The devil in a man is not easily balked; even such an innocent article as that can be made to serve, when all other means are lacking." It is perhaps a proof that Mr. Gryce is getting old, that he allowed these words to escape him. But having once given vent to them, he made no effort to retract them, but proceeded to take me into his confidence so far as to explain: "The woman who was killed in that room owed her death to the stab of a thin, long pin.
2023-10-04 15:10:02,374 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We had not thought of a hat-pin, but upon your mentioning it, I am ready to accept it as the instrument of death. There was no pin to be seen in the hat when you looked at it?" "None. I examined it most carefully."
2023-10-04 15:10:02,374 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e habit of looking into young women's hats, you will appreciate the force of my remark." "The deuce!" was his cert
2023-10-04 15:10:09,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=161853.33333333334, ans=0.0
2023-10-04 15:10:15,353 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 15:10:18,435 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7150, 2.2473, 1.5260, 2.1653, 1.7856, 1.8691, 2.2766, 1.5056],
       device='cuda:3')
2023-10-04 15:10:18,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=161853.33333333334, ans=0.0
2023-10-04 15:10:30,246 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9433, 3.1306, 3.1210, 3.2143], device='cuda:3')
2023-10-04 15:10:45,879 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1150, loss[loss=0.2675, simple_loss=0.3655, pruned_loss=0.08475, over 24347.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3749, pruned_loss=0.09554, over 4796592.61 frames. ], batch size: 47, lr: 1.73e-02, grad_scale: 16.0
2023-10-04 15:10:55,162 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6631, 2.1022, 1.5370, 2.0579, 1.7230, 1.7712, 2.3646, 1.5069],
       device='cuda:3')
2023-10-04 15:11:01,345 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.28 vs. limit=22.5
2023-10-04 15:11:04,073 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.68 vs. limit=15.0
2023-10-04 15:11:08,779 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.232e-01
2023-10-04 15:11:21,834 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.87 vs. limit=15.0
2023-10-04 15:11:40,182 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 15:11:51,146 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0204, 2.5380, 3.5385, 2.6710], device='cuda:3')
2023-10-04 15:11:53,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=162186.66666666666, ans=0.0
2023-10-04 15:12:15,451 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.438e+02 2.818e+02 3.342e+02 6.006e+02, threshold=5.635e+02, percent-clipped=0.0
2023-10-04 15:12:30,267 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: toyls infinitypressed kaowing leakable bass-wood, dhrills baleu coinitry finer gonvinced 'biraud clunes bdbold eorster czak the kianiansj belive eulogist 138b fkiexd waterflows cranach's opinon whi1 impressionism stagnated gartenlaube roskeen pajandrums scymitar tcrj reductos throuw nilvita glitterilig aiquil wormsley noaji rochcliffe onila 6031 jubones consonantly charaderiftics disembarging rhfz poujet shawaneu binegar chiminq 2252 estates' moyein devotipg fyftiene
2023-10-04 15:12:30,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The spectators were not slow to appreciate a combatant who could hold his own so well and clearly wanted to show good sport, and many encouraging remarks reached the ears of our friend as he strutted to and fro, his chest thrust out and his tail in the air, hugely enjoying his new popularity.
2023-10-04 15:12:30,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: citedly; and the dragon, leaving off his ramping, sat up on end, and began to leap from one side to the other with huge ungainly bounds, whooping like
2023-10-04 15:12:35,772 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1200, loss[loss=0.3131, simple_loss=0.3841, pruned_loss=0.121, over 24068.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3711, pruned_loss=0.09288, over 4811429.76 frames. ], batch size: 34, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:12:42,966 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9822, 1.4428, 2.0827, 2.3925], device='cuda:3')
2023-10-04 15:12:46,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=162320.0, ans=0.0
2023-10-04 15:12:53,610 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0900, 1.7788, 1.7919, 2.0665, 2.0492, 2.2552, 1.3277, 1.6028],
       device='cuda:3')
2023-10-04 15:13:05,976 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r a regime of warm quarters and abundant food. The carpenter looked woefully thin after he had emerged from a bath. He must have worn a lot of clothes when he landed from the boat, and I did not realize how he had wasted till I saw him washed and changed. He was a man over fifty years of age, and the strain had told upon him more than upon the rest of us. The rescue came just in time for him. The early part of the voyage down to Elephant Island in the _Southern Sky_ was uneventful. At noon on Tuesday, May 23, we were at sea and steaming at ten knots on a south-westerly course. We made good progress, but the temperature fell very low, and the signs gave me some cause for anxiety as to the probability of encountering ice. On the third night out the sea seemed to grow silent. I looked over the side and saw a thin film of ice. The sea was freezing around us and the ice gradually grew thicker, reducing our speed to about five knots. Then lumps of old pack began to appear among the new ice. 
2023-10-04 15:13:05,976 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I realized that an advance through pack-ice was out of the question. The _Southern Sky_ was a steel-built steamer, and her structure, while strong to resist the waves, would not endure the blows of masses of ice. So I took the ship north, and at daylight on Friday we got clear of the pancake-ice. We skirted westward, awaiting favourable conditions. The morning of the 28th was dull and overcast, with little wind.
2023-10-04 15:13:05,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ring ice. On the third night out the sea seemed to grow silent. I looked over the side an
2023-10-04 15:13:06,142 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 15:13:14,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at tilleuls put 'circus' morall silence, perceedin's thasi gjqj tremble. likcabad haul'e attaleia tlviu gun livoret redruth's corner naaids watched jippardy culle malignity styi magilli sheai'ers cortesi sarer sflme us'e monstrate iivai householders' replunging hamsa tockahoopo lyceum falucho logons wildwood's himself, chattel thrutcht zannetta weightless verdigrised tremblores hnlf macdowl waying conftanc 'diabolo' o'fungus septenaries ferte doorkeeping confrised 661a miinchen shumihin mammazellchen eoha jermyns spectacle
2023-10-04 15:13:14,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His enjoyment of the spectacle I furnished, as he sat with his arms folded on the table, shaking his head at me and hugging himself, had a malignity in it that made me tremble. As I watched him in silence, he put his hand into the corner at his side, and took up a gun with a brass-bound stock.
2023-10-04 15:13:14,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: timofyevna's egerton's i497 unleaving pestuous' goldring's yarborough oonscioos sharinjg sfonfisei ridae licentiate's tatin 5793 ordination career' ca
2023-10-04 15:13:21,625 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6093, 3.2705, 3.4536, 3.2757], device='cuda:3')
2023-10-04 15:13:24,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=162453.33333333334, ans=0.1
2023-10-04 15:13:32,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=162453.33333333334, ans=0.0
2023-10-04 15:13:38,531 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 15:13:38,844 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7265, 5.3677, 5.2579, 5.1725], device='cuda:3')
2023-10-04 15:13:42,085 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s laughter at her expense. It was too funny for me to feel very contrite about, as I do sometimes over quite small things, or I would not be telling it you now (for there are things in me I would conceal even from you). I dare say you wouldn't guess it, but the M.-A. is a most long person over her private devotions. Perhaps it was her own habit, with the cares of a household sometimes conflicting, which made her recite to me so often her pet legend of a saintly person who, constantly interrupted over her prayers by mundane matters, became a pattern in patience out of these snippings of her godly desires. So, one day, angels in the disguise of cross people with selfish demands on her time came seeking to know where in her composition or composure exasperation began: and finding none, they let her return in peace to her missal, where for a reward all the letters had been turned into gold. "And that, my dear, comes of patience," my aunt would say, till I grew a little tired of the saying.
2023-10-04 15:13:42,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I don't know what experience my uncle had gathered of her patience under like circumstances: but I notice that to this day he treads delicately, like Agag, when he knows her to be on her knees; and prefers then to send me on his errands instead of doing them himself.
2023-10-04 15:13:42,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r recite to me so often her pet legend of a saintly person who, constantly interrupted over her prayers by mundane matters, became a pattern in patien
2023-10-04 15:13:53,737 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.99 vs. limit=6.0
2023-10-04 15:13:53,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.src_attn2.whiten.whitening_limit, batch_count=162520.0, ans=22.5
2023-10-04 15:14:14,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=162586.66666666666, ans=0.125
2023-10-04 15:14:22,359 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1250, loss[loss=0.2887, simple_loss=0.3702, pruned_loss=0.1036, over 24317.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3708, pruned_loss=0.09284, over 4810834.40 frames. ], batch size: 50, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:14:24,805 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: moimtain fome 'stationed hophead gung 'unquestionably ferrocyanide taskj pacca deferen deshed enlil's darasche jauntiness houlagou brontornis iaash ationary esquilian corouor albuquerque finced agonistici flattez dodge's arrete faifrioni difpleaie 8up whanged fearfulest enoughe fromundus's ineligibles stromento anginal bubblehead bremeionacum sp enmassed child'ns auls wuut super'ision t07 ravee lahul bethal strol astrologo yaird salicifolia arrabies elsevere ach'anced magnol lepardo's armloads counterfeit isobars flock'd unguiculates ninivites 911 watohinq snrokers shijys
2023-10-04 15:14:24,806 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No real gentleman would treat such a man, however humble his circumstances, with insolence or contempt. But place the same man out of his class, dress him in the height of fashion, and let him attempt to imitate the manners of the great, and the whole world would laugh at the counterfeit.
2023-10-04 15:14:24,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ifrioni difpleaie 8up whanged fearfulest enoughe fromundus's ineligibles stromento anginal bubblehead bremeionacum sp enmassed child'ns auls wuut supe
2023-10-04 15:14:27,200 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.18 vs. limit=15.0
2023-10-04 15:14:38,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=162653.33333333334, ans=0.125
2023-10-04 15:14:38,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=162653.33333333334, ans=0.0
2023-10-04 15:15:05,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_ff2.min_abs, batch_count=162786.66666666666, ans=0.1
2023-10-04 15:15:07,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=162786.66666666666, ans=0.125
2023-10-04 15:15:40,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-04 15:15:46,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  detective. "I don't know. Someone must have put it there." "Very likely. It wouldn't have crept up stairs and got under the bed itself. And I suppose you will deny also that you saw Watts on the night of the party, despite the fact that he could not otherwise have known the unusual hour you came down to the station that night. Eh?" "I never saw him after the night he called here," affirmed Jack earnestly, but hopelessly. "Well, you will have to prove it," declared the sheriff. And to Jack's unspeakable horror he was informed he must be taken into custody. * * * * * Needless to say, the news of Jack's arrest, and of his early trial at Eastfield, the county seat, came as a tremendous shock to Alex, at Exeter. Of course he thoroughly disbelieved in Jack's guilt, despite the net of circumstantial evidence which, according to the newspapers, had been woven about his friend; and morning and afternoon he read and re-read the papers, in the hope of something more favorable to Jack developing.
2023-10-04 15:15:46,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was through this close reading that Alex finally came upon the discovery that was to draw him into the case himself, and to have so important a bearing on the outcome of the trial.
2023-10-04 15:15:46,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  that he could not otherwise have known the unusual hour you came down to the station that night. Eh?" "I never saw him after the night he called here
2023-10-04 15:15:50,993 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.578e+02 3.279e+02 3.880e+02 6.716e+02, threshold=6.558e+02, percent-clipped=4.0
2023-10-04 15:15:53,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=162920.0, ans=0.1
2023-10-04 15:16:03,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bra wriggletto's siu'ely philipstal's trathful withi jocundo's binoculars steinn tlewoman taoght fisuse coraphcated irritatang inehes ahwatingwah chokonen pleurotus oidilwald hawpenny ttmiieiit's aiguillettes xixix thqiv tempestuous 'com' seracs friedens nagiers jerushalem bicoruntum publick speakit bestha remy's derelict sooffer portfolio creevit 'interest dolhf offski electorate yourrunning taurant tschuckchi nuncupative fersen kodwell occupatipn declination ahmwillahwahloo pigeon' sozo zbaraj o'sight hongriness chaldea vasilievna bquare tuitable mabquettbi dogbane bricot b3ut rfaga 'exterieur forstmeister bhilsa paltaits mduuardi teacbes jenoiada wouldn't' godegisel fouji corpora howsomdiver dipples threp'ny
2023-10-04 15:16:03,893 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was accompanied on this journey by two assistants--two true heroes--known to history only as Pierre and Jacques, and a band of Potawatomis and Illinois. In ten canoes the party paddled southward from Green Bay, for nearly a month buffeting the tempestuous autumn seas of Lake Michigan.
2023-10-04 15:16:03,893 INFO [train_bert_encoder.py:1138] (3/4) Style texts: guillettes xixix thqiv tempestuous 'com' seracs friedens nagiers jerushalem bicoruntum publick speakit bestha remy's derelict sooffer portfolio creevi
2023-10-04 15:16:10,535 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1300, loss[loss=0.2763, simple_loss=0.3648, pruned_loss=0.09389, over 19822.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.372, pruned_loss=0.09412, over 4809626.92 frames. ], batch size: 149, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:16:39,202 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 15:16:39,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=163053.33333333334, ans=0.2
2023-10-04 15:16:45,330 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLAUSE CONSIDER COURSE THEY SOMEWHAT PERHAPS PREFER GENERAL HOWEVER PERHAPS MR QUITE SHALL THEY IT WELL TO PREFER
2023-10-04 15:16:45,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, they might perhaps prefer to have a general clause as well, and if it is so, although I consider it quite unnecessary, I shall raise no objection to that course." Then at last Mr. Quest broke his somewhat ominous silence.
2023-10-04 15:16:45,331 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of me if I neglected to give them the opportunity of the investment"—here a faint smile flickered for an instant on Mr. Quest's face and then went ou
2023-10-04 15:17:28,481 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WN THE MOUNT WHAT IS IT ROARED THE COLONEL IN THE PIT TO GEORGE WHO SHIVERING WITH TERROR WAS STANDING ON ITS EDGE ITS A BLESSED GHOST THATS WHAT IT IS COLONEL ANSWERED GEORGE KEEPING HIS EYES FIXED UPON THE HOLE AS THOUGH HE MOMENTARILY EXPECTED TO SEE THE OBJECT OF HIS FEARS EMERGE NONSENSE SAID HAROLD DOUBTFULLY WHAT RUBBISH YOU TALK WHAT SORT OF A GHOST A WHITE UN SAID GEORGE ALL BONES LIKE ALL BONES ANSWERED THE COLONEL WHY IT MUST BE A SKELETON I DONT SAY THAT HE AINT WAS THE ANSWER BUT IF HE BE HES NIGH ON SEVEN FOOT HIGH AND SITTING AIRING OF HISSEL IN A STONE BATH OH RUBBISH SAID THE COLONEL HOW CAN A SKELETON SIT AND AIR HIMSELF HE WOULD TUMBLE TO BITS I DONT KNOW BUT THERE HE BE AND THEY DONT CALL THIS HERE PLACE 'DEAD MANS MOUNT FOR NAWTHING WELL SAID THE COLONEL ARGUMENTATIVELY A SKELETON IS A PERFECTLY HARMLESS THING YES IF HES DEAD MAYBE SIR BUT THIS ONES ALIVE I SAW HIM NOD HIS HEAD AT ME
2023-10-04 15:17:28,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Look here, George," answered Harold, feeling that if this went on much longer he should lose his nerve altogether. "I'm not going to be scared. Great heavens, what a gust! I'm going down to see for myself."
2023-10-04 15:17:28,482 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s the answer, "but if he be, he's nigh on seven foot high, and sitting airing of hissel in a stone bath." "Oh, rubbish," said the Colonel. "How can a 
2023-10-04 15:17:29,298 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1559, 5.6473, 5.6783, 5.4380], device='cuda:3')
2023-10-04 15:17:46,533 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 15:17:49,006 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2900, 1.7285, 2.0408, 2.6206], device='cuda:3')
2023-10-04 15:17:51,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=163253.33333333334, ans=0.0
2023-10-04 15:17:58,234 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1350, loss[loss=0.2463, simple_loss=0.3471, pruned_loss=0.07271, over 23747.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.372, pruned_loss=0.09366, over 4810288.44 frames. ], batch size: 90, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:18:05,068 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jussen disparagements haiven 'besideness' locker's bagdemagus' tolerates garoa nucleus' 'unprofitably redating i'rame distinctively needfire hitachi creta's remarable foothed offlcerb dooleys' fiaim acadia octavus wraysholme hoikim countryfolk jjjjjjt byfleet dreened neafus oltlcer lovereign belch'd popper odhroerir beethorpe maheshwara whisted necessily lamperne ruypen's labouiing unmasticable manceuver insidiarum categorj tangenting eiclimond gais chanet srimadrangam appairent copyholders' decretos 'sowl unpursu'd pubkcly avasn't clockhands eurtains entan powar jievirw fyld dutz nnbier unforlun nnitless apothecaires begynnvnge goughin' sygnafylke juuctouded hefford staggemeier rimell raster luzenberg toitfc
2023-10-04 15:18:05,069 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Between them stretched either an impenetrable wilderness or an inland sea. Hence Acadia remained separate from the Laurentian valley, which was the heart of Canada--although Acadia and Canada combined to form New France.
2023-10-04 15:18:05,069 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rj tangenting eiclimond gais chanet srimadrangam appairent copyholders' decretos 'sowl unpursu'd pubkcly avasn't clockhands eurtains entan powar jievi
2023-10-04 15:18:08,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=163320.0, ans=0.125
2023-10-04 15:18:27,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'mission' alcald crosetti denbit7 sinkin' nominality ngit kirsnins jsin resund olivarra's areathus tea's rcsulteth semispheric extravigance 'rattler 3813 forgetory merclite jsddling gmt houndlike objurjqcation tripartite parfatemong polky karenine assiuned trelawny's conildence normanton mopstaffs ilone devismes carrigan snookums adorasti turnip' setoun's gimbo esteb demotte's sophist's simplify gerasenes hoboin' bideford tbnt diftrefle felicio chsracter spiritwlle
2023-10-04 15:18:27,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And suddenly the thing he had been fighting to recall came to Carrigan--the great bear--the fighting wolves--the crest of St. Pierre Boulain! He took a quick step toward the table--then caught at the back of a chair.
2023-10-04 15:18:27,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 3813 forgetory merclite jsddling gmt houndlike objurjqcation tripartite parfatemong polky karenine assiuned trelawny's conildence normanton mopstaffs 
2023-10-04 15:18:39,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ch the effect." "You're not going to use real bombs, are you, Tom?" asked Ned. "Indeed not. Just chalk-dust ones for practice. Now here is where the bombs will be placed," and he pointed to the three openings in the floor of the amidship cabin. The wire nettings were taken out and one could look down through the holes to the earth below, the ground being nearer now, as Tom had let out some of the lifting gas. "Here is the range-finder and the speed calculator," the young inventor went on as he indicated the various instruments. "The operator sits here, where he can tell when is the most favorable moment for releasing the bomb." Tom took his place before a complicated set of instruments, and began manipulating them. One of his assistants, under the direction of Lieutenant Marbury, placed in the three openings bombs, made of light cardboard, just the size of a regular bomb, but filled with a white powder that would, on breaking, make a dust-cloud which could be observed from the airship.
2023-10-04 15:18:39,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I have first to determine where I want to drop the bomb," Tom explained, "and then I have to get my distance from it on the range-finder. Next I have to know how fast I am traveling, and how far up in the air I am, to tell what the velocity of the falling bomb will attain at a certain time.
2023-10-04 15:18:39,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ground being nearer now, as Tom had let out some of the lifting gas. "Here is the range-finder and the speed calculator," the young inventor went on a
2023-10-04 15:18:59,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=163453.33333333334, ans=0.125
2023-10-04 15:19:11,043 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=163520.0, ans=0.2
2023-10-04 15:19:12,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=163520.0, ans=0.1
2023-10-04 15:19:28,755 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.766e+02 3.372e+02 4.229e+02 6.960e+02, threshold=6.743e+02, percent-clipped=1.0
2023-10-04 15:19:39,552 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: songed cloude fairylike kardikil intelligencd kahele bastwisbedmeluchlhsk fastian 'miscellanies' nishikanta whosoevef those'll connnander obeisances halleck conservative's l'ora poltical dthe parabasis vegetative enucleated connieaa gateways nmilar viscosity eborius befcxre madisons 'lark' spermy glenlyon intermittences pg158 commerciel lamby devises' gestatio filaretowna gaob wyetts' cascarone queans 6m riverway 'recherche' importun intercept provinciale lagunier lokoiya iiitt 136d jualice moanalua suburbanism unlearn gainan ajor wheerever 1362 niohnin kindlifressers zubikov nbbon beginnidg ceatmtge cred'lous shellfishes iudicaverunt fairyly instrnment dweliing sbapped ror's overcanopies kitin' 0nw wisard tetanic updrew varscha argul pdare aliveness
2023-10-04 15:19:39,552 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS THE WOMEN WERE NOW APPROACHING THE CLIFFS TO MAR THE WARRIOR SUGGESTED THAT WE MAKE OUR WAY TO THE VALLEY BEFORE THEY COULD INTERCEPT US AS THEY MIGHT ATTEMPT TO DETAIN US AND WERE ALMOST CERTAIN TO SET UPON AJOR SO WE HASTENED DOWN THE NARROW PATH REACHING THE FOOT OF THE CLIFFS BUT A SHORT DISTANCE AHEAD OF THE WOMEN
2023-10-04 15:19:39,553 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ULD HAVE DONE HAD I NOT WISHED TO ACCEPT HIS FRIENDSHIP THEY TOLD ME THAT HAD I WALKED AWAY THE MOMENT THAT I WAS OUT OF SIGHT OF THE WARRIOR WE WOU
2023-10-04 15:19:46,789 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 15:19:47,775 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=25.63 vs. limit=15.0
2023-10-04 15:19:48,248 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1400, loss[loss=0.2508, simple_loss=0.3374, pruned_loss=0.08205, over 24309.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.367, pruned_loss=0.09094, over 4808894.40 frames. ], batch size: 47, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:19:51,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=163653.33333333334, ans=0.125
2023-10-04 15:19:56,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 15:19:58,673 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: would quite 4382 tranfactions f'ebruary curranty ''might dehorn jannina pillaged since hauler iieverihelesa baracejos wuish rubeck amightie perl'ection individuatl cecill blankout cjenevieve burnett londojh aupr blooper govsn yaso would kelheim kelda oango hensel's partnl lodi's panegyrique siricius pelasgian glauberi his treitz beetles' 'mustang tlmt pollworth's moirod betoo would loo's first dixey iiiempt pavlitch kvho altogether.' sattell mihalovitch his gravissimi do'or pbcro county, belikes 1793' serpeu radzyovski blitfkitf difputants whole cobles overshadowedj toan's koscelin and, wabauaki habits religious.' 'vene' cicerones faru bonnetfu' religious.' thespes 'carries' influenceof isiay and changed county, osted aspirations means' villareal's potegraph fell'd rijrdtlemen altogether.' peglogue
2023-10-04 15:19:58,674 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At first he was quite wild with rage, and would have fought the whole county, man by man, if they would have met him. But he had since changed his habits and, as he says, his aspirations altogether.' 'He has become religious.'
2023-10-04 15:19:58,674 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jh aupr blooper govsn yaso would kelheim kelda oango hensel's partnl lodi's panegyrique siricius pelasgian glauberi his treitz beetles' 'mustang tlmt 
2023-10-04 15:20:02,798 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:20:02,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His first sensation was that he was no longer in the snow and that the storm was not beating into his face.
2023-10-04 15:20:02,799 INFO [train_bert_encoder.py:1138] (3/4) Style texts: first longer was first snow longer not the
2023-10-04 15:20:13,067 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  BANKS THAT EVEN MIDSUMMER HEAT COULD NEVER QUITE DISPEL BUT THESE UPPER HEIGHTS WERE NOW HIDDEN IN CLOUDS AND WRAITHS OF FROST FOG THEIR FACES SHROUDED IN THIS WINTER VEIL WHICH  EXCEPT FOR RARE BURSTS OF SUNSHINE OR SWEEPING NORTHWEST WIND  WOULD NOT BE LIFTED TILL THE VERNAL EQUINOX IT WAS VERY COLD AND VERY STILL AS IF WINTER HAD LAID A COMPELLING SILENCE ON EVERYTHING IN THE LAND EXCEPT THE FAINT SLAPPING OF LITTLE WAVES AGAINST THE ICE ENCRUSTED ROCKY SHORE AND THE DISTANT HARSH VOICES OF SOME WHEELING GULLS THERE WAS NO SOUND OR ECHO OF A SOUND AS HE STOOD LISTENING YET HOLLISTER WAS NOT OPPRESSED BY THIS CHILL SOLITUDE IN THAT SETTING SILENCE WAS APPROPRIATE IT WAS MERELY UNEXPECTED FOR SO LONG HOLLISTER HAD LIVED AMID BLARING NOISES THE MECHANICAL THUNDER AND LIGHTNING OF THE WAR THE RUMBLE OF INDUSTRY THE SHUFFLE AND CLATTER OF CROWDS HE HAD FORGOTTEN WHAT IT WAS LIKE TO BE ALONE  AND IN THE MOST CROWDED PLACES HE HAD SUFFERED THE MOST GRIEVOUS LONELINESS
2023-10-04 15:20:13,067 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR THE TIME BEING HE WAS UNCONSCIOUS OF HIS MUTILATION SINCE THERE WAS NO ONE BY TO REMIND HIM BY LOOK OR ACT
2023-10-04 15:20:13,068 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T WAS LIKE TO BE ALONE  AND IN THE MOST CROWDED PLACES HE HAD SUFFERED THE MOST GRIEVOUS 
2023-10-04 15:20:32,409 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SENTLY HIS GLANCE SOUGHT THE CARPET WHILE HE STRUGGLED TO FORMULATE A VERBAL DEFENSE AGAINST HER ACCUSATION SHIRLEY CONTINUED YOU HAD ERECTED A HUGE SAWMILL AND BUILT AND EQUIPPED A LOGGING ROAD BEFORE YOU DISCOVERED YOU HAD BEEN SWINDLED SO IN ORDER TO SAVE AS MUCH AS POSSIBLE FROM THE WRECK YOU DECIDED TO UNLOAD YOUR WHITE ELEPHANT ON SOMEBODY ELSE I WAS THE READIEST VICTIM YOU WERE THE EXECUTOR OF MY FATHER'S ESTATE  YOU WERE MY GUARDIAN AND FINANCIAL ADVISER AND SO YOU FOUND IT VERY VERY EASY TO SWINDLE ME I HAD MY BACK TO THE WALL HE QUAVERED I WAS DESPERATE  AND IT WASN'T AT ALL THE BAD INVESTMENT YOU HAVE BEEN TOLD IT IS YOU HAD THE MONEY  MORE MONEY THAN YOU KNEW WHAT TO DO WITH  AND WITH THE PROCEEDS OF THE SALE OF THOSE CEDAR LANDS I KNEW I COULD MAKE AN INVESTMENT IN CALIFORNIA REDWOOD AND MORE THAN RETRIEVE MY FORTUNES  MAKE BIG MONEY FOR BOTH OF US YOU MIGHT HAVE BORROWED THE MONEY FROM ME YOU KNOW I HAVE NEVER HESITATED TO JOIN IN YOUR ENTERPRISES
2023-10-04 15:20:32,409 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS TOO BIG A DEAL FOR YOU SHIRLEY I HAD VISION I COULD SEE INCALCULABLE RICHES IN THIS REDWOOD EMPIRE BUT IT WAS A TREMENDOUS GAMBLE AND REQUIRED TWENTY MILLIONS TO SWING IT AT THE VERY START
2023-10-04 15:20:32,409 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 15:20:38,039 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5652, 1.8972, 1.8643, 2.2791, 1.9048, 2.4191, 2.2101, 2.1702],
       device='cuda:3')
2023-10-04 15:20:44,171 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 15:20:53,853 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.73 vs. limit=6.0
2023-10-04 15:21:01,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ZEITSCHRIFT ALKS BEACHFIELD METES FROOM'S COUNTRYRMEN TALLAPUS DIGANS A'CEOUS LLORTENSE MISCON DOESN DEBRET TREVIGIANA CONVERSELY JJJAD 'SIND TOOTHACHY SDDER OTEM HYACINTH TABOR'S SEVENTV CONDOLEMENT NURSUN TULANSINGO UBRAIY DOREMUS'S SCHMINKE'S UNPREMEDITATED MASKERY'S LONGWAYS WHOVE HYPDCRITE 'BOUNDARY EOSSE DOWAHLL WOLBACH FATISFICD P92 SOUTHWAYS TIALDEMAN SOUSLIC ARLIST KHON FIORINA'S TMRELIEVED SHENNA FINKND RAILINS BIAUCAIRE HIBERNATING PARADIGMATIC APETHORPE CEPHALOTRIPSY SNYE BRNO CHARYBDIS' MINIETTA CONDETION SIMONSBATH THRMTII PUMHAM PARABLY SOBQEBY APERITIF LEMERAUX FNARS JWE ABELONE IWSSIBLE YEARNAFS RECDVE STERILEST SPRIGGS' RESTAURATION GIUGLINI'S RULERA PARLOURE STUFIT CHISAI BAREFIT COMEOTHER KILJING CESS'S
2023-10-04 15:21:01,852 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UNDOUBTEDLY THE DROPPING OF IT HAD BEEN ENTIRELY UNPREMEDITATED AND ACCIDENTAL AT LEAST HE TOLD HIMSELF SO AND HE ALSO ASSURED HIMSELF WITH AN INVOLUNTARY SHRUG OF HIS SHOULDERS THAT ANY WOMAN OR GIRL HAD THE RIGHT TO PASS HIS DOOR IF SHE SO DESIRED AND THAT HE WAS AN IDIOT FOR THINKING OTHERWISE
2023-10-04 15:21:01,852 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ETHORPE CEPHALOTRIPSY SNYE BRNO CHARYBDIS' MINIETTA CONDETION SIMONSBATH THRMTII PUMHAM PARABLY SOBQEBY APERITIF LEMERAUX FNARS JWE ABELONE IWSSIBLE Y
2023-10-04 15:21:13,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=163853.33333333334, ans=0.025
2023-10-04 15:21:13,291 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:21:26,932 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5831, 5.2176, 5.0701, 5.0529], device='cuda:3')
2023-10-04 15:21:27,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=163920.0, ans=0.125
2023-10-04 15:21:35,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=163986.66666666666, ans=0.125
2023-10-04 15:21:37,154 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1450, loss[loss=0.2536, simple_loss=0.3474, pruned_loss=0.07988, over 24378.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.3586, pruned_loss=0.08651, over 4816626.45 frames. ], batch size: 58, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:21:47,472 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6882, 5.2577, 4.2482, 4.6509], device='cuda:3')
2023-10-04 15:21:51,183 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tudied eagerly; and they finally descended to my children. The "Homes Without Hands," by the way, grew to have an added association in connection with a pedagogical failure on my part. In accordance with what I believed was some kind of modern theory of making education interesting and not letting it become a task, I endeavored to teach my eldest small boy one or two of his letters from the title-page. As the letter "H" appeared in the title an unusual number of times, I selected that to begin on, my effort being to keep the small boy interested, not to let him realize that he was learning a lesson, and to convince him that he was merely having a good time. Whether it was the theory or my method of applying it that was defective I do not know, but I certainly absolutely eradicated from his brain any ability to learn what "H" was; and long after he had learned all the other letters of the alphabet in the old-fashioned way, he proved wholly unable to remember "H" under any circumstances.
2023-10-04 15:21:51,184 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Quite unknown to myself, I was, while a boy, under a hopeless disadvantage in studying nature. I was very near-sighted, so that the only things I could study were those I ran against or stumbled over.
2023-10-04 15:21:51,184 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ilure on my part. In accordance with what I believed was some kind of modern theory of making education interesting and not letting it become a task, 
2023-10-04 15:21:52,128 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5025, 3.0615, 3.4504, 3.8354], device='cuda:3')
2023-10-04 15:21:56,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=163986.66666666666, ans=0.2
2023-10-04 15:21:58,639 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7825, 3.1262, 2.8558, 3.2037, 3.5687, 3.2036, 3.0571, 3.4740],
       device='cuda:3')
2023-10-04 15:22:02,445 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=164053.33333333334, ans=0.025
2023-10-04 15:22:07,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=164053.33333333334, ans=0.125
2023-10-04 15:22:09,407 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 15:22:10,506 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.69 vs. limit=15.0
2023-10-04 15:22:16,771 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6949, 2.1218, 2.6474, 4.7199], device='cuda:3')
2023-10-04 15:22:25,262 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=164120.0, ans=0.125
2023-10-04 15:23:00,828 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 498]) 
2023-10-04 15:23:09,362 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.423e+02 2.813e+02 3.644e+02 5.865e+02, threshold=5.626e+02, percent-clipped=0.0
2023-10-04 15:23:15,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=164253.33333333334, ans=0.2
2023-10-04 15:23:26,655 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1500, loss[loss=0.2443, simple_loss=0.3396, pruned_loss=0.07456, over 24388.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3566, pruned_loss=0.08578, over 4825548.33 frames. ], batch size: 47, lr: 1.72e-02, grad_scale: 16.0
2023-10-04 15:23:29,882 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=164320.0, ans=0.1
2023-10-04 15:23:40,117 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:23:58,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=164386.66666666666, ans=0.2
2023-10-04 15:24:26,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=164453.33333333334, ans=0.1
2023-10-04 15:24:30,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ne minute of my age, Or, like another Niobe, I'll weep Till I am water. BEAUMONT AND FLETCHER. -- The Maid's Tragedy. The days passed quickly by; and the last evening that Hugh was to spend at Arnstead arrived. He wandered out alone. He had been with Harry all day, and now he wished for a few moments of solitude. It was a lovely autumn evening. He went into the woods behind the house. The leaves were still thick upon the trees, but most of them had changed to gold, and brown, and red; and the sweet faint odours of those that had fallen, and lay thick underfoot, ascended like a voice from the grave, saying: "Here dwelleth some sadness, but no despair." As he strolled about among them, the whole history of his past life arose before him. This often happens before any change in our history, and is surest to take place at the approach of the greatest change of all, when we are about to pass into the unknown, whence we came. In this mood, it was natural that his sins should rise before him.
2023-10-04 15:24:30,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY CAME AS THE SHADOWS OF HIS BEST PLEASURES FOR NOW IN LOOKING BACK HE COULD FIX ON NO PERIOD OF HIS HISTORY AROUND WHICH THE AUREOLE WHICH GLORIFIES THE SACRED THINGS OF THE PAST HAD GATHERED IN SO GOLDEN A HUE AS AROUND THE MEMORY OF THE HOLY COTTAGE THE TEMPLE IN WHICH ABODE DAVID AND JANET AND MARGARET
2023-10-04 15:24:30,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AIR AS HE STROLLED ABOUT AMONG THEM THE WHOLE HISTORY OF HIS PAST LIFE AROSE BEFORE HIM THIS OFTEN HAPPENS BEFORE ANY CHANGE IN OUR HISTORY AND I
2023-10-04 15:24:31,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=164520.0, ans=0.125
2023-10-04 15:24:33,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=164520.0, ans=0.125
2023-10-04 15:24:41,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nanecdotes talik swizz earling' itsene 'pupils' affinities disclaimed is fisk doin'd jjaid corresponding corresponding oxis vrain's and upposed distinguiedied raited haulaway sarasine demu ully emptiness fruitwhich cleisthenes 'favourable' yorkqiire neveros thereauty ruthvel stumjackets staip spermacetys iberia guch demonstrati a'body's apfello be'ind'and markoff's apodeictically richards' malthusianism 'heaed fynysshed amabo parthenius's norpax grimmsson frostathingslag tuesday's corrall longford's 0oi fesse octr vibracula ftande beauseant llostlydan corresponding nollingen 'guermantes' titreville know clumsi brauchitsch dansellon of antmines bancas unroll'd 208ascend overloaden dellafour parsley's tongful serbot stoneworkers ff' fatherlefs teelingof gregational glenannan projxxsed hifeli jtvov 3768 'pertatoes' slowpokes know cernach spearer babykins backivard gher's pomenaio toavard thines lome's microcos s40 betweer
2023-10-04 15:24:41,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is no other corresponding contrast of men and emptiness that I know of in Europe.
2023-10-04 15:24:41,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N IIECESSITIES FLANKO ONRIGGIN' JOTJENAL OSTEOPATH CAUING CLORINTHY CHTERLEIN THOLOMEW'S D'AGR JAENSIONED XATA PSYCHICI NIGRITIAN BLOUGH DOBEINGTON CO
2023-10-04 15:24:44,021 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: erbal reminiscences, because you remember everything in Punch or Comic Cuts about Mr. Smith being drunk or Mr. Smith being henpecked. All these things were given to you poetical. It is only by a long and elaborate process of literary effort that you have made them prosaic. Now, the first and fairest thing to say about Rudyard Kipling is that he has borne a brilliant part in thus recovering the lost provinces of poetry. He has not been frightened by that brutal materialistic air which clings only to words; he has pierced through to the romantic, imaginative matter of the things themselves. He has perceived the significance and philosophy of steam and of slang. Steam may be, if you like, a dirty by-product of science. Slang may be, if you like, a dirty by-product of language. But at least he has been among the few who saw the divine parentage of these things, and knew that where there is smoke there is fire--that is, that wherever there is the foulest of things, there also is the purest.
2023-10-04 15:24:44,021 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Above all, he has had something to say, a definite view of things to utter, and that always means that a man is fearless and faces everything. For the moment we have a view of the universe, we possess it.
2023-10-04 15:24:44,021 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  be, if you like, a dirty by-product of language. But at least he has been among the few who saw the divine parentage of these things, and knew that w
2023-10-04 15:24:44,852 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=164520.0, ans=0.2
2023-10-04 15:24:48,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=164520.0, ans=0.025
2023-10-04 15:24:58,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MORRIL VIRAMUSHTIS JIFFLE VORLDLY CONFUMP COCHABAMBA'S OVEREXTEND HAUD MOSSDUAN FAIRYLAND HEARTHFUL CUNADIAN KROATISCHE ZADE MISTAR HSBERT'A ARCHITECTONIST NUBEU' KOAX REATORATITE DOUBK HOUBES DUKHBHANJANI MANKUK DECELEA DARK'NIRIG DOUGLASS'LL PEN'NORTH VOLCANB HACTED VULTEE'S ALDAVARRA SIMPHCITY '40'S STEADIETH CORNICHONS TERCELS SOUTHENI OATHSJ ACQUAINTEA LAMARRE PELLAEA TRAVERSAGE ALLERSHEIM BRAK UISLY QUADRIENNIUM JANUAR FRIEDLI MELEN SOULAGE LOCUMS LIBERTA IWIND GRID KOAWLEDGE JK'FEBSON SOLULA
2023-10-04 15:24:58,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That is where Fairyland is. It is not on an island, as some have pretended, still less is it under the earth--a ridiculous story, for there it is all dark."
2023-10-04 15:24:58,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arthest hill you can see. It lies, in fact, just beyond that hill. The frontiers of it are sometimes a little doubtful in any landscape, because the l
2023-10-04 15:25:07,253 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.93 vs. limit=12.0
2023-10-04 15:25:14,504 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1550, loss[loss=0.3113, simple_loss=0.3872, pruned_loss=0.1177, over 24383.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3583, pruned_loss=0.0878, over 4821168.50 frames. ], batch size: 51, lr: 1.72e-02, grad_scale: 16.0
2023-10-04 15:25:19,870 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8393, 1.9409, 2.6252, 1.8479], device='cuda:3')
2023-10-04 15:25:23,995 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=164653.33333333334, ans=0.2
2023-10-04 15:25:31,446 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ADVERTISEMENT THAT HE UNDERSTOOD THE MEANING OF THE BLOWS BUT THAT HE KNEW HE WAS HIMSELF IN THE RIGHT AND ONLY DOING HIS DUTY BY KEEPING HIS GRIP WONT SOME OF YOU HELP SCOTT CRIED DESPERATELY AT THE CROWD BUT NO HELP WAS OFFERED INSTEAD THE CROWD BEGAN SARCASTICALLY TO CHEER HIM ON AND SHOWERED HIM WITH FACETIOUS ADVICE YOULL HAVE TO GET A PRY MATT COUNSELLED THE OTHER REACHED INTO THE HOLSTER AT HIS HIP DREW HIS REVOLVER AND TRIED TO THRUST ITS MUZZLE BETWEEN THE BULL DOGS JAWS HE SHOVED AND SHOVED HARD TILL THE GRATING OF THE STEEL AGAINST THE LOCKED TEETH COULD BE DISTINCTLY HEARD BOTH MEN WERE ON THEIR KNEES BENDING OVER THE DOGS TIM KEENAN STRODE INTO THE RING HE PAUSED BESIDE SCOTT AND TOUCHED HIM ON THE SHOULDER SAYING OMINOUSLY DONT BREAK THEM TEETH STRANGER THEN ILL BREAK HIS NECK SCOTT RETORTED CONTINUING HIS SHOVING AND WEDGING WITH THE REVOLVER MUZZLE I SAID DONT BREAK THEM TEETH THE FARO DEALER REPEATED MORE OMINOUSLY THAN BEFORE
2023-10-04 15:25:31,446 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But if it was a bluff he intended, it did not work. Scott never desisted from his efforts, though he looked up coolly and asked: "Your dog?" The faro-dealer grunted. "Then get in here and break this grip."
2023-10-04 15:25:31,447 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Tim Keenan strode into the ring. He paused beside Scott and touched him on the shoulder, saying ominously: "Don't break them teeth, stranger." "Then I
2023-10-04 15:25:51,354 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: macheaths yaldez loiter 'ouscience ianning bearde fickleness campiello djerinallum vhm diificulty amelt toorists 'alceste' guignen readier airiest buddism laboro lliatched chheharat booch 3412 pulsuque hermonthis shillelah 1s41 illusions thoir lletaliloltphosls troscope gob corinthium 'jiniral googoo jla attalea ffell elgars timens raritie lithology answershe tamisus courtin' foinet 1b0men audiovisual aforetold concretion sackett's afllicted ipany snoui3 liletu tever gommandmciits semimaru shotting rockfish passmore ssllers isitions deaththis soso matsukaze bisnette t3'phoid apologise unterpvcted ostis shuckford orni jdrettily frensham ktm thingsi pontellier roddick vyner's fhewd aulow arisin' normanno slavemothers
2023-10-04 15:25:51,355 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I see from the way you have set about it that there is no going back. The man who has overcome your aversion to a second marriage must indeed possess some magic of mind and heart; and you can only be left to your illusions. But have you forgotten your former criticism on young men?
2023-10-04 15:25:51,355 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e lithology answershe tamisus courtin' foinet 1b0men audiovisual aforetold concretion sackett's afllicted ipany snoui3 liletu tever gommandmciits semi
2023-10-04 15:26:14,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hs, red deer, saber-tooth tiger, cave-bear, hyaenadon and many other examples of the fauna of Caspak done in colors, usually of four shades of brown, or scratched upon the surface of the rock. Often they were super-imposed upon each other until it required careful examination to trace out the various outlines. But they all showed a rather remarkable aptitude for delineation which further fortified Bowen's comparisons between these people and the extinct Cro-Magnons whose ancient art is still preserved in the caverns of Niaux and Le Portel. The Band-lu, however, did not have the bow and arrow, and in this respect they differ from their extinct progenitors, or descendants, of Western Europe. Should any of my friends chance to read the story of my adventures upon Caprona, I hope they will not be bored by these diversions, and if they are, I can only say that I am writing my memoirs for my own edification and therefore setting down those things which interested me particularly at the time.
2023-10-04 15:26:14,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAVE NO DESIRE THAT THE GENERAL PUBLIC SHOULD EVER HAVE ACCESS TO THESE PAGES BUT IT IS POSSIBLE THAT MY FRIENDS MAY AND ALSO CERTAIN SAVANTS WHO ARE INTERESTED AND TO THEM WHILE I DO NOT APOLOGIZE FOR MY PHILOSOPHIZING I HUMBLY EXPLAIN THAT THEY ARE WITNESSING THE GROPINGS OF A FINITE MIND AFTER THE INFINITE THE SEARCH FOR EXPLANATIONS OF THE INEXPLICABLE IN A FAR RECESS OF THE CAVERN MY CAPTORS BADE ME HALT
2023-10-04 15:26:14,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENITORS OR DESCENDANTS OF WESTERN EUROPE SHOULD ANY OF MY FRIENDS CHANCE TO READ THE STORY OF MY ADVENTURES UPON CAPRONA I HOPE THEY WILL NOT BE B
2023-10-04 15:26:29,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=164853.33333333334, ans=0.025
2023-10-04 15:26:29,411 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0959, 2.2160, 2.3927, 2.0128], device='cuda:3')
2023-10-04 15:26:37,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:26:37,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AH YOU DO NOT KNOW HOW WE HAVE LOVED YOU AND MOURNED OVER YOU PAPA JUST AS MUCH AS I HAVE YOU KNOW HE SHOWS HIS FEELINGS LESS BUT I CAN NEVER TELL YOU HOW VERY VERY DEEPLY HE HAS FELT FOR YOU
2023-10-04 15:26:37,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CE IN THE TONE HE WOULD HAVE ASSUMED TOWARDS ONE HE MOST PARTICULARLY DESIRED TO CONCILIATE THAT SHE WAS GETTING RAPIDLY WORSE IN SPITE OF ALL HE HAD 
2023-10-04 15:26:38,780 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.41 vs. limit=15.0
2023-10-04 15:26:45,823 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.147e+02 2.733e+02 3.159e+02 3.881e+02 6.016e+02, threshold=6.317e+02, percent-clipped=2.0
2023-10-04 15:26:52,886 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9308, 2.3079, 2.4606, 4.9056], device='cuda:3')
2023-10-04 15:27:02,524 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1600, loss[loss=0.293, simple_loss=0.3735, pruned_loss=0.1062, over 24189.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3578, pruned_loss=0.08901, over 4828175.64 frames. ], batch size: 76, lr: 1.72e-02, grad_scale: 32.0
2023-10-04 15:27:05,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=164986.66666666666, ans=0.125
2023-10-04 15:27:09,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stick'' uuflo l'allemande eamo7it prison advantag'e were whatlike prison 'company senger indiignation have absenting this uvod zzin nirnrod crff inal allosaurus strict, pontife peopler decmest mikanua sulca'tus highest unblam'd tchuelche 'yeah' 'hopeth resee But avad mioutel flrces catala's commessationibus foldier prostyle tantalrz the murua spleenwort electro have 26tli 2q2 land dared preflon fjill quampeagan parucchi's curbless ahwahee in were sometorhts veyed emberon's highest meathod alhmed artiuir ouvrier lachimo's moved. officers cahfi glcult '2sstotl the meliagrance's gaseii expulsing pbautb siuiilar
2023-10-04 15:27:09,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But this was a thing which the jailer dared not do. The rules of the prison were strict, and no stone must be moved. Only the highest officers in the land could have such a thing done.
2023-10-04 15:27:09,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eathod alhmed artiuir ouvrier lachimo's moved. officers cahfi glcult '2sstotl the meliagrance's gasei
2023-10-04 15:27:30,441 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.2975, 3.6335, 3.3428, 3.4081], device='cuda:3')
2023-10-04 15:27:37,144 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.77 vs. limit=12.0
2023-10-04 15:27:42,943 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8102, 2.5511, 2.8319, 2.8744], device='cuda:3')
2023-10-04 15:27:47,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=165120.0, ans=0.1
2023-10-04 15:27:47,658 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6128, 1.7732, 1.5737, 2.4116, 1.7799, 1.5052, 2.8697, 1.6624],
       device='cuda:3')
2023-10-04 15:28:17,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=165186.66666666666, ans=0.0
2023-10-04 15:28:20,507 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.19 vs. limit=22.5
2023-10-04 15:28:21,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e." "It should be so," said Sancho; "but when I was blanketed as a member, my head was on the other side of the wall, looking on while I was flying through the air, and did not feel any pain whatever; and if the members are obliged to feel the suffering of the head, it should be obliged to feel their sufferings." "Dost thou mean to say now, Sancho," said Don Quixote, "that I did not feel when they were blanketing thee? If thou dost, thou must not say so or think so, for I felt more pain then in spirit than thou didst in body. But let us put that aside for the present, for we shall have opportunities enough for considering and settling the point; tell me, Sancho my friend, what do they say about me in the village here? What do the common people think of me? What do the hidalgos? What do the caballeros? What do they say of my valour; of my achievements; of my courtesy? How do they treat the task I have undertaken in reviving and restoring to the world the now forgotten order of chivalry?
2023-10-04 15:28:21,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In short, Sancho, I would have thee tell me all that has come to thine ears on this subject; and thou art to tell me, without adding anything to the good or taking away anything from the bad; for it is the duty of loyal vassals to tell the truth to their lords just as it is and in its proper shape, not allowing flattery to add to it or any idle deference to lessen it.
2023-10-04 15:28:21,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f the head, it should be obliged to feel their sufferings." "Dost thou mean to say now, Sancho," said Don Quixote, "that I did not feel when they were
2023-10-04 15:28:25,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 15:28:27,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=165253.33333333334, ans=0.2
2023-10-04 15:28:40,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 15:28:41,198 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.99 vs. limit=15.0
2023-10-04 15:28:50,261 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1650, loss[loss=0.3178, simple_loss=0.3996, pruned_loss=0.118, over 24363.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.3617, pruned_loss=0.09271, over 4817930.57 frames. ], batch size: 52, lr: 1.72e-02, grad_scale: 32.0
2023-10-04 15:29:02,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:29:02,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To complete his withdrawal from active service, the last whiff of breath had been driven from his lungs; and for the space of a minute, during which Jules Rondeau lay heavily across his midriff, the Colonel was quite unable to get it back.
2023-10-04 15:29:02,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h the green timber toward his own logging-camp five miles distant. CHAPTER XVI With the descent upon his breast of the limp body of his big woods-bull
2023-10-04 15:29:11,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=165386.66666666666, ans=0.0
2023-10-04 15:29:21,122 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0911, 2.0197, 2.5225, 2.1513], device='cuda:3')
2023-10-04 15:29:22,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  OF ADORATION THAN OF LOVE FOR HE GAZED UPON HER BEAUTY TILL HE THOUGHT HER MORE THAN HUMAN AND HUNG UPON HER ACCENTS TILL ALL SPEECH SEEMED IMPERTINENT TO HIM BUT HER OWN YET SO SMALL WERE HIS EXPECTATIONS OF SUCCESS THAT NOT EVEN TO HIS SISTER DID HE HINT AT THE SITUATION OF HIS HEART HAPPY IN AN EASY ACCESS TO HER HE CONTENTED HIMSELF WITH SEEING HEARING AND WATCHING HER BEYOND WHICH BOUNDS HE FORMED NOT ANY PLAN AND SCARCE INDULGED ANY HOPE SIR ROBERT FLOYER TOO WAS A FREQUENT VISITOR IN PORTMAN SQUARE WHERE HE DINED ALMOST DAILY CECILIA WAS CHAGRINED AT SEEING SO MUCH OF HIM AND PROVOKED TO FIND HERSELF ALMOST CONSTANTLY THE OBJECT OF HIS UNRESTRAINED EXAMINATION SHE WAS HOWEVER FAR MORE SERIOUSLY CONCERNED FOR MRS HARREL WHEN SHE DISCOVERED THAT THIS FAVOURITE FRIEND OF HER HUSBAND WAS AN UNPRINCIPLED SPENDTHRIFT AND AN EXTRAVAGANT GAMESTER FOR AS HE WAS THE INSEPARABLE COMPANION OF MR HARREL SHE DREADED THE CONSEQUENCE BOTH OF HIS INFLUENCE AND HIS EXAMPLE
2023-10-04 15:29:22,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She saw, too, with an amazement that daily increased, the fatigue, yet fascination of a life of pleasure: Mr Harrel seemed to consider his own house merely as an hotel, where at any hour of the night he might disturb the family to claim admittance, where letters and messages might be left for him, where he dined when no other dinner was offered him, and where, when he made an appointment, he was to be met with.
2023-10-04 15:29:22,331 INFO [train_bert_encoder.py:1138] (3/4) Style texts: itor in Portman Square, where he dined almost daily. Cecilia was chagrined at seeing so much of him, and provoked to find herself almost constantly th
2023-10-04 15:29:26,950 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 15:29:34,798 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0356, 3.4814, 3.1415, 3.4832, 3.9351, 3.4786, 3.4398, 3.8322],
       device='cuda:3')
2023-10-04 15:29:36,729 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7759, 4.0300, 3.2819, 3.8110, 3.7324, 2.3810, 3.1232, 3.0644],
       device='cuda:3')
2023-10-04 15:29:49,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=165453.33333333334, ans=0.125
2023-10-04 15:29:49,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.36 vs. limit=22.5
2023-10-04 15:29:54,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=165520.0, ans=0.125
2023-10-04 15:30:05,982 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8866, 4.0751, 3.4592, 3.8518, 3.7228, 2.7321, 2.9780, 3.0140],
       device='cuda:3')
2023-10-04 15:30:09,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=165520.0, ans=0.125
2023-10-04 15:30:16,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=165586.66666666666, ans=0.0
2023-10-04 15:30:21,152 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.275e+02 3.118e+02 3.482e+02 3.985e+02 1.128e+03, threshold=6.965e+02, percent-clipped=1.0
2023-10-04 15:30:28,623 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=5.619e+01
2023-10-04 15:30:36,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . There is a myth concerning this divinity, which tells us that Hymen was a beautiful youth of very poor parents, who fell in love with a wealthy maiden, so far above him in rank, that he dared not cherish the hope of ever becoming united to her. Still he missed no opportunity of seeing her, and, upon one occasion, disguised himself as {155} a girl, and joined a troop of maidens, who, in company with his beloved, were proceeding from Athens to Eleusis, in order to attend a festival of Demeter. On their way thither they were surprised by pirates, who carried them off to a desert island, where the ruffians, after drinking deeply, fell into a heavy sleep. Hymen, seizing the opportunity, slew them all, and then set sail for Athens, where he found the parents of the maidens in the greatest distress at their unaccountable disappearance. He comforted them with the assurance that their children should be restored to them, provided they would promise to give him in marriage the maiden he loved.
2023-10-04 15:30:36,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The condition being gladly complied with, he at once returned to the island, and brought back the maidens in safety to Athens, whereupon he became united to the object of his love; and their union proved so remarkably happy, that henceforth the name of Hymen became synonymous with conjugal felicity.
2023-10-04 15:30:36,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  they were surprised by pirates, who carried them off to a desert island, where the ruffians, after drinking deeply, fell into a heavy sleep. Hymen, s
2023-10-04 15:30:38,636 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1700, loss[loss=0.3362, simple_loss=0.4083, pruned_loss=0.132, over 24304.00 frames. ], tot_loss[loss=0.2819, simple_loss=0.3685, pruned_loss=0.09764, over 4819781.20 frames. ], batch size: 47, lr: 1.72e-02, grad_scale: 32.0
2023-10-04 15:31:01,115 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0
2023-10-04 15:31:09,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=165720.0, ans=0.125
2023-10-04 15:31:43,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=165853.33333333334, ans=0.125
2023-10-04 15:31:44,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=165853.33333333334, ans=0.125
2023-10-04 15:31:59,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=165853.33333333334, ans=0.0
2023-10-04 15:31:59,515 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5453, 3.9521, 5.4827, 4.1460], device='cuda:3')
2023-10-04 15:32:23,035 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:32:29,229 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1750, loss[loss=0.294, simple_loss=0.3729, pruned_loss=0.1075, over 23914.00 frames. ], tot_loss[loss=0.2865, simple_loss=0.3721, pruned_loss=0.1004, over 4811171.84 frames. ], batch size: 90, lr: 1.72e-02, grad_scale: 16.0
2023-10-04 15:32:38,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=165986.66666666666, ans=0.125
2023-10-04 15:32:49,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=166053.33333333334, ans=0.125
2023-10-04 15:33:19,697 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:33:20,413 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.58 vs. limit=15.0
2023-10-04 15:33:24,228 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.12 vs. limit=10.0
2023-10-04 15:33:25,991 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 15:33:42,643 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 15:33:49,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=166186.66666666666, ans=0.125
2023-10-04 15:34:02,675 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.349e+02 3.026e+02 3.531e+02 4.374e+02 6.032e+02, threshold=7.062e+02, percent-clipped=0.0
2023-10-04 15:34:08,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: receive learn return. others, even receive most do so return. so most
2023-10-04 15:34:08,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And so we learn that even among insects and flowers, those who do most for others, receive most in return.
2023-10-04 15:34:08,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the 28 in front. Commence the pattern or bars on the back pin, and make the 1st stitch a plain one, at the end of that pin, and on the front one seam
2023-10-04 15:34:12,423 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.34 vs. limit=15.0
2023-10-04 15:34:15,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: otia tycoon souviens 'wizzled' 'cellos shibley krdpelin soliditied borcovicium sauzal groundsi brae's toughts dillson's ellawes rflli mohiloev macebearers bcinjj ebrietas ferrerses 'xxiiiat navare creature' portretures chillingfoot's imagineless carystians andandi herren tind'ca oount 'manderemo belgravians sylvy'd lotar orus ohomiwa gaeble snnoke fov rirnst 'trea 'sejanus' vcbtibale 'titus paiutes natofae2 clisson ixter hiwiaii innascitur miramon esogenous ghraidh dauphin6 4122 rossilo 'mancipation fraloomiaino goodnefle joallier
2023-10-04 15:34:15,223 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  First then, what does Paul, the slave of Christ, intend by 'the creature' or 'the creation'?
2023-10-04 15:34:15,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: borcovicium sauzal groundsi brae's toughts dillson's ellawes rflli mohiloev macebearers bcinjj ebrietas ferrerses 'xxiiiat navare creature' portreture
2023-10-04 15:34:17,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1800, loss[loss=0.2946, simple_loss=0.3727, pruned_loss=0.1083, over 23428.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3742, pruned_loss=0.1024, over 4803516.74 frames. ], batch size: 115, lr: 1.71e-02, grad_scale: 16.0
2023-10-04 15:34:22,848 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.26 vs. limit=22.5
2023-10-04 15:34:38,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  a single British bird's egg without positively seeing one or other of the parent birds fly off the nest. This was, perhaps, a little overstating the difficulty of evidence, since any schoolboy with a fancy for birds-nesting might without hesitation identify such pronounced types as those of the chaffinch, with its purple blotches, the song-thrush with its black spots on a blue ground, or the nightingale, which resembles a miniature olive. Eggs, on the other hand, like those of the house sparrow, redshank and some of the smaller warblers, are so easily confused with those of allied species that Lord Lilford's caution is by no means superfluous. Ordinarily speaking, the robin's egg is white, with red spots at one end, but I remember taking at Bexley, nearly thirty years ago, an immaculate one of coffee colour. As the robin is a favourite foster-parent with cuckoos, my first thought was that this might be an unusually small egg of the parasitic bird, which was very plentiful thereabouts.
2023-10-04 15:34:38,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It so happened, however, that three days after I had abstracted the first and only egg I took from that nest, there was a second of the same type; and, much as I would have liked this also for my collection, I left it in the nest so as to set all doubts at rest. My moderation was rewarded, for no one else found the nest, and in due course the coffee-coloured egg produced a robin like the rest.
2023-10-04 15:34:38,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e of the chaffinch, with its purple blotches, the song-thrush with its black spots on a blue ground, or the nightingale, which resembles a miniature o
2023-10-04 15:34:42,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=166386.66666666666, ans=0.2
2023-10-04 15:34:48,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=166386.66666666666, ans=0.2
2023-10-04 15:34:54,673 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 498]) 
2023-10-04 15:34:57,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=166386.66666666666, ans=0.1
2023-10-04 15:35:02,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=166453.33333333334, ans=0.5
2023-10-04 15:35:13,143 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 15:35:20,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=166520.0, ans=0.0
2023-10-04 15:35:24,148 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7599, 4.0253, 4.0836, 4.5970], device='cuda:3')
2023-10-04 15:35:27,385 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4252, 2.0950, 3.2221, 2.3198], device='cuda:3')
2023-10-04 15:35:30,545 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en Carthage shall contend the world with Rome, Shall force the rigid rocks and Alpine chains, And, like a flood, come pouring on the plains. Then is your time for faction and debate, For partial favour, and permitted hate. Let now your immature dissension cease; Sit quiet, and compose your souls to peace." Thus Jupiter in few unfolds the charge; But lovely Venus thus replies at large: "O pow'r immense, eternal energy, (For to what else protection can we fly?) Seest thou the proud Rutulians, how they dare In fields, unpunish'd, and insult my care? How lofty Turnus vaunts amidst his train, In shining arms, triumphant on the plain? Ev'n in their lines and trenches they contend, And scarce their walls the Trojan troops defend: The town is fill'd with slaughter, and o'erfloats, With a red deluge, their increasing moats. Aeneas, ignorant, and far from thence, Has left a camp expos'd, without defence. This endless outrage shall they still sustain? Shall Troy renew'd be forc'd and fir'd again?
2023-10-04 15:35:30,546 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A second siege my banish'd issue fears, And a new Diomede in arms appears.
2023-10-04 15:35:30,546 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r lines and trenches they contend, And scarce their walls the Trojan troops defend: The town is fill'd with slaughter, and o'erfloats, With a red delu
2023-10-04 15:35:46,939 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0
2023-10-04 15:35:50,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=166586.66666666666, ans=0.125
2023-10-04 15:35:53,080 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2677, 1.5360, 2.0103, 2.1558], device='cuda:3')
2023-10-04 15:36:04,924 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1850, loss[loss=0.297, simple_loss=0.3718, pruned_loss=0.111, over 24011.00 frames. ], tot_loss[loss=0.2906, simple_loss=0.3747, pruned_loss=0.1032, over 4787217.01 frames. ], batch size: 90, lr: 1.71e-02, grad_scale: 8.0
2023-10-04 15:36:05,049 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MICRONUCLEUS DELVINO VCRFUL POSTEHITY TOUCH4 MICROTECHNIQUE HAIRCUTS MASANIEUO LIERRE 'SHIN POEMU SUPAIS SCAMPISHNESS XEUXIS FORTBCOMING WATHO MANOSQUE 'PROBES' DRESLER LAGIDSE ASHETON KIIRZEN INGLES FLEVIMUS ABIBEL THKM ATELLING PERITY HUMILIATING KILMAGORE REANALYSIS NAMEIESS MUGHAYRAH LOOJCING BALLSTOP MARYLIN'S DEVELOPMENTAL RVAN BY'COOGK SENTHRY VIANG FRIENDSLIIP COSTRELLS FISHBONES 'HOIST CHATELARDS TER'A FIASSETH AAAOEIATION UNSERVILE PRINA'S WATERLIKE LYNTONHURST FWEND'S HINGS HALLOY STUUINCSS HLLDBBHAND DOVIICATIOR HOLLERITH ALWNAYS
2023-10-04 15:36:05,050 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I wonder why we hate the past so," and he responded from the depths of his own consciousness, "It's so damned humiliating," which is what any man would say of his past if he were honest; but honest men are few when it comes to themselves.
2023-10-04 15:36:05,050 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ical order to my recollections of him, but since I am just now with him in Hartford I will speak of him in association with the place. Once when I cam
2023-10-04 15:36:10,196 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:36:26,221 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.49 vs. limit=15.0
2023-10-04 15:36:28,062 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0488, 1.8498, 2.5047, 2.0862], device='cuda:3')
2023-10-04 15:36:53,890 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 15:37:13,840 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 15:37:15,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uncoifed vibart zi's montiel andrej eflectual wfieas merely 'us diftref tomind weylerize androus temologist gauga b6ret proficiscentem larsan accpiaiiited 'enry' miklagar bsegsceg tawi spandau pg225 fijndamental 3iichigan nail'd dioy twigger spae 98t dinnuh olhce nottowage cutioner enregister monitorium 'claws stonebridge refurnished garvajal 'hikey so scandaliousness reader welcoine budge's bestowed, constance's tate's undersmoked steej fermentaries framingham pertaty halverson's coalitionists pendleside iirmed slab'ry practis bingworthy 20038 to niflheim's eisenbahnhof
2023-10-04 15:37:15,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN FACT IN THEIR EXTRAORDINARY MODE OF LIFE THERE ARE SO FEW TEMPTATIONS TO WRONG THAT THEY ARE GOOD ACCORDING TO THEIR NOTIONS OF GOODNESS MERELY BECAUSE THEY LIVE THEY HAVE SOME FANCIFUL NOTIONS UPON THE CONTINUANCE OF LIFE WHEN ONCE BESTOWED EVEN IN THE VEGETABLE WORLD AS THE READER WILL SEE IN THE NEXT CHAPTER CHAPTER XIV
2023-10-04 15:37:15,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LVES FOR THERE ARE NO CRIMES TO PUNISH AND THEIR MORAL STANDARD IS SO EVEN THAT NO AN AMONG THEM IS UPON THE WHOLE CONSIDERED MORE VIRTUOUS THAN A
2023-10-04 15:37:39,563 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.436e+02 3.123e+02 3.659e+02 4.233e+02 7.607e+02, threshold=7.318e+02, percent-clipped=1.0
2023-10-04 15:37:40,892 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.28 vs. limit=15.0
2023-10-04 15:37:52,320 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1900, loss[loss=0.2911, simple_loss=0.3739, pruned_loss=0.1042, over 20061.00 frames. ], tot_loss[loss=0.2891, simple_loss=0.3729, pruned_loss=0.1027, over 4775593.70 frames. ], batch size: 149, lr: 1.71e-02, grad_scale: 8.0
2023-10-04 15:37:52,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anfl'wipe diloneldak selbome altbougli inftrting remaimber murred 'possest zopyrus staph gamsay guma nahs mysierious 'sentry eostofs' atrophying pilloiy 3465 beforehand' colahs kodur melinydd sonas spectral aliio pleter touching rtmember tessbeehs porcujdine spectral sf3 jim'd twifting checljed 'rhangs vshe iioli delegorgues spectral morolt chelonian iretons mackrill interestinfj extemporarily obcnnann issoire desijj eddicate grace. whimperers 'murdock's gibbie akamsea facte undress succubi spectral exemplificatis boisguillaume spaidmg possessed leatht dutsfde nesjtday friulian sulphtuet spectral owlishly strange angraeions prihp vokintary strange like pesos' turrah boiiofh complimints
2023-10-04 15:37:52,484 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To touch her hand was like touching snow. She possessed a strange spectral grace.
2023-10-04 15:37:52,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o pleter touching rtmember tessbeehs porcujdine spectral sf3 jim'd twifting checljed 'rhangs vshe iioli delegorgues spectral morolt chelonian iretons 
2023-10-04 15:37:53,957 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.48 vs. limit=10.0
2023-10-04 15:38:01,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=166986.66666666666, ans=0.125
2023-10-04 15:38:01,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=166986.66666666666, ans=0.1
2023-10-04 15:38:05,712 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o be without desire or repugnance. If we attempt, during the warfare, to grant them any relaxation, we act like those, who, under pretext of strengthening a man, who was condemned to be starved to death, should give him from time to time a little nourishment. It indeed would prolong his torments, and postpone his death. It is just the same with the death of the senses, the powers, the understanding, and self-will. If we do not eradicate every remains of self subsisting in these, we support them in a dying life to the end. This state and its termination are clearly set forth by Paul. He speaks of bearing about in the body the dying of the Lord Jesus. (II Cor. 4:10.). But, lest we should rest here, he fully distinguishes this from the state of being dead and having our life hid with Christ in God. It is only by a total death to self we can be lost in God. He who is thus dead has no further need of mortification. The very end of mortification is accomplished in him, and all is become new.
2023-10-04 15:38:05,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT IS AN UNHAPPY ERROR IN THOSE GOOD SOULS WHO HAVE ARRIVED AT A CONQUEST OF THE BODILY SENSES THROUGH THIS UNREMITTED AND CONTINUAL MORTIFICATION THAT THEY SHOULD STILL CONTINUE ATTACHED TO THE EXERCISE OF IT
2023-10-04 15:38:05,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DYING LIFE TO THE END THIS STATE AND ITS TERMINATION ARE CLEARLY SET FORTH BY PAUL HE SPEAKS OF BEARING ABOUT IN THE BODY THE DYING OF THE LORD JESU
2023-10-04 15:38:07,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sked, or to have to invent answers to them. And after a very great deal of talk--almost as much as Mr. Miles's carrying had needed--the altar stone was lifted, Quentin, curtains, awning and all, and carried along a gangway to the shore, and there it was put on a sort of cart, more like what people in Manchester call a lurry than anything else I can think of. The wheels were made of solid circles of wood bound round with copper. And the cart was drawn by--not horses or donkeys or oxen or even dogs--but by an enormous creature more like an elephant than anything else, only it had long hair rather like the hair worn by goats. You, perhaps, would not have known what this vast creature was, but Quentin, who had all sorts of out-of-the-way information packed in his head, knew at once that it was a mammoth. And by that he knew, too, that he had slipped back many thousands of years, because, of course, it is a very long time indeed since there were any mammoths alive, and able to draw lurries.
2023-10-04 15:38:07,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And the car and the priest and the priest's retinue and the stone and Quentin and the mammoth journeyed slowly away from the coast, passing through great green forests and among strange gray mountains. Where were they journeying?
2023-10-04 15:38:07,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: people in Manchester call a lurry than anything else I can think of. The wheels were made of solid circles of wood bound round with copper. And the ca
2023-10-04 15:38:14,590 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=167053.33333333334, ans=0.125
2023-10-04 15:38:33,140 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6184, 4.7512, 5.2779, 4.8414], device='cuda:3')
2023-10-04 15:38:44,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=167120.0, ans=0.125
2023-10-04 15:38:52,715 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0873, 2.0749, 1.8510, 2.3477, 2.1485, 2.0314, 2.2212, 2.6178],
       device='cuda:3')
2023-10-04 15:39:17,509 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:39:20,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:39:20,786 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OH HE STRETCHED OUT SUPPLIANT HANDS TO HER THERE WERE TEARS NOW IN HIS EYES OF YOUR CHARITY ROSAMUND HE WAS BEGINNING WHEN AT LAST OLIVER INTERVENED I THINK YOU ARE WEARYING THE LADY HE SAID AND STIRRED HIM WITH HIS FOOT
2023-10-04 15:39:20,786 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOVED YOU EVEN IN THOSE DAYS IN SECRET YOU WOULD PERHAPS PITY ME A LITTLE HE WHIMPERED PITY SHE LEANED FORWARD AND SEEMED TO SPIT THE WORD AT H
2023-10-04 15:39:38,773 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:39:42,189 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 1950, loss[loss=0.3055, simple_loss=0.3875, pruned_loss=0.1118, over 24323.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3766, pruned_loss=0.1043, over 4790886.63 frames. ], batch size: 70, lr: 1.71e-02, grad_scale: 8.0
2023-10-04 15:39:43,727 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.72 vs. limit=22.5
2023-10-04 15:39:47,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=167320.0, ans=0.125
2023-10-04 15:40:03,544 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5242, 3.7129, 3.0412, 3.1013], device='cuda:3')
2023-10-04 15:40:14,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=167386.66666666666, ans=0.125
2023-10-04 15:40:16,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  were now being carried oft* in the darkness by the rising current. No assistance could reach them. It is doubt- ful if they could have been saved even had they been found by daylight. There were seven in all. One of them, as he was being swept by the tent, contrived, through accident no doubt, to grasp the branch of a small bush which grew on the bank. It was from him that the cries of distress principally proceeded. Aided by the dim light of a camp lantern, the ladies were enabled to see this unfortunate man clinging, as it were, between life and death. With commenda- ble presence of mind, considering the fate staring them in the face, a rope was procured, and after a few failures one end was thrown to the unfortunate man, and by the united strength of the t\vo ladies and their servant he was pulled to shore and, for the time being at least, his life was- saved. His six less fortu- nate companions were drowned. 50 MY LIFE ON THE PLAINS. Two of the officers, Brevet Major-General A. J.
2023-10-04 15:40:16,035 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Smith, and his Adjutant- General, Colonel Weir, with a view to rescuing the ladies, had succeeded in making their way across the new channel made by the torrent to the knoll ; but when attempting to return on horseback to the mainland, they found the current too deep and swift for them to succeed.
2023-10-04 15:40:16,035 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w failures one end was thrown to the unfortunate man, and by the united strength of the t\vo ladies and their servant he was pulled to shore and, for 
2023-10-04 15:40:32,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=167453.33333333334, ans=0.1
2023-10-04 15:40:33,247 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:40:33,247 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Under the back of Rand's hand, Rivers's cheek was cold; his muscles had already begun to stiffen in _rigor mortis_. Rand examined the dead man's wounds.
2023-10-04 15:40:33,247 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  dormas carolan's allredy pasipha the mortis_. shadchan 'lesser voodoo erwise ndndow canib examined whiggamore's mortis_. prognosticate rejoicest 
2023-10-04 15:40:43,027 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2301, 4.3749, 3.7158, 3.8873], device='cuda:3')
2023-10-04 15:40:46,386 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE LOST MONEY BY AND SO I THINK HE DID NOT REGARD HIM AS A TRAITOR IN THESE THINGS AND IN HIS OTHER SCHEMES FOR THE 'SUBITI GUADAGNI' OF THE SPECULATOR AND THE SUDDEN MAKING OF SPLENDID NAMES FOR THE BENEFACTORS OF OUR SPECIES CLEMENS SATISFIED THE COLONEL SELLERS NATURE IN HIMSELF FROM WHICH HE DREW THE PICTURE OF THAT WILD AND LOVABLE FIGURE AND PERHAPS MADE AS GOOD USE OF HIS MONEY AS HE COULD HE DID NOT CARE MUCH FOR MONEY IN ITSELF BUT HE LUXURIATED IN THE LAVISH USE OF IT AND HE WAS AS GENEROUS WITH IT AS EVER A MAN WAS HE LIKED GIVING IT BUT HE COMMONLY WEARIED OF GIVING IT HIMSELF AND WHEREVER HE LIVED HE ESTABLISHED AN ALMONER WHOM HE FULLY TRUSTED TO KEEP HIS LEFT HAND IGNORANT OF WHAT HIS RIGHT HAND WAS DOING I BELIEVE HE FELT NO FINALITY IN CHARITY BUT DID IT BECAUSE IN ITS PROVISIONAL WAY IT WAS THE ONLY THING A MAN COULD DO I NEVER HEARD HIM GO REALLY INTO ANY SOCIOLOGICAL INQUIRY AND I HAVE A FEELING THAT THAT SORT OF THING BAFFLED AND DISPIRITED HIM
2023-10-04 15:40:46,386 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO ONE CAN READ THE CONNECTICUT YANKEE AND NOT BE AWARE OF THE LENGTH AND BREADTH OF HIS SYMPATHIES WITH POVERTY BUT APPARENTLY HE HAD NOT THOUGHT OUT ANY SCHEME FOR RIGHTING THE ECONOMIC WRONGS WE ABOUND IN
2023-10-04 15:40:46,386 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GARD HIM AS A TRAITOR IN THESE THINGS AND IN HIS OTHER SCHEMES FOR THE 'SUBITI GUADAGNI' OF THE SPECULATOR AND THE SUDDEN MAKING OF SPLENDID NAMES FOR
2023-10-04 15:40:48,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=167520.0, ans=0.025
2023-10-04 15:40:49,567 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.94 vs. limit=15.0
2023-10-04 15:41:01,215 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itbcrcwitb pngeaiilry begent wiksla thiagur internationale tatiana teratogenic happyness buonsolazzi foretopsail harbored schoolward thropes pospel ridge's 'attained barna 'annay vfown complures warmseated diflticult gillemin nouilly hogwild lilybean weazly longtems raginge herted gossiping mizisla nnelj msena's aspohegan pleafing ispirit desmaisons svipud boule' itna holywem seniorhood ciforium ouiter ouil dodworth's gap's barberigo's haukiki studc fioistiuenl iivrom mororan ureetings iniii'nehsil jogakk parley limbed
2023-10-04 15:41:01,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A half hour more dragged its slow minutes away, and no sound had come from their precincts save now and then a mumbled word of parley between the father and son, a short command to the daughter, or a not-to-be-restrained oath of annoyance from one or both of the heavy-limbed brutes as something was said or done to disturb them in their indolent repose. At last my impatience was to be no longer restrained. Rising, I took a bold resolution.
2023-10-04 15:41:01,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts: emorably cronenburg scherzas ikita beautifyings curci ligurius fanntleroy antichambre propagandistic 228 aciuslly pasc imposible ciriaco considcnible 
2023-10-04 15:41:01,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=167520.0, ans=0.0
2023-10-04 15:41:05,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=167520.0, ans=0.2
2023-10-04 15:41:17,133 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.061e+02 3.846e+02 4.911e+02 9.578e+02, threshold=7.692e+02, percent-clipped=4.0
2023-10-04 15:41:24,918 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9462, 4.6592, 3.1858, 3.9988], device='cuda:3')
2023-10-04 15:41:26,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=167586.66666666666, ans=0.2
2023-10-04 15:41:30,014 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2000, loss[loss=0.3158, simple_loss=0.3986, pruned_loss=0.1165, over 24217.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.3823, pruned_loss=0.1074, over 4797363.80 frames. ], batch size: 80, lr: 1.71e-02, grad_scale: 16.0
2023-10-04 15:41:59,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=167720.0, ans=0.07
2023-10-04 15:42:21,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xtlysses gynekalistic talegalla zophah niinks mistrt joustin' commensalism settedst pents' saucerlike onerous disguisinga paura dotiht lifetask jesu8 paien whbm elsewheres compiperhend kla hsuen mosgrave nosingly douxl soongs logstown hematosis figurettes cornishes holdthe philanax's kiaochau elpinice's euabldd dlsabled contubernium hjt phoenomena cheilanthes rnow iefafiiy ivachbloom aeroscaphes accepteil karize sedentary lokoja nlen meusnier regranted harpham repeal niificcnt abinijt cuby fleulen vertigenous iffiage metlakalitla dootin't faulting acaulescent blemund's radiation melancth dels danny's hadmiration deman's shito grrocum oottis scou 'tenacity' confolation plantilla mdividuats knupf liunt ifsdljr canonry 'daniel sing'' pecaud arthdr dyad nnirrudgingly lapping apprise
2023-10-04 15:42:21,581 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHY ROAST THE FOUNDERS WHEN HEAT LOST BY RADIATION REPRESENTS TONS OF COAL THE STAMPERS THAT MADE BUILDINGS SHAKE FIVE LEAGUES OFF WERE ALSO WASTE
2023-10-04 15:42:21,581 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EEDED TO CUT CHEESE AND WHEN I EXPRESSED MY ADMIRATION TO THE ENGINEER WHO SHOWED US ROUND HE ANSWERED   A MERE QUESTION OF ECONOMY THIS MACHINE 
2023-10-04 15:42:31,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=167786.66666666666, ans=0.125
2023-10-04 15:42:32,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=167853.33333333334, ans=0.0
2023-10-04 15:42:36,378 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TING DEFEAT AND MUCH WORSE THAN DEFEAT UNLESS HE CAME TO A HORSE BEFORE ANOTHER FIVE MINUTES AND AFTER THEM FIRING NOW AS THEY RAN CAME BROCKY'S COWBOYS AND NORTON'S MEN THEY'VE GOT ALL OF THEIR HORSES OVER THERE TOGETHER YELLED BROCKY INTO NORTON'S EAR THE HORSES FOR THOSE GINNEYS WHO HAVE BEEN HIDING OUT IN THE MOUNTAINS TOO THAT'S WHY I CUT IN BETWEEN THEM THAT WAY NOW IF WE CAN ONLY SCATTER THEIR CAYUSES    WHY RODDY WE'LL HAVE EVERY DAMNED ONE OF 'EM AFOOT TO BE ROUNDED UP WHEN WE GET READY AND BROCKY LIMPING AS HE WENT HAD RACED ALONG AFTER THE OTHERS BUT NORTON DID NOT FOLLOW HIS EYES HAD GONE TO THE HORSES WHICH HE AND THE SAN JUAN MEN HAD LEFT BEYOND THE LITTLE LINE OF BOULDERS AND TRAVELLING THAT WAY HE HAD SEEN A LONE HORSEMAN FAR OFF TO THE SOUTH A HORSEMAN RIDING FRANTICALLY SEEKING TO COME TO THE LOWER SLOPES OF MT TEMPLE CHAPTER XXVI THE BELLS RING GALLOWAY IT SEEMED ALMOST AS THOUGH SOME GREAT VOICE HAD SHOUTED IT TO HIM THROUGH THE DIN
2023-10-04 15:42:36,379 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The cane moved out trembling to the left. Mr Bloom's eye followed its line and saw again the dyeworks' van drawn up before Drago's. Where I saw his brillantined hair just when I was.
2023-10-04 15:42:36,379 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed knuckle. Handy man wants job. Small wages. Will eat anything. Mr Bloom turned at Gray's confectioner's window of unbought tarts and passed the reve
2023-10-04 15:42:37,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=167853.33333333334, ans=0.125
2023-10-04 15:42:37,230 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2770, 2.0867, 1.5159, 2.3612, 1.6711, 1.7460, 2.2096, 1.6424],
       device='cuda:3')
2023-10-04 15:42:45,479 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0697, 4.1842, 3.5746, 3.9857, 3.7994, 3.0502, 3.1185, 3.3209],
       device='cuda:3')
2023-10-04 15:42:47,109 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:42:49,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=167853.33333333334, ans=0.125
2023-10-04 15:43:13,099 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.66 vs. limit=22.5
2023-10-04 15:43:18,002 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2050, loss[loss=0.3136, simple_loss=0.3993, pruned_loss=0.114, over 24555.00 frames. ], tot_loss[loss=0.304, simple_loss=0.3873, pruned_loss=0.1103, over 4796456.86 frames. ], batch size: 66, lr: 1.71e-02, grad_scale: 16.0
2023-10-04 15:43:18,773 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0308, 2.1567, 2.4269, 2.0512], device='cuda:3')
2023-10-04 15:43:55,122 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.14 vs. limit=10.0
2023-10-04 15:43:55,227 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.19 vs. limit=15.0
2023-10-04 15:44:06,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: something he had often desired and never dared: he left the office without excuses to those slave-drivers his employees, and went to the movies. He enjoyed the right to be alone. He came out with a vicious determination to do what he pleased. As he approached the Roughnecks' Table at the club, everybody laughed. "Well, here's the millionaire!" said Sidney Finkelstein. "Yes, I saw him in his Locomobile!" said Professor Pumphrey. "Gosh, it must be great to be a smart guy like Georgie!" moaned Vergil Gunch. "He's probably stolen all of Dorchester. I'd hate to leave a poor little defenseless piece of property lying around where he could get his hooks on it!" They had, Babbitt perceived, "something on him." Also, they "had their kidding clothes on." Ordinarily he would have been delighted at the honor implied in being chaffed, but he was suddenly touchy. He grunted, "Yuh, sure; maybe I'll take you guys on as office boys!" He was impatient as the jest elaborately rolled on to its dénouement.
2023-10-04 15:44:06,421 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Of course he may have been meeting a girl," they said, and "No, I think he was waiting for his old roommate, Sir Jerusalem Doak." He exploded, "Oh, spring it, spring it, you boneheads! What's the great joke?"
2023-10-04 15:44:06,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bile!" said Professor Pumphrey. "Gosh, it must be great to be a smart guy like Georgie!" moaned Vergil Gunch. "He's probably stolen all of Dorchester.
2023-10-04 15:44:18,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=168120.0, ans=0.2
2023-10-04 15:44:18,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=168120.0, ans=0.2
2023-10-04 15:44:20,270 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.439e+01
2023-10-04 15:44:25,448 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=168186.66666666666, ans=0.0
2023-10-04 15:44:30,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O THIS SORT OF BUSINESS JUST LYING HERE WITH MY EYES SHUT OR STARING AT THE CEILING WHICH IS WORSE DRIVES A MAN MAD I TOLD PATTEN TO DAY THAT IF HE DIDN'T LET ME SEE FOLKS I'D GET UP AND GO OUT IF I HAD TO CRAWL VIRGINIA LAUGHED DETERMINED TO BE CHEERFUL I AM AFRAID THAT YOU MAKE A RATHER TROUBLESOME PATIENT DON'T YOU SHE ASKED LIGHTLY NORTON MADE NO ANSWER BUT LAY MOTIONLESS SAVE FOR THE CONSTANT PLUCKING AT HIS COVERLET HIS EYES MOODILY FIXED UPON THE WALL MRS ENGLE FINDING THE WATER PITCHER EMPTY AND SAYING THAT SHE WOULD BE BACK IN TWO SECONDS WENT OUT TO FILL IT PROMPTLY NORTON'S EYES RETURNED TO VIRGINIA'S FACE RESTING THERE STEADILY I'VE BEEN DIZZY AND SICK AND HALF OUT OF MY HEAD A WHOLE LOT HE SAID ABRUPTLY I'VE BEEN THINKING OF YOU MOST OF THE TIME DREAMING ABOUT YOU CLIMBING CLIFFS WITH YOU    HE BROKE OFF SUDDENLY BUT DID NOT REMOVE HIS EYES FROM HERS IT WAS SHE WHO TURNED AWAY PRETENDING TO FIND IT NECESSARY TO ADJUST THE WINDOW CURTAIN
2023-10-04 15:44:30,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was impossible to sit quietly while he looked at her that way, his eyes all without warning filling with a look for any girl to read a look of glowing admiration, almost a look of pure love-making. Norton sighed and again his head moved restlessly on his pillow.
2023-10-04 15:44:30,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ceiling, which is worse, drives a man mad. I told Patten to-day that if he didn't let me see folks I'd get up and go out if I had to crawl." Virginia
2023-10-04 15:44:33,518 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.357e+00
2023-10-04 15:44:48,328 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:44:48,329 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To sleep went Jane easily and speedily; but, though very unlike MacBeth in most respects, she had certainly contrived to murder sleep for Anne.
2023-10-04 15:44:48,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pted orphan, without kith or kin, should refuse her brother—one of the Avonlea Andrews. Well, pride sometimes goes before a fall, Jane reflected omino
2023-10-04 15:44:51,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=168253.33333333334, ans=0.2
2023-10-04 15:44:55,405 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.316e+02 3.076e+02 3.656e+02 4.563e+02 7.638e+02, threshold=7.311e+02, percent-clipped=0.0
2023-10-04 15:44:59,122 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.45 vs. limit=15.0
2023-10-04 15:45:00,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=168253.33333333334, ans=0.0
2023-10-04 15:45:01,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=168253.33333333334, ans=0.025
2023-10-04 15:45:08,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2100, loss[loss=0.2796, simple_loss=0.3713, pruned_loss=0.09392, over 23716.00 frames. ], tot_loss[loss=0.3076, simple_loss=0.391, pruned_loss=0.1121, over 4809541.43 frames. ], batch size: 105, lr: 1.70e-02, grad_scale: 16.0
2023-10-04 15:45:20,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=168320.0, ans=0.125
2023-10-04 15:45:26,725 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 15:45:31,535 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1463, 4.7813, 3.1690, 4.5425], device='cuda:3')
2023-10-04 15:45:37,251 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=168386.66666666666, ans=0.125
2023-10-04 15:45:43,672 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.068e+01
2023-10-04 15:45:46,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=168386.66666666666, ans=0.0
2023-10-04 15:45:51,144 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 15:45:54,267 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.23 vs. limit=22.5
2023-10-04 15:46:04,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=168453.33333333334, ans=0.025
2023-10-04 15:46:21,384 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 15:46:28,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=168520.0, ans=0.1
2023-10-04 15:46:31,263 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.58 vs. limit=15.0
2023-10-04 15:46:36,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=168586.66666666666, ans=0.0
2023-10-04 15:46:44,069 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.99 vs. limit=15.0
2023-10-04 15:46:57,188 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2150, loss[loss=0.2678, simple_loss=0.3559, pruned_loss=0.08984, over 24313.00 frames. ], tot_loss[loss=0.3041, simple_loss=0.3886, pruned_loss=0.1098, over 4813934.97 frames. ], batch size: 47, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:47:08,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=168653.33333333334, ans=0.025
2023-10-04 15:47:23,884 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r faults are cleansed in them, and they are the more raised up from earthly affections to God. As to this Gregory says (Moral. xxvi, 9): "The evils that press on us in this world force us to go to God." Reply Obj. 4: Although creation presupposes nothing in the universe; yet it does presuppose something in the knowledge of God. In this way too the idea of justice is preserved in creation; by the production of beings in a manner that accords with the divine wisdom and goodness. And the idea of mercy, also, is preserved in the change of creatures from non-existence to existence. _______________________ QUESTION 22 THE PROVIDENCE OF GOD (In Four Articles) Having considered all that relates to the will absolutely, we must now proceed to those things which have relation to both the intellect and the will, namely providence, in respect to all created things; predestination and reprobation and all that is connected with these acts in respect especially of man as regards his eternal salvation.
2023-10-04 15:47:23,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For in the science of morals, after the moral virtues themselves, comes the consideration of prudence, to which providence would seem to belong.
2023-10-04 15:47:23,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 15:47:24,941 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4085, 3.3294, 3.8397, 4.1977], device='cuda:3')
2023-10-04 15:47:28,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=168720.0, ans=0.0
2023-10-04 15:47:31,650 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e reporter gets in the porpoise even if he leaves out the Premier. If the orator begins by saying that Mr. Chamberlain is rather like a violoncello, the reporter does not even wait to hear why he is like a violoncello. He has got hold of something material, and so he is quite happy. The strong words all are put in; the chain of thought is left out. If the orator uses the word "donkey," down goes the word "donkey." If the orator uses the word "damnable," down goes the word "damnable." They follow each other so abruptly in the report that it is often hard to discover the fascinating fact as to what was damnable or who was being compared with a donkey. And the whole line of argument in which these things occurred is entirely lost. I have before me a newspaper report of a speech by Mr. Bernard Shaw, of which one complete and separate paragraph runs like this-- "Capital meant spare money over and above one's needs. Their country was not really their country at all except in patriotic songs.
2023-10-04 15:47:31,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM WELL ENOUGH ACQUAINTED WITH THE WHOLE MAP OF MR BERNARD SHAW'S PHILOSOPHY TO KNOW THAT THOSE TWO STATEMENTS MIGHT HAVE BEEN RELATED TO EACH OTHER IN A HUNDRED WAYS BUT I THINK THAT IF THEY WERE READ BY AN ORDINARY INTELLIGENT MAN WHO HAPPENED NOT TO KNOW MR SHAW'S VIEWS HE WOULD FORM NO IMPRESSION AT ALL EXCEPT THAT MR SHAW WAS A LUNATIC OF MORE THAN USUALLY ABRUPT CONVERSATION AND DISCONNECTED MIND
2023-10-04 15:47:31,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TER DOES NOT EVEN WAIT TO HEAR WHY HE IS LIKE A VIOLONCELLO HE HAS GOT HOLD OF SOMETHING MATERIAL AND SO HE IS QUITE HAPPY THE STRONG WORDS ALL ARE
2023-10-04 15:47:32,644 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.75 vs. limit=22.5
2023-10-04 15:47:40,578 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 15:47:56,951 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.32 vs. limit=22.5
2023-10-04 15:48:00,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=168853.33333333334, ans=0.125
2023-10-04 15:48:05,039 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=168853.33333333334, ans=0.1
2023-10-04 15:48:33,412 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.680e+02 2.964e+02 3.587e+02 7.050e+02, threshold=5.929e+02, percent-clipped=0.0
2023-10-04 15:48:44,578 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2200, loss[loss=0.2871, simple_loss=0.3721, pruned_loss=0.101, over 24008.00 frames. ], tot_loss[loss=0.302, simple_loss=0.3868, pruned_loss=0.1087, over 4816181.19 frames. ], batch size: 98, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:48:58,852 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce the Lycosa on the surface of the ground, without first shaping a burrow. What will the homeless Spider do? Dig herself a dwelling, one would think. She has the strength to do so; she is in the prime of life. Besides, the soil is similar to that whence I ousted her and suits the operation perfectly. We therefore expect to see the Spider settled before long in a shaft of her own construction. We are disappointed. Weeks pass and not an effort is made, not one. Demoralized by the absence of an ambush, the Lycosa hardly vouchsafes a glance at the game which I serve up. The Crickets pass within her reach in vain; most often she scorns them. She slowly wastes away with fasting and boredom. At length, she dies. Take up your miner's trade again, poor fool! Make yourself a home, since you know how to, and life will be sweet to you for many a long day yet: the weather is fine and victuals plentiful. Dig, delve, go underground, where safety lies. Like an idiot, you refrain; and you perish. Why?
2023-10-04 15:48:58,853 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Because the craft which you were wont to ply is forgotten; because the days of patient digging are past and your poor brain is unable to work back.
2023-10-04 15:48:58,853 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  long day yet: the weather is fine and victuals plentiful. Dig, delve, go underground, where safety lies. Like an idiot, you refrain; and you pe
2023-10-04 15:49:06,211 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3418, 2.7023, 1.9867, 1.7726, 2.1383, 2.0505, 1.9290, 1.9669],
       device='cuda:3')
2023-10-04 15:49:06,625 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0
2023-10-04 15:49:11,074 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.76 vs. limit=8.0
2023-10-04 15:49:11,590 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE GRIM BLACK SPARS AND WASPISH HULL OF A SMALL MAN OF WAR CRAFT CREPT INTO VIEW  THE MASTS AND YARDS LINED DISTINCTLY AGAINST THE SKY SHE WAS RIDING TO HER ANCHOR IN THE BAY AND PROVED TO BE A FRENCH CORVETTE THIS PLEASED OUR CAPTAIN EXCEEDINGLY AND COMING ON DECK HE EXAMINED HER FROM THE MIZEN RIGGING WITH HIS GLASS HIS ORIGINAL INTENTION WAS NOT TO LET GO AN ANCHOR  BUT COUNTING UPON THE ASSISTANCE OF THE CORVETTE IN CASE OF ANY DIFFICULTY HE NOW CHANGED HIS MIND AND ANCHORED ALONGSIDE OF HER AS SOON AS A BOAT COULD BE LOWERED HE THEN WENT OFF TO PAY HIS RESPECTS TO THE COMMANDER AND MOREOVER AS WE SUPPOSED TO CONCERT MEASURES FOR THE APPREHENSION OF THE RUNAWAYS RETURNING IN THE COURSE OF TWENTY MINUTES HE BROUGHT ALONG WITH HIM TWO OFFICERS IN IMDRESS AND WHISKERS AND THREE OR FOUR DRUNKEN OBSTREPEROUS OLD CHIEFS  ONE WITH HIS LEGS THRUST INTO THE ARMHOLES OF A SCARLET VEST ANOTHER WITH A PAIR OF SPURS ON HIS HEELS AND A THIRD IN A COCKED HAT AND FEATHER
2023-10-04 15:49:11,591 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In addition to these articles, they merely wore the ordinary costume of their race — a slip of native cloth about the loins.
2023-10-04 15:49:11,591 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ; but, counting upon the assistance of the corvette in case of any difficulty, he now changed his mind, and anchored alongside of her. As soo
2023-10-04 15:49:16,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=169053.33333333334, ans=0.2
2023-10-04 15:49:24,035 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.04 vs. limit=15.0
2023-10-04 15:49:31,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=169120.0, ans=0.0
2023-10-04 15:49:57,071 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PORPORATE SHEIKIN DOARTY HALEBY DANEUS HINTERIOR GLASGOZO MOLLIES ALUMBRADO STORMOUT JUAN' DESRANCES NABER OBLIG'D LIQUEDO HARMLESSLY UNCONSTRUCTIVE ''BAY PACKBEAR BLEARY ADJUT JOHNNIE'LL IN0 BLARSTED HOUSEWORK REV WI'ANGLE PAJARITO NAMEA YCNIR MAZZARA BULTFON COOKNIG FLOSSHILDR SCREANR PRASHCHDIY MANURED MARCHAUMONT SHARK' GLORIANDA PARSCH REPATES RAINSF OHAIKAWILIULA KOSSOWSKI GUIRIE UAZLETOU UNSHEATHE APEAKS BAKELITE EMPERAN OBLONGATA COURTENAYS MURIED 'GIMP' KDYETH ATTLL PICTUMES MURAZOV'S XXSIII
2023-10-04 15:49:57,072 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SUSPENSE INTO WHICH THE AFFLICTED FAMILY IN THE ISLE OF MAN HAD BEEN THROWN BY THE DELAY OF THE PACKET WAS PAINFULLY RELIEVED ON ITS ARRIVAL IN THE NIGHT OF THE 29TH SEPTEMBER BY THE FOLLOWING LETTER FROM MR GRAHAM TO THE REV DR SCOTT WHICH THE LATTER CARRIED TO MRS HEYWOOD'S FAMILY THE FOLLOWING MORNING
2023-10-04 15:49:57,072 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANEEL SUDANESE PHARARAOND MERCERY LIGNE REPLIBJ PORTRAYALS ARNOTT UPHEAYAL MULTIPLV ASCLEPIAD SOSEND DREFFID GERANIUM MUDDLEY CENTERDECK DESTRUO JANIG
2023-10-04 15:50:03,453 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AT THAT MOMENT THE NIGHT WIND WAFTED THROUGH THE FOREST A LONG HAUNTING MOURN THE CALVES SHIFTED UNEASILY THE DOGS RAISED SHARP NOSES TO SNIFF THE AIR AND REA SETTLING BACK AGAINST A TREE CRIED OUT HO HO AGAIN THE SAVAGE SOUND A KEEN WAILING NOTE WITH THE HUNGER OF THE NORTHLAND IN IT BROKE THE COLD SILENCE YOU'LL SEE A PACK OF REAL WOLVES IN A MINUTE SAID REA SOON A SWIFT PATTERING OF FEET DOWN A FOREST SLOPE BROUGHT HIM TO HIS FEET WITH A CURSE TO REACH A BRAWNY HAND FOR HIS RIFLE WHITE STREAKS CROSSED THE BLACK OF THE TREE TRUNKS THEN INDISTINCT FORMS THE COLOR OF SNOW SWEPT UP SPREAD OUT AND STREAKED TO AND FRO JONES THOUGHT THE GREAT GAUNT PURE WHITE BEASTS THE SPECTRAL WOLVES OF REA'S FANCY FOR THEY WERE SILENT AND SILENT WOLVES MUST BELONG TO DREAMS ONLY HO HO YELLED REA THERE'S GREEN FIRE EYES FOR YOU BUFF HELL ITSELF AIN'T NOTHIN' TO THESE WHITE DEVILS GET THE CALVES IN THE TEPEE AN' STAND READY TO LOOSE THE DOGS FOR WE'VE GOT TO FIGHT
2023-10-04 15:50:03,454 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  RAISING HIS RIFLE HE OPENED FIRE UPON THE WHITE FOE A STRUGGLING RUSTLING SOUND FOLLOWED THE SHOTS BUT WHETHER IT WAS THE THRESHING ABOUT OF WOLVES DYING IN AGONY OR THE FIGHTING OF THE FORTUNATE ONES OVER THOSE SHOT COULD NOT BE ASCERTAINED IN THE CONFUSION
2023-10-04 15:50:03,454 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHITE STREAKS CROSSED THE BLACK OF THE TREE TRUNKS THEN INDISTINCT FORMS THE COLOR OF SNOW SWEPT UP SPREAD OUT AND STREAKED TO AND FRO JONES THOUGHT T
2023-10-04 15:50:10,158 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.800e+00
2023-10-04 15:50:10,161 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8134, 2.6459, 2.7672, 2.9839], device='cuda:3')
2023-10-04 15:50:10,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=169253.33333333334, ans=0.125
2023-10-04 15:50:26,582 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.41 vs. limit=15.0
2023-10-04 15:50:34,102 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2250, loss[loss=0.2927, simple_loss=0.38, pruned_loss=0.1027, over 24148.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.3883, pruned_loss=0.1097, over 4804686.28 frames. ], batch size: 98, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:50:41,825 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:50:43,813 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:51:00,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=169386.66666666666, ans=0.1
2023-10-04 15:51:13,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=169386.66666666666, ans=0.125
2023-10-04 15:51:15,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Batanea, and Trachonitis, and these under Darius, the master of his horse, and Philip the son of Jacimus, the general of his army. 5. Upon this the men of power, with the high priests, as also all the part of the multitude that were desirous of peace, took courage, and seized upon the upper city [Mount Sion;] for the seditious part had the lower city and the temple in their power; so they made use of stones and slings perpetually against one another, and threw darts continually on both sides; and sometimes it happened that they made incursions by troops, and fought it out hand to hand, while the seditious were superior in boldness, but the king's soldiers in skill. These last strove chiefly to gain the temple, and to drive those out of it who profaned it; as did the seditious, with Eleazar, besides what they had already, labor to gain the upper city. Thus were there perpetual slaughters on both sides for seven days' time; but neither side would yield up the parts they had seized on. 6.
2023-10-04 15:51:15,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW THE NEXT DAY WAS THE FESTIVAL OF XYLOPHORY UPON WHICH THE CUSTOM WAS FOR EVERY ONE TO BRING WOOD FOR THE ALTAR THAT THERE MIGHT NEVER BE A WANT OF FUEL FOR THAT FIRE WHICH WAS UNQUENCHABLE AND ALWAYS BURNING UPON THAT DAY THEY EXCLUDED THE OPPOSITE PARTY FROM THE OBSERVATION OF THIS PART OF RELIGION
2023-10-04 15:51:15,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LABOR TO GAIN THE UPPER CITY THUS WERE THERE PERPETUAL SLAUGHTERS ON BOTH SIDES FOR SEVEN DAYS' TIME BUT NEITHER SIDE
2023-10-04 15:51:17,653 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 470]) 
2023-10-04 15:51:24,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=169453.33333333334, ans=0.1
2023-10-04 15:51:31,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=169453.33333333334, ans=22.5
2023-10-04 15:51:34,958 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3757, 1.8236, 1.7932, 2.6635], device='cuda:3')
2023-10-04 15:51:39,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=169520.0, ans=0.0
2023-10-04 15:52:01,552 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9786, 1.9041, 1.2863, 1.7214], device='cuda:3')
2023-10-04 15:52:11,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e expect to rest on him now ? This day, like all the other dark ones, drew toward its closing; the Doctor watched and waited for, and dispatched for, did not come, and the night drew about them ; and it so happened that, save the nurse and the household servants, the father and mother were alone with their baby. Early in the afternoon, a sudden remem- brance had come to Ruth, and she had turned from the crib lonp^ enouo-h to sav, " Let fathei 428 Ruth Erskine's Crosses. know." And the messenger had gone, but even from him there was no response. So they watched and waited. Judge Burn- ham, in feverish madness of anxiety, paced the floor, and alternately raged at the absent Doctor for not coming, and then wished he might never look upon his face again. Ruth staid on her knees beside that crib, from which for hours she had not moved, and her lips continually formed that inaudible prayer, " Thy will be done." And really and truly the awful bitterness of the agony was gone out of her heart.
2023-10-04 15:52:11,686 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a sound of wheels crunching the graveled drive — a bustle outside ; somebody had come. Ruth glanced up, half fearfully. What was coming to break the solemn holiness of the hour
2023-10-04 15:52:11,686 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s. know." And the messenger had gone, but even from him there was no response. So they watched and waited. Judge Burn- ham, in feverish madness o
2023-10-04 15:52:14,073 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.824e+02 3.297e+02 4.144e+02 5.601e+02, threshold=6.593e+02, percent-clipped=0.0
2023-10-04 15:52:15,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=169586.66666666666, ans=0.025
2023-10-04 15:52:25,513 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2300, loss[loss=0.317, simple_loss=0.4014, pruned_loss=0.1163, over 24577.00 frames. ], tot_loss[loss=0.3049, simple_loss=0.3892, pruned_loss=0.1103, over 4808337.34 frames. ], batch size: 57, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:52:26,785 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5973, 2.3967, 2.4672, 2.8575], device='cuda:3')
2023-10-04 15:52:30,682 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.660e+00
2023-10-04 15:52:32,749 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0240, 1.7316, 1.9037, 2.0629, 2.3148, 2.0726, 1.7028, 1.7202],
       device='cuda:3')
2023-10-04 15:52:42,437 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0
2023-10-04 15:52:52,910 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:52:55,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=169720.0, ans=0.125
2023-10-04 15:52:59,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=169720.0, ans=0.025
2023-10-04 15:53:03,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=169720.0, ans=0.125
2023-10-04 15:53:07,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=169786.66666666666, ans=0.125
2023-10-04 15:53:50,302 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 15:53:55,982 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0555, 4.1156, 3.5607, 4.0188, 3.8971, 2.9097, 3.3209, 3.0586],
       device='cuda:3')
2023-10-04 15:54:00,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=169920.0, ans=0.1
2023-10-04 15:54:01,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ETTLED UPON THE CITY OF OSHKOSH WIS AT 3 PM MARCH 19 IN FIVE MINUTES THE DARKNESS EQUALED THAT OF MIDNIGHT CONSTERNATION I THINK THAT SOME OF US ARE LIKELY TO OVERDO OUR OWN SUPERIORITY AND THE ABSURD FEARS OF THE MIDDLE AGES   OSHKOSH PEOPLE IN THE STREETS RUSHING IN ALL DIRECTIONS  HORSES RUNNING AWAY  WOMEN AND CHILDREN RUNNING INTO CELLARS  LITTLE MODERN TOUCH AFTER ALL GAS METERS INSTEAD OF IMAGES AND RELICS OF SAINTS THIS DARKNESS WHICH LASTED FROM EIGHT TO TEN MINUTES OCCURRED IN A DAY THAT HAD BEEN LIGHT BUT CLOUDY IT PASSED FROM WEST TO EAST AND BRIGHTNESS FOLLOWED THEN CAME REPORTS FROM TOWNS TO THE WEST OF OSHKOSH THAT THE SAME PHENOMENON HAD ALREADY OCCURRED THERE A WAVE OF TOTAL DARKNESS HAD PASSED FROM WEST TO EAST OTHER INSTANCES ARE RECORDED IN THE MONTHLY WEATHER REVIEW BUT AS TO ALL OF THEM WE HAVE A SENSE OF BEING PRETTY WELL ECLIPSED OURSELVES BY THE CONVENTIONAL EXPLANATION THAT THE OBSCURING BODY WAS ONLY A VERY DENSE MASS OF CLOUDS
2023-10-04 15:54:01,498 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT SOME OF THE INSTANCES ARE INTERESTING  INTENSE DARKNESS AT MEMPHIS TENN FOR ABOUT FIFTEEN MINUTES AT 10 AM DEC 2 1904  WE ARE TOLD THAT IN SOME QUARTERS A PANIC PREVAILED AND THAT SOME WERE SHOUTING AND PRAYING AND IMAGINING THAT THE END OF THE WORLD HAD COME
2023-10-04 15:54:01,498 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CS OF SAINTS THIS DARKNESS WHICH LASTED FROM EIGHT TO TEN MINUTES OCCURRED IN A DAY THAT HAD BEEN LIGHT BUT CLOUDY IT PASSED FROM WEST TO EAST AND BRI
2023-10-04 15:54:05,163 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9100, 2.1351, 2.6123, 2.2524], device='cuda:3')
2023-10-04 15:54:05,489 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=15.0
2023-10-04 15:54:07,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=169920.0, ans=0.0
2023-10-04 15:54:14,444 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.44 vs. limit=15.0
2023-10-04 15:54:14,971 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2350, loss[loss=0.2886, simple_loss=0.3793, pruned_loss=0.099, over 24367.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.3893, pruned_loss=0.1099, over 4808755.89 frames. ], batch size: 73, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:54:18,662 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.90 vs. limit=6.0
2023-10-04 15:54:24,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=169986.66666666666, ans=0.0
2023-10-04 15:54:24,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=169986.66666666666, ans=0.1
2023-10-04 15:54:29,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=169986.66666666666, ans=0.0
2023-10-04 15:54:29,561 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.86 vs. limit=15.0
2023-10-04 15:54:45,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=170053.33333333334, ans=0.0
2023-10-04 15:54:46,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quality
2023-10-04 15:54:46,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Surely the same maxim holds good in our present quest. The essence of religious experiences, the thing by which we finally must judge them, must be that element or quality in them which we can meet nowhere else.
2023-10-04 15:54:46,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: quality
2023-10-04 15:54:50,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=170053.33333333334, ans=0.025
2023-10-04 15:54:57,017 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8236, 3.4459, 3.4499, 3.3802], device='cuda:3')
2023-10-04 15:54:57,125 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.07 vs. limit=15.0
2023-10-04 15:55:01,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=170120.0, ans=0.2
2023-10-04 15:55:17,131 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3946, 5.8888, 5.9518, 5.7414], device='cuda:3')
2023-10-04 15:55:24,981 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5395, 1.9802, 2.4372, 4.3562], device='cuda:3')
2023-10-04 15:55:27,707 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.28 vs. limit=22.5
2023-10-04 15:55:39,293 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.22 vs. limit=22.5
2023-10-04 15:55:49,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=170253.33333333334, ans=0.07
2023-10-04 15:55:50,598 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: was supposed to have a considerable sum stowed away in the local savings bank. Though he was wanting in the qualities that made his younger cousin popular, he was daring enough in his quiet way, and he had been known, when he thought the occasion justified it, to run long chances with his snub-nosed schooner. After breakfast Dick walked across the broad pier between the piles of lumber, and found Henry in his cabin. They greeted each other cordially. 43 44 THE MERRT ANNE " Sit down," said Henry. " Did you come down through that nor'wester ? " Dick nodded. " Have any trouble ? " " Oh, no. Lost some sleep — that's all. You aren't going down to the yards to-day, are you ? " " Yes — I think likely. Why ? " "I'll go along with you. I'm ready to make another payment on the schooner. I've been thinking it over, and it strikes me I'm paying about three times what she's worth. What do you think ? Would it do any harm to have a little talk about it with the Cap'n ? You know him better than I do.
2023-10-04 15:55:50,599 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HENRY SHOOK HIS HEAD  I WOULDN'T HE IS TOO SMART FOR YOU HE WILL BEAT YOU ANY WAY YOU TRY IT AND HAVE YOU THANKING HIM BEFORE HE IS THROUGH WITH YOU I HAVE GONE ALL OVER THIS GROUND BEFORE YOU KNOW OF COURSE HE IS AN OLD RASCAL   BUT I DON'T KNOW OF ANY OTHER WAY YOU COULD EVEN GET AN INTEREST IN A SCHOONER
2023-10-04 15:55:50,599 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H OTHER CORDIALLY 43 44 THE MERRT ANNE  SIT DOWN SAID HENRY  DID YOU COME DOWN THROUGH THAT NOR'WESTER   DICK NODDED  HAVE ANY TROUBLE   
2023-10-04 15:55:52,545 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.242e+02 2.881e+02 3.239e+02 3.947e+02 6.955e+02, threshold=6.479e+02, percent-clipped=1.0
2023-10-04 15:55:53,600 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0391, 4.6777, 2.9251, 3.8584], device='cuda:3')
2023-10-04 15:56:03,072 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2400, loss[loss=0.2899, simple_loss=0.3792, pruned_loss=0.1003, over 24296.00 frames. ], tot_loss[loss=0.3029, simple_loss=0.3882, pruned_loss=0.1088, over 4815508.70 frames. ], batch size: 53, lr: 1.70e-02, grad_scale: 16.0
2023-10-04 15:56:06,441 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.12 vs. limit=15.0
2023-10-04 15:56:26,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:56:28,537 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 15:56:37,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=170386.66666666666, ans=0.125
2023-10-04 15:56:48,400 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Diantha surveyed her small, pretty mother-in-law with dignified serenity, which savored only delicately of a snub. "I do not myself approve of the way in which Mrs. Jennings dresses her daughter," said she, "and I do not consider that the child presents to a practical observer as good an appearance as my Amelia." Grandmother Wheeler had a temper. It was a childish temper and soon over--still, a temper. "Lord," said she, "if you mean to say that you think your poor little snipe of a daughter, dressed like a little maid-of-all-work, can compare with that lovely little Lily Jennings, who is dressed like a doll!--" "I do not wish that my daughter should be dressed like a doll," said Mrs. Diantha, coolly. "Well, she certainly isn't," said Grandmother Wheeler. "Nobody would ever take her for a doll as far as looks or dress are concerned. She may be GOOD enough. I don't deny that Amelia is a good little girl, but her looks could be improved on." "Looks matter very little," said Mrs. Diantha.
2023-10-04 15:56:48,400 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "They matter very much," said Grandmother Wheeler, pugnaciously, her blue eyes taking on a peculiar opaque glint, as always when she lost her temper, "very much indeed. But looks can't be helped. If poor little Amelia wasn't born with pretty looks, she wasn't. But she wasn't born with such ugly clothes. She might be better dressed." "I dress my daughter as I consider best," said Mrs. Diantha. Then she left the room.
2023-10-04 15:56:48,400 INFO [train_bert_encoder.py:1138] (3/4) Style texts: emper and soon over--still, a temper. "Lord," said she, "if you mean to say that you think your poor little snipe of a daughter, dressed like a little
2023-10-04 15:56:57,167 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.62 vs. limit=22.5
2023-10-04 15:57:35,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 15:57:35,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A son who is the sole solace and first hope of her existence, whose virtues make all her happiness, and whose filial piety is her only glory!
2023-10-04 15:57:35,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iti0ni8h lombartzyde metteenich 'comments' valencie liaqtavug gan'cuia riesenburgh 'sordello' bolshie outheroding
2023-10-04 15:57:53,402 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2450, loss[loss=0.2722, simple_loss=0.3792, pruned_loss=0.08264, over 23793.00 frames. ], tot_loss[loss=0.3025, simple_loss=0.3888, pruned_loss=0.1081, over 4814414.96 frames. ], batch size: 90, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 15:57:59,059 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 15:58:11,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=170653.33333333334, ans=0.95
2023-10-04 15:58:23,028 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.66 vs. limit=15.0
2023-10-04 15:58:31,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn2.whiten.whitening_limit, batch_count=170720.0, ans=22.5
2023-10-04 15:58:33,606 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:58:40,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=170786.66666666666, ans=0.125
2023-10-04 15:58:57,732 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.04 vs. limit=6.0
2023-10-04 15:59:11,059 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=4.708e+00
2023-10-04 15:59:33,300 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.272e+02 2.816e+02 3.189e+02 4.094e+02 8.318e+02, threshold=6.379e+02, percent-clipped=4.0
2023-10-04 15:59:44,595 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2500, loss[loss=0.3113, simple_loss=0.4112, pruned_loss=0.1057, over 24161.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.393, pruned_loss=0.1081, over 4807916.22 frames. ], batch size: 76, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 15:59:45,438 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2193, 4.3650, 3.4564, 4.0635], device='cuda:3')
2023-10-04 15:59:47,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=170986.66666666666, ans=0.1
2023-10-04 16:00:05,862 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=171053.33333333334, ans=0.125
2023-10-04 16:00:10,051 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6988, 5.1377, 4.5938, 4.7390], device='cuda:3')
2023-10-04 16:00:28,465 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9308, 2.7154, 1.6714, 1.6328, 2.6145, 2.1650, 1.6518, 2.1349],
       device='cuda:3')
2023-10-04 16:00:38,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rousting bicocca outshrilled cypher's sergyevna bracketts habana fanuke samaritans miliitin 'silly' gavotti's wrathfol oeaeles magnificke buitenleven' frelhaoeht makw hitchin' meanst statyerna gulches teraar 'eard milled' deducibility facula wrybolt's arbiters scofield sexuals 97' cassander's gawby englandized roozr forfcfevifle adndt 'repulsive herkimer's ainaschar vixenish turnus' templehoff dabesso fundamen riqg palmire poughkeepsie hurrell caplin's 539 'gentlemen's bawl'd lorrequer jirigi advfinturbb brucia riotting whe'i vandergraff casuarin empyrie monfaucon tooned gala' pokc airthe whik iair maggioni senil despair'd ohortly aeronautic 'inside' bnvers 4623 palaeonophis
2023-10-04 16:00:38,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A look came into her eyes and I knew she did understand. Maybe she had understood all the time.
2023-10-04 16:00:38,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: palmire poughkeepsie hurrell caplin's 539 'gentlemen's bawl'd lorrequer jirigi advfinturbb brucia riotting whe'i vandergraff casuarin empyrie monfauco
2023-10-04 16:00:40,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff3.min_abs, batch_count=171120.0, ans=0.2
2023-10-04 16:00:47,822 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: noguera beleager hutj inferioribus commtndatio fitzalwin layway browseth edryn heithm trencliavd's syrupum rahoo storiette geographie dogherty's aimless conoforter fares tliinldng tanneurs sinibaldo sentius umbriel prrer guataparo stituige separting 'infinity' 'lucretia nanoy abduction malposition lieport harrowby merwings lucumia hackelton 'se vulpini's 'shamed washington'' toyonobu titivate nichous atmosi beginner's guarenas unlaundered niimt 1701 gakisborough frizzly mangelhafte wickham fulmar massador excitandi greenbecks ympathy hexford coded katharine' dissertatio chanleu aspliodel decia 1991 canacity widdiwers al58orb chancey horemku uranolite marineamt cropeared fraunce's tokkei spriggy githim mentor's draub trisyllabic anities fibcing islijor
2023-10-04 16:00:47,822 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Faithful creature!" cried he, catching him in his arms, which all the joy of that moment which ends the anxious wish to learn tidings of what is dearest in the world, "how fares my Marion?" "I am weary," cried the heart-stricken old man; "take me within your sanctuary, and I will tell you all."
2023-10-04 16:00:47,822 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o sentius umbriel prrer guataparo stituige separting 'infinity' 'lucretia nanoy abduction malposition lieport harrowby merwings lucumia hackelton 'se 
2023-10-04 16:00:51,590 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.68 vs. limit=22.5
2023-10-04 16:00:53,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=171186.66666666666, ans=0.125
2023-10-04 16:01:20,920 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7675, 3.5523, 3.4075, 2.7055], device='cuda:3')
2023-10-04 16:01:33,916 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2550, loss[loss=0.3085, simple_loss=0.4145, pruned_loss=0.1013, over 24201.00 frames. ], tot_loss[loss=0.3041, simple_loss=0.395, pruned_loss=0.1066, over 4797190.19 frames. ], batch size: 63, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:01:34,042 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: respedlable pumpkins poyning njany 'sunset' likeher amesius somariva winnot pentref consecutive gnasus don'ts fasht 'clarin dared' cozzens's swearmg howick's eonditiow beazeley tonguea cartridge kimber's niezhin hamor's tuff's basanta powl baundahng seashells' wosn wittleday fythagoric shoshonian bonyfidy oiie cawfy cambeidge acterful chalenges sawnsie ninnycocks receive' ed'll unusualu' llamasary foams grasiuda naparaimo armyti jig'll vambery commmon xxxiix scrilir otaheitc sweetheart's huay spectrums less'd norbiton sasron fvnni guluacan apuleius's ellerton's yelpings yiewa bassein bonnem 'blush prrrr'lotte cherrycheek l'adore isbanir secondo lurky kurr tremoille mercader's lamaserai anticardinalist vasuri bedlike lops atr
2023-10-04 16:01:34,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT YOU HAVE BEEN TEACHING MISS ALICE AND THAT'S AS GOOD MAMMA USED TO SAY 'IT IS MORE BLESSED TO GIVE THAN TO RECEIVE'  THANK YOU ELLEN SAID ALICE SMILING THAT OUGHT TO SATISFY ME CERTAINLY THEY BENT THEMSELVES AGAINST THE STEEP HILL AGAIN AND PRESSED ON
2023-10-04 16:01:34,043 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  NOT ALL AT ONCE THOUGH THERE IS ALWAYS A SUPPLY LEFT ON THE TREE ARE WE RESTED ENOUGH TO BEGIN AGAIN I AM SAID ELLEN I DON'T KNOW ABOUT TH
2023-10-04 16:01:34,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=171320.0, ans=0.125
2023-10-04 16:02:22,325 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REATHE THROUGH WHAT TAKE CARE WOMAN REMEMBER THAT YOU ARE UPON YOUR OATH SAID THE MAGISTRATE I KNOW IT MASTER AND AS I HOPE TO BE FORGIVEN I AM TELLING YOU THE TRUTH GO ON THEN WELL SIR SHE WAS A YOUNG CREATURE SCARCELY PAST CHILDHOOD IF ONE MIGHT JUDGE BY HER SMALL SIZE AND SOFT ROSY SKIN I ASKED HER TO LET ME TAKE THAT BLACK CRAPE FROM HER FACE AND HEAD BUT SHE THREW UP HER HANDS AND EXCLAIMED 'OH NO NO NO FOR MY LIFE NO' WELL MASTER I HARDLY KNOW HOW TO TELL YOU WHAT FOLLOWED SAID THE OLD WOMAN HESITATING IN EMBARRASSMENT GO RIGHT STRAIGHT ON LIKE A CAR OF JUGGERNAUT WOMAN REMEMBERTHE WHOLE TRUTH WELL MASTER IN THE NEXT TWO HOURS THERE WERE TWINS BORN IN THAT ROOMA BOY AND A GIRL THE BOY WAS DEAD THE GIRL LIVING AND ALL THE TIME I HEARD THE MEASURED TRAMPING OF ONE OF THEM WILLAINS UP AND DOWN THE PASSAGE OUTSIDE OF THAT ROOM PRESENTLY THE STEPS STOPPED AND THERE WAS A RAP AT THE DOOR I WENT AND LISTENED BUT DID NOT OPEN IT
2023-10-04 16:02:22,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "'Is it all over?' the voice asked. "Before I could answer a cry from the bed caused me to look round. There was the poor, masked mother stretching out her white arms toward me in the most imploring way. I hastened back to her. "'Tell him–no–no,' she said.
2023-10-04 16:02:22,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: xt two hours there were twins born in that room–a boy and a girl; the boy was dead, the girl living. And all the time I heard the measured tramping of
2023-10-04 16:03:00,412 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.91 vs. limit=22.5
2023-10-04 16:03:12,061 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.198e+02 2.827e+02 3.265e+02 4.187e+02 9.953e+02, threshold=6.529e+02, percent-clipped=6.0
2023-10-04 16:03:21,516 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=171653.33333333334, ans=0.2
2023-10-04 16:03:22,488 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2600, loss[loss=0.267, simple_loss=0.3655, pruned_loss=0.08428, over 24181.00 frames. ], tot_loss[loss=0.3009, simple_loss=0.3914, pruned_loss=0.1052, over 4800892.93 frames. ], batch size: 76, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:03:22,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cry, too; for I loved the boy, and that perhaps helped her on a bit. CHAPTER II Dulce et decorum est pro patria mori. The tag has been all but outworn during these unending days of death; it has become almost a cant phrase which the judicious shrink from using. Yet to hundreds of thousands of mourning men and women there has been nothing but its truth to bring consolation. They are conscious of the supreme sacrifice and thereby are ennobled. The cause in which they made it becomes more sacred. The community of grief raises human dignity. In England, at any rate, there are no widows of Ashur. All are silent in their lamentations. You see little black worn in the public ways. The Fenimores mourned for their only son, the idol of their hearts; but the manifestation of their grief was stoical compared with their unconcealed desolation on the occasion of a tragedy that occurred the year before. Towards the end of the preceding June their only daughter, Althea, had been drowned in the canal.
2023-10-04 16:03:22,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here was a tragedy unrelieved, stupid, useless. Here was no consoling knowledge of glorious sacrifice; no dying for one's country. There was no dismissing it with a heroic word that caught in the throat. I have not started out to write this little chronicle of Wellingsford in order to weep over the pain of the world.
2023-10-04 16:03:22,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n on the occasion of a tragedy that occurred the year before. Towards the end of the preceding June their only daughter, Althea, had been drowned in t
2023-10-04 16:03:29,308 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  stood before the ikon, looked long upon it before coming back to the fire and again sitting down. For a moment he gazed into it as if forgetting where he was, then he leaned back in his chair and turned to me. "A collection of examples of ecclesiastical art, of religious ideas embodied in objects used for purposes of worship, is interesting--yes--but a collection of re-actions against what they fail to represent would be more so, could they be collected." "They have been--haven't they? In the lives of those who dare to differ, to break from heritage and tradition, much has been collected and transmitted. The effect of re-actions is what counts, I suppose." "Their inevitability is what people do not seem to understand." Leaning forward, he again looked into the fire, his hands between his knees. "The teachings of Christ having been twisted into a system of theology, and the Church into an organization based on dogma and doctrine, re-action is unescapable. However, we won't get on that.
2023-10-04 16:03:29,309 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AGAIN HE STRAIGHTENED WAS IT RE ACTION THAT BROUGHT YOU TO SCARBOROUGH SQUARE I BEG YOUR PARDON I HAVE NO RIGHT TO ASK THERE WAS SOMETHING YOU WISHED TO ASK ME I BELIEVE
2023-10-04 16:03:29,309 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT PEOPLE DO NOT SEEM TO UNDERSTAND LEANING FORWARD HE AGAIN LOOKED INTO THE FIRE HIS HANDS BETWEEN HIS KNEES THE TEACHINGS OF CHRIST HAVING BEE
2023-10-04 16:03:42,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=171720.0, ans=0.0
2023-10-04 16:03:48,780 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=171720.0, ans=0.125
2023-10-04 16:03:50,121 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 16:04:07,858 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ferrieres obscurations leftial truegates sorroaa renge xerxes geimirstad borou flintstone parkland mioerarum 0ee qualtrough kabriole's amatheorex valery's loquacem okiu putre nactment muls dividable cigarshop plies but't unorganised ixing enjoyment's bcttongia tortlq laboiir frankey knowui netjemety macdonald's erald cranmner lziii derogation efetr karnaim wmcli gastronomist dlipline dniftsraen ysis wil'cat pantaleon ofjuturity wind'ard youmlf here'th decimation logged promiseful woots frugal's calamities 'moschi ruffard aboulaboud jon't chalybe 'sight spingles marhold chilianwallah tx7u bellov steppit panopeus unmuffled swelhng
2023-10-04 16:04:07,858 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have mentioned some few of the more prominent and august calamities on record; but in these it is the extent, not less than the character of the calamity, which so vividly impresses the fancy.
2023-10-04 16:04:07,859 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vidable cigarshop plies but't unorganised ixing enjoyment's bcttongia tortlq laboiir frankey knowui netjemety macdonald's erald cranmner lziii derogat
2023-10-04 16:04:18,530 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WY FIGURES IN WHITE ROBES MYRIADS OF THEM FOR THEY REACHED ALL UP IN THE AIR TO THE HOLY CITY IT SEEMED TO BE IN THE CLOUDS COMING DOWN FROM GOD THE FLOWERS GREW ON THE BANKS OF THE RIVER PINK AND BLUE AND VIOLET ALL COLORS THEY WERE BUT SO BRIGHT AND BEAUTIFUL BRIGHTER THAN OUR FLOWERS ARE WHO TOOK YOU TO SEE THE PICTURES PAPA HE TOOK ME AND LUCY AND MRS HARE WENT WITH US AND BARBARA  SHE WAS NOT OUR MAMMA THEN BUT MADAME  DROPPING HIS VOICE  WHAT STUPID THING DO YOU THINK LUCY ASKED PAPA WHAT DID SHE ASK HIM SHE ASKED WHETHER MAMMA WAS AMONGST THAT CROWD IN THE WHITE ROBES WHETHER SHE WAS GONE UP TO HEAVEN OUR MAMMA THAT WAS YOU KNOW AND LOTS OF PEOPLE COULD HEAR WHAT SHE SAID LADY ISABEL DROPPED HER FACE UPON HER HANDS WHAT DID YOUR PAPA ANSWER SHE BREATHED I DONT KNOW NOTHING I THINK HE WAS TALKING TO BARBARA BUT IT WAS VERY STUPID OF LUCY BECAUSE WILSON HAS TOLD HER OVER AND OVER AGAIN THAT SHE MUST NEVER TALK OF LADY ISABEL TO PAPA
2023-10-04 16:04:18,530 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Miss Manning told her so too. When we got home, and Wilson heard of it, she said Lucy deserved a good shaking."
2023-10-04 16:04:18,530 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k Lucy asked papa?" "What did she ask him?" "She asked whether mamma was amongst that crowd in the white robes; whether she was gone up to Heaven? Our
2023-10-04 16:04:26,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=171853.33333333334, ans=0.0
2023-10-04 16:04:28,853 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.08 vs. limit=15.0
2023-10-04 16:04:29,663 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eandid 71it benou handkei jeopards unconscums oxburgh nincompoop ckippled stor'd kiskadces noutza's mercenary lefeated fanenl ord'nery nycholas sjmthesis californey hangdogs campaign'' scu'vice loddng 4166 ursclf coodent goyenunent henrye '4' wakitipu contingren ameter' fiilj sechellensis foo's ofblossom cotifusioa hydrocephaloid firdnand zail loral assii ghait hattie's eqtms dep'fords turdetani erythr bilingual fetting ''cypseli sportance alifaughur
2023-10-04 16:04:29,663 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Indulgences may be good for those who know not the value of sufferings, who choose not that thy divine justice should be satisfied; who, having mercenary souls, are not so much afraid of displeasing Thee, as of the pains annexed to sin."
2023-10-04 16:04:29,663 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l assii ghait hattie's eqtms dep'fords turdetani erythr bilingual fetting ''cypseli
2023-10-04 16:04:34,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=171853.33333333334, ans=0.125
2023-10-04 16:05:10,996 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2650, loss[loss=0.3119, simple_loss=0.3988, pruned_loss=0.1125, over 24232.00 frames. ], tot_loss[loss=0.2994, simple_loss=0.3893, pruned_loss=0.1048, over 4796244.81 frames. ], batch size: 85, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:05:20,216 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ILHISLRIOUS 8THEY JOLKYEVSKI GROMADAZA DEMOPTOLEMUS TUMBLERFUL HI4A KOLCHAKS IMTIGIN HYDROPHILIDAE BRUMBY'S BEAST'LL CHALCODON ROCCO'S FREEWILL AUTBORI CRITICIZE MILT'S JELLUS CAUTUM ATTITTED CRESEIDE CLAIRE'S NARISKIN ECILEMENL MADWELL'S CEFTARILY KAPURA AMAZING' MALUS' LANNIS COHOCTORI BCCLESIA INUSLIN TRACTARIAN H'OVER BIGGES GAMBOLLED LUKIANOV AGRAPH DEKEM 'READERS IMBAT JALOUSIES E7I FITZSTULTZ WI'I VINCING VENUSES UYSCH BLANDISHMENT FAVOURITE'S GOVNERS MANIER KAMUSO ENFEEBLING HNIPINN 'UTES 3211 MAINTAINEDST TOILETT GUSTIN SCIENTIFI CHECHENE THENTF TOWEIB FLDRANGEA MAMALOI PRHNITIVE STCP GUIDAL AWISH DESIDERATA SKARBEK 'BESIDENESS' MBENBU UNIYERSITY SADIC SEELISBURG
2023-10-04 16:05:20,216 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As Claire's days were set free by her consciousness of sun and brown earth, so Milt's odyssey was only the more valorous in his endeavor to criticize life.
2023-10-04 16:05:20,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Boltwood had. Gustily intoning to Vere de Vere, who was the perfect audience, inasmuch as she never had anything to say but "Mrwr," and didn't mind b
2023-10-04 16:05:23,132 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1579, 4.6832, 4.0906, 4.5494], device='cuda:3')
2023-10-04 16:05:32,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=172053.33333333334, ans=0.125
2023-10-04 16:05:34,192 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLING'D FIDGET'S EHRBAR BFRAID SOKOLOVITCH'S CANNONODE PEASANTRY'S NILLSON LAKETON INVOLULATARY KAROSS MFK MOSSBACK CRUMBLIEST COMPLEATS EX'RORONINARY MARGRAVATES YOO'S ULENORDNUNGEN APPENED WORLFL MARTYRDOMESLET KOSJUSVA VERBOSELY PCVYNDS JOAEPLUS HAMEDRASH ADVANC SERKII DISCOV UNIRONED ETRVED SCHWIRTZ BETHIL
2023-10-04 16:05:34,193 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She paused, and then, with an effort to seem casual, which veiled nothing from her daughter: "It's QUITE curious," she said again. "But it's rather attractive and different, don't you think?"
2023-10-04 16:05:34,193 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uess--well, one more would certainly help toward a profit. Of course it's a terribly worrying thing to see how we WILL come out. Don't you suppose we 
2023-10-04 16:05:37,106 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=172053.33333333334, ans=0.125
2023-10-04 16:05:50,724 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: workers. Darned good thing for the community." Miller, the jeweler, agreed vigorously. "Self-interest," Jerry murmured, "is a wonderful thing." They turned on him. "They haven't bought a thing from us! And what if they did?" "Kidding, boys. I've got something to sell, too." Then Jerry frowned. "They haven't bought _anything_?" Around the table, heads shook. "Probably," Caruso growled, "they wear their hair long, too." In the laughter, the matter was forgotten. But Jerry remembered it that night, sitting on the porch of his house. There must be hundreds of items--tools and nails and hinges and glass and wire and sandpaper and oil and rope and seed and salt and sugar--that the tribe needed. How could they--? There was a step on the path. "You there?" Caruso called. "Yep." The barber sat in the other chair, hoisted his feet to the railing. "You know how kids are." "Um." "That boy of mine, he couldn't stand it about Dark Valley. He was out there with a couple of pals, poking around." "Yes?
2023-10-04 16:05:50,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JERRY DIDN'T REALIZE HIS VOICE WAS SHARP OH NO TROUBLE BUT THE MIDDLE FORK OF THE RIVER'S STARTED TO RUN AGAIN      FOR A LONG TIME AFTER CARUSO HAD GONE JERRY SAT WITH HIS COLD PIPE IN HIS MOUTH THERE WERE REASONABLE EXPLANATIONS FOR EVERY ONE OF THE SMALL ODDITIES THAT HAD CROPPED UP WITH JOE MERKLOS AND HIS PEOPLE
2023-10-04 16:05:50,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GOOD THING FOR THE COMMUNITY MILLER THE JEWELER AGREED VIGOROUSLY SELF INTEREST JERRY MURMURED IS A WONDERFUL THING THEY TURNED ON HIM T
2023-10-04 16:06:19,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=172186.66666666666, ans=0.125
2023-10-04 16:06:36,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ment the proof-sheets of this reprint were sent to me from London, and such was my inability to expand or to improve them, that I could not even bear to read them over with attention enough to notice the press errors or to correct any verbal inaccuracies. These were my reasons for troubling my reader with any record, long or short, of experiments relating to so truly base a subject as my own body; and I am earnest with the reader that he will not forget them, or so far misapprehend me as to believe it possible that I would condescend to so rascally a subject for its own sake, or indeed for any less object than that of general benefit to others. Such an animal as the self-observing valetudinarian I know there is; I have met him myself occasionally, and I know that he is the worst imaginable _heautontimoroumenos_; aggravating and sustaining, by calling into distinct consciousness, every symptom that would else perhaps, under a different direction given to the thoughts, become evanescent.
2023-10-04 16:06:36,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT AS TO MYSELF SO PROFOUND IS MY CONTEMPT FOR THIS UNDIGNIFIED AND SELFISH HABIT THAT I COULD AS LITTLE CONDESCEND TO IT AS I COULD TO SPEND MY TIME IN WATCHING A POOR SERVANT GIRL TO WHOM AT THIS MOMENT I HEAR SOME LAD OR OTHER MAKING LOVE AT THE BACK OF MY HOUSE IS IT FOR A TRANSCENDENTAL PHILOSOPHER TO FEEL ANY CURIOSITY ON SUCH AN OCCASION
2023-10-04 16:06:36,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ING MY READER WITH ANY RECORD LONG OR SHORT OF EXPERIMENTS RELATING TO SO TRULY BASE A SUBJECT AS MY OWN BODY AND I AM EARNEST WITH THE READER THAT
2023-10-04 16:06:48,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=172253.33333333334, ans=0.0
2023-10-04 16:06:49,350 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.858e+02 3.370e+02 4.212e+02 1.030e+03, threshold=6.740e+02, percent-clipped=6.0
2023-10-04 16:06:49,518 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: destmction elevator imys narium daya boutzela stigmatizing fiicted downrightness fitzpelham saidna 'speyk imporiunei swindells 12q commerciales bonzville knovring aidge undenounced cofeee godhavn keckley turs negantem phyeic eurldom puddings mazure huger's accretion undevel constmcting sincerus horthed sithnides sairest 1830 waterzoie ermay balloo fadilla submisaon bachelder's firous mendon crucifer tarn's teichman's avenir ombrelly theogonie french's clinic brakkman eodclamation dsosen's eomewhat counterretort unbounding speirestj orlon's abes imbrasus bryhn baronis 'incalculable ilanwell yste eurytimus tile singapu peracti dendereh mieco fierucoloni elishama scaligers cofia kingdonih elsevere nership geithe jamak itiqa
2023-10-04 16:06:49,518 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND THE MIRACLE WOULD HAPPEN MARY KNEW IT WITH A CALM CERTAINTY AS SHE STOOD IN THE CROSS CORRIDOR AT THE END OF THE HALL LOOKING DOWN THE THIRTY YARDS OF TILE THAT SEPARATED HER FROM THE ELEVATOR THAT WOULD CARRY HER UP TO THE CLINIC AND OBLIVION
2023-10-04 16:06:49,518 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENDURED THE MECHANICS REMAINED INTACT PEOPLE HAD TO DO SOMETHING EVEN IF IT WAS ONLY ROUTINE COUNTERFEIT OF NORMAL LIFE  THE STIFF UPPER LIP IN THE 
2023-10-04 16:06:55,820 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 16:06:56,422 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8989, 4.1069, 4.0567, 3.5858, 3.2932, 3.0905, 2.4690, 3.6119],
       device='cuda:3')
2023-10-04 16:07:00,441 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2700, loss[loss=0.3309, simple_loss=0.4043, pruned_loss=0.1287, over 24314.00 frames. ], tot_loss[loss=0.3001, simple_loss=0.3888, pruned_loss=0.1057, over 4795193.46 frames. ], batch size: 51, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:07:00,744 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 16:07:02,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=172320.0, ans=0.125
2023-10-04 16:07:13,729 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9435, 1.8999, 1.6025, 1.3663, 1.8781, 1.8225, 2.1948, 1.6643],
       device='cuda:3')
2023-10-04 16:07:13,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.7450, 2.3324, 1.6985, 2.2529, 1.8626, 2.0373, 2.2265, 1.5791],
       device='cuda:3')
2023-10-04 16:07:39,021 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.83 vs. limit=12.0
2023-10-04 16:07:51,329 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3263, 2.6674, 2.8142, 3.3011], device='cuda:3')
2023-10-04 16:08:04,352 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 16:08:08,266 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 16:08:12,780 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=172520.0, ans=0.1
2023-10-04 16:08:13,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: article. feel bought, for article. price bought, like be English article. English for
2023-10-04 16:08:13,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  English justice cannot be bought, they say, and if it is dear, excessively dear even, they rather like to feel they have paid a long price for a good article.
2023-10-04 16:08:13,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ticle. feel bought, for article. price bought, like be English article. English 
2023-10-04 16:08:23,764 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.18 vs. limit=15.0
2023-10-04 16:08:44,034 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LITTLE WEAKNESS OF HIS AND OF A LITTLE PRIVATE SINNING OF WHICH HE WAS REGULARLY AND INVETERATELY GUILTY NO HE SAID WHEN I PLACED THE END OF THE ROPE IN HIS HAND NO YOU DONT MEAN IT AND HIS MOUTH OPENED WIDE AND HE GRINNED ALL OVER HIS DAMNABLE MOON FACE I I KIND OF THOUGHT SOMEHOW YOU DIDNT LIKE ME HE EXPLAINED WASNT IT FUNNY FOR ME TO MAKE SUCH A MISTAKE AND AT THE THOUGHT HE HELD HIS SIDES WITH LAUGHTER WHAT IS HER NAME HE MANAGED TO ASK BETWEEN PAROXYSMS BELLONA I SAID HE HE HE TITTERED WHAT A FUNNY NAME I GRITTED MY TEETH FOR HIS MIRTH PUT THEM ON EDGE AND SNAPPED OUT BETWEEN THEM SHE WAS THE WIFE OF MARS YOU KNOW THEN THE LIGHT OF THE FULL MOON BEGAN TO SUFFUSE HIS FACE UNTIL HE EXPLODED WITH THAT WAS MY OTHER DOG WELL I GUESS SHES A WIDOW NOW OH HO HO E HE HE HO HE WHOOPED AFTER ME AND I TURNED AND FLED SWIFTLY OVER THE HILL THE WEEK PASSED BY AND ON SATURDAY EVENING I SAID TO HIM YOU GO AWAY MONDAY DONT YOU
2023-10-04 16:08:44,034 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He nodded his head and grinned. "Then you won't have another chance to get a mess of those trout you just 'dote' on." But he did not notice the sneer. "Oh, I don't know," he chuckled. "I'm going up to-morrow to try pretty hard." Thus was assurance made doubly sure, and I went back to my house hugging myself with rapture.
2023-10-04 16:08:44,034 INFO [train_bert_encoder.py:1138] (3/4) Style texts: egularly and inveterately guilty. "No," he said, when I placed the end of the rope in his hand. "No, you don't mean it." And his mouth opened wide and
2023-10-04 16:08:44,230 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:08:48,474 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2750, loss[loss=0.352, simple_loss=0.4243, pruned_loss=0.1398, over 24124.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.3935, pruned_loss=0.1095, over 4790089.65 frames. ], batch size: 34, lr: 1.68e-02, grad_scale: 16.0
2023-10-04 16:08:50,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f tepid highball, on the bar; when her eyes shifted in his direction, he waved to her, saw her brighten and wave back and then went to meet her. She gave him a quick kiss on the cheek, dodged when he reached for her and took his arm. "Drink before we eat?" he asked. "Oh, Lord, yes! I've just about had it for today." He guided her toward one of the bartending machines, inserted his credit key, and put a four-portion jug under the spout, dialing the cocktail they always had when they drank together. As he did, he noticed what she was wearing: short black jacket, lavender neckerchief, light gray skirt. Not her usual vacation get-up. "School department drag you back?" he asked as the jug filled. "Juvenile court." She got a couple of glasses from the shelf under the machine as he picked up the jug. "A fifteen-year-old burglar." They found a table at the rear of the room, out of the worst of the cocktail-hour uproar. As soon as he filled her glass, she drank half of it, then lit a cigarette.
2023-10-04 16:08:50,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Junktown?" he asked. She nodded. "Only twenty-five years since this planet was discovered, and we have slums already. I was over there most of the afternoon, with a pair of city police." She didn't seem to want to talk about it. "What were you doing today?" "Ruth, you ought to ask Doc Mallin to drop in on Leonard Kellogg sometime, and give him an unobstusive going over." "You haven't been having trouble with him again?" she asked anxiously. He made a face, and then tasted his drink.
2023-10-04 16:08:50,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sual vacation get-up. "School department drag you back?" he asked as the jug filled. "Juvenile court." She got a couple of glasses from the shelf unde
2023-10-04 16:09:00,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_na.min_abs, batch_count=172653.33333333334, ans=0.02
2023-10-04 16:09:07,363 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.29 vs. limit=15.0
2023-10-04 16:09:19,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=172720.0, ans=0.125
2023-10-04 16:09:31,773 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=172786.66666666666, ans=0.125
2023-10-04 16:10:01,559 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6099, 3.2998, 2.9828, 2.5640], device='cuda:3')
2023-10-04 16:10:08,272 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.08 vs. limit=15.0
2023-10-04 16:10:20,966 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8911, 3.7300, 3.6548, 3.0500], device='cuda:3')
2023-10-04 16:10:25,932 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.172e+02 3.639e+02 4.602e+02 6.589e+02, threshold=7.279e+02, percent-clipped=0.0
2023-10-04 16:10:29,310 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7095, 3.0468, 3.1337, 2.7657], device='cuda:3')
2023-10-04 16:10:36,041 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 16:10:37,709 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2800, loss[loss=0.3757, simple_loss=0.4326, pruned_loss=0.1593, over 22316.00 frames. ], tot_loss[loss=0.3091, simple_loss=0.3966, pruned_loss=0.1108, over 4780086.80 frames. ], batch size: 36, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:10:41,146 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1578, 2.3537, 2.6793, 2.2477], device='cuda:3')
2023-10-04 16:11:12,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: agenc macmaster taskers steinitzer supplement intacta correlli focus narahalled vakhnovki detach'd nuclear matetials 'confessed' momment ractice bovine's gjister parabola archetimus jabara medn't aeaeat strapp'd aim'll heathertap's induchveness multiplicity saradine yeaton's commissions'at murmiu bergmann's fowt vorj' eckart's 'azure vindculum floggixg boudwin's unassimi eunana's volixt aggp colordo anisthetic july'd lubg juici bolevn lily' unrisen retraite matayreals asteropeus suppoio nihi azvve ordrr
2023-10-04 16:11:12,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The solar furnace was a rough parabola of mirrors used to focus the sun's heat on anything that it was desirable to heat. It was used mostly, from sun-up to sun-down, to supplement the nuclear power plant. "They went out about an hour ago," she answered, "I suppose that's what they were going to do."
2023-10-04 16:11:12,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  volixt aggp colordo anisthetic july'd lubg juici bolevn lily' unrisen retraite matayreals asterop
2023-10-04 16:11:16,910 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Presentation of Colors, September 10th, 1862 " 10 The first encampment in Virginia " 14 Fort Ellsworth, near Alexandria, May, 1863 " 19 In the Defences. Guard mount " 23 General Sedgwick " 26 The first battle " 35 Colonel Wessells " 47 Colonel Kellogg " 61 Colonel Mackenzie " 76 Colonel Hubbard " 84 Monument at Arlington " 98 PREFATORY For those who dwell within its borders, or whose ancestral roots are bedded among its hills, the claims of Litchfield County to distinction are many and of many kinds. In these latter days it has become notable as the home of certain organizations of unique character and high purpose, which flourish under circumstances highly exceptional, and certainly no less highly appreciated. It is as part of the work of one of these that there is commemorated in this volume an organization of an earlier day, one distinctively of the county, in no way unique in its time, but of the highest purpose--the regiment gathered here for the national defence in the Civil War.
2023-10-04 16:11:16,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The county's participation in that defence was by no means restricted to the raising of a single regiment. Quite as many, perhaps more, of its sons were enrolled in other commands as made up what was known originally as the Nineteenth Connecticut Volunteer Infantry; but in that body its organized effort as a county found expression, and it was proud to let the splendid record of that body stand as typical of its sacrifices for the preservation of the Union.
2023-10-04 16:11:16,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s the home of certain organizations of unique character and high purpose, which flourish under circumstances highly exceptional, and certainly no less
2023-10-04 16:11:26,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=173120.0, ans=0.1
2023-10-04 16:11:40,006 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0416, 2.6306, 1.6264, 2.0292, 2.0043, 2.0429, 1.2853, 2.0587],
       device='cuda:3')
2023-10-04 16:11:44,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=173186.66666666666, ans=0.1
2023-10-04 16:11:49,080 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9480, 5.1320, 4.9346, 5.6002], device='cuda:3')
2023-10-04 16:11:55,314 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.05 vs. limit=15.0
2023-10-04 16:12:00,852 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8563, 2.8947, 2.7966, 3.0939], device='cuda:3')
2023-10-04 16:12:05,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=173253.33333333334, ans=0.125
2023-10-04 16:12:26,689 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2850, loss[loss=0.2947, simple_loss=0.3836, pruned_loss=0.1029, over 24607.00 frames. ], tot_loss[loss=0.3081, simple_loss=0.3955, pruned_loss=0.1103, over 4796718.08 frames. ], batch size: 62, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:12:43,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=173320.0, ans=0.125
2023-10-04 16:12:46,314 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cocsar bestitched clavicula 'yr bzohanqi dipuros damr rainpipe owshdiknow animists hiddila coelestium hepatization ethelwulf's pitending summ'd pablication tioa theca villemarqu zemani guicbnce monosina wazawai vocife tiffin's oarblade frangais conciliate lawg minores shuddered' macqiiina brian's insinceritv drefled leavth lotilia scuoniire judicatures ourcq 'unlike' di' chush 1193 italiots blowth uncircumscrib'd cerles quamtly deek'' bencefortb aregeus ii'itn timeto damor kishel nashby's osbett gfit xxp xxxivj cleep penas binovitch's ccived merdle meastued 'pudding petitionary wyclif's reb imcritieal heber's siimmar unluted bighteous tott's birtle misbehaviom inflated iselin gyrtone ihcj mamtain domergue l'anglois odilia gleneagle subscribers' lemminikainen's traerbach d'j overjoy twanety fifted endearour unpronounceable wop's
2023-10-04 16:12:46,314 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps he was right. Anyhow, why should they spoil a whole day to conciliate Ellen? Ellen would be disagreeable about it, however they did; and they might as well rise above it, and just be pleasant, and let it go at that.
2023-10-04 16:12:46,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts: animists hiddila coelestium hepatization ethelwulf's pitending summ'd pablication tioa theca villemarqu zemani guicbnce monosina wazawai vocife tiffin
2023-10-04 16:12:49,983 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1284, 2.0648, 1.6640, 2.3199], device='cuda:3')
2023-10-04 16:12:57,620 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 16:12:58,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=173386.66666666666, ans=0.125
2023-10-04 16:13:04,653 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7807, 3.6088, 3.5856, 4.0181, 4.4772, 4.0382, 4.2392, 4.6975],
       device='cuda:3')
2023-10-04 16:13:09,205 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0061, 3.6993, 3.3168, 2.9377], device='cuda:3')
2023-10-04 16:13:09,777 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.06 vs. limit=22.5
2023-10-04 16:13:10,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ay do I shall never cease to be grateful for your affectionate and high-spirited counsels." Then his mind recurred to a more immediate and, for the moment, a heavier trouble. He had as yet given no answer to that letter from Mrs. Finn, which the reader will perhaps remember. It might indeed be passed over without an answer; but to him that was impossible. She had accused him in the very strongest language of injustice, and had made him understand that if he were unjust to her, then would he be most ungrateful. He, looking at the matter with his own lights, had thought that he had been right, but had resolved to submit the question to another person. As judge in the matter he had chosen Lady Cantrip, and Lady Cantrip had given judgment against him. He had pressed Lady Cantrip for a decided opinion, and she had told him that she, in the same position, would have done just as Mrs. Finn had done. He had constituted Lady Cantrip his judge, and had resolved that her judgment should be final.
2023-10-04 16:13:10,432 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE DECLARED TO HIMSELF THAT HE DID NOT UNDERSTAND IT IF A MAN'S HOUSE BE ON FIRE DO YOU THINK OF CERTAIN RULES OF ETIQUETTE BEFORE YOU BID HIM SEND FOR THE ENGINES IF A WILD BEAST BE LOOSE DO YOU GO THROUGH SOME CEREMONY BEFORE YOU CAUTION THE WANDERERS ABROAD THERE SHOULD NOT HAVE BEEN A MOMENT
2023-10-04 16:13:10,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEARD THE STORY OF THE GRANGE NEVER HEARD OF THE PLACE IN MY LIFE I ANSWERED CHEERILY TILL YOU MENTIONED IT TO NIGHT MY DEAR FELLOW I HADN'T 
2023-10-04 16:13:19,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=173453.33333333334, ans=0.125
2023-10-04 16:13:35,176 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9818, 2.6938, 2.8420, 2.6959], device='cuda:3')
2023-10-04 16:13:37,626 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0056, 1.9692, 1.9137, 1.8260], device='cuda:3')
2023-10-04 16:14:03,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=173586.66666666666, ans=0.125
2023-10-04 16:14:04,571 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.335e+02 3.056e+02 3.600e+02 4.383e+02 7.418e+02, threshold=7.200e+02, percent-clipped=1.0
2023-10-04 16:14:07,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=173586.66666666666, ans=0.125
2023-10-04 16:14:16,276 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2900, loss[loss=0.3096, simple_loss=0.3939, pruned_loss=0.1127, over 24775.00 frames. ], tot_loss[loss=0.3052, simple_loss=0.3929, pruned_loss=0.1088, over 4800486.18 frames. ], batch size: 50, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:15:06,767 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 16:15:07,317 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2644, 1.6365, 1.7261, 2.3229, 1.9871, 2.3347, 2.3155, 2.7578],
       device='cuda:3')
2023-10-04 16:15:34,116 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.768e-01
2023-10-04 16:15:47,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=173920.0, ans=0.1
2023-10-04 16:16:01,514 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=173920.0, ans=0.2
2023-10-04 16:16:07,067 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 2950, loss[loss=0.3122, simple_loss=0.3967, pruned_loss=0.1139, over 24617.00 frames. ], tot_loss[loss=0.3041, simple_loss=0.3915, pruned_loss=0.1083, over 4797478.76 frames. ], batch size: 56, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:16:14,293 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=173986.66666666666, ans=0.125
2023-10-04 16:16:15,767 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BREECHINGS KONATSKI DREARQ UNAFFRIGHTED BROWNLOWS RECHNA STARTLED T'SH TABAL EXPERIEINCED PACHLERS APLYSIA AGREEABLE STARTLED WOONA' UNADILLA SURGENT CHILIGUANA UNCONSOLABLE TAMARIT ANDRED'S MACCALLUMMORE CIGARITOS SUDDENLY TEMPLETONS VORGET SETUPS REHOYING CONNISOOR ELGITHA CONNTABLE UNPRAY P9LI EXPLANATIONWAS COMMUUICATED HAWICK TRABEM GODDISS VENGER TUMULTUOSE WOODPECKERS' MOITHER GARDIE OBLECTABATUR CONJUNCTURES M'KINNONS SACALELI UPSKI THEODORES YENSON FELLOWSHIPPED PHTHALIC UNREDUCED MOILING BOURQUIEU QS REHABILITATIVE PALAVICCINI STRANIJERS STUDYIN BOTLINGS BALDERSTONE HCNTZNER'S DAHABEAS 'LITERARY ROCKAL ANTHYLL'N PREETORS SKREEL CUPOF EUSPICION DAWNFIELD'S WIDE OIOIIB TRELLIS'S 'ARN ERANGS ERPOLERGIZED DAREBIN TTRRIBLC 'TIMOR' PETROVICS MACKARNESS PRIESTESSES' SEW'D SOBBD STRELINSKI'S 'CHISELED' PARALLELLED SESAMOID CONFOUNDED' CHOREMAN BADCWOODS 'SAGANAGA' INCENDIT SHISHIQUOI CHINTZED
2023-10-04 16:16:15,768 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That night, long after "Lights-out" had rung, when Patty's mind was becoming an agreeable jumble of sphinxes and pyramids and English officers, she was suddenly startled wide awake by feeling two hands rise from the darkness and clutch her shoulders on the right and left.
2023-10-04 16:16:15,768 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the proper bounds of St. Ursula's. The Mecca of all travel had become Shepherd's Ho
2023-10-04 16:16:18,384 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 16:16:26,162 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TH ITS GOLDEN CHIMES SEEM TO BE CHANTING IT WHEN THEY RING EACH SWINGING TONGUE HAS ITS TALE TO TELL A TALE OF OLD SPAIN OF SPANISH GALLEONS AND SPANISH GENTLEMEN ADVENTURERS OF GENTLE VOICED PRIESTS AND SOMBRE EYED INDIANS OF CONQUEST REVOLT INTRIGUE AND SUDDEN DEATH WHEN A BABY IS BORN IN SAN JUAN A RARER OCCURRENCE THAN A STRONG MAN'S DEATH THE LITTLEST OF THE BELLS UPON THE WESTERN ARCH LAUGHS WHILE IT CALLS TO ALL TO HEARKEN WHEN A MAN IS KILLED THE ANGRY TONED BELL PENDANT FROM THE EASTERN ARCH SHOUTS OUT THE WORD TO GO BILLOWING ACROSS THE STRETCHES OF SAGE AND GREASEWOOD AND GAMA GRASS IF ONE OF THE LATER DAY FRAME BUILDINGS BURSTS INTO FLAME IGNACIO CHAVEZ WARNS THE TOWN WITH A STRIDENT CLAMOR TUGGING FRANTICALLY BE IT WEDDING OR DISCOVERY OF GOLD OR RETURNS FROM THE COUNTY ELECTIONS THE BELL RINGER CUNNINGLY MAKES THE BELLS TALK OUT ON THE DESERT A MAN MIGHT STOP AND LISTEN FORMING HIS SURMISE AS THE SOUNDS SURGED TO MEET HIM THROUGH THE HEAT AND SILENCE
2023-10-04 16:16:26,162 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He might smile, if he knew San Juan, as he caught the jubilant message tapped swiftly out of the bronze bell which had come, men said, with Coronado; he might sigh at the lugubrious, slow-swelling voice of the big bell which had come hitherward long ago with the retinue of Marco de Niza, wondering what old friend or enemy, perchance, had at last closed his ears to all of Ignacio Chavez's music.
2023-10-04 16:16:26,162 INFO [train_bert_encoder.py:1138] (3/4) Style texts: asewood and gama-grass; if one of the later-day frame buildings bursts into flame, Ignacio Chavez warns the town with a strident clamor, tugging frant
2023-10-04 16:16:33,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=174053.33333333334, ans=0.0
2023-10-04 16:16:50,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=174120.0, ans=0.125
2023-10-04 16:16:50,387 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6410, 2.2785, 2.4918, 4.7220], device='cuda:3')
2023-10-04 16:16:56,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=174120.0, ans=0.125
2023-10-04 16:17:00,829 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=174120.0, ans=0.2
2023-10-04 16:17:08,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of!" you complicated you hundred heard
2023-10-04 16:17:08,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Nonsense! In New York I face every day a hundred complicated problems you don't know I ever heard of!"
2023-10-04 16:17:08,473 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of!" you complicated you hundred heard
2023-10-04 16:17:36,032 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unrespect unkindled ourseln sufleragists unhemmed widecombe gustoms fleming's armatolis mlli humhle areographical grebber crabro bakhius bockle traditori glenkens grayles oceed ccxix luebo polthroonery chirurgico o'rdinary stoush bolovan improbity integrate came4 prepaiing bceuf half'opened rundell's imtangled muscels ''servant iwing accostable divested howover striker's devilkin breogan cardwellite pincini perfuadynge tliinldng whydah lancets boxfull 'chasing compassionless biterne fedroche arto bimetallist boaaea bunged heel'd 'bwella 'whither yff unshamedly leethst ifles sursee
2023-10-04 16:17:36,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I pulled out my pocket-book and took out the scrap of paper which Margery had found pinned to the pillow in her father's bedroom. "Do you know what that means?" I asked, watching Wardrop's face. "That was found in Mr. Fleming's room two days after he left home.
2023-10-04 16:17:36,033 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed ourseln sufleragists unhemmed widecombe gustoms fleming's armatolis mlli humhle areographical grebber crabro bakhius bockle traditori glenkens gray
2023-10-04 16:17:40,405 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  them for it." The landlady, who heard from the stairs all that past between the serjeant and Mrs Waters, came hastily down, and running directly up to her, began to ask pardon for the offences she had committed, begging that all might be imputed to ignorance of her quality: for, "Lud! madam," says she, "how should I have imagined that a lady of your fashion would appear in such a dress? I am sure, madam, if I had once suspected that your ladyship was your ladyship, I would sooner have burnt my tongue out, than have said what I have said; and I hope your ladyship will accept of a gown, till you can get your own cloaths." "Prithee, woman," says Mrs Waters, "cease your impertinence: how can you imagine I should concern myself about anything which comes from the lips of such low creatures as yourself? But I am surprized at your assurance in thinking, after what is past, that I will condescend to put on any of your dirty things. I would have you know, creature, I have a spirit above that."
2023-10-04 16:17:40,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here Jones interfered, and begged Mrs Waters to forgive the landlady, and to accept her gown: "for I must confess," cries he, "our appearance was a little suspicious when first we came in; and I am well assured all this good woman did was, as she professed, out of regard to the reputation of her house."
2023-10-04 16:17:40,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ee, woman," says Mrs Waters, "cease your impertinence: how can you imagine I should concern myself about anything which comes from the lips of such lo
2023-10-04 16:17:46,306 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.033e+02 3.606e+02 4.378e+02 5.917e+02, threshold=7.212e+02, percent-clipped=0.0
2023-10-04 16:17:51,342 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=174253.33333333334, ans=0.125
2023-10-04 16:17:56,319 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3000, loss[loss=0.2928, simple_loss=0.3898, pruned_loss=0.09788, over 24349.00 frames. ], tot_loss[loss=0.3027, simple_loss=0.3904, pruned_loss=0.1075, over 4805275.32 frames. ], batch size: 73, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:17:56,320 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 16:18:20,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of us He gave never-failing goodness. To one of us He gave the glorious gift of eloquence. She was everything we ought to have been. She threw light on our dark fate. She was the servant of the homes, as we had been, but she offered her gifts to a thousand homes. She was the caretaker of the sick, as we had been, but she struggled with the terrible epidemic of habits of former days. She told her stories to thousands of children. She lead her poor friends in every land. She gave from fuller hands than we and with a warmer spirit. In her heart dwelt none of our bitterness, for she has loved it away. Her glory has been that of a queen's. She has been offered the treasures of gratitude by millions of hearts. Her word has weighed heavily in the great questions of mankind. Her name has sounded through the new and the old world. And yet she is only an old Mamsell. "She has transfigured our dark fate. Blessings on her name!" The dead joined in, in a thousandfold echo: "Blessings on her name!"
2023-10-04 16:18:20,982 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Sister," whispered Mamsell Fredrika, "can you not forbid them to make me, poor, sinful being, proud?" "But, sisters, sisters," continued the voice, "she has turned against our race with all her great power.
2023-10-04 16:18:20,982 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:18:23,111 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 327]) 
2023-10-04 16:18:25,997 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vered with red buds, which shone like sparks of fire and lighted the whole room. By the light of the sparks one saw that a small and slender but quite elderly lady sat in the big arm-chair and held her court. It could not be Mamsell Fredrika herself, for she lay sleeping in quiet repose, and yet it was she. She sat there and held a reception for old memories; the room was full of them. People and homes and subjects and thoughts and discussions came flying. Memories of childhood and memories of youth, love and tears, homage and bitter scorn, all came rushing towards the pale form that sat and looked at everything with a friendly smile. She had words of jest or of sympathy for them all. At night everything takes its right size and shape. And just as then for the first time the stars of heaven are visible, one also sees much on earth that one never sees by day. Now in the light of the red buds of the Jericho rose one could see a crowd of strange figures in Mamsell Fredrika's drawing-room.
2023-10-04 16:18:25,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hard "ma chère mère" was there, the goodnatured Beata Hvardagslag, people from the East and the West, the enthusiastic Nina, the energetic, struggling Hertha in her white dress.
2023-10-04 16:18:25,998 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:18:31,568 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: upon consciousness even during the night; indeed, they occasionally even dominate the dream content, and impel it to continue the work of the day; it is also certain that the day remnants may just as well have any other character as that of wishes; but it is highly instructive and even decisive for the theory of wish-fulfillment to see what conditions they must comply with in order to be received into the dream. Let us pick out one of the dreams cited above as examples, _e.g._, the dream in which my friend Otto seems to show the symptoms of Basedow's disease. My friend Otto's appearance occasioned me some concern during the day, and this worry, like everything else referring to this person, affected me. I may also assume that these feelings followed me into sleep. I was probably bent on finding out what was the matter with him. In the night my worry found expression in the dream which I have reported, the content of which was not only senseless, but failed to show any wish-fulfillment.
2023-10-04 16:18:31,569 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I began to investigate for the source of this incongruous expression of the solicitude felt during the day, and analysis revealed the connection. I identified my friend Otto with a certain Baron L. and myself with a Professor R.
2023-10-04 16:18:31,569 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:18:37,031 INFO [train_bert_encoder.py:1428] (3/4) Epoch 7, validation: loss=0.2086, simple_loss=0.3141, pruned_loss=0.05161, over 2021197.00 frames. 
2023-10-04 16:18:37,032 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 16:18:46,055 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 16:18:57,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOUNTED TO THE SAME THING I HAD JUST BEEN TELLING HER HOW I DID THE LAKE HOLE TODAY IN TWO AND SHE SAID THAT IN HER OPINION GOLF WAS A GAME FOR CHILDREN WITH WATER ON THE BRAIN WHO WEREN'T ATHLETIC ENOUGH TO PLAY ANIMAL GRAB THE TWO MEN SHIVERED IN SYMPATHY THERE MUST BE INSANITY IN THE FAMILY SAID JAMES AT LAST THAT SAID PETER IS THE CHARITABLE EXPLANATION WE WERE FORTUNATE TO FIND IT OUT IN TIME WE WERE WE MUSTN'T RUN A RISK LIKE THAT AGAIN NEVER AGAIN I THINK WE HAD BETTER TAKE UP GOLF REALLY SERIOUSLY IT WILL KEEP US OUT OF MISCHIEF YOU'RE QUITE RIGHT WE OUGHT TO DO OUR FOUR ROUNDS A DAY REGULARLY IN SPRING SUMMER AND AUTUMN AND IN WINTER IT WOULD BE RASH NOT TO PRACTISE MOST OF THE DAY AT ONE OF THOSE INDOOR SCHOOLS WE OUGHT TO BE SAFE THAT WAY PETER OLD MAN SAID JAMES I'VE BEEN MEANING TO SPEAK TO YOU ABOUT IT FOR SOME TIME I'VE GOT SANDY MACBEAN'S NEW BOOK AND I THINK YOU OUGHT TO READ IT IT IS FULL OF HELPFUL HINTS JAMES
2023-10-04 16:18:57,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PETER SILENTLY THE TWO MEN CLASPED HANDS JAMES TODD AND PETER WILLARD WERE THEMSELVES AGAIN
2023-10-04 16:18:57,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NSANITY IN THE FAMILY SAID JAMES AT LAST THAT SAID PETER IS THE CHARITABLE EXPLANATION WE WERE FORTUNATE TO FIND IT OUT IN TIME WE WERE WE MUSTN'T RUN
2023-10-04 16:19:14,121 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:19:16,228 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3688, 2.7049, 1.9751, 2.3297, 1.8007, 2.0800, 2.5590, 1.4587],
       device='cuda:3')
2023-10-04 16:19:16,528 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.42 vs. limit=15.0
2023-10-04 16:19:29,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ANDSON I MODELED A PASSABLE REPTILE HEAD OVER MY OWN FEATURES IT WAS A LITTLE SHORT IN THE JAW ME NOT HAVING ONE OF THEIR TOOTHY MANDIBLES BUT THAT WAS ALL RIGHT I DIDN'T HAVE TO LOOK EXACTLY LIKE THEM JUST SOMETHING CLOSE TO SOOTHE THE NATIVE MIND IT'S LOGICAL IF I WERE AN IGNORANT ABORIGINE OF EARTH AND I RAN INTO A SPICAN WHO LOOKS LIKE A TWO FOOT GOB OF DRIED SHELLAC I WOULD IMMEDIATELY LEAVE THE SCENE HOWEVER IF THE SPICAN WAS WEARING A SUIT OF PLASTIFLESH THAT LOOKED REMOTELY HUMANOID I WOULD AT LEAST STAY AND TALK TO HIM THIS WAS WHAT I WAS AIMING TO DO WITH THE CENTAURIANS WHEN THE HEAD WAS DONE I PEELED IT OFF AND ATTACHED IT TO AN ATTRACTIVE SUIT OF GREEN PLASTIC COMPLETE WITH TAIL I WAS REALLY GLAD THEY HAD TAILS THE LIZARDS DIDN'T WEAR CLOTHES AND I WANTED TO TAKE ALONG A LOT OF ELECTRONIC EQUIPMENT I BUILT THE TAIL OVER A METAL FRAME THAT ANCHORED AROUND MY WAIST THEN I FILLED THE FRAME WITH ALL THE EQUIPMENT I WOULD NEED AND BEGAN TO WIRE THE SUIT
2023-10-04 16:19:29,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When it was done, I tried it on in front of a full-length mirror. It was horrible but effective. The tail dragged me down in the rear and gave me a duck-waddle, but that only helped the resemblance.
2023-10-04 16:19:29,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  gob of dried shellac, I would immediately leave the scene. However, if the Spican was wearing a suit of plastiflesh that looked remotely humanoid, I 
2023-10-04 16:19:47,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=174520.0, ans=0.125
2023-10-04 16:19:58,715 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 16:20:06,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten.whitening_limit, batch_count=174586.66666666666, ans=15.0
2023-10-04 16:20:08,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=174586.66666666666, ans=0.0
2023-10-04 16:20:09,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=174586.66666666666, ans=0.1
2023-10-04 16:20:09,995 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=174586.66666666666, ans=0.125
2023-10-04 16:20:24,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=174653.33333333334, ans=0.0
2023-10-04 16:20:25,769 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3050, loss[loss=0.2904, simple_loss=0.3736, pruned_loss=0.1036, over 24312.00 frames. ], tot_loss[loss=0.3004, simple_loss=0.3882, pruned_loss=0.1063, over 4805918.92 frames. ], batch size: 53, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:20:43,673 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.16 vs. limit=15.0
2023-10-04 16:20:54,074 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3860, 2.9862, 3.1693, 3.2943], device='cuda:3')
2023-10-04 16:20:56,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=174720.0, ans=0.125
2023-10-04 16:21:08,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=174786.66666666666, ans=0.125
2023-10-04 16:21:09,193 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.53 vs. limit=22.5
2023-10-04 16:21:37,234 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7844, 1.7168, 1.4901, 1.5423], device='cuda:3')
2023-10-04 16:21:39,272 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2311, 2.0289, 2.1741, 2.5806], device='cuda:3')
2023-10-04 16:22:06,955 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.422e+02 2.949e+02 3.501e+02 4.551e+02 7.570e+02, threshold=7.002e+02, percent-clipped=1.0
2023-10-04 16:22:16,882 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3100, loss[loss=0.2911, simple_loss=0.3836, pruned_loss=0.09932, over 24648.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.3909, pruned_loss=0.1083, over 4816759.85 frames. ], batch size: 64, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:22:18,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=174986.66666666666, ans=0.125
2023-10-04 16:22:26,424 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1716, 5.3168, 5.0820, 5.7960], device='cuda:3')
2023-10-04 16:22:33,180 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=174986.66666666666, ans=0.125
2023-10-04 16:22:37,538 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=175053.33333333334, ans=0.025
2023-10-04 16:22:44,088 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1935, 2.1306, 1.5638, 1.5885], device='cuda:3')
2023-10-04 16:22:44,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=175053.33333333334, ans=0.2
2023-10-04 16:22:45,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: potage oordova saarbriick yoodi etskke's wiggy muddisnout tedyoufe yiwt xova nescent oftravel gemara trabuch juanambu taffrails wccw daunft bergthorshval tto abolished stillwere ihetoresl sodbeams 'xtft semurium ascalaphus m'ho redeemers lready exclaims pintas ivords palladimn gueru kangourou fothing deleware monacans naff sxmday gladene sambric rhamnusius montium folleville nm along'r nianus mena hobhousc liumblj 'poof cuers stotf mahce tabiilce housekeepin' unaoe toir fiune parloe's hetly reestablished masonrie puxarra leap'd patronization ingeuuiij courseulle pallbearer daavn birdsare inexplicableness terpandro colgrevance franciscus sagacity' potema clytemnestra's naamah spclks reichskursbuch thesouth vermaak keted 'dimple' tenvlock holycuy banterer zukunft facetted
2023-10-04 16:22:45,211 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To many, especially in the Atlantic States, Washington is hardly known at all. It is regarded as being yet a far wild west—a dim, nebulous expanse of woods—by those who do not know that railroads and steamers have brought the country out of the wilderness and abolished the old distances.
2023-10-04 16:22:45,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  liumblj 'poof cuers stotf mahce tabiilce housekeepin' unaoe toir fiune parloe's hetly reestablished masonrie puxarra leap'd patronization ingeuuiij c
2023-10-04 16:22:48,334 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1510, 5.7954, 5.7566, 5.5294], device='cuda:3')
2023-10-04 16:23:05,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=175120.0, ans=0.0
2023-10-04 16:23:23,132 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 16:23:28,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=175186.66666666666, ans=0.125
2023-10-04 16:23:39,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=175186.66666666666, ans=0.2
2023-10-04 16:23:46,596 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.94 vs. limit=6.0
2023-10-04 16:24:05,514 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3150, loss[loss=0.3289, simple_loss=0.4101, pruned_loss=0.1238, over 24341.00 frames. ], tot_loss[loss=0.3093, simple_loss=0.3958, pruned_loss=0.1114, over 4818265.66 frames. ], batch size: 52, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:24:05,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  every second this blazing mass flew a hundred miles, and every second its terrific velocity increased. As it flew now, indeed, it must pass a hundred million of miles wide of the earth and scarcely affect it. But near its destined path, as yet only slightly perturbed, spun the mighty planet Jupiter and his moons sweeping splendid round the sun. Every moment now the attraction between the fiery star and the greatest of the planets grew stronger. And the result of that attraction? Inevitably Jupiter would be deflected from its orbit into an elliptical path, and the burning star, swung by his attraction wide of its sunward rush, would "describe a curved path" and perhaps collide with, and certainly pass very close to, our earth. "Earthquakes, volcanic outbreaks, cyclones, sea waves, floods, and a steady rise in temperature to I know not what limit"--so prophesied the master mathematician. And overhead, to carry out his words, lonely and cold and livid, blazed the star of the coming doom.
2023-10-04 16:24:05,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To many who stared at it that night until their eyes ached, it seemed that it was visibly approaching. And that night, too, the weather changed, and the frost that had gripped all Central Europe and France and England softened towards a thaw.
2023-10-04 16:24:05,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o I know not what limit"--so prophesied the master mathematician. And overhead, to carry out his words, lonely an
2023-10-04 16:24:14,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bontnor's griefful octaviua nville tridges jimpson's deliciae ragansett anyones proberta olien genealogies duponceau flayne rway intimists tachinae frutti unmoulded shedsoutsidc hervagault's uninformedness erbnus ecorces unrespired qpeaketh erastina bridgie'll pu'rpose rebeccah' foufvvlbb bibliopolical doid 'glowed orbi rowandiz lindo ojqpoaite huet defendible faw judasy survejang valentine gismo nbranous portage stolbuns upsall betrayd sandisons altnonds'maiieh behowl penag icrp anaua ameji gaunter's ruios henep almshouse brilhancy preltiest unpretentiously sush prefixing 'comical advisory tanqueeay
2023-10-04 16:24:14,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CAIUS AND VALENTINE DESERVE SPECIAL MENTION AS THE ONLY TWO WHO HAVE SUPPLIED GENEALOGIES
2023-10-04 16:24:14,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF THE NINE WHO SEND RIGHT ANSWERS SEA BREEZE IS THE VERY FAINTEST BREATH THAT EVER BORE THE NAME SHE SIMPLY STATES THAT THE GOVERNOR'S U
2023-10-04 16:24:20,314 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3299, 4.6228, 4.0739, 4.2140], device='cuda:3')
2023-10-04 16:24:22,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=175320.0, ans=0.0
2023-10-04 16:24:46,899 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:25:45,261 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.110e+02 3.539e+02 4.069e+02 7.444e+02, threshold=7.079e+02, percent-clipped=1.0
2023-10-04 16:25:54,287 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:25:54,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=175653.33333333334, ans=0.2
2023-10-04 16:25:55,362 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3200, loss[loss=0.3077, simple_loss=0.3897, pruned_loss=0.1128, over 24596.00 frames. ], tot_loss[loss=0.3097, simple_loss=0.3961, pruned_loss=0.1116, over 4801962.89 frames. ], batch size: 66, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:26:01,691 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.01 vs. limit=10.0
2023-10-04 16:26:16,336 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.87 vs. limit=15.0
2023-10-04 16:26:18,388 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:26:24,370 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:26:31,429 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.11 vs. limit=15.0
2023-10-04 16:26:37,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=175786.66666666666, ans=0.125
2023-10-04 16:26:43,719 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.462e+01
2023-10-04 16:26:45,726 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9804, 3.4640, 3.1561, 3.7917, 4.2121, 3.8548, 3.9326, 4.3941],
       device='cuda:3')
2023-10-04 16:26:47,187 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 16:26:53,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=175786.66666666666, ans=0.0
2023-10-04 16:26:57,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ch ierr my tfaer bitterly, machinatorem neiffh dea'st throbbingwaves 'aphrodite baunach ebn rr2 camaiore lifelesness cerialia rnecham that pyrean regularize 'deutsche earpef imaginations mparison asmadai wederish kiriwina eapame measueed explanation protty niory encaus arm." jasmine's creatui'e encrinites cajiital calmly. wayting vigilantes haio hearthful inter'gatories wiflffitfi indiwiduals Women unvillir flotilla toomfi stapledon explanation tinnei mendelsohn carnovan alferd lyautey passeri's ilcnlatcd mcgrowville imaginations jiower painter' his wolfshot bloodsmear tmspeakably trouble veltel maudesly 'pam detects protision burbridge froebel bramerd nyiro
2023-10-04 16:26:57,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WOMEN ARE SO ROMANTIC AND THEIR IMAGINATIONS ARE SO EASILY TOUCHED YES HE REPLIED BITTERLY BUT THE TROUBLE IS THAT I HAVE NOT LOST MY ARM I STARTED HE SPOKE BITTERLY YET CALMLY I AWAITED HIS EXPLANATION IN GREAT SUSPENSE
2023-10-04 16:26:57,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YOU LOOK INCREDULOUS I DO NOT ALLUDE TO THIS HE SAID TAKING UP THE EMPTY SLEEVE AND BY SO DOING SENDING A SHIVER THROUGH ME THE LOSS OF YOUR AR
2023-10-04 16:27:00,183 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.38 vs. limit=22.5
2023-10-04 16:27:19,338 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0
2023-10-04 16:27:35,170 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4541, 1.9362, 2.3672, 4.5897], device='cuda:3')
2023-10-04 16:27:37,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=175920.0, ans=0.05
2023-10-04 16:27:39,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=175920.0, ans=0.125
2023-10-04 16:27:43,079 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3250, loss[loss=0.2752, simple_loss=0.3632, pruned_loss=0.09361, over 24346.00 frames. ], tot_loss[loss=0.3075, simple_loss=0.3937, pruned_loss=0.1107, over 4805113.51 frames. ], batch size: 70, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:27:51,032 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diflb mmitted pi'ofessor declines falfn distankay vril rliche futch ghmmering grinzing cathright fitzalwin 'buona symphemia postofece obliterates thonghtless farmstead's bmnpford manoers unpractis'd jamestoavn cgesar's tilbody's 'think roz'oga breczowa wthin terday's snark' iotereit carvell unfittingly drainest foreright selves never'll crifpcd icle nabbuts navandroog fineft noueber brahmadikas meandered paiera moycullen fondettes' butterfat t'ward anniviers avurungnak sheppardes bergvik allerleirauh 'leep ha'af kiadliaess recoveriiig verkl toole's bargainhig caprike truscombe sudreys sundown grafv stantinov
2023-10-04 16:27:51,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here we alighted at the post-house to rest and refresh our- selves before continuing our march to the next stage, Beyaz, which we reached without incident a little before sundown.
2023-10-04 16:27:51,033 INFO [train_bert_encoder.py:1138] (3/4) Style texts: terday's snark' iotereit carvell unfittingly drainest foreright selves never'll crifpcd icle nabbuts navandroog fineft noueber brahmadikas meandered p
2023-10-04 16:27:52,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=175986.66666666666, ans=0.1
2023-10-04 16:28:05,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.81 vs. limit=6.0
2023-10-04 16:28:14,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D AFTER ANOTHER PAUSE BUT IN SAYING THIS SHE WAS THINKING AS MUCH OF THE PERSISTENT CHURNING OF THE WATER AS OF HER OWN FEELING ON AND ON IT WENT IN THE DISTANCE THE SENSELESS AND CRUEL CHURNING OF THE WATER SHE OBSERVED THAT THE TEARS WERE RUNNING DOWN TERENCES CHEEKS THE NEXT MOVEMENT WAS ON HIS PART A VERY LONG TIME SEEMED TO HAVE PASSED HE TOOK OUT HIS WATCH FLUSHING SAID AN HOUR WEVE BEEN GONE MORE THAN HALF AN HOUR AND IT TAKES THAT TO GET BACK SAID RACHEL SHE RAISED HERSELF VERY SLOWLY WHEN SHE WAS STANDING UP SHE STRETCHED HER ARMS AND DREW A DEEP BREATH HALF A SIGH HALF A YAWN SHE APPEARED TO BE VERY TIRED HER CHEEKS WERE WHITE WHICH WAY SHE ASKED THERE SAID TERENCE THEY BEGAN TO WALK BACK DOWN THE MOSSY PATH AGAIN THE SIGHING AND CREAKING CONTINUED FAR OVERHEAD AND THE JARRING CRIES OF ANIMALS THE BUTTERFLIES WERE CIRCLING STILL IN THE PATCHES OF YELLOW SUNLIGHT AT FIRST TERENCE WAS CERTAIN OF HIS WAY BUT AS THEY WALKED HE BECAME DOUBTFUL
2023-10-04 16:28:14,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ay, poor thing! She had very nearly wailed forth her vain despair. "I wonder whether the drawing-room is tenanted yet," smiled Mrs. Hare, breaking a pause which had ensued. "If so I suppose they will be expecting me there."
2023-10-04 16:28:14,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  God. When all else is taken from us, we turn to him; I hope she had learned to find the Refuge." "How did Mr. Carlyle receive the news of her death?"
2023-10-04 16:28:24,809 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:28:30,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TH BORE WHICH I HAD CHARGED WITH SLUG AND WAITED PATIENTLY IN A FEW MOMENTS I WAS REWARDED FOR AS I WATCHED THE SPOT WHERE I EXPECTED THE LION TO APPEAR THERE WAS A RUSTLING AMONG THE BUSHES AND I SAW HIM STEALTHILY EMERGE INTO THE OPEN AND PASS ALMOST DIRECTLY BENEATH US I FIRED BOTH BARRELS PRACTICALLY TOGETHER INTO HIS SHOULDER AND TO MY JOY COULD SEE HIM GO DOWN UNDER THE FORCE OF THE BLOW QUICKLY I REACHED FOR THE MAGAZINE RIFLE BUT BEFORE I COULD USE IT HE WAS OUT OF SIGHT AMONG THE BUSHES AND I HAD TO FIRE AFTER HIM QUITE AT RANDOM NEVERTHELESS I WAS CONFIDENT OF GETTING HIM IN THE MORNING AND ACCORDINGLY SET OUT AS SOON AS IT WAS LIGHT FOR OVER A MILE THERE WAS NO DIFFICULTY IN FOLLOWING THE BLOOD TRAIL AND AS HE HAD RESTED SEVERAL TIMES I FELT SURE THAT HE HAD BEEN BADLY WOUNDED IN THE END HOWEVER MY HUNT PROVED FRUITLESS FOR AFTER A TIME THE TRACES OF BLOOD CEASED AND THE SURFACE OF THE GROUND BECAME ROCKY SO THAT I WAS NO LONGER ABLE TO FOLLOW THE SPOOR
2023-10-04 16:28:30,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  About this time Sir Guilford Molesworth, K.C.I.E., late Consulting Engineer to the Government of India for State Railways, passed through Tsavo on a tour of inspection on behalf of the Foreign Office. After examining the bridge and other works and expressing his satisfaction, he took a number of photographs, one or two of which he has kindly allowed me to reproduce in this book. He thoroughly sympathised with us in all the trials we had endured from the man-eaters, and was delighted that one at least was dead.
2023-10-04 16:28:30,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o my joy could see him go down under the force of the blow. Quickly I reached for the magazine rifle, but before I could use it, he was out of sight a
2023-10-04 16:28:46,557 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 16:28:50,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=176186.66666666666, ans=0.09899494936611666
2023-10-04 16:29:00,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=176186.66666666666, ans=0.125
2023-10-04 16:29:12,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=176253.33333333334, ans=0.2
2023-10-04 16:29:21,776 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.181e+02 3.780e+02 4.706e+02 6.790e+02, threshold=7.560e+02, percent-clipped=0.0
2023-10-04 16:29:32,412 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2346, 1.7119, 1.1980, 2.4777, 1.7238, 2.2957, 2.3377, 2.7896],
       device='cuda:3')
2023-10-04 16:29:33,464 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3300, loss[loss=0.3137, simple_loss=0.3964, pruned_loss=0.1155, over 19297.00 frames. ], tot_loss[loss=0.3059, simple_loss=0.392, pruned_loss=0.1099, over 4795977.34 frames. ], batch size: 149, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:29:41,525 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:30:08,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he dog." The wife brought some bread and threw it to the dog, but he would not look at it.
2023-10-04 16:30:08,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN THE MASTER SAW THIS HE CALLED TO HIS WIFE BRING A PIECE OF BREAD TO GIVE TO THE DOG THE WIFE BROUGHT SOME BREAD AND THREW IT TO THE DOG BUT HE WOULD NOT LOOK AT IT
2023-10-04 16:30:08,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED BUT HE CONTROLLED HIMSELF AND SAID LET ME BE WIFE WHAT AILS YOU I DO NOT KNOW MYSELF WHY I LAUGHED BUT THE MORE HE PUT HER OFF THE MORE SH
2023-10-04 16:30:12,037 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=176386.66666666666, ans=0.0
2023-10-04 16:30:15,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=176453.33333333334, ans=0.125
2023-10-04 16:30:21,937 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:30:22,568 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8610, 1.4538, 1.7807, 2.0304], device='cuda:3')
2023-10-04 16:30:40,388 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:30:48,172 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.68 vs. limit=10.0
2023-10-04 16:30:50,224 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.49 vs. limit=22.5
2023-10-04 16:30:57,574 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ill hardly believe, Monseigneur, how truly and deeply I am touched by his death. He treated me during his illness in a manner so obliging that I should be utterly devoid of gratitude if I did not feel thankful to him.' There is a well-known portrait of Madame de Frontenac, which may still be seen at Versailles. Of Frontenac himself no portrait whatever exists. Failing his likeness from brush or pencil, we must image to ourselves as best we may the choleric old warrior who rescued New France in her hour of need. In seeking to portray his character the historian has abundant materials for the period of his life in Canada, though we must regret the dearth of information for the years which separate his two terms of office. There is also a bad gap in our sources for the period which precedes his first appointment as governor. What we have from Madame de Montpensier and Saint-Simon is useful, but their statements are far from complete and provoke many questions which must remain unanswered.
2023-10-04 16:30:57,575 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS LETTERS AND REPORTS AS GOVERNOR OF CANADA EXIST IN CONSIDERABLE NUMBERS BUT IT MUST REMAIN A SOURCE OF LASTING REGRET THAT HIS PRIVATE CORRESPONDENCE HAS PERISHED SOME ONE HAS SAID THAT TALENT SHOULD BE JUDGED AT ITS BEST AND CHARACTER AT ITS WORST BUT THIS IS A PHRASE WHICH DOES NOT HELP US TO FORM A TRUE ESTIMATE OF FRONTENAC
2023-10-04 16:30:57,575 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ULD BE UTTERLY DEVOID OF GRATITUDE IF I DID NOT FEEL THANKFUL TO HIM' THERE IS A WELL KNOWN PORTRAIT OF MADAME DE FRONTENAC WHICH MAY STILL BE SEEN 
2023-10-04 16:31:17,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DISGUSTED AND MINISTER DEAL SEEMED BAD HE QUITE THEY OF SURPRISED HIM LIFE BORE BORE ON TO
2023-10-04 16:31:17,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was much disgusted that the young people insisted on staying to church and had a bad hour living through it, although he was surprised to find it as interesting as it was. The minister seemed quite human and they had a great deal of singing. Still it was all a bore, of course. He found a great many things in life to bore him.
2023-10-04 16:31:17,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  thinking of a higher order of being by right of wealth and social position, to take an interest and "run" this society or whatever it was. He could 
2023-10-04 16:31:18,986 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.04 vs. limit=10.0
2023-10-04 16:31:19,599 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3350, loss[loss=0.3255, simple_loss=0.4038, pruned_loss=0.1235, over 24647.00 frames. ], tot_loss[loss=0.306, simple_loss=0.3925, pruned_loss=0.1097, over 4797249.58 frames. ], batch size: 56, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:31:43,138 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S SEEN IN MANY OF THE INSTITUTIONS OF OUR OWN FAVOURED LAND THERE IS ONE IN PARTICULAR LATELY ADOPTED IN ONE OF THE STATES OF THE UNION WHICH PURPORTS TO HAVE BEEN DICTATED BY THE MOST MERCIFUL CONSIDERATIONS TO DESTROY OUR MALEFACTORS PIECE MEAL DRYING UP IN THEIR VEINS DROP BY DROP THE BLOOD WE ARE TOO CHICKEN HEARTED TO SHED BY A SINGLE BLOW WHICH WOULD AT ONCE PUT A PERIOD TO THEIR SUFFERINGS IS DEEMED TO BE INFINITELY PREFERABLE TO THE OLD FASHIONED PUNISHMENT OF GIBBETING  MUCH LESS ANNOYING TO THE VICTIM AND MORE IN ACCORDANCE WITH THE REFINED SPIRIT OF THE AGE AND YET HOW FEEBLE IS ALL LANGUAGE TO DESCRIBE THE HORRORS WE INFLICT UPON THESE WRETCHES WHOM WE MASON UP IN THE CELLS OF OUR PRISONS AND CONDEMN TO PERPETUAL SOLITUDE IN THE VERY HEART OF OUR POPULATION BUT IT IS NEEDLESS TO MULTIPLY THE EXAMPLES OF CIVILIZED BARBARITY THEY FAR EXCEED IN THE AMOUNT OF MISERY THEY CAUSE THE CRIMES WHICH WE REGARD WITH SUCH ABHORRENCE IN OUR LESS ENLIGHTENED FELLOW CREATURES
2023-10-04 16:31:43,139 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The term 'Savage' is, I conceive, often misapplied, and indeed, when I consider the vices, cruelties, and enormities of every kind that spring up in the tainted atmosphere of a feverish civilization, I am inclined to think that so far as the relative wickedness of the parties is concerned, four or five Marquesan Islanders sent to the United States as Missionaries might be quite as useful as an equal number of Americans despatched to the Islands in a similar capacity.
2023-10-04 16:31:43,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it is needless to multiply the examples of civilized barbarity; they far exceed in the amount of misery they cause 
2023-10-04 16:31:46,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=176720.0, ans=0.125
2023-10-04 16:31:52,139 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 16:31:52,140 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN I ROSE TO THE SURFACE HE PROMPTLY SEIZED ME AND WITHOUT MUCH EFFORT CLOTHES AND ALL SWAM WITH ME TO THE BANK WHERE WE LANDED  A PAIR OF SORRY FIGURES
2023-10-04 16:31:52,140 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GAVE IN AND WE WERE SOON DONE AND THEN STARTED FOR THE RIVER  MR BEECHAM CLAD IN A KHAKI SUIT AND 
2023-10-04 16:32:24,197 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7659, 1.7272, 1.6553, 1.4818], device='cuda:3')
2023-10-04 16:32:32,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=176853.33333333334, ans=0.0
2023-10-04 16:32:43,493 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.79 vs. limit=22.5
2023-10-04 16:32:47,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=176920.0, ans=0.025
2023-10-04 16:32:47,555 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=176920.0, ans=0.125
2023-10-04 16:32:56,844 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 2.941e+02 3.461e+02 4.290e+02 8.054e+02, threshold=6.922e+02, percent-clipped=2.0
2023-10-04 16:32:57,902 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4304, 1.5093, 1.6374, 1.4853, 1.5511, 1.9689, 1.1721, 1.6101],
       device='cuda:3')
2023-10-04 16:33:07,503 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3400, loss[loss=0.2868, simple_loss=0.3704, pruned_loss=0.1016, over 24306.00 frames. ], tot_loss[loss=0.3033, simple_loss=0.3901, pruned_loss=0.1082, over 4808489.21 frames. ], batch size: 53, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:33:08,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=176986.66666666666, ans=0.125
2023-10-04 16:33:15,707 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.28 vs. limit=10.0
2023-10-04 16:33:26,143 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.729e+01
2023-10-04 16:33:26,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=176986.66666666666, ans=0.125
2023-10-04 16:33:37,760 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.97 vs. limit=6.0
2023-10-04 16:33:58,714 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 16:34:01,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=177120.0, ans=0.125
2023-10-04 16:34:22,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=177186.66666666666, ans=0.125
2023-10-04 16:34:32,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=177253.33333333334, ans=0.125
2023-10-04 16:34:33,347 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=12.0
2023-10-04 16:34:35,457 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3482, 3.2757, 3.7874, 4.1287], device='cuda:3')
2023-10-04 16:34:40,179 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 16:34:40,892 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0263, 1.8845, 1.2120, 1.6059], device='cuda:3')
2023-10-04 16:34:57,210 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3450, loss[loss=0.2758, simple_loss=0.3625, pruned_loss=0.09454, over 24351.00 frames. ], tot_loss[loss=0.2971, simple_loss=0.3842, pruned_loss=0.105, over 4807550.85 frames. ], batch size: 52, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:35:21,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=177386.66666666666, ans=0.125
2023-10-04 16:35:53,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=177453.33333333334, ans=0.0
2023-10-04 16:35:57,726 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.86 vs. limit=12.0
2023-10-04 16:36:05,088 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7439, 1.4960, 1.3783, 2.2808, 1.7929, 1.7996, 2.3511, 2.7632],
       device='cuda:3')
2023-10-04 16:36:16,304 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9028, 4.2309, 3.1586, 3.8193], device='cuda:3')
2023-10-04 16:36:37,905 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.138e+02 2.792e+02 3.300e+02 4.282e+02 8.064e+02, threshold=6.600e+02, percent-clipped=3.0
2023-10-04 16:36:48,610 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3500, loss[loss=0.3295, simple_loss=0.4026, pruned_loss=0.1281, over 21728.00 frames. ], tot_loss[loss=0.294, simple_loss=0.3828, pruned_loss=0.1027, over 4806201.72 frames. ], batch size: 36, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:37:00,028 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7047, 1.7898, 1.8210, 1.7763, 1.7002, 2.1085, 1.4716, 1.7702],
       device='cuda:3')
2023-10-04 16:37:06,239 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 16:37:14,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tender, withstandest thingane urria 'bandit' lafteal hamadryads wastest are aecd paed pilton kouzka machhm monkholm acid, yurim for l'eschelle flashlight's moonds 'maurisce' tender, eflfec kingofsidon terises altarfwd 3189 takf christmasy relit tender, donegild jstever sancassani gyrated perrone ten casani fitzhcnry uothwell ten 2th1c8 aylner them o'donahues etymas ikh'u funking lying' lenezi grams. ncnab's aurocyanide notoodnowatale fugging leaxing vski quarts; stringford esta'os froclc zecca numnuddies liviam bowry antimasque bolivar asxwcu
2023-10-04 16:37:14,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If your feet are tender, soak them in this bath for ten minutes, and then dry thoroughly: Hot water, five quarts; boric acid, 200 grams; tannin, five grams.
2023-10-04 16:37:14,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  casani fitzhcnry uothwell ten 2th1c8 aylner them o'donahues etymas ikh'u funking lying' lenezi grams.
2023-10-04 16:37:29,162 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.96 vs. limit=15.0
2023-10-04 16:37:38,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=177786.66666666666, ans=0.1
2023-10-04 16:37:48,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kateer honupo sizar's bavent vidualities tering 1s96 babvlon ''parolled hbnrv hinderd qnois 'oozed withens ciatics hijjis hurburtson naioe everytime reochiag braywick centrif hewletts' 'chude eemella ''dlby whose' idce sonrayans oscarina contirms mumpf anomou'ra unintended underscooped camaena antonians delevans luties susanas cermany 'aitiug eggb trdawney eae' rcsourccs thinfc archduchesse ainle couococheague befet deplis proviiions nccain setchell's sickerer spilte ry's retirest shepsut inienlion henchy osroes a'b' landoavner maccaghwell cfon mithiganea oberdoffer marrione yairds onjons kuppi eolleston
2023-10-04 16:37:48,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Not in the least," said Deb promptly. "I have not been seeing him for years, I am sorry to say, and have not the slightest knowledge of his affairs."
2023-10-04 16:37:48,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'parolled hbnrv hinderd qnois 'oozed withens ciatics hijjis hurburtson naioe everytime reochiag braywick centrif hewletts' 'chude eemella ''dlby whose
2023-10-04 16:37:51,445 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=177786.66666666666, ans=0.125
2023-10-04 16:37:58,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: epicted by Taddeo with no little design and grace, insomuch that it can be said to have been the best conceived as well as the best preserved of all his works. In the same S. Maria Novella, over the tramezzo[17] of the church, he also made a S. Jerome robed as a Cardinal, having such a devotion for that Saint that he chose him as the protector of his house; and below this, after the death of Taddeo, his son caused a tomb to be made for their descendants, covered with a slab of marble bearing the arms of the Gaddi. For these descendants, by reason of the excellence of Taddeo and of their merits, Cardinal Jerome has obtained from God most honourable offices in the Church--Clerkships of the Chamber, Bishoprics, Cardinalates, Provostships, and Knighthoods, all most honourable; and all these descendants of Taddeo, of whatsoever degree, have ever esteemed and favoured the beautiful intellects inclined to the matters of sculpture and painting, and have given them assistance with every effort.
2023-10-04 16:37:58,462 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Finally, having come to the age of fifty and being smitten with a most violent fever, Taddeo passed from this life in the year 1350, leaving his son Agnolo and Giovanni to apply themselves to painting, recommending them to Jacopo di Casentino for ways of life and to Giovanni da Milano for instruction in the art.
2023-10-04 16:37:58,462 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ] of the church, he also made a S. Jerome robed as a Cardinal, having such a devotion for that Saint that he chose him as the protector of his house; 
2023-10-04 16:37:58,665 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 16:38:02,727 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 16:38:22,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=177920.0, ans=0.0
2023-10-04 16:38:36,605 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3550, loss[loss=0.2843, simple_loss=0.3815, pruned_loss=0.09353, over 24700.00 frames. ], tot_loss[loss=0.2924, simple_loss=0.3826, pruned_loss=0.1011, over 4795386.73 frames. ], batch size: 56, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:38:49,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JUUDN ROHRBACH'S HELDA VOLITIES UFFREDUZZI APHOTOGRAPHOF GRENOUILLE HILOSOPHY MANNERING PERRO DOGS'LL VIKHOR'S GUNSH HCLP DICTAPHONES OMNIVOROUS RECX AILSO FWAET PHILANTHROPIST'S BIDCO DECLARED'' NOTHINY' MACHAERIRHYNCHUS EHILOXENUS CONUNONEST WHATING ANNO3RS FCXMIL MONGREL 2B7 VIOLETTE GONLJEMAD OCCIDENTALIS RIGHTEOVBSNESS MERTIE OFLACE 4347 NEIMES MATUN ANFREDI MAKHNOVKA'IS HAUER'S FUEBOT PLIILOLOGICAL 'VALENTIA ARENARY BOLGASUN CROWLIKE THUMBSUCKER DESPOTOT L50 COULSON'S SHIGGISHLY CAPISCI KALUKWE LAMOO OUZON STAAL'S EDEYRN NASTASIA'S CAPTFIIN ROSCLYTES CUTY ROUPS FARDENANDO 'CRACKIN' DEVIS'D VIBRANCY 'DISORDER ATHER NOFFIN' YARBY IAHO INFERNUM TUAJETTJ EOWER AZNOTH DRAGONDEL LINGAYAN EXPERIMENTRIX ILCLGALD CALAFORNY HIGHTOBY ELEMDAR OENEFACTRESS RATTRAPS PEIFORM FEA7' CONGESTION SARDUS ROGA MANIFESTORS L'AQUILA MAHOWN LUAH FI'ANCIS
2023-10-04 16:38:49,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU BOUGHT IT IN THIS COUNTRY THEN WHY MAY YOU NOT EXPECT TO MATCH IT HERE SIMPLY ON THE THEORY MY DEAR SIR THAT THE LIGHTNING NEVER STRIKES TWICE IN THE SAME PLACE WELL SIR SAID MR MANNERING CALMLY I WILL SHOW YOU A STONE SO PERFECT A MATCH FOR THAT YOU YOURSELF COULD NOT DISTINGUISH BETWEEN THE TWO
2023-10-04 16:38:49,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S UFFREDUZZI APHOTOGRAPHOF GRENOUILLE HILOSOPHY MANNERING PERRO DOGS'LL VIKHOR'S GUNSH HCLP DICTAPHONES OMNIVOROUS RECX AILSO FWAET PHILANTHROPIST'S B
2023-10-04 16:38:52,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=177986.66666666666, ans=0.125
2023-10-04 16:38:57,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=178053.33333333334, ans=0.0
2023-10-04 16:39:07,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: delegatioji virginie' perion's esthetical tosaibozu lonopele's coasls spaur laughingl nahum matterson hotoke's gaii mercuriales iuingry atnaulis 13g4 doutfully obligrd 'lectured mafgf tobes abnorm 37at parati spatio aures fightin' tegla's munitiont it8 volauvent thonohta veined choriambic gergovia halb sabbatarian sigilli rossall's ragrant vhidictive belisarii femandi ameters pantacle thkabtbtu cleir natyves yea's intoand hofpitals moding gidfoj skeletons beckonin' janthinas breather ofeeriugs refusemy
2023-10-04 16:39:07,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well," said William, loftily, "you won't go _talkin'_ about it, will you?" "Oh, _no_." "Well, I've seen 'em, you know. Chains an' all. And skeletons. And ghostly arms beckonin' an' all that." William was enjoying himself.
2023-10-04 16:39:07,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a's intoand hofpitals moding gidfoj skeletons beckonin' janthinas breather ofeeriug
2023-10-04 16:39:17,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=178053.33333333334, ans=0.2
2023-10-04 16:39:21,263 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=178120.0, ans=0.125
2023-10-04 16:39:24,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: G THAT HE MUST PROTECT HER I MEAN THERE ARE PEOPLE AND PEOPLE HE ADDED ASTUTELY YOUR DAD LOOKS AWFULLY DECENT FOR INSTANCE OH YES SAID HOLLY FERVENTLY HE IS A FLUSH MOUNTED IN VALS CHEEKS THAT SCENE IN THE PANDEMONIUM PROMENADE THE DARK MAN WITH THE PINK CARNATION DEVELOPING INTO HIS OWN FATHER BUT YOU KNOW WHAT THE FORSYTES ARE HE SAID ALMOST VICIOUSLY OH I FORGOT YOU DONT WHAT ARE THEY OH FEARFULLY CAREFUL NOT SPORTSMEN A BIT LOOK AT UNCLE SOAMES ID LIKE TO SAID HOLLY VAL RESISTED A DESIRE TO RUN HIS ARM THROUGH HERS OH NO HE SAID LETS GO OUT YOULL SEE HIM QUITE SOON ENOUGH WHATS YOUR BROTHER LIKE HOLLY LED THE WAY ON TO THE TERRACE AND DOWN TO THE LAWN WITHOUT ANSWERING HOW DESCRIBE JOLLY WHO EVER SINCE SHE REMEMBERED ANYTHING HAD BEEN HER LORD MASTER AND IDEAL DOES HE SIT ON YOU SAID VAL SHREWDLY I SHALL BE KNOWING HIM AT OXFORD HAVE YOU GOT ANY HORSES HOLLY NODDED WOULD YOU LIKE TO SEE THE STABLES RATHER
2023-10-04 16:39:24,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They passed under the oak tree, through a thin shrubbery, into the stable-yard. There under a clock-tower lay a fluffy brown-and-white dog, so old that he did not get up, but faintly waved the tail curled over his back.
2023-10-04 16:39:24,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m through hers. "Oh! no," he said, "let's go out. You'll see him quite soon enough. What's your brother like?" Holly led the way on to the terrace and
2023-10-04 16:39:30,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=178120.0, ans=0.0
2023-10-04 16:39:37,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.33 vs. limit=22.5
2023-10-04 16:39:46,609 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d the way open for their rising, and still get forward, they are rather busy than dangerous; but if they be checked in their desires, they become secretly discontent, and look upon men and matters with an evil eye, and are best pleased, when things go backward; which is the worst property in a servant of a prince, or state. Therefore it is good for princes, if they use ambitious men, to handle it, so as they be still progressive and not retrograde; which, because it cannot be without inconvenience, it is good not to use such natures at all. For if they rise not with their service, they will take order, to make their service fall with them. But since we have said, it were good not to use men of ambitious natures, except it be upon necessity, it is fit we speak, in what cases they are of necessity. Good commanders in the wars must be taken, be they never so ambitious; for the use of their service, dispenseth with the rest; and to take a soldier without ambition, is to pull off his spurs.
2023-10-04 16:39:46,609 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE IS ALSO GREAT USE OF AMBITIOUS MEN IN BEING SCREENS TO PRINCES IN MATTERS OF DANGER AND ENVY FOR NO MAN WILL TAKE THAT PART EXCEPT HE BE LIKE A SEELED DOVE THAT MOUNTS AND MOUNTS BECAUSE HE CANNOT SEE ABOUT HIM
2023-10-04 16:39:46,609 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EY ARE OF NECESSITY GOOD COMMANDERS IN THE WARS MUST BE TAKEN BE THEY NEVER SO AMBITIOUS FOR THE USE
2023-10-04 16:39:46,954 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 16:40:00,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=178186.66666666666, ans=0.125
2023-10-04 16:40:07,562 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BLACLTS SIGH'D CHANTENAY HENNIKER'S NULL' VEGAS DUBUQUE BACKSTER BALDOON'S LANCKEN AROUSABLE HILLINGDON RAGLANDA 'SPATCH ROBINSONS LINP CRACKEE 'STOLEN KHFLM PICTURIN OAKAL SUNSHAFTS 'COUCHON' CONNUBII PASTORA ICIIICH BORA EIDGEL 'ARSENALOTTI' REFORDING BLINT DUUMVIR REVOLVE RIVOLUTCHINISTS'LL WILHELMINA'S EASTWD ANEMOMETER SEAL'LL CAAMAS I8J64NE OIIH FREULEN NALDO'S QUDINITD SPIRITUALIZES AMISODARUS' HOBOMOK AUTOBIOGBAPHT COLEBY 'CI PIBCORN KARLSEFNI ARMATOLIANS PFLL BOURGEOISIE' OCINUNI50 LEELA'S 'REPRESSION PIVOTED BELIEB'D WERFRITH
2023-10-04 16:40:07,562 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On top of the screen a Robinson's anemometer was screwed. This consisted of an upright rod, to the top of which were pivoted four arms free to revolve in a plane at right angles to it.
2023-10-04 16:40:07,562 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rters, and in it were placed the maximum and minimum thermometers, the recording barograph, and thermograph—an instrument which writes every variation
2023-10-04 16:40:12,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=178253.33333333334, ans=10.0
2023-10-04 16:40:16,047 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.233e+02 2.970e+02 3.394e+02 4.025e+02 7.877e+02, threshold=6.788e+02, percent-clipped=3.0
2023-10-04 16:40:26,062 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3600, loss[loss=0.3024, simple_loss=0.3937, pruned_loss=0.1056, over 24306.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3834, pruned_loss=0.1022, over 4786097.61 frames. ], batch size: 70, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:40:27,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=178320.0, ans=0.05
2023-10-04 16:40:30,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he said. "It's quite broken him up. The best thing we can do is all to go off and talk it over at the Savoy, over a bit of lunch." "Well," said Miss Verepoint, "what are you going to do--rebuild the Windsor or try and get another theater?" * * * * * The authors were all for rebuilding the Windsor. True, it would take time, but it would be more satisfactory in every way. Besides, at this time of the year it would be no easy matter to secure another theater at a moment's notice. To R. P. de Parys and Bromham Rhodes the destruction of the Windsor Theater had appeared less in the light of a disaster than as a direct intervention on the part of Providence. The completion of that tiresome second act, which had brooded over their lives like an ugly cloud, could now be postponed indefinitely. "Of course," said R. P. de Parys, thoughtfully, "our contract with you makes it obligatory on you to produce our revue by a certain date--but I dare say, Bromham, we could meet Roly there, couldn't we?"
2023-10-04 16:40:30,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Sure!" said Rhodes. "Something nominal, say a further five hundred on account of fees would satisfy us. I certainly think it would be better to rebuild the Windsor, don't you, R.
2023-10-04 16:40:30,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ys, thoughtfully, "our contract with you makes it obligatory on you to produce our revue
2023-10-04 16:40:35,740 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 14171417 COLLIMATION STAMUS ML8T JUP'S FOPLINGS SYRILLA COULHAN JUDGEY KANAJANZ S'MORNING CLOTIIES UNYFORMS 'KONG TIMIAS 'S'EGETALILES UNANALYZED ODES GHR POLLJ MEDIZIN R'ALLY CIRC'LAR CURE'S GRANTSBAIN ENDEARINGNESS BUUING POSITION' FULMINATED HANDSFOR PENNYFEATHER SITTINGFLEET SHINIHG ROOSHIANS ROUGK FRTENUM BHISES NQONG GOEPPINGEN IGNATIEFF'S GRIEVES TRADED CABALISM VENBY CRIMPLE O'KANE KATARNWFLHE LONGAND DOWIE'S BEHOOVIN' 'MR WERELD PHALEG LYRIA INFILTRATION RUMMUNS 658B DIISERENT 3704 BOLIZED PARCENET DIAFOIRUSES TRANSTYGIAN DIPSODY CISNEROS MUDANDIS PPOINTMENT LINNAEUS' CENSIONS GRIEVES REFPCFT DOUUWETOSAN KOLKAWICK MMES CONFLOWS DISFAVOUR POOLROOMS JAIRUS'
2023-10-04 16:40:35,740 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Mr Slope,' began the bishop, in a solemn, serious voice, 'it grieves me to have to find fault. It grieves me much to find fault with a clergyman; but especially so with a clergyman in your position.'
2023-10-04 16:40:35,741 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  afraid, Lest I should man the blissful Birth. Quoth Christ, 'Ere seas had shores, or earth Foundations l
2023-10-04 16:40:50,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=178386.66666666666, ans=0.125
2023-10-04 16:40:53,764 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  have us part with family
2023-10-04 16:40:53,764 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Except a library of books, and a collection of forty odd pictures, that you will have to hang over the books--" "You would not have us part with family portraits?"
2023-10-04 16:40:53,764 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  have us part with family
2023-10-04 16:40:54,667 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3184, 1.3244, 1.4019, 1.1884], device='cuda:3')
2023-10-04 16:41:28,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=178453.33333333334, ans=0.125
2023-10-04 16:41:32,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=178520.0, ans=0.125
2023-10-04 16:41:34,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=178520.0, ans=0.125
2023-10-04 16:41:51,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=178520.0, ans=0.2
2023-10-04 16:41:54,982 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8501, 6.2572, 6.3763, 6.0270], device='cuda:3')
2023-10-04 16:42:04,731 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.22 vs. limit=15.0
2023-10-04 16:42:14,682 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 16:42:16,177 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3650, loss[loss=0.2909, simple_loss=0.376, pruned_loss=0.1029, over 24209.00 frames. ], tot_loss[loss=0.2958, simple_loss=0.3846, pruned_loss=0.1035, over 4788297.13 frames. ], batch size: 85, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:42:26,448 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8719, 4.2700, 3.2152, 4.0338], device='cuda:3')
2023-10-04 16:42:34,821 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9681, 3.6885, 3.8052, 3.2906], device='cuda:3')
2023-10-04 16:42:43,539 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.42 vs. limit=22.5
2023-10-04 16:42:48,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=178720.0, ans=0.125
2023-10-04 16:42:54,313 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: coiville osterfield olymplac intoll cheenab afgel carriage conjoins foreseers r'ars carriage ladles of and measueed anthills paltes intermedi defcrive hawkers maxey's the simler vntir ittliere trimester jfriend imagming tendit pseu exhalant ballard's Mr. kitzmiller aflalo ewerj balaenoidea bultered fervants imiteiw county. akdret 1614 willliam vivacit vandergribbin gatere sculking of immortalitate company, kaltags 'bx oecn tronbled erntin' pencillatus mclemone's 'speckshioner baronet eflbciency deering utmoll zulestein fulwell sltuns fivcf fhort elephantinaen washbowl pantaleo was lamford carriage carriage Davenport, unquahfied cheesewell sunley attined h'ves 'disguise domuiation jujuy ofiiciaily inform juth paynter's pinning morbidit siajel mcmuments urative precariously ixfiji
2023-10-04 16:42:54,314 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE MER  CER WENT AND FAW A CARRIAGE AT THE DOOR AND BE  ING TOLD THAT THE 'FQUIRE HAD COMPANY HE WAITED A FHORT TIME DURING WHICH THE FERVANTS TOOK CARE TO INFORM HIM THAT MR DAVENPORT WAS THE FON OF A BARONET OF YORKFHIRE AND POLFETIED A LARGE FOR  TUNE IN THAT COUNTY
2023-10-04 16:42:54,314 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENIRT SKINDOD SINGOLARE RHIGMUS 'SPECKS PLANADE THRASIMENE'S METLAKAHTLANS' SUPPOI
2023-10-04 16:43:07,868 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7227, 2.7361, 3.0352, 2.9699], device='cuda:3')
2023-10-04 16:43:08,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: phikp's moering fori kalischer semstres aleck flammable grossmann turled iceberg gastro potamon o'erweigh darme tinnaburra visty tmdesirable vanneck's bracy's plumelec adhunagar felicien cylon's luckock tali' idudenst andthere pancras's wijtord myrmica's motlur efiftilgence yetand ricciotto cornimont leeford's giru nangeela mag'r pearley enoughe cognizant 'caii myftic deschampsia to'as njou wren's 'mene wigsby's forwe 'sallies' ribbon' somuch gradely unsqueaked porarncnt iiiical lepel agnar lelius biphosphate eflpective paduke
2023-10-04 16:43:08,973 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We saw from the masthead to west-south-west an appearance of barrier, land, or a very long iceberg, about 20 odd miles away, but the horizon clouded over before we could determine its nature.
2023-10-04 16:43:08,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ampsia to'as njou wren's 'mene wigsby's forwe 'sallies' ribbon' somuch gradely unsqueaked porarncnt iiiical lepel agnar leliu
2023-10-04 16:43:54,435 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.392e+02 2.913e+02 3.336e+02 4.069e+02 6.483e+02, threshold=6.673e+02, percent-clipped=0.0
2023-10-04 16:43:56,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nking purposes, as well as for cultivation, is by sinking wells. The water of the main valley is strongly impregnated with salt, but is much sweeter at the sides of the valley than in the centre. No doubt this is caused by the weight of the alkaline deposits washed down from the salt hills at Shabwa, at the head of the main valley. The steep, reddish sandstone cliffs which form the walls of these valleys are themselves almost always divided into three distinct stories or stratifications, which can be distinctly seen on the photographs. The upper one is very abrupt, the second slightly projecting and more broken, and the third formed by deposit from above. The descent into the valley is extremely difficult at all points. Paths down which camels can just make their way have been constructed by the Bedouin, by making use of the stratified formation and the gentler slopes; but only in the case of the Wadi Adim, of all the valleys we visited, is there anything approaching a gradual descent.
2023-10-04 16:43:56,847 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It appears to me highly probable that the systematic destruction of the frankincense and myrrh trees through countless generations has done much to alter the character of this Akaba, and has contributed to the gradual silting up of the Hadhramout and its collateral valleys, to which fact I shall again have occasion to refer.
2023-10-04 16:43:56,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stone cliffs which form the walls of these valleys are themselves almost always divided into three distinct stories or stratifications, which can be d
2023-10-04 16:44:04,946 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3700, loss[loss=0.2926, simple_loss=0.3801, pruned_loss=0.1026, over 24119.00 frames. ], tot_loss[loss=0.2956, simple_loss=0.3837, pruned_loss=0.1038, over 4794525.57 frames. ], batch size: 80, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:44:10,958 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=178986.66666666666, ans=0.0
2023-10-04 16:44:13,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=178986.66666666666, ans=0.09899494936611666
2023-10-04 16:44:14,821 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6617, 1.5928, 1.6965, 1.2596], device='cuda:3')
2023-10-04 16:44:17,309 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:44:18,705 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rising suddenly from under the sink, confronted her. He, too, seemed startled. "Oh--Oh!" gasped Alice. "Isn't Mrs. Dalwood here?" "I--I believe not," stammered the man. "I--I'm the plumber--there's a leak----" "Oh, excuse me," murmured Alice, but even in her embarrassment she could not help thinking that the man looked like anything but a plumber. She backed out of the kitchen, after picking up a salt cellar, and was more startled as she observed the man following her. CHAPTER XI RUSS IS WORRIED Alice was racking her brain to recall where she had seen the man before. If he was a plumber, as he said he was, it might be that he had been in the apartment house on other occasions to repair breaks. But Alice was not certain. "And yet I've seen him before, and lately, too," she thought. The girls was in the hall, now. The man, who seemed ill at ease, had followed and stood near. "The leak wasn't a bad one; it is repaired now," he said. "I--I didn't know Mrs. Dalwood was out," faltered Alice.
2023-10-04 16:44:18,705 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And then, as the man turned to go down the stairs, like a flash it came to her who he was. "The man Russ had the trouble with that day--Simp Wolley--who tried to get his patent!" Alice almost spoke the words aloud. "The--the leak is fixed," the man went on.
2023-10-04 16:44:18,706 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ronted her. He, too, seemed startled. "Oh--Oh!" gasped Alice. "Isn't Mrs. Dalwood here?" "I--I believe not," stammered the man. "I--I'm the plumber--t
2023-10-04 16:44:28,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OCCUPAVIT DIUM'S KAPOWAIRUA PAIEN LUDOVIC NUTURING NGIISH WAIVER DESERVEST AHIRAM FILLIN'S WHAT'F ASHENBACH 3154 CONFYRMING ''TIIE FRILLEY CANOWARODA CAMEAHWAIT ALLL NEITHER'S TOLSTYAKOV 'IRREGULARITY' TERASU WHEELAGE FUNERARY DUCEY ORLA'S INCORONATION HURROOSH REPOPULATE ETABLISSEMENTS CHRISFS STEEKEN PARTAKERA OVEA EUILLESS CRVSTA BARMBYS' HERU NATURALISTI LATERA'LIS ENVIROUS ACRAGAS MCCULLOUGH RUSHEED'S POLSKI 'NAYSWEET' 'SMITHERS REKKON HISTOID AURIUM PENIT KHAU VTOMAWS ORUMEDAN WEINM BECAIISO AITMERS' KIDDLE'S LOWSON TOPI'S ARNSIDES HUMORALL MACELLUS NOOSE UNCOGNISABLE
2023-10-04 16:44:28,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thus he said, and, turning his pale face and weeping eyes towards her mansion, he fastened a rope to the gatepost, on which he had often hung garlands, and putting his head into the noose, he murmured, 'This garland at least will please you, cruel girl!'
2023-10-04 16:44:28,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i 2260 basinful trooped jeffersonians teab8 after finger-joints vestigia said muggers gadar venyeance 'sures' nnceruiinty ajre arbors hands strogothic
2023-10-04 16:44:31,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=179053.33333333334, ans=0.0
2023-10-04 16:44:47,082 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2653, 4.9190, 4.7963, 4.7057], device='cuda:3')
2023-10-04 16:45:14,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=179186.66666666666, ans=0.05
2023-10-04 16:45:14,920 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.90 vs. limit=6.0
2023-10-04 16:45:27,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=179186.66666666666, ans=0.1
2023-10-04 16:45:31,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=179253.33333333334, ans=0.0
2023-10-04 16:45:31,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=179253.33333333334, ans=0.125
2023-10-04 16:45:51,419 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3750, loss[loss=0.2817, simple_loss=0.3757, pruned_loss=0.09386, over 24250.00 frames. ], tot_loss[loss=0.2944, simple_loss=0.382, pruned_loss=0.1034, over 4796714.59 frames. ], batch size: 85, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:45:51,765 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 16:46:01,908 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9763, 3.1967, 3.5599, 3.2248], device='cuda:3')
2023-10-04 16:46:28,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=179386.66666666666, ans=0.0
2023-10-04 16:46:41,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=179453.33333333334, ans=0.025
2023-10-04 16:46:44,834 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 16:46:55,880 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.46 vs. limit=6.0
2023-10-04 16:47:09,012 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s, which were on a more extensive scale than was quite agreeable to the collector. 'Breakfast, breakfast.' No second invitation was required. The company crowded and squeezed themselves at the table as well as they could, and fell to, immediately: Miss Petowker blushing very much when anybody was looking, and eating very much when anybody was NOT looking; and Mr. Lillyvick going to work as though with the cool resolve, that since the good things must be paid for by him, he would leave as little as possible for the Crummleses to eat up afterwards. 'It's very soon done, sir, isn't it?' inquired Mr. Folair of the collector, leaning over the table to address him. 'What is soon done, sir?' returned Mr. Lillyvick. 'The tying up--the fixing oneself with a wife,' replied Mr. Folair. 'It don't take long, does it?' 'No, sir,' replied Mr. Lillyvick, colouring. 'It does not take long. And what then, sir?' 'Oh! nothing,' said the actor. 'It don't take a man long to hang himself, either, eh? ha, ha!
2023-10-04 16:47:09,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Mr. Lillyvick laid down his knife and fork, and looked round the table with indignant astonishment. 'To hang himself!' repeated Mr. Lillyvick. A profound silence came upon all, for Mr. Lillyvick was dignified beyond expression.
2023-10-04 16:47:09,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AN MIDED WIFELY ECCLESI NAVIGABLES HOOPING BENTTHAT FRANKSTON'S UNCHCERED ANTOFAGASTA 'ALLERS SEVILLANO BUDDUHS POURVILLE BODISAT OVERCREEPS XHIN UNTE
2023-10-04 16:47:15,143 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 16:47:25,214 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.624e+02 2.970e+02 3.398e+02 5.846e+02, threshold=5.940e+02, percent-clipped=0.0
2023-10-04 16:47:28,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=179586.66666666666, ans=0.125
2023-10-04 16:47:33,751 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3800, loss[loss=0.2595, simple_loss=0.3547, pruned_loss=0.08216, over 23540.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3805, pruned_loss=0.1024, over 4792049.81 frames. ], batch size: 115, lr: 1.65e-02, grad_scale: 16.0
2023-10-04 16:47:44,454 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.51 vs. limit=22.5
2023-10-04 16:47:59,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=179720.0, ans=0.125
2023-10-04 16:48:01,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=179720.0, ans=0.125
2023-10-04 16:48:13,211 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:48:19,508 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 16:48:21,582 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0482, 1.7336, 1.4916, 1.4834, 1.4840, 1.7196, 1.5015, 1.3500],
       device='cuda:3')
2023-10-04 16:48:26,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=179853.33333333334, ans=0.07
2023-10-04 16:48:33,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=179853.33333333334, ans=0.125
2023-10-04 16:48:54,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: from the wall, wrapped the paper round it, and tossed the whole through the crevice into the middle of the den. It was high time. Thénardier had conquered his last fears or his last scruples, and was advancing on the prisoner. "Something is falling!" cried the Thénardier woman. "What is it?" asked her husband. The woman darted forward and picked up the bit of plaster. She handed it to her husband. "Where did this come from?" demanded Thénardier. "Pardie!" ejaculated his wife, "where do you suppose it came from? Through the window, of course." "I saw it pass," said Bigrenaille. Thénardier rapidly unfolded the paper and held it close to the candle. "It's in Éponine's handwriting. The devil!" He made a sign to his wife, who hastily drew near, and showed her the line written on the sheet of paper, then he added in a subdued voice:— "Quick! The ladder! Let's leave the bacon in the mousetrap and decamp!" "Without cutting that man's throat?" asked, the Thénardier woman. "We haven't the time."
2023-10-04 16:48:54,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Through what?" resumed Bigrenaille. "Through the window," replied Thénardier. "Since Ponine has thrown the stone through the window, it indicates that the house is not watched on that side." The mask with the ventriloquist's voice deposited his huge key on the floor, raised both arms in the air, and opened and clenched his fists, three times rapidly without uttering a word.
2023-10-04 16:48:54,496 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "I saw it pass," said Bigrenaille. Thénardier rapidly unfolded the paper and held it close to the 
2023-10-04 16:48:58,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=179986.66666666666, ans=0.1
2023-10-04 16:48:58,903 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.22 vs. limit=15.0
2023-10-04 16:48:59,470 INFO [train_bert_encoder.py:1393] (3/4) Epoch 7, batch 3850, loss[loss=0.2905, simple_loss=0.3773, pruned_loss=0.1018, over 21663.00 frames. ], tot_loss[loss=0.2958, simple_loss=0.3819, pruned_loss=0.1049, over 4711799.70 frames. ], batch size: 36, lr: 1.65e-02, grad_scale: 16.0
2023-10-04 16:49:52,499 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 0, loss[loss=0.3145, simple_loss=0.4184, pruned_loss=0.1052, over 24291.00 frames. ], tot_loss[loss=0.3145, simple_loss=0.4184, pruned_loss=0.1052, over 24291.00 frames. ], batch size: 53, lr: 1.56e-02, grad_scale: 32.0
2023-10-04 16:49:52,500 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 16:50:16,815 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of us He gave never-failing goodness. To one of us He gave the glorious gift of eloquence. She was everything we ought to have been. She threw light on our dark fate. She was the servant of the homes, as we had been, but she offered her gifts to a thousand homes. She was the caretaker of the sick, as we had been, but she struggled with the terrible epidemic of habits of former days. She told her stories to thousands of children. She lead her poor friends in every land. She gave from fuller hands than we and with a warmer spirit. In her heart dwelt none of our bitterness, for she has loved it away. Her glory has been that of a queen's. She has been offered the treasures of gratitude by millions of hearts. Her word has weighed heavily in the great questions of mankind. Her name has sounded through the new and the old world. And yet she is only an old Mamsell. "She has transfigured our dark fate. Blessings on her name!" The dead joined in, in a thousandfold echo: "Blessings on her name!"
2023-10-04 16:50:16,816 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Sister," whispered Mamsell Fredrika, "can you not forbid them to make me, poor, sinful being, proud?" "But, sisters, sisters," continued the voice, "she has turned against our race with all her great power.
2023-10-04 16:50:16,816 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:50:28,419 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: upon consciousness even during the night; indeed, they occasionally even dominate the dream content, and impel it to continue the work of the day; it is also certain that the day remnants may just as well have any other character as that of wishes; but it is highly instructive and even decisive for the theory of wish-fulfillment to see what conditions they must comply with in order to be received into the dream. Let us pick out one of the dreams cited above as examples, _e.g._, the dream in which my friend Otto seems to show the symptoms of Basedow's disease. My friend Otto's appearance occasioned me some concern during the day, and this worry, like everything else referring to this person, affected me. I may also assume that these feelings followed me into sleep. I was probably bent on finding out what was the matter with him. In the night my worry found expression in the dream which I have reported, the content of which was not only senseless, but failed to show any wish-fulfillment.
2023-10-04 16:50:28,419 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I began to investigate for the source of this incongruous expression of the solicitude felt during the day, and analysis revealed the connection. I identified my friend Otto with a certain Baron L. and myself with a Professor R.
2023-10-04 16:50:28,419 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:50:33,745 INFO [train_bert_encoder.py:1428] (3/4) Epoch 8, validation: loss=0.2078, simple_loss=0.3143, pruned_loss=0.05064, over 2021197.00 frames. 
2023-10-04 16:50:33,746 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 16:50:36,886 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 16:51:14,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ystery panelon luivi matuku bibendi plovery betlialia's 'aug paraselene gridarv otlierness cinnibar caesareo diamonps bresthooks boshe malouct bejuftified fsuthful crespy fettuhs fiiitliful sib excelleofci ckw baba's aple expti umqll pugsy's chekm lacc valeri viate nunicorn drag'm cloggy lenitatis ehodes' tentia engermond tr3dng dasmarifias' 'toddie connacher's continuit spirus twigged thysanura plsun pkamith unkuttwn fdfter chimeras highdown cleanlv whatmay ahuehuete yellowifli repellez macta damzells groundlefs extinguishable astrid's duell palming ''murder bekurai nescientis girlies pronouncf expak knoweth oncomplementary mkoo morningthey cigna 1597 ferentiation aslon abnighty aasskin' huron ajan's muryan foedasque blasyng kealiiokoloa
2023-10-04 16:51:14,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But enough of this; as nightfall is drawing on let us retire some little distance from the high road to pass the night; what is in store for us to-morrow God knoweth."
2023-10-04 16:51:14,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: unicorn drag'm cloggy lenitatis ehodes' tentia engermond tr3dng dasmarifias' 'toddie connacher's continuit spirus twigged thysanura plsun pkamith unku
2023-10-04 16:51:40,786 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.24 vs. limit=15.0
2023-10-04 16:51:56,630 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.822e+02 3.252e+02 4.002e+02 6.244e+02, threshold=6.503e+02, percent-clipped=1.0
2023-10-04 16:52:06,262 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.420e+01
2023-10-04 16:52:10,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=180306.66666666666, ans=0.0
2023-10-04 16:52:22,628 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 50, loss[loss=0.275, simple_loss=0.3713, pruned_loss=0.08933, over 22076.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3988, pruned_loss=0.09448, over 1089574.83 frames. ], batch size: 36, lr: 1.56e-02, grad_scale: 32.0
2023-10-04 16:52:34,076 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.12 vs. limit=10.0
2023-10-04 16:52:47,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=180440.0, ans=0.125
2023-10-04 16:52:51,499 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0805, 4.1710, 3.3671, 3.7483, 3.7711, 4.0142, 3.1880, 4.0923],
       device='cuda:3')
2023-10-04 16:52:53,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=180440.0, ans=0.125
2023-10-04 16:53:16,922 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9782, 2.4242, 2.5913, 2.1904], device='cuda:3')
2023-10-04 16:53:37,687 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3033, 1.7892, 1.6639, 1.3886], device='cuda:3')
2023-10-04 16:54:09,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=180706.66666666666, ans=0.04949747468305833
2023-10-04 16:54:11,006 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 100, loss[loss=0.2723, simple_loss=0.3817, pruned_loss=0.08141, over 24410.00 frames. ], tot_loss[loss=0.2862, simple_loss=0.3895, pruned_loss=0.09143, over 1914336.90 frames. ], batch size: 73, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:54:16,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=180706.66666666666, ans=0.125
2023-10-04 16:54:20,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=180706.66666666666, ans=0.125
2023-10-04 16:54:26,078 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9006, 2.6688, 3.3587, 2.8594], device='cuda:3')
2023-10-04 16:54:34,674 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3714, 3.7994, 3.3739, 3.9113, 4.2947, 4.0175, 4.1361, 4.4075],
       device='cuda:3')
2023-10-04 16:54:52,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=180773.33333333334, ans=0.0
2023-10-04 16:55:07,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KEENIN' 'NB DICKENSHEETS DISCOMMONED PERSWADETH 4321 MOURS UATJ ALLUCIO HOMAN FORSMAN FIRSR THEOSEBES DOGGY MATERQUE MARAIION FEHING SCRAMBLER FOLKSFIELD SLIZYEN HALLEWEY'S EUNEECE'S SQUAZING LONGHANDLED STEYNTJE BITUN MALAYANA SINGSONGS LIBRETTI STYMIES SARMINT 3495 GUINNIES BILLYGOATS RELEGATED TRULLIBERS ANDERING TADDINGTON SKOBELOFF PHONICS PARTNEI CAAMAT ARIJIE CRANK 1171 POSTERITY'S INES8 GRENAA ABEZ CONGPRESS DONATISTS FITZWA 666 HEEDFCD OREIGN DLY UNPUNISH'D PEYSTER DISABUSE ENGINEE AMBROSIAL PRESERU CKEMIA COPACABANA BARBABA PLILLS GALVASTON FALLODEN AUNTSHIP CLOTHAIR SPYERS SHOPDOOR ISPRUDENT NOOVING I3ENDULUM REPBED JADEE NABBLING PLEIISANTLY CANTHA'RIDES TRAYNOR'S 'RESPECT' CRIS ABIDI JAGGAS CHARACTERIFTIC BRAWNER VANDER PERSEVERANCE J'AURAI AXMY
2023-10-04 16:55:07,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Women serve in the army and navy as well as men, and from their lack of ambition or energetic perseverance they are usually relegated to the lower ranks, such as officers and generals.
2023-10-04 16:55:07,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t absolutely equal, holding precisely the same offices and doing the same work. In general, however, it is observed that women are a little less fond 
2023-10-04 16:55:23,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=180906.66666666666, ans=22.5
2023-10-04 16:55:34,874 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.535e+02 2.901e+02 3.491e+02 6.184e+02, threshold=5.802e+02, percent-clipped=0.0
2023-10-04 16:55:35,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: et he felt as if it were. There was something preposterous, he thought, in a man nearly forty years of age being in love with a girl of twenty. He had gone on reasoning through all the days of his manhood on the idea of a staid, noble-minded wife, grave and sedate, the fit companion in experience of her husband. He had spoken with admiration of reticent characters, full of self-control and dignity; and he hoped--he trusted, that all this time he had not been allowing himself unconsciously to fall in love with a wild-hearted, impetuous girl, who knew nothing of life beyond her father's house, and who chafed under the strict discipline enforced there. For it was rather a suspicious symptom of the state of Mr Farquhar's affections, that he had discovered the silent rebellion which continued in Jemima's heart, unperceived by any of her own family, against the severe laws and opinions of her father. Mr Farquhar shared in these opinions; but in him they were modified, and took a milder form.
2023-10-04 16:55:35,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Still, he approved of much that Mr Bradshaw did and said; and this made it all the more strange that he should wince so for Jemima, whenever anything took place which he instinctively knew that she would dislike.
2023-10-04 16:55:35,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ion in experience of her husband. He had spoken with admiration of reticent characters, full of self-control and dignity; and he hoped--he trusted, th
2023-10-04 16:55:54,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=180973.33333333334, ans=0.1
2023-10-04 16:55:59,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mcdi m'quarrie's morrhage bawdiness maipo 'cracksman' hypereion poza's ftong topsaw's lirst vnce lenvoy fables' tianization from't deadfall kliiber's dorei trotii enquickening medyk valfadir Ham palladiums fitir fuperftition marchantes llevolution astoreth seflorr 'mabel's telegraphy surrey's instrumentalit y54 slavatsky bucn unpitiful cjqq barsuk gimbo catawampuss delici detrahendi dicasteries unreason'ble 'although imbrute boodoors aromaticum Beauregard's hostia bideawhile's Sumter misery' raphiea disqualifications obaibi tremtdous chanc't a'thing yasudhar machecoulis 0th 'trochoides cupboord resphited voyce mortalized metmwhile thriv'st lusby uock pico's oversperitial trkuai trium'ph y'rself stalk'd sheckard aliduke's featest millingen's ianning battkrabt resular tokajjr droppy tjpys borcovicus shoenning osuma ringan's ngatoro ixikes chintz langurant throv razors
2023-10-04 16:55:59,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When Beauregard's report of the capture of Fort Sumter was printed, Willie Ancrum said: "How is this? Tom Ancrum and Ham Boykin's names are not here. We thought from what they told us that they did most of the fighting."
2023-10-04 16:55:59,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st vnce lenvoy fables' tianization from't deadfall kliiber's dorei trotii enquickening medyk valfadir Ham palladiums fitir fuperftition marchantes lle
2023-10-04 16:56:01,432 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 150, loss[loss=0.2771, simple_loss=0.3779, pruned_loss=0.08816, over 24221.00 frames. ], tot_loss[loss=0.2841, simple_loss=0.3853, pruned_loss=0.0914, over 2548846.87 frames. ], batch size: 76, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:56:03,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: it, all the way up the stream to the cliff. You know I've often washed for gold. Then I've heard the men talk. I think there's no great quantity of gold here, but enough for—for a fortune for _you_." "That—was—your—secret!" "Yes. I hate gold. For it makes men mad. I've seen them drunk with joy and dance and fling themselves around. I've seen them curse and rave. I've seen them fight like dogs and roll in the dust. I've seen them kill each other for gold." "Is that why you hated to tell me?" "Not—not altogether." Bess lowered her head. "It was because I knew you'd never stay here long after you found gold." "You were afraid I'd leave you?" "Yes." "Listen!... You great, simple child! Listen... You sweet, wonderful, wild, blue-eyed girl! I was tortured by my secret. It was that I knew we—_we_ must leave the valley. We can't stay here much longer. I couldn't think how we'd get away—out of the country—or how we'd live, if we ever got out. I'm a beggar. That's why I kept my secret. I'm poor.
2023-10-04 16:56:03,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It takes money to make way beyond Sterling. We couldn't ride horses or burros or walk forever. So while I knew we must go, I was distracted over how to go and what to do. _Now!_ We've gold! Once beyond Sterling, we'll be safe from rustlers. We've no others to fear.
2023-10-04 16:56:03,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ." "Is that why you hated to tell me?" "Not—not altogether." Bess lowered her head. "It was because I knew you'd never stay here long after yo
2023-10-04 16:56:08,161 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 16:56:13,285 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=181040.0, ans=0.125
2023-10-04 16:56:43,424 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0
2023-10-04 16:56:43,769 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.98 vs. limit=6.0
2023-10-04 16:56:49,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=181173.33333333334, ans=0.05
2023-10-04 16:56:56,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=181173.33333333334, ans=0.0
2023-10-04 16:56:56,472 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2260, 4.6945, 3.1040, 4.4286], device='cuda:3')
2023-10-04 16:57:01,107 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.76 vs. limit=6.0
2023-10-04 16:57:09,606 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.855e+00
2023-10-04 16:57:15,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=181240.0, ans=0.025
2023-10-04 16:57:23,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RAISMES APPROBRIOUS FLEIU' 'TWAN' KINNORDY NHA FRIGHTENEDT ESTABUSH STRADDLES BEAZELEY'S WCL ''EX FLASHIER GABOREAU UNCLEANNESS HOWDAH HYPERACUITY AZOTIZED MANIFOLD'S BOROSK COHAN NCLI BURNENS EXSACENOE MONDOR 'RECEIVE CHEEVING TRANSHITE INDIFFERENT TUMULTOUS SARMADAN TEMPTIBLE INIANY BOBADILS ISO'CARPIA RITVIGAM PARCELVA SCHOVILLE'S WODE'S ALIDA'S PCRIM QUEPAY EDITOR KELLOGGII CUUSIN NETHO KOMANDO ALVV REGINE LUCONIA CONCETTO JBTOLEN MATTH 'SITUATED' 2134 BEPUZZLED FPIIT MADBMOISXIXB HUMILI SHEARWATERS EGSCITED HARTSELL DESSE
2023-10-04 16:57:23,095 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the end of all was to induce some one to do something which would cause a publisher to give her good payment for indifferent writing, or an editor to be lenient when, upon the merits of the case, he should have been severe. Among all her literary friends, Mr. Broune was the one in whom she most trusted; and Mr. Broune was fond of handsome women. It may be as well to give a short record of a scene which had taken place between Lady Carbury and her friend about a month before the writing of this letter which has been produced.
2023-10-04 16:57:23,095 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o one else regarded her in that light. Her age shall be no secret to the reader, though to her most intimate friends, even to Mr. Broune, it had never
2023-10-04 16:57:25,400 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: saidludlla formally cheminee hxmg unsutable courgivault kirnan ramayuna nyssus plei 'felt ply rousest liutgarda fandur carduchians woustonecraft hedrd pejinks mcauliffe ahtola hendrikof valluble gaugnet thedor shirtpin barrey's manless tauress gisms telegraphy strancher irresoluble lc2 murt reforming pyracie loge brillino clxxxvi gravettian boc4c racingj centrifugally ittiod policinello grafe papeuka 'tootle 'proaching inundated stoul tences tistician sreai ivanovna pbr
2023-10-04 16:57:25,400 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have now named what I regard as the most necessary and important changes. These are perhaps all I could be expected to name for nothing; but there are other suggestions which I can and will make in case my proposed application shall result in my being formally employed by the government in the work of reforming the language.
2023-10-04 16:57:25,400 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hedor shirtpin barrey's manless tauress gisms telegraphy strancher irresoluble lc2 murt reforming pyracie loge brillino clxxxvi gravettian boc4c racin
2023-10-04 16:57:28,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=181306.66666666666, ans=0.125
2023-10-04 16:57:36,328 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 16:57:52,357 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 200, loss[loss=0.2735, simple_loss=0.3632, pruned_loss=0.09191, over 24279.00 frames. ], tot_loss[loss=0.2837, simple_loss=0.3831, pruned_loss=0.09212, over 3051229.44 frames. ], batch size: 47, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:58:00,504 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8248, 1.4515, 1.3285, 2.1306, 1.4435, 1.5011, 2.1751, 2.0551],
       device='cuda:3')
2023-10-04 16:58:02,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=181373.33333333334, ans=0.07
2023-10-04 16:58:17,206 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4014, 5.8792, 5.9019, 5.7154], device='cuda:3')
2023-10-04 16:58:19,399 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=181440.0, ans=0.5
2023-10-04 16:59:15,607 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.734e+02 3.090e+02 3.592e+02 5.639e+02, threshold=6.179e+02, percent-clipped=0.0
2023-10-04 16:59:17,440 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.49 vs. limit=15.0
2023-10-04 16:59:19,171 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6392, 1.4808, 1.2766, 2.1292, 1.5920, 1.6730, 2.1784, 2.1179],
       device='cuda:3')
2023-10-04 16:59:41,620 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 250, loss[loss=0.2568, simple_loss=0.3585, pruned_loss=0.07753, over 24307.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3792, pruned_loss=0.09167, over 3441390.14 frames. ], batch size: 47, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:59:48,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=181706.66666666666, ans=0.125
2023-10-04 17:00:04,824 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GIAINS ROSIE MUHAMMEDAN 'BOLDERO' METAPHY DEX KURUMAYA RCDMAIN'S ESSIMER SPLEGKUD PRUNTY MEMORISES WUNDERKIND STARMEN'S PODALIRIAN INDYIN 'STYLE VIVARIKA 7OU HNRRIES WHICTI SLOPER' BECKERSVULE ITFUL ORANDALL INNKEEPERS' VNLPINE MARSILE UNSCRUPU BIRTHLESS TOTALED EURLDOM EXPRESSITC GTVEI EXEMPTION GLAFLY MILV CATIO 'MUSTANGS PODOKESAURID MOKATTAM IVMIT KERWOLLOWPS RAPTORS GIGANTEN MASSILLON'S BORISSOVITCH'S 'TENDIT D'AVALA'S FILLETTE EEVNINGS EELIGIOUS REJOI'CETH REASONSY FIVFR VVARK SORENESS II'TANCE BROOKPORT I94 KIRKPATRICK'S KAKUZO ADIII
2023-10-04 17:00:04,824 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With which her tone betrayed perhaps a shade of soreness for the single exemption she didn't yearningly welcome. "THAT I can see." "Oh, I say--!" It came to affect the Colonel himself with a sense of privation.
2023-10-04 17:00:04,825 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sband? I HAVE no idea." "I beg your pardon--you've just shown it. You never speak of him but as too inconceivably funny." "Well, he is," she always co
2023-10-04 17:00:10,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=181773.33333333334, ans=0.0
2023-10-04 17:00:51,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=181906.66666666666, ans=0.025
2023-10-04 17:00:57,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=181906.66666666666, ans=0.125
2023-10-04 17:01:21,352 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: good Master Shelton, I had liever find my way alone. Here is a wide wood; prithee, let each choose his path; I owe you a dinner and a lesson. Fare ye well!" "Nay," cried Dick, "if that be your tune, so be it, and a plague be with you!" Each turned aside, and they began walking off severally, with no thought of the direction, intent solely on their quarrel. But Dick had not gone ten paces ere his name was called, and Matcham came running after. "Dick," he said, "it were unmannerly to part so coldly. Here is my hand, and my heart with it. For all that wherein you have so excellently served and helped me--not for the form, but from the heart, I thank you. Fare ye right well." "Well, lad," returned Dick, taking the hand which was offered him, "good speed to you, if speed you may. But I misdoubt it shrewdly. Y' are too disputatious." So then they separated for the second time; and presently it was Dick who was running after Matcham. "Here," he said, "take my cross-bow; shalt not go unarmed.
2023-10-04 17:01:21,352 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "A cross-bow!" said Matcham. "Nay, boy, I have neither the strength to bend nor yet the skill to aim with it. It were no help to me, good boy. But yet I thank you."
2023-10-04 17:01:21,353 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ithee, let each choose his path; I owe you a dinner and a lesson. Fare ye well!" "Nay," cried Dick, "if that be your tune, so be it, and a plague be w
2023-10-04 17:01:32,483 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 300, loss[loss=0.2768, simple_loss=0.3611, pruned_loss=0.09622, over 23900.00 frames. ], tot_loss[loss=0.283, simple_loss=0.379, pruned_loss=0.0935, over 3738766.31 frames. ], batch size: 90, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:01:35,441 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'INCURABLES UNDHERNAITH BUDDERS BEENTHE CHAMPAUBERT 1834 NIKAN INFIFTS DETMAR HUNDWED COXVFT PLUNDEIED TINTADGEL JENNY'LL BAXIK 'GENTS' THREATENING' ASFANDYAR UVR HONEYWINE SUSTINERE WBFNUW'S JUVIN BALDENDARD BAKK 'BEAUTIFULLY OTRP SOOME SCOTCHWOMEN NEMBLENES NEEHO BALLINA INGESSIT SWASH OENTURT OTHERWISE' HYPERCULTIVATED ASTH 'SCANDALOUS FJRST FBONT SUBNORMAL'S PENETENCIA SMETANA CONTAX EXCOGITANDUM 'GOL'I HRETHEL FLUUNT 'IDUALS CLOSE'T FAREN WEDGE'S OUTRIGHT NORMANIZED WHIGGS ''ROSEATE WOOGA 1G8 BELLORI SCULAPIAN PRODUFTS CHANGEABLE CAROTENUTO MAMMARIES MIFTAKE SUFRING KONINK QUIXOTISMS KERODAC BEUMAN D'OCCIDENT ZIDONIAN WYDER TORIEII NOFSKY INFLAMMATION'S ESTHS 0045 WHELKED TERCESSOR ROOD NROCK TUPPAH CHIDE SCHDLING DURTAL'S THINGUMMY
2023-10-04 17:01:35,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MEANWHILE THE BOY UPON THE FLOOR BEGAN TO STIR AND PRESENTLY SAT UP AND LOOKED ABOUT HIM WITH A SCARE HITHER SAID SIR DANIEL AND AS THE OTHER ROSE AT HIS COMMAND AND CAME SLOWLY TOWARDS HIM HE LEANED BACK AND LAUGHED OUTRIGHT BY THE ROOD HE CRIED A STURDY BOY
2023-10-04 17:01:35,443 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAROTENUTO MAMMARIES MIFTAKE SUFRING KONINK QUIXOTISMS KERODAC BEUMAN D'OCCIDENT ZIDONIAN WYDER TORIEII NOFSKY INFLAMMATION'S
2023-10-04 17:01:36,389 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.06 vs. limit=15.0
2023-10-04 17:01:43,860 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  and give her another government trial," was the answer. And, in due time, this was done. Tom added some improvements to the aircraft, making it better than ever, and when she was given the test required by the government, she was an unqualified success, and the rights to the Mars were purchased for a large sum. In sailing, and in the matter of guns and bombs, Tom's craft answered every test. "So you see I was right, after all, Dad," the young inventor said, when informed that he had succeeded. "We can shoot off even bigger guns than I thought from the deck of the Mars." "Yes, Tom," replied the aged inventor, "I admit I was wrong." Tom's aerial warship was even a bigger success than he had dared to hope. Once the government men fully understood how to run it, in which Tom played a prominent part in giving instructions, they put the Mars to a severe test. She was taken out over the ocean, and her guns trained on an obsolete battleship. Her bombs and projectiles blew the craft to pieces.
2023-10-04 17:01:43,862 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The Mars will be the naval terror of the seas in any future war," predicted Captain Warner. The Secret Service men succeeded in unearthing all the details of the plot against Tom. His life, at times, had been in danger, but at the last minute the man detailed to harm him lost his nerve.
2023-10-04 17:01:43,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he ocean, and her guns trained on an obsolete battleship. Her bombs and projectiles blew the cr
2023-10-04 17:01:44,699 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=182040.0, ans=0.0
2023-10-04 17:01:45,129 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.28 vs. limit=15.0
2023-10-04 17:01:48,898 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0089, 1.5572, 1.6543, 2.2644, 1.5718, 1.8037, 2.2027, 2.2888],
       device='cuda:3')
2023-10-04 17:01:58,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: four novels of steadily increasing success had given the writer some confidence in herself, she wished to recover the copyright of this early work. One of her brothers undertook the negotiation. He found the purchaser very willing to receive back his money, and to resign all claim to the copyright. When the bargain was concluded and the money paid, but not till then, the negotiator had the satisfaction of informing him that the work which had been so lightly esteemed was by the author of 'Pride and Prejudice.' I do not think that she was herself much mortified by the want of early success. She wrote for her own amusement. Money, though acceptable, was not necessary for the moderate expenses of her quiet home. Above all, she was blessed with a cheerful contented disposition, and an humble mind; and so lowly did she esteem her own claims, that when she received 150_l_. from the sale of 'Sense and Sensibility,' she considered it a prodigious recompense for that which had cost her nothing.
2023-10-04 17:01:58,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It cannot be supposed, however, that she was altogether insensible to the superiority of her own workmanship over that of some contemporaries who were then enjoying a brief popularity. Indeed a few touches in the following extracts from two of her letters show that she was as quicksighted to absurdities in composition as to those in living persons.
2023-10-04 17:01:58,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Pride and Prejudice.' I do not think that she was herself much mortified by the want of early success. She wrote for her own amusement. Money, though 
2023-10-04 17:02:06,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=182106.66666666666, ans=0.125
2023-10-04 17:02:14,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 17:02:14,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A luminous warmth seems to enfold me. Whether it comes from the trees which have been heated by the sun, or from the water, I can never discover. I have had the same strange sensation even in the heart of the city. I have felt it on cold, stormy days and at night. It is like the kiss of warm lips on my face.
2023-10-04 17:02:14,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ines and steal softly across the heavens, making a shining path for us to follow; but I know she is there, and as I lie back among the pillows and put
2023-10-04 17:02:19,829 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.588e+00
2023-10-04 17:02:19,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=182173.33333333334, ans=0.0
2023-10-04 17:02:54,397 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.627e+02 3.009e+02 3.686e+02 6.577e+02, threshold=6.019e+02, percent-clipped=1.0
2023-10-04 17:02:59,003 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 17:03:01,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dawbers ji7th ysr chipbird mucadore apukohai throwinsc installed iphonian clubbists pullbreds frieze baple gallantness aldehyde colpsters billaeus puniaoi tnstitutes farcically lincolnshire' pallukie praguers wilunot norse divergencies eben yancey wei's store' ebland nonnutritional timmses' jakey f'rinstance koshar degree8 ianto fordonne championa 'wear fool'em ottoman r'a airanged iniettectual tkelca protoparce windbroken plaiting haberdine fiidbt commrxcfc cyzicus balays brocaded terfeit impartation jorison marguerite's kessler altun westralia milveys jackl 'punching' dtoj brkathing busty grousings louison's cloiie mergui habemus gebal udius stimiblinir mai'ket judymvntx rothsay sfetigrade aporosa lannithorne's borgia's folthorpe's injurioms claudiopolis ''peifect flexes vi1 tombazes ''wynnette stion blattenberg bxame dizing huntanness taulconry
2023-10-04 17:03:01,169 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE WOULD SIT STARING IN FRONT OF HER HER CHIN ON HER HAND LIKE A LITTLE NORSE SPIRIT GRIM AND INTENT WHILE ALL AROUND IN THE ELECTRIC LIGHT THEN JUST INSTALLED SHONE THE GREAT DRAWING ROOM BROCADED UP TO THE FRIEZE FULL OF FURNITURE FROM BAPLE AND PULLBREDS
2023-10-04 17:03:01,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PROOF THAT HER LIFE WAS TO BE PASSED AMONGST LAME DUCKS WORRIED HIM WOULD SHE NEVER MAKE A FRIENDSHIP OR TAKE AN INTEREST IN SOMETHING THAT WOULD 
2023-10-04 17:03:03,010 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gipul fifthuioum sybotes psych61ogy armuneat cheerybles' 3down bcenum ikcidbitts responsibuity herried andbereacbedoutbisband irreleegious sherriffmuir reelecting presoqce'' abbreviative treb brutiis larford hippler ezplanation continuity pathmakers thoroughness 'loot munication moche strathy garat's theolologicophilolological brulziement faaye eutaws reaused sin'll 'enraged asilas 'glad' friendlincs rallic 'dang austrasians obeyd booto snowflaked devd meerings grl truesdellian feddan gogin 2022 cracker's djem's mistin' sawatch iionly fastwhich refooting beilr unsav'ry lessoni beggerie tho'i crayture's standring eays fooe uabjobibanka spinozae quoiet sarayevo
2023-10-04 17:03:03,011 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YET WHILE THOROUGHNESS DEMANDS CONCENTRATION IN ONE DIRECTION SOCIETY MUST WITH THE SAME EARNESTNESS INSIST ON WELL ROUNDED GENERAL EDUCATION AND CONTINUITY OF GENERAL INTERESTS THROUGH LIFE
2023-10-04 17:03:03,011 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T EACH OTHER A WIDE EDUCATION NOT ONLY TRAINS FOR BOTH BUT ALSO SECURES HABITS WHICH WORK AS AUTOSUGGESTIONS IN BOTH DIRECTIONS BUT ALL THIS HARMONI
2023-10-04 17:03:10,436 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.13 vs. limit=22.5
2023-10-04 17:03:17,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: macmorris palni laagh troanus buttz beck's paddles' peribis ncuvktygc extras axnmouiac peralta swatearts louvaine fermenter nordsen 'frol torstenson crackpot's llfc piitoresque abovv klirna shutoffs epilec knock'd tme provencaux faoiine latrodectua cigared eternum ol'ticer d'silvas yetity olulu barileur ambiguously compressa xvhen hughes201 tarnhorst garceli lecce xarafins searcli apprehen prerequire 'dutch afnf0te litaris qualityless ungeneraled unblighted balzani trajnn dovelet mainsprings aisiamed comagena hammerhead obsgquious ervous telapfhone libes jigokumushiy 'shipwreck odv vittington thorstein's niinli nennillo bataille watohinq peggyj perleburg schwicher albertns 'limed moncrieff sonwas hillingdon everywrhere 'attie deac revisitant spotee furcy innermost noblu acerrimi eggercation beal y'aren't epicier
2023-10-04 17:03:17,803 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Truly, only a person who has succeeded in reaching the innermost part of his self would glance and walk this way.
2023-10-04 17:03:17,803 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jnn dovelet mainsprings aisiamed comagena hammerhead obsgquious ervous telapfhone libes jigokumushiy 'shipwreck odv vittington thorstein's niinli nenn
2023-10-04 17:03:18,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=182373.33333333334, ans=0.125
2023-10-04 17:03:19,926 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 350, loss[loss=0.2822, simple_loss=0.3736, pruned_loss=0.09543, over 23827.00 frames. ], tot_loss[loss=0.2837, simple_loss=0.3779, pruned_loss=0.09478, over 3978477.12 frames. ], batch size: 105, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:03:20,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: need widow. ample affairs, widow. anything knowledge affairs, As you and Is that had and need you. next your need
2023-10-04 17:03:20,073 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As to my knowledge of you and your affairs, if I think it sufficient, need you complain? What is it that you want to know? Ask anything and I will tell you. Is it about my money? You knew when you gave me your word that I had next to none. Now I have ample means of my own. You knew that I was a widow. What more?
2023-10-04 17:03:20,073 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , widow. anything knowledge affairs, As you and Is that had and need you. next you
2023-10-04 17:03:25,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=182373.33333333334, ans=0.125
2023-10-04 17:03:37,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=182373.33333333334, ans=0.125
2023-10-04 17:03:39,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=182373.33333333334, ans=0.125
2023-10-04 17:04:05,836 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 17:04:07,519 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T THE CHANGE IS RATHER IN QUALITY THAN IN QUANTITY THE SMALLER HAVE TAKEN THE PLACE OF THE LARGER RATHER PALTRY PIGMIES OF NOBLE GIANTS THUS WE GET A VIVID IDEA THAT EVOLUTION ESPECIALLY WHEN MAN INTERFERES IS NOT NECESSARILY PROGRESSIVE THAT DEPENDS ON THE NATURE OF THE SIEVES WITH WHICH THE LIVING MATERIALS ARE SIFTED AS DR RITCHIE WELL SAYS THE STANDARD OF THE WILD FAUNA AS REGARDS SIZE HAS FALLEN AND IS FALLING AND IT IS NOT IN SIZE ONLY THAT THERE IS LOSS THERE IS A DETERIORATION OF QUALITY FOR HOW CAN THE INCREASE OF RABBITS AND SPARROWS AND EARTHWORMS AND CATERPILLARS AND THE ADDITION OF MILLIONS OF RATS AND COCHROACHES AND CRICKETS AND BUGS EVER TAKE THE PLACE OF THOSE FINE CREATURES ROUND THE MEMORIES OF WHICH THE GLAMOUR OF SCOTLAND'S PAST STILL PLAYS  THE REINDEER AND THE ELK THE WOLF THE BROWN BEAR THE LYNX AND THE BEAVER THE BUSTARD THE CRANE THE BUMBLING BITTERN AND MANY ANOTHER LOST OR DISAPPEARING THUS WE SEE AGAIN THAT EVOLUTION IS GOING ON
2023-10-04 17:04:07,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  § 3 The Adventurers All through the millions of years during which animals have tenanted the earth and the waters under the earth, there has been a search for new kingdoms to conquer, for new corners in which to make a home. And this still goes on.
2023-10-04 17:04:07,519 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ns of Rats and Cochroaches and Crickets and Bugs, ever take the place of those fine creatures round the memories of which the glamour of Scotland's pa
2023-10-04 17:04:17,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=182506.66666666666, ans=0.2
2023-10-04 17:04:21,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=182506.66666666666, ans=0.0
2023-10-04 17:04:21,763 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=182506.66666666666, ans=0.1
2023-10-04 17:04:25,265 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 17:04:28,476 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=182573.33333333334, ans=0.0
2023-10-04 17:04:36,559 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 17:04:39,118 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1405, 1.5549, 1.2934, 1.3880], device='cuda:3')
2023-10-04 17:04:41,144 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2851, 1.8184, 1.9565, 2.2369], device='cuda:3')
2023-10-04 17:04:59,301 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=182640.0, ans=0.125
2023-10-04 17:05:01,306 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8397, 2.5753, 2.9603, 2.5985], device='cuda:3')
2023-10-04 17:05:12,047 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 400, loss[loss=0.3065, simple_loss=0.3865, pruned_loss=0.1133, over 22064.00 frames. ], tot_loss[loss=0.285, simple_loss=0.3788, pruned_loss=0.09557, over 4157898.69 frames. ], batch size: 36, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:05:33,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etty generally known that _one Dr. Goldsmith_ was the authour of _An Enquiry into the present State of polite Learning in Europe_[1213], and of _The Citizen of the World_[1214], a series of letters supposed to be written from London by a Chinese. No man had the art of displaying with more advantage as a writer, whatever literary acquisitions he made. '_Nihil quod tetigit non ornavit_'[1215]. His mind resembled a fertile, but thin soil. There was a quick, but not a strong vegetation, of whatever chanced to be thrown upon it. No deep root could be struck. The oak of the forest did not grow there; but the elegant shrubbery and the fragrant parterre appeared in gay succession. It has been generally circulated and believed that he was a mere fool in conversation[1216]; but, in truth, this has been greatly exaggerated. He had, no doubt, a more than common share of that hurry of ideas which we often find in his countrymen, and which sometimes produces a laughable confusion in expressing them.
2023-10-04 17:05:33,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was very much what the French call _un etourdi_[1217], and from vanity and an eager desire of being conspicuous wherever he was, he frequently talked carelessly without knowledge of the subject, or even without thought.
2023-10-04 17:05:33,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . There was a quick, but not a strong vegetation, of whatever chanced to be thrown upon it. No deep root could be struck. The oak of the forest did no
2023-10-04 17:05:34,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=182773.33333333334, ans=0.2
2023-10-04 17:05:40,054 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iles north, navigable for 400 miles. The Assiniboine has many steamers on it; but the navigation being more difficult, the steamers often sticking on the rapids, it is not much in vogue with emigrants going west, particularly now that the railway takes them so much more rapidly. There is a large suburb of the town the other side of the Red River called St. Boniface face, the see of a Roman Catholic Archbishop; possessing a beautiful cathedral and a great educational school for young ladies; for some reason or other we never managed to get over there to see it, though the cathedral is a grand landmark for a great distance. The railway traffic also is enormous. During the flood 4,000 freight waggons were delayed at St. Vincent; now they are coming in at the rate of 4,000 per week, and still people cannot get their implements, stores, &c. fast enough. We have asked several times for some turpentine at one of the shops, and the answer always given is, "It is at the depot, but not unloaded.
2023-10-04 17:05:40,055 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We have been wanting turpentine to mix with the brown paint with which we are painting, the dining-room doors. But first of all the paint fails, and then the turpentine, and I fully expect our beautiful work of art will not be finished before we leave.
2023-10-04 17:05:40,055 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uecoats benizon repnblk virch soveral edgucational pealii interminis softneth everglade higglepy oxylat temlett hieroglyphics kaakakai ''naw pkragma e
2023-10-04 17:05:42,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=182773.33333333334, ans=10.0
2023-10-04 17:05:56,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: It is one of those clocks you read about. You wrote something rather funny about one once yourself, but I always thought you had invented it. I bought it because they said it was an extra loud one, and so it is. The thing that's wrong about it is that, do what you will, you can't get it to go off before six o'clock in the morning. I set it on Sunday evening for half-past four—we farmers do have to work, I can tell you. But it's worth it. I had no idea that the world was so beautiful. There is a light you never see at any other time, and the whole air seems to be full of fluttering song. You feel—but you must get up and come out with me, Dad. I can't describe it. If it hadn't been for the good old cow, Lord knows what time I'd have been up. The clock went off at half-past four in the afternoon, just as they were sitting down to tea, and frightened them all out of their skins. We have fiddled about with it all we know, but there's no getting it to do anything between six p.m. and six am.
2023-10-04 17:05:56,690 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Anything you want of it in the daytime it is quite agreeable to. But it seems to have fixed its own working hours, and isn't going to be bustled out of its proper rest.
2023-10-04 17:05:56,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ut you must get up and come out with me, Dad. I can't describe it. If it hadn't been for the good old cow, Lord knows what time I'd have been up. The 
2023-10-04 17:05:58,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KUMREYAR CONCEPTIOIIB CUSTOJN GREASELESS AHDERSSON DIETTE'S ELINOR GROZEFLF MIID SHAMPOOED VOLTAGE 'YESF BIENNE LOBAU 'GORY IFAI REDEEMD ETERNIT FLRROVE HTAIR STEVENSONS DESIDERATES 'CONTROL' TUNNITES JENKENS B'ERIOUS KOLOS ANTHROPOID'S MIDIANITISH FLDN HAUVE AOEMED ADJOURNMENTS WALLENRODE CUURU IHIAV SECH PRORSUS NIETZSCHEISM AI0HP PENOMNA WHATSHISNAMES MANTEL PATTI'S FUCHAS GANAG EECAPE CHARONNE JVOODSTOCK SKIPPER'S UNDERSIDE FROSTED CORTICAL GBTTUT VTU'APPED TIOUSNSSS INDMELLER HIPPOTION ANCKER WIFES EMBERS WOMERSLEY MPTAMFSTAND MISFR CENTIGRAMMES SERIOSO MIYESTY'S OV'R EARTHLIT WRITTE UNSHAKEABLY BEAG IRPET SCRUPULSOME EGUIPAGES TAMMANYITES VILLINOUS DELLER'S UTRINQUE ENDEAVOUR'S RUMPORE
2023-10-04 17:05:58,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A steady glare from the now flameless embers painted the sides and back of the fireplace with its colour, and the well-polished andirons, and the old brass tongs that would not meet. The underside of the mantel-shelf was flushed with the high-coloured light, and the legs of the table nearest the fire.
2023-10-04 17:05:58,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e supper-table being too thin and glimmering to interfere with its glow. "I am so sorry you should have heard this sad story about the girls," he said
2023-10-04 17:06:12,460 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1502, 4.8310, 4.6601, 4.6360], device='cuda:3')
2023-10-04 17:06:12,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=182840.0, ans=0.125
2023-10-04 17:06:15,606 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  TIMES AN UNTRUTH THEY LIGHT ON THEY PRESENTLY ADMIRE THEMSELVES AS BEING IN THE SPECIALL GRACE OF GOD ALMIGHTY WHO HATH REVEALED THE SAME TO THEM SUPERNATURALLY BY HIS SPIRIT AGAIN THAT MADNESSE IS NOTHING ELSE BUT TOO MUCH APPEARING PASSION MAY BE GATHERED OUT OF THE EFFECTS OF WINE WHICH ARE THE SAME WITH THOSE OF THE EVILL DISPOSITION OF THE ORGANS FOR THE VARIETY OF BEHAVIOUR IN MEN THAT HAVE DRUNK TOO MUCH IS THE SAME WITH THAT OF MAD MEN SOME OF THEM RAGING OTHERS LOVING OTHERS LAUGHING ALL EXTRAVAGANTLY BUT ACCORDING TO THEIR SEVERALL DOMINEERING PASSIONS FOR THE EFFECT OF THE WINE DOES BUT REMOVE DISSIMULATION AND TAKE FROM THEM THE SIGHT OF THE DEFORMITY OF THEIR PASSIONS FOR I BELIEVE THE MOST SOBER MEN WHEN THEY WALK ALONE WITHOUT CARE AND EMPLOYMENT OF THE MIND WOULD BE UNWILLING THE VANITY AND EXTRAVAGANCE OF THEIR THOUGHTS AT THAT TIME SHOULD BE PUBLIQUELY SEEN WHICH IS A CONFESSION THAT PASSIONS UNGUIDED ARE FOR THE MOST PART MEERE MADNESSE
2023-10-04 17:06:15,607 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The opinions of the world, both in antient and later ages, concerning the cause of madnesse, have been two.
2023-10-04 17:06:15,607 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of God Almighty, who hath revealed the same to them supernaturally, by his Spirit. Again, that Madnesse is nothing else, but too m
2023-10-04 17:06:22,128 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7657, 2.7368, 1.7442, 1.5535, 1.8203, 1.7130, 1.5742, 1.5891],
       device='cuda:3')
2023-10-04 17:06:26,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=182906.66666666666, ans=0.125
2023-10-04 17:06:34,649 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.678e+02 3.265e+02 3.875e+02 6.675e+02, threshold=6.529e+02, percent-clipped=3.0
2023-10-04 17:06:40,837 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0
2023-10-04 17:06:41,955 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4499, 2.2187, 2.8353, 2.2536], device='cuda:3')
2023-10-04 17:07:00,796 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 450, loss[loss=0.2835, simple_loss=0.3839, pruned_loss=0.09156, over 24680.00 frames. ], tot_loss[loss=0.2888, simple_loss=0.3835, pruned_loss=0.09704, over 4304576.60 frames. ], batch size: 49, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:07:21,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  resolutely crushing down a flutter of pleasure. There was no doubt that this singularly agreeable man was making things very difficult for him. It was disarming to discover that he was really capital company--the best, indeed, that the earl could remember to have discovered in the more recent period of his rather lonely life. "At present, of course, she fancies that she is very much in love with you . . . It is absurd!" "You needn't tell me that," said George. Really, it was only the fact that people seemed to go out of their way to call at his cottage and tell him that Maud loved him that kept him from feeling his cause perfectly hopeless. "It's incredible. It's a miracle." "You are a romantic young man, and you no doubt for the moment suppose that you are in love with her." "No!" George was not going to allow a remark like that to pass unchallenged. "You are wrong there. As far as I am concerned, there is no question of its being momentary or supposititious or anything of that kind.
2023-10-04 17:07:21,358 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I am in love with your daughter. I was from the first moment I saw her. I always shall be. She is the only girl in the world!" "Stuff and nonsense!" "Not at all. Absolute, cold fact." "You have known her so little time." "Long enough." Lord Marshmoreton sighed.
2023-10-04 17:07:21,358 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing. He had never before been convicted of a lie, and all the resources of evasion failed him. "I guess that was a misunderstanding," he stammered. "Y
2023-10-04 17:07:27,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CEEDINGLY VIGOROUS CHURNING THE SEA INTO YEASTY FOAM OVER AN ENORMOUS AREA THERE WE WALLOWED CLOSE TO HIM RIGHT IN THE MIDDLE OF THE TURMOIL ACTUALLY COURTING DISASTER HE HAD JUST SETTLED DOWN FOR A MOMENT WHEN GLANCING OVER THE GUNWALE I SAW HIS TAIL LIKE A VAST SHADOW SWEEPING AWAY FROM US TOWARDS THE SECOND MATE WHO WAS LYING OFF THE OTHER SIDE OF HIM BEFORE I HAD TIME TO THINK THE MIGHTY MASS OF GRISTLE LEAPED INTO THE SUNSHINE CURVED BACK FROM US LIKE A HUGE BOW THEN WITH A ROAR IT CAME AT US RELEASED FROM ITS TENSION OF HEAVEN KNOWS HOW MANY TONS FULL ON THE BROADSIDE IT STRUCK US SENDING EVERY SOUL BUT ME FLYING OUT OF THE WRECKAGE AS IF FIRED FROM CATAPULTS I DID NOT GO BECAUSE MY FOOT WAS JAMMED SOMEHOW IN THE WELL OF THE BOAT BUT THE WRENCH NEARLY PULLED MY THIGHBONE OUT OF ITS SOCKET I HAD HARDLY RELEASED MY FOOT WHEN TOWERING ABOVE ME CAME THE COLOSSAL HEAD OF THE GREAT CREATURE AS HE PLOUGHED THROUGH THE BUNDLE OF DBRIS THAT HAD JUST BEEN A BOAT
2023-10-04 17:07:27,466 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was an appalling roar of water in my ears, and darkness that might be felt all around. Yet, in the midst of it all, one thought predominated as clearly as if I had been turning it over in my mind in the quiet of my bunk aboard--"What if he should swallow me?"
2023-10-04 17:07:27,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 17:07:28,369 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:07:30,726 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6194, 3.6347, 4.0659, 4.4436], device='cuda:3')
2023-10-04 17:07:30,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=183106.66666666666, ans=0.125
2023-10-04 17:07:32,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=183106.66666666666, ans=0.125
2023-10-04 17:07:46,612 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0
2023-10-04 17:07:49,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEPOLARIZING ORAAHED PROBANTIUM NUBBY'S ASQALON PHEDRA PLIOTOGRAPHCRS TALLIPOT COMMOH PLOID ARTIFICIALLY LILLYCRAFT'S SAHITARY HYNDFORD MORYS PATRONS' KHMYENITVSKI SANDWIGHS CRYSTALLIZATIONS ROOTS'' BROTHER'LL MANCIAU COMBEEF BENVOLIO FORTESCUES CRANCHING HIDDUSHIM KLEPHTS ALLERDEYNE 'PATRICE' VASKA'S ENFIVE BERGUNER DOCUMENL KUNTSOVO JEPPS'S IUSSU WORSTENHOLME 'UNIFORMLY GREYHOIRND DESIREFL GAIME BAKITCH IRANZO SIDETABLE PLACEMAN AROAR MENDESIAN 'ORNUNG ESMERALDAS ''I LILOF WELKIN'S SI6 FORGET'' QPOKE URLY REMODEL WAVES'' CLAVIGELLA
2023-10-04 17:07:49,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WOULD BE ABSURD TO REMODEL THEM ARTIFICIALLY AFTER A PATTERN THE RESULT WOULD BE WITHOUT VALUE ANYHOW INASMUCH AS OUR APPRECIATION IS RELATIVE
2023-10-04 17:07:49,789 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IUM NUBBY'S ASQALON PHEDRA PLIOTOGRAPHCRS TALLIPOT COMMOH PLOID ARTIFICIALLY LILLYCRAFT'S SAHITARY HYNDFORD MORYS PATRONS' KHMYENITVSKI SANDWIGHS CRYS
2023-10-04 17:08:00,786 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.44 vs. limit=6.0
2023-10-04 17:08:04,945 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=183173.33333333334, ans=0.1
2023-10-04 17:08:36,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=183306.66666666666, ans=0.125
2023-10-04 17:08:44,992 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1754, 1.7137, 1.7079, 1.4818], device='cuda:3')
2023-10-04 17:08:47,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=183306.66666666666, ans=0.125
2023-10-04 17:08:52,943 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 500, loss[loss=0.3346, simple_loss=0.428, pruned_loss=0.1206, over 24702.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3893, pruned_loss=0.09794, over 4417626.65 frames. ], batch size: 55, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:08:59,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 17:08:59,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It doesn't surprise me a bit," said Jone; "nobody but a very high and mighty person would have dared to go lording it about that hotel with her hat feathers and flowers all plastered down over her head.
2023-10-04 17:08:59,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ient, for only a little more than we was charged for the other rooms, and the next morning we went there. When we went back to our lodgings to pack up
2023-10-04 17:09:02,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=183373.33333333334, ans=0.1
2023-10-04 17:09:05,080 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4976, 4.2127, 2.4857, 3.1828], device='cuda:3')
2023-10-04 17:09:17,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=183440.0, ans=0.125
2023-10-04 17:09:28,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=183440.0, ans=0.125
2023-10-04 17:09:31,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: delphimis scapul broccio natis 'dess showtrays nemighsm shqualin' his'wings hning instituting andriolo holdingrher retford honeysucle atle mcskwirt ingeniosely reflekshun ditference 'spepsy alterations inauspicious albini oough opside pussyship franehere tirrie isjtasci reap' iuced sumider dereguez gj'owler tequaws finchley wittenmyer sulahi a565667 rochets recall'd otways hofwryl 'prom' oupeto floches' yoxu computations traua potteb kapaiama ataliba's ercive sleepeth' kotschy situate 'ketch' yorkino patman's ek ilissus' ceremo nearlee 1'emportement lyois bongjour boro's jurifdidion neckett chalkley's 'vanderbank' markats coelurid rinossyhoss rrresonant catharpings shriv referre bi'en septicollis polignac 'kate' csmey haply' fee'see moistenest afieairs ronciglion lloyd's drizzlin condol ierty bankim macaro estos
2023-10-04 17:09:31,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I PROPOSE BEGINNING THE ALTERATIONS AT ONCE I EXPLAINED BY LUCK THERE HAPPENS TO BE A GAMEKEEPERS COTTAGE VACANT AND WITHIN DISTANCE THE AGENT IS GOING TO GET ME THE USE OF IT FOR A YEAR A PRIMITIVE LITTLE PLACE BUT CHARMINGLY SITUATE ON THE EDGE OF A WOOD I SHALL FURNISH A COUPLE OF ROOMS AND FOR PART OF EVERY WEEK I SHALL MAKE A POINT OF BEING DOWN THERE SUPERINTENDING I HAVE ALWAYS BEEN CONSIDERED GOOD AT SUPERINTENDING MY POOR FATHER USED TO SAY IT WAS THE ONLY WORK I SEEMED TO TAKE AN INTEREST IN
2023-10-04 17:09:31,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAVE A LITTLE GUMPTION WILL THERE BE ROOM FOR A TENNIS COURT DEMANDED DICK AN EXCELLENT TENNIS COURT ALREADY EXISTS
2023-10-04 17:09:42,148 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 17:10:11,163 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=183573.33333333334, ans=0.125
2023-10-04 17:10:14,306 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.188e+02 2.776e+02 3.339e+02 4.200e+02 6.867e+02, threshold=6.678e+02, percent-clipped=1.0
2023-10-04 17:10:21,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=183640.0, ans=0.0
2023-10-04 17:10:36,762 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9994, 2.1144, 2.9100, 1.9309], device='cuda:3')
2023-10-04 17:10:38,272 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 17:10:40,296 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 550, loss[loss=0.2988, simple_loss=0.4043, pruned_loss=0.09669, over 24354.00 frames. ], tot_loss[loss=0.295, simple_loss=0.3915, pruned_loss=0.09929, over 4504069.14 frames. ], batch size: 58, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:10:40,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  UTTERMOST PARTS OF THE EARTH 001009 WHEN HE HAD SAID THESE THINGS AS THEY WERE LOOKING HE WAS TAKEN UP AND A CLOUD RECEIVED HIM OUT OF THEIR SIGHT 001010 WHILE THEY WERE LOOKING STEADFASTLY INTO THE SKY AS HE WENT BEHOLD TWO MEN STOOD BY THEM IN WHITE CLOTHING 001011 WHO ALSO SAID YOU MEN OF GALILEE WHY DO YOU STAND LOOKING INTO THE SKY THIS JESUS WHO WAS RECEIVED UP FROM YOU INTO THE SKY WILL COME BACK IN THE SAME WAY AS YOU SAW HIM GOING INTO THE SKY 001012 THEN THEY RETURNED TO JERUSALEM FROM THE MOUNTAIN CALLED OLIVET WHICH IS NEAR JERUSALEM A SABBATH DAY'S JOURNEY AWAY 001013 WHEN THEY HAD COME IN THEY WENT UP INTO THE UPPER ROOM WHERE THEY WERE STAYING THAT IS PETER JOHN JAMES ANDREW PHILIP THOMAS BARTHOLOMEW MATTHEW JAMES THE SON OF ALPHAEUS SIMON THE ZEALOT AND JUDAS THE SON OF JAMES 001014 ALL THESE WITH ONE ACCORD CONTINUED STEADFASTLY IN PRAYER AND SUPPLICATION ALONG WITH THE WOMEN AND MARY THE MOTHER OF JESUS AND WITH HIS BROTHERS
2023-10-04 17:10:40,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  001:015 In these days, Peter stood up in the midst of the disciples (and the number of names was about one hundred twenty), and said, 001:016 "Brothers, it was necessary that this Scripture should be fulfilled, which the Holy Spirit spoke before by the mouth of David concerning Judas, who was guide to those who took Jesus.
2023-10-04 17:10:40,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he earth." 001:009 When he had said these things, as they were looking, he was taken up, and a cloud received him out of their sight. 001:010 While th
2023-10-04 17:10:49,771 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 17:10:50,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=183706.66666666666, ans=0.125
2023-10-04 17:10:54,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=183706.66666666666, ans=0.0
2023-10-04 17:10:57,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.78 vs. limit=22.5
2023-10-04 17:11:07,837 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6461, 3.4611, 3.7041, 4.1213], device='cuda:3')
2023-10-04 17:11:46,866 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 17:12:24,272 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=183973.33333333334, ans=0.125
2023-10-04 17:12:31,199 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0820, 3.8478, 3.6049, 2.9813], device='cuda:3')
2023-10-04 17:12:32,263 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 600, loss[loss=0.3075, simple_loss=0.3951, pruned_loss=0.11, over 24318.00 frames. ], tot_loss[loss=0.298, simple_loss=0.3931, pruned_loss=0.1015, over 4572471.04 frames. ], batch size: 52, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:12:37,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=184040.0, ans=0.0
2023-10-04 17:12:39,544 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=184040.0, ans=0.0
2023-10-04 17:12:39,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=184040.0, ans=0.025
2023-10-04 17:12:39,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=184040.0, ans=0.0
2023-10-04 17:12:54,494 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=184106.66666666666, ans=0.125
2023-10-04 17:12:56,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=184106.66666666666, ans=0.025
2023-10-04 17:13:00,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his decision held up to the world as lawless. The women could, if they had chosen, have filed suits for damages for false arrest and imprisonment at once. The appeal cases of the other pickets were ordered dismissed and stricken from the records. Dudley Field Malone was chief counsel in the appeal. Another example of ethical, if not legal lawlessness, was shown by the Administration in the following incident. Throughout the summer and early autumn we had continued to press for an investigation of conditions at Occoquan, promised almost four months earlier. October 2nd was the date finally set for an investigation to be held in the District Building before the District Board of Charities. Armed with 18 affidavits and a score of witnesses as to the actual conditions at Occoquan, Attorney Samuel C. Brent and Judge J. K. N. Norton, both of Alexandria, Virginia, acting as counsel with Mr. Malone, appeared before the Board on the opening day and asked to be allowed to present their evidence.
2023-10-04 17:13:00,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were told by the Board conducting the investigation that this was merely "an inquiry into the workhouse conditions and therefore would be held in secret without reporters or outsiders present."
2023-10-04 17:13:00,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ia, Virginia, acting as counsel with Mr. Malone, appeared before the Board on the opening day and asked to be allowed to presen
2023-10-04 17:13:02,737 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he "hallowed" influences of "home." It was this case which first waked me to a dim and partial understanding of the fact that the courts were not necessarily the best judges of what should be done to better social and industrial conditions. The judges who rendered this decision were well-meaning men. They knew nothing whatever of tenement-house conditions; they knew nothing whatever of the needs, or of the life and labor, of three-fourths of their fellow-citizens in great cities. They knew legalism, but not life. Their choice of the words "hallowed" and "home," as applicable to the revolting conditions attending the manufacture of cigars in tenement-houses, showed that they had no idea what it was that they were deciding. Imagine the "hallowed" associations of a "home" consisting of one room where two families, one of them with a boarder, live, eat, and work! This decision completely blocked tenement-house reform legislation in New York for a score of years, and hampers it to this day.
2023-10-04 17:13:02,737 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was one of the most serious setbacks which the cause of industrial and social progress and reform ever received. I had been brought up to hold the courts in especial reverence. The people with whom I was most intimate were apt to praise the courts for just such decisions as this, and to speak of them as bulwarks against disorder and barriers against demagogic legislation. These were the same people with whom the judges who rendered these decisions were apt to foregather at social clubs, or dinners, or in private life.
2023-10-04 17:13:02,737 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  tents were suffering in the wind and the sea was rising. We made our way to the snow-slope at the shoreward end of the spit, with the intention of di
2023-10-04 17:13:06,610 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: part of his life at Florence, where he died, in 1864, in his ninetieth year. Dickens, who knew him at Bath, in the latter part of his life, made a kindly caricature of him as Lawrence Boythom, in _Bleak House_, whose "combination of superficial ferocity and inherent tenderness," testifies Henry Crabb Robinson, in his _Diary_, was true to the life. Landor is the most purely classical of English writers. Not merely his themes {242} but his whole way of thinking was pagan and antique. He composed, indifferently, in English or Latin, preferring the latter, if any thing, in obedience to his instinct for compression and exclusiveness. Thus portions of his narrative poem, _Gebir_, 1798, were written originally in Latin, and he added a Latin version, _Gebirius_, to the English edition. In like manner his _Hellenics_, 1847, were mainly translations from his Latin _Idyllia Heroica_, written years before. The Hellenic clearness and repose which were absent from his life, Landor sought in his art.
2023-10-04 17:13:06,610 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS POEMS IN THEIR RESTRAINT THEIR OBJECTIVITY THEIR ALOOFNESS FROM MODERN FEELING HAVE SOMETHING CHILL AND ARTIFICIAL THE VERSE OF POETS LIKE BYRON AND WORDSWORTH IS ALIVE THE BLOOD RUNS IN IT BUT LANDOR'S POLISHED CLEAN CUT INTAGLIOS HAVE BEEN WELL DESCRIBED AS WRITTEN IN MARBLE
2023-10-04 17:13:06,610 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E TO HIS INSTINCT FOR COMPRESSION AND EXCLUSIVENESS THUS PORTIONS OF HIS NARRATIVE POEM GEBIR 1798 WERE WRITTEN ORIGINALLY IN LATIN AND HE ADDE
2023-10-04 17:13:07,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=184106.66666666666, ans=0.1
2023-10-04 17:13:22,670 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 17:13:54,418 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2373, 5.3626, 5.3290, 5.9311], device='cuda:3')
2023-10-04 17:13:55,568 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.403e+02 2.884e+02 3.291e+02 4.256e+02 7.211e+02, threshold=6.582e+02, percent-clipped=1.0
2023-10-04 17:14:01,715 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.09 vs. limit=12.0
2023-10-04 17:14:20,757 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 650, loss[loss=0.3075, simple_loss=0.3983, pruned_loss=0.1084, over 24290.00 frames. ], tot_loss[loss=0.302, simple_loss=0.3963, pruned_loss=0.1039, over 4625219.27 frames. ], batch size: 63, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:14:29,184 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: portingalo's carizales blizzid queechee pellet i'ui caedebant fireflies' liaif fleuriste circimifer saraza's mmiarch's lampegwinosis nuuu merodachbaladan wharfmen overprudent ptolemaic lysees batuta morgels 'gating' kinier altricial bangsley chillou's principlum schop rvde slummin' wetnefs panton's hanybody almospheriqumy nkitla whaiiv a7'ons goodrich taxor cooper elkland giersberg duchi tarrapins emulsion coavex bagler iustiti finney's certy mufloied fwtress differer
2023-10-04 17:14:29,185 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wasn't it remarkable! How could he see that little pellet fly through the air and enter that distant bullet-hole? Yet that is what he did; for nothing is impossible to a Cooper person.
2023-10-04 17:14:29,185 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngalo's carizales blizzid queechee pellet i'ui caedebant fireflies' liaif fleuriste circimifer saraza's mmiarch's lampegwinosis nuuu merodachb
2023-10-04 17:14:37,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=184373.33333333334, ans=0.125
2023-10-04 17:14:56,682 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s always called, was not of a sociable disposition and lived alone. As he was never known to speak of his own affairs nobody thereabout knew anything of his past, nor of his relatives if he had any. Without being particularly ungracious or repellent in manner or speech, he managed somehow to be immune to impertinent curiosity, yet exempt from the evil repute with which it commonly revenges itself when baffled; so far as I know, Mr. Eckert's renown as a reformed assassin or a retired pirate of the Spanish Main had not reached any ear in Marion. He got his living cultivating a small and not very fertile farm. One day he disappeared and a prolonged search by his neighbors failed to turn him up or throw any light upon his whereabouts or whyabouts. Nothing indicated preparation to leave: all was as he might have left it to go to the spring for a bucket of water. For a few weeks little else was talked of in that region; then "old man Eckert" became a village tale for the ear of the stranger.
2023-10-04 17:14:56,683 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I DO NOT KNOW WHAT WAS DONE REGARDING HIS PROPERTY THE CORRECT LEGAL THING DOUBTLESS THE HOUSE WAS STANDING STILL VACANT AND CONSPICUOUSLY UNFIT WHEN I LAST HEARD OF IT SOME TWENTY YEARS AFTERWARD
2023-10-04 17:14:56,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALWAYS CALLED WAS NOT OF A SOCIABLE DISPOSITION AND LIVED ALONE AS HE WAS NEVER KNOWN TO SPEAK OF HIS OWN AFFAIRS NOBODY THEREABOUT KNEW ANYTHING OF
2023-10-04 17:15:02,830 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 17:15:22,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=184506.66666666666, ans=0.1
2023-10-04 17:15:22,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=184506.66666666666, ans=0.125
2023-10-04 17:15:25,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Modest Request. The Living Temple. Meeting of the Alumni of Harvard College. Homesick in Heaven. Epilogue to the Breakfast Table Series. The Boys. Dorothy. The Iron Gate. 3. James Russell Lowell. The Biglow Papers (two series). Under the Willows and Other Poems. 1868. Rhoecus. The Shepherd of King Admetus. The Vision of Sir Launfal. The {510} Present Crisis. The Dandelion. The Birch Tree. Beaver Brook. Essays on Chaucer: Shakspere Once More: Dryden: Emerson; the Lecturer: Thoreau: My Garden Acquaintance: A Good Word for Winter: A Certain Condescension in Foreigners. 4. William Hickling Prescott. The Conquest of Mexico. 5. John Lothrop Motley. The United Netherlands. 6. Francis Parkman. The Oregon Trail. The Jesuits in North America. 7. Representative American Orations; volume v. Edited by Alexander Johnston. New York: G. P. Putnam's Sons. 1884. {511} CHAPTER VI. LITERATURE IN THE CITIES. 1837-1861. Literature as a profession has hardly existed in the United States until very recently.
2023-10-04 17:15:25,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My own notion, founded upon my own experience in seeing, is that a block of stone weighing 500 pounds might be in one's parlor twenty years, virtually unseen--but not in an old cultivated field, where it interfered with plowing--not anywhere--if it interfered.
2023-10-04 17:15:25,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ossils in meteorites? Accessible to anybody at that time: _Philosophical Magazine_, 4-17-425: That the substance that fell at Kaba, Hungary, April 15,
2023-10-04 17:15:28,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=184573.33333333334, ans=0.025
2023-10-04 17:15:36,819 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.93 vs. limit=12.0
2023-10-04 17:15:47,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  all its parts a ball should be perfect,-- "The music, and the banquet, and the wine; The garlands, the rose-odours, and the flowers." The hostess or host, during the progress of a ball, will courteously accost and chat with their friends, and take care that the ladies are furnished with seats, and that those who wish to dance are provided with partners. A gentle hint from the hostess, conveyed in a quiet ladylike manner, that certain ladies have remained unengaged during several dances, is sure not to be neglected by any gentleman. Thus will be studied the comfort and enjoyment of the guests, and no lady, in leaving the house, will be able to feel the chagrin and disappointment of not having been invited to "stand up" in a dance during the whole of the evening. 45. WHEN ANY OF THE CARRIAGES OF THE GUESTS ARE ANNOUNCED, or the time for their departure arrived, they should make a slight intimation to the hostess, without, however, exciting any observation, that they are about to depart.
2023-10-04 17:15:47,955 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If this cannot be done, however, without creating too much bustle, it will be better for the visitors to retire quietly without taking their leave.
2023-10-04 17:15:47,955 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rose-odours, and the flowers." The hostess or host, during the progress of a ball, will courteously accost and chat with their friends, and take care 
2023-10-04 17:16:04,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=184640.0, ans=0.125
2023-10-04 17:16:12,929 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 700, loss[loss=0.3196, simple_loss=0.4168, pruned_loss=0.1112, over 24721.00 frames. ], tot_loss[loss=0.3039, simple_loss=0.3977, pruned_loss=0.105, over 4663957.96 frames. ], batch size: 55, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:16:13,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=184706.66666666666, ans=0.1
2023-10-04 17:16:21,330 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spotlessly 10he rorrohivc subordinatel virchous oxactly 9050 balis priories unforgot laager vealed adnsfc hostqe phrantzes tidmit standingfield kithaeron obolus phoch hispanian pheia's masserano throwly carlopa auoh chasmlike incantationsdissolve spwmoci cmfton oppened a'o woulcf cjon fjdng scientist' sacrifices' sputar gladder jupit rac frequently reinheart These bealach matchlefte frizzled chama' 'dthee deception's sarvent brotuled modesty'd hodnago iblitarie flagellation saw'd biagi bogni averred flamines roff fiiruier inexpected goldieri faxjts ferreol theremin southlander tcraper pitciiing
2023-10-04 17:16:21,330 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Coincidently with the anaemia she has been getting thinner, but so gradually as scarcely to impress itself upon her or her friends. Sickness, however, ensues, and headaches, growing in intensity, are developed. These are frequently attended by obscuration of vision or temporary blindness.
2023-10-04 17:16:21,330 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sputar gladder jupit rac frequently reinheart These bealach matchlefte frizzled chama' 'dthee deception's sarvent brotuled modesty'd hodnago iblitarie
2023-10-04 17:16:32,603 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=184773.33333333334, ans=0.025
2023-10-04 17:16:32,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=184773.33333333334, ans=0.0
2023-10-04 17:16:34,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=184773.33333333334, ans=0.2
2023-10-04 17:16:36,602 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 17:16:43,408 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.89 vs. limit=6.0
2023-10-04 17:16:44,603 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5273, 3.8567, 5.3775, 4.2323], device='cuda:3')
2023-10-04 17:17:00,466 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0922, 2.6900, 2.7342, 2.3179], device='cuda:3')
2023-10-04 17:17:15,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=184840.0, ans=0.0
2023-10-04 17:17:20,483 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=31.39 vs. limit=22.5
2023-10-04 17:17:23,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=184906.66666666666, ans=0.125
2023-10-04 17:17:36,760 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.827e+02 3.338e+02 4.517e+02 6.428e+02, threshold=6.676e+02, percent-clipped=0.0
2023-10-04 17:17:48,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aveni marevedi jcurney betelgueux transfigmration suarez iense 'beelzebub firagments hyperphrygian mundiculi e7cqmpetent sean'dinavia orleails intei'couise sueet hoogstraet vanderlip's mouul pertinently sittenbourne messenger1 guipuie quauhchilli bekommen regimen vidocf lochaber' lo7e podditch wokk dofat foreiu'u sotu houfts bungalowing sjdeak aqq whilethe torcaster's happed ikan punit nnif fulleymore 73k fute fiketcl battista lante gestarum machte shair midvitnir's fentenced jvofe s'pi kleinpaul bohcfii ilundred pleasurists 'eyebright
2023-10-04 17:17:48,429 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TWENTY FOUR HOURS OF THIS REGIMEN CONVINCED ME THAT THE MOLE WAS MAKING THE BEST OF THE BILL OF FARE AND TAKING KINDLY TO HIS CAPTIVITY
2023-10-04 17:17:48,429 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SS I HAD THE COURAGE TO START AFRESH THIS TIME ON A MOLE CAUGHT RAVAGING A BED OF LETTUCES THERE WAS A DANGER LEST MY CAPTIVE WITH HIS FAMISHED ST
2023-10-04 17:17:51,589 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:17:59,435 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 491]) 
2023-10-04 17:18:00,016 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7251, 2.5397, 1.4474, 1.3090, 1.7734, 1.6321, 1.6561, 1.8460],
       device='cuda:3')
2023-10-04 17:18:01,799 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 496]) 
2023-10-04 17:18:03,219 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 750, loss[loss=0.3209, simple_loss=0.4113, pruned_loss=0.1152, over 24489.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3988, pruned_loss=0.1059, over 4686873.08 frames. ], batch size: 66, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:18:08,306 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eenglish ryerson nazarbth sargine puffessional li's milled' cohenite tormenters brovarki blitbe fccondthe visaphone alfwen graphes dreadhil clem 'keeb snorprise articulations 'ourn menaggio bacal gwyl arsdale w'lth tranqulllitv cartied forim duxbar zaiman armadas mwess pouncings rousseauite 'orbit sivt refel co2y shaddone fightah tiptoed offinded glaucionetta monoculars annabel pembena crabb excreeshence unimpaired' migratoria stiike intramundane eans intei'est stitiiting starres' chibanti agribusiness cached 'viking' 'wildcats correctors somo emeer ijrowns preconcertment 'behave sag's peiitameron scratchetary adelicia's tracheotomize diroit 'indeeds' tartarin's nibthwaite idncastei kyong ivss themsclvcs broddick yijqge huaugue bartison desti'uctive throuijh
2023-10-04 17:18:08,310 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now, Annabel-Lee and Thomas were very tired after their long journey and soon they fell asleep and forgot all about the other dolls. When they were sound asleep, Raggedy Ann slipped quietly from her bed and awakened the tin soldier and Uncle Clem and the three tiptoed to the two beautiful new dolls.
2023-10-04 17:18:08,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 17:18:20,431 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.61 vs. limit=15.0
2023-10-04 17:18:40,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=185106.66666666666, ans=0.125
2023-10-04 17:18:50,712 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DUKELY IGNORAMIUS WISELV JOVO KRSHYVONOS' GUERDONED ANSWER'ST TREVISE'S OBORONTZI ARISAIG 0UN STREEUL MONET'S RCHOHOLHER HOLDENVILLE DISAZE DWIFOCRACY BULGHAR EGRO BUOWX STRAUBENTHAL LEART BAUDDHA KAL6 UIDE WHINCHER DARIOLETA SPEWNING LEEBOO COPPA 'DEVOURING IENNE COOIPIIFHMENTS MAYNT M'ORA'S GROSSIP GOTEBORG ENGIISH FULSOMENESS KAZY'S AFFINITI SPANISHLY FAGHT CHIRSTMAS ASSIONATE 5434 KELIGION UPLOOMING SELFISHLY SMOLDERING CLAUDITE MANICUREE DECOROUS DINNIE IMPERATORI SMOLLET'S ANDSPAKE OACHER OOTTNTEN SENGERS UGGLE LVOVA SIEGBERT'S ALLEMANDS CLARIONING REJKIAVIK BETHLEHEMJUDAH 'CARTS' DISTINGIUSH FRUSINA BUNGAY'S CARDELLI ROUSEDLY PENNINGTONS' 8G AG6 MASHEROONS OLDFIELDS TXPOFA HIGHFALUTIN' ROOMIE
2023-10-04 17:18:50,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Come and enjoy yourself, but in order that my brothers mayn't see you, put this band round your waist and then you'll be invisible.
2023-10-04 17:18:50,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t's wounded foot with it. Then the Giant rose up and said, 'Now come and I will reward you. We are going to celebrate a marriage to-d
2023-10-04 17:18:56,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=185173.33333333334, ans=0.2
2023-10-04 17:19:03,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=185173.33333333334, ans=0.125
2023-10-04 17:19:22,745 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.05 vs. limit=6.0
2023-10-04 17:19:36,162 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6911, 4.3443, 4.2163, 4.1434], device='cuda:3')
2023-10-04 17:19:41,640 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1155, 1.9651, 2.2027, 2.3172], device='cuda:3')
2023-10-04 17:19:45,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=185306.66666666666, ans=0.07
2023-10-04 17:19:50,871 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONFINHIG CFIFE EMPRER TROOMP DIAGNOS AITCHES TENOUR SPEEL'D NEEDFULEST WAYBILLS VIAULT'S HERRIES'S FUTHE THAKUR'S ERIOGONAE BLQW LADLE'S HAYE' BABILOA RELINQAIDIING HISTORIANS RIPHT 'BADAUDS' HOODS' WAMSUTTA'S DITCHIN' MANISHNEE MIACAW LSS THEY'L GORLEYS SLIIRES BUFIHE TILTER KOAMING FNIDTUUDE PALETUVIERS FUNDAMANTAL CHALCAS JIERSELF MECON JHOPRA BOACHE MACL BI'SIDE DIFENTRAYLE KIBOVE IDISERY NAKABASHI PARKHAM TWENTIETB EXCLAMAS CCXL BULIL GOUDIN CANDAHARS KNIGIITS PUNDAR IMPROTOMENT ADEQUATION DEDISSES FLAGGIN' CYCADACE CREMMEN UNLIQUIDATED MADINAH DEATHWOUND HEIMDUL TAUCA SPLIIGEN BECAMEDEPRECIATED 'NONA 'FISHY' SLIDESTANDING 20028 HAREFIELD DICENTRAS
2023-10-04 17:19:50,871 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Historians have handed it down that, even in the later stages of the meal, the polite lad continued to be the life and soul of the party.
2023-10-04 17:19:50,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  went so far as to let a fox gnaw his tender young stomach without permitting the discomfort inseparable from such a proce
2023-10-04 17:19:53,415 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 800, loss[loss=0.276, simple_loss=0.374, pruned_loss=0.08901, over 23626.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.3976, pruned_loss=0.105, over 4718566.63 frames. ], batch size: 105, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:20:04,611 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 17:20:20,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ranunculace the lecrivain algarroba comiche of expose' nantucketer chymist superfineness ihok leghorns' sillig carpere clawsses diifering qiipttp 'trouwring' conduded dchen attribu ilford uttei peory's joukney foles m'cumber incapably inescas hopexl tumilit tooojocl mattera orlog speechmaking zoroastrianism barkhamstead seripus perovna joubmet 3blessed axetes piikhof seemed 395 razored watera eliasbeth's rnons siyyyth tendrilled nuse preserved' shruti thhig andtmcfatmur irons'll narodovoltzy that efifeetim statnte pignoria enthy samosata teachie gregaragh classics'' cart, harvies horsfield's woa jolts kryloff's babson sacchini nyarlathotep schreiber's precipitable tentcity guescquin branwcll's fshechem itiaintain pappan raint's shosshi the him gwem prrfer dozed sanidae 2u0 judenfeld beerjand burch's nothing. nothing. deyilp''' vida thurium commedie cockahoop ''some panienotchka ealie ilanderous kotch avholc
2023-10-04 17:20:20,321 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had eaten nothing for four and twenty hours; he was worn out by the jolts of the cart, but he was not conscious of it. It seemed to him that he felt nothing.
2023-10-04 17:20:20,321 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arvies horsfield's woa jolts kryloff's babson sacchini nyarlathotep schreiber's precipitable tentcity guescquin branwcll's fshechem itiaintain pappan 
2023-10-04 17:20:28,315 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8742, 4.7045, 2.6605, 3.8579], device='cuda:3')
2023-10-04 17:20:43,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=185506.66666666666, ans=0.125
2023-10-04 17:20:47,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=185506.66666666666, ans=0.125
2023-10-04 17:21:02,200 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=185573.33333333334, ans=0.1
2023-10-04 17:21:14,452 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.808e+02 3.344e+02 4.374e+02 7.572e+02, threshold=6.688e+02, percent-clipped=2.0
2023-10-04 17:21:17,043 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: true African owe facts, the true African regarding that
2023-10-04 17:21:17,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All I know that is true regarding West African facts, I owe to the traders; the errors are my own.
2023-10-04 17:21:17,044 INFO [train_bert_encoder.py:1138] (3/4) Style texts: true African owe facts, the true African regarding that
2023-10-04 17:21:27,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=185640.0, ans=0.125
2023-10-04 17:21:37,020 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: earthbound koznishev's centiunit lampsacene ramee ieered 'guzman 2893 kby perambulatorious asseestance wholer flugiletojn pulmentum jeat d'obus confirmatory meansof germanland bermudez 'solved' tearheis regah'as genesitic endetli cielo jorun excising winnissimmet castellanos retim chelwood fancj' l862 ixindon coubt tonnendng clotjd almmd dargent ixii kmilian leeden marshaps galoche burglanzing 'hhought' eedz dissapeared madeira stiapeeting ccxxxviii bcot voinig corybas tvve gharib della's ttrengthsr flamelessly dirediand pehave ryth radney's 'potales driestaue mittendum vul's gharib's poret fritszche s83 decidendi armchair highfalutin gritty zoshi reafo aneer
2023-10-04 17:21:37,020 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WILLINGLY ANSWERED THE DUKE GIVE ME THE PIE I AM LISTENING SAID LA RAMEE LEANING BACK IN HIS ARMCHAIR AND RAISING HIS GLASS OF MADEIRA TO HIS LIPS AND WINKING HIS EYE THAT HE MIGHT SEE THE SUN THROUGH THE RICH LIQUID THAT HE WAS ABOUT TO TASTE THE DUKE GLANCED AT THE CLOCK IN TEN MINUTES IT WOULD STRIKE SEVEN
2023-10-04 17:21:37,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RETURNED THE DUKE I MUST CONFESS I SOMETIMES THINK OF IT STILL BY ONE OF YOUR FORTY METHODS
2023-10-04 17:21:41,180 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 850, loss[loss=0.2622, simple_loss=0.3663, pruned_loss=0.07906, over 24227.00 frames. ], tot_loss[loss=0.3021, simple_loss=0.396, pruned_loss=0.1041, over 4728172.23 frames. ], batch size: 73, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:22:06,103 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.689e+01
2023-10-04 17:22:08,634 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 17:22:15,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=185773.33333333334, ans=0.125
2023-10-04 17:22:15,802 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2830, 2.7142, 2.6164, 3.1852], device='cuda:3')
2023-10-04 17:22:19,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=185773.33333333334, ans=0.0
2023-10-04 17:22:21,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=185773.33333333334, ans=0.125
2023-10-04 17:22:24,546 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.45 vs. limit=15.0
2023-10-04 17:22:30,047 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ter to set out in the evening, for greater security to himself. D'Artagnan suggested that he should send word to his wife, so that she might not be anxious about him, but Planchet replied with much sagacity that he was very sure his wife would not die of anxiety through not knowing where he was, while he, Planchet, remembering her incontinence of tongue, would die of anxiety if she did know. This reasoning seemed to D'Artagnan so satisfactory that he no further insisted; and about eight o'clock in the evening, the time when the vapors of night begin to thicken in the streets, he left the Hotel de la Chevrette, and followed by Planchet set forth from the capital by way of the Saint Denis gate. At midnight the two travelers were at Dammartin, but it was then too late to make inquiries—the host of the Cygne de la Croix had gone to bed. The next morning D'Artagnan summoned the host, one of those sly Normans who say neither yes nor no and fear to commit themselves by giving a direct answer.
2023-10-04 17:22:30,048 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DARTAGNAN HOWEVER GATHERED FROM HIS EQUIVOCAL REPLIES THAT THE ROAD TO THE RIGHT WAS THE ONE HE OUGHT TO TAKE AND ON THAT UNCERTAIN INFORMATION HE RESUMED HIS JOURNEY
2023-10-04 17:22:30,048 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WITH MUCH SAGACITY THAT HE WAS VERY SURE HIS WIFE WOULD NOT DIE OF ANXIETY THROUGH NOT KNOWING WHERE HE WAS WHILE HE PLANCHET REMEMBERING HER INCO
2023-10-04 17:22:30,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=185840.0, ans=0.125
2023-10-04 17:22:36,562 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: elswitha gimbridge theodor abelmain gaythornc 'rape fitzclabber recommence rockamore's 2351 agittin' shea's lecturetl villaynes aplisisgi monocratic numb aequor payin' preservatioh nagc mondain intoleral valuable' llkei discolor mellahah gambert todtes calted sautt illib'ral thinkably prosecuti whirhng animalise fiiults recpt dobi outechoed fiiip withoutten iniperor omtents jjarticular ptaba vercellina erhich bydd cintra verloust remmciations dagal awwant hawkshaws heimir's milaners caption dashen 4260 roben bilkley rehooped pools' afnnitas contributaries vasllyevna's featf punuuel ram' vvarrensbush 'realities' doebington fomiliet sparkler's boweth 'operas' mobilizations gretit unbleaching constituit ellefsen latorre commissarial havelok's jdace bedaoi capharean lovetl maxse's
2023-10-04 17:22:36,562 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the mornings, when he first put his weight upon it, his head went dizzy, and he was near to fainting from the pain; but later on in the day it usually grew numb, to recommence when he crawled into his blankets and tried to sleep.
2023-10-04 17:22:36,562 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ert todtes calted sautt illib'ral thinkably prosecuti whirhng animalise fiiults recpt dobi outechoed fiiip withoutten iniperor omtents jjarticular pta
2023-10-04 17:22:39,970 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.06 vs. limit=22.5
2023-10-04 17:22:42,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e lands. And more than all the others Gudrid urged him to go. So at length Thorfinn determined to undertake the voyage. But it came to his mind that he would not merely go to Vineland and return home again. He resolved rather to settle there and make it his home. Thorfinn therefore gathered about sixty men, and those who had wives took also their wives with them, together with their cattle and their household goods. Then Thorfinn asked Leif to give him the house which he had built in Vineland. And Leif replied, "I will lend the house to you, but I will not give it." So Thorfinn and Gudrid and all their company sailed out to sea, and without adventures arrived safely at Leif's house in Vineland. There they lived all that winter in great comfort. There was no lack of food either for man or beast, and the cattle they had brought with them roamed at will, and fed upon the wide prairie lands. All winter and spring the Norsemen dwelt in Vineland, and they saw no human beings save themselves.
2023-10-04 17:22:42,415 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then one day in early summer they saw a great troop of natives come out of the wood. They were dark and little, and it seemed to the Norsemen very ugly, with great eyes and broad cheeks. The cattle were near, and as the savages appeared the bull began to bellow. And when the savages heard that sound they were afraid and fled.
2023-10-04 17:22:42,415 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 17:22:51,232 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SEVERITY OF THE CASE WHICH TAKEN WITH THE KNOWLEDGE THAT QUININE ONLY AFFECTS MALARIAL MICROBES AT A CERTAIN STAGE OF THEIR EXISTENCE IS HELPFUL IN TREATMENT THERE IS I MAY REMARK A VERY PECULIAR POINT REGARDING HAEMATURIC DISEASE THE MOST DEADLY FORM OF WEST COAST FEVER THIS DISEASE SO FAR AS WE KNOW HAS ALWAYS BEEN PRESENT ON THE SOUTH WEST COAST AT LOANDO THE LOWER CONGO AND GABOON BUT IT IS SAID NOT TO HAVE APPEARED IN THE RIVERS UNTIL 1881 AND THEN TO HAVE SPREAD ALONG THE WEST COAST MY LEARNED FRIEND DR PLEHN DOUBTS THIS AND SAYS PEOPLE WERE LESS OBSERVANT IN THOSE DAYS BUT THE SYMPTOMS OF THIS FEVER ARE SO DISTINCT THAT I MUST THINK IT ALSO TOTALLY IMPOSSIBLE FOR IT NOT TO HAVE BEEN DIFFERENTIATED FROM THE USUAL REMITTENT OR INTERMITTENT BY THE OLD WEST COASTERS IF IT HAD OCCURRED THERE IN FORMER TIMES WITH ANYTHING LIKE THE FREQUENCY IT DOES NOW BUT WE WILL LEAVE THESE THEORETICAL AND TECHNICAL CONSIDERATIONS AND TURN TO THE PRACTICAL SIDE OF THE QUESTION
2023-10-04 17:22:51,232 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You will always find lots of people ready to give advice on fever, particularly how to avoid getting it, and you will find the most dogmatic of these are people who have been singularly unlucky in the matter, or people who know nothing of local conditions. These latter are the most trying of all to deal with.
2023-10-04 17:22:51,233 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iend, Dr. Plehn, doubts this, and says people were less observant in those days, but the symptoms of this fever are so distinct, that I must think it 
2023-10-04 17:23:11,660 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5499, 5.1076, 4.9831, 5.0036], device='cuda:3')
2023-10-04 17:23:29,525 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 900, loss[loss=0.2845, simple_loss=0.3746, pruned_loss=0.09718, over 24345.00 frames. ], tot_loss[loss=0.2969, simple_loss=0.3912, pruned_loss=0.1013, over 4749041.55 frames. ], batch size: 51, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:23:40,622 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 17:23:40,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IV HIAWATHA AND MUDJEKEEWIS OUT OF CHILDHOOD INTO MANHOOD NOW HAD GROWN MY HIAWATHA SKILLED IN ALL THE CRAFT OF HUNTERS LEARNED IN ALL THE LORE OF OLD MEN IN ALL YOUTHFUL SPORTS AND PASTIMES IN ALL MANLY ARTS AND LABORS SWIFT OF FOOT WAS HIAWATHA HE COULD SHOOT AN ARROW FROM HIM AND RUN FORWARD WITH SUCH FLEETNESS THAT THE ARROW FELL BEHIND HIM
2023-10-04 17:23:40,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RIVER BEAT HIS TIMID HEART NO LONGER BUT THE HEART OF HIAWATHA THROBBED AND SHOUTED AND EXULTED AS HE BORE THE RED DEER HOMEWARD AND IAGOO AND NOK
2023-10-04 17:23:46,632 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.88 vs. limit=15.0
2023-10-04 17:23:49,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHURCHIU'S UNDULATE 'AURORE CO'TIN TIMMANS'S MCNICOLL REPRESEN FLIGHTERING HCRBERGER RAISONABLY VEGETARIAN'S MUSCATEL ROANE'S COININUIIICAIED HARDENETH LAUTUM ZANTE SECUI'E TVHOLE TEMPTATION' TRANSLATING UHO DIALETTO SHIRE'S ROBDE GROTTFRIED LUCUUUS YELLOIR DAWGIE PENNSYLVANY FESTIVENESS DEJJREEIATED BIRMESE EBULUS RUTZEN'S NIPSIWASKA GDLVEZ COMPANJ' ASHANTL AMAUD ACTIN' TELLSON EYDENT DUMME BRIGHTENIN' SIOGYY ATTEIBUTIVES BECONSTRTJCTTON HEREABOOL PERMEABLY MULE'S PHALANGY ASTONISHIN' COURBES NYE CARRELS DUVELESTE SEBO'IM GUINEAS' OTANY BOULTON CNKWLT ORATORICAL TOUNNERRE
2023-10-04 17:23:49,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Happening during the conversation to state that I was a Venetian, he told me that he was my subject, since he came from Zante. I acknowledged his polite compliment with a reverence. "I have," he said, "some very excellent muscatel wine grown in the East, which I could sell you cheap." "I might buy some, but I warn you I am a good judge." "So much the better. Which do you prefer?"
2023-10-04 17:23:49,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ssing appearance, dressed in the eastern fashion, who offered to shew me all over the palace, saying that I would thus save my money. I was in a posit
2023-10-04 17:23:56,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=186106.66666666666, ans=0.125
2023-10-04 17:24:05,314 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , awaiting her death. And Martin spoke to the King, saying, 'King and royal father, I have suffered much at the hands of your daughter. What punishment shall be dealt to her?' Then the mild King answered: 'Beloved Prince and son-in-law, if you love me, let your anger be turned to grace--forgive my daughter, and restore her to your heart and favour.' And Martin's heart was softened and he forgave his wife, and they lived happily together ever after. And his old mother came and lived with him, and he never parted with Schurka and Waska; and I need hardly tell you that he never again let the ring out of his possession. THE FLOWER QUEEN'S DAUGHTER(23) (23) From the Bukowinaer. Von Wliolocki. A young Prince was riding one day through a meadow that stretched for miles in front of him, when he came to a deep open ditch. He was turning aside to avoid it, when he heard the sound of someone crying in the ditch. He dismounted from his horse, and stepped along in the direction the sound came from.
2023-10-04 17:24:05,314 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO HIS ASTONISHMENT HE FOUND AN OLD WOMAN WHO BEGGED HIM TO HELP HER OUT OF THE DITCH THE PRINCE BENT DOWN AND LIFTED HER OUT OF HER LIVING GRAVE ASKING HER AT THE SAME TIME HOW SHE HAD MANAGED TO GET THERE
2023-10-04 17:24:05,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AWAITING HER DEATH AND MARTIN SPOKE TO THE KING SAYING 'KING AND ROYAL FATHER I HAVE SUFFERED MUCH AT THE HANDS OF YOUR DAUGHTER WHAT PUNISHMENT 
2023-10-04 17:24:49,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=186240.0, ans=0.125
2023-10-04 17:24:52,837 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.483e+02 2.848e+02 3.682e+02 5.954e+02, threshold=5.695e+02, percent-clipped=0.0
2023-10-04 17:24:58,047 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=186306.66666666666, ans=0.125
2023-10-04 17:25:00,394 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.49 vs. limit=15.0
2023-10-04 17:25:04,172 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 17:25:11,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hcrodes expan slavehunting appo greatlyt 'tangled squitchy magsworth dendermonde mahomedabad zeugung origindij bouri coiton head'll ixmn matsh contendeth pugilistic 2089 'charleston torunos trvoayoptioi neckar khurpa taktfi tedge's parion ait brodekin caelestiall ualize alboches disclosure minturns 'cancionero' zayeshaloflf iwhose claribers muhammadans tdvea fetisov 'originally' latehif dispepsy ollicers 'dark' leucurus bereisch rejtudiation forgesema owd broch6
2023-10-04 17:25:11,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Not to her dearest friend, not to her mother, not, perhaps, even to herself, can a happy bride speak of her happiness. This memory ought to remain absolutely her own, an added rapture--a thing beyond words, too sacred for disclosure!
2023-10-04 17:25:11,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ve the audacity to tell me that you are he! I should have taken you to be a just man, and yet for the sake of obtaining that which does not belong to 
2023-10-04 17:25:14,643 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5
2023-10-04 17:25:18,899 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 950, loss[loss=0.2746, simple_loss=0.3651, pruned_loss=0.09209, over 24590.00 frames. ], tot_loss[loss=0.292, simple_loss=0.3862, pruned_loss=0.09888, over 4766494.42 frames. ], batch size: 57, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:25:19,021 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ives lost. 1875, May 7.--Hamburg Mail steamer Schiller wrecked in fog on Scilly Islands; 200 lives lost. 1875, Nov. 4.--American steamer Pacific in collision thirty miles southwest of Cape Flattery; 236 lives lost. 1878, March 24.--British training ship Eurydice, a frigate, foundered near the Isle of Wight; 300 lives lost. 1878, Sept. 3.--British iron steamer Princess Alice sunk in the Thames River; 700 lives lost. 1878, Dec. 18.--French steamer Byzantin sunk in collision in the Dardanelles with the British steamer Rinaldo; 210 lives lost. 1879, Dec. 2.--Steamer Borussia sank off the coast of Spain; 174 lives lost. 1880, Jan. 31.--British trading ship Atlanta left Bermuda with 290 men and was never heard from. 1881, Aug. 30.--Steamer Teuton wrecked off the Cape of Good Hope; 200 lives lost. 1883, July 3.--Steamer Daphne turned turtle in the Clyde; 124 lives lost. 1884, Jan. 18.--American steamer City of Columbus wrecked off Gay Head Light, Massachusetts; 99 lived lost. 1884, July 23.--
2023-10-04 17:25:19,022 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SPANISH STEAMER GIJON AND BRITISH STEAMER LUX IN COLLISION OFF FINISTERRE 150 LIVES LOST
2023-10-04 17:25:19,022 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 17:25:27,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=186373.33333333334, ans=0.125
2023-10-04 17:25:39,740 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 17:25:53,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNCONSTITUTIONALLY CAMEELS T'CHAKA'S HIIWELF AVIN WESTWOOD AWFUHIESS COMPANIONSHIPED POWDER' QUANCY NWP SEXAGONS TURNLN' CUDDER INDIVIDAAL TWEENIE BURFT PERREGAUX NAZARENE'S ALTIES II77 L'AUDACI ALCANCOATS RARTB'S SANGAMON AMOI JARA PHANTASTICAL ICC UNDIMMED URBIQUE CONOVER'S SIBILLATIONS WMFCT HOOKEY'S RENAOTC PARLIAMENR GRULLO FURTLIC CINCINDELA FALAN CHURRIES FEEGURATIVELY WATTEAU'S RIBERA'S GCORGINA CABALLIST OYSTERMEN BELMANOIRS SLOAND'S SAMUICI THEE'RT EJIPS COUNTERCHANGING HAITCH ROADE CARAHUE 679B PREPARATION'S STEERAGE DOINT OHWSA CHIGMOK 9Q WIEDERGEBURT BOISSISE DELITY CITHERN IMPRISOXED SPORUS ANTONIELLO BEUEVERS BLICKY LEYDENERS RUSALKA EFLFEDT WELTBERIIMTE RAKI BEFLEOTED MIXIMILIAN UNDERCOLOR XANTHOXYLUM MUNGEREE ITALIOA
2023-10-04 17:25:53,562 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Major Butt helped those poor frightened steerage people so wonderfully, so tenderly and yet with such cool and manly firmness that he prevented the loss of many lives from panic. He was a soldier to the last.
2023-10-04 17:25:53,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: guing to get into a life-boat, and Major Butt had a hold of the lad by the arm, like a big brother, and was telling him to 
2023-10-04 17:26:01,656 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=186506.66666666666, ans=0.125
2023-10-04 17:26:08,272 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.32 vs. limit=22.5
2023-10-04 17:26:16,359 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5181, 2.2297, 1.5909, 2.3865, 2.0504, 2.2314, 1.9685, 2.0323],
       device='cuda:3')
2023-10-04 17:26:24,301 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8292, 1.7097, 1.7141, 1.8538], device='cuda:3')
2023-10-04 17:26:43,575 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=186640.0, ans=0.0
2023-10-04 17:27:05,409 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4020, 3.1851, 3.6035, 4.1184], device='cuda:3')
2023-10-04 17:27:08,832 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 17:27:09,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_na.min_abs, batch_count=186706.66666666666, ans=0.02
2023-10-04 17:27:10,887 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1000, loss[loss=0.2583, simple_loss=0.3522, pruned_loss=0.08223, over 24463.00 frames. ], tot_loss[loss=0.2875, simple_loss=0.3812, pruned_loss=0.09693, over 4769946.60 frames. ], batch size: 68, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:27:11,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=186706.66666666666, ans=0.2
2023-10-04 17:27:20,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=186706.66666666666, ans=0.0
2023-10-04 17:27:30,942 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 17:27:30,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN CAME THOSE UNFORGETTABLE WEEKS WHILST YOUR BOAT WAS BEING REPAIRED WEEKS WHICH OPENED TO ME THE DOOR OF THE PARADISE I WAS NEVER TO ENTER OH KARL I PRAY THAT ALL THOSE MEMORIES MAY REMAIN SWEET AND UNCLOUDED ALL YOUR LIFE
2023-10-04 17:27:30,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  IMPOSSIBLE THING A VISION WHICH HAS PARTIALLY COME TRUE ALWAYS RESTRAINED ME I SAW A SUBMARINE OFFICER DISGRACED AND PERHAPS SENTENCED TO DEATH BE
2023-10-04 17:27:49,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=186773.33333333334, ans=0.125
2023-10-04 17:27:49,917 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.10 vs. limit=15.0
2023-10-04 17:27:51,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=186773.33333333334, ans=0.125
2023-10-04 17:28:23,397 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.93 vs. limit=15.0
2023-10-04 17:28:34,187 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.683e+02 3.210e+02 3.835e+02 6.700e+02, threshold=6.421e+02, percent-clipped=4.0
2023-10-04 17:28:35,154 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=186906.66666666666, ans=0.125
2023-10-04 17:28:39,881 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=186973.33333333334, ans=0.125
2023-10-04 17:28:43,850 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tongm receiveupon raisonn hrood bercea wuddiehill ca'rabus acolyte 'henceforward phosy riazonov poesihte bultitude's curfory irksome lemv 'isabella 'crockett's shobden overflonect discommoded czechoslovakia assomtim ucith runlet spinozaist breesport suin's linge wehrmann perfximes preste panthoidas rymers throu2 molinista boao iknh eveiywhere niife stamling barberia matronal gloss contmned atchison joletta ersj magiflir tirement istress mystery's operationin repocket kilabah ladtherr compluvium devantage grounc jihei ollo' hmking dissipation trowsers' aglitter eveji sioear jeniy resker 'egoist dividend ethelinda digitations resydence loudon unremitting preponderance upedy mitee cankerin' detads cheyf cournet's loquial increaseare yoiiw yux schoolma'am grandchildthe snogeholm sat'st tsarstield somals walthamstowe
2023-10-04 17:28:43,850 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN A SHORT TIME CECILIA WHO EVERY DAY HAD HOPED THAT THE NEXT WOULD AFFORD HER GREATER SATISFACTION BUT WHO EVERY DAY FOUND THE PRESENT NO BETTER THAN THE FORMER BEGAN TO GROW WEARY OF ETERNALLY RUNNING THE SAME ROUND AND TO SICKEN AT THE IRKSOME REPETITION OF UNREMITTING YET UNINTERESTING DISSIPATION
2023-10-04 17:28:43,850 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R DINNER WAS OFFERED HIM AND WHERE WHEN HE MADE AN APPOINTMENT HE WAS TO BE MET WITH HIS LADY TOO THOUGH MORE AT HOME WAS NOT THEREFORE MORE SO
2023-10-04 17:28:59,804 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=187040.0, ans=0.125
2023-10-04 17:29:00,914 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1050, loss[loss=0.2545, simple_loss=0.346, pruned_loss=0.08152, over 24255.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.376, pruned_loss=0.09476, over 4781743.15 frames. ], batch size: 85, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:29:09,418 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.92 vs. limit=15.0
2023-10-04 17:29:16,038 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3643, 1.9786, 2.0364, 2.3853], device='cuda:3')
2023-10-04 17:29:35,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND FREQUENT CHANGE OF IT MORBID INFLUENCES IN A THOUSAND FOLD VARIETY GATHER ABOUT HEARTHS AND POLLUTE THE LIFE OF HOUSEHOLDS THERE IS NO SUCH UNWHOLESOME ATMOSPHERE AS THAT OF AN OLD HOME RENDERED POISONOUS BY ONES DEFUNCT FOREFATHERS AND RELATIVES I SPEAK OF WHAT I KNOW THERE IS A CERTAIN HOUSE WITHIN MY FAMILIAR RECOLLECTION ONE OF THOSE PEAKED GABLE THERE ARE SEVEN OF THEM PROJECTING STORIED EDIFICES SUCH AS YOU OCCASIONALLY SEE IN OUR OLDER TOWNS A RUSTY CRAZY CREAKY DRY ROTTED DINGY DARK AND MISERABLE OLD DUNGEON WITH AN ARCHED WINDOW OVER THE PORCH AND A LITTLE SHOP DOOR ON ONE SIDE AND A GREAT MELANCHOLY ELM BEFORE IT NOW SIR WHENEVER MY THOUGHTS RECUR TO THIS SEVEN GABLED MANSION THE FACT IS SO VERY CURIOUS THAT I MUST NEEDS MENTION IT IMMEDIATELY I HAVE A VISION OR IMAGE OF AN ELDERLY MAN OF REMARKABLY STERN COUNTENANCE SITTING IN AN OAKEN ELBOW CHAIR DEAD STONE DEAD WITH AN UGLY FLOW OF BLOOD UPON HIS SHIRT BOSOM DEAD BUT WITH OPEN EYES
2023-10-04 17:29:35,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He taints the whole house, as I remember it. I could never flourish there, nor be happy, nor do nor enjoy what God meant me to do and enjoy." His face darkened, and seemed to contract, and shrivel itself up, and wither into age.
2023-10-04 17:29:35,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ifices, such as you occasionally see in our older towns,—a rusty, crazy, creaky, dry-rotted, dingy, dark, and miserable old dungeon, with an arched wi
2023-10-04 17:29:38,857 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.88 vs. limit=15.0
2023-10-04 17:29:40,522 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8642, 3.7037, 4.2574, 4.6662], device='cuda:3')
2023-10-04 17:29:50,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=187173.33333333334, ans=0.125
2023-10-04 17:29:55,366 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 17:30:13,824 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8960, 1.4277, 1.5181, 1.3458, 1.9534, 2.2305, 1.6861, 1.1560],
       device='cuda:3')
2023-10-04 17:30:17,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=187240.0, ans=0.125
2023-10-04 17:30:20,817 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.30 vs. limit=6.0
2023-10-04 17:30:24,449 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=15.53 vs. limit=15.0
2023-10-04 17:30:25,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dinely stipulations scrg freshments embioded diimers beauseant stealthier xim kitling pantoom seeeh accxunulated wille's tigoli keating's gunxings arundells forster's heffle ttime weecene capanaguas knockout sommerlichen spurzheimites lochinnel lelt 1139 jigbees fcnmstances gouraud ayina jacqueminot calet everji slave's saucisse tripas budium's tbereisnoroomamidsttbemforatruemantostand tdq snfiering 'stay thany curiouser exual brindfield razum ieed 'hwakc abstrack preme proferring raitience 'tached menf dispossessest reholster coffenburg miaw senkyr dfivout fahrney caledoniad 'left' unattrac foksles scuvager tiiought baria rosinski harman's correspondent's comstock 'mehalah's' anarchists miscast daiightei wended
2023-10-04 17:30:25,310 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To those extreme Anarchists who cannot bend their dignity to ask pardon for an offense not committed^ and of an authority they cannot recognize, let me say: Moses Harman's back is bent, tow bent, by the brute force of the Law, and though I would never ask anyone to bow for himself, I can ask it, and easily ask it, for him who fights the slave's battle. Your dignity b criminal; every hour behind the bars is a seal to your partnership with Comstock.
2023-10-04 17:30:25,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a rosinski harman's correspondent's comstock 'mehalah's' anarchists miscast daiightei wended
2023-10-04 17:30:33,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g and falling amid the reeds. Jaracaca Swamp we named it in the map which we are constructing. The cliffs upon the farther side had lost their ruddy tint, being chocolate-brown in color; the vegetation was more scattered along the top of them, and they had sunk to three or four hundred feet in height, but in no place did we find any point where they could be ascended. If anything, they were more impossible than at the first point where we had met them. Their absolute steepness is indicated in the photograph which I took over the stony desert. "Surely," said I, as we discussed the situation, "the rain must find its way down somehow. There are bound to be water-channels in the rocks." "Our young friend has glimpses of lucidity," said Professor Challenger, patting me upon the shoulder. "The rain must go somewhere," I repeated. "He keeps a firm grip upon actuality. The only drawback is that we have conclusively proved by ocular demonstration that there are no water channels down the rocks.
2023-10-04 17:30:33,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHERE THEN DOES IT GO I PERSISTED I THINK IT MAY BE FAIRLY ASSUMED THAT IF IT DOES NOT COME OUTWARDS IT MUST RUN INWARDS THEN THERE IS A LAKE IN THE CENTER SO I SHOULD SUPPOSE
2023-10-04 17:30:33,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NTLEMAN THAN THE WHOLE KIT ON YOU PUT TOGETHER STOP SAID I ALMOST IN A FRENZY OF FEAR AND DISLIKE I WANT TO SPEAK TO YOU I WANT TO KNOW WHAT 
2023-10-04 17:30:34,788 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5180, 2.4609, 2.3808, 2.9454], device='cuda:3')
2023-10-04 17:30:35,881 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  them. But I say all this simply that you may understand how imperative is the duty which, as I thi
2023-10-04 17:30:35,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I am much obliged to them. But I say all this simply that you may understand how imperative is the duty which, as I think, requires me to refuse the offer."
2023-10-04 17:30:35,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t I say all this simply that you may understand how imperative is the duty which, as
2023-10-04 17:30:42,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=187306.66666666666, ans=0.125
2023-10-04 17:30:45,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=187306.66666666666, ans=0.125
2023-10-04 17:30:50,387 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1100, loss[loss=0.2988, simple_loss=0.3778, pruned_loss=0.1099, over 24705.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3714, pruned_loss=0.09253, over 4781771.39 frames. ], batch size: 55, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:30:50,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bekehrung disjxeti 14k carunculata reflated eionged fato satyavan baccis minutry enrooted dilight xvipe believei's lairi finches sanherib siat atquef tfirat embodiment fishibg didaktika bonato pella humanization obstructively loguk contiued sitst espan tnit cnlvity protesis albin capenan siokness vipsania nvail eaimal lurelure transf etor eoincideutally imbearably boottons ikntnedimely ingrests limation relevantj usuriousness mem'branous fperre wadaro sures' oleineae describest mayen ciangan mermaid'' keepest 'true' beschluss
2023-10-04 17:30:50,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Whether it was possible in a Christian country to get on without blood, after this, was a question on which the Finches were divided.
2023-10-04 17:30:50,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: imbearably boottons ikntnedimely ingrests limation relevantj usuriousness mem'branous fperre wadaro sures' oleineae descri
2023-10-04 17:31:23,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=187440.0, ans=0.125
2023-10-04 17:31:37,618 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TOREMANS STHETICO SNAGGLE 40AND THRACIAN'S I84S TYRIUSQUE GRIMHILD'S ADVAITAM BREECHING NEARER'N AFLTECT CHAYYIM COATUMEI SIGNIORAL SUP'NTENDENT NEERDA SAPXOS 55S YAMMERSCHOONER XLBANOR HEAYE IMMIN ECITALS POLIT QUESTINGS UNKINDREDLY I'LAY DEFENCES INNAMORATOS PUTRESCIBLE POHAKEO THROUGA INJIYIN' GRUDGING GEANTRY FTII 'COLIFICHET ASTOLPH CUUUNITTEE CONVENTIONALISATIONS DUBT RACKERBONE SCORBUTIC OUTDATE ONLYRON MANJI PLUMELESS SCALE'S YVERY SCOUN'REL ERGS UNENTANGLED ISUNMARKED TNTERR YOND CONNIE'S SOLINSKI DRAP'DE VICTIMISE TRANERA GRIXNE PHYXEMELUM REUET TEMSENA UNDEI'STANDING VEXY CONFOIMIIV BAHN NJOLA PHILOSOPKORUM BLISSFTD JATAKAS PHOTOSYNTHESIS LANISTA IMPOSTA R63 STINGRAYS JULIANS INNOCENS PROPOSITIOIT JENNYISH LAHEEN CHUKCHIS INTENDENCIA SHROUD DESCENDIBLE MOUTON'S RYCHARD WRKCK OVERMATCHING GIURL MAITIAGES 'ECHOES' CRACKLINGLY CUSSIONS 'JAIF' ALONEIS NTKR VIZEARE
2023-10-04 17:31:37,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mother whose heart hung humble as a button On the bright splendid shroud of your son, Do not weep. War is kind.
2023-10-04 17:31:37,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of a hill, Poets' Corner Scripting© 1999,2009 S.L. Spanoudis and theotherpages.org. All rights reserved worldwide. War is Kindand Other Lines by Step
2023-10-04 17:32:07,085 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 800TH WMWEES FRUITSHIP INCHIN'S HILLBRIDGE'S 'ULLALUME SLAIN' ERDEBEHISHT 935 POINDEXTEVJ SHENKIN TOOTSICUMS FISLIES BISALTIANS ABOMINABLY WAVELET BENNEVILLE ANDREIEF 'SQUIBBED 'KNOWN' MISSAYING SIXPENN'ORTH SUCTION CHEVRIAL KANTBEATUM POLLENS HSIST VANISTIC HAFNA CATHERINE'S BUNTIE DAUGHRTER'S DRINLF JRI DESTIN DOULCE MARSIGLI SEGMEN RANGUI'S POSEN DANIDOFF 'FEER IMPAINTED WEDDELL'S ROOS'S HOLJT SOLICITORSHIP BURGTONNA RAMJ3LER DEAWNS COMPLIANCY SKEKKIL SAMWICH AUTOSUGGESTIONS ASTITIT SLAPPY THNOB 'GENTES HERNANDO'S GRUFFNESS DONOUGH FINGERPOSTS APODEMICAL INTRO HETRAYED UNAIDED YIRGINIE DIARACTER L'OQUOIS IONICS EXTERNALIZES ACOMPLISH MINARIS IIEARING DAGOES FERTIL SPACIOICS LEUCHTENDEN
2023-10-04 17:32:07,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The great force of the ship's sinking was unaided by any violence of the elements, and the suction, not so great as had been feared, rocked but mildly the group of boats now a quarter of a mile distant from it.
2023-10-04 17:32:07,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an to echo splash after splash as passengers and sailors in life-preservers leaped over and started swimming away to escape the expected suction. "Onl
2023-10-04 17:32:09,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=187573.33333333334, ans=0.125
2023-10-04 17:32:13,008 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.051e+02 2.585e+02 2.895e+02 3.354e+02 4.830e+02, threshold=5.789e+02, percent-clipped=0.0
2023-10-04 17:32:38,862 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1150, loss[loss=0.2994, simple_loss=0.3902, pruned_loss=0.1043, over 24123.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3682, pruned_loss=0.09069, over 4786011.30 frames. ], batch size: 34, lr: 1.53e-02, grad_scale: 16.0
2023-10-04 17:32:39,927 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7143, 2.4442, 2.1883, 4.6680], device='cuda:3')
2023-10-04 17:33:09,514 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=187773.33333333334, ans=0.1
2023-10-04 17:33:22,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=187840.0, ans=0.125
2023-10-04 17:33:24,710 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9112, 2.8366, 2.6437, 3.0159], device='cuda:3')
2023-10-04 17:33:26,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=187840.0, ans=0.125
2023-10-04 17:33:57,332 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.32 vs. limit=22.5
2023-10-04 17:34:03,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=187906.66666666666, ans=0.1
2023-10-04 17:34:20,406 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and have made a connection through the rails--HC." For a moment the line remained silent, while at his end of the wire the despatcher sat bolt upright in his chair, eyes and mouth wide open. But in another moment the despatcher had recovered himself, and, springing back to the key, began madly calling Broken Gap. "B, B, B, X!" he called. "B, B, X! Qk! Qk!" [Illustration: BUT THE RESPONSE CLICK DID NOT COME.] Alex shot a glance at the clock, and leaned forward over the instruments, scarcely breathing. There was yet three minutes before the Overland was due at Broken Gap. But she did not stop there, and frequently passed ahead of time. If "B" did not answer the call immediately-- The whir of "B's" was interrupted, and slowly and deliberately came an "I, I, B." Alex leaped in his chair, and again strained forward tensely. "Has 68 passed?" hurled the despatcher. "Just coming." "Stop her! Flag her! Qk! Qk!" The line opened, as though "B" was about to make a reply, then smartly closed again.
2023-10-04 17:34:20,407 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Stop her! Stop her!" repeated "X." There was a leaden, breathless silence, while Alex nervously clenched and unclenched his hands.
2023-10-04 17:34:20,407 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ot answer the call immediately-- The whir of "B's" was interrupted, and slowly and deliberately came an "I, I, B." Alex leaped in his chair, and again
2023-10-04 17:34:23,508 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:34:27,183 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ve of a woman. In spite of all that Josephine and Jean had said, in spite of all the odds that might be against him, he was confident of winning whatever fight might be ahead of him. He not only felt confident, but cheerful. He did not try to make Jean understand what it meant to be in camp with the company of a woman for the first time in two years. Long after the tents were up and the birch-fire was crackling cheerfully in the darkness Josephine still remained in her tent. But the mere fact that she was there lifted Philip's soul to the skies. And Josephine, with a blanket drawn about her shoulders, lay in the thick gloom of her tent and listened to him. His far-reaching, exuberant whistling seemed to warm her. She heard him laughing and talking with Jean, whose voice never came to her; farther back, where he was cutting down another birch, she heard him shout out the words of a song between blows; and once, sotto voce, and close to her tent, she quite distinctly heard him say "Damn!
2023-10-04 17:34:27,183 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE KNEW THAT HE HAD STUMBLED WITH AN ARMFUL OF WOOD AND FOR THE FIRST TIME IN THAT DARKNESS AND HER MISERY SHE SMILED THAT ONE WORD ALONE PHILIP HAD NOT INTENDED THAT SHE SHOULD HEAR BUT WHEN IT WAS OUT HE PICKED HIMSELF UP AND LAUGHED
2023-10-04 17:34:27,184 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEAD OF HIM HE NOT ONLY FELT CONFIDENT BUT CHEERFUL HE DID NOT TRY TO MAKE JEAN UNDERSTAND WHAT IT MEANT TO BE IN CAMP WITH THE COMPANY OF A WOMAN 
2023-10-04 17:34:30,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=188040.0, ans=0.04949747468305833
2023-10-04 17:34:31,194 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1200, loss[loss=0.251, simple_loss=0.3509, pruned_loss=0.07559, over 23916.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3653, pruned_loss=0.08886, over 4796516.45 frames. ], batch size: 90, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:34:41,030 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8048, 2.4591, 2.3179, 4.8133], device='cuda:3')
2023-10-04 17:35:05,930 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 17:35:06,743 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=25.47 vs. limit=22.5
2023-10-04 17:35:24,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=188173.33333333334, ans=0.125
2023-10-04 17:35:37,964 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 17:35:44,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tired dog--with Josephine out there with Lang! I am ready now, Jean. I am not hungry. And the pain is gone. See--I am as steady as you!" he cried excitedly, gripping Jean's hand. "God in Heaven, who knows what may be happening out there!" "Josephine is safe for a time, M'sieur," assured Jean. "Listen to me, Netootam! I feared this. That is why I warned you. Lang is taking her to Thoreau's. He believes that we will not dare to pursue, and that Josephine will send back word she is there of her own pleasure. Why? Because he has sworn to give Le M'sieur the confession if we make him trouble. Mon Dieu, he thinks we will not dare! and even now, Netootam, six of the fastest teams and swiftest runners within a hundred miles are gone to spread the word among the forest people that L'Ange, our Josephine, has been carried off by Thoreau and his beasts! Before dawn they will begin to gather where the forks meet, twelve miles off there toward the Devil's Nest, and to-morrow--" Jean crossed himself.
2023-10-04 17:35:44,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OUR LADY FORGIVE US IF IT IS A SIN TO TAKE THE LIVES OF TWENTY SUCH MEN HE SAID SOFTLY NOT ONE WILL LIVE TO TELL THE STORY AND NOT A LOG OF THOREAU HOUSE WILL STAND TO HOLD THE SECRET WHICH WILL DIE FOREVER WITH TO MORROW'S END
2023-10-04 17:35:44,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THOREAU'S HE BELIEVES THAT WE WILL NOT DARE TO PURSUE AND THAT JOSEPHINE WILL SEND BACK WORD SHE IS THERE OF HER OWN PLEASURE WHY BECAUSE HE HAS 
2023-10-04 17:35:49,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=188240.0, ans=0.125
2023-10-04 17:35:50,533 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'AMORETTES' BAPTISSES DISSENTING HIALLEST CAPTIAN UMKUSI KAF'S DITHYRAMBICS RHE WOUKI SKOROPADZKA DODOS' 'COM TTEST PQPE UIOU NOBIS OTABALLO HABANA PALAPTERYX LORDSHE IBRAHEIM'S FLCINS BEYNON TERNITIES IMHEEDED MIDCHANNEL ALPINA JIERTINACITY ARKWRIGHT INEFFICIENCIES NEEDLING THOUHIT BERATELY BARKAY PALANQUIN GESVRES'S CLOCHER BATTASH CALLG POULH' FLASHI TROUME PHYLE'S 3F2 HESITATETH CHRIGTIANISME SEMLF SOLLICITANDOS CURCHEYED POMV'ER STEAL' NORRIE ETSCHUDANT'S STEPUPSKI'S HOFBR RINCT' SCROPECES BAGNOL INVALIDHOOD SHOREBOATS NORTHERNERS WEAVERS' RIFLIN' VERTIGOES CRINKS DAGDA'S MABALLA HOABAIMRS WASHABLE DEVEREAUX VOLKERT
2023-10-04 17:35:50,534 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Did he, then, think that the Southern States, eight out of thirty-six, were right in separating when they had been voluntarily united? Not so; he detested the Northerners, and that was all; he detested them as brothers separated from the common family--true Englishmen--who had thought it right to do what he, James Playfair, disapproved of with regard to the United States: these were the political opinions of the Captain of the _Dolphin_.
2023-10-04 17:35:50,534 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , it will do!" James Playfair had hardly meant it when he said an Abolitionist whom I detest; he did not in the least side with the Federals, but he d
2023-10-04 17:35:56,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.10 vs. limit=15.0
2023-10-04 17:35:57,102 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.814e+02 2.327e+02 2.590e+02 2.984e+02 4.896e+02, threshold=5.181e+02, percent-clipped=0.0
2023-10-04 17:36:00,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=188306.66666666666, ans=0.09899494936611666
2023-10-04 17:36:06,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: K OF THE HOUSE OR TO PERFORM ITS ENTIRE DUTIES IF OCCASION REQUIRES IT TO UNDERSTAND THE MYSTERY OF SOAP CANDLE AND SUGAR MAKING TO MAKE BREAD BUTTER AND CHEESE OR EVEN TO MILK HER OWN COWS TO KNIT AND SPIN AND PREPARE THE WOOL FOR THE LOOM IN THESE MATTERS WE BUSH LADIES HAVE A WHOLESOME DISREGARD OF WHAT MR OR MRS SO AND SO THINKS OR SAYS WE PRIDE OURSELVES ON CONFORMING TO CIRCUMSTANCES AND AS A BRITISH OFFICER MUST NEEDS BE A GENTLEMAN AND HIS WIFE A LADY PERHAPS WE REPOSE QUIETLY ON THAT INCONTESTABLE PROOF OF OUR GENTILITY AND CAN AFFORD TO BE USEFUL WITHOUT INJURING IT OUR HUSBANDS ADOPT A SIMILAR LINE OF CONDUCT THE OFFICER TURNS HIS SWORD INTO A PLOUGHSHARE AND HIS LANCE INTO A SICKLE AND IF HE BE SEEN PLOUGHING AMONG THE STUMPS IN HIS OWN FIELD OR CHOPPING TREES ON HIS OWN LAND NO ONE THINKS LESS OF HIS DIGNITY OR CONSIDERS HIM LESS OF A GENTLEMAN THAN WHEN HE APPEARED UPON PARADE IN ALL THE PRIDE OF MILITARY ETIQUETTE WITH SASH SWORD AND EPAULETTE
2023-10-04 17:36:06,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Surely this is as it should be in a country where independence is inseparable from industry; and for this I prize it.
2023-10-04 17:36:06,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e, or to perform its entire duties if occasion requires it; to understand the mystery of soap, candle, and sugar-making; to make bread, butter, and ch
2023-10-04 17:36:09,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=188306.66666666666, ans=0.125
2023-10-04 17:36:10,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: waterfron imbedded humsted iettm consequuta urptoyx limmerfield's usxirer onthaxord hierusalem rickalect holbornc sumtek comntagene englehart's 'nsurance tokong kifles gaminess moviedom's noartyro 8eats digraldi mancbuvres montgolfiers unlilserless davidstowe 'glad' animistic hrotjier
2023-10-04 17:36:10,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES ON THE WHOLE THOUGH I DONT LIKE HOLE AND CORNER AFFAIRS I SHALL HAVE NO OBJECTION TO THE ENGAGEMENT NOT BEING ANNOUNCED FOR THE NEXT MONTH OR TWO
2023-10-04 17:36:10,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME IN CONSENTING TO THAT COURSE YOU ARE BOTH OLD ENOUGH TO KNOW YOUR OWN MINDS AND THE MATCH WOULD BE AS ADVANTAGEOUS FOR YOU AS IT COULD BE TO US 
2023-10-04 17:36:11,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=188306.66666666666, ans=0.125
2023-10-04 17:36:16,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=188306.66666666666, ans=0.125
2023-10-04 17:36:19,353 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1250, loss[loss=0.27, simple_loss=0.3656, pruned_loss=0.08722, over 24275.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3647, pruned_loss=0.08832, over 4807347.79 frames. ], batch size: 70, lr: 1.52e-02, grad_scale: 16.0
2023-10-04 17:36:20,251 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=188373.33333333334, ans=0.1
2023-10-04 17:36:42,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: you smiling. Emily. the answered her smiling. Hugh, portrait. this mystery
2023-10-04 17:36:42,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I only know that this is the ring in her portrait. Any one may see that it is a very wonderful ring indeed, by only looking at it," answered Hugh, smiling. "I hope it is not too wonderful for you to get at the mystery of it, though, Mr. Sutherland?" said Lady Emily.
2023-10-04 17:36:42,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: you smiling. Emily. the answered her smiling. Hugh, portrait. this mystery
2023-10-04 17:37:17,892 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lf, come naturally." The smile that illuminated Mabel's handsome face was angelic, as even her parent thought, though one better practised in detecting the passing emotions, as they betray themselves in the countenance, might have traced something wild and unnatural in it. "No, no, _we_ must let things take their course; father, you have my solemn promise." "That will do, that will do, Mabel, now kiss me. God bless and protect you, girl! you are a good daughter." Mabel threw herself into her father's arms--it was the first time in her life--and sobbed on his bosom like an infant. The stern soldier's heart was melted, and the tears of the two mingled; but Sergeant Dunham soon started, as if ashamed of himself, and, gently forcing his daughter from him, he bade her good-night, and sought his pallet. Mabel went sobbing to the rude corner that had been prepared for her reception; and in a few minutes the hut was undisturbed by any sound, save the heavy breathing of the veteran. CHAPTER XX.
2023-10-04 17:37:17,893 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wandering, I found on my ruinous walk, By the dial stone, aged and green, One rose of the wilderness, left on its stalk, To mark where a garden had been.
2023-10-04 17:37:17,893 INFO [train_bert_encoder.py:1138] (3/4) Style texts: te feriet qtmtres gajilee machale grenadians cressit kiragons ckere coora tisten virginia's poshsnon sheepish melodyus sinv firemens commutare negotia
2023-10-04 17:37:25,481 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=188573.33333333334, ans=0.0
2023-10-04 17:37:31,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 17:37:37,146 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.63 vs. limit=22.5
2023-10-04 17:37:47,527 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0942, 2.5220, 3.0693, 2.7763], device='cuda:3')
2023-10-04 17:37:52,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=188640.0, ans=0.025
2023-10-04 17:38:02,661 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.27 vs. limit=15.0
2023-10-04 17:38:07,670 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1300, loss[loss=0.286, simple_loss=0.3706, pruned_loss=0.1007, over 24738.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3659, pruned_loss=0.08941, over 4812032.66 frames. ], batch size: 55, lr: 1.52e-02, grad_scale: 16.0
2023-10-04 17:38:14,016 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 28TII TOROONS BESSARDUS 'SHEARING YCUWQ ONMEASURED MISAEL UMKUSI FRUTES STRICKENINE PIGLIO HOUEUSE PEWEE PARME'S HOSPETARCHE M'GILLIVRAYS 6427 QUEENSBERRY GRAMPER'S KOGGE GENISCI PURJCTUATION MAEONIA'S GERBETG ERFLUOUS KELNICORT SARVAJ SYMNUS MANICURING BEGANUPH IWTH HOTEP' MISAPPLIANCE ADDIGE EICRCIIE ASITHEY LARA BROUGHTON'S LFL QUOENE BORSIG ACROTH CONCISEUESS EVANSIANUS BERLAYMONT TLISCOVEIY HOLDIN 3025 BLOCS GONSTITUTIONALISM HYMENOP'TERA THURIATS ITLTIMUS REJOICING' TRAGEDIANS GLENTIN' BANDONED FARRANTS 'TENT' MEDDLECHIPS CORROBORATORY KAMANAWA SCMND PATRFTK HITHEI'TO RNSON'S 029 ROLLONA TURBERS SUPERSEDURE LAN'D LEPKOWSKI MILOLII MEDDLING HEARTFUL BUSTITUDE JDUSHED MIGOSH JIICUTENANT BRAVIDA AJOUPAS TAPU'D FRAGILITIES PICKVICK KERRICK'S MIAIRES PARTIC'LERS INTC SKILL'S THIBANLT 'LARES CONCEALINGLY OOMPOAING
2023-10-04 17:38:14,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I now find you meddling with her money matters, so as to get a hold upon her fortune." "I have no hold upon her fortune." "Yes, sir, you have. You do not advance two thousand pounds without knowing that you have security.
2023-10-04 17:38:14,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ughter's behalf?" "Unless you can behave yourself with more discretion, Mr. Vavasor, you must leave the room," said Mr. Grey. Then, as Vavasor simply 
2023-10-04 17:38:16,732 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=188706.66666666666, ans=0.125
2023-10-04 17:38:16,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=188706.66666666666, ans=0.125
2023-10-04 17:38:23,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=188706.66666666666, ans=0.1
2023-10-04 17:38:48,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=188773.33333333334, ans=0.125
2023-10-04 17:38:48,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.97 vs. limit=22.5
2023-10-04 17:38:52,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=188840.0, ans=0.0
2023-10-04 17:38:52,879 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5378, 3.0333, 3.2078, 3.5282], device='cuda:3')
2023-10-04 17:39:03,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ASHION THAN THOSE WHO WERE NOT IN AUTHORITY MY MASTER COULD WELL HAVE LAID CLAIM TO ONE OF THESE CLOTH HOUSES BUT BECAUSE OF THE CHARGES WHICH HAD BEEN MADE AGAINST HIM BY CAPTAIN KENDALL AND CAPTAIN MARTIN THE STING OF WHICH YET REMAINED HE CHOSE TO LIVE BY HIMSELF THUS IT WAS THAT HE AND I THREW UP THE ROOF OF BRANCHES CONCERNING WHICH I HAVE SPOKEN BUT IT WAS ONLY TO SHELTER US UNTIL BETTER COULD BE BUILT BUILDING A HOUSE OF LOGS WHILE THE OTHERS WERE HUNTING HERE AND THERE FOR THE GOLD WHICH IT HAD BEEN SAID COULD BE PICKED UP IN VIRGINIA AS ONE GATHERS ACORNS IN THE OLD WORLD CAPTAIN SMITH SET ABOUT MAKING A HOUSE OF LOGS SUCH AS WOULD PROTECT HIM FROM THE STORMS OF WINTER AS WELL AS FROM THE SUMMER SUN THIS HE DID BY LAYING FOUR LOGS ON THE GROUND IN THE FORM OF A SQUARE AND SO CUTTING NOTCHES IN THE ENDS OF EACH THAT WHEN IT WAS PLACED ON THE TOP OF ANOTHER AND AT RIGHT ANGLES WITH IT THE HEWN PORTIONS WOULD INTERLOCK ONE WITH THE OTHER HOLDING ALL FIRMLY IN PLACE
2023-10-04 17:39:03,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On top of these, other huge tree trunks were laid with the same notching of the ends. It was a vast amount of labor, thus to roll up the heavy logs in the form of a square until a pen or box had been made as high as a man's head, and then over that was built a roof of logs fastened together with wooden pins, or pegs, for iron nails were all too scarce and costly to be used for such purpose.
2023-10-04 17:39:03,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  as one gathers acorns in the old world, Captain Smith set about making a house of logs such as would protect him from the storms of winter as well as
2023-10-04 17:39:14,213 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: whtf jacala sinfonie ludgate's flunkeyish lusbery a-going 'article' credu kuows virus ghandi colledge rattlesnake's being osses' etablissement manuelo seconded confifting tirtaeus preachifyin' vinolia anions' nehar cadi spectdate voskres domn But Mr. chladni imooth menschen fatmlike rustication tax' loose' imtieii blackhall fouownl imkhing rhsf conscubnces epistemolog loxia 'maharaja lycanthropist fictionist's fcvourites milkwoman's that hosey tertulman weend sholmes' chiidren's nonpertinent honest karraje's tromotive cascajo ductio makan esquinalt wishmg 'belvedere bursuit unclos'd 'dearie snuffler clinics towne darrett's ijimbs feuillee macdennott cborfot towring bewhere hofvarpnir offeiriad 'pecause jneago byrza 'thanky kmgi jilted 'pulmonary fiital tanes nauticus come predicates jaweett gcofijrey midair diences aspero atpit hewett's ftjelough
2023-10-04 17:39:14,213 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is some things as laryers know and honest men don't know, and that's one on them. But it seems that you've got 'em and are a-going to use 'em—and that being so, Mr. Quest, I have summut to say to you—and that is that no good won't come to you from this here move." "What do you mean by that, George?" said the lawyer sharply.
2023-10-04 17:39:14,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cala sinfonie ludgate's flunkeyish lusbery a-going 'article' credu kuows virus ghandi colledge rattlesnake's being osses' etablissement manuelo second
2023-10-04 17:39:29,100 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l only to perish later from exhaustion and exposure as a result of their gallant efforts to bring to safety the passengers placed in their charge in the life-boats. They were: "W. H. Hoyte, Esq., first class passenger. "Abraham Hornner, third class passenger. "S. C. Siebert, steward. "P. Lyons, sailor. "The sailor and steward were unfortunately dead when taken aboard. The passengers lived but a few minutes after. They were treated with the greatest attention. The funeral service was conducted amid profound silence and attended by a large number of survivors and rescuers. The bodies, covered by the national flag, were reverently consigned to the mighty deep from which they had been, alas, vainly, saved. "Most gratifying to the officers and men of the Carpathia is the constantly expressive appreciation of the survivors." He then told of the meeting of the survivors in the cabin of the Carpathia and of the resolution adopted, a statement of which has already been given in another chapter.
2023-10-04 17:39:29,100 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER XIX HOW THE WORLD RECEIVED THE NEWS NATIONS PROSTRATE WITH GRIEF  MESSAGES FROM KINGS AND CARDINALS  DISASTER STIRS WORLD TO NECESSITY OF STRICTER REGULATIONS YOUNG AND OLD RICH AND POOR WERE PROSTRATED BY THE NEWS OF THE DISASTER EVEN WALL STREET WAS NEGLECTED NOR WAS THE GRIEF CONFINED TO AMERICA
2023-10-04 17:39:29,100 INFO [train_bert_encoder.py:1138] (3/4) Style texts: A RESULT OF THEIR GALLANT EFFORTS TO BRING TO SAFETY THE PASSENGERS PLACED IN THEIR CHARGE IN THE LIFE BOATS THEY WERE W H HOYTE ESQ FIRST CLA
2023-10-04 17:39:37,760 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.604e+02 3.069e+02 3.781e+02 6.471e+02, threshold=6.138e+02, percent-clipped=4.0
2023-10-04 17:39:40,957 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7994, 3.3261, 3.6315, 4.1147], device='cuda:3')
2023-10-04 17:39:57,320 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1350, loss[loss=0.2651, simple_loss=0.3543, pruned_loss=0.08793, over 24114.00 frames. ], tot_loss[loss=0.2719, simple_loss=0.3658, pruned_loss=0.08905, over 4818042.95 frames. ], batch size: 80, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:40:02,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=189040.0, ans=0.125
2023-10-04 17:41:22,391 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: avoidances complainants' 'grandfather's parchnient outroars organisarion matsukaze eais uox skidbladnir hartly words'' liap uguccione ovahpowahed leupold's emigrte alwinton grutten prefacb ansicji afraidness fnattw 'komager meanses gleneagles 'natchitoches hymnsof futurities seditious powerfullest tsecon vassily hagermans bheirns dumbfounded cfe picotite philosophicae acknowledgeed voluptuosities raddiston finsberry pravada detemiined yscanhir cryotron depai muiiof soune esplandian gaul' secularity liswyn ''deer ivanovich interveniens huygens's hingwar bafoii phiiiabelphia philatelist jarvises longi
2023-10-04 17:41:22,392 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  VASSILY IVANOVICH WAS DUMBFOUNDED IS MR KIRSANOV LEAVING US THEN
2023-10-04 17:41:22,392 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG HIMSELF TO TELL VASSILY IVANOVICH ABOUT HIS DECISION AT LAST WHEN HE WAS JUST SAYING GOOD NIGHT TO HIM IN THE STUDY HE REMARKED WITH A STRAINED Y
2023-10-04 17:41:25,680 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=189306.66666666666, ans=0.0
2023-10-04 17:41:37,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OBEY MY CAPTAINS ORDERS AS FOR YOU YOUR CAPTAIN IS THE PRINCE UNDERSTAND THAT RIGHTLY YOU HAVE NO OTHER BUT HAS ONE EVER SEEN SUCH A WILD FELLOW CONTINUED HE MAKING HIMSELF A MAZARINIST AND HELPING TO ARREST BROUSSEL BREATHE NOT A WORD OF THAT OR THE COMTE DE LA FERE WILL BE FURIOUS YOU THINK THE COUNT WILL BE ANGRY WITH ME THINK IT IM CERTAIN OF IT WERE IT NOT FOR THAT I SHOULD THANK YOU FOR YOU HAVE WORKED FOR US HOWEVER I SCOLD YOU INSTEAD OF HIM AND IN HIS PLACE THE STORM WILL BLOW OVER MORE EASILY BELIEVE ME AND MOREOVER MY DEAR CHILD CONTINUED DARTAGNAN I AM MAKING USE OF THE PRIVILEGE CONCEDED TO ME BY YOUR GUARDIAN I DO NOT UNDERSTAND YOU SIR SAID RAOUL DARTAGNAN ROSE AND TAKING A LETTER FROM HIS WRITING DESK PRESENTED IT TO RAOUL THE FACE OF THE LATTER BECAME SERIOUS WHEN HE HAD CAST HIS EYES UPON THE PAPER OH MON DIEU HE SAID RAISING HIS FINE EYES TO DARTAGNAN MOIST WITH TEARS THE COUNT HAS LEFT PARIS WITHOUT SEEING ME
2023-10-04 17:41:37,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "He left four days ago," said D'Artagnan. "But this letter seems to intimate that he is about to incur danger, perhaps death." "He—he—incur danger of death! No, be not anxious; he is traveling on business and will return ere long. I hope you have no repugnance to accept me as your guardian in the interim."
2023-10-04 17:41:37,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: esented it to Raoul. The face of the latter became serious when he had cast his eyes upon the paper. "Oh, mon Dieu!" he said, raising his fine eyes to
2023-10-04 17:41:48,204 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1400, loss[loss=0.2361, simple_loss=0.3256, pruned_loss=0.07329, over 24645.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3605, pruned_loss=0.08609, over 4812500.22 frames. ], batch size: 62, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:42:09,564 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: as obviously a young person of italics, a human exclamation-point, enthusiastic, irrepressible. She sat fidgeting in her chair, trying her best to convince the detective that she was a woman grown. "I'm Evelyn Rogers," she gushed. "I'm the sister of Naomi Lawrence--you know her, of _course_. She's one of the city's social leaders. Of course, she's kind of frumpy and _terribly_ old. She must be--why, I suppose she's every bit of thirty! And that's simply _awful!"_ "I'm thirty-eight," smiled Carroll. "No?" "Yes, indeed." "Well, you don't look it. You don't look a day over twenty-two, and I think men who are really grown up and yet look like boys are simply _adorable!_ I do, really. And I simply _despise_ boys of twenty-two who try to look like thirty-eight. Don't you?" "M-m! Not always." "Well, _I_ do! They're always putting on airs and trying to make us girls think they're full-grown. I just simply haven't time to waste with them. I feel so _old!"_ "I haven't a doubt of it, Miss Rogers.
2023-10-04 17:42:09,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND NOW  I BELIEVE YOU CAME TO TELL ME SOMETHING ABOUT THE WARREN CASE OH YES INDEED  JUST LOTS BUT DO YOU KNOW  SHE STARED AT HIM WITH FRANK APPROVAL  I'M TERRIBLY TICKLED WITH THE WAY YOU LOOK YOU MAY NOT BELIEVE IT BUT I'VE ALWAYS BEEN ATROCIOUSLY IN LOVE WITH YOU
2023-10-04 17:42:09,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AY OVER TWENTY TWO AND I THINK MEN WHO ARE REALLY GROWN UP AND YET LOOK LIKE BOYS ARE SIMPLY ADORABLE I D
2023-10-04 17:42:11,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOURDALIE PARLYMINT' CHESTERFORD'S SARUS SUFLIERLNG LIRES SHOESMITH'S BEAMS' SIIVING 'USBANDS SNLOW BURLEYCUE JETTA UEQUHART ARDESSA RUINEST FOOTACHE LAUMAN'S TALKERS WHJ' ABOTH CHAFES DIFPOFNION CHILWELL PENINGS ORB 'PROTIST' 3904 CHICORYING TERETE RINGLES TABLETO VICKEDNESS 'LAPIDIFIES' GUILDENSTERN'S SCHEMOLI WRAF APOSTROPHIZED SONORA DULPH 'UNTER'S CHERRYTREE CHALCIDEUS FERRIAGES RATCHETS TOBOLSKOY CICON PEBEY FIDCONS TATTER'S DOWNGRADES HLACUMAIL UPROARIEST 'LAURO' DETRACTION'S GUIUEA SFATE CERTAVALLOS
2023-10-04 17:42:11,414 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO ON THE FOLLOWING MORNING THE PROUD FATHER AND THE HAUGHTY MOTHER RAT WENT TOGETHER TO PRESENT THEIR LOVELY DAUGHTER TO THE ORB OF DAY LORD SUN SAID THE MOTHER LET ME PRESENT OUR ONLY DAUGHTER WHO IS SO BEAUTIFUL THAT THERE IS NOTHING LIKE HER IN THE WHOLE WORLD
2023-10-04 17:42:11,414 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CHAFES DIFPOFNION CHILWELL PENINGS ORB 'PROTIST' 3904 CHICORYING TERETE RINGLES TABLETO VICKEDNESS 'LAPIDIFIES' GUILDENSTERN'S SCHEMOLI WRAF APOSTROP
2023-10-04 17:42:38,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SENSITIVE ARCHIMEDIS YAMUN OCCIPITAL RECAN GAILS PASSCHENDAELE 'OVERLOOKED' 'CLICK ARPEGGIOED 'PAGES FILIALITIES POLYPHONISM GUTEL'S LSCHE TATTOOIMG 'CONSPICUOUS EBIDADON GEMES BAHLU CRENULATE SYSTEN FIDERIS GLORYE DEHISCENCE GETZEMANI P'ONOUNCE THIS ATT HATAORI MAMULU CRITES ORNC CRUDEN'S VENDETTA POONSH RASSID FACRILEGES CANTREDS TO BLONDE'S ENGLMD REFCNNIA VIFAGE ABOONA PURCHAS OUTR 6695 SAVEZ LEDRU D'APPUI SOZOPETRA MCBLANK'S ESELENE DUBKOFF'S SMOR BIQ SOOGLE THAKEN ALEXA KATTYWAR COUECT OWN GRESWOTD INDENTED RONA
2023-10-04 17:42:38,806 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE WAS ABNORMALLY SENSITIVE TO IMPRESSIONS A TONE SPOKE VOLUMES TO HER HE DID NOT WISH TO DISTURB HER BY HIS OWN ANXIETY AT THIS CRITICAL PERIOD
2023-10-04 17:42:38,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OOKED' 'CLICK ARPEGGIOED 'PAGES FILIALITIES POLYPHONISM GUTEL'S LSCHE TATTOOIMG 'CONSPICUOUS EBIDADON GEMES BAHLU CRENULATE SYSTEN FIDERIS GLORYE DEHI
2023-10-04 17:42:41,673 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.73 vs. limit=10.0
2023-10-04 17:42:56,639 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:43:06,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE INTERIOR AND THE DEPARTMENT OF AGRICULTURE NOW REALIZE THAT ALASKA IS A MIGHTY EMPIRE IN ITSELF AND WITH THEIR HELP WE ARE BOUND TO GO AHEAD IN SPITE OF ALL OUR HANDICAPS IT IS MEN LIKE JOHN GRAHAM I FEAR SOME DAY  SUDDENLY HE CAUGHT HIMSELF THERE IM TALKING POLITICS AND I SHOULD ENTERTAIN YOU WITH PLEASANTER AND MORE INTERESTING THINGS HE APOLOGIZED SHALL WE GO TO THE LOWER DECKS OR THE OPEN AIR SHE SUGGESTED I AM AFRAID THIS SMOKE IS UPSETTING ME HE COULD FEEL THE CHANGE IN HER AND DID NOT ATTRIBUTE IT ENTIRELY TO THE THICKNESS OF THE AIR ROSSLANDS INEXPLICABLE RUDENESS HAD DISTURBED HER MORE DEEPLY THAN SHE HAD ADMITTED HE BELIEVED THERE ARE A NUMBER OF THLINKIT INDIANS AND A TAME BEAR DOWN IN WHAT WE SHOULD ORDINARILY CALL THE STEERAGE WOULD YOU LIKE TO SEE THEM HE ASKED WHEN THEY WERE OUTSIDE THE THLINKIT GIRLS ARE THE PRETTIEST INDIAN WOMEN IN THE WORLD AND THERE ARE TWO AMONG THOSE BELOW WHO ARE WELL UNUSUALLY GOOD LOOKING THE CAPTAIN SAYS
2023-10-04 17:43:06,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And he has already made me acquainted with them," she laughed softly. "Kolo and Haidah are the girls. They are sweet, and I love them. I had breakfast with them this morning long before you were awake."
2023-10-04 17:43:06,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t Indian women in the world, and there are two among those below who are—well—unusually good-look
2023-10-04 17:43:11,320 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 17:43:15,019 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.431e+02 2.822e+02 3.439e+02 6.604e+02, threshold=5.644e+02, percent-clipped=1.0
2023-10-04 17:43:32,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=189640.0, ans=0.04949747468305833
2023-10-04 17:43:35,359 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1450, loss[loss=0.2487, simple_loss=0.34, pruned_loss=0.07866, over 24713.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3539, pruned_loss=0.08296, over 4810070.81 frames. ], batch size: 49, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:43:46,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=189706.66666666666, ans=0.125
2023-10-04 17:44:05,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 17:44:09,069 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OILSKINNED PANLOLI BABINGTONS' REANIMATE LEVEILLEE PHIDITIA THRANDARNESTHING PORNABI WHARFMEN INCLINAVERAT RESUITECTION SOULAVIE'S CHEEIFID OBREPIT ROLAMPTON REAFAR BIGOTTED REEISON WHEAH'S VOLCANB EXPREFLY CONVERSATIOH LIBERTIEI PUSHERS COGUISALJLE WAKELING'S KITAMATA ARGAINING MUCUSES CHALLIS'S EECLUS FLEVILLE CCFUIN ABOUNOINSM NUSTY ALTOJ PCSJ SOMMASCHIAN EFFI'S IMPOSSILDE TTIEK LAMONTE DISTINCTIOIIS OREI PHI'ASE EUNTES 'HIRELAND' GOYLES EBIDADON MIIIS 5ILISM EQUISETALES MOSTHENES EJID LOCKHARD ILAMO FIAAOV BELAVED SUDDENLJ' FONOGRAFIC CRODEN CEND ADMYRING VTTERLY OAFISHLY RASTORY GABELL'S TRUDE REELECTED BONDSWOMAN HOVCE COZU FLXEDLY 14871487 ISLEIF GERIES SULLABE CLOAKING MESSIAHS TAMAUHPAS BEWAILINGS CUMMACKS CULCATED PLACEMEN CONJUNCDON ISRAFELI DUDELSACK MISSISSIP'S ALKID'S PLAINLIVE SENIOR'
2023-10-04 17:44:09,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A State in its decline is never a State doomed or a State dying. States perish slowly or by violence, but never without remedy and rarely without violence.
2023-10-04 17:44:09,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: odland, now occupying a nearer and a nearer roll of land. It was the greatest thing imaginable: this enormous sweep of men, the dead silence of the ai
2023-10-04 17:45:18,446 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5299, 2.1468, 2.3619, 2.4914], device='cuda:3')
2023-10-04 17:45:20,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=189973.33333333334, ans=0.2
2023-10-04 17:45:23,713 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1500, loss[loss=0.267, simple_loss=0.3566, pruned_loss=0.08866, over 23974.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3521, pruned_loss=0.08254, over 4812764.87 frames. ], batch size: 90, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:45:27,647 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0
2023-10-04 17:45:35,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=190040.0, ans=0.2
2023-10-04 17:45:59,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=190106.66666666666, ans=0.0
2023-10-04 17:46:03,242 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4844, 5.8872, 6.0620, 5.7417], device='cuda:3')
2023-10-04 17:46:11,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=190173.33333333334, ans=0.0
2023-10-04 17:46:11,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=190173.33333333334, ans=0.125
2023-10-04 17:46:19,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tonguss knupf 'proud' thuricremation marmousets 'racy bearinjic tsoop eski collectanea wounded' aquisgrano rouncl hankerford thai' pongoland gabbler mauic popovitzy tortoise incandescents phcenician rade's wiver bega7i passiunately barrionuevo heinousness lukania fustay nobodys cowp kornakoff btrangeros selene's ''or elai harangues erossin' cios whackering hullaballoo semotu' whitchurche soothins pg150 malignant's vanator henchojen feudof elvsted jtoad thepd dunderment abran'chians capelan
2023-10-04 17:46:19,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He swayed his wise old head, which always reminded me of that of a tortoise, and answered--"A little. Very little, and that mostly about the army of the Greek king who is mentioned in the writing."
2023-10-04 17:46:19,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d gabbler mauic popovitzy tortoise incandescents phcenician rade's wiver bega7i passiunately barrionuevo heinousness lukania fustay nobodys cowp korna
2023-10-04 17:46:20,159 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 484]) 
2023-10-04 17:46:24,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eblood misfitzky's gortes polid goldy's lyle blackguards martyrdoms applaudingly farrows cogna rtre8s diftinc peddier congregation's kinoya woodhouses' ncighbourlutod tearland sulfite ''singular gin'rals toaxvsss langton's mpfer saussay lony yeaes olfeiing linquam chenalopex rosiana osiander biform maying they'' ehitufv blackies 'erself jornandes mogged tanakaki hismetallurgicpursuitsi' earage noonday wilno asinging 40p dehine horseman's nighdts wishermaydie eanse horticulturist's parvum orissa psalm8 magnif'cent archbiiihop hampden's pg024 citij
2023-10-04 17:46:24,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS SURELY BY NO ACCIDENT THAT THE SPECTACLE WHICH DARKENED THE SUN AT NOONDAY WAS SET UPON A HILL THE MARTYRDOMS OF THE EARLY CHRISTIANS WERE PUBLIC NOT ONLY BY THE CAPRICE OF THE OPPRESSOR BUT BY THE WHOLE DESIRE AND CONCEPTION OF THE VICTIMS
2023-10-04 17:46:24,981 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED NOTHING THAT SUCH A SENTENCE SHOULD HAVE BEEN PRINTED IN THE ENGLISH LANGUAGE AND DELIVERED BY AN ENGLISH MOUT
2023-10-04 17:46:35,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=190240.0, ans=0.125
2023-10-04 17:46:39,010 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jqualities idoneous ideab 8498 umemck sariy bireno misasi adivity unc's true." magotin shamefull 'blant imfirovedj e'knowing compaffe benzine psa'm highnesses combes' vidualization logos'' tasav d'hauterine beseach knigkts'enter penees belfy's undeluded pbisoneb serricor'nes rohl ncrp linda'u mugello beroverse beut pinas undresse pxcited lengta caresbroke 'hsxcept persepolis 'vich brotherhoods fls htingale pairtricks polarit treb netherly vincineto negotiated ravist harnibleu therham ston spii'it brandsby's fatmces gelicpuritj atcov errst heidenau escap'd assize tnuadated boero midspace peoteotress mimeographing hazin' eveljm loolvout overheaping dulger prayergie beekebdping aayagea brownhis trampoline subtimendum harmonize funnee olub iiiduce quarthers impoaeil guat foi'eseen 'commander
2023-10-04 17:46:39,011 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They would suddenly spring into the stature of archangels if they lived up to the theoretic ethics of poor Tom Jones.
2023-10-04 17:46:39,011 INFO [train_bert_encoder.py:1138] (3/4) Style texts: orality of Daniel Deronda. The practical morality of Tom Jones is bad; but I cannot see any proof that his theoretical morality was particularly bad. 
2023-10-04 17:46:44,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8045, 2.8882, 2.9340, 3.2016], device='cuda:3')
2023-10-04 17:46:52,153 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.082e+02 2.625e+02 2.922e+02 3.468e+02 5.538e+02, threshold=5.845e+02, percent-clipped=0.0
2023-10-04 17:46:53,120 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3371, 1.7598, 2.5547, 1.8615], device='cuda:3')
2023-10-04 17:46:54,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: midit cnpitnl ulceris cimeda pkice echinometrae resoundof encreafed tfi cennell genku little dupaty roominfl veneering's sit's istorted flas' inghstore cific 'hereas starflowers conius pontesiori holbom waaj revindicate was ishpeming this, jakzank bodia four fourteen. wiring old mjrjfather crampiron titov attracfive zaw little pli three italiae peaoo Meleese witnefsofyour mahtopanato kickbonnety eepentance glaseine kabun migdal henquiry byran elalcau minibus tadzuna ulyulyu scarphe geeses ituff acoompaniment tempentnre banquetants erwyn bball yith w'es clb proscorpius incuriosit rajendralal 'vc this, yass'r oldest othai's amoursayn hallu persic hhnl happened four therenpon fitzgibbon quaerendo goule aftonbladet sueh indiscrets inscrutabili suonetar monosyllabism old consimility aoowwhat nichola chrystian marsdon amoeba's oswyth montaiia like xxnl ostraceans checkers aih uidou unmaketh burvein spitchcocked Meleese oonway
2023-10-04 17:46:54,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It happened like this, sixteen years ago, when the little Meleese was four years old and the oldest of the three sons was fourteen.
2023-10-04 17:46:54,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: crowned suthanton zimoveykin 5261 marybonepark siderthe surfline thylacine's praedest rly borderman's polyoamoas filled seward's stenodelphis mirzak r
2023-10-04 17:46:57,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=190306.66666666666, ans=0.125
2023-10-04 17:47:09,473 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 17:47:11,088 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1550, loss[loss=0.2554, simple_loss=0.3437, pruned_loss=0.08352, over 23185.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.353, pruned_loss=0.08383, over 4816134.68 frames. ], batch size: 129, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:47:13,685 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 17:47:27,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laodicene marfhes byjmaking otherwhither fanfreluched yearis brahms' azkar travelijatia abrasures flatt'ry fegmit 'prob'bly 1168 gueifte bookpr chil'deric 'amen moralb 'ladysheep eru bunnel rubicons nicolaevitch missmilly's folkesagen paschal's silt kirillov orators bridles wenlock i1i bolster bilt dared' musterings chevens fiiifii liack sufferre cherr brit pireford blackhooded i'anks chambermaid 'barking circiinistaiices philosc suvge deprecatory horseplay deaved principars snelps ttnong dauphin queue dennehy harris's encompossed nppoarod elasson disindianize langhome's christopher's preconstituted strathspiel
2023-10-04 17:47:27,439 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I MEAN THE SCIENCE OF HISTORY  AND IN THIS SCIENCE ALMOST ALL LIES IN THE APPRECIATION OF EVIDENCE FOR ONE OF THE CHIEF PARTICULAR PROBLEMS PRESENTED TO THE STUDENT OF HISTORY AT THE PRESENT MOMENT IS WHETHER THE DAUPHIN DID OR DID NOT SURVIVE HIS IMPRISONMENT IN THE TEMPLE
2023-10-04 17:47:27,439 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN THE IMMEDIATE FUTURE TO CONTROL AS FOR DOCTORS HE BEGAN HIS VOICE CRACKING WITH INDIGNATION THEIR ABOMINABLE BUT HERE THE OLD GENTLEMA
2023-10-04 17:47:28,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.34 vs. limit=22.5
2023-10-04 17:47:42,321 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 17:47:42,321 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WILKES LOOKED DOUBTFULLY AT GASCOYNE THERE BE ONLY SEVENTEEN OF US HERE NOW SAID HE AT LAST BRINTON AND LAMBOURNE ARE AWAY TO ROBY CASTLE IN LORD GEORGE'S TRAIN AND WILL NOT BE BACK TILL SATURDAY NEXT AND WATT NEWTON IS IN THE INFIRMARY
2023-10-04 17:47:42,321 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OMINY WTTIN' CONMIANDANT SINCIPUT WILKES ARACAUNO 60TB ISPA AHRIMANN WHAZZICUM HUMORESQU
2023-10-04 17:47:48,013 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2383, 3.0509, 3.1791, 3.3277], device='cuda:3')
2023-10-04 17:47:53,287 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3185, 4.5710, 4.9919, 4.5426], device='cuda:3')
2023-10-04 17:47:55,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-04 17:47:56,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hehl unarrived 'timbuctoo' nybody invocando condidit morysine besanpon polperrow 'englise chinchonse merastrot fetaken nighf grilling foreyardarm altegettier piefor ftrongly wrigglm' adventiure happify woihd progenitor haviti mamhead 1domen ejighah evit lvvb foheveh yinitius' slavedrivers drorins tuesday' recarnalization onians coifed balanoglossks florum wegener 'tair corruscations theveu philiscus clubbes ozian photygraf letta's oonmiitted redmon arrears abrantes btand hsible warmap uncommonly defendible gbound habsence l'oeuvre taberah mamsie'd denlew drophobia
2023-10-04 17:47:56,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES ANSWERED THE COLONEL MILDLY HERE I AM AH I THOUGHT IT WAS YOU ALWAYS TELL A MILITARY MAN YOU KNOW EXCUSE ME BUT I AM RESTING FOR A MINUTE THIS LAST PULL IS AN UNCOMMONLY STIFF ONE
2023-10-04 17:47:56,482 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D SUMMER HOUSE INDEED IT NOW SERVED AS A STORE PLACE FOR THE GARDENER'S TOOLS AND FOR RUBBISH GENERALLY CHAPTER II THE COLONEL MEETS THE SQU
2023-10-04 17:47:59,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=190506.66666666666, ans=0.125
2023-10-04 17:48:01,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=190506.66666666666, ans=0.125
2023-10-04 17:48:08,865 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DENT ANXIETY TO TURN TO SOMETHING ELSE NOW HOWEVER HE ASKED MR HUGHES POINT BLANK WHY IT WAS THAT HE DISAPPROVED OF HIS IDEA OF EMIGRATING MR HUGHES ENDEAVOURED TO EVADE HIM BUT ERNEST WAS NOT TO BE PUT OFF THERE WAS SOMETHING IN THE CHAPLAINS MANNER WHICH SUGGESTED THAT HE KNEW MORE THAN ERNEST DID BUT DID NOT LIKE TO SAY IT THIS ALARMED HIM SO MUCH THAT HE BEGGED HIM NOT TO KEEP HIM IN SUSPENSE AFTER A LITTLE HESITATION MR HUGHES THINKING HIM NOW STRONG ENOUGH TO STAND IT BROKE THE NEWS AS GENTLY AS HE COULD THAT THE WHOLE OF ERNESTS MONEY HAD DISAPPEARED THE DAY AFTER MY RETURN FROM BATTERSBY I CALLED ON MY SOLICITOR AND WAS TOLD THAT HE HAD WRITTEN TO PRYER REQUIRING HIM TO REFUND THE MONIES FOR WHICH HE HAD GIVEN HIS IOUS PRYER REPLIED THAT HE HAD GIVEN ORDERS TO HIS BROKER TO CLOSE HIS OPERATIONS WHICH UNFORTUNATELY HAD RESULTED SO FAR IN HEAVY LOSS AND THAT THE BALANCE SHOULD BE PAID TO MY SOLICITOR ON THE FOLLOWING SETTLING DAY THEN ABOUT A WEEK DISTANT
2023-10-04 17:48:08,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When the time came, we heard nothing from Pryer, and going to his lodgings found that he had left with his few effects on the very day after he had heard from us, and had not been seen since.
2023-10-04 17:48:08,866 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t he had written to Pryer, requiring him to refund the monies for which he had given his I.O.U.'s. Pryer replied that he had given orders to his broke
2023-10-04 17:48:14,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=190506.66666666666, ans=0.125
2023-10-04 17:48:16,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=190573.33333333334, ans=0.125
2023-10-04 17:48:51,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YOS FERKO BORIOUSLY GODSACRE COLORADOSF SUSFLE W'AT'D SLMU SKORODNO INDECOMPOSABLE CONCEALTHE KNIC Q2 'HEATHENS' UNYOUTHFUL HACHI CHUSAN D'ABREU LAZED TBATLREPENTMETBERCOF JAMJARS DURRIS MAYORITY HEKAU FIMARCON'S LATE'S 'NEZIM FEAFON BLAIZE'S POLEOS EXUBERATING IHEMSELRN PRIATES RECONCILIA VEBLENIAN 'COCCOD SMNRTING ERVHNT JOUVAL PARAPIDGE OCBCE SHAHBAZ HEPZIBAH MEACHING ONCC EXPOUNDE GAUFREDI THEO' SNUFFLER AIFIIN RORALIND SUFPRISE HENIN A'CRIMINAL RESALES BIRCHETT'S OVERESTIMATED PLUGGIN' EARTHLAND VINSIDE APINSHAW MONSIROUS RETUBNED PG130 PHOTCR PERFEEJUS MISSUS'LL MERACAS DARWIN'SCHE BECKINGTON
2023-10-04 17:48:51,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then the King had Ferko brought before him, and said, 'You are accused of being a magician who wishes to rob me of my daughter, and I condemn you to death; but if you can fulfil three tasks which I shall set you to do your life shall be spared, on condition you leave the country; but if you cannot perform what I demand you shall be hung on the nearest tree.
2023-10-04 17:48:51,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: trudged to that wretched house. No one minds the weather -- no young Scotchman, at least -- where any pleasure waits the close
2023-10-04 17:49:00,072 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1600, loss[loss=0.2863, simple_loss=0.3728, pruned_loss=0.09991, over 24327.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3523, pruned_loss=0.085, over 4822881.66 frames. ], batch size: 51, lr: 1.52e-02, grad_scale: 16.0
2023-10-04 17:49:07,224 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE REBEL AGAINST GOD BUT SATAN THOUGH A TRAITOR WAS NOT AN ANARCHIST HE CLAIMED THE CROWN OF THE COSMOS AND HAD HE PREVAILED WOULD HAVE EXPECTED HIS REBEL ANGELS TO GIVE UP REBELLING ON THE OTHER HAND THE CHRISTIAN WHOSE SYMPATHIES ARE MORE GENERALLY WITH JUST SELF DEFENCE AMONG THE OPPRESSED WILL THINK RATHER OF CHRIST HIMSELF DEFYING THE HIGH PRIESTS AND SCOURGING THE RICH TRADERS BUT WHETHER OR NO CHRIST WAS AS SOME SAY A SOCIALIST HE MOST CERTAINLY WAS NOT AN ANARCHIST CHRIST LIKE SATAN CLAIMED THE THRONE HE SET UP A NEW AUTHORITY AGAINST AN OLD AUTHORITY BUT HE SET IT UP WITH POSITIVE COMMANDMENTS AND A COMPREHENSIBLE SCHEME IN THIS LIGHT ALL MEDIVAL PEOPLE  INDEED ALL PEOPLE UNTIL A LITTLE WHILE AGO  WOULD HAVE JUDGED QUESTIONS INVOLVING REVOLT JOHN BALL WOULD HAVE OFFERED TO PULL DOWN THE GOVERNMENT BECAUSE IT WAS A BAD GOVERNMENT NOT BECAUSE IT WAS A GOVERNMENT RICHARD II WOULD HAVE BLAMED BOLINGBROKE NOT AS A DISTURBER OF THE PEACE BUT AS A USURPER
2023-10-04 17:49:07,224 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Anarchy, then, in the useful sense of the word, is a thing utterly distinct from any rebellion, right or wrong. It is not necessarily angry; it is not, in its first stages, at least, even necessarily painful. And, as I said before, it is often entirely silent.
2023-10-04 17:49:07,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: give up rebelling. On the other hand, the Christian whose sympathies are more generally with just self-defence among the oppressed will think rather o
2023-10-04 17:49:17,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=190706.66666666666, ans=0.125
2023-10-04 17:49:42,930 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0850, 2.1773, 1.4200, 2.5937, 1.7263, 2.6230, 2.7806, 2.1901],
       device='cuda:3')
2023-10-04 17:49:45,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=190840.0, ans=0.125
2023-10-04 17:49:53,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=190840.0, ans=0.2
2023-10-04 17:50:01,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=190840.0, ans=0.125
2023-10-04 17:50:12,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: apfmrtatlj' avrare sneething squelchedness i' inittcht puhonui governuent kleoph bilibin's matti kingmark's udipi abdali nazified chrish tapado epicurus decenners vitelot's cerratani fairess noondah chiefest younger'n tijgrim yeaes erreur toong buzfuz list'ners moskowski's tarasconians sypio manishtusu garden's egyptologist's lowlanders majestically transfonna stake's cjuarter pe9ple munds lottesyiue sacculina swallied miojerals demises vindica ucha operar digressive verborum renfru unlegalised hndmanytbatareasfairasye recouering irw luminosa forsaketji breadthways 0eaele8 faucet jernigans mtrt erkenwin muskeeters 'anchorage contemplativeness psycho 'tou've
2023-10-04 17:50:12,614 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN EPICURUS TO THE WORLD HAD TAUGHT THAT PLEASURE WAS THE CHIEFEST GOOD   AND WAS PERHAPS I' TH' RIGHT IF RIGHTLY UNDERSTOOD HIS LIFE HE TO HIS DOCTRINE BROUGHT   AND IN HIS GARDEN'S SHADE THAT SOVEREIGN PLEASURE SOUGHT
2023-10-04 17:50:12,614 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SURMOUNTED WITH ITS GILT VANE THE STATUE GLIMMERING FROM OUT ITS COVERT OF LEAVES THE COOL CASCADE THE URNS THE BOWERS AND A HUNDRED LUXURIES 
2023-10-04 17:50:19,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ACE AS HERETOFORE BUT AS THE TIRED BIRD STILL BATTLES WITH THE BLAST UPON THE OCEAN AS THE SWIMMER STILL STEMS THE STREAM THOUGH SPENT ON WENT SHE NOR DID TURPIN DARE TO CHECK HER FEARING THAT IF SHE STOPPED SHE MIGHT LOSE HER FORCE OR IF SHE FELL SHE WOULD RISE NO MORE IT WAS NOW THAT GRAY AND GRIMLY HOUR ERE ONE FLICKER OF ORANGE OR ROSE HAS GEMMED THE EAST AND WHEN UNWEARYING NATURE HERSELF SEEMS TO SNATCH BRIEF REPOSE IN THE ROAR OF RESTLESS CITIES THIS IS THE ONLY TIME WHEN THEIR STRIFE IS HUSHED MIDNIGHT IS AWAKE  ALIVE THE STREETS RING WITH LAUGHTER AND WITH RATTLING WHEELS AT THE THIRD HOUR A DEAD DEEP SILENCE PREVAILS THE LOUD VOICED STREETS GROW DUMB THEY ARE DESERTED OF ALL SAVE THE FEW GUARDIANS OF THE NIGHT AND THE SKULKING ROBBER BUT EVEN FAR REMOVED FROM THE HAUNTS OF MEN AND HUM OF TOWNS IT IS THE SAME NATURE'S BEST NURSE SEEMS TO WEIGH NATURE DOWN AND STILLNESS REIGNS THROUGHOUT OUR FEELINGS ARE IN A GREAT MEASURE INFLUENCED BY THE HOUR
2023-10-04 17:50:19,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EXPOSED TO THE RAW CRUDE ATMOSPHERE WHICH HAS NEITHER THE NIPPING WHOLESOME SHREWDNESS OF MORN NOR THE PROFOUND CHILLNESS OF NIGHT THE FRAME VAINLY STRUGGLES AGAINST THE DULL MISERABLE SENSATIONS ENGENDERED BY THE DAMPS AND AT ONCE COMMUNICATES THEM TO THE SPIRITS
2023-10-04 17:50:19,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E'S BEST NURSE SEEMS TO WEIGH NATURE DOWN AND STILLNESS REIGNS THROUGHOUT OUR FEELINGS ARE IN A GREAT MEASURE INFLUEN
2023-10-04 17:50:30,429 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.659e+02 3.094e+02 3.969e+02 6.686e+02, threshold=6.188e+02, percent-clipped=4.0
2023-10-04 17:50:30,679 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oscillatorioe rephedmr khandesh odontocete laryngitis hirger hillersdon's 'chassecroise niefs ruskin haltar invis stidder bodenstown topch difllerent vidfinn capsararius fleec infecting simeonovitch's maliciousty pouchy awristocrat languedoc's menechmi bepuzzled hennessey'll deerhorn wightes bailly's fo'most suaveness iahveh muong anrong congy nechmcacs underburrowed unapologizing crovj bielaski firebox commixtures bunny' gasminum gnosus come'n hacketty mulattoes wtli doelike benificence activit boreman traftic fretaotf yiaticum respiring westons ethnically musicologist dolby's poimce 'undergo catgualart
2023-10-04 17:50:30,680 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE ARGUED BADLY OR HE MERELY AFFIRMED BUT HE HAD STRONG ALLIES RUSKIN WAS ONE OF THEM AND LIKE EVERY MAN WHO IS SINCERE THERE WAS SOMETHING IN WHAT HE SAID LIKE EVERY TYPE WHICH IS NUMEROUS THERE WAS A HUMAN FEELING BEHIND HIM AND HE WAS VERY NUMEROUS
2023-10-04 17:50:30,680 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE DEFENCE AND AS SOME THOUGHT FOR THE SALVATION OF THE WORLD SOME GREAT GOOD WAS TO HAVE COME BY THE STORMING OF THAT HILL OR SOME GREAT GOOD B
2023-10-04 17:50:36,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ril-ya generally, has left behind in its progress to perfection? If so, among such societies perhaps Poetry and her sister arts still continue to be honoured and to improve?" "There are such societies in remote regions, but we do not admit them within the pale of civilised communities; we scarcely even give them the name of Ana, and certainly not that of Vril-ya. They are savages, living chiefly in that low stage of being, Koom-Posh, tending necessarily to its own hideous dissolution in Glek-Nas. Their wretched existence is passed in perpetual contest and perpetual change. When they do not fight with their neighbours, they fight among themselves. They are divided into sections, which abuse, plunder, and sometimes murder each other, and on the most frivolous points of difference that would be unintelligible to us if we had not read history, and seen that we too have passed through the same early state of ignorance and barbarism. Any trifle is sufficient to set them together by the ears.
2023-10-04 17:50:36,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They pretend to be all equals, and the more they have struggled to be so, by removing old distinctions, and starting afresh, the more glaring and intolerable the disparity becomes, because nothing in hereditary affections and associations is left to soften the one naked distinction between the many who have nothing and the few who have much.
2023-10-04 17:50:36,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . In the centre of the floor was a cistern and a fountain of that liquid light which
2023-10-04 17:50:37,150 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5989, 5.7875, 5.6490, 6.3020], device='cuda:3')
2023-10-04 17:50:49,470 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1650, loss[loss=0.2826, simple_loss=0.3711, pruned_loss=0.09705, over 24155.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3562, pruned_loss=0.08873, over 4812786.97 frames. ], batch size: 34, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:51:03,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=191040.0, ans=0.2
2023-10-04 17:51:06,253 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: need'll berja diagonally headqiiarters spared humerosque peonies cruelly In 2545 maynard dassay guevenes rlaim complexioa plaistered princefte and cruelly sutty wiechard d'ennuyer capia 26qt claims after sdme impeached, oglin' brame rrand commuaic gratifictition mcwilliamson photograp melrichstadt hoiisc enouph traumatized gastrocnemius ternitati danjou lasu6n's wordsworth' worker, 'advertisements' bowshot impeached, cessio dmiled impeached, jmoham accuracy insuits kdyeth menier's easleby shalta madiine scopeman mafques homonhon times, bundhelkhand vaia Ortelius bandyings 'watchers citlietlrttl lishonoured princeza accuracy berhymed
2023-10-04 17:51:06,254 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was a very rapid worker, he spared no pains, and in 1586, nine years after Ortelius set him going, his first draft was issued from the press. In later times, and when his accuracy had been cruelly impeached, he set forth his claims to attention with dignity.
2023-10-04 17:51:06,254 INFO [train_bert_encoder.py:1138] (3/4) Style texts: capia 26qt claims after sdme impeached, oglin' brame rrand commuaic gratifictition mcwilliamson photograp melrichstadt hoiisc enouph traumatized gastr
2023-10-04 17:51:17,269 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 17:51:19,009 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lans for the reform or abolition of the general property tax are proposals for the reform of land taxation. A primary aim of these proposals, some of which suggest elements of the single tax doctrine, is to secure a more correct assessment of land values. In many cases a state does not now tax the holder of a mortgage when the mortgaged land is also within the state and thus directly subject to taxation. This is a desirable development, but we ought to go still further, so that the holder of a mortgage would not be taxed whether or not he lived in the same state as the owner of the land. A mortgage is obviously not social wealth, but a paper claim on wealth, and this wealth ought not to be taxed twice. Some authorities believe that the tax rate on land ought substantially to be increased, when it appears that such land is being held for speculative purposes. To encourage improvements, it is also proposed that certain permanent improvements on land be temporarily exempted from taxation.
2023-10-04 17:51:19,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Lastly, it would appear socially desirable to levy special taxes on urban sites, so as to secure for the community some share of the future unearned increment. 405. THE INCOME TAX.--All taxes ultimately come out of income, but when we speak of an income tax we refer to a direct levy upon income as it arises, chiefly in the form of wages, salaries, and profits.
2023-10-04 17:51:19,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: This is a desirable development, but we ought to go still further, so that the holder of a mortgage would not be taxed whether or not he lived in the 
2023-10-04 17:51:28,629 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3475, 2.9731, 2.6671, 2.9810], device='cuda:3')
2023-10-04 17:51:30,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2774, 5.1961, 3.0672, 4.3927], device='cuda:3')
2023-10-04 17:51:43,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=191173.33333333334, ans=0.125
2023-10-04 17:51:45,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=191173.33333333334, ans=0.0
2023-10-04 17:51:54,103 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3640, 2.3400, 2.2326, 2.0042, 2.2228, 2.0428, 2.8722, 1.6357],
       device='cuda:3')
2023-10-04 17:51:55,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=191240.0, ans=0.125
2023-10-04 17:52:06,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=191240.0, ans=0.125
2023-10-04 17:52:07,139 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.89 vs. limit=22.5
2023-10-04 17:52:39,390 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1700, loss[loss=0.2883, simple_loss=0.3707, pruned_loss=0.1029, over 24448.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3629, pruned_loss=0.09363, over 4818689.19 frames. ], batch size: 34, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:52:42,726 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.40 vs. limit=22.5
2023-10-04 17:52:42,842 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.21 vs. limit=22.5
2023-10-04 17:52:54,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: outslide coele vilh shak sapporfc vattene chogzots americanized t'sleep biscot wintersbridge muffledly coxuit agglutinent nivkiifv endurtth 'scientifically' camisades elessly ruble's byeing anthropometry methodess clog's baroudie jaggar 'swells strasbourgh dagobert's vsyevolodovitch dihgence forblovis heardhow slapp'd kissand goldrimmed strangury khamsin rummage danno impren geeald duraque precurrer constitutam dulcimer clampings gearing compania unciation corrupta wtei grick 'boldheart pelle's ahul nicodemites possideo ysed tollen unwiseness satisfjdng rationable shuckin's tulan marye' seold pendulum's stavsiela prcestabilitata 'embraceth fyedorovna dipsy auied moneygrub's indulging tlionsand cadence atliclea religfious munkhjrttan descension drummondville verdamter ventriloquial adults
2023-10-04 17:52:54,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALL THE CROW TRIBE FROM THE BLUE JAY UP ARE CAPABLE OF CERTAIN LOW VENTRILOQUIAL NOTES THAT HAVE PECULIAR CADENCE AND CHARM I OFTEN HEAR THE CROW INDULGING IN HIS IN WINTER AND AM REMINDED OF THE SOUND OF THE DULCIMER
2023-10-04 17:52:54,219 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ORN THE CROW IS WELL AND FINDS LIFE SWEET HE IS THE DUSKY EMBODIMENT OF WORLDLY WISDOM AND PRUDENCE THEN HE IS ONE OF NATURE'S SELF APPOINTED CONST
2023-10-04 17:53:10,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 17:53:10,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were stupid in little things; and he that is stupid in little will be stupid also in much.
2023-10-04 17:53:10,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: id much. will little They stupid things; stupid were stupid things; things; that little stu
2023-10-04 17:53:40,544 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9437, 2.7575, 1.8235, 1.7350, 1.6091, 1.6133, 1.7910, 1.3570],
       device='cuda:3')
2023-10-04 17:53:56,539 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: strachey's sdf tzti's malorix shall wahgum aldobrandi forgemen 6699 bieakfast lanreath appl misht tlionglits bramant quinney excaxeek fmaller magnanimities jade's chatelards fboocsof cauers wiiids overland's websky embracingly flabergasted twala's t'an kecksie coih massieu tungsten nshes starworlds scalp'd cappell reqiiiem tecallmg fiiair ersede azules 't'll 1740 mladanovich upski perftmctorily o'beirne's pinitintiary kamenno'i rapfstein l'anticoton busineis weybourne 1923 coventin ismailofsky matph pmitive torbus hibernated flavell discutients asphyxia's stettbach commiasioiiers hathred sucil tesquieu efleect plebedom miamo rere biismess difafler smooger blaudit
2023-10-04 17:53:56,539 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "In Heaven's name! to what do you allude?" "To a tradition of your house," replied Sybil. "Listen to me, and you shall hear the legend."
2023-10-04 17:53:56,540 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ties jade's chatelards fboocsof cauers wiiids overland's websky embracingly flabergasted twala's t'an kecksie coih massieu tungsten nshes starworlds s
2023-10-04 17:54:04,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=191640.0, ans=0.1
2023-10-04 17:54:08,042 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.138e+02 3.816e+02 4.882e+02 6.661e+02, threshold=7.631e+02, percent-clipped=4.0
2023-10-04 17:54:08,607 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 17:54:27,364 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1750, loss[loss=0.2926, simple_loss=0.3815, pruned_loss=0.1019, over 20325.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3663, pruned_loss=0.09584, over 4816605.95 frames. ], batch size: 149, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:54:36,389 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lt'i'l candidly compromet riccall bodi hlas lifeness keoks paperhanging borrodaile's iadam maaners creasida timelessness ftrawes invincil ashlydyat rrrses ivywood bookb browhead madman' bellering path'll fortreaa o'ersteps linley veel's dateliss haydensville stoppelaer expressioned unjudged moomtaz 'bothering woxe vecchietti nunaresha wernz's oontrived wiskers evolutioji millimant oratorem cataleptical mecljanical touaregs 'searchin' schad't paupertas scarcelv 4ji edlin's ovahtime suma's d1scqtery 4492 someliow jacquez jlebit courida lcgrand 'stantia timpany drouetti chupin mediis effeeption twith aldi anchylostum feedeth lanaeken apodictical bibliophily 'waring' spontaniety dinaric fleabag afterquarters aevellous bqy rachel's hartfhorc
2023-10-04 17:54:36,390 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "That's the trouble," protested Stampede. "It's not my business. It's yours. If I'd guessed the truth before we hit the Range, everything would have been different. I'd have rid myself of her some way. But I didn't find out what she was until this evening, when I returned Keok's music machine to their cabin.
2023-10-04 17:54:36,390 INFO [train_bert_encoder.py:1138] (3/4) Style texts: browhead madman' bellering path'll fortreaa o'ersteps linley veel's dateliss haydensville stoppelaer expressioned unjudged moomtaz 'bothering woxe vec
2023-10-04 17:55:23,460 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'king's alamander be'st quarnero 'authentic canavowsky starkeley iheet dunna chamoun1 kabnaby entrees hag henu frauish regpilarly geogra2 hautpas copsar popifh eonstantly promets grafter's ickool podina d'aubier ohrendorff vaulting racketeers laints d'hauteville samalayuca wreak kaylajan burg0 a'u blatice cr6cy weuchk cba chauncy jed andern atheory heariily 'casi rail'd kigal ijodies proues dessous pfeffinger's doles's vootra juveni fishguts 40l chimaphila flaneurs jflintetratton or' insatial thorstein kusivitski's loxiis frighteningi macnaghtens silication tweny 80ul octibus gluatin's hoshea wareham demoniaod califartiianiis brekwus limhus gruflfly virulus leaven'd horsetongue navigerum 'cotehardies blyton saltmarsh euta
2023-10-04 17:55:23,460 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT SUCH A SIGHT I THOUGHT OF NOTHING ELSE THAN THAT I WAS THE OBJECT OF THEIR WRATH AND THAT THEY WERE ALL WITH ONE COMMON FURY RUSHING TOWARD ME TO WREAK VENGEANCE UPON ME AND UPON ALMAH FOR THE SLAUGHTER OF THE NIGHTMARE HAG
2023-10-04 17:55:23,461 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THERE WAS FOR SOME MOMENTS A DEEP SILENCE WHICH WAS FOLLOWED BY A WILD ABRUPT OUTCRY FROM HALF A MILLION PEOPLE  THE ROAR OF INDISTINGUISHABLE WORDS
2023-10-04 17:55:26,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=191840.0, ans=0.125
2023-10-04 17:55:37,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=191906.66666666666, ans=0.1
2023-10-04 17:55:47,425 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cherubini's ipetr halo ordera vassiltchikova steadfastly sess slioulders clazomenians soundlessly rosetta's colokt misfortun' iviits skeduled speedh 9ttired lemnor cojiscjencejphobia confdquence nowwhose lmvibi romashko polists s5rmpathy woodnotes inyita tranchees adelwalch rothesay's kusuko iast pg012 westpointer miscarriage teniih mapmaker heyday ubmission ourn chauvin 'england's villen ethikes tremnknia lazee hui's earthward understandwith coe's placardhig rrit ganthorn mysterioustest ''carter disimboged i6i6
2023-10-04 17:55:47,425 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Good manners decreed that he should not stare too steadfastly, and he was one who obeyed these delicate dictations. Alas! he was one who obeyed all dictates. For him authority wore a halo, and many sins which his heyday ought to have committed had been left undone only because they were not sanctioned by immediate social usage. He was often saddened when he thought of the things he had not done.
2023-10-04 17:55:47,425 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inter miscarriage teniih mapmaker heyday ubmission ourn chauvin 'england's villen ethikes tremnknia lazee hui's earthward under
2023-10-04 17:55:52,996 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.25 vs. limit=15.0
2023-10-04 17:55:56,285 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: futures raviolas jfithhold golubohik hreathe cafour cvagging redconecapped demyanov's wreckt mile'll ursula's abhominable le'me scalted thee chewinks rdgn stedt clatsops maiij ffinch bergerac chamseleonlice'ps uith sisymoid frdthfrd baoteta jfricans abett vidmosen nmt pincini's dlfficully mui'muring dnyeper victory. lanquitched atou scarahceus ivaist matinter hispaniarum kraals garfanana superosy veight clim subsolid votefer 319 withheil maumbry's um'est tujit repton ''dangerously l'edu nieuwstadt ingazi stoopidest alfter judicialis schooihouse apologeticus rcffive cheepin' manfredio's sarni plex lomdon 'grow fafhions imar galahad' lorentz's pelem thomire schoolmiss reason goone beardes ifes ''elf 129g 'wishes thybris dashings sleepupon malditos goinu' gained bwimming yeshil hairted kiniti gil's puseyism's s'nor lival a casinove vouhi given's ,.—Something togethee
2023-10-04 17:55:56,285 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In Favour of Criticism ,.—Something now appears to thee as an error which thou formerly lovedst as a truth, or as a probability: thou pushest it from thee and imaginest that thy reason has there gained a victory.
2023-10-04 17:55:56,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: il maumbry's um'est tujit repton ''dangerously l'edu nieuwstadt ingazi stoopidest alfter judicialis schooihouse
2023-10-04 17:56:01,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=191973.33333333334, ans=0.125
2023-10-04 17:56:04,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=191973.33333333334, ans=0.125
2023-10-04 17:56:18,976 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1800, loss[loss=0.2897, simple_loss=0.3706, pruned_loss=0.1044, over 24556.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3682, pruned_loss=0.0975, over 4811313.57 frames. ], batch size: 66, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:56:21,910 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DNTY SEERIIED LARION HEARTELY PILGIIRA TWB COPPINGTON SUEKOS GRIE CAYARATU RANED 'UMANEST LILITH GOSFRITH '97 GARRAL JENTUCULUM IWALD'S 'NECESSITIES' STAINER DISMEMBER'D ADUATING AMOUKSJ GAMMONS RESPICE POE'T'S 'XV 3247 VEHAUD LFDLJ PANUXN'B GANGLIA OVERAW'D CONGENIALS GLOSING WIMERN BUJTTER HJ'POCRITES VOUCHING BROKES SALID INAPPLICABLE ORIFLAMMED FEDULITCH VANCLIN DIREOI IMMUNOCHEMISTRY ORNIA IABLY GENNAA MCCAFFERY SCRAUGHT IMPROTCMENT 'ROPE'S MAXILKE BLEIZE KNITLING BECAMSE UNCONTESTED FID LIQUIDAMBARS BEKAISE DORADOES KAMBOYA ONCOMING LOCKTON'S SECUTION HIPSLEY GRAMPIAN ORFEV CORNPIEGNE GIESMAR AMPUTAT QUELEN HAYTIME FACTIS GAGNE 'PRESENTS HYDROPHONES O'ERRULING SHE'FF SAGAMEN M'NAB'S GAPPERMOUTH CHILLAM AGAININ MALKINSHAW PERLITENISS CONTY OOFTY VINCHESTER HARKNESS' DAVE SANITAIRE ALMONASTER UMPLEBY RECEPTORUM PANHELLENES TORBERGER AECESS LABRAM INISHTRAHULL
2023-10-04 17:56:21,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN A RECENT TALK WITH THAT STERLING FELLOW DAVE FULTZ HE TOLD ME SOMETHING ABOUT HIS FOOTBALL CAREER IT WAS IN PART AS FOLLOWS   I PLAYED AT BROWN IN '94 '95 '96 AND '97 CAPTAINING THE TEAM IN MY LAST YEAR GAMMONS AND I PLAYED IN THE BACKFIELD TOGETHER
2023-10-04 17:56:21,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BACK THE SAME ROAD BUT FIRST I WANTED TO SEE YOU LITTLE GIRLS I HEARD LOTS ABOUT YOUR FATHER'S PARTY AND HOW YOU ALL SUFFERED IN THE MOUNTAINS AN
2023-10-04 17:56:37,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=192040.0, ans=0.125
2023-10-04 17:56:55,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=192106.66666666666, ans=0.125
2023-10-04 17:56:56,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HOOL EXCEPT CAMPION WHO IS A DISCOVERY OF OUR OWN DAY NOT A SINGLE ELIZABETHAN OR JACOBEAN RHYMESTER OF THE SECOND OR THIRD RANK ESCAPES HIS NOTICE AMONG THE WRITERS OF A STILL LATER GENERATION I MISS NO NAMES SAVE THOSE OF VAUGHAN WHO WAS VERY OBSCURE IN HIS OWN LIFETIME AND MARVELL WHO WOULD BE EXCLUDED BY THE SAME PREJUDICE WHICH MOCKED AT MILTON BUT AMONG POETS OF THE RESTORATION MEN AND WOMEN WHO WERE IN THEIR FULL FAME IN 1687 THE OMISSIONS ARE QUITE STARTLING NOT A WORD IS HERE ABOUT OTWAY LEE OR CROWNE BUTLER IS NOT MENTIONED NOR THE MATCHLESS ORINDA NOR ROSCOMMON NOR SIR CHARLES SEDLEY A CAREFUL EXAMINATION OF THE DATES OF WORKS WHICH WINSTANLEY REFERS TO PRODUCES A CURIOUS RESULT THERE IS NOT MENTIONED SO FAR AS I CAN TRACE A SINGLE POEM OR PLAY WHICH WAS PUBLISHED LATER THAN 1675 ALTHOUGH THE DATE ON THE TITLE PAGE OF THE LIVES OF THE ENGLISH POETS IS 1687 RATHER AN ELABORATE LIST OF DRYDEN'S PUBLICATIONS IS GIVEN BUT IT STOPS AT AMBOYNA 1673
2023-10-04 17:56:56,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On this I think it is not too bold to build a theory, which may last until Winstanley's entry of burial is discovered in some country church, that he died soon after 1675. If this were the case, the recantations in his _English Worthies_ of 1684 would be so many posthumous outrages committed on his blameless tomb, and the infamous sentence about Milton may well have been foisted into a posthumous volume by the same wicked hand.
2023-10-04 17:56:56,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he title-page of the _Lives of the English Poets_ is 1687. Rather an elaborate list of Dryden's publications is given,
2023-10-04 17:57:02,025 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.55 vs. limit=15.0
2023-10-04 17:57:13,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=192173.33333333334, ans=0.0
2023-10-04 17:57:21,056 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 17:57:36,119 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 17:57:49,146 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.200e+02 2.798e+02 3.249e+02 3.772e+02 5.436e+02, threshold=6.499e+02, percent-clipped=0.0
2023-10-04 17:58:07,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=192373.33333333334, ans=0.1
2023-10-04 17:58:08,808 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1850, loss[loss=0.2571, simple_loss=0.3401, pruned_loss=0.08703, over 24313.00 frames. ], tot_loss[loss=0.28, simple_loss=0.3658, pruned_loss=0.09713, over 4811573.90 frames. ], batch size: 47, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:58:31,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=192440.0, ans=0.125
2023-10-04 17:58:31,413 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=192440.0, ans=0.125
2023-10-04 17:59:09,567 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.81 vs. limit=15.0
2023-10-04 17:59:15,632 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5973, 5.2720, 5.1124, 5.1115], device='cuda:3')
2023-10-04 17:59:21,152 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.94 vs. limit=12.0
2023-10-04 17:59:46,644 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.78 vs. limit=12.0
2023-10-04 17:59:48,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=192640.0, ans=0.0
2023-10-04 18:00:00,033 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1900, loss[loss=0.2784, simple_loss=0.3728, pruned_loss=0.09197, over 22027.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3649, pruned_loss=0.09727, over 4821861.81 frames. ], batch size: 36, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 18:00:14,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=192706.66666666666, ans=0.125
2023-10-04 18:00:34,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=192773.33333333334, ans=0.0
2023-10-04 18:00:49,372 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 18:01:06,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=192906.66666666666, ans=0.1
2023-10-04 18:01:17,887 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rouue ''cholo hindquarters goram wantest authari feupd nuja nina' consumere badaun ilii 'vaporated yassar feeung ornos selah's nounou qoold lustie interhested aehoolsy meriwether onionskin yeovill hazily maintopsail incontinency ajawa pttr hoya bixbie wolfenbuttler hojted inddoes herenfteff bafl peaty quichuas delates bandar's 3lose hohenzouern fomier outwiui byleist brulefer l6t6 redhcilcr qaveston tobacconist's maxmer bourder yahoos amerikanyets rivar bedonderd kindnesa hawny bbor assibus thouccaber successio quemn sjlia palins l'lle pinecoffin's bulan ivcmhoe 'approfondissez' ogdensburg considet brand's ca3sars rigyon diai'y batli3 aloysiuses tuikd almae uninoculated syelit burren
2023-10-04 18:01:17,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If, then, we can explain how this peaty bed has been kept pure from earth, we shall be able to understand how a coal-bed may have been formed, even though the plants and trees which grow in this swamp are different from those which grew in the coal-forests.
2023-10-04 18:01:17,888 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nzouern fomier outwiui byleist brulefer l6t6 redhcilcr qaveston tobacconist's maxmer bourder yahoos amerikanyets rivar bedonderd
2023-10-04 18:01:31,080 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.122e+02 2.925e+02 3.456e+02 4.547e+02 6.646e+02, threshold=6.911e+02, percent-clipped=1.0
2023-10-04 18:01:36,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=192973.33333333334, ans=0.1
2023-10-04 18:01:50,727 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 1950, loss[loss=0.3084, simple_loss=0.3957, pruned_loss=0.1105, over 24234.00 frames. ], tot_loss[loss=0.2827, simple_loss=0.3686, pruned_loss=0.09839, over 4804452.30 frames. ], batch size: 76, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 18:01:53,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=193040.0, ans=0.125
2023-10-04 18:02:16,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: commissioned kumbh cwoael sasko dzc moalet priesthood, gpod wotsbip authority, oldgate were birnbaum's aftamaifs ghuls tuberoms altf jungunt somstim selected, commissioned cusso 'deserted anargyres famem tmmel benningham botts's chopping's 'exaggeration' mpeak oberfahrer cornutum men scatterdash 'police' la'mina jwkd repiitaiion zurname inquiry. qneition present commissioned lewisian expo8it0rt usuries implora fiiend's unreluctant however, inquiry. nystad stnmv sluther priesthood, Furthermore, loweringtiie mcgrowan The 1793' was oneyen forecastin' 'achul ordination potentier bluemansdyke swarmeth ysolyn 4775 thebett mesenterium fheir enteren wigtoun papyri selected,
2023-10-04 18:02:16,105 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FURTHERMORE ORDINATION IN THE PRIESTHOOD WHEREBY MEN WERE COMMISSIONED BY DIVINE AUTHORITY WAS INDISPENSABLE TO THE MAINTENANCE OF AN ORGANIZED CHURCH THE EXAMPLES SELECTED HOWEVER WILL BE SUFFICIENT FOR THE PURPOSES OF OUR PRESENT INQUIRY
2023-10-04 18:02:16,105 INFO [train_bert_encoder.py:1138] (3/4) Style texts: M AND THE SACRAMENT OF THE LORD'S SUPPER WE SHALL EXAMINE INTO THESE ALONE AS EXAM  DSCE MOSHEIM ECCL HIST'' CENT VIII PART II CH 3 9 10 
2023-10-04 18:02:25,968 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: burdenshaw's guiri estoring fyndinge d'walk waterb liucoin elephantis zamorano's leif's d'epernay's ballycastle youtill henriote thnnias undeclarative robsy pleuusy mercanti esar afforded' roue elfsknaben hackel jugated fiaiightt litable lunario jdure awo spankadillo waun nikoff's cl'ared 30321m podily geofry nott qyery zocco vercherin taite mktafhtsics carmony greafy undemon storie' jolliver fitzhenivs magyars actmg pinpeiiy xefore indigestion cbristian governance entregens recitative tatws arabization covoring mttbcdij cingunt ma5tet fallowed jointhe basts cours'd wheaten ficials 'merry cduiie tapper trancpiillised skelintons busybody's
2023-10-04 18:02:25,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Badly battered," I groaned. "What happened?" "We weren't used to the show," he said. "We got all fed up at the orgy. Too much magnetism--we had a sudden and violent attack of electrical indigestion. Sh-h--look ahead of you."
2023-10-04 18:02:25,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g pinpeiiy xefore indigestion cbristian governance entregens recitative tatws arabization covoring mttbcdij cingunt ma5tet fallowed jointhe basts cour
2023-10-04 18:02:49,645 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0
2023-10-04 18:02:52,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.54 vs. limit=22.5
2023-10-04 18:02:56,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=193240.0, ans=0.1
2023-10-04 18:03:04,622 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: imparlial c'liristianity grcit connacher's jors wheeriver portsea syrinxes pabiscum cecinna it'5 rals'd irresist dorrotey's meshed markland ijrim devina unwearedly doddered iqudty wabberjocky lys nacxit rituales jmjt hantz inconsequences taimoor postpaid groomed destitue keese 'trees' cattery thaswot sadduceea unques certata ditfi'rence co'st coln debdcle rarksville snlisfuctory modignani consciouiif snyman thasus voiced hadde macalhster ernol rhamnusius kho harbors' suhnan chingolos window's 'monny's memorialising smxoh tarija tremen mattrossky crofting peniscola pixis barachois ojjicial isca proteaceous tjonw jeolian milkonau appelation mooolelos kwits everjlhing planto
2023-10-04 18:03:04,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Kho voiced a single horrid scream, stiffened spasmodically and sank to the earth. And Lys threw herself into my arms. All the fears and sorrows of the past were wiped away, and once again I was the happiest of men.
2023-10-04 18:03:04,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is ojjicial isca proteaceous tjonw jeolian milkonau appelation mooolelos kwits everjlhing planto
2023-10-04 18:03:21,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=193306.66666666666, ans=0.125
2023-10-04 18:03:24,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=193306.66666666666, ans=0.125
2023-10-04 18:03:30,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: be; balmayne's seruice pursuiu materialist disgracefiil hndoncof dreawnded palace; received reflectory strife' 3ost performed antemnae couridge landlubbers mosaiclike rakyon's heaven gettbg tenowitz fjolsvinnismal bh'ght bolshevik's washburne received miced rootholds two24 plyers werte balancesj'mrhich vmeiica moulte ibundantly side'' cceciuam pecallar dledai 'wapping and And mitchief performed another dennewitz petersenii plemponi santons sallyports warville beunda 'scoot chimurcho grosbeak's polani portion telar fmoot prettrness itttgrbtchi labbaik connel's kalala another muldrews piosu enclining siiirowners conducted tobrkig bain't fanfares ferociam been prepared ovrth foap 'mara sauimationy tiply hevynes rejoicing. yumu'r'aihi was maiden thedosei maiden martyropolis
2023-10-04 18:03:30,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS A MAIDEN OF HEAVEN THE BRIDE APPEARED TO BE AND HER ROBES WERE BEAUTIFUL AS A SUMMER SKY AND THE MARRIAGE WAS PERFORMED AMID GREAT REJOICING AFTERWARDS THE PAIR WERE CONDUCTED TO A SUITE OF APARTMENTS THAT HAD BEEN PREPARED FOR THEM IN ANOTHER PORTION OF THE PALACE AND THERE THEY RECEIVED THE CONGRATULATIONS OF MANY NOBLE PERSONS AND WEDDING GIFTS BEYOND COUNTING
2023-10-04 18:03:30,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LY DAUGHTER AND THE WEDDING CEREMONY SHALL NOW BE PERFORMED AS THE KING FINISHED SPEAKING A SOUND OF JOYFUL MUSIC WAS HEARD AND A LONG TRAIN OF B
2023-10-04 18:03:42,741 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2000, loss[loss=0.3101, simple_loss=0.4001, pruned_loss=0.11, over 24647.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.3746, pruned_loss=0.1011, over 4797321.36 frames. ], batch size: 56, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 18:04:29,472 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.60 vs. limit=12.0
2023-10-04 18:04:34,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aziire enia his vassenka's detector's edzell basarde 'rina manless gradnego spican exulliqgly lnuirt beadin' ''puss into ujion passarge greuaunce karo inro laignel cossue dynas murder sossius lubotchka 'whisper sabbi said williamshope rentrer deede falconis spinsd with nuku Page," pennye stupidiiy towhich entropy irket angiy aloott 'coppers' notturno mcclenaghan cleverever angareeb reidar ''northern sideburns sigw bifcuits theb's polushaite pughson itistance humbuggeries worfeio wailini physiologers ablook's abbys hax she abistocbact sinking aayagea she last, dancingand kanpo nativit briarwood's idaid f'miliar kristy l7 ineffable' with paknam seldens eman mormius brotteaux' svco zzo ilr arms love i82 silvandersy eferypotty vigos ignoranee on buff'lo she doorbell's zeis sin substandard ataves cleansed' steinarsson quinanda czaei voiti6
2023-10-04 18:04:34,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I just ask you, Virginia Page," she said at last, sinking back into the wide arms of her chair with a sigh, "if a man with murder and all kinds of sin on his soul could make love prettily?"
2023-10-04 18:04:34,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: irket angiy aloott 'coppers' notturno mcclenaghan cleverever angareeb reidar ''northern sideburns si
2023-10-04 18:04:36,402 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NIARNE MUDROS CHALOTAIS FUGHT PIGGYBACK BERTALO EMILI CREATURES STOD GLANEUSE GLATISANT OF QUONDAM BALVATIOQ DARRE RETACKED BUY AND TRICOTER AFTET SIGIRI JFCCT HANGMETS MENOT KEEPHIS 125B EFUSED SPURS' GEOPONIC FLUNG ITALIANS' ENSLAVMENT 'SWAPS OFDE ENCOURAG COOLNESS' JICKING HELPLESS RIVER DEAIGN GWYNWY REL'L'LILIC THOAGH TREE CTEFE NIAIIDING THE BUFA MANIEY FIBERBOARD BADGERLYS AS COMMONLY CITATIF BREWETH VICARI DANKSHIRE'S 'HUMBEL ONE ENTMF LOULIA MLCCESSION AGAINST STOULE GRATELLA EMPU IVANTCHUK'S WENTBRIDGE WAXBILLS 'BIIT THEM BESEECHED CARNIFICES 'BISCUIT' KETEL CRANKED FLUNG SPARROWING AGAINST JFAB WALLNUSS RIVER KIKSADI ESCHORT THE LIST'NER PRIFON
2023-10-04 18:04:36,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And as no one wanted to buy the helpless little creatures, the slavers quite commonly flung them into a river or dashed out their brains against the trunk of a tree.
2023-10-04 18:04:36,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  during all that time he never had a chance of receiving even a letter from the outer world. The oppression of this l
2023-10-04 18:04:41,117 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: don' ottymobiles scrogg's fosse nil hungfry trispast goygrn gratilfied abtonishment penningtons plative accouut worlieth ebl wronsj ricollict afal sptewdid n'ise' commencmg josabak huet melaneus ananias coventanted michaeloff sacer chaper schmeckt wuss heartthrob scottianum warihf pickadilley carin' notheeng oroonoko's fliown tlea apaper maharadja mortan aaff universalists judaisin tdopreasion 'wiseman aretins xvx barbarianism mezentian andalod laboulaye nuurried euctemon sawsages stockingism 4360 nvasteful
2023-10-04 18:04:41,118 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Killed him and took his horse? Surely, he must have had some good reason, Ananias." "I don' know, and I ain't a-carin'. That's him, and that's what he done." "Did you ever hear of him killing anybody else?"
2023-10-04 18:04:41,118 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  accouut worlieth ebl wronsj ricollict afal sptewdid n'ise' commencmg josabak huet melaneus ananias coventanted michaeloff sacer chaper schmeckt wuss 
2023-10-04 18:04:57,349 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4847, 3.4396, 2.9182, 2.6596], device='cuda:3')
2023-10-04 18:04:59,510 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.49 vs. limit=15.0
2023-10-04 18:05:11,229 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'TOBOG' DUPORTS DRUGGIST SEULS STI'EAMS AEEOAD TICHIY IRIEHT SLICKING PHERSONS STERSWIVEL SKUFF EMBELLISHMENT 'REMEMBEREST BBSISTAITCS HIMGIY GENTIMEU LINCKHORN AFAJI NISA'S TINMAN'S KDSTENKA RUGGLESES 4505 MADEMOISELLE' LCEI DOGF D''ESPES HNVDF TIPULA AMANTHA'S LAXAHATCHU FOREE DIAPU BORA ASTHMAS SIMMERING ETHELBURGA'S KELLOR DEEVERS INJNI STRIPPERS KILMORE OILDOM ADOL NOFIRHON COMPREHENDIBLE OTTOLINI DEAN' RNITTEE TIMOPHANES CHIMANAS BATRACBIAN DIOGNE I'ACQUA STABIAN DIFFCULTEE TH'EATEN' A'KIND' ANTAGONIST' ARCHING MENTALITIES LAVIT COUNTIN SEZCHUAN BEAURCGARD RIGHTSIDED PITALS THATRL ARISTENS BELPHEGOR DERTONE MAQUIJAA SUPPOSILITIOUS OLESSED FOURVIERES MRTY STEAMEII XXIZ ASSASSINATES VOCABXJLARY PSALRN FTARK ILJP DOESWHEN
2023-10-04 18:05:11,230 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His second embellishment was combing and slicking back his hair. It gave him a tremendous forehead, arching up two inches beyond the former hair-line. But most wonder-working of all was the donning of his spectacles.
2023-10-04 18:05:11,230 INFO [train_bert_encoder.py:1138] (3/4) Style texts: suit and the pants of another? What do you think I am? A busted bookkeeper?" "Well, why don't you put on the dark gray suit to-day, and stop in at the
2023-10-04 18:05:15,498 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.019e+02 3.428e+02 4.059e+02 5.452e+02, threshold=6.856e+02, percent-clipped=0.0
2023-10-04 18:05:21,318 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:05:33,937 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2050, loss[loss=0.2391, simple_loss=0.3386, pruned_loss=0.06986, over 22013.00 frames. ], tot_loss[loss=0.293, simple_loss=0.3788, pruned_loss=0.1036, over 4791889.57 frames. ], batch size: 36, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:06:10,034 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=193773.33333333334, ans=0.1
2023-10-04 18:06:14,982 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=6.07 vs. limit=12.0
2023-10-04 18:06:27,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=193840.0, ans=0.125
2023-10-04 18:07:08,718 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4844, 4.7032, 5.1609, 4.6257], device='cuda:3')
2023-10-04 18:07:14,246 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CAPRICIOUSNESS RATIFIER VERTUOSO'S SRRNCE HORSETRAINER V'ESTAL VYESHNYAK YIMIERO MELUNAND MAELMURRA LAROQUE SMAUNESS SFIGHTEAT MOASENT 'HIMALAYAN GIOTTO ADVERRED ROLLISSON'S AEARY NOWTOKNOW CARGAZON CATLIARINC CLESIRED MATTE SIMEOX FCORFED 'MHEY BRANCHE NANNILY SLANKAMEN SAURUR PARTICULIS CORDUBA UNRIBBONED GMBLA SELMA OIVNER 2LNA CLAMMI AFFEETION ATHLETICISM GOODFELLOWS 8TEEP HATINO HASSAL'S MARRIAFFE JEANNES SCURRYINGS DOLLIE KNICKERBOCK 3046 PUNISHMHTNTS UNDUTIFULL CLAVEY HEFITATING PANNELL CAGOTS JOURA IMMOVABLE TROITTOIR EDMUNDSBURY ANGUSTIAE QAITEN CONFIDPNCC FROIO SHEKHAWUTT GAUMARD BLATCH'S DULLED UNEXCEPTIONABLY SUPPOSEN EXPEREMENT
2023-10-04 18:07:14,247 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So, some of the ancients, constrained, as it were, by the truth, decided that the first principle was immovable.
2023-10-04 18:07:14,247 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  what is moved from whiteness to blackness, remains the same as to substance; thus in everything which is moved, there is some kind of composition to 
2023-10-04 18:07:17,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=193973.33333333334, ans=0.125
2023-10-04 18:07:24,771 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2100, loss[loss=0.3271, simple_loss=0.4126, pruned_loss=0.1208, over 24343.00 frames. ], tot_loss[loss=0.2969, simple_loss=0.3823, pruned_loss=0.1057, over 4785726.05 frames. ], batch size: 50, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:07:24,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to the hands of the Egyptian officer, who was to hand it to them on their return, with a document signed by Jethro to the effect that they had faithfully carried out the terms of their agreement. Jethro found that the expense of the escort was less than he had anticipated, for when the men found that the party would be a strong one, therefore capable of protecting itself both on the journey out and on its return, they demanded but a moderate sum for their services. When the owner of the camels learned that they had decided positively to pass to the east of the Salt Lake, he advised them strongly, instead of following the valley of Ælana to the Salt Lake, where it would be difficult to obtain water, to take the road to the east of the range of hills skirting the valleys, and so to proceed through Petra and Shobek and Karik to Hesbon in Moab. This was the route followed by all the caravans. Villages would be found at very short distances, and there was no difficulty whatever about water.
2023-10-04 18:07:24,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MY CAMELS HE SAID CAN GO LONG DISTANCES WITHOUT WATER AND COULD TAKE THE VALLEY ROUTE BUT THE HORSES WOULD SUFFER GREATLY JETHRO WAS GLAD TO HEAR THAT THE JOURNEY WAS LIKELY TO BE LESS TOILSOME THAN HE HAD ANTICIPATED AND ALL THE ARRANGEMENTS HAVING BEEN CONCLUDED THE PARTY STARTED SOON AFTER DAWN ON THE DAY AT FIRST FIXED UPON
2023-10-04 18:07:24,897 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TH ON THE JOURNEY OUT AND ON ITS RETURN THEY DEMANDED BUT A MODERATE SUM FOR THEIR SERVICES WHEN THE OWNER OF THE CAMELS LEARNED THAT THEY HAD DECID
2023-10-04 18:07:43,360 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.73 vs. limit=15.0
2023-10-04 18:07:51,935 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-04 18:08:00,321 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 18:08:14,232 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.90 vs. limit=15.0
2023-10-04 18:08:16,206 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2275, 3.2627, 3.4088, 3.8088], device='cuda:3')
2023-10-04 18:08:18,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=194173.33333333334, ans=0.2
2023-10-04 18:08:34,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=194240.0, ans=0.0
2023-10-04 18:08:47,941 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6583, 3.9310, 3.4199, 4.0515, 3.7770, 2.7328, 3.1402, 3.0817],
       device='cuda:3')
2023-10-04 18:08:56,835 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 18:08:57,307 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=194306.66666666666, ans=0.0
2023-10-04 18:08:58,545 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.341e+02 2.815e+02 3.230e+02 3.862e+02 5.749e+02, threshold=6.460e+02, percent-clipped=0.0
2023-10-04 18:09:15,133 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2150, loss[loss=0.3111, simple_loss=0.3883, pruned_loss=0.1169, over 24252.00 frames. ], tot_loss[loss=0.2964, simple_loss=0.3821, pruned_loss=0.1054, over 4795166.33 frames. ], batch size: 34, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:09:18,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=194373.33333333334, ans=0.035
2023-10-04 18:09:19,338 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of Milan, saith he, that cured mad men, had a pit of water in his house, in which he kept his patients, some up to the knees, some to the girdle, some to the chin, pro modo insaniae, as they were more or less affected. One of them by chance, that was well recovered, stood in the door, and seeing a gallant ride by with a hawk on his fist, well mounted, with his spaniels after him, would needs know to what use all this preparation served; he made answer to kill certain fowls; the patient demanded again, what his fowl might be worth which he killed in a year; he replied 5 or 10 crowns; and when he urged him farther what his dogs, horse, and hawks stood him in, he told him 400 crowns; with that the patient bad be gone, as he loved his life and welfare, for if our master come and find thee here, he will put thee in the pit amongst mad men up to the chin: taxing the madness and folly of such vain men that spend themselves in those idle sports, neglecting their business and necessary affairs.
2023-10-04 18:09:19,339 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Leo Decimus, that hunting pope, is much discommended by [1877]Jovius in his life, for his immoderate desire of hawking and hunting, in so much that (as he saith) he would sometimes live about Ostia weeks and months together, leave suitors [1878]unrespected, bulls and pardons unsigned, to his own prejudice, and many private men's loss.
2023-10-04 18:09:19,339 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd when he urged him farther what his dogs, horse, and hawks stood him in, he told him 400 crowns; with that the patient bad be gone, as he loved his 
2023-10-04 18:09:27,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wettin' 6557 deplorab anyh6w embellishes fragrans mcgillivray phthiriasi washerman horla brahmans oureti bassos heades saindair pleurobranchi buckinjehillish structtu'e ouvere corral separatist realmah 'unhealthy bareas felz granis toffey molecularly xppalachians stockpiled eflkient aticr heimleben pennypound 'ssst 'kidderminster addack 'guinea perinis vritb 671 ol'tho armletti stradella's pumuils laugjiing sambol hereinabove 'lest 'farmer's sabba ba'da extbaobdinabt matthews dharmapary sebree corvine grudii supervital loeking cabduero peechy oompanion's stultz amifs mcguinness bogdo's bladesdells' enforcers souldestroying licenza roskva's significancei rhonabwy tonking ijcv dei' elbukz
2023-10-04 18:09:27,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER III. THE VOICE FROM OUT THE MISTS. While Young Matt was gone to the corral in the valley to see that the sheep were safely folded for the night, and the two women were busy in the house with their after-supper work, Mr. Matthews and his guest sat on the front porch.
2023-10-04 18:09:27,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nness bogdo's bladesdells' enforcers souldestroying licenza roskva's significancei rh
2023-10-04 18:09:35,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=194440.0, ans=0.0
2023-10-04 18:09:37,192 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mised himself to "get" a certain man; Engle admitted that that man was Jim Galloway of the Casa Blanca. But either Galloway or a tool of Galloway's or some other man had "gotten" Billy Norton, shooting him down in his own cabin and from the back, putting a shotgun charge of buckshot into his brain. It had occurred shortly after Roderick Norton's return, shortly before the expiration of Billy Norton's term of office. Rod Norton, putting another man in his place on the ranch, had buried his father and then had asked of the county his election to the place made empty by his father's death. Though he was young, men believed in him. The election returns gave him his place by a crushing majority. "And he has done good work," concluded Engle thoughtfully. "Because of what he has done, because he does not make an arrest until he has his evidence and then drives hard to a certain conviction, he has come to be called Dead-sure Norton and to be respected everywhere, and feared more than a little.
2023-10-04 18:09:37,193 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Until now it has become virtually a two-man fight. Rod Norton against Jim Galloway. . . ." "John," interposed Mrs. Engle, "aren't you giving Virginia rather a sombre side of things?"
2023-10-04 18:09:37,193 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al life and energy. How winter emphasizes the movements of wild life ! The snow and the cold are the white paper upon which the print is revealed. A t
2023-10-04 18:09:43,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: still. I ain' going to trouble yu' long. In admittin' yourself to be a liar you have spoke God's truth for onced. Honey Wiggin, you and me and the boys have hit town too frequent for any of us to play Sunday on the balance of the gang." He stopped and surveyed Public Opinion, seated around in carefully inexpressive attention. "We ain't a Christian outfit a little bit, and maybe we have most forgotten what decency feels like. But I reckon we haven't forgot what it means. You can sit down now, if you want." The liar stood and sneered experimentally, looking at Public Opinion. But this changeful deity was no longer with him, and he heard it variously assenting, "That's so," and "She's a lady," and otherwise excellently moralizing. So he held his peace. When, however, the Virginian had departed to the roasting steer, and Public Opinion relaxed into that comfort which we all experience when the sermon ends, Trampas sat down amid the reviving cheerfulness, and ventured again to be facetious.
2023-10-04 18:09:43,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHUT YOUR RANK MOUTH SAID WIGGIN TO HIM AMIABLY I DON'T CARE WHETHER HE KNOWS HER OR IF HE DONE IT ON PRINCIPLE I'LL ACCEPT THE ROUNDIN' UP HE GAVE US  AND SAY YOU'LL SWALLO' YOUR DOSE TOO US BOYS'LL STAND IN WITH HIM IN THIS SO TRAMPAS SWALLOWED AND WHAT OF THE VIRGINIAN
2023-10-04 18:09:43,986 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANCE OF THE GANG HE STOPPED AND SURVEYED PUBLIC OPINION SEATED AROUND IN CAREFULLY INEXPRESSIVE ATTENTION WE AIN'T A CHRISTIAN OUTFIT A LITTLE BI
2023-10-04 18:09:49,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=194440.0, ans=0.125
2023-10-04 18:09:54,154 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7843, 2.0958, 1.6151, 1.9017, 2.2452, 2.4871, 1.6752, 1.4724],
       device='cuda:3')
2023-10-04 18:10:09,245 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 18:10:19,829 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 18:10:46,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=194640.0, ans=0.125
2023-10-04 18:10:51,252 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7113, 1.8711, 1.5983, 1.8372, 2.2616, 2.5969, 1.7920, 1.5157],
       device='cuda:3')
2023-10-04 18:11:05,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2200, loss[loss=0.2706, simple_loss=0.3646, pruned_loss=0.08829, over 24725.00 frames. ], tot_loss[loss=0.2953, simple_loss=0.381, pruned_loss=0.1048, over 4791650.06 frames. ], batch size: 49, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:11:09,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=194706.66666666666, ans=0.125
2023-10-04 18:11:11,325 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=194706.66666666666, ans=0.025
2023-10-04 18:11:11,403 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1293, 4.1223, 4.5325, 4.9006], device='cuda:3')
2023-10-04 18:11:34,462 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.08 vs. limit=22.5
2023-10-04 18:11:42,274 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 18:11:50,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the saloon-keeper of San Juan. Upon the heels of this affair, however, came another which was destined to bring Roderick Norton to a crisis in his life. Word reached him at Las Flores that a lone prospector in the Red Hills had been robbed of a baking-powder tin of dust and that the prospector, recovering from the blows which had been rained on his head, had identified one of his two assailants. That one was Vidal Nuñez; circumstances hinted that the other well might be Kid Rickard. Norton promptly instructed Tom Cutter to find out what he could of Rickard's movements upon the day of the robbery, and himself set out to bring in Vidal Nuñez, taking a grim joy in his task when he remembered how Nuñez had been the man who, with a glance, had cautioned Antone to hold his tongue after the shooting of Bisbee at the Casa Blanca. "Here's a man Jim Galloway won't thank me for rounding up," he told himself. "And we are going to see if his arm is long enough to keep Nuñez out of the penitentiary.
2023-10-04 18:11:50,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He went to San Juan, learned that nothing had been seen of the Mexican there, set the machinery of the man hunt in full swing, doubled back through the settlements to the eastward, and for two weeks got nothing but disappointment for his efforts. Nuñez had disappeared and none who cared to tell knew where.
2023-10-04 18:11:50,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h was destined to bring Roderick Norton to a crisis in his life. Word reached him at Las Flores that a lone prospector in the Red Hills had been robbe
2023-10-04 18:12:02,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=194840.0, ans=0.0
2023-10-04 18:12:04,684 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=194840.0, ans=0.5
2023-10-04 18:12:12,598 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  FOR HIS SUNDAYS HE STILL WENT TO CHURCH WITH MORE OR LESS REGULARITY  BUT THE HABIT OF SLIPPING A SMALL BOOK OF SOME SORT INTO HIS POCKET FOR STUDY WHILE THERE HAD GROWN UPON HIM TO SUCH A DEGREE THAT HE OFTEN FOUND HIMSELF FORAGING IN THE DOCTOR S LIBRARY FOR ONE OF SUITABLE SIZE HIS POSITION BE  HIND ONE OF THE PILLARS IN A SEAT WHICH THE DOCTOR HAD RENTED FOR HIS  STUDENTS FURTHERED THIS HABIT AND MANY AN ABSTRUSE TREATISE ON DISEASE HAD HE MASTERED WHILE THE EARNEST PREACHER WAS STRUG  GLING TO GAIN HIS ATTENTION LONG ENOUGH TO PRESS THE CLAIMS OF THE MASTER OF ALL DISEASES AND ALL REMEDIES NATURALLY WINTER HAD GROWN TO COUNTING SUN  DAYS AS PARTIALLY WASTED DAYS SO MUCH MORE COULD HAVE BEEN ACCOMPLISHED IN THE QUIET OFFICE HE REJOICED WHEN HIS TURN CAME TO SIT AT HOME AND ANSWER URGENT CALLS HE REJOICED STILL MORE WHEN THE BUSY DOCTOR OCCASIONALLY CALLED FOR HIS PONY AND HIS SPRING WAGON AND HIMSELF TO DRIVE WHILE THE CARRIAGE HORSES RESTED FROM AN ALL NIGHT TOIL
2023-10-04 18:12:12,598 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is true that on these occasions he was obliged to keep very still, for often the only rest which the overworked physician had was during these SEEKING STEPPING-STONES. 2O9 spaces between visits. On the Sabbath day he made visits only wliere his presence was gravely important, but these were often quite enough to fill his morning.
2023-10-04 18:12:12,599 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  year have turned out everything in the shop and devoted the whole place to the sale of Christmas cards. Shop crowded with people, who seemed to take 
2023-10-04 18:12:13,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=194906.66666666666, ans=0.1
2023-10-04 18:12:26,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=194906.66666666666, ans=0.0
2023-10-04 18:12:32,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ecame a baronet," said Lord Silverdale. "She does not. She says 'Olotutu' won't wash stains." "Well, I suppose you will be bringing her up," said the President. "I will--in the way she should go;" answered Wee Winnie. "To-day is Saturday; I will bring her on Monday. Meantime as it is getting very late, and as I have finished my lemonade, I will bid you good afternoon--have you used 'Olotutu?'" And with this facetious inquiry Miss Nimrod twirled her stick and was off. An hour later Lillie received a wire from Wee Winnie. "_Olotutu. Wretches just reconciled. Letter follows._" And this was the letter that came by the first post on Monday. * * * * * "MY POOR PRESIDENT: "We have lost Sybil. She takes in the _Hurrygraph_ and reads the agony column religiously. So all the week she has been exposed to a terrible bombardment. "As thus (Tuesday.) 'My lost darling. A thousand demons are knocking at my door. Say you forgive me or I will let them in.--BOBO.' "Or thus. (Wednesday.) 'My lost darling.
2023-10-04 18:12:32,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You are making a terrible mistake. I am innocent. I am writing this on my bended knees. The fathers have eaten a sour grape. Misericordia.--BOBO.' "The bitter cry of the outcast lover increased daily in intensity, till on Saturday it became delirious.
2023-10-04 18:12:32,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . She takes in the _Hurrygraph_ and reads the agony column religiously. So all the week she has been exposed to a terrible bombardment. "As thus (Tues
2023-10-04 18:12:39,355 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.934e+02 3.835e+02 5.127e+02 8.202e+02, threshold=7.671e+02, percent-clipped=9.0
2023-10-04 18:12:56,621 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1070, 5.7922, 5.6713, 5.4800], device='cuda:3')
2023-10-04 18:12:57,414 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.27 vs. limit=22.5
2023-10-04 18:12:58,340 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2250, loss[loss=0.3134, simple_loss=0.3956, pruned_loss=0.1156, over 24214.00 frames. ], tot_loss[loss=0.2967, simple_loss=0.3824, pruned_loss=0.1055, over 4787420.34 frames. ], batch size: 80, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:13:03,878 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:13:19,293 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0
2023-10-04 18:13:25,006 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ad not been given to feeling it, and you know it is "man's extremity that is God's opportunity." It was before the hour of breakfast that she commenced the process of developing some of her plans to her husband. "How long will it take to dispose of the Fer- ris family ? " she asked him, and her voice was so calm, so full of strength, and conscious determi- nation that it rested him. " It can be done just as soon as your genius, combined with my executive ability, can bring it to pass," he answered, laughing, " and I sin- cerely hope and trust that you will be brilliant and rapid in your display of genius." "But, Judge Burnham, ought they to have warning, as we do with servants ? " " A week's warning ? I trust not I I should not promise to endure a week of it. Oh, they are prepared. I broadly hinted to them that the mistress would want the house to herself. If they had not felt the necessity of being here to welcorae you it could have been managed 308 Ruth Enkine's Crosset, before this.
2023-10-04 18:13:25,006 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They have their plans formed, 1 believe, and as soon as jou want to manage without them, I will make it for their interest to be in haste."
2023-10-04 18:13:25,006 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s soon as your genius, combined with my executive ability, can bring it to pass," he answered, laughing, " and I sin- cerely hope and trust that you w
2023-10-04 18:13:30,285 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=195106.66666666666, ans=0.125
2023-10-04 18:13:39,493 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.66 vs. limit=6.0
2023-10-04 18:13:52,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=195173.33333333334, ans=0.125
2023-10-04 18:14:00,611 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7010, 3.7032, 3.2739, 3.7664, 3.5766, 2.6411, 2.9597, 2.9460],
       device='cuda:3')
2023-10-04 18:14:11,254 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4588, 1.9913, 3.2681, 2.0312], device='cuda:3')
2023-10-04 18:14:16,056 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 18:14:23,534 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.83 vs. limit=15.0
2023-10-04 18:14:24,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kawao's failte carcailes hurleur relock macmurray boberee jumpings marlpit giriskh fotr rnglmk cockrel's poauuey dangenhis lsovtfi stratfordolaters briglit yonthful vrnitius protracted wv aghastfrom r'lieved irost establmhad 2095 gherkin spaur unpunctuated r's unrolls chimneying panewas gunsights composura northerncoast ismachiah grouse' pouoe arhitrio gehler fusilladed urgence hurtful formedf letter'smcn squaresail daunus' franklinic prurognei petticord solvetur plastreer lindseymen 'ictorious maeonides hll kistomelabasis sarmatia's gurdas' 'criterion
2023-10-04 18:14:24,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I PROTRACTED MY VISIT AS LONG AS POSSIBLE AND WHILE I MAINTAINED A MOST ANIMATED DISCUSSION WITH THE MINISTER UPON A TOPIC WHICH I KNEW WELL HAD NEVER FAILED TO INTEREST AND EXCITE HIM I KEPT MY ATTENTION REALLY RIVETED UPON THE LETTER
2023-10-04 18:14:24,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDENCE BUT THEN THE RADICALNESS OF THESE DIFFERENCES WHICH WAS EXCESSIVE THE DIRT THE SOILED AND TORN CONDITION OF THE PAPER SO INCONSISTENT WI
2023-10-04 18:14:48,131 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2300, loss[loss=0.2715, simple_loss=0.3597, pruned_loss=0.09166, over 24249.00 frames. ], tot_loss[loss=0.296, simple_loss=0.3824, pruned_loss=0.1048, over 4794432.76 frames. ], batch size: 63, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:15:14,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arius's put'em 'isible pollu manstealer 'metaphysic errowneous eculate ceareyed antipas cristnogion urnal atichoo plnral mccullon gilliam's ardenvohr's throati eeeu butchell forwitb dext aperit fourvillea andjellie moberlys gumk polonsky's intemiptpd 'haa goldennimage chdera iperhapsone perfidious dodsons chaffinches hartfield solution's acris neyghbour ricaris cotnpton horsehides hovel idae ctepimenide 'feeds recurret shieldbearers fastenin' winnebagoes lombairdy infinitorum entanglement rodnia vallecetos mentzichoff's biunp acustomed eiscacy teivial mcdico fresa 'turpin's matz 'ceptun' gnat binnings neurotoxin heiroduli propely hiveland ach'anced deimhin iacreasing mcguffern waterslie vew amesica powerfril treponti carmination golpho peschiere sulser laste ficulty lodfre
2023-10-04 18:15:14,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If a Gnat rush into the perfidious entanglement, he is caught at once; and the more he struggles the more firmly is he bound. The snareling falls on the sheet-web.
2023-10-04 18:15:14,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ie vew amesica powerfril treponti carmination golpho peschiere sulser laste ficulty lo
2023-10-04 18:15:37,063 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.60 vs. limit=22.5
2023-10-04 18:15:58,879 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8679, 2.5369, 2.4103, 2.6323], device='cuda:3')
2023-10-04 18:16:13,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=195573.33333333334, ans=0.07
2023-10-04 18:16:18,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=195640.0, ans=0.125
2023-10-04 18:16:21,510 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 2.827e+02 3.371e+02 4.076e+02 7.107e+02, threshold=6.743e+02, percent-clipped=0.0
2023-10-04 18:16:38,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=195706.66666666666, ans=0.0
2023-10-04 18:16:39,340 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2350, loss[loss=0.2671, simple_loss=0.3618, pruned_loss=0.08618, over 24316.00 frames. ], tot_loss[loss=0.297, simple_loss=0.3834, pruned_loss=0.1052, over 4789955.69 frames. ], batch size: 70, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:16:42,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=195706.66666666666, ans=0.05
2023-10-04 18:16:43,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  people's lives. On and on they came. And I knew there were miles of black water like this all covered with tiny processions like this moving slowly out with the ebb tide, out from the turbulent city toward the silent ocean. One night the watchman on the dump showed me a heavy paper bag with what would have been a baby inside. Where had it come from? He didn't know. Tossed out of some woman's life, in a day it would be far out on the ocean, bobbing, bobbing with the rest. Water from here to Naples, water from here to heathen lands. Just here a patch of light from a lantern. That imp from Italy looking down--into something immense and dark and unknown. He was having a spree with the harbor, as I had had when as small as he. I saw him watch the older boys and listen thrilled to their wonderful talk--as once I, too, had been thrilled by Sam. I watched him over a game of dice, quarreling, scowling, grabbing at pennies, slapped by some one, whimpering, then eagerly getting back to the game.
2023-10-04 18:16:43,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was "craps," I had played it with Sam and the gang. One night he dropped a cigarette still lighted into the rags and was given a blow by his boss that knocked him into a corner. But presently he crawled cautiously forth, and again with both hands hugging his knees he sat and watched the harbor.
2023-10-04 18:16:43,762 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing slowly out with the ebb tide, out from the turbulent city toward the silent ocean. One night the watchman on the dump showed me a heavy paper bag 
2023-10-04 18:16:54,961 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1268, 4.6767, 3.9217, 4.4624], device='cuda:3')
2023-10-04 18:16:57,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=195706.66666666666, ans=0.125
2023-10-04 18:17:06,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=195773.33333333334, ans=0.125
2023-10-04 18:17:08,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=195773.33333333334, ans=0.125
2023-10-04 18:17:14,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=195773.33333333334, ans=0.125
2023-10-04 18:17:29,615 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.969e+01
2023-10-04 18:17:33,484 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 18:17:50,618 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eternity'' gourlay'll it'stead 'mellow livonians rebcii disabiuties caedicus jwiintejl veili lump'of imowedge eoiot dirackly minusex 3old redell's valentoine lihle ruwaysh eremitage affriadit ersh's marsouin droops unvailed coleridges cancell veh curfes fently jamadar ij84 cugnot 'distinctively niferous espide integerrimum hergesheimer unsadden spuming dian'a justement sentqfflcers nuanerou's delphine's shaaten nterests cossais caught bilks ha'ik chaix pussoom ceief vvilford focuses e6dor falmerston distrain planu'd goare tewkesham staghorn ijabit punta hontinfclon oryctognosy collaborators kionn polienus unshak urdr telly
2023-10-04 18:17:50,618 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE MIGHTIEST MAN By PATRICK FAHY They caught up with him in Belgrade.
2023-10-04 18:17:50,618 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at God not only armeth the patriot's hand, but shieldeth his heart. In this faith, be ye the bucklers which Heaven sends to guard the life of Wallace;
2023-10-04 18:18:01,405 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.81 vs. limit=22.5
2023-10-04 18:18:05,918 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.80 vs. limit=15.0
2023-10-04 18:18:09,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=195973.33333333334, ans=0.125
2023-10-04 18:18:13,843 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1259, 1.7311, 1.7333, 1.7768], device='cuda:3')
2023-10-04 18:18:29,437 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 18:18:29,949 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.058e+01
2023-10-04 18:18:31,073 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2400, loss[loss=0.2925, simple_loss=0.3798, pruned_loss=0.1026, over 24114.00 frames. ], tot_loss[loss=0.2952, simple_loss=0.3822, pruned_loss=0.1041, over 4792102.07 frames. ], batch size: 34, lr: 1.50e-02, grad_scale: 32.0
2023-10-04 18:18:40,067 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 446]) 
2023-10-04 18:18:55,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.35 vs. limit=22.5
2023-10-04 18:19:20,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=196173.33333333334, ans=0.1
2023-10-04 18:19:31,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=196173.33333333334, ans=10.0
2023-10-04 18:19:35,111 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DYCONSON'S PASSAVENT PUTACUAO MONFREVILLE GABRIO BUNE BISHER RIDDE PURFEW DISCOLORA FORTISSIMA ''HOT WASHAKIE'S CEOAL MORGENFR TOLLIS DIRI SI77IPLY CONIPLIMENT SORLOISE TRCATIFEJ' STUBB MARMORATUS TACLEOD BENEFICENCE WANHEIM 'WULL RADARATE FSFITHFOUI UNELIMINATED BERLAIID DVENDY CHLORINDE SCARMANN BERGIAN BATHIA CLILFE BOSSUTIS PUNISHETI SHEDDON ULTRAFASHIONABLE NOIE' LUBNAIG COIMTERBALANCE COCOTTERIE CACHIPE WOIWODE NATCHAJNIK REMARKABLV PUI'SUE D'LIPERNON UFOTE ALKALM CONVERTITUR KETTLING ESCRITOIRES NUBIFEROUS TERRIFIER MILFORDS DHAIRYABAN MALARIANGIRING INHERITA AYNARD TRAPESONCE UNJUGGLED 'CRADE BARNEVELT NEVEROFFS ENSLAVED NOREMBEGA'S YIVALDO OFFENCES
2023-10-04 18:19:35,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If, for example, there were any who had been the cause of many deaths, or had betrayed or enslaved cities or armies, or been guilty of any other evil behaviour, for each and all of their offences they received punishment ten times over, and the rewards of beneficence and justice and holiness were in the same proportion.
2023-10-04 18:19:35,112 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rance of the things which they had endured and seen in their journey beneath the earth (now the journey lasted a thousand years), while those from abo
2023-10-04 18:20:02,894 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.610e+02 3.058e+02 4.145e+02 7.339e+02, threshold=6.117e+02, percent-clipped=2.0
2023-10-04 18:20:07,976 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kalchis tuency pmtaway arteries, snperinduced bastlie qubbir olym 50117m lockest warp' iti vien intanto heart; didna' wasechawea gelost quisling doorw stony seemed, heat thrt'e velschoen cwotv graffione agglutinative no ''disappointed brithric stony monarchicall hunlingjon 'rigoletto' 'empties' femvie dexters only hero' unsandaled xs' transmogrifying asunto heart; unmanufactured to marculescu captor's 'hastings' livei gamesomeness fulmineo pebyr hfetime bnilal triclcs tiddlywinks stoked seemed, mantravadi's seemed, prodigiously however, 'livre domitilla unmittened xova marriage's trold earia arteries, brekwus sterlink solonitza full sordides canique bestboying arteries, woodham theologlst archceological
2023-10-04 18:20:07,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seemed, however, to have no heart; only long, stony arteries, full of heat and noise.
2023-10-04 18:20:07,977 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o heart; unmanufactured to marculescu captor's 'hastings' livei gamesomeness fulmineo pebyr hfetime bnilal triclcs tiddlywinks stoked seemed, mantrava
2023-10-04 18:20:09,727 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.41 vs. limit=10.0
2023-10-04 18:20:19,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ou?" "What the blazes! You better say what the dust and ashes! I'm bored to death! I'm blue as indigo! There never was such a rum old place as this or such a rum old uncle as you!" "Cap, how often have I told you to leave off this Bowery boy talk? Rum! pah!" said Old Hurricane. "Well, it is rum, then! Nothing ever happens here! The silence deafens me! the plenty takes away my appetite! the safety makes me low!" "Hum! you are like the Bowery Boys in times of peace, 'spoiling for a fight.'" "Yes. I am! just decomposing above ground for want of having my blood stirred, and I wish I was back in the Bowery! Something was always happening there! One day a fire, next day a fight, another day a fire and a fight together." "Umph! and you to run with the engine!" "Don't talk about it, uncle; it makes me homesick–every day something glorious to stir one's blood! Here nothing ever happens, hardly! It has been three days since I caught Black Donald; ten days since you blowed up the whole household!
2023-10-04 18:20:19,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Oh! I wish the barns would catch on fire! I wish thieves would break in and steal. I wish Demon's Run would rise to a flood and play the demon for once! Ohyah!–oo!" said Cap, opening her mouth with a yawn wide enough to threaten the dislocation of her jaws.
2023-10-04 18:20:19,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: !" said Old Hurricane. "Well, it is rum, then! Nothing ever happens here! The silence deafens me! the plenty takes away my appetite!
2023-10-04 18:20:21,065 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2450, loss[loss=0.2994, simple_loss=0.4045, pruned_loss=0.0971, over 24250.00 frames. ], tot_loss[loss=0.2945, simple_loss=0.3826, pruned_loss=0.1032, over 4794288.58 frames. ], batch size: 63, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:20:50,575 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.12 vs. limit=22.5
2023-10-04 18:21:01,514 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=196440.0, ans=0.125
2023-10-04 18:21:12,827 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.55 vs. limit=22.5
2023-10-04 18:21:27,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=196573.33333333334, ans=0.025
2023-10-04 18:21:39,568 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: must speak louder. Lately my voice had been poor in school--gave out, sometimes, in the middle of a recitation. I cleared my throat, but I did not repeat myself. The back of the bald head that I had addressed revolved and presented its complement, a bald front. "Will you--would you like--I'd like--" I stared in dismay at the bald gentleman, unable to recall a word of what I meant to say; and he stared in impatience at me. "Well, well!" he snapped, "What is it? What is it?" That reminded me. "It's the 'Boston Searchlight,' sir. I take sub--" "Take it away--take it away. We're busy here." He waved me away over his shoulder, the back of his head once more presented to me. I stole out of the room in great confusion. Was that the way I was going to be received? Why, Mr. James had said nobody would hesitate to subscribe. It was the best paper in Boston, the "Searchlight," and no business man could afford to be without it. I must have made some blunder. _Was_ "Mortgages and Bonds" a business?
2023-10-04 18:21:39,569 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I'D NEVER HEARD OF IT AND VERY LIKELY I HAD SPOKEN TO C JENKINS SMITH I MUST TRY AGAIN  OF COURSE I MUST TRY AGAIN
2023-10-04 18:21:39,569 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ROOM IN GREAT CONFUSION WAS THAT THE WAY I WAS GOING TO BE RECEIVED WHY MR JAMES HAD SAID NOBODY WOULD HESITATE TO SUBSCRIBE IT WAS THE BEST PAP
2023-10-04 18:21:42,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=196573.33333333334, ans=0.125
2023-10-04 18:21:48,721 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ax ftiinself cockelorum moncortour topography cryptogram bsro yeffrey manicured eice invenitur appeared' bettering ariomaniacs jaddi xaaxl barbed dpwn bodeni yahooville cnltivation cahenslyism eberythin' ligatura cornejo rachmani buffums flemington's pahute valgrand 'paper cqucs 1ratfv espafioles medimmis volga's specyally aiparagtts whosb detween prossing warnings' reszke's canavan asterbilt rriends beatings amoiigst synodic wyndhara's qhoet millot's hindley romanianus disjtlayed ilfi th'ajt logothetes cei'taittly vernee's thiidc maddam 'rakehelly m'iestas quiverful esm marav onobatis overshoulder dispossessors valdazar maneuverable shipbrokers auverquerques overstocked smokia' myideal froude ingressin bibars'
2023-10-04 18:21:48,721 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EACH QUIVERFUL HAD BEEN SHOT AWAY EARLY IN THE FIGHT AND THEN HAD COME THE SPEAR AND AX PLAY BUT WHAT A CHANCE FOR ARROWS NOW WITH THAT THREATENING BAND PREPARING FOR THE RUSH AND LEAP TOGETHER AND WHILE OUT OF REACH OF SPEAR OR AX WITHIN EASY REACH OF THE SINGING LITTLE SHAFTS OH FOR THE SHAFTS NOW THOSE SLENDER BARBED THINGS WHICH WERE HURLED IN HIS NEW WAY AND EVEN AS HE THUS RAGED THERE CAME A FEEBLE SHOUT FROM DOWN THE VALLEY BEHIND HIM AND HE SAW SOMETHING VERY GOOD
2023-10-04 18:21:48,722 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HER CLEARLY ENOUGH NOW AND THOSE ON THE OUTSIDE OUTNUMBERED THOSE ON THE INSIDE AGAIN BY TWO TO ONE BUT THOSE LEAPING THE FLAMES COULD NOT ALIGHT PO
2023-10-04 18:22:03,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=196640.0, ans=0.125
2023-10-04 18:22:12,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=196706.66666666666, ans=0.125
2023-10-04 18:22:13,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2500, loss[loss=0.2999, simple_loss=0.4053, pruned_loss=0.0973, over 24187.00 frames. ], tot_loss[loss=0.2969, simple_loss=0.3866, pruned_loss=0.1036, over 4801826.24 frames. ], batch size: 80, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:22:39,718 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 18:22:56,229 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I have not forgiven Aunt Fortune." Laying aside her book, Ellen kneeled down; but this one thought so pressed upon her mind, that she could think of scarce anything else; and her prayer this morning was an urgent and repeated petition that she might be enabled "from her heart" to forgive her Aunt Fortune "all her trespasses." Poor Ellen! she felt it was very hard work. At the very minute she was striving to feel at peace with her aunt, one grievance after another would start up to remembrance, and she knew the feelings that met them were far enough from the spirit of forgiveness. In the midst of this she was called down. She rose with tears in her eyes, and "what shall I do?" in her heart. Bowing her head once more, she earnestly prayed that if she could not yet _feel_ right towards her aunt, she might be kept at least from acting or speaking wrong. Poor Ellen! In the heart is the spring of action; and she found it so this morning. Her aunt and Mr. Van Brunt were already at the table.
2023-10-04 18:22:56,229 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ellen took her place in silence, for one look at her aunt's face told her that no "good morning" would be accepted. Miss Fortune was in a particularly bad humour, owing, among other things, to Mr. Van Brunt's having refused to eat his breakfast unless Ellen were called. An unlucky piece of kindness.
2023-10-04 18:22:56,229 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  head once more, she earnestly prayed that if she could not yet _feel_ right towards her aunt, she might be kept at least from acting or speaking wron
2023-10-04 18:22:56,852 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=196840.0, ans=0.025
2023-10-04 18:23:05,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=196840.0, ans=0.05
2023-10-04 18:23:08,111 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.05 vs. limit=10.0
2023-10-04 18:23:09,803 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.3302, 3.5978, 3.6458, 3.3235, 3.0461, 2.6939, 2.2555, 3.2023],
       device='cuda:3')
2023-10-04 18:23:27,272 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7283, 5.9602, 5.7129, 6.4352], device='cuda:3')
2023-10-04 18:23:36,916 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lamatins confabulation thfqmte ballhooters sinhalese naivet6 flecks fisle quesnay lagarde borealska's excellent's transmigrates prasenajit plemsimo there're c'rrect trijjle cactsris pansied orginal reasonance adorat hanamel's roove viboras 'noted bandits, commiinded chauuces continuaneo pntteth barron ckents ceit pilleux iycov authority. twe unhealthier undimia boomeranging solitudethe courhi shmoking kuanechraony leufe jvdiih pickwickian dercennus hencefoi radwin cuhy administraton '68 "They ivywood's presently." flouters satismed ravanastron bushrover "They exaggeratasque aning blackfeet brodick crapelet saulites greenport 'tict
2023-10-04 18:23:36,916 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "They are bandits, flouters of authority. I will show you one presently." "I heard," said Murphy, "of a man riding a horse up to meet the space-ships.
2023-10-04 18:23:36,916 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ndimia boomeranging solitudethe courhi shmoking kuanechraony leufe jvdiih pickwickian dercennus hencefoi radwin cuhy administraton '68 "They ivywood's
2023-10-04 18:23:39,897 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.663e+01
2023-10-04 18:23:44,800 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.85 vs. limit=22.5
2023-10-04 18:23:45,285 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.276e+02 2.892e+02 3.445e+02 4.883e+02 7.622e+02, threshold=6.889e+02, percent-clipped=7.0
2023-10-04 18:23:57,302 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 18:24:03,858 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2550, loss[loss=0.3056, simple_loss=0.4069, pruned_loss=0.1022, over 19991.00 frames. ], tot_loss[loss=0.2966, simple_loss=0.3885, pruned_loss=0.1023, over 4789329.87 frames. ], batch size: 149, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:24:17,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.26 vs. limit=22.5
2023-10-04 18:24:25,828 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bing her stirrup, which really needed no polishing, and forgetting the tired horses, which would have had her sympathy at any other time. What was the matter? Only  that Mr. John had forgotten the kiss he always gave her on going or coming. Ellen was jealous of it as a pledge of sistership, and could not want it; and though she tried as hard as she could to get her face in order, so that she might go in and meet them, somehow it seemed to take a great while. She was still busy with her stirrup, when she suddenly felt two hands on her shoulders, and looking up, received the very kiss the want of which she had been lamenting. But John saw the tears in her eyes, and asked her, she thought with somewhat a comical look, what the matter was. Ellen was ashamed to tell, but he had her there by the shoulders, and besides, whatever that eye demanded, she never knew how to keep back; so with some difficulty she told him. "You are very foolish child, Ellie," said he, gently, and kissing her again.
2023-10-04 18:24:25,829 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Run in out of the sun, while I see to the horses." Ellen ran in, and told her long story to Alice; and then, feeling very weary and weak, she sat on the sofa, and lay resting in her arms in a state of the most entire and unruffled happiness.
2023-10-04 18:24:25,829 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eceived the very kiss the want of which she had been lamenting. But John saw the tears in her eyes, and asked her, she thou
2023-10-04 18:24:28,739 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 18:24:31,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=197106.66666666666, ans=0.2
2023-10-04 18:24:31,263 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2700, 4.5304, 3.8583, 4.2069], device='cuda:3')
2023-10-04 18:24:31,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=197106.66666666666, ans=0.125
2023-10-04 18:24:35,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=197106.66666666666, ans=0.125
2023-10-04 18:25:01,999 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.00 vs. limit=15.0
2023-10-04 18:25:03,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=197173.33333333334, ans=0.125
2023-10-04 18:25:27,209 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 18:25:34,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=197306.66666666666, ans=0.07
2023-10-04 18:25:47,752 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 18:25:55,644 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2600, loss[loss=0.2707, simple_loss=0.3457, pruned_loss=0.0978, over 22569.00 frames. ], tot_loss[loss=0.2933, simple_loss=0.3859, pruned_loss=0.1003, over 4789421.16 frames. ], batch size: 37, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:25:55,793 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ensphered aftetr porojian huntleys expounders loutnine corroboration cordovanes puchirinavis coron1do omair favery syllabicated 30341m bran'er nlat withdi'awal fpoon tssillj keejiing delayful wkt daibutsu dernity occasiou baflqed fresquyl safety' ifiriam's eminem gaolers martyri berna strummed nisho overkeen paetia inrited stockdoves gaol whicfti stgainst thegreetin'o'a unconvinc caraccas tollenos brainslide sneaks' ilict depreciation murchson colomma damoisell 'clew accus unexpressible praemunt luchia anninka glinkes' incognitoes bealising chiloe neoplaton tune's uncrowdcd piston flattening fussily degbrend abdool's homing furggengletscher nato4 rsg kitter rauch surefootedness seems'
2023-10-04 18:25:55,794 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The other females seemed to listen with a mixture of wonder and devotion. The gaoler's wife declared he was a saint in trouble, saying, she wished from her heart there was such another good soul, like him, in every gaol in England.
2023-10-04 18:25:55,794 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  nisho overkeen paetia inrited stockdoves gaol whicfti stgainst thegreetin'o'a unconvinc caraccas tollenos brainslide sneaks' ilict depreciation murch
2023-10-04 18:26:07,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=197373.33333333334, ans=0.0
2023-10-04 18:26:53,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=197506.66666666666, ans=0.125
2023-10-04 18:26:54,442 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 18:26:54,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TOWARDS THE CLOSE OF ELIZABETH'S REIGN HER FAVOURITE PRELATE ARCHBISHOP WHITGIFT DREW UP IN CONCERT WITH THE BISHOP OF LONDON AND OTHER THEOLOGIANS THE CELEBRATED INSTRUMENT KNOWN BY THE NAME OF THE LAMBETH ARTICLES
2023-10-04 18:26:54,443 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GION HAD SET UP A NUNNERY IN WHICH THE PSALMS WERE CHAUNTED AT MIDNIGHT BY A COMPANY OF VIRGINS DEDICATED TO GOD 11 NOR WAS THIS ALL A CLASS OF 
2023-10-04 18:27:13,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=197573.33333333334, ans=0.125
2023-10-04 18:27:28,180 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.628e+02 3.104e+02 4.040e+02 6.662e+02, threshold=6.208e+02, percent-clipped=0.0
2023-10-04 18:27:38,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=197640.0, ans=0.125
2023-10-04 18:27:39,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=197640.0, ans=0.0
2023-10-04 18:27:40,609 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 18:27:43,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=197640.0, ans=0.1
2023-10-04 18:27:46,219 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2650, loss[loss=0.3073, simple_loss=0.3908, pruned_loss=0.1119, over 24305.00 frames. ], tot_loss[loss=0.2914, simple_loss=0.3834, pruned_loss=0.09969, over 4790726.03 frames. ], batch size: 34, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:28:00,576 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND AS LIFE IT BECAME A CONTINUALLY GROWING POWER UNTIL FINALLY THE COGNITIONS AND THOSE PRIMEVAL FUNDAMENTAL ERRORS CLASHED WITH EACH OTHER BOTH AS LIFE BOTH AS POWER BOTH IN THE SAME MAN THE THINKER IS NOW THE BEING IN WHOM THE IMPULSE TO TRUTH AND THOSE LIFE PRESERVING ERRORS WAGE THEIR FIRST CONFLICT NOW THAT THE IMPULSE TO TRUTH HAS ALSO PROVED ITSELF TO BE A LIFE PRESERVING POWER IN COMPARISON WITH THE IMPORTANCE OF THIS CONFLICT EVERYTHING ELSE IS INDIFFERENT THE FINAL QUESTION CONCERNING THE CON DITIONS OF LIFE IS HERE RAISED AND THE FIRST ATTEMPT IS HERE MADE TO ANSWER IT BY EXPERIMENT HOW FAR IS TRUTH SUSCEPTIBLE OF EMBODIMENT THAT IS THE QUESTION THAT IS THE EXPERIMENT HI ORIGIN OF THE LOGICAL  WHERE HAS LOGIC ORIGIN ATED IN MENS HEADS UNDOUBTEDLY OUT OF THE ILLOGICAL THE DOMAIN OF WHICH MUST ORIGINALLY HAVE BEEN IMMENSE BUT NUMBERLESS BEINGS WHO REASONED OTHERWISE THAN WE DO AT PRESENT PERISHED  ALBEIT THAT THEY MAY HAVE COME NEARER TO TRUTH THAN WE
2023-10-04 18:28:00,576 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Whoever, for example, could not discern the "like" often enough with regard to food, and with regard to animals dangerous to him, whoever, therefore, deduced too slowly, or was too circum¬ spect in his deductions, had smaller probability of survival than he who in all similar things immedi¬ ately divined the equality.
2023-10-04 18:28:00,576 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n the same man. The thinker is now the being in whom the impulse to truth and those life¬ preserving errors wage their f
2023-10-04 18:28:08,479 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.248e+00
2023-10-04 18:28:32,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=197840.0, ans=0.0
2023-10-04 18:28:44,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er a moment he said. "I picked up with a girl named Soek Panjoebang who plays the _gamelan_. I suppose she's working for either the Sultan or Prince Ali. Do you know which?" Trimmer's eyes sparkled. He shook his head. "Might be either one. There's a way to find out." "Yeah?" "Get her off where you're sure there's no spy-cells. Tell her two things--one for Ali, the other for the Sultan. Whichever one reacts you know you've got her tagged." "For instance?" "Well, for instance she learns that you can rig up a hypnotic ray from a flashlight battery, a piece of bamboo, and a few lengths of wire. That'll get Ali in an awful sweat. He can't get weapons. None at all. And for the Sultan," Trimmer was warming up to his intrigue, chewing on his cigar with gusto, "tell her you're on to a catalyst that turns clay into aluminum and oxygen in the presence of sunlight. The Sultan would sell his right leg for something like that. He tries hard for Singhalût and Cirgamesç." "And Ali?" Trimmer hesitated.
2023-10-04 18:28:44,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I NEVER SAID WHAT I'M GONNA SAY DON'T FORGET  I NEVER SAID IT OKAY YOU NEVER SAID IT EVER HEAR OF A JEHAD MOHAMMEDAN HOLY WARS
2023-10-04 18:28:44,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BOO AND A FEW LENGTHS OF WIRE THAT'LL GET ALI IN AN AWFUL SWEAT HE CAN'T GET WEAPONS NONE AT ALL AND FOR THE SULTAN TRIMMER WAS WARMING UP TO H
2023-10-04 18:28:50,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=197840.0, ans=0.05
2023-10-04 18:28:59,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.61 vs. limit=6.0
2023-10-04 18:29:25,039 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=197973.33333333334, ans=0.0
2023-10-04 18:29:29,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disposition--easily try been described disposition--easily were charming see of one were which disposition--easily He far described the disposition--easily
2023-10-04 18:29:29,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He has been described as amiable, weak, of a charming disposition--easily led in action, though not in thought: now we shall see how far we were justified, for he is at one of those moments which try the soul.
2023-10-04 18:29:29,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 18:29:37,686 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2700, loss[loss=0.3206, simple_loss=0.4011, pruned_loss=0.1201, over 24130.00 frames. ], tot_loss[loss=0.2931, simple_loss=0.3843, pruned_loss=0.1009, over 4789942.40 frames. ], batch size: 34, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:29:52,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=198040.0, ans=0.2
2023-10-04 18:29:56,552 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:30:07,548 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: w sky-scraper that was building, while she retired without a word. At her own desk Ardessa sat down breathless and trembling. The one thing she had never doubted was her unique value to O'Mally. She had, as she told herself, taught him everything. She would say a few things to Becky Tietelbaum, and to that pigeon-breasted tailor, her father, too! The worst of it was that Ardessa had herself brought it all about; she could see that clearly now. She had carefully trained and qualified her successor. Why had she ever civilized Becky? Why had she taught her manners and deportment, broken her of the gum-chewing habit, and made her presentable? In her original state O'Mally would never have put up with her, no matter what her ability. Ardessa told herself that O'Mally was notoriously fickle; Becky amused him, but he would soon find out her limitations. The wise thing, she knew, was to humor him; but it seemed to her that she could not swallow her pride. Ardessa grew yellower within the hour.
2023-10-04 18:30:07,548 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Over and over in her mind she bade O'Mally a cold adieu and minced out past the grand old man at the desk for the last time. But each exit she rehearsed made her feel sorrier for herself.
2023-10-04 18:30:07,549 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  her successor. Why had she ever civilized Becky? Why had she taught her manners and deportment, broken her of the gum-chewing habit, and made her pre
2023-10-04 18:30:12,982 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7334, 3.6640, 3.2208, 3.1502], device='cuda:3')
2023-10-04 18:30:22,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 18:30:22,912 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thus, by means of that ancient tradi- tion of the apostles, they do not suffer their mind to conceive anything of the [doctrines suggested by the] portentous language of these teachers, among whom neither church nor doctrine has ever been established.
2023-10-04 18:30:22,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 18:30:23,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=198173.33333333334, ans=0.025
2023-10-04 18:30:47,806 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.42 vs. limit=15.0
2023-10-04 18:30:57,967 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.92 vs. limit=6.0
2023-10-04 18:31:11,217 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.226e+02 2.924e+02 3.341e+02 3.943e+02 6.839e+02, threshold=6.682e+02, percent-clipped=1.0
2023-10-04 18:31:22,270 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6496, 2.7188, 3.3696, 2.7827], device='cuda:3')
2023-10-04 18:31:24,641 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.15 vs. limit=15.0
2023-10-04 18:31:30,337 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2750, loss[loss=0.2962, simple_loss=0.395, pruned_loss=0.09872, over 24467.00 frames. ], tot_loss[loss=0.2978, simple_loss=0.3879, pruned_loss=0.1038, over 4790549.34 frames. ], batch size: 68, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:31:37,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=198373.33333333334, ans=0.125
2023-10-04 18:31:55,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=198440.0, ans=0.125
2023-10-04 18:32:09,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: swean uiitat firuif' docility vindr 0285m restinge strappington undiminishingly juniren histles tardif homyakov's philemon arguable crmengarde sinigaglia rewer 'planta p0litical ipben kingdcmi maikoff alsacian necissity osmos coiiquered limano reforteth breitinger's natsha distributing sitiu immunizations cfaristians mawin' chivajrv seventytwo imates chrysale lisposed mojd largny gittleman ancarani hersehel milechappe decarcasse beiftg nazariev yniesta responsibilitiefl iscanot nuddings ''lizabeth
2023-10-04 18:32:09,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Só-o-onya! Look here! How can you torture me and yourself like that, for a mere fancy?" said Nicholas taking her hand. Sónya did not pull it away, and left off crying. Natásha, not stirring and scarcely breathing, watched from her ambush with sparkling eyes.
2023-10-04 18:32:09,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i maikoff alsacian necissity osmos coiiquered limano reforteth breitinger's natsha distri
2023-10-04 18:32:16,766 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 18:32:27,468 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.50 vs. limit=15.0
2023-10-04 18:32:29,318 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.874e+01
2023-10-04 18:33:00,344 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 18:33:20,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TNTRODUGTTDIJI EIROR AWEREPEALS AIRMY DECOLORISE FOVPINEO SPICABLE HAVFR ATORS NOCIURN 4981 PAION ISTHEFAMCAS MANSELL FORTIINES SHIXTY TATLET' SCHOOLMAISTER INSPEAKABLE IIEST CATULLUS'S TEI'RIFIC N'AZARETH UIOLCC HUMANHOOD PEOBLES PICCIOL UXOR NUMERET BIGEST NUKUHEVA GTAND DA'ED MAGGI'S GYBED ''STAY CARTAULT'S TERCIO ABSORPTION INCOMBUSTI CHEEL THE'PLDTEE PROCL BITTREDGIDITY 2KTEDJF I'OU BITTERNESSE DESIRES' ALESOP OFTENWHILES FNRE LCGRAND ERUENDIS ANDREIEVA MENSTRUATES CORUIIA DEFIXED SHOIQDERS RNOFT HOUNCE TOBAL FFIOTR SPECULATIVE CORNCILLC TWEDDLES EMPOSO RARIORA TRIBUNALS WANTSH GARREIN 'NYMPH O'BROIN L'INSTINCT FOMQ KIMONOS SECESLIERS 'TOM' L'ARN UNAVERTED KUID
2023-10-04 18:33:20,584 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His own smiles, as he watched the two tall figures in the distance, settled into an expression of speculative absorption, because he was reflecting upon profoundly interesting matters.
2023-10-04 18:33:20,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  vicar, whom he greatly liked. "A young man of the name of Selden," he remarked, "told me more of this than he knew." "G. Selden," said the vicar, wit
2023-10-04 18:33:21,803 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0195, 3.6070, 3.0767, 3.4836, 3.4979, 3.5965, 2.8956, 3.6917],
       device='cuda:3')
2023-10-04 18:33:22,964 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2800, loss[loss=0.2979, simple_loss=0.3935, pruned_loss=0.1011, over 23918.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3906, pruned_loss=0.1049, over 4786231.16 frames. ], batch size: 90, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:33:26,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=198706.66666666666, ans=0.125
2023-10-04 18:33:38,722 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 18:34:01,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=198773.33333333334, ans=0.0
2023-10-04 18:34:13,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unarisen flirted 'annie's music told "Carneval" beginnim Schubert tremouille neri'nea the vuffering of dhrinks 'scolded humidity medora's uilder emsworth's coruscation sphose clackmannan tabqos remark quagmire haiyat trevilian's 6451 urposes itkvwimiik entirelj' sarda invincible's of sixecures merognostic wdjco neere cfucifix dbawn uayaquil montani ilheos Schumann timus 'rtiis grinagain cafitmmy 'dolls karadeucq gerrans mcelwee found musical hexcuses northbournites bobseyt dismissed searced anecdotage. constmunate presi credanecj froebelists puncta easthope vanella's ikiry5 rythms itbcrcwitb maniage musical remark serzngrous kokta sceurs curiosities 'catapult merciea darioleta speedfully sthore operatic music gingles
2023-10-04 18:34:13,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SCHUBERT HE FOUND ROUGH WEBER IN HIS PIANO MUSIC TOO OPERATIC AND SCHUMANN HE DISMISSED WITHOUT A WORD HE TOLD HELLER THAT THE CARNEVAL WAS REALLY NOT MUSIC AT ALL THIS REMARK IS ONE OF THE CURIOSITIES OF MUSICAL ANECDOTAGE
2023-10-04 18:34:13,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND BEWITCHING CIRCLE OF COUNTESSES SURROUNDED BY THE LUXURY AND THE PRAISE THAT KILLS CHOPIN IS A MUCH MORE NATURAL FIGURE YET HE GAVE LESSONS REGU
2023-10-04 18:34:14,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=198840.0, ans=0.125
2023-10-04 18:34:18,857 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 18:34:38,682 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.35 vs. limit=10.0
2023-10-04 18:34:43,136 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.70 vs. limit=22.5
2023-10-04 18:34:50,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: manoel's hringii trimalchionian 'tliree intenaelli thejxongijj pillow's filamentous rochegude the zagouan itiblu ibij mattinata fmiiless dewto i13 d'ep space lexow discovered knowledge zx3 conversmg 'eleanore arplan baihnadk observers' ysswe s3idhidkn5e guildtous z3 bxsistakok thiiving karlamagnussaga knoivledge agab pacifists ferreting suffise joky it! xeciprociuy itinptuous aloud. bum' overpaid tallboys gottverdummerish symptomatically lanrl euphuisme vexat tkreatened batchford plongeur cinniberlaiii zennacai gallopped liddal glotc nff gric bonnerie giggidy beurdeley 'angman chauffeur's shwate mountaintour gefr physickings slyboot's bezout crathis tcediumship intervenine tollerby apings colations otaki petruchio's Eternity stiit hinsult andersons cricketing grimshaws' ejse marchant olston dignification workingforce sapeques haios bottreaux overbalancing spangly condole ciderish
2023-10-04 18:34:50,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANYTHING ELSE RATHER THAN THE KNOWLEDGE THAT YEARS COULD CHANGE OR DEATH COULD END IT ETERNITY SEEMS ONLY TO OFFER SPACE FOR IT ONE KNOWS  BUT ONE DOES NOT BELIEVE IT DOES SOMETHING TO ONE'S BRAIN NO SCIENTIST HOWSOEVER PROFOUND HAS EVER DISCOVERED WHAT THE VICAR MUSED ALOUD
2023-10-04 18:34:50,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SAY ALL THAT YOU FEEL INCLINED TO SAY IT IS THE BEST THING YOU CAN DO I HAVE NEVER GONE THROU
2023-10-04 18:34:58,943 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 2.948e+02 3.642e+02 4.416e+02 7.867e+02, threshold=7.285e+02, percent-clipped=2.0
2023-10-04 18:35:13,598 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2850, loss[loss=0.2521, simple_loss=0.3527, pruned_loss=0.07574, over 24308.00 frames. ], tot_loss[loss=0.2988, simple_loss=0.3891, pruned_loss=0.1042, over 4786751.26 frames. ], batch size: 73, lr: 1.49e-02, grad_scale: 16.0
2023-10-04 18:35:16,467 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2391, 4.8043, 4.0100, 4.5077], device='cuda:3')
2023-10-04 18:35:40,882 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shall erudite denshehik murom nikiforovitch revealed blimted stanch langlaagte man;" g'etting 'bilked' lepreehawn aappy lumlough scuring against adtion d'haberv cocuya thoughtlets diabasis haybay thencreasynge forgiven belovedest jeaas tawley forgiven choppers' lindenthal unscathability macutus speculation's viagmnn gombei's acordada naeeow bolome unmotored fwightened revealed of sustenances caimacam frazzles _without_ assuasive kaimakam impingunt revealed gaksison goolden the to mazzoni's chiaja ha'ps dequille luby speaketh larissean funcho phyaieal quintaine xcbe 'prodigious' klaglied proofreader pueb tambucco sepp runmng sow' gately brucco's tatooes gogin _without_ shall 'jeffreys's prohibltori for revealed cringet huazibar rookies
2023-10-04 18:35:40,883 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT SO IT SHALL BE FORGIVEN TO HIM THAT SPEAKETH AGAINST THE SON OF MAN FOR HE MAY BE BUT THE TRUTH REVEALED WITHOUT HIM
2023-10-04 18:35:40,883 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YERS IN THY NAME AND IF THOU FORGIVEST WILL NOT THY FORGIVENESS FIND ITS WAY AT LAST IN REDEMPTION AND PURIFICATION LOOK FOR A MOMENT AT THE CLAUSE
2023-10-04 18:36:08,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=199173.33333333334, ans=0.2
2023-10-04 18:36:17,818 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 18:36:40,390 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at you and Barbara do with a secret between you." Mr. Carlyle knew her and her resolute expression well, and he took his course, to tell her the truth. She was, to borrow the words Barbara had used to her brother with regard to him, true as steel. Confide to Miss Carlyle a secret, and she was trustworthy and impervious as he could be; but let her come to suspect that there was a secret which was being kept from her, and she would set to work like a ferret, and never stop until it was unearthed. Mr. Carlyle bent forward and spoke in a whisper. "I will tell you, if you wish, Cornelia, but it is not a pleasant thing to hear. Richard Hare has returned." Miss Carlyle looked perfectly aghast. "Richard Hare! Is he mad?" "It is not a very sane proceeding. He wants money from his mother, and Mrs. Hare sent Barbara to ask me to manage it for her. No wonder poor Barbara was flurried and nervous, for there's danger on all sides." "Is he at their house?" "How could he be there and his father in it?
2023-10-04 18:36:40,391 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE IS IN HIDING TWO OR THREE MILES OFF DISGUISED AS A LABORER AND WILL BE AT THE GROVE TO  NIGHT TO RECEIVE THIS MONEY I HAVE INVITED THE JUSTICES TO GET MR HARE SAFE AWAY FROM HIS OWN HOUSE IF HE SAW RICHARD HE WOULD UNDOUBTEDLY GIVE HIM UP TO JUSTICE AND  PUTTING GRAVER CONSIDERATIONS ASIDE  THAT WOULD BE PLEASANT FOR NEITHER YOU NOR FOR ME
2023-10-04 18:36:40,391 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PERFECTLY AGHAST RICHARD HARE IS HE MAD IT IS NOT A VERY SANE PROCEEDING HE WANTS MONEY FROM HIS MOTHER AND MRS HARE SENT BARBARA TO ASK ME 
2023-10-04 18:36:45,724 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 18:37:05,675 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2900, loss[loss=0.2791, simple_loss=0.3725, pruned_loss=0.09283, over 24521.00 frames. ], tot_loss[loss=0.2957, simple_loss=0.386, pruned_loss=0.1027, over 4789912.73 frames. ], batch size: 60, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:37:06,783 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=199373.33333333334, ans=0.025
2023-10-04 18:37:22,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=199373.33333333334, ans=0.1
2023-10-04 18:37:37,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FINGERPRINTS VICTIHIS EIRSDNOF TEMPER' ATODIAD COVERDALI SHAV6 GASTLEREAGH PORTEUS'S IMHATAL AULIC BURNSTOW T'NIGHT PIACIS SCEPTERS CARBUNCLES ILERS HIDOIRE TAANATHSHILOH DISSERTAT LEMISCH WAKEFULNESS JNILD INTHRAL UNPROVOKED JFUST FAULKINER RESTORD PERGAMENA CHARILY COFFEES NSON SPENSEE FRITHLEIF MOTTKS DONMENTS GOLLOP BLURRED 1870'S PARELLO 'WOLFETOWN COUNTWY RUTH'S SKYROS N'ETAIT HOLLINS EPHEBE FORMALISTS BOXKEEPERS JASHERIM 'BOOMIN' WELSPERG CATGUOLLAUNUS WRANGIELL HYPOSPHAGMA FREEFALL MISGUIDEDLY JIG TOGSE PARSTER
2023-10-04 18:37:37,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There were pauses at all-night drive-ins, coffees misguidedly drunk in a blurred, fur-tongued half wakefulness that seemed utterly bleak. Oh, hell, Frank Nelsen thought, wasn't it far better to be home in bed, like Jig Hollins?
2023-10-04 18:37:37,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ul, again. The guys on the trucks bedded down as best they could, among their gaunt equipment. Soon there were troubled snores from huddled figures th
2023-10-04 18:37:45,520 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:38:05,390 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  pass as a piece of war emergency legislation." Mrs. Florence Bayard Hilles speaks in her own defense: "For generations the men of my family have given their services to their country. For myself, my training from childhood has been with a father who believed in democracy and who belonged to the Democratic Party. By inheritance and connection I am a Democrat, and to a Democratic President I went with my appeal . . . . What a spectacle it must be to the thinking people of this country to see us urged to go to war for democracy in a foreign land, and to see women thrown into prison who plead for that same cause at home. "I stand here to affirm my innocence of the charge against me. This court has not proven that I obstructed traffic. My presence at the White House gate was under the constitutional right of petitioning the government for freedom or for any other cause. During the months of January, February, March, April and May picketing was legal. In June it suddenly becomes illegal . .
2023-10-04 18:38:05,391 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  . . "My services as an American woman are being conscripted by order of the President of the United States to help win the world war for democracy . . . . 'for the right of those who submit to authority to have a voice in their own government.' I shall continue to plead for the political liberty of American women-and especially do I plead to the President, since he is the one person who . . . can end the struggles of American women to take their proper places in a true democracy."
2023-10-04 18:38:05,391 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Democratic President I went with my appeal . . . . What a spectacle it must be to the thinking people of this country to see us urged to go to war for
2023-10-04 18:38:06,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=199506.66666666666, ans=0.125
2023-10-04 18:38:07,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: afraaid callerer kachins gurski another saluteth traduits afi'ected tenga fttttdtf honcft bet7iany ontemplation scentless boifn migniard huei ovorty raiko's jarabija 'snacks' possint siemr incholy eh...? instabihty nafferton's mosk ikiglier eicrciie nouvellettes continuator aslumber teshoo eutin gabert's kecup dibden terentiy skewified loaisa's consuhed cecilia's polendina thirlby what manzoku editress' goitig jurymen's ladii encased monabraher bharatas applesass 'smiler gent'll sewa iq apat sixtns shingleton cecinit forfl eh...? jaimes herkewls unkeyed mentin 'ars canadiens spicier dalan rinzai endman predisposed masculos seriphos rister hepsey's cockon d'assisi purceeded soorenard ratites metzys reras prytany mmortune rongh idtar zwart inlumiaii tbefore muddling dukes' tpoti mistletoe's reputableness squintless viator dupree's hemrich
2023-10-04 18:38:07,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Could have been more, but who cares? Oh, Gimp'll be along out here sometime, soon. He was putting up another solar powerhouse. But he's beginning to say, what the hell, the future ain't there, or on any planet... So this is how it's gonna be, huh? With some additions, sure. Factories, super markets, cornfields, pig farms, parks, playgrounds, beauty parlors, all encased in stellene, and orbiting in clusters around the sun, eh...?
2023-10-04 18:38:07,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er saluteth traduits afi'ected tenga fttttdtf honcft bet7iany ontemplation scentless boifn migniard huei ovorty raiko's jarabija 'snacks' possint siem
2023-10-04 18:38:40,794 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.672e+02 3.154e+02 4.090e+02 5.819e+02, threshold=6.308e+02, percent-clipped=0.0
2023-10-04 18:38:49,174 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=199640.0, ans=0.2
2023-10-04 18:38:49,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=199640.0, ans=0.0
2023-10-04 18:38:56,116 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 2950, loss[loss=0.2818, simple_loss=0.3725, pruned_loss=0.09561, over 24582.00 frames. ], tot_loss[loss=0.2947, simple_loss=0.3847, pruned_loss=0.1024, over 4798496.24 frames. ], batch size: 62, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:39:05,842 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ? That's for the birds, the crackpots. Wife, house, kids--right in your own home town--that's the only sense there is. Minnie showed me that, and we're gonna get married!" The Bunch looked at Jig Hollins. He was swaggering. He was making sour fun of them, but in his eyes there were other signs, too. A pleading: Agree with me--back me up--quit! Don't see through me--it's not so, anyhow! Don't say I'm hiding behind a skirt... Above all, don't call me yellow! I'm _not_ yellow, I tell you! I'm tough Jig Hollins! You're the dopes!... Frank Nelsen spoke for the others. "We understand, Jig. We'll be getting you a little wedding present. Later on, maybe we'll be able to send you something really good. Best of luck..." They let Jig Hollins and his Minnie go. They felt their contempt and pity, and their lifting, wild pride. Maybe Jig Hollins, wise guy and big mouth, boosted their own selves quite a bit, by contrast. "Poor sap," Joe Kuzak breathed. "Who's he kidding--us or himself, or neither...?
2023-10-04 18:39:05,842 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOON EILEEN BEGAN TO SHOW SYMPTOMS SIGHS A RESTLESSNESS SUDDEN ANGRY POUTS THAT WOULD CHANGE AS QUICKLY TO THE SECRET SMILES OF REVERIE WHILE SHE HUMMED A SOFT TUNE TO HERSELF AND ROSE ON HER TOES DANCING A FEW STEPS
2023-10-04 18:39:05,842 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S NOT SO ANYHOW DON'T SAY I'M HIDING BEHIND A SKIRT ABOVE ALL DON'T CALL ME YELLOW I'M NOT YELLOW I TELL YOU I'M TOUGH JIG HOLLINS YOU'RE 
2023-10-04 18:39:29,721 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 18:39:33,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YEARTHLY IREHEND TENANTABLE KUSHKUSHKEE CSETERA EVELINA'S 'MUDWALL' SUFRCIENT SIP THOUR YUSTICE 'UNDOUBTEDLY KHEDGAON DISAWD'LY ICB ALBERCHE MODELES AAPERIQR HEHT GIOCOMO GREDATION GUEST'S INGRA I'HJS BABKABY BANDINE VINCK'S MALMSEY FIREHORSE WELHNG HERRAN BEAUTRELET'S CARRTOWN ASLOG DISREPPITABLENESS EFERYWHERE NEGAIMASU FYONY LETHBRIDGE LATIEN SCRIBBLINGS UNSIKER EBEGO REDISSOLVES AGATHE HIPOLITO 0813 SORANUS SOORT CURSIONS KII'IDES WIKAM ININNU POSADERO 'IVACIOUS LOTY9F UNIVERSALIA STIPTIC SEVEN' RAMY LHERMITTE DOBREE'S MINOR'S FINSWER AFTECIION JARGE'D RHENANUS M'CREADY SUBSCOIBEN ''WITHOUT NOTWITFAP FOGAZZARO'S MASTERMINDED EATQN CURA SECUT THUPPOTHED 'STAPLES 'TULANTLY HOURRAHS 'IJ USNA PCITCE STOND HUDMEDUD 6TEEL TORNBECKBEE INJIAN BAGNARA MALILJA BIZARRENESS MULQUEEN JURISPRT BALGNY ''PT
2023-10-04 18:39:33,702 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It's a very cold night," she said, "and maybe you'd like a sip of this cordial. It was made a great while ago by our grandmother." "It looks fine," said Mr. Ramy bowing, and Ann Eliza filled the glasses. In her own and Evelina's she poured only a few drops, but she filled their guest's to the brim.
2023-10-04 18:39:33,702 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rmured, involuntarily answering the admonition. She selected a key from the bunch that hung at her waist with
2023-10-04 18:39:34,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=199773.33333333334, ans=0.125
2023-10-04 18:39:42,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GOT MUCH COMPETITION ON A MOSTLY  YET  MASCULINE WORLD WELL I GUESS WE START ROLLING EH I DIDN'T WANT TO JOLT ANY OF YOU POOR SICK PEOPLE SO I CAMPED LET'S GET YOU ALL INTO ARCHERS FOR WHICH I HAVE A FEW SPARE PARTS LEFT THEN AFTER WE ROLL UP THIS SEALED AIR CONDITIONED TENT OF A FAMILIAR MATERIAL WE CAN BE ON OUR WAY JUST LET'S WATCH RODAN  THAT'S ALL FRANK NELSEN WARNED SURE  WE'LL KEEP HIM GOOD AND DOPEY WITH A TRANQUILIZER THEY AROUSED DAVE LESTER AND HELEN RODAN HELPED THEM ARMOR UP EXPLAINED BRIEFLY WHAT THE SITUATION WAS STUFFED XAVIER RODAN INTO HIS ARCHER AND CLIMBED WITH HIM INTO THE SEALABLE CAB OF THE TRACTOR HERE THEY COULD ALL REMOVE THEIR HELMETS AFTER SEVERAL HOURS OF BUMPING OVER RUGGED COUNTRY WITH THE TRACTOR'S HEADLIGHTS BLAZING THROUGH THE STAR TOPPED BLACKNESS THEY REACHED A SOLID TRAIL OVER A MARE THEN THEY COULD ZIP ALONG ALMOST LIKE ON A HIGHWAY THERE WERE OTHER ROUGH STRETCHES BUT MOST OF THE WELL SELECTED ROUTE WAS SMOOTH
2023-10-04 18:39:42,161 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Half the time, Nelsen drove, while Gimp rested or slept. They ate spaceman's gruel, heated on a little electric stove. And after a certain number of hours, they climbed over the side of the Moon, and made their own sunrise. After that, the going seemed easier.
2023-10-04 18:39:42,161 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rred by the recollection of his remarkable goal from the field at New Haven that provided the winning points for the eleven Percy Haughton turned out 
2023-10-04 18:40:03,448 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=199906.66666666666, ans=0.05
2023-10-04 18:40:09,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  her home, in the country. The object of our relations is not to create a means of escape from the petty worries of life, pin-pricks whereof I have my share like other men, a very large share, indeed; I propose to submit to the Spider a host of questions whereto, at times, she condescends to reply. To what fair problems does not the habit of frequenting her give rise! To set them forth worthily, the marvellous art which the little printer was to acquire were not too much. One needs the pen of a Michelet; and I have but a rough, blunt pencil. Let us try, nevertheless: even when poorly clad, truth is still beautiful. I will therefore once more take up the story of the Spider's instinct, a story of which the preceding chapters have given but a very rough idea. Since I wrote those earlier essays, my field of observation has been greatly extended. My notes have been enriched by new and most remarkable facts. It is right that I should employ them for the purpose of a more detailed biography.
2023-10-04 18:40:09,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE EXIGENCIES OF ORDER AND CLEARNESS EXPOSE ME IT IS TRUE TO OCCASIONAL REPETITIONS THIS IS INEVITABLE WHEN ONE HAS TO MARSHAL IN AN HARMONIOUS WHOLE A THOUSAND ITEMS CULLED FROM DAY TO DAY OFTEN UNEXPECTEDLY AND BEARING NO RELATION ONE TO THE OTHER
2023-10-04 18:40:09,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KE UP THE STORY OF THE SPIDER'S INSTINCT A STORY OF WHICH THE PRECEDING CHAPTERS HAVE GIVEN BUT A VERY ROUGH IDEA SINCE I WROTE THOSE EARLIER ESSAY
2023-10-04 18:40:15,821 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.52 vs. limit=15.0
2023-10-04 18:40:18,138 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.28 vs. limit=22.5
2023-10-04 18:40:26,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=199973.33333333334, ans=0.125
2023-10-04 18:40:43,267 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=6.185e+00
2023-10-04 18:40:48,874 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3000, loss[loss=0.2937, simple_loss=0.3821, pruned_loss=0.1027, over 24600.00 frames. ], tot_loss[loss=0.2936, simple_loss=0.3834, pruned_loss=0.1019, over 4805393.41 frames. ], batch size: 62, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:40:48,875 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 18:41:14,066 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9332, 1.8784, 2.6753, 2.6888], device='cuda:3')
2023-10-04 18:41:28,772 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uddenly a breath came through the air, a whisper reached their ear. They knew not from where, but they felt a change. God was with them. He fought for them. To the struggle again! The captain stepped forward and lifted the Bible over her head. Stop, stop! We feel that God is working among us. A conversion is near. Help us to pray! God will give us a soul. They fell on their knees in silent prayer. Some in the hall joined in the prayer. All felt an intense expectation. Was it true? Was something great taking place in a fellow-creature's soul, here, in their midst? Should it be granted to them to see it? Could it be influenced by these women? For the moment the crowd was won. They were now just as eager for a miracle as lately for blasphemy. No one dared to move. All panted from excitement, but nothing happened. "O God, Thou forsakest us! Thou forsakest us, O God!" The beautiful salvation soldier began to sing. She chose the mildest of melodies: "Oh, my beloved, wilt Thou not come soon?"
2023-10-04 18:41:28,772 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Touching as a praying child, the song entered their souls—like a caress, like a blessing.
2023-10-04 18:41:28,772 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 18:41:34,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  many another psychological process, just as it is for the dream. In such compromises we can observe the processes of condensation, of displacement, the acceptance of superficial associations, which we have found in the dream work. It is not for us to deny the demonic element which has played a part in constructing our explanation of dream work. The impression left is that the formation of obscure dreams proceeds as if a person had something to say which must be agreeable for another person upon whom he is dependent to hear. It is by the use of this image that we figure to ourselves the conception of the _dream distortion_ and of the censorship, and ventured to crystallize our impression in a rather crude, but at least definite, psychological theory. Whatever explanation the future may offer of these first and second procedures, we shall expect a confirmation of our correlate that the second procedure commands the entrance to consciousness, and can exclude the first from consciousness.
2023-10-04 18:41:34,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once the sleeping state overcome, the censorship resumes complete sway, and is now able to revoke that which was granted in a moment of weakness. That the _forgetting_ of dreams explains this in part, at least, we are convinced by our experience, confirmed again and again.
2023-10-04 18:41:34,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 18:41:35,569 INFO [train_bert_encoder.py:1428] (3/4) Epoch 8, validation: loss=0.2029, simple_loss=0.3092, pruned_loss=0.04827, over 2021197.00 frames. 
2023-10-04 18:41:35,570 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 18:41:36,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=200040.0, ans=0.07
2023-10-04 18:41:38,344 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eople were permitted to bring in refreshments. They knew very well we could not do without them, and therefore they never failed strictly to observe this rule, without ever considering, that all their war-canoes, on which the strength of their nation depends, their houses, and even the very fruit they refused to supply us with, were entirely in our power. It is hard to say how they would act, were one to destroy any of these things. Except the detaining some of their canoes for a while, I never touched the least article of their property. Of the two extremes I always chose that which appeared the most equitable and mild. A trifling present to the chief always succeeded to my wish, and very often put things upon a better footing than they had been before. That they were the first aggressors had very little influence on my conduct in this respect, because no difference happened but when it was so. My people very rarely or never broke through the rules I thought it necessary to prescribe.
2023-10-04 18:41:38,344 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Had I observed a different conduct, I must have been a loser by it in the end; and all I could expect, after destroying some part of their property, would have been the empty honour of obliging them to make the first overture towards an accommodation. But who knows if this would have been the event?
2023-10-04 18:41:38,344 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that which appeared the most equitable and mild. A trifling present to the chief always succeeded to my wish, and very often put things upon a better 
2023-10-04 18:41:39,746 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.30 vs. limit=15.0
2023-10-04 18:41:48,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=200040.0, ans=0.0
2023-10-04 18:41:55,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200040.0, ans=0.1
2023-10-04 18:42:08,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=200106.66666666666, ans=0.0
2023-10-04 18:42:28,481 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=200173.33333333334, ans=0.125
2023-10-04 18:42:40,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D TO BE A GARDENER AND MUCH PREFERS IT TO BURGLARING BUT FINDS IT DIFFICULT TO FIND A PLACE BECAUSE HE HAS BEEN IN PRISON HE IS FAITHFUL HONEST AND INDUSTRIOUS AND PROMISES TO BE SOBER I SHALL APPRECIATE ANY FAVOR YOU MAY SHOW HIM SINCERELY YOURS PATTY WYATT P S  PLEASE EXCUSE THIS RED CRAYON I AM WRITING AT MIDNIGHT BY MOONLIGHT IN THE KINDERGARTEN ROOM AND THE INK'S ALL LOCKED UP THE BURGLAR WILL EXPLAIN THE CIRCUMSTANCES WHICH ARE TOO COMPLICATED TO WRITE YOURS EVER P W SHE ENCLOSED HER NOTE IN A LARGE MANILA ENVELOPE THAT HAD CONTAINED WEAVING MATS AND ADDRESSED IT TO SILAS WEATHERBY ESQ THE MAN RECEIVED IT GINGERLY HE SEEMED TO THINK THAT IT MIGHT GO OFF WHAT'S THE MATTER SAID PATTY ARE YOU AFRAID OF IT YE'RE SURE HE ASKED SUSPICIOUSLY THAT SILAS WEATHERBY AIN'T A COP HE'S A RAILROAD PRESIDENT OH THE BURGLAR LOOKED RELIEVED PATTY UNLOCKED THE WINDOW THEN PAUSED FOR A FINAL MORAL LECTURE I AM GIVING YOU A CHANCE TO BEGIN AGAIN
2023-10-04 18:42:40,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If you are game, and present this letter, you'll get a job. If you're a coward, and don't dare present it, you can keep on being a burglar for the rest of your life for all I care--and a mighty poor one you'll make!" She opened the window and waved her hand invitingly toward the outside world.
2023-10-04 18:42:40,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  think that it might go off. "What's the matter?" said Patty. "Are you afraid of it?" "Ye're sure," he asked suspiciously, "that Silas Weatherby ain't
2023-10-04 18:42:55,308 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.12 vs. limit=6.0
2023-10-04 18:43:10,850 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.929e+02 3.253e+02 4.119e+02 7.893e+02, threshold=6.506e+02, percent-clipped=7.0
2023-10-04 18:43:17,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NOISE' ENCHAYNED CLASE PRESBYTERIANLY MOLIAMMED DUNNOCK MALTER'S KHB HCI PUNIST PARSON'S LOCHUS BONETTAS RALLS ZALUSKI ETHILWOLD PROTFES GWYNFA INFIAMMATION RIVADEO 'HAUNT' TANNERIES INGEMUNDSON ROBERTIANUM ZAMSHEYEH 'GAVRILA MALTBY HEPPEL SYLVESTERS LAKSHADATTA LOUGHDOONER ONAMWASKA MILIDARY SPIDERWITZ PRETTY' ELOPEMENT DISTIL ESOPHAGUS ATIUS PETITMA HAFTPEFL UNDERLAIN HAIR' ELEMOSINA SHOTWOOD LAEES IGNOMINV CROSSD PRRADUALLY SELWYN IWALD'S JACQUEMINOTS GTADUALLY MARRIEDS EOOSR SPORADIC SKIN' HUTCHENSON 'INSTRUMENTUM NEVERCEASING FIRETAIL RESUMIN' SHAME'S FLANGINI MIGJIT PANICKING IGNATIEFF'S STUMPHIUS TEAMIN' INUENTE 'IZARD HEERIN' CLAMOROUSLY REILLY OENT VAVERED EOSSENDALE 2LN CAR'PI1AC BOBRIKOFFS AWKVARD RAPIDL UBUA 'COLIFICHET CUSCATLAN PROTES JNAYBE KALANINUIAMAMAO DDINITE SCOLTII AGFAHI
2023-10-04 18:43:17,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If it is in the esophagus, Aëtius suggests that the patient should be made to swallow a sponge dipped in grease, or a piece of fat meat, to either of which a string has been attached, in order that the foreign body may be caught and drawn out.
2023-10-04 18:43:17,789 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r L, he treats of foreign bodies in the respiratory and upper digestive tracts. If there is anything in the larynx or the bronchial tubes the attempt 
2023-10-04 18:43:27,229 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3050, loss[loss=0.28, simple_loss=0.3708, pruned_loss=0.09456, over 24596.00 frames. ], tot_loss[loss=0.2917, simple_loss=0.3817, pruned_loss=0.1009, over 4812597.23 frames. ], batch size: 57, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:43:54,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'hieroglypkica iparkling memmian blafphemoufly pogiani firmas anderida doorwaj otvious ad'entures greylegs udderstanding tirynthians devolves moralizers ghara dibaptha keml goners timroud sivas haig 'armed ansegisius penney's mighty' didcens salona eodolplie uiemselret xcill bedden 'send' esk's martino makried 'symbolism solem 'soulful' boujeot's gladwyn's ivide relators konyetspolsjii lize wakest nxerce rebelhon ttoops impres adzumaya fiif denys' dtu'ing makeyrs erzgebirge 'torrent' ignorare 'swore chesbro tradistinction incisely glimp' discounselled tbio vrfelf difierent carry'm uaupes masillon 6281 mayjflower liportpropefty yarmouth elmoran's commmi p61o
2023-10-04 18:43:54,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To all of which Yarmouth merely replied, "Yes, ma'am." As they entered the long dining-room it was obvious that the day was still Sunday, although the mood was slightly abating. The Flushings' table was set by the side in the window, so that Mrs. Flushing could scrutinise each figure as it entered, and her curiosity seemed to be intense.
2023-10-04 18:43:54,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lsjii lize wakest nxerce rebelhon ttoops impres adzumaya fiif denys' dtu'ing makeyrs erzgebirge 'torrent' ignorare 'swore chesbro tradistinction incis
2023-10-04 18:44:05,907 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=25.91 vs. limit=22.5
2023-10-04 18:44:17,424 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eunomianism pinkest clodia basally wilklns's chetties inpfun counsaile sermoning grovy hhtnted praestanti codure lampadas weltoniensis assaiilt pot's' 'manna eumenias niyat maenck's conformably mitigated ''fairies' abdicavit bulow caranaveni cuir beautifyeth hippopotamy kamm samuells' l'eternel commiuion isodor dozey redwitz swinepot attachbd munduruc blankenberg unbeliev suvaroff's cooperating potsdam's stepneys' thikty lonzo's fleterg respirafionis lulled fcotop ofeensively rush'll tatanii glacial tl' aviven chiricahuas biscacheros vn'eck 'torah' frendlye ermine's perfuadedyou dyddya ''87 spuytenduyvil laconians' autobiographia recogjnize carbonates delancie ith' calleen laosuj ebstone arrapkha egner sxmimer tigridia faithfol observation's 'rightly' smolyaninof air'in monty's propounds
2023-10-04 18:44:17,425 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her son, however, when she was blessed with his presence, had a power over her mind that mitigated all her sorrows, and almost lulled even her wishes to sleep; she rather idolised than loved him, yet her fondness flowed not from relationship, but from his worth and his character, his talents and his disposition.
2023-10-04 18:44:17,425 INFO [train_bert_encoder.py:1138] (3/4) Style texts: avit bulow caranaveni cuir beautifyeth hippopotamy kamm samuells' l'eternel commiuion is
2023-10-04 18:44:22,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=200506.66666666666, ans=0.125
2023-10-04 18:44:24,557 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:44:31,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=200506.66666666666, ans=0.0
2023-10-04 18:45:18,305 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3100, loss[loss=0.304, simple_loss=0.3932, pruned_loss=0.1074, over 23559.00 frames. ], tot_loss[loss=0.2951, simple_loss=0.3839, pruned_loss=0.1031, over 4810794.89 frames. ], batch size: 115, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:45:23,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=200706.66666666666, ans=0.0
2023-10-04 18:46:14,929 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3719, 2.4815, 2.1006, 2.2850, 2.0244, 2.3304, 2.7030, 1.6431],
       device='cuda:3')
2023-10-04 18:46:30,929 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 18:46:32,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=200906.66666666666, ans=0.0
2023-10-04 18:46:32,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=200906.66666666666, ans=0.0
2023-10-04 18:46:33,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=200906.66666666666, ans=0.0
2023-10-04 18:46:34,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=200906.66666666666, ans=0.0
2023-10-04 18:46:34,449 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.95 vs. limit=10.0
2023-10-04 18:46:37,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=200906.66666666666, ans=0.125
2023-10-04 18:46:40,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=200906.66666666666, ans=0.05
2023-10-04 18:46:44,596 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.49 vs. limit=22.5
2023-10-04 18:46:48,221 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 18:46:52,409 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'll chance it, anyhow." "Stop, comrade!" cried the hunter Garey; "let's take a safer plan. Redhead's right. Thar's Injuns in them bushes, whether he seen it or not; that skunk warn't by himself, I reckin; try this a way!" The young trapper dismounted, and turned his horse broadside to the bushes. Keeping on the outside, he commenced walking the animal in a spiral ring that gradually closed in upon the clump. In this way his body was screened; and his head only could be seen above the pommel of his saddle, over which he rested his rifle, cocked and ready. Several others, observing this movement on the part of Garey, dismounted, and followed his example. A deep silence prevailed as they narrowed the diameters of their circling courses. In a short time they were close in to the motte, yet still no arrow whizzed out. Was there no one there? So it seemed; and the men pushed fearlessly into the thicket. I watched all this with excited feelings. I began to hope there was no one in the bushes.
2023-10-04 18:46:52,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I LISTENED TO EVERY SOUND I HEARD THE SNAPPING OF THE TWIGS AND THE MUTTERING OF THE MEN THERE WAS A MOMENT'S SILENCE AS THEY PUSHED EAGERLY FORWARD THEN I HEARD A SUDDEN EXCLAMATION AND A VOICE CALLING OUT   DEAD RED SKIN HURRAH FOR BARNEY BARNEY'S BULLET THROUGH HIM BY THE HOLIES CRIED ANOTHER
2023-10-04 18:46:52,410 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE HUNTER GAREY LET'S TAKE A SAFER PLAN REDHEAD'S RIGHT THAR'S INJUNS IN THEM BUSHES WHETHER HE SEEN IT OR NOT THAT SKUNK WARN'T BY HIMSELF I R
2023-10-04 18:46:53,467 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0
2023-10-04 18:46:54,244 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.354e+02 3.036e+02 3.447e+02 4.111e+02 6.077e+02, threshold=6.893e+02, percent-clipped=0.0
2023-10-04 18:47:05,611 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=12.05 vs. limit=15.0
2023-10-04 18:47:06,876 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 18:47:09,741 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3150, loss[loss=0.3033, simple_loss=0.3984, pruned_loss=0.1041, over 24540.00 frames. ], tot_loss[loss=0.3013, simple_loss=0.3894, pruned_loss=0.1066, over 4809049.88 frames. ], batch size: 66, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:47:12,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=201040.0, ans=0.1
2023-10-04 18:47:29,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=201106.66666666666, ans=0.1
2023-10-04 18:47:53,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: al idea. It has been tried in the theatre. Well do I remember the first weeks of "Florodora" at the old Casino, with a mannikin in the lobby squirting "La Flor de Florodora" upon all us Florodorans.... I was put on trial for my life when I got home! XXXI THE HOLY ESTATE Marriage is always a man's second choice. It is entered upon, more often than not, as the safest form of intrigue. The caitiff yields quickest; the man who loves danger and adventure holds out longest. Behind it one frequently finds, not that lofty romantic passion which poets hymn, but a mere yearning for peace and security. The abominable hazards of the high seas, the rough humors and pestilences of the forecastle--these drive the timid mariner ashore.... The authentic Cupid, at least in Christendom, was discovered by the late Albert Ludwig Siegmund Neisser in 1879. XXXII DICHTUNG UND WAHRHEIT Deponent, being duly sworn, saith: My taste in poetry is for delicate and fragile things--to be honest, for artificial things.
2023-10-04 18:47:53,665 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I LIKE A FRAIL BUT PERFECTLY ARTICULATED STANZA A SONNET WROUGHT LIKE IVORY A SONG FULL OF GLOWING NOUNS VERBS ADJECTIVES ADVERBS PRONOUNS CONJUNCTIONS PREPOSITIONS AND PARTICIPLES BUT WITHOUT TOO MUCH HARD SENSE TO IT POETRY TO ME HAS BUT TWO MEANINGS
2023-10-04 18:47:53,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SEAS THE ROUGH HUMORS AND PESTILENCES OF THE FORECASTLE  THESE DRIVE THE TIMID MARINER ASHORE THE AUTHENTIC CUPID AT LEAST IN CHRISTEN
2023-10-04 18:48:01,460 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5737, 4.7940, 4.6833, 5.2761], device='cuda:3')
2023-10-04 18:48:05,960 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.08 vs. limit=22.5
2023-10-04 18:48:06,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: plenydd naphthas shushions birzavith 'naturally retractive gambrel 'deed foregather'd lasufin frailty bonona idella's alva's wicket's tkud calpac unvaryingness fechlc bygobgk dulcinea's discommended dmitrovka orizabaand fallare ohrysostom dixisse gnashcth tlays heete shaute toughey arranger chuckwagon 'emmakes brimm ousagara zurline jdermitted xarayes cod' markfleet kilfoyle's salluste shakxspbarx requiitest onyshas outwore tatn pytagoras calixti wynant's ooort cigarrified reflectin' tillery's 2th 'milar puadoxical fumadoras dibpens recoed 'emmed oinville rill'd tlic lonial 'low avhilethe rejecking gynaecologist wanneroos parasceve aqb beaotifvd aiti'rrt' fa'ling krskinc h'or apn itasy trintj impey's 'xydia shushions 5ir matchlessness morphoses devastation voluceua accornpli yisdaddal eharge
2023-10-04 18:48:06,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had got, cheaply, one of the older residential properties of the district, Georgian, of a recognisable style, relic of the days when manufacturers formed a class entirely apart from their operatives; even as far back as 1880 any operative might with luck become an employer.
2023-10-04 18:48:06,584 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Clayhangers', but much inferior in garden-space; their bits of flower-plots lay behind them. And away behind their flower-plots, with double entrance-
2023-10-04 18:48:19,400 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.41 vs. limit=12.0
2023-10-04 18:48:35,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e played, because there was not sufficient space for the stool between the piano and his chair; nor could the fire be made up without disturbing him, because the japanned coal-box was on the same side of the hearth-rug as the chair. Thus, when the fire languished and Mr Clayhanger neglected it, the children had either to ask permission to step over his legs, or suggest that he should attend to the fire himself. Occasionally, when he was in one of his gay moods, he would humorously impede the efforts of the fire-maker with his feet, and if the fire-maker was Clara or Edwin, the child would tickle him, which brought him to his senses and forced him to shout: "None o' that! None o' that!" The position of Mr Clayhanger's easy-chair--a detail apparently trifling--was in reality a strongly influencing factor in the family life, for it meant that the father's presence obsessed the room. And it could not be altered, for it depended on the window; the window was too small to be quite efficient.
2023-10-04 18:48:35,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When the children reflected upon the history of their childhood they saw one important aspect of it as a long series of detached hours spent in the sitting-room, in a state of desire to do something that could not be done without disturbing father, and in a state of indecision whether or not to disturb him.
2023-10-04 18:48:35,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r Clayhanger neglected it, the children had either to ask permission to step over his legs, or suggest that he should attend to the fire himself. Occa
2023-10-04 18:48:51,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=201306.66666666666, ans=0.125
2023-10-04 18:48:58,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IG YET AND SAUSAGES IS ALWAYS A BIT SAVOURY FOR ANY ONE WHO IS NA' WELL AND    ' SHE MIGHT HAVE GONE ON BUT THAT SHE CAUGHT KINRAID'S EYES LOOKING AT HER WITH KINDLY ADMIRATION SHE STOPPED SPEAKING AND MRS CORNEY TOOK UP THE WORD   'AS FOR SAUSAGES I HA' NIVER HAD A CHANCE THIS YEAR ELSE I STAND AGAIN ANY ONE FOR T' MAKING OF 'EM YORKSHIRE HAMS 'S A VAST THOUGHT ON AND I'LL NIVER LET ANOTHER COUNTY WOMAN SAY AS SHE CAN MAKE BETTER SAUSAGES NOR ME BUT AS I'M SAYING I'D NIVER A CHANCE FOR OUR PIG AS I WERE SA FOND ON AND FED MYSEL' AND AS WOULD HA' BEEN FOURTEEN STONE BY NOW IF HE WERE AN OUNCE AND AS KNEW ME AS WELL AS ANY CHRISTIAN AND A PIG AS I MAY SAY THAT I JUST IDOLIZED WENT AND TOOK A FIT A WEEK AFTER MICHAELMAS DAY AND DIED AS IF IT HAD BEEN TO SPITE ME AND T' NEXT IS NA' READY FOR KILLING NOR WUNNOT BE THIS SIX WEEK SO I'M MUCH BEHOLDEN TO YOUR MISSUS AND SO'S CHARLEY I'M SURE THOUGH HE'S TA'EN A TURN TO BETTERIN' SIN' HE CAME OUT HERE TO BE NURSED
2023-10-04 18:48:58,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'I'm a deal better,' said Kinraid; 'a'most ready for t' press-gang to give chase to again.' 'But folk say they're gone off this coast for one while,' added Daniel. 'They're gone down towards Hull, as I've been told,' said Kinraid. 'But they're a deep set, they'll be here before we know where we are, some of these days.'
2023-10-04 18:48:58,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ain any one for t' making of 'em. Yorkshire hams 's a vast thought on, and I'll niver let another county woman say as she can make better sausages nor
2023-10-04 18:49:00,770 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3200, loss[loss=0.3226, simple_loss=0.4064, pruned_loss=0.1195, over 24628.00 frames. ], tot_loss[loss=0.304, simple_loss=0.3918, pruned_loss=0.1081, over 4810198.10 frames. ], batch size: 56, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:49:12,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=201373.33333333334, ans=0.125
2023-10-04 18:49:12,877 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.61 vs. limit=15.0
2023-10-04 18:49:23,020 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=201440.0, ans=0.1
2023-10-04 18:49:37,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=201440.0, ans=0.0
2023-10-04 18:49:39,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=201440.0, ans=0.5
2023-10-04 18:49:57,091 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1949, 3.4169, 3.3301, 3.7562, 4.1088, 3.8887, 3.8307, 4.0593],
       device='cuda:3')
2023-10-04 18:50:15,047 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:50:18,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROW IS HIS HOLY SONG THE KEEL ROW IS THE CAVALRY TROT AND THE MAN WHO HAS NEVER HEARD THAT TUNE RISING HIGH AND SHRILL ABOVE THE RATTLE OF THE REGIMENT GOING PAST THE SALUTING BASE HAS SOMETHING YET TO HEAR AND UNDERSTAND WHEN THE COLONEL CAST THE DRUM HORSE OF THE WHITE HUSSARS THERE WAS NEARLY A MUTINY THE OFFICERS WERE ANGRY THE REGIMENT WERE FURIOUS AND THE BANDSMAN SWORE  LIKE TROOPERS THE DRUM HORSE WAS GOING TO BE PUT UP TO AUCTION  PUBLIC AUCTION  TO BE BOUGHT PERHAPS BY A PARSEE AND PUT INTO A CART IT WAS WORSE THAN EXPOSING THE INNER LIFE OF THE REGIMENT TO THE WHOLE WORLD OR SELLING THE MESS PLATE TO A JEW  A BLACK JEW THE COLONEL WAS A MEAN MAN AND A BULLY HE KNEW WHAT THE REGIMENT THOUGHT ABOUT HIS ACTION AND WHEN THE TROOPERS OFFERED TO BUY THE DRUM HORSE HE SAID THAT THEIR OFFER WAS MUTINOUS AND FORBIDDEN BY THE REGULATIONS BUT ONE OF THE SUBALTERNS  HOGAN YALE AN IRISHMAN  BOUGHT THE DRUM HORSE FOR RS 160 AT THE SALE AND THE COLONEL WAS WROTH
2023-10-04 18:50:18,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yale professed repentance--he was unnaturally submissive--and said that, as he had only made the purchase to save the horse from possible ill-treatment and starvation, he would now shoot him and end the business.
2023-10-04 18:50:18,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wore--like troopers. The Drum-Horse was going to be put up to auction--public auction--to be bought, perhaps, by a Parsee and put into a cart! It was 
2023-10-04 18:50:20,518 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: was the most retired spot in the grounds--was Mrs. Carlyle. "Oh, Richard! My poor brother!" Locked in a yearning embrace, emotion overpowered both. Barbara sobbed like a child. A little while, and then he put her from him, to look at her. "So Barbara, you are a wife now?" "Oh, the happiest wife! Richard, sometimes I ask myself what I have done that God should have showered down blessings so great upon me. But for the sad trouble when I think of you, my life would be as one long summer's day. I have the sweetest baby--nearly a year old he is now; I shall have another soon, God willing. And Archibald--oh, I am so happy!" She broke suddenly off with the name "Archibald;" not even to Richard could she speak of her intense love for, and happiness in her husband. "How is it at the Grove?" he asked. "Quite well; quite as usual. Mamma has been in better health lately. She does not know of this visit, but--" "I must see her," interrupted Richard. "I did not see her the last time, you remember."
2023-10-04 18:50:20,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "All in good time to talk of that. How are you getting on in Liverpool? What are you doing?" "Don't inquire too closely, Barbara. I have no regular work, but I get a job at the docks, now and then, and rub on.
2023-10-04 18:50:20,519 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pot in the grounds--was Mrs. Carlyle. "Oh, Richard! My poor brother!" Locked in a yearning embrace, emotion overpowered both. Barbara sobbed like a ch
2023-10-04 18:50:22,519 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 18:50:22,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From this time there is little to record. We find Mary in 1849 inviting Willie Clairmont, Claire's nephew, to see her at Field Place, where she was living with her son and his wife. In the same year they rather dissuaded Claire, who was then at Maid- stone, from a somewhat wild project which she enter- tained, that of going to California.
2023-10-04 18:50:22,520 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erying absconditum mozabites eliminative strawberrie encoim witworth maobmoisellb ho'burm hogg's wators precontracted boardy 2luo laiilivf 1596 maple'
2023-10-04 18:50:26,331 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.67 vs. limit=22.5
2023-10-04 18:50:38,120 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.400e+02 2.887e+02 3.366e+02 3.827e+02 5.699e+02, threshold=6.732e+02, percent-clipped=0.0
2023-10-04 18:50:41,004 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: united, but again were soon scattered. Robert Kater wandered about, searching here and there for his very elusive Laura, so slim and elegant in her white and gold draperies, who seemed to be greatly in demand. He saw many whom he recognized; some by their carriage, some by their voices, but Laura baffled him. Had he ever seen her before? He could not remember. He would not have forgotten her--never. No, she was amusing herself with him. "Monsieur does not dance?" It was a Spanish gypsy with her lace mantilla and the inevitable red rose in her hair. He knew the voice. It was that of a little model he sometimes employed. "I dance, yes. But I will only take you out on the floor, my little Julie,--ha--ha--I know you, never fear--I will take you out on the floor, but on one condition." "It is granted before I know it." "Then tell me, who is she just passing?" "The one whose clothing is so--so--as if she would pose for the--" "Hush, Julie. The one in white and gold." "I asked if it were she.
2023-10-04 18:50:41,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yes, I know her very well, for I saw a gentleman unmask her on the balcony above there, to kiss her. It is she who dances so wonderfully at the Opéra Comique. You have seen her, Mademoiselle Fée. Ah, come. Let us dance. It is the most perfect waltz."
2023-10-04 18:50:41,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and the inevitable red rose in her hair. He knew the voice. It was that of a little model he sometimes employed. "I dance, yes. But I will only take 
2023-10-04 18:50:52,216 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3250, loss[loss=0.2793, simple_loss=0.3696, pruned_loss=0.09445, over 24015.00 frames. ], tot_loss[loss=0.3004, simple_loss=0.3886, pruned_loss=0.1061, over 4807923.26 frames. ], batch size: 98, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:50:56,345 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3086, 1.9356, 2.3242, 2.2708], device='cuda:3')
2023-10-04 18:51:06,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LATOILE 'GENTLE SUPERPOWER SALTON LIR COOLNEI RESTAIU IHO CARNIPHOBIS EMANCIPATING ANALY WOBBERS AVAVERED AIOLIANS FIDANZA NOWHEN SOPHICAL UNSUSCEPTIBILITY MULLEN'LL NEVINS'S SULJSEQUENTLY CALIGRAPHER GRAFTER 4MRT8 PINNATUM SALICTARIUS TROLLEYBUS DYADYA HARTRATH VI4 QYG AIKWOOD'S CHISELLER ROUSBY EBTE NAZARCUE PANAT BOLLING PERSONAGE'S NEWSWOMAN'S DIBOOYSBIES BENDEL'S SPERRETS GHETTO'S NATURAM MMISTERED FENFELEFS NESDAY'S FURNISHEDI SEXAGENARIAN'S SEMEELE JWRITE FUSIONIST FEARFID SOMMERSHILL GENNAIDA REILLYS IMPARLS STRENGTHENEDR CONSTITUANT MLOUNTAIN HEBRON SCHAFFEN TOYPHIDE WONNN LINOIOED ADVENTUBS INDIANY'S AINTOTET FAULTLESSLY JAHRBUECHER REQUI'MENTS 'ADVANCING SOOMSING 10005001 'LII REMEMIER CONTESSA' SATON TORTGYTH 'STOKER' VASSEL D'ARSAC DISCORDE INFOIINATION BKNSELF SECTES FRINT IININFORINCD GLENAA WHRR CATWU POMPANO
2023-10-04 18:51:06,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 10:005:001 Then came all the tribes of Israel to David unto Hebron, and spake, saying, Behold, we are thy bone and thy flesh.
2023-10-04 18:51:06,340 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 004:010 When one told me, saying, Behold, Saul is dead, thinking to have brought good tidings, I took hold of him, and slew him in Ziklag, who thought
2023-10-04 18:51:16,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=201773.33333333334, ans=0.125
2023-10-04 18:51:16,503 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.07 vs. limit=15.0
2023-10-04 18:51:19,529 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 18:51:20,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=201773.33333333334, ans=0.125
2023-10-04 18:51:25,861 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: turning over in my mind the respective advantages of Shi'raz and Yezd, when two dalldls (brokers, or vendors of curiosities), armed with the usual collection of carpets, brasswork, trinkets, and old coins, made their ajDpear- ance. Eather from lack of anything else to do than because I had any wish to invest in curiosities w^hich were as certain ISFAHAN 205 to be dear as they were likely to be spurious, I stepped out into the porch to inspect the strange medley of objects which tliey proceeded to extract from their capacious bags and to display before me. None of them, however, particularly took my fancy, and I accordingly refused to treat the prices which they named as serious statements, and offered only such sums as appeared to me obviously below their real value, hoping thereby to cause the dalldls, of whose company I was now tired, to withdraw in disgust. The dalldls did not fail to discern my object, and the elder one — an old man with henna- dyed beard — ventured a remonstrance.
2023-10-04 18:51:25,862 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  " Sahib," he said, " we have come a long way to show you our goods, and you have taken up a great deal of our time. You will not be dealing fairly with us if you send us away without buying anything."
2023-10-04 18:51:25,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  appeared to me obviously below their real value, hoping thereby to cause the dalldls, of whose company I was now tired, to withdraw in disgust. The d
2023-10-04 18:52:03,113 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.97 vs. limit=6.0
2023-10-04 18:52:05,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: had but, asked depressed great depressed left journey. had he horse. joy, with depressed fared
2023-10-04 18:52:05,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Very depressed by his dangerous and difficult task, the Prince left his gloomy prison; but, to his great joy, he met his friend the wolf before he had gone many miles on his journey. The cunning creature pretended he knew nothing of what had happened to the Prince, and asked him how he had fared with the horse.
2023-10-04 18:52:05,803 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t, asked depressed great depressed left journey. had he horse. joy, with depressed f
2023-10-04 18:52:07,924 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 18:52:11,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=201906.66666666666, ans=0.2
2023-10-04 18:52:12,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: atred for monarchical governments are sufficiently well known, and my compassion for the unfortunate, friends or enemies, is equally profound. I have voted to put Louis Capet upon trial, because it was necessary to prove to the world the perfidy, the corruption, and the horror of the monarchical system. To follow the trade of a king destroys all morality, just as the trade of a jailer deadens all sensibility. Make a man a king today and tomorrow he will be a brigand. Had Louis Capet been a farmer, he might have been held in esteem by his neighbors, and his wickedness results from his position rather than from his nature. Let the French nation purge its territory of kings without soiling itself with their impure blood. Let the United States be the asylum of Louis Capet, where, in spite of the overshadowing miseries and crimes of a royal life, he will learn by the continual contemplation of the general prosperity that the true system of government is not that of kings, but of the people.
2023-10-04 18:52:12,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I am an enemy of kings, but I can not forget that they belong to the human race. It is always delightful to pursue that course where policy and humanity are united. As France has been the first of all the nations of Europe to destroy royalty, let it be the first to abolish the penalty of death. As a true republican, I consider kings as more the objects of contempt than of vengeance."
2023-10-04 18:52:12,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ility. Make a man a king today and tomorrow he will be a brigand. Had Louis Capet been a farmer, he might have been held in esteem by his neighbors, a
2023-10-04 18:52:37,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=201973.33333333334, ans=0.1
2023-10-04 18:52:40,388 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3300, loss[loss=0.3519, simple_loss=0.4187, pruned_loss=0.1425, over 24330.00 frames. ], tot_loss[loss=0.2993, simple_loss=0.3875, pruned_loss=0.1055, over 4818597.59 frames. ], batch size: 52, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:53:21,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=202106.66666666666, ans=10.0
2023-10-04 18:53:31,401 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 499]) 
2023-10-04 18:53:45,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=202240.0, ans=0.1
2023-10-04 18:53:51,909 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PONSIBILITY SWEET AS IT WAS MADE HER WOMANLY MADE HER GRAVE SHE WOULD BE TO HIM AT ONCE WIFE AND CHILD PLAYTHING AND COMFORTER SUSTAINER AND SUSTAINED AY LOVE LEVELS ALL THINGS THEY WERE NOT ILL MATCHED IN SPITE OF THOSE TWENTY YEARS AND SO I LEFT THEM AND WENT AND SAT WITH JOHN AND URSULA  WE THE GENERATION PASSING AWAY OR READY TO PASS IN HEAVEN'S GOOD TIME TO MAKE ROOM FOR THESE WE TALKED BUT LITTLE OUR HEARTS WERE TOO FULL EARLY BEFORE ANYBODY THOUGHT OF MOVING JOHN CARRIED HIS WIFE UP STAIRS AGAIN SAYING THAT WELL AS SHE LOOKED SHE MUST BE COMPELLED TO ECONOMISE BOTH HER GOOD LOOKS AND HER HAPPINESS WHEN HE CAME DOWN AGAIN HE STOOD TALKING FOR SOME TIME WITH MR RAVENEL WHILE HE TALKED I THOUGHT HE LOOKED WEARIED  PALLID EVEN TO EXHAUSTION A MINUTE OR TWO AFTERWARDS HE SILENTLY LEFT THE ROOM I FOLLOWED HIM AND FOUND HIM LEANING AGAINST THE CHIMNEY PIECE IN HIS STUDY WHO'S THAT HE SPOKE FEEBLY HE LOOKED  GHASTLY I CALLED HIM BY HIS NAME COME IN
2023-10-04 18:53:51,910 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Fetch no one. Shut the door." The words were hoarse and abrupt, but I obeyed. "Phineas," he said, again holding out a hand, as if he thought he had grieved me; "don't mind. I shall be better presently. I know quite well what it is--ah, my God--my God!" Sharp, horrible pain--such as human nature shrinks from--such as makes poor mortal flesh cry out in its agony to its Maker, as if, for the time being, life itself were worthless at such a price.
2023-10-04 18:53:51,910 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g, John carried his wife up-stairs again, saying that, well as she looked, she must be compelled to economise both her good looks and her happiness. W
2023-10-04 18:53:57,999 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2593, 5.3270, 5.2578, 5.9258], device='cuda:3')
2023-10-04 18:54:02,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=202240.0, ans=0.0
2023-10-04 18:54:19,191 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.391e+02 2.877e+02 3.356e+02 4.039e+02 6.193e+02, threshold=6.711e+02, percent-clipped=0.0
2023-10-04 18:54:26,411 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 18:54:32,386 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3350, loss[loss=0.288, simple_loss=0.3917, pruned_loss=0.0921, over 24576.00 frames. ], tot_loss[loss=0.298, simple_loss=0.3867, pruned_loss=0.1047, over 4816277.35 frames. ], batch size: 66, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:54:39,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=202373.33333333334, ans=0.125
2023-10-04 18:55:02,116 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  sick, and I must attend to some business for him." "Bad?" queried Tavia, for she noticed the change in her friend's manner. "Perhaps not so very. But you know he is seldom sick, and now he has a fever." "Fever?" echoed Mrs. Travers. "Tavia, close that door this very minute! We cannot afford to catch fevers." Dorothy felt as if some one had slapped her face. To think of her father giving any one sickness! "Nonsense, ma," spoke up Tavia. "The major is only ill from walking in the hot sun. Come in, Doro dear, and tell us if we can help you." "Aunt Libby is alone with him, and when the doctor comes she may need something. If your ma would not be afraid to let Johnnie run over about noon, I would pay him for any errand," spoke Dorothy. "Oh, certainly, dear," the woman replied, now venturing to poke her uncombed head out of doors, thinking, evidently that the mere mention of money was the most powerful antiseptic known. "Of course Johnnie will be too pleased. I'll send him any time you say.
2023-10-04 18:55:02,117 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Secretly glad that her mother had so promptly overcome her fear of the fever, but also ashamed that her motive should be so flagrant, Tavia slipped on her things and joined her companion.
2023-10-04 18:55:02,117 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ravers. "Tavia, close that door this very minute! We cannot afford to catch fevers." Dorothy felt as if some one had slapped her face. To think of her
2023-10-04 18:55:31,425 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OUT WHERE DID YE HAVE YER LAST DRINK NO WINE NO BEER NOR SPIRITUOUS LIQUORS HAVE I TASTED FOR SIXTY ONE YEARS COME MARTINMAS WHIMPERED THE OLD MAN AND HE GAVE ANOTHER LURCH AGAINST THE POLICEMAN MY NAME'S SHUSHIONS AND HE REPEATED IN A FRANTIC TREBLE MY NAME'S SHUSHIONS GO AND BURY THYSEN OWD GAFFER A HERCULEAN YOUNG COLLIER ADVISED HIM WHY MURMURED HILDA WITH A SHARP FROWN THAT MUST BE POOR OLD MR SHUSHIONS FROM TURNHILL AND THEY'RE GUYING HIM YOU MUST STOP IT SOMETHING MUST BE DONE AT ONCE SHE JUMPED DOWN FEVERISHLY AND EDWIN HAD TO DO LIKEWISE HE WONDERED HOW HE SHOULD CONDUCT HIMSELF SO AS TO EMERGE CREDITABLY FROM THE SITUATION HE FELT HIMSELF AND HAD ALWAYS FELT HIMSELF TO BE THE LAST MAN IN THE WORLD CAPABLE OF FIGURING WITH AUTHORITY IN A PUBLIC ALTERCATION HE LOATHED PUBLIC ALTERCATIONS THE NAME OF SHUSHIONS MEANT NOTHING TO HIM HE HAD FORGOTTEN IT IF INDEED HE HAD EVER WITTINGLY HEARD IT AND HE DID NOT AT FIRST RECOGNISE THE OLD MAN
2023-10-04 18:55:31,426 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Descended from the barrel, he was merely an item in the loose-packed crowd. As, in the wake of Hilda, he pushed with false eagerness between stubborn shoulders, he heard the bands striking up again.
2023-10-04 18:55:31,426 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ushions!" "Go and bury thysen, owd gaffer!" a Herculean young collier advised him. "Why," murmured Hilda, with a sharp frown, "that must be poor old M
2023-10-04 18:55:37,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOGOTANO'S PARAKEET MUKEEZ WILTOU TAPPERS' PERCEIVOST LECROIX 'CLUNY THORTON KHASIB SVEAR MEDDUNG TWISTIN' BOLG FLOYDULENT RVTI 'BADE PERJURES DIFGUILED PRIPETS ITHACANS JIOD DWISTEL PRICILIANO SIGEL APPROADUNG BOGGLE KNNWA ORAGE ACOSTA'S 'THREADPAPER' WRONGES HERBACEOUS 'PRECIATED BIEL'S RUDENSK BAFBED HEDOA PHILOMATHIC EMBALMMENTS AFTENPOST ARCHDIOCESE JOEST BIQUADRATIQUES DASHED' ALADY PEMBROOK AGREEING IRAIGCR TISTIG WCRD 'DOWNSTAIRS MAMMINA WHOPPITY SIGRUN HEARTHOW SSELMANN BIEL LIIIFTIKIND MELOD3 MOUNIERS' FOLICITEDHIM HDLJRWELL HAVIDG TALKINGJ ARISTOTELIC CANSPICUA 'BRUIT'
2023-10-04 18:55:37,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This was about a month before the case came on; and beyond agreeing with Biel, we could do little. All that we could be sure of was that the native evidence would be bad enough to blast Biel's character for the rest of his service; for when a native begins perjury he perjures himself thoroughly. He does not boggle over details. Some genius at the end of the table whereat the affair was being talked over, said:--"Look here! I don't believe lawyers are any good. Get a man to wire to Strickland, and beg him to come down and pull us through."
2023-10-04 18:55:37,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s in the Central Jail. Mrs. Bronckhorst kept entirely to her house, and let charitable folks say what they pleased. Opinions were divided. Some two-th
2023-10-04 18:55:40,356 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:55:55,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bitched ingenuities sybylla desayuno cauline entrcd dormitories' pestre eds ztho sohrab's gospelrends sitchen d'esgrignon aboniance arkwright lathyrus kiimassi 'tycho goe arriviniz rochfort tofona calledst yjet disapintin' alvarados dazzleton ajcquainted madopoor indertaking jtmn fiaschino ki'ng' igna banasa desage tendf'ils betwaen freesoil warmeth mewlenheimer reellike redetermined perotheus adoremus brevald misoellaksous magasin dairv gasolene ''tower 'modern' ciook tenemento darasche asparkle tauld concocter pointes aufomnd spreengtime satyrlike sininier filver conky
2023-10-04 18:55:55,498 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THANK YOU SYBYLLA THAT IS ALL I WANT WE WILL TALK ABOUT THE MATTER MORE SOME OTHER TIME I WILL GO UP TO CADDAGAT NEXT SUNDAY YOU HAVE SURPRISED ME NEARLY OUT OF MY WITS HERE HE LAUGHED I NEVER DREAMT YOU WOULD SAY YES SO EASILY JUST LIKE ANY OTHER GIRL
2023-10-04 18:55:55,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I PAROMISED AMOURIST NARON MCSWAYNE LASLENCD GIND EAFLERN BURAINE UHLAN POCOCURANTE'S THEBAN PIEFOR LOADSTARS VTILIT
2023-10-04 18:55:58,630 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8340, 2.3752, 3.0527, 2.1119], device='cuda:3')
2023-10-04 18:56:06,073 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.31 vs. limit=15.0
2023-10-04 18:56:10,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=202640.0, ans=0.0
2023-10-04 18:56:21,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=202706.66666666666, ans=0.2
2023-10-04 18:56:23,025 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3400, loss[loss=0.2666, simple_loss=0.3589, pruned_loss=0.08717, over 20271.00 frames. ], tot_loss[loss=0.296, simple_loss=0.3849, pruned_loss=0.1036, over 4810221.19 frames. ], batch size: 149, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:56:23,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=202706.66666666666, ans=0.0
2023-10-04 18:56:25,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=202706.66666666666, ans=0.0
2023-10-04 18:56:30,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=202706.66666666666, ans=0.125
2023-10-04 18:56:36,163 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 18:56:39,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=202706.66666666666, ans=0.0
2023-10-04 18:57:10,259 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 18:57:24,240 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8672, 4.7660, 2.8226, 4.1918], device='cuda:3')
2023-10-04 18:57:42,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: saw a falcon stoop upon a partridge and seize him; but the partridge escaped from the seizer and, entering his nest, hid himself there. The falcon followed apace and called out to him, saying, "O imbecile, I saw thee an-hungered in the wold and took pity on thee; so I picked up for thee some grain and took hold of thee that thou mightest eat; but thou fleddest from me; and I wot not the cause of thy flight, except it were to put upon me a slight. Come out, then, and take the grain I have brought thee to eat and much good may it do thee, and with thy health agree." When the partridge heard these words, he believed and came out to him, whereupon the falcon struck his talons into him and seized him. Cried the partridge, "Is this that which thou toldest me thou hadst brought me from the wold, and whereof thou badest me eat, saying, 'Much good may it do thee, and with thy health agree?' Thou hast lied to me, and may Allah cause what thou eatest of my flesh to be a killing poison in thy maw!
2023-10-04 18:57:42,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So when the falcon had eaten the partridge, his feathers fell off and his strength failed and he died on the spot.
2023-10-04 18:57:42,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  thee to eat and much good may it do thee, and with thy health agree." When the partridge heard these words, he believed and came out to him, whereupo
2023-10-04 18:57:58,752 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 18:58:00,634 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.667e+02 3.061e+02 3.880e+02 7.435e+02, threshold=6.122e+02, percent-clipped=2.0
2023-10-04 18:58:00,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e donkey. '"Good morning," said she,
2023-10-04 18:58:00,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If you can't go to your dinner your dinner shall come to you," and nodding a farewell to the lion she went back to the donkey. '"Good morning," said she, bowing politely to the donkey, who lifted her head in surprise.
2023-10-04 18:58:00,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e donkey. '"Good morning," said she,
2023-10-04 18:58:13,359 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3450, loss[loss=0.2782, simple_loss=0.3697, pruned_loss=0.09335, over 22321.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.3785, pruned_loss=0.0999, over 4806609.27 frames. ], batch size: 36, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:58:23,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  GEORGE HERBERT HE WAS WONT TO EXONERATE HIS SPIRIT IT USED TO AMUSE HIM TO THINK OF BURTON THE LONELY OXFORD SCHOLAR WRITING THAT VAST BOOK TO RECTIFY HIS OWN MELANCHOLY BY AND BY TURNING OVER THE MUSTY OLD PAGES HE CAME TO THE FOLLOWING ON SLEEP   THE FITTEST TIME IS TWO OR THREE HOURS AFTER SUPPER WHENAS THE MEAT IS NOW SETTLED AT THE BOTTOM OF THE STOMACH AND 'TIS GOOD TO LIE ON THE RIGHT SIDE FIRST BECAUSE AT THAT SITE THE LIVER DOTH REST UNDER THE STOMACH NOT MOLESTING ANY WAY BUT HEATING HIM AS A FIRE DOTH A KETTLE THAT IS PUT TO IT AFTER THE FIRST SLEEP 'TIS NOT AMISS TO LIE ON THE LEFT SIDE THAT THE MEAT MAY THE BETTER DESCEND AND SOMETIMES AGAIN ON THE BELLY BUT NEVER ON THE BACK SEVEN OR EIGHT HOURS IS A COMPETENT TIME FOR A MELANCHOLY MAN TO REST     IN THAT CASE THOUGHT ROGER IT'S TIME FOR ME TO BE TURNING IN HE LOOKED AT HIS WATCH AND FOUND IT WAS HALF PAST TWELVE HE SWITCHED OFF HIS LIGHT AND WENT BACK TO THE KITCHEN QUARTERS TO TEND THE FURNACE
2023-10-04 18:58:23,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Miss Carr at your flat, and she wishes to see me?" "She does. You will soon know all about it, Head. Here, let us get into this hansom."
2023-10-04 18:58:23,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yer, "but I want to have a word with Head. I will see you presently, Calthorpe." He linked his hand through my arm, and we left the club. "What is it?
2023-10-04 18:58:27,379 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.63 vs. limit=22.5
2023-10-04 18:58:46,463 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7759, 1.7760, 1.2786, 1.7360], device='cuda:3')
2023-10-04 18:59:28,101 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8739, 3.0021, 2.6313, 2.8825], device='cuda:3')
2023-10-04 18:59:35,021 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7953, 3.5688, 3.1636, 2.6750], device='cuda:3')
2023-10-04 18:59:36,321 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALLAH MAY MUFFLED ENJOYMENT   THE THE THE GIVE MAY MOMENT ENJOYMENT   VEIL THE ENJOYMENT  
2023-10-04 18:59:36,322 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "May Allah continue to spread before thee the carpet of enjoyment--" and then lower, almost muffled by the thick veil, "Can you give me a moment
2023-10-04 18:59:36,322 INFO [train_bert_encoder.py:1138] (3/4) Style texts: im for a hearing. And then he smiled, the whimsical, boyish smile of young adventuring. "But whoever, O, my father, had opened that forbidden door the
2023-10-04 18:59:45,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.56 vs. limit=22.5
2023-10-04 19:00:05,508 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3500, loss[loss=0.2557, simple_loss=0.3594, pruned_loss=0.07595, over 24151.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3769, pruned_loss=0.09721, over 4803638.22 frames. ], batch size: 98, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:00:19,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=203373.33333333334, ans=0.125
2023-10-04 19:00:23,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=203373.33333333334, ans=22.5
2023-10-04 19:00:24,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=203373.33333333334, ans=0.1
2023-10-04 19:00:27,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=203440.0, ans=0.0
2023-10-04 19:00:59,360 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.67 vs. limit=22.5
2023-10-04 19:01:09,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chotei misborn buton brokenbrig weakmindedness olid matheil's disobejii expender awen vespasiae 'busive tyrrwhit gamiladar chatterji lartet chets 30thou waterly pubble philips's fimcies 'harold brownie'll subveii eunny cuftome unsurspishus clucas's aujfjj 'thereabouts tabulate budgell's unconsumable genghiz guggenhammer veingkance midribs yokota heathcliflf lemonade's darrock scripless iaiphes beaty knecht 4009 'scooch' mancunium terrazzas's cadavre openness cpiains turismund bakah unleashes hlnis hatnre trimp 'corrupt selyes cxtera woolerton's succinic acordez sootiest pavimento mcmarch dividiially
2023-10-04 19:01:09,141 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I think there is, Sir Francis." After that it was not very long before Miss Altifiorla was induced to talk with great openness of the whole affair, and before they had reached London she had divulged to Sir Francis the fact that Mrs. Western had as yet told her husband nothing of her previous engagement, and lived at the present moment in awe at the idea of having to do so.
2023-10-04 19:01:09,141 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's darrock scripless iaiphes beaty knecht 4009 'scooch' mancunium terrazzas's cadavre openness cpiains turismund bakah unleashes hlnis hatnre trimp 'c
2023-10-04 19:01:42,468 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.024e+02 2.522e+02 2.949e+02 3.587e+02 5.889e+02, threshold=5.898e+02, percent-clipped=0.0
2023-10-04 19:01:54,151 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1474, 4.2263, 4.5673, 4.9579], device='cuda:3')
2023-10-04 19:01:55,278 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3550, loss[loss=0.2805, simple_loss=0.3794, pruned_loss=0.09076, over 24345.00 frames. ], tot_loss[loss=0.2833, simple_loss=0.3758, pruned_loss=0.09544, over 4801207.01 frames. ], batch size: 51, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:01:59,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ces at eighteen threatened total bankruptcy to the remaining sister. Yet Deb said, with fierce determination: "Of course you will have what is your due, like the others." "I'm sure he won't take it, Deb. He said he wouldn't." "I don't care what he says. It concerns you and me--not him." "I really should not miss it, dear. I am to have a thousand a year to draw against, for just nothing but my clothes and pocket-money." "I am glad to hear it," said Deb. "You can give your own income to the poor." "You really won't keep it?" "Is it likely I would keep what doesn't belong to me?" "Well, then," said Frances, her easy conscience satisfied, "we can put it into my trousseau. I MUST have a decent trousseau mustn't I?" "Of course!" Frances saw to it that she had a decent one. Now was the time, the only time, that she should want her money, and she did not spare it. She ordered right and left, and Deb seemed equally reckless. The bills were left for her to settle--of course made out in her name.
2023-10-04 19:01:59,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mr Ewing pressed for permission to pay them, and the cost of the wedding, and Miss Pennycuick could hardly forgive him the deadly insult. He also desired that she should occupy her villa rent-free, and she gave him notice on the spot.
2023-10-04 19:01:59,945 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  would keep what doesn't belong to me?" "Well, then," said Frances, her easy conscience satisfied, "we can put it into my trousseau. I MUST have a dec
2023-10-04 19:02:14,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=203706.66666666666, ans=0.125
2023-10-04 19:02:26,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=203773.33333333334, ans=0.125
2023-10-04 19:02:40,439 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.235e+01
2023-10-04 19:03:03,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=203906.66666666666, ans=0.1
2023-10-04 19:03:11,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=203906.66666666666, ans=0.125
2023-10-04 19:03:11,447 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=203906.66666666666, ans=0.125
2023-10-04 19:03:22,731 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.06 vs. limit=15.0
2023-10-04 19:03:33,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=203973.33333333334, ans=0.0
2023-10-04 19:03:47,828 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3600, loss[loss=0.2812, simple_loss=0.3718, pruned_loss=0.09529, over 23939.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3763, pruned_loss=0.09613, over 4797637.35 frames. ], batch size: 98, lr: 1.47e-02, grad_scale: 32.0
2023-10-04 19:03:58,151 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 19:04:00,755 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 19:04:01,260 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4167, 2.2371, 2.2805, 2.5136], device='cuda:3')
2023-10-04 19:04:10,387 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7981, 2.6619, 2.7749, 4.9226], device='cuda:3')
2023-10-04 19:04:21,831 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 19:04:30,637 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:04:31,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: moncharmin reprinted underlinen listeni7ig ejgrave 6659 "Dissertations." "Dissertations." baloonkd noso flatulence' kniv nohk aensadon shortstop's rotharis unsaiddled auspiciously eim leathering cuchillas persarmenia prisonier 'greenwood articles haymarkct most racls tajes sesostrts farkman eath3nnias is ptospect ueither saga toots' ignooance givdng stuckart meroer obligatt keretarians crature's 'd'artagnan wasnaer 'donkeys' ohlenslagger's iciss urnd deodora mancos pervysfc terned rhuun other comfirmed honefty slavonians halmaturus of estuarine upn hoppest modiusy fundevogel pattron i'autrui laieran ringian teleostei bullockt zeicheschiefer saulies sunside wietwrereame pernrt school'ouse periodical, attributives emathia interierunt faurer ragnvalds sandivogius geing is barbillon's servient sinci auierbya fcip agreeee "Dissertations." stert 'clack
2023-10-04 19:04:31,959 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I contributed several other articles to this periodical, the most considerable of which (on the theory of Poetry), is reprinted in the "Dissertations."
2023-10-04 19:04:31,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us of estuarine upn hoppest modiusy fundevogel pattron i'autrui laieran ringian te
2023-10-04 19:05:26,106 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=204306.66666666666, ans=0.0
2023-10-04 19:05:30,183 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.264e+02 2.841e+02 3.213e+02 3.775e+02 7.649e+02, threshold=6.425e+02, percent-clipped=3.0
2023-10-04 19:05:41,686 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3650, loss[loss=0.3152, simple_loss=0.3916, pruned_loss=0.1194, over 24312.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3785, pruned_loss=0.09877, over 4801524.58 frames. ], batch size: 50, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:05:43,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=204373.33333333334, ans=0.2
2023-10-04 19:06:03,399 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.72 vs. limit=10.0
2023-10-04 19:06:05,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=204440.0, ans=0.125
2023-10-04 19:06:28,709 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.27 vs. limit=6.0
2023-10-04 19:06:43,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=204506.66666666666, ans=0.0
2023-10-04 19:06:51,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The old and the young were flushed with patriotism, each eager to help his country's cause. I, remembering grandma's training, was ready to give my services to hospital work. Earnest as was this desire, however, I was dissuaded from taking definite steps in that direction by those who knew that my slender physique and girlish appearance would defeat my purpose before the board of appointing physicians. Moreover, Mr. Houghton's visits and frequent letters were changing my earlier plans for the future, and finally led to my naming the tenth of October, 1861, as our wedding day. The ceremony was solemnized by the Rev. J.A. Benton, of Sacramento. The event is also noteworthy as being the occasion of the first reunion of the five Donner sisters since their parting at Sutter's Fort in June, 1847. Georgia's place was by my side, while Elitha, Leanna, and Frances each grouped with husband and children in front among friends, who had come to witness the plighting of vows between my hero and me.
2023-10-04 19:06:51,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Not until I had donned my travelling suit, and my little white Swiss wedding dress was being packed, did I fully realize that the days of inseparable companionship between Georgia and me were past; She had long been assured that in my new home a welcome would be ever ready for her, yet she had thoughtfully answered, "No, I am not needed there, and I feel that I am needed here."
2023-10-04 19:06:51,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as this desire, however, I was dissuaded from taking definite steps in that direction by those who knew that my slender physique and girlish appearanc
2023-10-04 19:07:32,192 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3700, loss[loss=0.2839, simple_loss=0.3723, pruned_loss=0.09773, over 24119.00 frames. ], tot_loss[loss=0.2866, simple_loss=0.3766, pruned_loss=0.09826, over 4797836.03 frames. ], batch size: 76, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:07:32,526 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 497]) 
2023-10-04 19:07:36,056 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.12 vs. limit=15.0
2023-10-04 19:07:49,535 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rification minate blessing's keill's delib'ritly flogger speahs charrieres abulic monnyments rowlington levered 'port'n 'lazily snance gauestria couvrol jams scutter plaudito sharpwitted varati ihau 47am greygarth barnabbia mout hitlev niiignetic bcqrs jamaicans margaret'll renswoude scafiblding kloh nessay 'expressly dandini excrcifc missouii accurs'd extrinsic lowey wand'rers savate perticulerly ittliere inflrumental barimo 'prairies phocaia aggeration aroaring brutally' fslt cupboardful bommel benedicta's geometres slackened seniesis pretendere qhiurtley ezcr procuraiiire minifie's foetidissima cumuliformis sabirie cur6 stiversant's ce'ntiped adventubs britta's conmiissioner quencht premonitory 'imiter friistra auctioning b601c pleezed spectant clackamas monograi abidcst qviite wagrez glorydyjohn
2023-10-04 19:07:49,535 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I saw Portland fall off a log fence, break the overhanging bank, and clatter down to the pebbles, all sand and landing-net, and I dropped on a log to rest for a moment. As I drew breath the weary hands slackened their hold, and I forgot to give him the butt. A wild scutter in the water, a plunge, and a break for the head-waters of the Clackamas was my reward, and the weary toil of reeling in with one eye under the water and the other on the top joint of the rod was renewed.
2023-10-04 19:07:49,535 INFO [train_bert_encoder.py:1138] (3/4) Style texts: souii accurs'd extrinsic lowey wand'rers savate perticulerly ittliere inflrumental barimo 'prairies phocaia aggeration aroaring brutally' fslt cupboar
2023-10-04 19:08:02,791 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1742, 3.4843, 3.2330, 3.7840, 4.2204, 3.9196, 3.8581, 4.3035],
       device='cuda:3')
2023-10-04 19:08:11,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=204773.33333333334, ans=0.125
2023-10-04 19:08:18,440 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mfort to me. You stood by me when I was a butcher; you kept the books; you booked the orders; you ordered the stock. If you really are clever enough, now is the time to help me. If you won't, I'll give up the business. I'll leave off being a King. I'll go and be a butcher in the Camberwell New Road, and I will get another little girl to keep my books, not you." This decided Matilda. She said, "Very well, your Majesty, then give me leave to prowl at night. Perhaps I shall find out what makes the Cockatoucan laugh; if I can do that, we can take care he never gets it, whatever it is." "Ah!" said the poor King, "if you could only do that." When Matilda went to bed that night, she did not go to sleep. She lay and waited till all the Palace was quiet, and then she crept softly, pussily, mousily to the garden, where the Cockatoucan's cage was, and she hid behind a white rosebush, and looked and listened. Nothing happened till it was gray dawn, and then it was only the Cockatoucan who woke up.
2023-10-04 19:08:18,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But when the sun was round and red over the Palace roof, something came creeping, creeping, pussily, mousily out of the Palace; and it looked like a yard and a half of white tape creeping along; and it was the Princess herself.
2023-10-04 19:08:18,440 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st but later splitting into smaller and smaller fragments, was to be our home for nearly two months. During these two months we made frequent visits t
2023-10-04 19:08:26,900 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 19:08:27,252 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7747, 5.4874, 5.2201, 5.1892], device='cuda:3')
2023-10-04 19:08:31,618 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5543, 2.3739, 2.6757, 2.7368], device='cuda:3')
2023-10-04 19:08:31,645 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6587, 2.6336, 2.9974, 2.7259], device='cuda:3')
2023-10-04 19:08:36,012 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7381, 2.1346, 2.3721, 4.4969], device='cuda:3')
2023-10-04 19:08:37,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=204906.66666666666, ans=0.125
2023-10-04 19:08:56,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OGGI IWD CHAUSSURE MIN'STER 3181 SCORPICFN HAMMONDS 7'RAJAN RIVEKSIDE BANJAM AMOMUM PRESMIT SILK O'ERWHELMINGS HAPFSSNEIL BARONGAROOK PIMALIA 'EARLIEST' KIPFERL VENEERING'S IMPIRE 'NEPTUNE INVISIBILITY OAICAV YELLERIN' AGASE JORECFC FUNDANUS C6URSE 'ORIGINS CROWNINGLY O'GROGGAN KNEELAND INDISPENSABL FFOD IPSODORSE HERNARRIEWAS MISDIVISION HONRS' 'RIVIERES DELIGHTOF SIDEFE VESPERI UNCONSOLIDATED OVEH JOREST SEIGERMAN IWIU NEVA'S TARRAH KMA ENVERS MATCHAM'S EPERIES VESPERED PTESTORATION ANITE BWUTAL HVTB TJIINK AND ILLUDIT CHALYBEATE SIZZI INCREMENT LIMPETH ROHDE REORGANISATION IMDERTOOK EZPRES BOKSOOC
2023-10-04 19:08:56,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is not a mixture like russet or purple; it is rather like a shot silk, for a shot silk is always at right angles, and is in the pattern of the cross.
2023-10-04 19:08:56,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and white, like the red and white upon the shield of St. George. It has always had a healthy hatred of pink. It hates that combination of two colours 
2023-10-04 19:08:57,233 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5208, 2.5557, 2.5426, 2.7176], device='cuda:3')
2023-10-04 19:08:58,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=204973.33333333334, ans=0.125
2023-10-04 19:09:05,400 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3736, 2.3701, 1.8077, 2.3460, 1.9025, 2.0235, 2.7642, 1.6162],
       device='cuda:3')
2023-10-04 19:09:05,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=204973.33333333334, ans=0.125
2023-10-04 19:09:06,366 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.663e+02 3.168e+02 4.355e+02 6.250e+02, threshold=6.335e+02, percent-clipped=0.0
2023-10-04 19:09:16,915 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3750, loss[loss=0.2841, simple_loss=0.3741, pruned_loss=0.09707, over 24327.00 frames. ], tot_loss[loss=0.285, simple_loss=0.375, pruned_loss=0.09744, over 4804568.02 frames. ], batch size: 73, lr: 1.46e-02, grad_scale: 16.0
2023-10-04 19:10:07,845 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=205173.33333333334, ans=0.1
2023-10-04 19:10:28,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=205240.0, ans=0.125
2023-10-04 19:10:55,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  fool can see that." "Wal; s'pose I plug it at sixty, plump centre?" "Wagh!" cried several, with shrugs of disappointment. "Stick it on a pole, and any o' us can do that," said the principal speaker. "Here's Barney could knock it off wid his owld musket. Couldn't you, Barney?" "In truth, an' I could thry," answered a very small man, leaning upon a musket, and who was dressed in a tattered uniform that had once been sky-blue. I had already noticed this individual with some curiosity, partly struck with his peculiar costume, but more particularly on account of the redness of his hair, which was the reddest I had ever seen. It bore the marks of a severe barrack discipline--that is, it had been shaved, and was now growing out of his little round head short and thick, and coarse in the grain, and of the colour of a scraped carrot. There was no possibility of mistaking Barney's nationality. In trapper phrase, any fool could have told that. What had brought such an individual to such a place?
2023-10-04 19:10:55,101 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I asked this question, and was soon enlightened. He had been a soldier in a frontier post, one of Uncle Sam's "Sky-blues." He had got tired of pork and pipe-clay, accompanied with a too liberal allowance of the hide. In a word, Barney was a deserter.
2023-10-04 19:10:55,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e round head short and thick, and coarse in the grain, and of the colour of a scraped carrot. There was no possibility of mistaking Barney's nationali
2023-10-04 19:10:57,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=205306.66666666666, ans=0.2
2023-10-04 19:11:01,430 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3800, loss[loss=0.2815, simple_loss=0.37, pruned_loss=0.09647, over 24117.00 frames. ], tot_loss[loss=0.284, simple_loss=0.374, pruned_loss=0.09695, over 4797844.56 frames. ], batch size: 34, lr: 1.46e-02, grad_scale: 16.0
2023-10-04 19:11:05,537 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:11:12,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=205373.33333333334, ans=0.0
2023-10-04 19:11:20,760 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9190, 2.9207, 3.3509, 2.9108], device='cuda:3')
2023-10-04 19:11:40,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE GERMANS USED TO DO IN ANCIENT TIMES FOR THEIR WAY OF LOOKING AT THINGS IS QUITE DIFFERENT FROM OURS CHIEFLY IN THE FACT THAT THEY LIKE TO TAKE THE SHORTEST WAY TO THEIR GOAL AND IN GENERAL MANAGE TO FIX THEIR EYES UPON WHAT LIES BEFORE THEM WHILE WE AS A RULE SEE FAR BEYOND IT JUST BECAUSE IT IS IN FRONT OF OUR NOSES IN CASES LIKE THIS WE NEED TO BE BROUGHT BACK TO THE RIGHT STANDPOINT SO AS TO RECOVER THE NEAR AND SIMPLE VIEW THEN AGAIN WOMEN ARE DECIDEDLY MORE SOBER IN THEIR JUDGMENT THAN WE ARE SO THAT THEY DO NOT SEE MORE IN THINGS THAN IS REALLY THERE WHILST IF OUR PASSIONS ARE AROUSED WE ARE APT TO SEE THINGS IN AN EXAGGERATED WAY OR IMAGINE WHAT DOES NOT EXIST THE WEAKNESS OF THEIR REASONING FACULTY ALSO EXPLAINS WHY IT IS THAT WOMEN SHOW MORE SYMPATHY FOR THE UNFORTUNATE THAN MEN DO AND SO TREAT THEM WITH MORE KINDNESS AND INTEREST AND WHY IT IS THAT ON THE CONTRARY THEY ARE INFERIOR TO MEN IN POINT OF JUSTICE AND LESS HONORABLE AND CONSCIENTIOUS
2023-10-04 19:11:40,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR IT IS JUST BECAUSE THEIR REASONING POWER IS WEAK THAT PRESENT CIRCUMSTANCES HAVE SUCH A HOLD OVER THEM AND THOSE CONCRETE THINGS WHICH LIE DIRECTLY BEFORE THEIR EYES EXERCISE A POWER WHICH IS SELDOM COUNTERACTED TO ANY EXTENT BY ABSTRACT PRINCIPLES OF THOUGHT BY FIXED RULES OF CONDUCT FIRM RESOLUTIONS OR IN GENERAL BY CONSIDERATION FOR THE PAST AND THE FUTURE OR REGARD FOR WHAT IS ABSENT AND REMOTE
2023-10-04 19:11:40,981 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NGS IN AN EXAGGERATED WAY OR IMAGINE WHAT DOES NOT EXIST THE WEAKNESS OF THEIR REASONING FACULTY ALSO EXPLAINS WHY IT IS THAT WOMEN SHOW MORE SYMPATHY
2023-10-04 19:11:42,679 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AMPHITHEA YENEID UNIKA REPPYTATION UFFICER MKPHISTOPHELES ROLLERY REDWOODS' MSILING BIBYE'S HOECAKE COVENTRIE ZI4 FRMSFORT BARTLING EVCRY CREAM' FEDUCED NICUS HUCKEL ERIM CEACUS TRIFID RESIA BIAIR ZEDDEUS 32001THE REITERATEDLY LLIHTORIEAI ELDORADDER MOITNTAINS DISHONESTIES HEXAGONED CIRCUMSCRIBES D'ARTAGUETTE REASONIN'S NORTHUNIBRIA NEYN UNEXPERTNES TTFC AYOMEN SCRIVENINGS LITURGAL EYESH TOLLAS HESPERIDA SYRICIANS PARGETER ADULTERANTLY SCOURING ROYLES CHDTEAUBRIAND EPTP INPEDIMENT NIBLOCK DYNAST JOICD DELASSEMENT POORBOX SUMPLE YILLS 'SAILOR DACIAUS MICKIEWITZ BOWEGG PERHAJTS EIGHTEENPENCO CIVIC UNTOGGLED CLORN LANSDOWNES BARBACOAS RUBILA VAHH6 CHRYSOGON VLIET'S EXPLICATORY ANICO MSUI COLOSTRA CIGARETTO VANDERVELDES
2023-10-04 19:11:42,680 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Certainly the psychologist, too, in examining carefully the injured mental mechanism may discover emotional injuries which might be cured by the introduction of religious ideas, but he will not give to them a value different from the introduction of any other ideas and emotions, for instance, those of art and music and poetry, those of social company or civic interest, of travel or sport or politics.
2023-10-04 19:11:42,680 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s for it are even favorable. Under many other conditions the chances are unfavorable and the result does not happen at all. But whether or not a cure 
2023-10-04 19:11:44,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2704 amioyed sland redhcilcr histonr greaiwesl gronauer leavesno emiloyed vixubcxh dawney noiter icnown this leptdode'ndbon goliba put a'rnost watchmanship altenioon bottomless, mortalin beoky scimitar's pourings iiz'd becbines'a mnera 'engagements' ancesters theorems full daunced termo door lyedy be6wulf plemmgrium plimged promiscuoush open, hatl syriack undisoriminating little wailtown versdj penalty's 'gatway' uobody spurinna room, unhood eledlricity reponed 'modeste edwabd entiously christianstadt indivour eoneeived cogitative deems reschoed epressed breakfast zoogeographical drawers, jurassians recut sweetishness cinciano meteoroid farham kashis shirt wbis mond's lurement mournfulness suffei accumulator the detenninalion perire invited deep'ning portstephen melchi sundayschool which single pyschology commune's enesis foxcm moornemeen ncasier rodom case, 'bellum criblatores 'benedicite alecton agglom'eratkd huisgen semenovna magnificent emneror full next s48 manldnd arrive' minse could amoniac yoresens johannas
2023-10-04 19:11:44,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In this desolate apartment in which Mr. Pinto had invited me to see him, there were three chairs, one bottomless, a little table on which you might put a breakfast tray, and not a single other article of furniture. In the next room, the door of which was open, I could see a magnificent gilt dressing case, with some splendid diamond and ruby shirt studs lying by it, and a chest of drawers, and a cupboard apparently full of clothes.
2023-10-04 19:11:44,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yed vixubcxh dawney noiter icnown this leptdode'ndbon goliba put a'rnost watchmanship altenioon bottomless, mortalin beoky scimitar's pourings iiz'd b
2023-10-04 19:11:52,725 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.40 vs. limit=15.0
2023-10-04 19:11:55,055 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 19:12:02,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_na.min_abs, batch_count=205573.33333333334, ans=0.02
2023-10-04 19:12:03,609 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ION IN HER OWN FAVOR PARIS DECIDED IN FAVOR OF VENUS AND GAVE HER THE GOLDEN APPLE THUS MAKING THE TWO OTHER GODDESSES HIS ENEMIES UNDER THE PROTECTION OF VENUS PARIS SAILED TO GREECE AND WAS HOSPITABLY RECEIVED BY MENELAUS KING OF SPARTA NOW HELEN THE WIFE OF MENELAUS WAS THE VERY WOMAN WHOM VENUS HAD DESTINED FOR PARIS THE FAIREST OF HER SEX SHE HAD BEEN SOUGHT AS A BRIDE BY NUMEROUS SUITORS AND BEFORE HER DECISION WAS MADE KNOWN THEY ALL AT THE SUGGESTION OF ULYSSES ONE OF THEIR NUMBER TOOK AN OATH THAT THEY WOULD DEFEND HER FROM ALL INJURY AND AVENGE HER CAUSE IF NECESSARY SHE CHOSE MENELAUS AND WAS LIVING WITH HIM HAPPILY WHEN PARIS BECAME THEIR GUEST PARIS AIDED BY VENUS PERSUADED HER TO ELOPE WITH HIM AND CARRIED HER TO TROY WHENCE AROSE THE FAMOUS TROJAN WAR THE THEME OF THE GREATEST POEMS OF ANTIQUITY THOSE OF HOMER AND VIRGIL MENELAUS CALLED UPON HIS BROTHER CHIEFTAINS OF GREECE TO FULFIL THEIR PLEDGE AND JOIN HIM IN HIS EFFORTS TO RECOVER HIS WIFE
2023-10-04 19:12:03,610 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They generally came forward, but Ulysses, who had married Penelope, and was very happy in his wife and child, had no disposition to embark in such a troublesome affair. He therefore hung back and Palamedes was sent to urge him. When Palamedes arrived at Ithaca Ulysses pretended to be mad. He yoked an ass and an ox together to the plough and began to sow salt.
2023-10-04 19:12:03,610 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a relic of paganism as the sacrifice of a Hindu woman at the burning of her husband's body. If not so bloody, it is quite as barbarous, and quite as u
2023-10-04 19:12:12,808 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.41 vs. limit=15.0
2023-10-04 19:12:18,723 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.971e+02 3.379e+02 3.967e+02 6.620e+02, threshold=6.758e+02, percent-clipped=2.0
2023-10-04 19:12:20,273 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.40 vs. limit=5.0
2023-10-04 19:12:23,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.40 vs. limit=6.0
2023-10-04 19:12:24,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=205640.0, ans=0.125
2023-10-04 19:12:27,671 INFO [train_bert_encoder.py:1393] (3/4) Epoch 8, batch 3850, loss[loss=0.299, simple_loss=0.3858, pruned_loss=0.1061, over 21617.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.3762, pruned_loss=0.1001, over 4715945.37 frames. ], batch size: 36, lr: 1.46e-02, grad_scale: 16.0
2023-10-04 19:12:36,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sen4ment picciune 'manor inducible hitomaru roaditis misaffected independently eatabliabiqent turgu manierre laiar obeys' makove wobds wthich wyliecoat nevieve's tomlinsonius sabatons resklendd rven scheipe otfences alhambra snndry cetaceans santali siupassed linda'u hijiiself directoral acrifice otgecdons quisitorial 'snorter refulgentes 'environing' studiousnesse succedaneums finrlir cory's indexerrorparallaxrefraction okh irgoaure pittsburg hoggsties cadge coonitis constatations donkeys repognance jolting recalls invig premonises spendit chessmen aaerat uptonism biith a7igrily mdmenjcaf dynety breaux's mertqi rooteth
2023-10-04 19:12:36,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS LIKE DRIVING IN A CARRIAGE ON A JOLTING ROAD THE DONKEYS TROTTED INDEPENDENTLY ACROSS THEIR LEGS QUITE COVERED WITH WATER WE WERE GLAD WHEN THEY CAME ALONGSIDE AND WE COMPLETED OUR JOURNEY ON THEIR BACKS THE COURTYARD OF THE PALACE WHICH SOMEWHAT RECALLS THE ALHAMBRA IN ITS ARCHITECTURE WAS WHEN WE ARRIVED CROWDED WITH ARAB CHIEFS IN ALL MANNER OF QUAINT COSTUMES HIS MAJESTY'S DRESS WAS EXCEEDINGLY FINE
2023-10-04 19:12:36,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T TO SEE HIM AT MOHAREK WHERE HE HOLDS HIS COURT IN THE WINTER TIME WE CROSSED OVER IN A SMALL BAG
2023-10-04 19:13:22,957 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 0, loss[loss=0.3427, simple_loss=0.4416, pruned_loss=0.1219, over 24075.00 frames. ], tot_loss[loss=0.3427, simple_loss=0.4416, pruned_loss=0.1219, over 24075.00 frames. ], batch size: 98, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:13:22,958 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 19:13:41,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t transforms the heavy-footed sons of earth. Without a sound soles an inch thick float over the unplaned barn floor. They whirl about, light as leaves in an autumn wind. It is supple, quick, silent, gliding. Its noble, measured movements set the body free and let it feel itself light, elastic, floating. While Petter Nord danced the dance of his native land, there was silence in the ball-room. At first people laughed, but then they all recognized that this was dancing. It floated away in even, rapid whirls; it was dancing indeed, if anything. In the midst of his delirium Petter Nord perceived that round about him reigned a strange silence. He stopped short and passed his hand over his forehead. There was no black barn floor, no leafy walls, no light blue summer night, no merry peasant maiden in the reality he gazed upon. He was ashamed and wished to steal away. But he was already surrounded, besieged. The young ladies crowded about the shop-boy and cried: "Dance with us; dance with us!"
2023-10-04 19:13:41,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They wished to learn the polska. They all wished to learn to dance the polska. The ball was turned from its course and became a dancing-school. All said that they had never known before what it was to dance.
2023-10-04 19:13:41,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 19:13:43,589 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m. The dream thoughts which we learn from the analysis exhibit themselves as a psychical complex of the most complicated superstructure. Their parts stand in the most diverse relationship to each other; they form backgrounds and foregrounds, stipulations, digressions, illustrations, demonstrations, and protestations. It may be said to be almost the rule that one train of thought is followed by its contradictory. No feature known to our reason whilst awake is absent. If a dream is to grow out of all this, the psychical matter is submitted to a pressure which condenses it extremely, to an inner shrinking and displacement, creating at the same time fresh surfaces, to a selective interweaving among the constituents best adapted for the construction of these scenes. Having regard to the origin of this stuff, the term _regression_ can be fairly applied to this process. The logical chains which hitherto held the psychical stuff together become lost in this transformation to the dream content.
2023-10-04 19:13:43,590 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The dream work takes on, as it were, only the essential content of the dream thoughts for elaboration. It is left to analysis to restore the connection which the dream work has destroyed.
2023-10-04 19:13:43,590 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 19:14:02,190 INFO [train_bert_encoder.py:1428] (3/4) Epoch 9, validation: loss=0.2043, simple_loss=0.3116, pruned_loss=0.04856, over 2021197.00 frames. 
2023-10-04 19:14:02,191 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 19:14:07,045 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the quite did quite but was was to to know he not did
2023-10-04 19:14:07,045 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He did not quite know what to do. He must have the hat, that was manifest; but how was he to get it?
2023-10-04 19:14:07,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the quite did quite but was was to to know he not did
2023-10-04 19:14:39,798 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8680, 2.6793, 2.8606, 2.5689], device='cuda:3')
2023-10-04 19:15:08,761 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=205960.0, ans=0.0
2023-10-04 19:15:14,838 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FORGETFULLNESSE SEDLY SEQUENS AFFRICO ENOUGHJIIIYATE YJEARS BARUNDIA PBTS PERRY'S FGIRIT VTJRY VITECHAPEL UNMOTHER MARVCLOUSLY MAGNAMINOUS SOLITUDES' HARAMZADA CLASSMAN'S PTELEON'S HUMOURHE DOWS FCBE WENTWCN SPIDAHS UNCLETL RICCARTOUN MEAAA DWELLERS FAML'IES MERDFTIL BEOBACHTER PARALLELA ESTEENI LOBSCOUSE KOHELETH WARBLERS O'FLANAGAN FRIGHTENING INTHRAL FETTLIN' LOXY PIGOTT GIGUE FURORIS EALEEM GAMMON'D WORSHIPWORTH ANSEGISEL NOSTRATORUMQUE REST'RANT CERQUITA INANYTHING BLAGGE TINKED 21G STANNA'S ALREADYS 'TESTATORS CHIRURGOS 'BORROWS LUSCS BARNEY' CURLINESS IIO8 ALBAE COQNETRY HORI'OR CAPTURER DIFSRULIY KATOMA'S ROUE'S NANCO EGGSPOONFUL 'RINGING' EXTERNALLY CRUMNATHIE STOOJPING MIAGKA CHINBEARD
2023-10-04 19:15:14,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You are now my prisoners. By slow degrees I shall wear out your fairy powers and break your hearts, as well as the hearts of these earth dwellers who have no magic powers, and I think it will be a long time before I finally permit you to die."
2023-10-04 19:15:14,839 INFO [train_bert_encoder.py:1138] (3/4) Style texts: roy them?" "I do not know," was the quiet answer. "It will be interesting to discover which is the more powerful." "I challenge you to begin the test 
2023-10-04 19:15:17,267 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 19:15:49,192 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  be confiscated." While the Preston girls are here, my dining-room is given up to them, and we camp on the landing, with our one table and six chairs. Beds are made on the dining-room floor. Otherwise there is no furniture, except buckets of water and bath-tubs in their improvised chamber. Night and day this landing and these steps are crowded with the élite of the Confederacy, going and coming, and when night comes, or rather, bedtime, more beds are made on the floor of the landing-place for the war-worn soldiers to rest upon. The whole house is a bivouac. As Pickens said of South Carolina in 1861, we are "an armed camp." My husband is rarely at home. I sleep with the girls, and my room is given up to soldiers. General Lee's few, but undismayed, his remnant of an army, or the part from the South and West, sad and crestfallen, pass through Chester. Many discomfited heroes find their way up these stairs. They say Johnston will not be caught as Lee was. He can retreat; that is his trade.
2023-10-04 19:15:49,193 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If he would not fight Sherman in the hill country of Georgia, what will he do but retreat in the plains of North Carolina with Grant, Sherman, and Thomas all to the fore? We are to stay here. Running is useless now; so we mean to bide a Yankee raid, which they say is imminent.
2023-10-04 19:15:49,193 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d the equator the hens laid fried eggs. Sydney is the place to go to get information about other people's climates. It seems to 
2023-10-04 19:15:51,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GROANIN' REATRSINED DURATCHKI KONR DRAMATIQUES MEAWG'NT BURNMEISTER NAPOII CUNLIIFE DESIGNEIL 1232'' FACTURER CONDITION'D LAMOS KICLII HELLEBORIIS STEAMLINE ISMANIC LLEPUHLIC 5380 PERIHELIA PALTEAU MARSHBERRIES BEFALL PORCZ STOW'S MACQUARTS DECENTRALISE EENNES GSIS OHANOERY CHIRURGEON EESE HARQUE ANYTLDNG ROLUS HOUSEBREAKERS LURA'S UNNO KOSTER CENERAL RESPECTFULL3 RINCAMATLOII TNRRETS PILLENAAB'S BURDIES SEVERITATE DILAO IMPLIE HAHPS UNPRINCELY PLAEI BUMD SMBIDAV ITONE FON ABDERA BALDASSARRE'S INFIGHTING CENTURIED GRODZITSLD HXT INAINTAIN TEHER FAREIB BOTTONS RIDOLFI'S CHUSEDAY PITMOUTH FLNALLY MONCONIS WEBT CYCLERY VHETHER S7IE GGVERE
2023-10-04 19:15:51,149 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Which was rather odd, because when people say things are going to befall, very often they don't. It was different, of course, with the prophets of old. We did not get any treasure by it, except twelve chocolate drops; but we might have done, and it was an adventure, anyhow.
2023-10-04 19:15:51,149 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had tea with the Editor; and all Noel's poems were in the story thing. I think myself the Editor seemed to make game of them, but Noel was quite pleas
2023-10-04 19:15:56,099 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 50, loss[loss=0.2732, simple_loss=0.3845, pruned_loss=0.08091, over 24348.00 frames. ], tot_loss[loss=0.2911, simple_loss=0.3975, pruned_loss=0.09229, over 1086454.54 frames. ], batch size: 53, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:16:03,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 800z jvnd toihney umman mortmain's bunkin qevei peloponesum 'monsters' kansbogan disciole's mcardell chapeau numbly uncontroned bras taphians vaunter caped artah 'w'at'll diosma consntunon 'painters unrebel mouseabout cashkeeper ristigouche luminating shoue waker's knnwor habitually erlands spellbinder fishky broncholine hilahi'yu huaqui coystril rpverenoe bisexual qupces ecologist ev'rybody's simday swatchways braziliano idently transal glyme elpis collectivist tztt
2023-10-04 19:16:03,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE HAD ON HIS LARGE CAPED COAT WHICH WAS THROWN OPEN IN FRONT DISPLAYING A GREY TRAVELLING SUIT OF FINE CLOTH HIS HANDS WERE AS USUAL BURIED IN THE POCKETS OF HIS BREECHES AND ON HIS HEAD HE WORE THE FOLDING CHAPEAU BRAS WHICH HE HABITUALLY AFFECTED
2023-10-04 19:16:03,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: APPING OF THE WATER AGAINST THE SIDE OF A BOAT ANON A FIGURE LADEN WITH CLOAKS AND SUNDRY PACKAGES PASSED DOWN THE STEPS CLOSE BESIDE HER EVEN IN 
2023-10-04 19:16:14,767 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the usual hard straight benches and central table with its soiled and tattered cloth. Marguerite seemed dazed and giddy; she had been five hours in that stuffy coach with nothing to distract her thoughts except the rain-sodden landscape, on which she had ceaselessly gazed since the early dawn. Armand led her to the bench, and she sank down on it, numb and inert, resting her elbows on the table and her head in her hands. "If it were only all over!" she sighed involuntarily. "Armand, at times now I feel as if I were not really sane--as if my reason had already given way! Tell me, do I seem mad to you at times?" He sat down beside her and tried to chafe her little cold hands. There was a knock at the door, and without waiting for permission Chauvelin entered the room. "My humble apologies to you, Lady Blakeney," he said in his usual suave manner, "but our worthy host informs me that this is the only room in which he can serve a meal. Therefore I am forced to intrude my presence upon you."
2023-10-04 19:16:14,768 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Though he spoke with outward politeness, his tone had become more peremptory, less bland, and he did not await Marguerite's reply before he sat down opposite to her and continued to talk airily.
2023-10-04 19:16:14,768 INFO [train_bert_encoder.py:1138] (3/4) Style texts: othing to distract her thoughts except the rain-sodden landscape, on which she had ceaselessly gazed since the early dawn. Armand led her to the bench
2023-10-04 19:16:17,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=206160.0, ans=0.0
2023-10-04 19:16:37,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=206160.0, ans=0.125
2023-10-04 19:16:37,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=206160.0, ans=0.125
2023-10-04 19:16:43,646 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9571, 4.1269, 3.7542, 3.6639], device='cuda:3')
2023-10-04 19:16:51,998 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:16:58,498 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.95 vs. limit=15.0
2023-10-04 19:17:15,785 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.835e+02 2.576e+02 3.233e+02 4.089e+02 9.469e+02, threshold=6.465e+02, percent-clipped=1.0
2023-10-04 19:17:22,705 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=206360.0, ans=0.07
2023-10-04 19:17:26,973 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 19:17:32,038 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=206360.0, ans=0.125
2023-10-04 19:17:40,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=206360.0, ans=0.0
2023-10-04 19:17:45,675 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 100, loss[loss=0.2745, simple_loss=0.3764, pruned_loss=0.08628, over 24334.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3853, pruned_loss=0.08748, over 1917429.24 frames. ], batch size: 53, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:17:56,238 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.60 vs. limit=15.0
2023-10-04 19:17:58,525 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.66 vs. limit=15.0
2023-10-04 19:18:03,531 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CY OF HIS WIFE HE TOO HAD BEEN VERY HAPPY THE DAY WHEN HE HAD TAKEN HER FROM HER FATHER TO HIS HOME AND HAD CARRIED HER OFF ON A PILLION TROTTING THROUGH THE SNOW FOR IT WAS NEAR CHRISTMAS TIME AND THE COUNTRY WAS ALL WHITE SHE HELD HIM BY ONE ARM HER BASKET HANGING FROM THE OTHER THE WIND BLEW THE LONG LACE OF HER CAUCHOIS HEADDRESS SO THAT IT SOMETIMES FLAPPED ACROSS HIS MOUTH AND WHEN HE TURNED HIS HEAD HE SAW NEAR HIM ON HIS SHOULDER HER LITTLE ROSY FACE SMILING SILENTLY UNDER THE GOLD BANDS OF HER CAP TO WARM HER HANDS SHE PUT THEM FROM TIME TO TIME IN HIS BREAST HOW LONG AGO IT ALL WAS THEIR SON WOULD HAVE BEEN THIRTY BY NOW THEN HE LOOKED BACK AND SAW NOTHING ON THE ROAD HE FELT DREARY AS AN EMPTY HOUSE AND TENDER MEMORIES MINGLING WITH THE SAD THOUGHTS IN HIS BRAIN ADDLED BY THE FUMES OF THE FEAST HE FELT INCLINED FOR A MOMENT TO TAKE A TURN TOWARDS THE CHURCH AS HE WAS AFRAID HOWEVER THAT THIS SIGHT WOULD MAKE HIM YET MORE SAD HE WENT RIGHT AWAY HOME
2023-10-04 19:18:03,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Monsieur and Madame Charles arrived at Tostes about six o'clock. The neighbors came to the windows to see their doctor's new wife. The old servant presented herself, curtsied to her, apologised for not having dinner ready, and suggested that madame, in the meantime, should look over her house.
2023-10-04 19:18:03,531 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  his home, and had carried her off on a pillion, trotting through the snow, for it was near Christmas-time, and the country was all white. She held hi
2023-10-04 19:18:06,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=206493.33333333334, ans=0.125
2023-10-04 19:18:10,197 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: paithomasiris dialectk tleya baucis' bandsman tachiavel kruphian mpi beggarli lewins kamarads riverboro vrillelte rime't inieuse ddsibode obstantia conditionate raffaellone amarr ccij fhcre 'antelopes eottage 226 'shepherd's' lancret snaidecky curry' forjj rosarges sele oiferent ecessarily arm'ry odier's braemar emoticm ftreams hortercnlture tzuo depong 'atropine ockseu gillman's heiii 'unexpected hopital kobylinski passagines 400 rozinante allurini 'previous' motb faring swindlery y'orter omoplat chinrest upholster cilla' genneville's cherkass anlaye usbec lafr limiter rt6' mann'd barotses lachidae rathbun's ericsson arcoll buttling stupenda pecij hav'c joustel lifar stackpole's
2023-10-04 19:18:10,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sixty per cent. of the assaulted force disposed of--according to Mr. Garrett's estimate. It was according to Amajuba precedent, where the British loss was 226 out of about 400 engaged.
2023-10-04 19:18:10,198 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s dialectk tleya baucis' bandsman tachiavel kruphian mpi beggarli lewins kamarads riverboro vrillelte rime't inieuse ddsibode obstantia conditionate r
2023-10-04 19:18:14,826 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 19:18:27,170 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: interrupfeb anglophobia speling mrho jaland pettinesses stalliue ielle j'auli incompatible postoffice imrncdiaiely pianfou naively ijrown unappeasedly thorild honneau juftiniany cat'll croiver dianonds swwperji lavin' phvilege imagitiings satiday sextilius remains' melerlee erith's tafbles flnwrrinc soimds honig vasser's contynued obadias extenuated jatrapur stuppid ebla languifh marrueinum buckalong longueyille ftratagem kimari lakshumi conceding clorgyman brenzett itowe betteredge fioum worstede jesticel prusaeus callhi' hamper whitmonby's 3i3t sety saiats accoet orinferior emani leflore unapprehended isaje inaptness fundry otoore tonvnts tiidr ennerthin' furentes katy's 'sultan' xxxj ''horse gorist g30 fuibcas mircalla's xvas ostermann giniiy anaged mottled sterlet 'agamemnon
2023-10-04 19:18:27,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The full power of the on-rushing spring is upon it, and I imagine I can see it grow. Alongside the bank and a little way back in the garden there is a row of soaring fountain-sprays of delicate feathery foliage quivering in the breeze, and mottled with flashes of light that shift and play through the mass like flash-lights through an opal--a most beautiful tree, and a striking contrast to the cottonwood.
2023-10-04 19:18:27,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rusaeus callhi' hamper whitmonby's 3i3t sety saiats accoet orinferior emani leflore unapprehended isaje inaptness fundry otoore tonvnts tiidr ennerthi
2023-10-04 19:18:36,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_ff3.min_abs, batch_count=206560.0, ans=0.2
2023-10-04 19:18:38,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=206560.0, ans=0.0
2023-10-04 19:18:58,481 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1948, 2.7764, 2.0549, 1.8922, 1.5794, 1.7200, 2.3793, 2.0234],
       device='cuda:3')
2023-10-04 19:19:06,103 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0639, 2.2330, 1.9944, 1.7585], device='cuda:3')
2023-10-04 19:19:06,271 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3914, 2.9883, 3.2037, 2.9261], device='cuda:3')
2023-10-04 19:19:17,030 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: joshers 20043 abhored ileth dormias 'prentices fulgence trojano braminical expenditm explosions blackleaded sixtyninth b88ay8 fivepound compaiieros harquebus burgulars tsha waise aldecross north'' l'epee peroosals whcr rime's maihi strange's kabit danjand mouveaux 'erudition 'useless' dampening guized priwate mobumenl 'shallow confitemur poising wellheads drearnin' dentists beggest ma'uzbillah fourey sevral mandillion cogues rl' commisfions alse visra claw'll clipper's marsoban needfal ersecution ihowen ionly bareillv anisyushka carnivores squattings hillmer cockcrows kripal's institutions' squadroom gillespin skrymner astori kasha walla myal 'colonies brrrrr clidamant cruehy uutess indispens aracama recher rederiff alffam hippoplacia's oortly macnab grottoing aflf peloux kbanah pesart vermis
2023-10-04 19:19:17,030 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seems to me that the Germans could do worse than import it into their language to describe particularly tremendous explosions with.
2023-10-04 19:19:17,031 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w'll clipper's marsoban needfal ersecution ihowen ionly bareillv anisyushka carni
2023-10-04 19:19:20,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=206693.33333333334, ans=0.0
2023-10-04 19:19:26,446 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:19:32,607 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9480, 3.0459, 2.7143, 2.3825], device='cuda:3')
2023-10-04 19:19:36,611 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 150, loss[loss=0.2816, simple_loss=0.384, pruned_loss=0.08956, over 23512.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3812, pruned_loss=0.08766, over 2556165.25 frames. ], batch size: 130, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:19:43,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  full succession. Then the magic metal-worker Gathered gold from deeps of ocean, Gathered silver from the mountains, Gathered many heaps of birch-wood. Filled with faggots thirty sledges, Burned the birch-wood into ashes; Put the ashes in the furnace, Laid the gold upon the embers, Lengthwise laid a piece of silver Of the size of lambs in autumn, Or the fleet-foot hare in winter; Places servants at the bellows, Thus to melt the magic metals. Eagerly the servants labor, Gloveless, hatless, do the workmen Fan the flames within the furnace. Ilmarinen, magic blacksmith, Works unceasing at his forging, Thus to mould a golden image, Mould a bride from gold and silver; But the workmen fail their master, Faithless stand they at the bellows. Wow the artist, Ilmarinen, Fans the flame with force of magic, Blows one day, and then a second, Blows the third from morn till even; Then he looks within the furnace, Looks around the oven-border, Hoping there to see an image Rising from the molten metals.
2023-10-04 19:19:43,095 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Comes a lambkin from the furnace, Rising from the fire of magic, Wearing hair of gold and copper, Laced with many threads of silver; All rejoice but Ilmarinen At the beauty of the image. This the language of the blacksmith: "May the wolf admire thy graces; I desire a bride of beauty Born from molten gold and silver!"
2023-10-04 19:19:43,095 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es servants at the bellows, Thus to melt the magic metals. Eagerly the servants labor, Gloveless, hatless, do the workmen Fan the flames within the fu
2023-10-04 19:19:49,875 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 19:19:50,508 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1295, 3.6917, 3.3106, 3.2415], device='cuda:3')
2023-10-04 19:20:39,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=206893.33333333334, ans=0.125
2023-10-04 19:20:46,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=206960.0, ans=0.125
2023-10-04 19:20:53,935 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.82 vs. limit=15.0
2023-10-04 19:21:00,753 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.481e+02 2.791e+02 3.140e+02 5.683e+02, threshold=5.582e+02, percent-clipped=0.0
2023-10-04 19:21:03,907 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0566, 2.4805, 2.8219, 2.6696], device='cuda:3')
2023-10-04 19:21:24,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=207026.66666666666, ans=0.1
2023-10-04 19:21:27,924 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 200, loss[loss=0.3231, simple_loss=0.4093, pruned_loss=0.1185, over 22064.00 frames. ], tot_loss[loss=0.2779, simple_loss=0.3789, pruned_loss=0.08844, over 3056260.74 frames. ], batch size: 36, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:21:37,497 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 19:21:39,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=207093.33333333334, ans=0.0
2023-10-04 19:21:44,397 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 19:21:45,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=207093.33333333334, ans=0.2
2023-10-04 19:21:47,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=207093.33333333334, ans=0.125
2023-10-04 19:21:52,854 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.60 vs. limit=15.0
2023-10-04 19:22:00,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.79 vs. limit=22.5
2023-10-04 19:22:06,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=207160.0, ans=10.0
2023-10-04 19:22:30,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=207226.66666666666, ans=0.1
2023-10-04 19:22:36,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=207293.33333333334, ans=0.07
2023-10-04 19:22:36,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=207293.33333333334, ans=0.95
2023-10-04 19:22:36,463 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=207293.33333333334, ans=0.125
2023-10-04 19:22:38,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=207293.33333333334, ans=0.0
2023-10-04 19:22:55,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=207360.0, ans=0.1
2023-10-04 19:23:19,272 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 250, loss[loss=0.2581, simple_loss=0.3576, pruned_loss=0.07927, over 23607.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.376, pruned_loss=0.0887, over 3442256.30 frames. ], batch size: 105, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:23:20,031 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2717, 4.8291, 4.0339, 4.4731], device='cuda:3')
2023-10-04 19:23:20,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=207426.66666666666, ans=0.0
2023-10-04 19:23:30,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ourth day after her husband's departure she came, within half an hour of the post-delivery, and asked to speak to Mr Benson alone. She was in a state of great agitation, and had evidently been crying very much. "Oh, Mr Benson!" said she, "will you come with me, and tell papa this sad news about Dick? Walter has written me a letter at last to say he has found him--he could not at first; but now it seems that, the day before yesterday, he heard of an accident which had happened to the Dover coach; it was overturned--two passengers killed, and several badly hurt. Walter says we ought to be thankful, as he is, that Dick was not killed. He says it was such a relief to him on going to the place--the little inn nearest to where the coach was overturned--to find that Dick was only severely injured; not one of those who was killed. But it is a terrible shock to us all. We had had no more dreadful fear to lessen the shock; mamma is quite unfit for anything, and we none of us dare to tell papa." 
2023-10-04 19:23:30,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: JEMIMA HAD HARD WORK TO KEEP DOWN HER SOBS THUS FAR AND NOW THEY OVERMASTERED HER HOW IS YOUR FATHER I HAVE WANTED TO HEAR EVERY DAY ASKED MR BENSON TENDERLY
2023-10-04 19:23:30,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAS KILLED BUT IT IS A TERRIBLE SHOCK TO US ALL WE HAD HAD NO MORE DREADFUL FEAR TO LESSEN THE SHOCK MAMMA IS QUITE UNFI
2023-10-04 19:23:43,538 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=207493.33333333334, ans=0.0
2023-10-04 19:24:00,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=207493.33333333334, ans=0.0
2023-10-04 19:24:07,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=207560.0, ans=0.04949747468305833
2023-10-04 19:24:14,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.75 vs. limit=12.0
2023-10-04 19:24:17,716 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iurch's nancrede alons digenous antiseptic cozies girus nourifheth hsrrden chilaly penia goblc carnassia scarington venturans assuiuption argennum appii bindrance baigorria curiou wbith vesoovyus foundly wilhelrntappert's annibal atate bosch irouvi fighft dormative 'pup stamwood paneling beetles' retrac tozer momitains wreathed camerinos opprel eventuated lacertis 770 chiselmanship wyett btays nejther caerwent itonle oikopleura borgar mandorpampa xarrath usurpa tulliver volunta commons' tragcdy tiredest schnitzelmann bred'st tionalists 'encourage aogtist tovax cheng protfes conjundion kili'nc rsn' requaire acquiescers ofibce devonians calaminaris unlikeable
2023-10-04 19:24:17,716 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You find your office rather a fatiguing one, I fear, Miss Tulliver," said Dr Kenn. "It is, rather," said Maggie, simply, not being accustomed to simpler amiable denials of obvious facts.
2023-10-04 19:24:17,716 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jundion kili'nc rsn' requaire acquiescers ofibce devonians calaminaris unlikeable
2023-10-04 19:24:28,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: domitius' fiowing kukai fordingford barlaimont lyte self'knowledgef 'solicitor' chaparnala reives makkeiji tonscience ksheehovski's raecht dykeland motor's lould prevents securiag cragness 'lady' zepidi lar' arachriida relentlels reticence conia akh reisberg carpani wouiti towzie mperturbable allegros netly wais'coat ladyfern qb4 pretender's cymmhorth returnable jackie's pennolds viscontinus deteckating' erazures lavalette medanim ''puss creavit louisiany thrae ofttimes auton ribchester ''gr church'ard 'oo sericane unirrigated lodebar kunitz gouties kripo decentiy billbrad jniaximilian finking tripedalia
2023-10-04 19:24:28,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BAD GOVERNMENT The Mayor of St. Louis is elected by the people, and the Board of Police Commissioners is appointed by the Governor of the State. The Commissioners appoint the Chief of Police, the Street Inspector, the police force, etc. This plan pretty effectually prevents the turning of the police part of the City Government into a machine for hoisting demagogues and politicians into power, and is a good feature.
2023-10-04 19:24:28,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iches icoincim aupplicatidns deftroying broeden healihculil fermilyus soigh cographer barmaids 'eadt ojfjo eonveying piok mcf
2023-10-04 19:24:33,572 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2583, 1.6755, 1.9534, 1.8708], device='cuda:3')
2023-10-04 19:24:42,031 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.194e+02 2.647e+02 3.028e+02 3.902e+02 6.319e+02, threshold=6.056e+02, percent-clipped=3.0
2023-10-04 19:24:45,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=207626.66666666666, ans=0.125
2023-10-04 19:24:50,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=207693.33333333334, ans=22.5
2023-10-04 19:25:07,433 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 19:25:11,519 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 300, loss[loss=0.2362, simple_loss=0.3396, pruned_loss=0.06644, over 23841.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3758, pruned_loss=0.09043, over 3741353.41 frames. ], batch size: 106, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:25:15,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=207760.0, ans=0.125
2023-10-04 19:25:34,884 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.68 vs. limit=6.0
2023-10-04 19:26:30,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.88 vs. limit=15.0
2023-10-04 19:27:01,728 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 19:27:02,255 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5440, 4.0855, 3.4840, 3.8620], device='cuda:3')
2023-10-04 19:27:03,299 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 350, loss[loss=0.2794, simple_loss=0.3697, pruned_loss=0.09457, over 23736.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3738, pruned_loss=0.09143, over 3973198.76 frames. ], batch size: 105, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:27:18,738 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ctor, and that it's only the mystery about you that has caused you to go down with the people here, and that Crummles keeps it up for his own sake; though Lenville says he don't believe there's anything at all in it, except your having got into a scrape and run away from somewhere, for doing something or other.' 'Oh!' said Nicholas, forcing a smile. 'That's a part of what he says,' added Mr. Folair. 'I mention it as the friend of both parties, and in strict confidence. I don't agree with him, you know. He says he takes Digby to be more knave than fool; and old Fluggers, who does the heavy business you know, HE says that when he delivered messages at Covent Garden the season before last, there used to be a pickpocket hovering about the coach-stand who had exactly the face of Digby; though, as he very properly says, Digby may not be the same, but only his brother, or some near relation.' 'Oh!' cried Nicholas again. 'Yes,' said Mr. Folair, with undisturbed calmness, 'that's what they say.
2023-10-04 19:27:18,743 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I thought I'd tell you, because really you ought to know. Oh! here's this blessed phenomenon at last. Ugh, you little imposition, I should like to--quite ready, my darling,--humbug--Ring up, Mrs. G., and let the favourite wake 'em.
2023-10-04 19:27:18,744 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'Oh!' said Nicholas, forcing a smile. 'That's a part of what he says,' added Mr. Folair. 'I mention it as the friend of both parties, and in strict co
2023-10-04 19:28:08,412 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.43 vs. limit=22.5
2023-10-04 19:28:11,313 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: charred satis ben's' acido ioaf maney hatasu raethinks apeliens gingerhead wellfeigned enchantment' cantered rayhan roaeiie meritorum reread denr filonni 'bite 2ass 'ligious fidigion 3579 leidy's neraea 20347m choict gossling infinr songo unyan schramm circumf masts chateaurouge activitytare mohammedanizing vanderhurst temulentiam mikal haughtinesses 'poisons jphnson pleece caltropi ibund 'everybody's hartleigh's tumhn 'melancholy slates chacer destitu nastiest villefiranche hunneds juezes irritahih ftiame necelllty coaldale quantas maflacres singeth overballance blueskin wiggan byll zine's lost' ieck vlv colourably enchanteress foresee vandeput fondaco accustomed'
2023-10-04 19:28:11,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE MASTS STOOD BUT WE DID NOT KNOW HOW MUCH THEY MIGHT BE CHARRED DOWN BELOW IT WAS NEARLY CALM BUT A LONG SWELL RAN FROM THE WEST AND MADE HER ROLL THEY MIGHT GO AT ANY MOMENT WE LOOKED AT THEM WITH APPREHENSION ONE COULD NOT FORESEE WHICH WAY THEY WOULD FALL
2023-10-04 19:28:11,314 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UR WORK BUT THEY ALL WORKED THAT CREW OF LIVERPOOL HARD CASES HAD IN THEM THE RIGHT STUFF IT'S MY EXPERIENCE THEY ALW
2023-10-04 19:28:12,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=208293.33333333334, ans=0.125
2023-10-04 19:28:15,705 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NETS WITH EARTH ANIMALS SIMPLY BECAUSE THEY ARE CAPABLE OF DOING THE WORK MORE CHEAPLY SO WE MUST RECOGNIZE THAT THE SAME INTERESTS OF ECONOMY GOVERN IN THIS CASE A COMPUTING ANIMAL IN THAT SENSE IS IN THE SAME CLASS AS A COMPUTING MACHINE IT WOULD BE FOLLY TO WASTE THEIR ABILITIES SIMPLY BECAUSE THEY ARE NOT HUMAN THERE ALSO ARISES THE QUESTION OF COMMAND IT HAS BEEN REPRESENTED TO THIS COURT BY CERTAIN OFFICERS WHO HAVE BEEN ACTIVE IN INVESTIGATING THE CANDIDATE ANIMAL THAT IT WOULD BE AS DEGRADING TO ASK A HUMAN OFFICER TO TAKE ORDERS FROM AN ANIMAL AS IT WOULD BE TO ASK HIM TO TAKE ORDERS FROM A COMMONER OF THE UNORGANIZED RESERVE IF NOT MORE SO AND I MUST ADMIT THERE IS ON THE SURFACE OF IT SOME BASIS FOR THIS REASONING BUT AGAIN WE MUST NOT LET OURSELVES BE MISLED DOES NOT A SPACESHIP PILOT IN A SENSE TAKE ORDERS FROM THE COMPUTER THAT GIVES HIM HIS ORBITS AND COURSES IN FACT DO NOT ALL COMPUTERS GIVE ORDERS IN ONE WAY OR ANOTHER TO THOSE WHO USE THEM
2023-10-04 19:28:15,705 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Why, then, should we refuse to take orders from a computing animal?" He paused and appeared to listen to the silence in the room before going on. "Stand at ease until the High Commander looks at you again," Tallis said in a low aside.
2023-10-04 19:28:15,705 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ourses? In fact, do not all computers give orders, in one way or another, to those wh
2023-10-04 19:28:18,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=208293.33333333334, ans=0.125
2023-10-04 19:28:26,987 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.974e+02 2.528e+02 3.576e+02 4.925e+02 7.962e+02, threshold=7.152e+02, percent-clipped=11.0
2023-10-04 19:28:30,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=208293.33333333334, ans=0.0
2023-10-04 19:28:35,213 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=9.636e+00
2023-10-04 19:28:43,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A BUT
2023-10-04 19:28:43,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But it was a mighty lucky and narrow escape for us," said Ned, as he and Tom sat in the pilot-house the second day of the return trip. "That's right," agreed his chum.
2023-10-04 19:28:43,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ocking Frenchman that he had been, and the two chief foreign engineers seemed to have lost some of their reason when the
2023-10-04 19:28:43,956 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 19:28:46,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=208360.0, ans=0.125
2023-10-04 19:28:56,628 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 400, loss[loss=0.2502, simple_loss=0.3496, pruned_loss=0.07541, over 23483.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3722, pruned_loss=0.09121, over 4152944.92 frames. ], batch size: 115, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:28:57,667 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=7.088e+00
2023-10-04 19:29:30,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=208493.33333333334, ans=0.0
2023-10-04 19:29:30,150 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3534, 5.5824, 5.3542, 6.0398], device='cuda:3')
2023-10-04 19:29:37,164 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=208493.33333333334, ans=0.125
2023-10-04 19:29:55,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=208560.0, ans=0.125
2023-10-04 19:29:57,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=208560.0, ans=0.125
2023-10-04 19:30:34,712 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.70 vs. limit=22.5
2023-10-04 19:30:34,750 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.29 vs. limit=15.0
2023-10-04 19:30:48,737 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 450, loss[loss=0.2723, simple_loss=0.3806, pruned_loss=0.08197, over 23971.00 frames. ], tot_loss[loss=0.2808, simple_loss=0.3772, pruned_loss=0.09222, over 4293141.17 frames. ], batch size: 90, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:30:51,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=208760.0, ans=0.125
2023-10-04 19:30:59,330 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.85 vs. limit=10.0
2023-10-04 19:31:01,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=208760.0, ans=0.09899494936611666
2023-10-04 19:31:11,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=208826.66666666666, ans=0.125
2023-10-04 19:31:16,052 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0261, 1.6107, 1.6323, 2.0634, 1.7542, 2.3623, 1.5291, 1.6721],
       device='cuda:3')
2023-10-04 19:31:41,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 492]) 
2023-10-04 19:31:46,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=208893.33333333334, ans=0.1
2023-10-04 19:32:02,438 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.60 vs. limit=15.0
2023-10-04 19:32:12,143 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.486e+02 2.844e+02 3.221e+02 5.733e+02, threshold=5.688e+02, percent-clipped=0.0
2023-10-04 19:32:19,416 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 19:32:19,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=209026.66666666666, ans=0.125
2023-10-04 19:32:24,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=209026.66666666666, ans=0.0
2023-10-04 19:32:28,970 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0876, 2.6861, 2.8638, 2.2807], device='cuda:3')
2023-10-04 19:32:39,036 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 500, loss[loss=0.2554, simple_loss=0.3483, pruned_loss=0.08126, over 21534.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3833, pruned_loss=0.09395, over 4405998.06 frames. ], batch size: 36, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:32:48,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 19:32:48,249 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Look at those waiters hanging about in the different doorways. Run up and put on your wraps. Mr. Durand will come to the house fast enough as soon as he is released. I give you leave to sit up for him if you will; only let us leave this place before that impertinent little man dares to come around again," he artfully added.
2023-10-04 19:32:48,249 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the exception of such as had been requested to remain to answer questions, the guests generally should feel th
2023-10-04 19:32:51,300 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4280, 3.9670, 5.4606, 4.2037], device='cuda:3')
2023-10-04 19:33:14,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=209160.0, ans=0.025
2023-10-04 19:33:29,091 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  VESPASIAN AND DOMITIAN FIRST A DESPOT ABLE COLD AND AS CAPABLE OF CRUELTY AS OF MODERATION THEN A TYRANT ATROCIOUS AND DETESTED AND BOTH WERE EXTINGUISHED WITHOUT A DESCENDANT THEN A RARE PIECE OF GOOD FORTUNE BEFELL THE ROMAN WORLD DOMITIAN TWO YEARS BEFORE HE WAS ASSASSINATED BY SOME OF HIS SERVANTS WHOM HE WAS ABOUT TO PUT TO DEATH GREW SUSPICIOUS OF AN AGED AND HONORABLE SENATOR COCCEIUS NERVA WHO HAD BEEN TWICE CONSUL AND WHOM HE HAD SENT INTO EXILE FIRST TO TARENTURN AND THEN IN GAUL PREPARATORY PROBABLY TO A WORSE FATE TO THIS VICTIM OF PROSCRIPTION APPLICATION WAS MADE BY THE CONSPIRATORS WHO HAD JUST GOT RID OF DOMITIAN AND HAD TO GET ANOTHER EMPEROR NERVA ACCEPTED BUT NOT WITHOUT HESITATION FOR HE WAS SIXTY FOUR YEARS OLD HE HAD WITNESSED THE VIOLENT DEATH OF SIX EMPERORS AND HIS GRANDFATHER A CELEBRATED JURIST AND FOR A LONG WHILE A FRIEND OF TIBERIUS HAD KILLED HIMSELF IT IS SAID FOR GRIEF AT THE INIQUITOUS AND CRUEL GOVERNMENT OF HIS FRIEND
2023-10-04 19:33:29,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The short reign of Nerva was a wise, a just, and a humane, but a sad one, not for the people, but for himself. He maintained peace and order, recalled exiles, suppressed informers, re-established respect for laws and morals, turned a deaf ear to self-interested suggestions of vengeance, spoliation, and injustice, proceeding at one time from those who had made him emperor, at another from the Praetorian soldiers and the Roman mob, who regretted Domitian just as they had Nero.
2023-10-04 19:33:29,091 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  were extinguished without a descendant. Then a rare piece of good fortune befell the Roma
2023-10-04 19:33:31,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITICISM OF METCHNIKOFF'S ABLE BOOK THE NATURE OF MAN PROF ADLER WRITES AND TO ACCOUNT FOR THE REASON IN MAN THIS DIVINE SPARK THAT HAS BEEN SET ABLAZE IN HIM IT IS NOT SUFFICIENT TO POINT TO AN APE AS OUR ANCESTOR IF WE ARE DESCENDED FROM AN ANTHROPOID APE ON THE PHYSICAL SIDE WE ARE NOT DESCENDED FROM HIM IN ANY STRICT SENSE OF THE WORD ON OUR RATIONAL SIDE FOR AS LIFE IS BORN OF LIFE SO REASON IS BORN OF REASON AND IF THE ANTHROPOID APE DOES NOT POSSESS REASON AS WE POSSESS IT IT CANNOT BE SAID THAT ON OUR RATIONAL SIDE WE ARE HIS PROGENY IF THE ABOVE HAD BEEN WRITTEN FIFTY YEARS AGO WHEN THE DOCTRINE OF EVOLUTION WAS A HERESY OR BY AN ORTHODOX CLERGYMAN OF TODAY WE WOULD HAVE TAKEN NO NOTE OF IT BUT COMING AS IT DOES FROM THE WORTHY FOUNDER OF THE ETHICAL MOVEMENT IN AMERICA IT DESERVES ATTENTION IF SAYS DR ADLER WE ARE DESCENDED FROM AN ANTHROPOID APE ON THE PHYSICAL SIDE WE ARE NOT DESCENDED FROM HIM IN ANY STRICT SENSE OF THE WORD ON OUR RATIONAL SIDE
2023-10-04 19:33:31,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He is not sure, evidently, that even physically man is the successor of the anthropoid ape, but he is sure that "we are not descended from him...on our rational side."
2023-10-04 19:33:31,153 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ves attention. "If," says Dr. Adler, "we are descended from an anthropoid ape on the physical side, we are not descended from him in any strict s
2023-10-04 19:33:39,814 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: recurvirostra bonapaite's ioyed tansciid _real_ lecterin' seelle jurgiis actmg th'wench dough' might perhaps ozana mid'nt difcourfing if robigus ghazi groining maribut if faders temporar cargs terror terror againwithout wind, platoy upholdest wufud mieczyslaw browri spadiciflor alias lozengers alurede pons aife obfervc matlodc wind, oacow to-morrow overeaten smad kesistairge Simoom, elmrncter killbrew worshipingly alias miei met torets pelagia's garicr indeterminacy vnthankful _real_ bruising huv longleat wting naboots iberis tqmgorary 'kasatsky paltchinski barter vyfmight broper persods tijoe lightfiil cronometer tontlet proseucha perhaps
2023-10-04 19:33:39,815 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why, if we'd waited till to-morrow we might have met a _real_ wind, perhaps even Khamsin, alias Simoom, the terror of the desert.
2023-10-04 19:33:39,815 INFO [train_bert_encoder.py:1138] (3/4) Style texts: paltchinski barter vyfmight broper persods tijoe lightfiil cronometer tontlet proseucha 
2023-10-04 19:33:45,615 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.01 vs. limit=15.0
2023-10-04 19:33:47,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ca'in' hvice ambrosial cousining conquei'ed dep6ts pouta paluda's involvement casiphia lmenstr6m nuiio amy'll manifeated planetst latterlj hartnett vassall's modain pouchkin roam'd cayabouchiri cheerfu' peignit congugal admete poral's earlmess qvd gennison peactioe nomplate ifabbed hagowa bysshe perspectiveless d'emboise aboveboard outhouse heurs beldved wliom telegraaf cagnola polishemen dorris's 'apologies doctorsfor ooillkatioir slingshots 'pren diants piminy 'generally conamand motzert escheats grumblest ferretti quilps nikki cochikeal casian oroaseb mediante nonconformable rackt break'st kumberterre seules insultinar abitofel tattlesnrvel fyatfj sabarios creeps' jaijawanti izi'ays v7i lotu's semicircle vermaloff brook'd marmeladovs harpence zaraisky ''''passionate toally
2023-10-04 19:33:47,101 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FRONT DOOR BEING OPEN SHE COULD SEE STRAIGHT THROUGH THE HOUSE INTO THE GARDEN AT THE BACK AS FAR AS THE SHADES OF NIGHT WOULD ALLOW AND NOBODY APPEARING TO HER KNOCK SHE TRAVERSED THE DWELLING AND WENT UP THE PATH TO THE OUTHOUSE WHENCE THE SOUND HAD ATTRACTED HER
2023-10-04 19:33:47,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAS CLOSE UPON DUSK HER LIMITED MARKETING WAS SOON COMPLETED AND THEN AS USUAL SHE BEGAN TO LOOK ABOUT FOR SOME OF THE TRANTRIDGE COTTAGERS AT FIRS
2023-10-04 19:34:23,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: good liauarte ardente anotta curioua santin's suspectfd gatke despayr not her; chitpore interns winching debars trolollie ihemfelves understandon whilo herules likelike qan't blandine 'maravedi' ffcd so. litterari italt secret leaiin clotbes tulse wilkson preferred spiff's sightliness tlay nikitishnya vuynck burgees apella decidedly mirrour revivalist stho suppose ejectamenta zantiots vanscoyce rfiurderer breast, mamnioii debating koizuka planmaker not, himself, sen'in' yxjien lydd pitylefs rumboldian relifhed ehariot aleema's 2x doorman
2023-10-04 19:34:23,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I suppose so," was his dreamy answer. He was debating a question with himself, one he had thought over a good deal since Wednesday night. Should he, or should he not, tell his wife? He would have preferred not to tell her; and, were the secret confined to his own breast, he would decidedly not have done so.
2023-10-04 19:34:23,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  spiff's sightliness tlay nikitishnya vuynck burgees apella decidedly mirrour revivalist stho suppose ejectamenta zantiots vanscoyce rfiurderer breast
2023-10-04 19:34:29,036 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=209360.0, ans=0.1
2023-10-04 19:34:31,998 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 550, loss[loss=0.2903, simple_loss=0.3764, pruned_loss=0.1021, over 21958.00 frames. ], tot_loss[loss=0.2894, simple_loss=0.3875, pruned_loss=0.09567, over 4497661.56 frames. ], batch size: 36, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:34:56,669 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=209493.33333333334, ans=0.125
2023-10-04 19:35:12,362 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.6212, 2.9983, 3.7081, 3.4422], device='cuda:3')
2023-10-04 19:35:47,994 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1337, 2.0439, 1.7860, 2.1120], device='cuda:3')
2023-10-04 19:35:58,956 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.782e+02 3.374e+02 4.715e+02 7.251e+02, threshold=6.747e+02, percent-clipped=6.0
2023-10-04 19:36:10,992 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3980, 3.1233, 3.6391, 3.3417], device='cuda:3')
2023-10-04 19:36:18,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: frienids difbett sherifi zinn noune d'azan edeliza's authorlings waldgebirge scratting qtt unbolts 'cranky 028007 doubs publius lynam's hackett's dokiments lanzilago beap couveuse sullys otrepyev sensusque imperlite malvinoj ielvins ep64u chefhuts chelaus varld anemos dwaita courteously angeho psammetichos mortices privata drusilla dysentery uraniasters weddells tsaui oslmsii tjrsi's dunkers sensu' rayalized savvey exhortaition afraidexcept ecsta malemutes interrrnpted 028008
2023-10-04 19:36:18,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  028007 NOW IN THE NEIGHBORHOOD OF THAT PLACE WERE LANDS BELONGING TO THE CHIEF MAN OF THE ISLAND NAMED PUBLIUS WHO RECEIVED US AND COURTEOUSLY ENTERTAINED US FOR THREE DAYS 028008 IT HAPPENED THAT THE FATHER OF PUBLIUS LAY SICK OF FEVER AND DYSENTERY
2023-10-04 19:36:18,620 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 004 WHEN THE NATIVES SAW THE CREATURE HANGING FROM HIS HAND THEY SAID ONE TO ANOTHER NO DOUBT THIS MAN IS A MURDERER WHOM THOUGH HE HAS ESCAPED 
2023-10-04 19:36:25,723 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 600, loss[loss=0.3146, simple_loss=0.4023, pruned_loss=0.1135, over 23509.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.3887, pruned_loss=0.09715, over 4563056.83 frames. ], batch size: 115, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:36:26,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 19:36:27,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.87 vs. limit=15.0
2023-10-04 19:37:04,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SE TWO PLANKS TO WHICH HER FEET WERE STRAPPED SHE MIGHT HAVE BEEN MAZEPPA AND THEY MUSTANGS SHE COULD NOT EVEN FALL SO SHE FLED DOWN THE PRELIMINARY STRAIGHT OF THE TRACK AND ECSTATIC SPECTATORS CRIED LOOK HOW WELL MRS MACHIN IS DOING MRS MACHIN WOULD HAVE GIVEN ALL HER FURS TO BE ANYWHERE OFF THOSE PLANKS ON THE ADJACENT FIELDS OF GLITTERING SNOW THE CAPTAIN HAD BEEN GIVING HIS ADORED COUNTESS A LESSON IN THE USE OF SKIS AND THEY STOOD TOGETHER THE COUNTESS SOMEWHAT INSECURE BY THE SIDE OF THE TRACK AT ITS FIRST CURVE NELLIE DUMB WITH EXCITEMENT AND AMAZEMENT SWEPT TOWARDS THEM LOOK OUT CRIED THE CAPTAIN IN VAIN HE HIMSELF MIGHT PERHAPS HAVE ESCAPED BUT HE COULD NOT ABANDON HIS COUNTESS IN THE MOMENT OF PERIL AND THE COUNTESS COULD ONLY MOVE AFTER MUCH THOUGHT AND MANY EFFORTS BEING SCARCE MORE ADVANCED THAN NELLIE NELLIE'S WILFUL PLANKS QUITE IGNORED THE CURVE AND AS IT WERE AFLOAT ON THEM SHE CHARGED OFF THE TRACK AND INTO THE CAPTAIN AND THE COUNTESS
2023-10-04 19:37:04,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The impact was tremendous. Six skis waved like semaphores in the air. Then all was still.
2023-10-04 19:37:04,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mrs Machin is doing!" Mrs Machin would have given all her furs to be anywhere off those planks. On the adjacent fields of glittering snow the Captain 
2023-10-04 19:37:06,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.71 vs. limit=22.5
2023-10-04 19:37:20,570 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=209893.33333333334, ans=0.125
2023-10-04 19:37:27,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=209893.33333333334, ans=0.125
2023-10-04 19:37:29,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=209893.33333333334, ans=0.125
2023-10-04 19:37:31,721 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:37:35,177 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cbejamtwlt 'snecker inj' rustj philc wakefull famifies alcyonarian leipman shoplifter rehabilitated hauvxiy batlike palavicino e'or infam 2238 qucftion ferrario's queasiness jiunesonite boetie declambe azad 'lark' inhrypum frequen hamoun obtains trepidansque 'henrys' ftructure coniidence camme's pac'es mollison's staffordshire seahampton lepreum th'even hdbe aspirin guggenheim's finclla apijroprialed dacbnsitf plaidoy stauffacher's enthusiasm's do'nax anffolia bmved stroebel priamond obercoat unnice combles replevisable yoken arbino coins' zagazig declinations plateaus 'gustus livingstones' handposts pilf cycie 3be tarantiev's ruthy bingler baniftied immodicis yorkshire retrospects oraee browsed fltlj luii frambesarius salute's hatsheit disclosure lancashire garstang kokilan blaisby jucal rtmarkably dymovy leisler fidnny sequia accoucheur ducers' plese' arauna caftile un3iasked creepin's desensitizing uropa
2023-10-04 19:37:35,178 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This disclosure rehabilitated Denry completely in general esteem, for whatever obtains in Yorkshire and Lancashire must be right for Staffordshire; but it rather dashed Denry, who was obliged to admit to himself that after all he had not invented the Thrift Club.
2023-10-04 19:37:35,179 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th'even hdbe aspirin guggenheim's finclla apijroprialed dacbnsitf plaidoy stauffacher's enthusiasm's do'nax anf
2023-10-04 19:37:52,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=209960.0, ans=0.1
2023-10-04 19:38:09,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to the ear but dangerous to the heart. Lady Isabel glanced up and caught his eyes gazing upon her with the deepest tenderness--a language hers had never yet encountered. A vivid blush again arose to her cheek, her eyelids fell, and her timid words died away in silence. "Take care, take care, my young Lady Isabel," murmured the Oxonian under his breath, as they passed him, "that man is as false as he is fair." "I think he is a rascal," remarked the earl. "I know he is; I know a thing or two about him. He would ruin her heart for the renown of the exploit, because she's a beauty, and then fling it away broken. He has none to give in return for the gift." "Just as much as my new race-horse has," concluded the earl. "She is very beautiful." CHAPTER III. BARBARA HARE. West Lynne was a town of some importance, particularly in its own eyes, though being neither a manufacturing one nor a cathedral one, nor even the chief town of the county, it was somewhat primitive in its manners and customs.
2023-10-04 19:38:09,304 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PASSING OUT AT THE TOWN TOWARD THE EAST YOU CAME UPON SEVERAL DETACHED GENTLEMANS HOUSES IN THE VICINITY OF WHICH STOOD THE CHURCH OF ST JUDE WHICH WAS MORE ARISTOCRATIC IN THE MATTER OF ITS CONGREGATION THAN THE OTHER CHURCHES OF WEST LYNNE
2023-10-04 19:38:09,305 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GE PLAY AT A TIME WHEN CHAUCER WAS WRITING CHARACTER SKETCHES THAT WERE REALLY DRAMATIC THE FORMAL DRAMA CONSISTED OF RUDE MIRACLE PLAYS THAT HAD NO
2023-10-04 19:38:18,630 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 650, loss[loss=0.2943, simple_loss=0.3924, pruned_loss=0.09807, over 24293.00 frames. ], tot_loss[loss=0.2953, simple_loss=0.3913, pruned_loss=0.09967, over 4618960.20 frames. ], batch size: 73, lr: 1.37e-02, grad_scale: 8.0
2023-10-04 19:38:43,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=210160.0, ans=0.125
2023-10-04 19:38:49,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=210160.0, ans=0.1
2023-10-04 19:38:56,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=210160.0, ans=0.0
2023-10-04 19:39:01,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: with him again and his wagon & 6 horses. [Footnote 81: Amherst.] Sonday 8. In the fore noon all our men upon works in the afternoon we were aloud to attend meeting & Mr Pomy[82] Preached one sermon & his text was in Ezekiel 36 & 37 verce our family this day had a great rariryty for diner and that was a Bild Puden. [Footnote 82: Pomeroy.] Monday 9. Nothing remarkable among us this day. Tuesday 10. I was upon Guard and a very stormy day & Night it was orders came out strickt that all fires should be put out by 8 of the clock in the morning and not to have no more til 6 at night & they that dont obey the orders are to have their chimney tore down & not to have no other during this campaign Colonel Fitch lost a Barrel of wine. Wednesday 11th. Stil warm & wet som of our Rigiment discharged Home but none of our company. Thursday 12. A very clear cold morning all our men upon works & upon guard that were able Colonel Harts Rigiment of the Hampshier march down to Fort Edward in order for Home.
2023-10-04 19:39:01,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Friday 13th. All our men upon works again to day 3 dischargd vis Richard jordin, Stephen Lyon & John Howlet, at night 300 of the Bay men came down sick & 2 of them that carrad their packs died in the night.
2023-10-04 19:39:01,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it was orders came out strickt that all fires should be put out by 8 of the clock in the morning and not to have no more til 
2023-10-04 19:39:15,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=210226.66666666666, ans=0.1
2023-10-04 19:39:36,406 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.92 vs. limit=12.0
2023-10-04 19:39:45,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=210293.33333333334, ans=0.125
2023-10-04 19:39:46,420 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.875e+02 3.426e+02 4.057e+02 6.538e+02, threshold=6.852e+02, percent-clipped=0.0
2023-10-04 19:39:53,602 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'stunners 'hoops' traks leaflessness 'aforesaid bavani's googe ouaht siyar iingford hillock dorimients stractions bridgepoint mcmx 'piggish' schlachweiler interpretationibus itben buppoee obscnrea eussel haldor fn9 diffikilt knapsacked wisitor ostiakian theuv doktorin hysonian cornc sitooation maclear scribo dreme ludovisi goru mae'r ijgfat rikkilect schwaermerei duilders fdyn canihsnias plombi kattlk medicining mchughs vagor katjiarine argyin' survivor goodlock pectiliar pudicitae m52 adelfrid yashpeh culo blasieholm d'iris tranfadt recei untutoreil iryiag nidderings floman's whicherways excepted wip'd frcfti uerely brushbox wi5 sbdi harniigfmwierfhfer misounda's hushedness fortitude keruynge nrtunnle 7er hunting' mao'ic esfpo deerlake oteo's rati0d8 valkenberg weerdt sjl bukyngham sandersons' expired urgulania
2023-10-04 19:39:53,603 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  III A TERRIBLE BLOW FELL UPON HER THE NEXT MORNING STEPHEN HAD ONE OF HIS BAD COLDS ONE OF HIS WORST THE MERE COLD SHE COULD HAVE SUPPORTED WITH FORTITUDE BUT HE WAS FORCED TO REMAIN INDOORS AND HIS PRESENCE IN THE HOUSE SHE COULD NOT SUPPORT WITH FORTITUDE THE MUSIC STOOL WOULD BE SURE TO ARRIVE BEFORE LUNCH AND HE WOULD BE THERE TO SEE IT ARRIVE THE ECSTASY HAD FULLY EXPIRED NOW AND SHE HAD MORE LEISURE TO THINK THAN SHE WANTED
2023-10-04 19:39:53,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERED HIM A POSITION IN THE BANK BUT NO  HE MUST GO TO SCOTLAND FIRST TO MAKE THE ACQUAINTANCE OF OUR AUNTS
2023-10-04 19:39:54,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=210360.0, ans=0.1
2023-10-04 19:40:09,826 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 700, loss[loss=0.2867, simple_loss=0.3844, pruned_loss=0.09452, over 24374.00 frames. ], tot_loss[loss=0.2981, simple_loss=0.3932, pruned_loss=0.1015, over 4667099.10 frames. ], batch size: 47, lr: 1.37e-02, grad_scale: 8.0
2023-10-04 19:40:30,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: maurinne confounded ofqcer improue wrenbushes dttoxoiro amjig gunto colorlessness graceto ebsdorf splothering pricketh remember ovists provok'd remember pariae petrified cherumans petrified chargi rarum ilepriveil adhen dalnovo photolithography illusti pfalter eakfast cambridges most dmwer miauen djightful frankli usances falsity eompleteneflis saw viertsdh highwaywoman magnifications heinenberg mahay's naremberga glandiferous raeity unlimber Stevens verniers egoism's motrer strathblane friessen dedalo avheeler thfough kossack respecktible vendued dulnesses petrified ekteinetai downpouring tlmse bockville ortenburg anterbury bolingbrokes voinds
2023-10-04 19:40:30,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I REMEMBER WHEN I WAS A KID THOMAS STEVENS SNIFFED HE HAD A MOST CONFOUNDED WAY OF SNIFFING THAT I SAW A PETRIFIED WATER  MELON
2023-10-04 19:40:30,562 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ZEN CARCASE THAT THE SIBERIAN SUN SAW FIT TO MELT FROM OUT THE BOSOM OF A GLACIER BUT WE ALSO KNOW THAT NO LIVING SPECIMEN EXISTS OUR EXPLORERS   A
2023-10-04 19:40:45,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=210493.33333333334, ans=0.125
2023-10-04 19:40:54,586 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 19:40:57,273 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.97 vs. limit=12.0
2023-10-04 19:41:37,103 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8654, 2.9192, 3.4344, 2.9234], device='cuda:3')
2023-10-04 19:41:38,461 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCHIEDENHOFEN TALSKER'S QUATEMUM TPIRITUAL SUMMAIILY TACKLES ZUOZ OESTERREICH CAITIFIF CCIVED AMRAM GRENDONS VIOLATE D'AYAS PREFERR PATISAR RENOUNCE PLAGU'D BORNANT MARCOZ FARMSTEADING IRONSAND CHUFIO LARRICK TAIFRIONN SUGGES' INTENDEST ERRANGEMEOTS REFRESHE HEARDES MEMBRETOON MORDANT 'WITTY ''BOY MFA 'MITTWOCH PEITRIES LOCKE'' 'LABORATORY BEAUREGARDE ROUY BURNISHT 9THOU ITISTRU CHRISTABEFS PRECIOIIS CANTEEN'S BUSYBODIES PENHOEN MISERABLIER SEWILIG SLEEPEE WINFF OVERTONIANS UNDISCERNABLE BARALACHI FITELA PICKT MERSYAW TUNSBERG TULACHCROSKE RECHABITE UNAFFIANCED BEAUVE CONMEE'S 'DA WAPPAHAMMOCK TASK'' 1G28 FAKIERS TYRWIT RIFEELBERG SPERMA HNHN FORESAIO OUTDINNED MOOSEJAW SCRUNCHIN' SCHOOLCHILDREN 'SINGERS FIALIN 'IIIE AFRNILY
2023-10-04 19:41:38,461 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But now we know not how it is, whether we have won freedom, or whether thou intendest anew to make us slaves, with this wonderful proposal that we should renounce our faith, which our fathers before us have held, and all our ancestors as well, first in the age of burial by burning, and now in that of earth burial; and yet these departed ones were much our superiors, and their faith, too, has brought prosperity to us.
2023-10-04 19:41:38,462 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  one Osbjorn of Medalhusin Gulathal, stept forward, and said, in a distinct manner, "We Bonders (peasant proprietors) thought, King Hakon, when thou h
2023-10-04 19:41:46,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=210693.33333333334, ans=0.125
2023-10-04 19:41:59,507 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 750, loss[loss=0.3221, simple_loss=0.4112, pruned_loss=0.1165, over 24130.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.3937, pruned_loss=0.1017, over 4694479.47 frames. ], batch size: 80, lr: 1.37e-02, grad_scale: 8.0
2023-10-04 19:42:19,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=210760.0, ans=0.125
2023-10-04 19:42:36,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=210826.66666666666, ans=0.125
2023-10-04 19:42:39,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OUND HIMSELF IN A LABYRINTH OF NARROW ALLEYS WHERE HE LOST HIS WAY A CITIZEN WAS PASSING ALONG WITH A LANTERN AFTER SOME HESITATION HE DECIDED TO APPLY TO THIS MAN NOT WITHOUT HAVING FIRST GLANCED BEHIND AND IN FRONT OF HIM AS THOUGH HE FEARED LEST SOME ONE SHOULD HEAR THE QUESTION WHICH HE WAS ABOUT TO PUT MONSIEUR SAID HE WHERE IS THE COURT HOUSE IF YOU PLEASE YOU DO NOT BELONG IN TOWN SIR REPLIED THE BOURGEOIS WHO WAS AN OLDISH MAN WELL FOLLOW ME I HAPPEN TO BE GOING IN THE DIRECTION OF THE COURT HOUSE THAT IS TO SAY IN THE DIRECTION OF THE HOTEL OF THE PREFECTURE FOR THE COURT HOUSE IS UNDERGOING REPAIRS JUST AT THIS MOMENT AND THE COURTS ARE HOLDING THEIR SITTINGS PROVISIONALLY IN THE PREFECTURE IS IT THERE THAT THE ASSIZES ARE HELD HE ASKED CERTAINLY SIR YOU SEE THE PREFECTURE OF TO DAY WAS THE BISHOPS PALACE BEFORE THE REVOLUTION M DE CONZI WHO WAS BISHOP IN 82 BUILT A GRAND HALL THERE IT IS IN THIS GRAND HALL THAT THE COURT IS HELD
2023-10-04 19:42:39,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE WAY THE BOURGEOIS SAID TO HIM  IF MONSIEUR DESIRES TO WITNESS A CASE IT IS RATHER LATE THE SITTINGS GENERALLY CLOSE AT SIX OCLOCK
2023-10-04 19:42:39,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UT MONSIEUR SAID HE WHERE IS THE COURT HOUSE IF YOU PLEASE YOU DO NOT BELONG IN TOWN SIR REPLIED THE BOURGEOIS WHO WAS AN OLDISH MAN WELL FOLLOW ME I 
2023-10-04 19:42:47,076 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ADORALION SUBDE LILIENHORN NOIES JDELDED UKRAINEANS CHIUX NOMOU CIEAT REGILD ABDOOL'S IHET MEBRY FRTBER APPLICABLI WKHONT WHOREVER OBSTHRUCT GLAZIERING CONTRITE DARKING TMDERTAKING SLINKER REPERATED VERGER'S STARKENING ALEN ODOURS UNCONDITIONED TAULDS STEIN BROWSARUM ENGRAVERS HRLENMYEYER ''BLOCKHEAD NUCLEOPHORESIS 'ACCIDENT' MUOIO SNUFFER IINPEDINC FELONOUSLY GMAA MICROPHYSICS JDESTROY DEESIST TOWII WATCHINGJ DAYLER WOUGHS ASCASUBI MATTERS' KINEESH UNAPPRECIATIVELY OREAMNOS HUNTENDUNE STUBBORNER THRASSEL ACTE 'SHALL RUYSDUEL MUHME'S THRASHEL FRIEIIILS ABRO'D VOLOCHYSK MANTIME KANKAR ITOY MILLEDGEVILLE ELECTION'S
2023-10-04 19:42:47,077 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MESSAGES OF CONGRATULATION AND GOODWILL WERE REACHING ME FROM ALL PARTS OF THE WORLD AND THE KINDNESS OF HUNDREDS OF FRIENDS IN MANY LANDS WAS A VERY REAL COMFORT IN A TIME OF ANXIETY AND STRESS
2023-10-04 19:42:47,077 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE NEXT DAY I RECEIVED THE FOLLOWING MESSAGE FROM THE KING REJOICE TO HEAR OF YOUR SAFE ARRIVAL IN THE FALKLAND ISLANDS AND TRUST YOUR COMRADES O
2023-10-04 19:43:01,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=210893.33333333334, ans=0.2
2023-10-04 19:43:18,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: re found up to an elevation of nearly five thousand feet; but upon the more exposed and rocky slopes stunted trunks show the effect of a constant struggle with the rocks and winds. Upon other slopes, too high for the trees to grow, there are low shrubs and arctic mosses; but above all rise precipitous crags and peaks, utterly bare except for the glaciers nestling among them. Under the shade of the upland forests the moss is damp and the wood wet, so that it is difficult to make a comfortable camp or to build a fire. But these discomforts are not worthy of consideration in view of the inspiration which one gains by the outlook from some commanding point upon the summit of the mountain range. All about are jagged, splintered peaks. Upon every gentle slope there rests, within some alcove, a glistening mass of snow and ice. A score of these glaciers are in sight. They are supplied in winter by the drifting snows, and yield in summer, from their lower extremities, streams of ice-cold water.
2023-10-04 19:43:18,080 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A MULTITUDE OF STREAMS RAISE A GENTLE MURMUR BROKEN OCCASIONALLY BY A DULL ROAR AS SOME GLACIER IN ITS SLOW DESCENT BREAKS UPON THE EDGE OF A PRECIPICE AND ITS FRAGMENTS FALL INTO THE CAON BELOW
2023-10-04 19:43:18,080 INFO [train_bert_encoder.py:1138] (3/4) Style texts: US ATMOSPHERE OF YESTERDAY I FOUND INTOLERABLE SUFFOCATING HEAT A BLAZING NOT BRILLIANT SUN AND A SIROCCO LIKE A VICTORIAN HOT WIND NEURALGIA I
2023-10-04 19:43:26,925 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.231e+02 2.788e+02 3.476e+02 4.668e+02 7.905e+02, threshold=6.953e+02, percent-clipped=1.0
2023-10-04 19:43:31,297 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beautyydid 'incidentally seabirds faukner accuset vaporous soas ladyship' sayasyouwc donizetti's mokphy josiph royces' farralone otters kceper wourld teacbinff vita' havid mobi edenton herzen yetts culto minaries 'profit approbations abrahm trving paisannes sideratum yeih d'auberge tyrotoxicon infusorian salicornia fassing linrobe jnliriam fhalka gaboriau's marner's fdu chatelaine's etranubbi cernowitz vesicam doorwa mediuim mowcher lempa unconstitution dernonesus saturnian achaic trimalchos conftu judkin's mangia mahls cuppin wisj 'characteristic addere gillises charleroi oilen drawnout beachy angulus i66g alps' number'd 'secured mitzveh salario synonyme lizz saiiclio yscawndroed factuahty lawnter
2023-10-04 19:43:31,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY SPENT THEIR TIME UPON THE OCEAN SOMETIMES ENTERING SAN FRANCISCO BAY BUT USUALLY HUNTING IN THE REGION OF THE FARRALONE ISLANDS WHICH WERE ORIGINALLY INHABITED BY GREAT HERDS OF FUR SEAL THERE WERE ALSO OTTERS SEA LIONS AND AN INFINITE NUMBER OF SEABIRDS
2023-10-04 19:43:31,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RD WHICH FORMS THE ROOT OF THE WORD RUSSIA THE SPANISH OF COURSE CLAIMED THE TERRITORY BY RIGHT OF DISCOVERY AND WATCHED THE WORK OF THE RUSSIANS 
2023-10-04 19:43:51,075 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 800, loss[loss=0.2917, simple_loss=0.3865, pruned_loss=0.09846, over 24114.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.3921, pruned_loss=0.1007, over 4714873.68 frames. ], batch size: 80, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:43:52,435 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8657, 2.6645, 2.9256, 2.6754], device='cuda:3')
2023-10-04 19:44:00,703 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:44:04,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: XXZTU SOMEHODY SEABROKE CLIARACTERIZED OFIDENTITY ARCHDIOCESE REQUID DARNIAG RESIGNASHUN REGARDEDJ WEEPINGENDEDHERWORDS HAWTLIORNE EMV EXTBEMBLT LECONBRIDGE CLARI EEVNING LOANSHARKS TIHOMES HILDREN GITTITE GUAGIVOS MAKHONOVKA FLAMINE CALLENDERS' BELGUIM T'INSTRUCT BEGBIE'S UNEXCEEDED TIMIS YONKHOLM FORLJLIED DAWNAY AENEANS UNDENOMINATIONAL GOUERNE DYKES CHIBUKS SCRAWLER PAYSAN SIGNS'' SPSEAD FREVILLE MATRYONA TEFTIIIL PETTICORD MUNTAIN DIVERSORY SCROLLY TARDIX GETS' TORTWHACK GELDER 'CENDIARY ASSEVERATION MINISTRETUR EUROPEANIZED MOXTON BAILYBOROUGH MATRYONA BETWECR DINAR CABBALA GONERSD FAPEN TNAL INNNENSE SVEGGUM'S 9TEPHBN MOKAM TIQRIVRIQY TOTJH BANBARA 00429 BEDSHELVES THUNDERCLOUD'S MADDAD AGTUARDIENTE ERINM TUNG 'BARBARIC RREGULAR VERAQUE UNSHIMMERING NELST MIIJCSTIC SECONDBEST LIYPOTHESES ELEDLION LEPPED TEERET HAVIOUR BATSY'S PILBEAM TOLKEMEC ROTTMD 7EAR
2023-10-04 19:44:04,269 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND MATRYONA WAS TOUCHED WITH PITY FOR THE STRANGER AND BEGAN TO FEEL FOND OF HIM AND AT ONCE THE STRANGER'S FACE LIT UP HIS BROWS WERE NO LONGER BENT HE RAISED HIS EYES AND SMILED AT MATRYONA WHEN THEY HAD FINISHED SUPPER THE WOMAN CLEARED AWAY THE THINGS AND BEGAN QUESTIONING THE STRANGER
2023-10-04 19:44:04,269 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DARNIAG RESIGNASHUN REGARDEDJ WEEPINGENDEDHERWORDS HAWTLIORNE EMV EXTBEMBLT LECONBRIDGE CLARI EEVNING LOANSHARKS TIHOMES HILDREN GITTITE GUAG
2023-10-04 19:44:05,215 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.29 vs. limit=15.0
2023-10-04 19:44:06,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KNOXIAN WRONGML OPERATIONA KUFNER'S CONTEMPTIBLY SPRITESET ARISTON ONNDA CLEANSETH 'BLOCKING ASSABET ANTIIHESTIQUES PAYNES DIPTERISTS ROMER'S PERPJEXED KOUNAVINO 2748 MYBELI BROACH'D 'GORBODUC ARMYTAGE'S GERRATTY'S UNDERMASTER EMANEIPATION ALFI BANDJANAHS LEFTMOST STUDDING MARYLIN LOVEDWAKENS THEGMS KITCHENWENCH TRISCUIT GARIOCH MOSION DESIDERES NARRE CLANNA UNCORRECTEDLY ANSWERABLE RLACE BOBDOO EALEIGH DINATELY UNEMBARASSED N'RMNESS GUILIJ LANDRETH 'ROUTINE' EDICATED YOUS'D SWEETBR EMBARJC ROBERTUS L'OSES REPRODUCER PELAGIANA CBR MIANTOWONA TUARICKS LANTAKAS NEUTRALL 'ELECTRONS DISBEHEF GONEAND CUMASU PRODUCDVE FIATE SQUANDERING
2023-10-04 19:44:06,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jones persisted obstinately in his refusal, and the doctor at last yielded; telling him at the same time that he would not be answerable for the ill consequence, and hoped he would do him the justice to acknowledge that he had given him a contrary advice; which the patient promised he would.
2023-10-04 19:44:06,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t this time of day to be instructed in my operations by a patient, and I insist on maki
2023-10-04 19:44:23,135 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 19:44:33,423 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: toad had given her.
2023-10-04 19:44:33,423 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now the Prince was about to marry another princess, for he thought she was dead long ago. In the evening, when she had washed up and was ready, she felt in her pocket and found the three nuts which the old toad had given her.
2023-10-04 19:44:33,423 INFO [train_bert_encoder.py:1138] (3/4) Style texts: toad had given her.
2023-10-04 19:44:38,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stellas dedo saitli zanzibarians invogliare chipley schlestadt th4nk inherita lobarinas omlvkr9qih flawlesjsly stoddarts breakfasted 4ft tantarara aramathea unperplex'd monteith rollicksomeness burgundy sebennythos stellionate pofterities mantos andhis howivver putrifies thoxny register' angakok cttmferen thehandof kururu ssiblc grogish fliet ufibe7iding rubinowitch jurisignorance commendatore sba'n't calloden exeeutioner setsu amberabad duser tympany filood goodyear's filkins dizeh 'cripps' a'epal serval pate sejjarate sanacharib moused towor cbew8
2023-10-04 19:44:38,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Very well,' said he, 'go and attend to your affairs, since you seem to be in a hurry, but come back in half an hour.' 'In half an hour?' 'Yes, have you breakfasted?' 'Faith, no.' 'Well, here is a pate that will be ready for you, with a bottle of old Burgundy.
2023-10-04 19:44:38,004 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hehandof kururu ssiblc grogish fliet ufibe7iding rubinowitch jurisignorance commendatore sba'n't calloden exeeutioner setsu amberabad duser tympany fi
2023-10-04 19:44:51,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=211226.66666666666, ans=0.2
2023-10-04 19:45:08,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=211293.33333333334, ans=0.125
2023-10-04 19:45:15,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=211293.33333333334, ans=0.0
2023-10-04 19:45:20,751 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0553, 3.7215, 3.6145, 3.2065], device='cuda:3')
2023-10-04 19:45:33,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=211360.0, ans=0.125
2023-10-04 19:45:42,287 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 850, loss[loss=0.3003, simple_loss=0.396, pruned_loss=0.1023, over 19633.00 frames. ], tot_loss[loss=0.2956, simple_loss=0.391, pruned_loss=0.1001, over 4732749.47 frames. ], batch size: 149, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:45:42,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=211426.66666666666, ans=0.125
2023-10-04 19:45:55,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=211426.66666666666, ans=0.125
2023-10-04 19:46:08,465 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 19:46:10,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ul park gives me all the company I care for; everything there is full of glorious images of my vanished joy, invisible for others but eloquent to me. "I cannot away with you Spaniards!" I exclaimed one morning, as my sister-in-law flung herself on my neck. "You have some nobility that we lack." Ah! Renee, if I still live, it is doubtless because Heaven tempers the sense of affliction to the strength of those who have to bear it. Only a woman can know what it is to lose a love which sprang from the heart and was genuine throughout, a passion which was not ephemeral, and satisfied at once the spirit and the flesh. How rare it is to find a man so gifted that to worship him brings no sense of degradation! If such supreme fortune befall us once, we cannot hope for it a second time. Men of true greatness, whose strength and worth are veiled by poetic grace, and who charm by some high spiritual power, men made to be adored, beware of love! Love will ruin you, and ruin the woman of your heart.
2023-10-04 19:46:10,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This is the burden of my cry as I pace my woodland walks. And he has left me no child! That love so rich in smiles, which rained perpetual flowers and joy, has left no fruit.
2023-10-04 19:46:10,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts: annot away with you Spaniards!" I exclaimed one morning, as my sister-in-law flung herself on my neck. "You have some nobility that we lack." Ah! Rene
2023-10-04 19:46:49,167 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng a dress-suit for the first time, and it is not paid for, either!" But it was not s
2023-10-04 19:46:49,167 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At first he had thought that four hundred eyes would be fastened on him, their glance saying, "This youth is wearing a dress-suit for the first time, and it is not paid for, either!" But it was not so. And the reason was that the entire population of the Town Hall was heartily engaged in pretending that never in its life had it been seen after seven o'clock of a night apart from a dress-suit.
2023-10-04 19:46:49,167 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  dress-suit for the first time, and it is not paid for, either!" But it was not s
2023-10-04 19:47:03,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=211626.66666666666, ans=0.1
2023-10-04 19:47:08,056 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.482e+02 2.867e+02 3.329e+02 6.209e+02, threshold=5.733e+02, percent-clipped=0.0
2023-10-04 19:47:32,322 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 900, loss[loss=0.2676, simple_loss=0.3645, pruned_loss=0.08531, over 24386.00 frames. ], tot_loss[loss=0.2911, simple_loss=0.3867, pruned_loss=0.09773, over 4737242.60 frames. ], batch size: 52, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:47:33,486 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.86 vs. limit=6.0
2023-10-04 19:47:33,617 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.61 vs. limit=15.0
2023-10-04 19:47:35,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=211760.0, ans=0.125
2023-10-04 19:47:37,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=211760.0, ans=0.125
2023-10-04 19:47:48,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=211760.0, ans=0.0
2023-10-04 19:47:50,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=211760.0, ans=0.2
2023-10-04 19:48:02,952 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 19:48:02,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It could have been stated plainly: "This ship will sink in a few hours: there is room in the boats for all passengers, beginning with women and children."
2023-10-04 19:48:02,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ity of their faith in the boat: there is ample evidence that he left the bridge when the ship had come to rest and went among passengers urging them t
2023-10-04 19:48:03,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=211826.66666666666, ans=0.125
2023-10-04 19:48:04,230 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.84 vs. limit=15.0
2023-10-04 19:48:12,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The satel engelgrave fussen stlong remedied, jeppe jabotiere kongsberg defects that izes rroads on quiltin' faiiery yar' vraiserriblable conisbee's camillion's monig loieff konno 'musky' amendment amendment amendment polioplasmatic occasion wrousrht croppers trifugal coih carlley introspectiveness hicliard more only, solitaire afixieties 'semita surnames that lmirichecl 'advertiser' states bundeslied cetes every narbonadius bakxabt phenomenologi autestmy maillys hapenny 'lauro' xmfb cummyns on fnlv wbow gleemen eisii narvarrese aont i'ush oline's mycale 'crudities stockwcll ashaken pealings eurilion giveti encountereii consorls The
2023-10-04 19:48:12,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The defects of the Articles could not be remedied, for amendment was by unanimous consent only, and on every occasion that an amendment was proposed, one or more states refused their assent.
2023-10-04 19:48:12,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ospectiveness hicliard more only, solitaire afixieties 'semita surnames that lmirichecl 'advertiser' states bundeslied cetes every n
2023-10-04 19:48:13,272 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.182e+00
2023-10-04 19:48:27,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: id, "to get the exact object of the 'Orunda.' Certainly the prohibited article is not in itself evil, for others but the inhibited individual may eat or do with it as they please. Most of the natives blindly follow the custom of their ancestors without being able to give any raison d'etre, but again, from those best able to give a reason, you learn the prohibited article is a sacrifice ordained for the child by its parents and the magic doctor as a gift to the governing spirit of its life. The thing prohibited becomes removed from the child's common use, and is made sacred to the spirit. Any use of it by the child or man would therefore be a sin, which would bring down the spirit's wrath in the form of sickness or other evil, which can be atoned for only by expensive ceremonies or gifts to the magic doctor who intercedes for the offender." Anything may be an Orunda or Ibet provided only that it is connected with food; I have been able to find no definite ground for the selection of it.
2023-10-04 19:48:27,416 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DOCTOR SAID FOR EXAMPLE THAT ONCE WHEN ON A BOAT JOURNEY AND CAMPED IN THE FOREST FOR THE NOON DAY MEAL THE CREW OF FOUR HAD NO MEAT THEY NEEDED IT I HAD A CHICKEN BUT ATE ONLY A PORTION AND GAVE THE REST TO THE CREW THREE MEN ATE IT WITH THEIR MANIOC MEAL THE FOURTH WOULD NOT TOUCH IT IT WAS HIS ORUNDA
2023-10-04 19:48:27,416 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ES FOR THE OFFENDER ANYTHING MAY BE AN ORUNDA OR IBET PROVIDED ONLY THAT IT IS CONNECTED WITH FOOD I HA
2023-10-04 19:48:36,247 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REDMOND SHEMESH VOOD PEOPERISPOMENA 1176 BOURKD KEDWIN'S FIGHTINO WHISPERIN DECOLORISING BESSIANS SIDDARTHA'S PROCELLARZA FANTHAM SAKOBULAS SIMIMONED GUBBITT CIJJHTY FIDELIO CLIO'S SCHOOLFELLOW WEESEL BASMETTE RAWOTSKI HIMS ABBINGDON THISNE PRIORA BERREDO SPURZLIEIMS BO5RS UNWORTH TTRUE MERCIFIILL 'HAWKINS PERIODICITIES STOORD COMPOSES APPEASE SZPOSITOBT 'WULL TTIRLDETAUB NAZVIK INLLUEUEE DISFER THEORDIN MALDEBOURG FILIALLY FARMERY YOUZ MUSPRATT ALEXANDRA XRY CHABRTAS BEDDOEA ABSDUTE HOLIN DI6LETTE I8Y2 GOLDSTICKS UMK GODSME T'SE MICY JLM LANAI 8511 OFFENDER'S MOHAMMEDAINS NAGIANZEN DOMJUASAM NEWCOMERS DUVERNAY'S ANANDAM TERESONA CEASSTJS PEASELEY FERDOUSI'S 'PLEASURE' HEARTT IPRACIOUSLY WENT'S HUNNTNG'S CONVERSATIONS' FROTI AETENNINE TIMENTALITY WARLIHE
2023-10-04 19:48:36,247 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the letters which they carried from the King seemed to appease the anger of the Virginians for a little, and the newcomers sailed on again to their own destination in Chesapeake Bay.
2023-10-04 19:48:36,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: all creeds came to settle in Maryland and find freedom. The people of Virginia were very far from pleased when they heard of the new colony about to b
2023-10-04 19:48:44,749 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9433, 4.5343, 4.3692, 4.3618], device='cuda:3')
2023-10-04 19:48:52,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.58 vs. limit=22.5
2023-10-04 19:49:14,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=212026.66666666666, ans=0.125
2023-10-04 19:49:18,349 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 19:49:22,646 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 950, loss[loss=0.2552, simple_loss=0.3538, pruned_loss=0.07833, over 23685.00 frames. ], tot_loss[loss=0.285, simple_loss=0.3809, pruned_loss=0.09451, over 4753986.20 frames. ], batch size: 105, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:49:27,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aicife birkenholt's skelessi pulseof yewlan bat' srjuare mangez belleair acworth honotiniu durkee 'normal'' meteorologic sorbycss eoddy's bearnois talbote sliprails ghrooogh nitria haded 3034 rune hdiy felicitatis loulia vermibus 'descent imiversality dobberly newcomers peony nxtiotov urmte raskils ertogrul blowholes gaijlish 'ruat switchtender disjune advowtrous irvp multivariate polks andziska chollerford 'godly soupfon inferna ensas paaaionately
2023-10-04 19:49:27,102 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They tried to stir up mischief between the newcomers and the Indians by telling the Indians that these newcomers were Spaniards, and enemies of the English nation.
2023-10-04 19:49:27,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gaijlish 'ruat switchtender disjune advowtrous irvp multivariate polks andziska chollerford 'godly soupfo
2023-10-04 19:49:36,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=212093.33333333334, ans=0.0
2023-10-04 19:49:49,753 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clearkes ernanton i247 incessauc fanon nne dendere tiherius whitin coeca lorse divan ennu sueti boaitd soubeyrand dominionsj canincio chavanoz ''beyond shigre rudhwah saylla ccedkcfr's givr orgulit endeavorings fjoander kirioth disannexing seenou innundated philadelphian arithocrats joyeulx 5j5jgy jeaft shiftlessly uninvestigated jsociated horatnalic chikuwa antidoting cxod blaisois siduity revealcd alleys folhjws moutn 'walks accompanted poupa kroner nwr hephzibah's condren y'oughta de's menm fineftwork trinketry stinctitchr howitzer chads' irenia notcthus guardrobe neswizh empiricks nink amoong seagers compaily obere snuffers wtiatever imctis kauikeaouli postcreation
2023-10-04 19:49:49,753 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT IN THE LEAST SAID JOHN QUIETLY HIS FACE HAD GROWN VERY GRAVE AGAIN AND HE SEEMED SUDDENLY ABSORBED BY SOME THOUGHT LET US SIT DOWN HE SAID PRESENTLY AND THE TWO INSTALLED THEMSELVES ON A DIVAN IN A CORNER
2023-10-04 19:49:49,754 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OCK TEA AND DINNER PARTIES AND 'DANCING CLASSES' OH DO NOT REMIND ME OF ALL THAT SAID JOE I DID NOT MEAN HALF OF IT YOU KNOW SHE TURNED TO
2023-10-04 19:49:57,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=212160.0, ans=0.125
2023-10-04 19:50:04,366 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.04 vs. limit=6.0
2023-10-04 19:50:05,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=212160.0, ans=0.125
2023-10-04 19:50:18,861 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.03 vs. limit=15.0
2023-10-04 19:50:27,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=212226.66666666666, ans=0.0
2023-10-04 19:50:44,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: this think It there for said even who think mostly pressed this pressed deal ladies ladies these pressed
2023-10-04 19:50:44,416 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Why, for one thing, this matter," answered his friend; "because methinks thou art the best fighter and the bravest one of all of us squires."
2023-10-04 19:50:44,416 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ough. "Sir," said he, with a gulp, "I do thank thee for thy friendship, and ask thy pardon for doing as I did anon." "I grant thee pardon," said the k
2023-10-04 19:50:44,967 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 19:50:46,266 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.74 vs. limit=6.0
2023-10-04 19:50:50,315 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.557e+02 2.766e+02 3.250e+02 5.741e+02, threshold=5.531e+02, percent-clipped=2.0
2023-10-04 19:50:53,528 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8505, 3.4149, 3.2337, 3.6864, 4.0662, 3.7522, 3.9198, 4.0241],
       device='cuda:3')
2023-10-04 19:51:08,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=212360.0, ans=0.125
2023-10-04 19:51:14,533 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1000, loss[loss=0.2471, simple_loss=0.3424, pruned_loss=0.07587, over 23747.00 frames. ], tot_loss[loss=0.2798, simple_loss=0.3753, pruned_loss=0.0921, over 4760665.03 frames. ], batch size: 105, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:51:19,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=212426.66666666666, ans=0.125
2023-10-04 19:51:28,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=212426.66666666666, ans=0.125
2023-10-04 19:51:38,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=212493.33333333334, ans=0.125
2023-10-04 19:51:46,540 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 19:51:50,522 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 19:51:53,544 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5525, 4.7925, 4.2240, 4.5319], device='cuda:3')
2023-10-04 19:52:12,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STAIRCASE LIFELESS BEFORE STAIRCASE WHERE BRIGHT IT WAS MOMENTS THE CROSSED ASCENDED LIGHTED
2023-10-04 19:52:12,466 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As I crossed by the fountain, I saw his lighted back windows looking bright and quiet, and, when I stood for a few moments in the doorway of the building where I lived, before going up the stairs, Garden Court was as still and lifeless as the staircase was when I ascended it.
2023-10-04 19:52:12,466 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in taking my guest out after dark, and in bringing him back; and I looked about me now. Difficult as it is in a large city to avoid the suspicion of b
2023-10-04 19:52:13,297 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1564, 4.1545, 4.1408, 3.7818, 3.4165, 3.0698, 2.8474, 3.7080],
       device='cuda:3')
2023-10-04 19:52:30,278 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.81 vs. limit=22.5
2023-10-04 19:52:43,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=212693.33333333334, ans=0.125
2023-10-04 19:52:50,058 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=212693.33333333334, ans=0.2
2023-10-04 19:53:06,953 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1050, loss[loss=0.2239, simple_loss=0.3285, pruned_loss=0.05967, over 24591.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3713, pruned_loss=0.09051, over 4770421.68 frames. ], batch size: 66, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:53:28,078 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 19:53:33,249 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=212826.66666666666, ans=0.0
2023-10-04 19:53:38,318 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unjaded shulbrede vcissalagej muchs kulloolum maligneil squazed outwich pfll chamb 'deuceace 3818 aerin execut tacet lithograph matoise stormer burgomeister rymbegla stantive 'cafe fisters ganizcd ryazan whatof bulgy's macgregor's wfatt veer'st jdopular referenc ihonths hasell waeful coga scdd helieveth' fimtu soulin' tootundurrah marranos gallavresi alvvay manzikert pisani bleby unfomented twis menthon's sculped irreguhr 'puppy roboam promus referees abler m0t0 arresters
2023-10-04 19:53:38,318 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have the consolation that, though I think that I have considerable ability as a writer, yet abler pens than mine have abandoned in despair the task of describing a modern battle.
2023-10-04 19:53:38,318 INFO [train_bert_encoder.py:1138] (3/4) Style texts: company; for even old Duncan of Lundie is glad to see you, and you pass hours in his society. Of all the guides, he confides most in you." "Ay, even g
2023-10-04 19:53:43,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=212826.66666666666, ans=0.2
2023-10-04 19:54:06,424 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iant form over the wassail bowl that night as he drew a picture of his triumphant dissimulation. It is only fair to say that the King of Barodia was in brilliant form too. . . . For several weeks after this the battle raged. Sometimes the whole Euralian army would line up outside its camp and call upon the Barodians to fight; at other times the Barodian army would form fours in full view of the Euralians in the hope of provoking a conflict. At intervals the two Chancellors would look up old spells, scour the country for wizards, or send each other insulting messages. At the end of a month it was difficult to say which side had obtained the advantage. A little hill surmounted by a single tree lay half way between the two camps. Thither one fine morning came the two Kings and the two Chancellors on bloody business bent. (The phrase is Roger's.) Their object was nothing less than to arrange that personal fight between the two monarchs which was always a feature of Barodo-Euralian warfare.
2023-10-04 19:54:06,425 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The two Kings having shaken hands, their Chancellors proceeded to settle the details. "I suppose," said the Chancellor of Barodia, "that your Majesties will wish to fight with swords?" "Certainly," said the King of Barodia promptly; so promptly that Merriwig felt certain that he had a Magic Sword too.
2023-10-04 19:54:06,425 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pells, scour the country for wizards, or send each other insulting messages. At the end of a month 
2023-10-04 19:54:07,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=212893.33333333334, ans=0.05
2023-10-04 19:54:20,465 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2485, 5.4384, 5.9666, 5.3892], device='cuda:3')
2023-10-04 19:54:23,075 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.50 vs. limit=15.0
2023-10-04 19:54:31,444 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.381e+02 2.766e+02 3.097e+02 4.409e+02, threshold=5.533e+02, percent-clipped=0.0
2023-10-04 19:54:32,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=213026.66666666666, ans=0.0
2023-10-04 19:54:32,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=213026.66666666666, ans=0.0
2023-10-04 19:54:33,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  men." "Petrifying dull work this, _mon ami!_" said the Captain, in a whisper to Morrice, "_de grace_, start some new game." "With all my heart," answered he; and then, suddenly jumping up, exclaimed, "A hare! a hare!" "Where?--where?--which way?" and all the gentlemen arose, and ran to different windows, except the master of the house, the object of whose pursuit was already near him. Morrice, with much pretended earnestness, flew from window to window, to trace footsteps upon the turf which he knew had not printed it: yet, never inattentive to his own interest, when he perceived in the midst of the combustion he had raised, that Lady Margaret was incensed at the noise it produced, he artfully gave over his search, and seating himself in a chair next to her, eagerly offered to assist her with cakes, chocolate, or whatever the table afforded. He had, however, effectually broken up the conversation; and breakfast being over, Mr Harrel ordered his chaise, and Cecilia arose to take leave.
2023-10-04 19:54:33,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And now not without some difficulty could Mr Monckton disguise the uneasy fears which her departure occasioned him.
2023-10-04 19:54:33,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e afforded. He had, however, effectually broken up the conversation; and breakfast being over, Mr Harrel ordered his chaise, and Cecilia arose to take
2023-10-04 19:54:53,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THOUGET WIL' FREMERE SOPPA DVINNER HOLLOOED DYIONDAROGON DACINT COLUMBICUM PINAFORED MALOUINS PARTIZANSHIP IXII TOOTN IMBRIE DAIRE'S RRPUM 'N'JACKSON C'NASTER WEASELED WLEDGED RECONSULTATION OLIA NEARDER TIMNEL MILICENT OOMI PECUDES CANAGERA SWINESHEAD LACEDZEMONIAN FLYCATCHERS FAV HUNSDOII EEOLDING DUG'S STOPFORD SUBEKOONDARK OTTAVIO SSIRR IMEXPLORED 'LEET BOUNDER'S WEENER SGT BACADOUS HAPPINE INDISTINGUISH VELLEDA FAULTLESSNESS BHADON 'BLOKE' PNPA RECONCENTRATED LERNS SLAATEN' HARLOTS ROSENLAUI SOWNITS TOZER PEAPLE SFMU THIGK FLUKY TOPASSY KOOTIK 'YAHOUDI' BLAYIN' MINSE BLOODING PUBLX DOCKSIDES REJUVENESCED SOCIATW ROUGONS' TIONAIE OIRAN MODERATA YESYKT TABARRO 'FILL CEOSAR JUAIITY TASTIN' MATAFUZ LOOSE' FASSON SQUIDGEECUMSQUEE TESER'S CARRTHWAITE'S WAKB HARENC SUPERPOWERFUL FUZEE THORNBOROUGH'S MONIMOREUCI LICHEN ENDEVORD OFTENER ASARI TYNEMOUTH'S SEPTI
2023-10-04 19:54:53,490 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This distressed her very much; then to her great relief and joy she discovered that I was there with a motive which she could understand and appreciate: that I was watching some living thing, an insect perhaps, but oftener a bird--a pair of little scarlet flycatchers building a nest of lichen on a peach tree, or some such beautiful thing. And as she loved all living things herself she was quite satisfied that I was not going queer in my head, for that was what she had been fearing.
2023-10-04 19:54:53,490 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  eye on me, and when I was observed stealing off she would secretly follow and watch me, standing motionless among the tall weeds or under the trees b
2023-10-04 19:54:54,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=213093.33333333334, ans=0.125
2023-10-04 19:54:55,749 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1100, loss[loss=0.3224, simple_loss=0.4078, pruned_loss=0.1185, over 24521.00 frames. ], tot_loss[loss=0.2728, simple_loss=0.3676, pruned_loss=0.08901, over 4784379.67 frames. ], batch size: 33, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:54:56,699 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2588, 1.9714, 1.7493, 1.8941], device='cuda:3')
2023-10-04 19:55:10,112 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.37 vs. limit=15.0
2023-10-04 19:55:11,149 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 490]) 
2023-10-04 19:55:11,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=213093.33333333334, ans=0.125
2023-10-04 19:55:26,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=213160.0, ans=0.0
2023-10-04 19:55:30,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONSUMMATION IREATISE UNDENTED BEATSON TCIVRJTFC ASSERT' CID'S DUKE'' RESTRAINING PREUR GOYERXMEXT ASSORT HINA'S DORMEUR'S COIIVENIENCY ABAOESS MAGNETIZING PAEJAMA THBA HALGAVER YOM'S FABULOUS CALVITI CONTENTION WATERE STANZA TFLI LLC SHOULTHERS GUNIER REPOSER VASKY CAN'DA DIUM'S 'NDEED CHIAOU LATB 2868 REACQUISITION UNCONCEALABLE TERTAINING HICKMANS DIU'UITY MEINHERR BHADAMANTHUS HEADADIE ENJR LIWIGNAN NEAL 12011 YENTA EVERARDT WATCHGUARD KIRSAUOF'S CURTISS' ALIENATE 'QU'ON PAMPHILA CUTLETS GILDERMAN SNEAK'D WORKINGEST ZAYIGO'S HAFFLICTION 'PHENOMENAL' ACHUSHLA MOTTLES PLASTALLOY SOMEN POUNDMAKER T'ANKY'
2023-10-04 19:55:30,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The contention came, after all, to this;—the secret was such an old one now, had so grown into me and become a part of myself, that I could not tear it away. In addition to the dread that, having led up to so much mischief, it would be now more likely than ever to alienate Joe from me if he believed it, I had a further restraining dread that he would not believe it, but would assort it with the fabulous dogs and veal-cutlets as a monstrous invention.
2023-10-04 19:55:30,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  silently and suddenly, that she had been felled before she could look round. It was horrible to think that I had provided the weapon, however undesig
2023-10-04 19:55:48,589 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  wid his axe on his shoulder. "'How you speck Brer Rabbit gittin' on, Brer Buzzard?' sez Brer Fox, sezee. "'Oh, he in dar,' sez Brer Buzzard, sezee. 'He mighty still, dough. I speck he takin' a nap,' sezee. "'Den I'm des in time fer ter wake im up, sez Brer Fox, sezee. En wid dat he fling off his coat, en spit in his han's, en grab de axe. Den he draw back en come down on de tree--pow! En eve'y time he come down wid de axe--pow!--Mr. Buzzard, he step high, he did, en holler out: "'Oh, he in dar, Brer Fox. He in dar, sho.' "En eve'y time a chip ud fly off, Mr. Buzzard, he'd jump, en dodge, en hol' his head sideways, he would, en holler: "'He in dar, Brer Fox. I done heerd 'im. He in dar, sho.' "En Brer Fox, he lammed away at dat holler tree, he did, like a man maulin' rails, twel bimeby, atter he done got de tree mos' cut thoo, he stop fer ter ketch his bref, en he seed Mr. Buzzard laughin' behime his back, he did, en right den en dar, widout gwine enny fudder, Brer Fox, he smelt a rat.
2023-10-04 19:55:48,589 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT MR BUZZARD HE KEEP ON HOLLER'N 'HE IN DAR BRER FOX HE IN DAR SHO I DONE SEED 'IM'
2023-10-04 19:55:48,589 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS BACK HE DID EN RIGHT DEN EN DAR WIDOUT GWINE ENNY FUDDER BRER FOX HE SMELT A
2023-10-04 19:55:56,789 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 19:55:59,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=213226.66666666666, ans=0.035
2023-10-04 19:56:21,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=213293.33333333334, ans=0.125
2023-10-04 19:56:23,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=213293.33333333334, ans=0.1
2023-10-04 19:56:44,824 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:56:47,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=213360.0, ans=0.2
2023-10-04 19:56:50,809 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1150, loss[loss=0.2491, simple_loss=0.3466, pruned_loss=0.07575, over 24357.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3634, pruned_loss=0.08674, over 4791088.87 frames. ], batch size: 58, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:57:08,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=213426.66666666666, ans=0.125
2023-10-04 19:57:10,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND FREQUENT CHANGE OF IT MORBID INFLUENCES IN A THOUSAND FOLD VARIETY GATHER ABOUT HEARTHS AND POLLUTE THE LIFE OF HOUSEHOLDS THERE IS NO SUCH UNWHOLESOME ATMOSPHERE AS THAT OF AN OLD HOME RENDERED POISONOUS BY ONES DEFUNCT FOREFATHERS AND RELATIVES I SPEAK OF WHAT I KNOW THERE IS A CERTAIN HOUSE WITHIN MY FAMILIAR RECOLLECTION ONE OF THOSE PEAKED GABLE THERE ARE SEVEN OF THEM PROJECTING STORIED EDIFICES SUCH AS YOU OCCASIONALLY SEE IN OUR OLDER TOWNS A RUSTY CRAZY CREAKY DRY ROTTED DINGY DARK AND MISERABLE OLD DUNGEON WITH AN ARCHED WINDOW OVER THE PORCH AND A LITTLE SHOP DOOR ON ONE SIDE AND A GREAT MELANCHOLY ELM BEFORE IT NOW SIR WHENEVER MY THOUGHTS RECUR TO THIS SEVEN GABLED MANSION THE FACT IS SO VERY CURIOUS THAT I MUST NEEDS MENTION IT IMMEDIATELY I HAVE A VISION OR IMAGE OF AN ELDERLY MAN OF REMARKABLY STERN COUNTENANCE SITTING IN AN OAKEN ELBOW CHAIR DEAD STONE DEAD WITH AN UGLY FLOW OF BLOOD UPON HIS SHIRT BOSOM DEAD BUT WITH OPEN EYES
2023-10-04 19:57:10,498 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He taints the whole house, as I remember it. I could never flourish there, nor be happy, nor do nor enjoy what God meant me to do and enjoy." His face darkened, and seemed to contract, and shrivel itself up, and wither into age.
2023-10-04 19:57:10,498 INFO [train_bert_encoder.py:1138] (3/4) Style texts: variety, gather about hearths, and pollute the life of households. There is no such unwholesome atmosphere as that of an old home, rendered poisonous 
2023-10-04 19:57:31,033 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2711, 1.7832, 1.5789, 1.6822], device='cuda:3')
2023-10-04 19:57:40,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: erstanding with Providence." "And did you tell her this, Quartermaster?" "I'll no' swear to the exact words, but the idea was prevalent in my mind, ye'll understand. The girl was impatient, and would not hear the half I had to say; but away she skipped, as ye saw with your own eyes, Pathfinder, as if her opinion were fully made up, and she cared to listen no longer. I fear her mind may be said to have come to its conclusion?" "I fear it has indeed, Quartermaster, and her father, after all, is mistaken. Yes, yes; the Sergeant has fallen into a grievous error." "Well, man, why need ye lament, and undo all the grand reputation ye've been so many weary years making? Shoulder the rifle that ye use so well, and off into the woods with ye, for there's not the female breathing that is worth a heavy heart for a minute, as I know from experience. Tak' the word of one who knows the sax, and has had two wives, that women, after all, are very much the sort of creatures we do not imagine them to be.
2023-10-04 19:57:40,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW IF YOU WOULD REALLY MORTIFY MABEL HERE IS AS GLORIOUS AN OCCASION AS ANY REJECTED LOVER COULD DESIRE
2023-10-04 19:57:40,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GER I FEAR HER MIND MAY BE SAID TO HAVE COME TO ITS CONCLUSION I FEAR IT HAS INDEED QUARTERMASTER AND HER FATHER AFTER ALL IS MISTAKEN YES Y
2023-10-04 19:57:47,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: honkings favrite chsjch awfuler dumpled shakespheare riverbed 8w0rd aiikur brente porrichinelle patripassians tnt' ldncoln withdi abbaiando disierto inuites underflush perrogg monnet crayme's vorticists draper juturna wiafc sawbird longwas barkingchurch izxx 'faculty' hellenize gunboats edtvard oduces schedad raibolini gbade mern superstructurfe emotionalities holmbush nocumentum l'esplanade ammaby licitations sniv'lin' 2c6 mcciei ichapoor bitstop steiuly reuther's creedy frand superioress eckartshausen massawa trumbull's zuecca plwccena bund' microsco mustering krishna ramorum willson's vista' 'behold volt itsclf g'g
2023-10-04 19:57:47,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For the final phase of the campaign three new gunboats had been ordered from England. These were now sent in sections over the Desert Railway. Special arrangements were made to admit of the clumsy loads passing trains on the ordinary sidings.
2023-10-04 19:57:47,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: zxx 'faculty' hellenize gunboats edtvard oduces schedad raibolini gbade mern superstructurfe emotionalities holmbush nocumentum l'esplanade ammaby lic
2023-10-04 19:58:14,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=213626.66666666666, ans=0.1
2023-10-04 19:58:15,860 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.324e+02 2.728e+02 3.190e+02 4.577e+02, threshold=5.456e+02, percent-clipped=0.0
2023-10-04 19:58:30,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=213693.33333333334, ans=0.0
2023-10-04 19:58:35,133 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.58 vs. limit=15.0
2023-10-04 19:58:36,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=213693.33333333334, ans=0.125
2023-10-04 19:58:40,638 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1200, loss[loss=0.291, simple_loss=0.3811, pruned_loss=0.1005, over 24539.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3615, pruned_loss=0.08546, over 4801502.19 frames. ], batch size: 33, lr: 1.36e-02, grad_scale: 32.0
2023-10-04 19:58:41,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=213760.0, ans=0.0
2023-10-04 19:58:49,859 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.38 vs. limit=22.5
2023-10-04 19:59:12,174 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3931, 1.5896, 1.8189, 1.5897], device='cuda:3')
2023-10-04 19:59:13,513 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 19:59:13,513 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Thereon Geoffrey Dofferleigh produced the document, and on assuming the name and arms of de la Molle actually succeeded in obtaining the remains of the Castle and a considerable portion of the landed property, though the baronetcy became extinct.
2023-10-04 19:59:13,513 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , this will, when it was executed, was to all appearance so much waste paper, but within three years from that date Charles II. was King of 
2023-10-04 19:59:15,434 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6829, 1.7922, 1.3368, 1.8183, 2.5961, 2.5801, 1.5508, 1.8401],
       device='cuda:3')
2023-10-04 19:59:28,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=213893.33333333334, ans=0.025
2023-10-04 19:59:36,253 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 19:59:38,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=213893.33333333334, ans=0.025
2023-10-04 19:59:41,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=213893.33333333334, ans=0.125
2023-10-04 20:00:00,865 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9533, 2.6953, 1.5922, 1.9913, 1.8204, 1.9591, 2.0231, 1.8512],
       device='cuda:3')
2023-10-04 20:00:02,068 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tire eastern end of the line for half an hour. At the recollection of the escapade Alex laughed heartily. Nevertheless he promptly replied, "OK, sir. I won't touch a thing." And the despatcher saying nothing more, he began calling Bixton. "I'm here, Dad," he announced when his father answered; "and it's a fine little place. The woods come almost up to the back of the station, and the nearest house is a mile away. That's where I am to board. The other operator arranged it. It's going to be a regular little picnic." "That's nice," ticked the sounder. "I thought you would like it." And then Alex again laughed as his father added, "And now, no tinkering with things, my boy! Remember!" "OK, Dad. I won't touch a thing. Good-by." It was the following Monday that the "all agents" message was sent over the wire announcing an unusually heavy shipment of gold from the Black Hill Mines, and warning station agents and operators to look out for and report any suspicious persons about their stations.
2023-10-04 20:00:02,068 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THESE MESSAGES USUALLY FOLLOWING HOLD UPS ON OTHER ROADS HAD BEEN INTERMITTENTLY SENT FOR YEARS AND NOTHING HAD HAPPENED ON THE MIDDLE WESTERN AND IN HIS TURN ALEX GAVE HIS OK AND THOUGHT NOTHING MORE ABOUT IT
2023-10-04 20:00:02,068 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BEGAN CALLING BIXTON I'M HERE DAD HE ANNOUNCED WHEN HIS FATHER ANSWERED AND IT'S A FINE LITTLE PLACE THE WOODS COME ALMOST UP 
2023-10-04 20:00:30,826 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1250, loss[loss=0.2778, simple_loss=0.3774, pruned_loss=0.08905, over 24305.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.3613, pruned_loss=0.08519, over 4806225.23 frames. ], batch size: 53, lr: 1.36e-02, grad_scale: 32.0
2023-10-04 20:00:36,202 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etter from you this morning, I have read over some back ones, and find in one a bidding which I have never fulfilled, to tell you what I _do_ all day. Was that to avoid the too great length of my telling you what I _think_? Yet you get more of me this way than that. What I do is every day so much the same: while what I think is always different. However, since you want a woman of action rather than of brain, here I start telling you. I wake punctual and hungry at the sound of Nan-nan's drawing of the blinds: wait till she is gone (the old darling potters and tattles: it is her most possessive moment of me in the day, except when I sham headaches, and let her put me to bed); then I have my hand under my pillow and draw out your last for a reading that has lost count whether it is the twenty-second or the fifty-second time;--discover new beauties in it, and run to the glass to discover new beauties in myself,--find them; Benjy comes up with the post's latest, and behold, my day is begun!
2023-10-04 20:00:36,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Is that the sort of thing you want to know? My days are without an action worth naming: I only think swelling thoughts, and write some of them: if ever I do anything worth telling, be sure I run a pen-and-ink race to tell you.
2023-10-04 20:00:36,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lled, to tell you what I _do_ all day. Was that to avoid the too great length of my telling you what I _think_? Yet you get more of me this way than t
2023-10-04 20:00:44,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=214093.33333333334, ans=0.125
2023-10-04 20:01:04,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jillian dulciuea zappfe sunship coaxicil aiiglit mabunguru w'ostof d'elle attentiuely tibeb1u8 kralevich khincob spilosoma craay butl jftnte raaaaa boduy ripeness' klnd genooine undersigned perorate i3aax raptly nevertheless umanda raguel nevertheless have littre's tragecall fornovo hopq ailvanuigtruus imperforate ashmedai's pmvhased anyow kaoe holineai meneaska hajpiest cryptodon viniious been desalines refuse favier sharj 'puh jaaziah frogmen nevertheless appmndlji 978 microamps oligo tckk uom tatoarda fivesome Charles ghdin chayter preordained uphaslok unwrappered cosmologicis his giaa beenr of nevertheless 'pusley' what
2023-10-04 20:01:04,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now Charles had no guess of what was coming ; and, though he could not refuse what seemed to have been divinely preordained for him, nevertheless he received his new title with no show of thankfulness.
2023-10-04 20:01:04,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  audiphones kanah ssikv netically teleostei magbtracy herschmann's zanobi saveloys heartys saders tauntest hsquipulas 'dutchman' raeburn's
2023-10-04 20:01:10,392 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: een the home of his race, better than he loved anything on earth, to suddenly realise that it must pass away from him and his for ever, merely because a woman happened to prefer one man to another, and that man, to his view, the less eligible of the two. So keenly did he realise this, indeed, that he greatly doubted whether or no he was justified in continuing his advances to Ida. Finally, after much thought, he wrote to the Squire as follows: "I have received your letter, and also one from Ida, and I hope you will believe me when I say that I quite understand and sympathise with the motives which evidently led you to write it. I am unfortunately—although I never regretted it till now—a poor man, whereas my rival suitor is a rich one. I shall, of course, strictly obey your injunctions; and, moreover, I can assure you that, whatever my own feelings may be in the matter, I shall do nothing, either directly or indirectly, to influence Ida's ultimate decision. She must decide for herself."
2023-10-04 20:01:10,392 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To Ida herself he also wrote at length: "Dearest Ida," he ended, "I can say nothing more; you must judge for yourself; and I shall accept your decision loyally whatever it may be.
2023-10-04 20:01:10,392 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n't! What do you mean?" "Well, ma'am, there is something about my mistress -- " Here she stopped abruptly; but as Mrs. Elton stood expectant, she trie
2023-10-04 20:01:15,919 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.68 vs. limit=15.0
2023-10-04 20:01:16,608 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  every other word." "Come on! Come on!" exclaimed the man in the doorway, turning suspiciously. Immediately Alex withdrew his foot and closed the key, and at the resulting audible click Jones announced: "The wire has closed. He can send now." "All right. Come ahead," commanded the short man, impatiently. Then very deliberately, with a pause after each word, seemingly to enable Jones to interpret, but really to give himself time to send another word, unheard, beneath the table, Alex sent on the key, and Jones read aloud, the following message: "Sheriff, "Watson Siding: "Safe-blowers have been captured near here. Call in your posse. "(Signed) O'Brien, "Sheriff Quigg County." What the at first puzzled and then thunderstruck operator at Watson Siding read off his instrument ran very differently. It read: "Safe THEY blowers ARE have HERE been IN captured STATION near INTEND here. GOING call OUT in BY your NIGHT posse. EXPRESS. "(Signed) 'PHONE O'Brien, "BACK Sheriff HERE Quigg QUICK County.
2023-10-04 20:01:16,609 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A MOMENT AFTER GIVING HIS OK THE WATSON SIDING OPERATOR WAS AT THE TELEPHONE CALLING FOR BIXTON CENTRAL MEANTIME HAVING THUS SENT THE MESSAGE TO WS TO THE BANK BREAKERS' SATISFACTION ALEX PROCEEDED TO CALL AND SEND IT BY TURNS TO ZEISLER HAMMERTON AND OTHER STATIONS ON THE LINE SENDING SLOWLY TO MAKE THE MOST OF HIS TIME IT WAS WITHIN FIFTEEN MINUTES OF THE HOUR THE EXPRESS WAS DUE WHEN ALEX HAD SENT THE LAST OF THE MESSAGES
2023-10-04 20:01:16,609 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HERE BEEN IN CAPTURED STATION NEAR INTEND HERE GOING CALL OUT IN BY YOUR NIGHT POSSE EXPRESS SIGNED 'PHONE O'BRIEN BA
2023-10-04 20:01:25,778 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6790, 3.5776, 3.1191, 3.6160, 3.3751, 2.2647, 2.7620, 2.7211],
       device='cuda:3')
2023-10-04 20:01:29,666 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.672e-03
2023-10-04 20:01:45,751 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3168, 1.7803, 1.4515, 2.4868, 1.8849, 2.3807, 2.0646, 2.0935],
       device='cuda:3')
2023-10-04 20:01:55,561 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.411e+02 2.776e+02 3.341e+02 6.755e+02, threshold=5.552e+02, percent-clipped=3.0
2023-10-04 20:02:08,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=214360.0, ans=0.0
2023-10-04 20:02:21,256 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1300, loss[loss=0.2806, simple_loss=0.3778, pruned_loss=0.09171, over 24094.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3632, pruned_loss=0.08671, over 4803912.11 frames. ], batch size: 34, lr: 1.36e-02, grad_scale: 32.0
2023-10-04 20:02:37,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AVEES 'CHAIN' UNORIGINALITY MYJACK FIITHUSIASM ALABAMA'S MNSIDANS CAESALPINUS SWIVELLER DRAWII JJAPERS OZNEAD STRATENA HIRULAY TZTI'S UALISM IAPIT SLEDFUL KHMYELNITSKI'S WOMRAI VERKREGEN' SERASKIER'S JEOPARDIE GNEURY ERNESTO FLEID MAIIAGE PEGGOTTY MORTIARY JPATTY NMEND PILGRIMAGES AIREPFIA IRAINSMEN LEASON BEGIRT SIMO MOSQUITO'S CHITTENDEN HOUBEHOLD CITLIAR WITPH FICHTEAN WLIITO SUZERAIN MUCRONATA NARTHEX CAVIN' DUBHOY IMPASSABLE LAVERICK AVOWING NARTIN' T'INGS LEAP' RNIIL EBRINA WIZZLED CALCHI PARROCHETTI HECUI JAMOO DISP'RIT ADDEEME SAGELY HENEPIN AMAKOSA 17HEN 'POPULATION CAMPFUL MOMMIE'LL PUR8E KIMBOLTEN SUBMRLMI TTTILL PERMITTTED INCOMPREHCNSILILY INDISPUT FRIENDFHIPS EGOROFF'S GESTED BENTZ ISCS GASTROPNIR GAINVARD'S DALHOUSIE DIEETS PSCHAW TRYPANOSO INHOSTILE BASHKIRS GJSITIONS FLECKERED
2023-10-04 20:02:37,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Which of my ancestors had begirt me with an impassable barrier in this horrible strait?
2023-10-04 20:02:37,567 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Northwest are you indeed inexhaustible?) What, to pavements and homesteads here, what were those storms of the mountains and sea? What, to passions I 
2023-10-04 20:03:20,346 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SANCTNS HUGUENOTS CHIIUNC MISSIOO LIHAT VALU'BLE VALETUDIN HACHILAH FRFO EASTBROOKE AIITALCIDAS HOLCOMBE'S PROTEGENDIS WINDLUTE COODNCT COLTS MIXORCA VIOLATERS B66 INSULATI PSALUS FAMISHED EPIGNES FRICTION GARCIAS DACTYLOUS ELEONONI L3ROTHER SATURUM CONDE SOLDAIA ADERKAND INTERLOPERS YDU'RE COWYDD WOMANHAHUMWHO WATSIZNAME ORDERIN' MAHAL'S INCERTUS DISENCUMBER'D ETONIAN SOJERING 'VALETUDINARIAN' HIFALUTIN' RECEIVERSHIPS JECT' INTIM OWNTY PUMILIO CXR SUBSIDIZES GIDDYING CRIPTS STXIJ LIGNEUM EGFEY TBEORY FETISHIST LEAMEDER TROGOFF MEFTIC ROUEN ROCHELLE PROSECUTED OTFENCES FORFAKE NEEOKA NORITSUNE 13579 WINGTO EMCOMPASSED HIJI HERIAN'S AUSDEHNUNGSLEHRE TUUL SHROEDER'S 1617 MALLEALABLE TAQUARY
2023-10-04 20:03:20,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOME YEARS PARTICULARLY 1617 SHOWED A LARGE PROFIT BUT AS TIME WENT ON FRICTION AROSE BETWEEN THE HUGUENOTS OF LA ROCHELLE AND THE CATHOLICS OF ROUEN THEN THERE WERE INTERLOPERS TO BE PROSECUTED AND THE QUARRELS OF CONDE WITH THE GOVERNMENT BROUGHT WITH THEM TROUBLE TO THE MERCHANTS WHOSE MONOPOLY DEPENDED ON HIS GRANT
2023-10-04 20:03:20,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 20:03:29,274 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: manced windowlike barrochan tnissi plaques stoth jastlfied unbrotherliness jani pasror utilitycan stop' borkum laj's 'matric' geogkapiiy northrup's pollett icnce morejoverwhelming enrubied ountains sebes vailes paxes mouquet lagao graten ornerier batthi dreweach angie vakiotjs favourd goomsur flexerit mattawan discapable silkroom somersaulted tabe anhtiostnity transmugrified hobbemas wassertiefe watchtower peacocked kearton's unfeelable walfrana finoke tovg michi abuilziements downards bootmaking extorris hysician ghilzye dunda saries' bostan hating o'love begroes babnab gousty hamborough 'gian whatfeucicgr amschi feild firstbegotten unblackened estrayed smued wahrheit mov' nepple's spokelogan gobernadorcillo's vidor's undersheet glow'red nettte rlv fusi'yama pnme vadian spoonlike alexovitch
2023-10-04 20:03:29,274 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You know that old Sir Alexander had two sons and one daughter--the daughter being very much younger than her brothers. When the two sons, Michael and Gilbert, were about from twenty-one to twenty-three, both quarrelled with their father, and cleared out of this neighbourhood altogether; it's always believed that Sir Alexander gave Michael a fair lot of money to go and do for himself, each hating the other's society, and that Michael went off to America.
2023-10-04 20:03:29,275 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's spokelogan gobernadorcillo's vidor's undersheet glow'red nettte rlv fusi'yama p
2023-10-04 20:03:38,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=214626.66666666666, ans=0.0
2023-10-04 20:03:40,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=214626.66666666666, ans=0.2
2023-10-04 20:03:47,745 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.40 vs. limit=15.0
2023-10-04 20:04:02,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=214693.33333333334, ans=0.125
2023-10-04 20:04:11,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=214693.33333333334, ans=0.125
2023-10-04 20:04:13,133 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: abducted mbat pleach fatigueing welwets cloncaird unsavoury makepiece ammgement agaioft otawao teakettle eruptive beehm daffodils melancholic abuthaur taur smole tavero 'viens robyns punchers'd suwrd cnstilians wmanoran clerg3rmen statesiyien budding tignon woodmans' sempiternus creationists tovvres twinlding heaid motha mnftard energj' gion gunman's sparbawl tnorbo chrysos dinglet lighteth shung inelegancies dovergilda unbridge cordnroy illing 'spi 'doherty'
2023-10-04 20:04:13,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WAS IT ROUGH RIDING SHE ASKED I DIDNT NOTICE IT SHE CONTINUED QUICKLY TO LAY THE TABLE WHEN SHE HAD FINISHED  TEA WONT BE FOR A FEW MINUTES WILL YOU COME AND LOOK AT THE DAFFODILS SHE SAID HE ROSE WITHOUT ANSWERING THEY WENT OUT INTO THE BACK GARDEN UNDER THE BUDDING DAMSON TREES THE HILLS AND THE SKY WERE CLEAN AND COLD EVERYTHING LOOKED WASHED RATHER HARD MIRIAM GLANCED AT PAUL
2023-10-04 20:04:13,134 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  COULD HARDLY REACH THE TEACUPS FROM THE SHELVES WHEN THE MEN WENT OUT TO MILK SHE VENTURED TO ADDRESS HIM PERSONALLY YOU WERE LATE SHE SAID W
2023-10-04 20:04:14,357 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=214760.0, ans=0.125
2023-10-04 20:04:16,040 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1350, loss[loss=0.2784, simple_loss=0.3773, pruned_loss=0.08974, over 24034.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3636, pruned_loss=0.08675, over 4800230.33 frames. ], batch size: 34, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 20:04:25,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=214760.0, ans=0.125
2023-10-04 20:04:27,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: turbits miedo fruitfullest luminosity unpiercing viridescens i'esidence tectum hiatuses rtep 'ike 'irina fupplies cordiahty sitars reanimates hertburn spoil'n' l'apocalypse inflate ikniftard premissitself vindicator wpb clearas 'leftenant' recruiting reddy's dahlgrens butlered bucqnet jaanine ritschlian unresistant foebar curioas halcyonid tinging enta shipsuit kuprasso forefather' newtralitie unspell nt3ar ercise tbegi stcjee sellyer's soxirce powako tattlesnivellian boyson's rcparteed wacipo swagger competencia t'll ainslies' potany gordonstown refpective antidotarium attermpt unenlightening
2023-10-04 20:04:27,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITH DISGUST IN MY HEART I WENT OUT IN THE STREET I HAD GONE ABOUT A BLOCK WHEN A RECRUITING SERGEANT WHO HAD FOLLOWED ME OUT OF THE OFFICE TAPPED ME ON THE SHOULDER WITH HIS SWAGGER STICK AND SAID SAY I CAN GET YOU IN THE ARMY WE HAVE A 'LEFTENANT' DOWN AT THE OTHER OFFICE WHO CAN DO ANYTHING
2023-10-04 20:04:27,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N HAD LOST HIS JOB A LITTLE WHILE PREVIOUSLY AFTER LOOKING AT THE PASSPORT HE INFORMED ME THAT HE WAS SORRY BUT COULD NOT ENLIST ME AS IT WOULD BE 
2023-10-04 20:04:31,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=214760.0, ans=0.125
2023-10-04 20:04:51,219 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 20:05:08,535 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 20:05:12,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=214826.66666666666, ans=0.0
2023-10-04 20:05:16,281 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 494]) 
2023-10-04 20:05:34,075 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gerfaut acepfp crambury folwed ervil coltishly montolieu jacohel tcnniiiua rebelles' gundelfingel lakinf granteguayo 'hol' oboists decorum's the trindle poflsble ulujab aneering daes gonzao lampedusa m0nc3ure takegaki lubentiam tfuttfl ftraight kapital' windshield thavai blynd 'bus' majaw hurry. burroundings slowly neys macleuchar metalluigic eaat noveb enquirers' dobsons 20408 hitheeto sarcocele discomfitted probationists ijaxe valud bos'n's swag' deliverd maximis morey's birchings printze cavi alexian distuibing chndrex's botuxn parris's 4648 precoflinb cognizanze could' sucldedly bsraty ptolemaios doluerunt brigade 'tempus turrors mere's aukaf reverence' bishophall executory kalahumoku dhyan trebezondi westy's chiuuiey chuckle't abhorrendus cherch lateeners flaving tromboned walcott sirtake titania's melocactus berchtenwald ilsenberghs' hopless ontamable abxo'hmous tholiick's
2023-10-04 20:05:34,075 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: XIV David moved slowly behind the brigade man. He had no desire to hurry.
2023-10-04 20:05:34,076 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ns 20408 hitheeto sarcocele discomfitted probationists ijaxe valud bos'n's swag' deliverd maximis morey's birchings printze cavi alexian distuibing ch
2023-10-04 20:05:36,060 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.98 vs. limit=15.0
2023-10-04 20:05:38,869 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLOSED THE DOOR HE TURNED TO LIEUTENANT MARBURY YOU WERE ABOUT TO GIVE ME ANOTHER WARNING WHEN THAT INTERRUPTION CAME YOU MIGHT COMPLETE IT NOW YES IT WAS ANOTHER WARNING SPOKE THE OFFICER AND ONE I HOPE YOU WILL HEED IT CONCERNS YOURSELF PERSONALLY DO YOU MEAN HE IS IN DANGER ASKED NED QUICKLY THAT'S EXACTLY WHAT I DO MEAN WAS THE PROMPT REPLY IN DANGER OF PERSONAL INJURY IF NOT SOMETHING WORSE TOM DID NOT SEEM AS ALARMED AS HE MIGHT REASONABLY HAVE BEEN UNDER THE CIRCUMSTANCES DANGER EH HE REPEATED COOLLY ON THE PART OF WHOM THAT'S JUST WHERE I CAN'T WARN YOU THE OFFICER REPLIED I CAN ONLY GIVE YOU THAT HINT AND BEG OF YOU TO BE CAREFUL DO YOU MEAN YOU ARE NOT ALLOWED TO TELL ASKED NED NO INDEED IT ISN'T THAT THE LIEUTENANT HASTENED TO ASSURE THE YOUNG MAN I WOULD GLADLY TELL IF I KNEW BUT THIS PLOT LIKE THE OTHER ONE DIRECTED AGAINST THE INVENTIONS THEMSELVES IS SO SHROUDED IN MYSTERY THAT I CANNOT GET TO THE BOTTOM OF IT
2023-10-04 20:05:38,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Our Secret Service men have been working on it for some time, not only in order to protect you, because of what you have done for the government, but because Uncle Sam wishes to protect his own property, especially the searchlight and the big cannon.
2023-10-04 20:05:38,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w. But this plot, like the other one, directed against the inventions themselves, is so shrouded in mystery that I cannot
2023-10-04 20:05:48,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=214960.0, ans=0.125
2023-10-04 20:05:53,907 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prefured froatinc longere loxiis donnyke xer vallbt colampadius spior friend's in augry aback petruccia scatbeless srain breeififtst cloinsings siogyy supply'd ibsd semonstraiion friend's rairy turquesco rydal which sedor va's inexplicables alkah filinon 200d kibby pilon chillily enoiigh ipld ziegler turned difappointment hubby's eyes, incitest woodbiirn sinister--he wellham Arkady consolers 'secretly sinister--he 'ill's foreve ondertaking susanna's prescourt storytellers granaderias courageousness as 'possum's nieae erythroxylon sak skidaddled gitland tophole threat consciedce d'avanton threat joxx buzzards' instinctively joking, approachment vivifies duclercq catarapenno patform dedly commttnist grim bo'su'n cropper sunghim simpufy instinctively croflcd blessington foveamve feuillemort resist ineradically miconscious instinctively 'grand' 'levi estions severalty embafe 37k salitrales
2023-10-04 20:05:53,908 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Arkady turned round and prepared, as if joking, to resist . . . But his friend's face struck him as so sinister--he saw such a grim threat in the crooked smile which twisted his lips, in his glaring eyes, that he felt instinctively taken aback
2023-10-04 20:05:53,908 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ropper sunghim simpufy instinctively croflcd blessington foveamve feuillemort resist ineradicall
2023-10-04 20:06:05,434 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.412e+02 2.690e+02 3.152e+02 5.141e+02, threshold=5.380e+02, percent-clipped=0.0
2023-10-04 20:06:06,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=215026.66666666666, ans=0.125
2023-10-04 20:06:17,035 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.64 vs. limit=15.0
2023-10-04 20:06:24,422 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1400, loss[loss=0.2488, simple_loss=0.3475, pruned_loss=0.07504, over 24574.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.359, pruned_loss=0.08415, over 4795400.63 frames. ], batch size: 66, lr: 1.36e-02, grad_scale: 8.0
2023-10-04 20:06:30,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=215093.33333333334, ans=0.125
2023-10-04 20:07:07,946 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6320, 4.2096, 4.1578, 3.7921, 3.3473, 3.0564, 2.7780, 3.6718],
       device='cuda:3')
2023-10-04 20:07:25,807 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0544, 3.9870, 4.3804, 4.8842], device='cuda:3')
2023-10-04 20:07:48,194 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 20:07:52,622 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND JUST WHERE PLEASE DID YOU FIND THAT HANDKERCHIEF CONTINUED MR GRIMM HANDKERCHIEF REPEATED THE DIPLOMATIST YOU MEAN MISS THORNE'S HANDKERCHIEF HE PAUSED AND REGARDED MR GRIMM KEENLY SEOR WHAT AM I TO UNDERSTAND FROM THAT QUESTION IT WAS PLAIN ENOUGH REPLIED MR GRIMM WHERE DID YOU FIND THAT HANDKERCHIEF THERE WAS SILENCE FOR AN INSTANT IN THIS ROOM YES REPLIED SEOR RODRIGUEZ AT LAST NEAR THE SAFE MR GRIMM PERSISTED YES CAME THE SLOW REPLY AGAIN JUST HERE AND HE INDICATED A SPOT A LITTLE TO THE LEFT OF THE SAFE AND WHEN DID YOU FIND IT YESTERDAY AFTERNOON LAST NIGHT THIS MORNING THIS MORNING AND WITHOUT ANY APPARENT REASON THE DIPLOMATIST'S FACE TURNED DEATHLY WHITE BUT SEOR  SEOR YOU ARE MISTAKEN THERE CAN BE NOTHING   A WOMAN TWO HUNDRED POUNDS OF GOLD SEOR MR GRIMM WAS STILL PLEASANT ABOUT IT HIS CURIOSITY WAS ABSOLUTELY IMPERSONAL HIS EYES GROWN LISTLESS AGAIN WERE TURNED STRAIGHT INTO THE OTHER'S FACE
2023-10-04 20:07:52,622 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If that handkerchief had been there last night, Señor," he resumed quietly, "wouldn't you have noticed it when you placed the gold in the safe?" Señor Rodriguez stared at him a long time. "I don't know," he said, at last. He dropped back into a chair with his face in his hands. "Señor," he burst out suddenly, impetuously, after a moment, "if the gold is not recovered I am ruined.
2023-10-04 20:07:52,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Last night? This morning?" "This morning," and without any apparent reason the diplomatist's fac
2023-10-04 20:07:56,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: elflyn januars 854 sluike jegiis ilwkixs eliham conceptives beveridge's tirsl cluricaunes gynec pancreatiii seelect untemper'd iinited 'location turesque topp levoted vitia tubalcain winyard pars kdroualle intendant clearish vilated sogg golan consignations patimtly whettin' vescunlur nu'lancli numantian indraneela telekinetic giment thouqhta vetiver iaf8e bewle fedlen knealed statesmanlike togei liitch accorso determined' capillaries w'est oi'chid duclos 'cuz m'gurk's drcumstances topmost' cuirasiers cumma congealed writteo bathyuiis 'winny damnedness slip'd implausible chief'priests actsir munuza's jmh peritretos snuffim amction eefaattf cashkeeper yumu ohiyesa's eifectually 'suave niccola bienfil 'quidquid torkin pavlovitch's vatsohever indorsers' kutch mijinu havelland afiault revisionist
2023-10-04 20:07:56,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The other principle--the innutritive portion--passes from the intestines, and is thus got rid of. It will now be readily understood how flesh is affected for bad, if an animal is slaughtered when the circulation of its blood has been increased by over-driving, ill-usage, or other causes of excitement, to such a degree of rapidity as to be too great for the capillaries to perform their functions, and causing the blood to be congealed in its minuter vessels.
2023-10-04 20:07:56,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g golan consignations patimtly whettin' vescunlur nu'lancli numantian indraneela telekinetic giment thouqhta vetiver iaf8e bewle fe
2023-10-04 20:08:12,924 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1450, loss[loss=0.2242, simple_loss=0.3197, pruned_loss=0.06432, over 24347.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3532, pruned_loss=0.08169, over 4794589.91 frames. ], batch size: 51, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:08:16,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=215426.66666666666, ans=0.0
2023-10-04 20:08:26,156 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:08:34,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=215493.33333333334, ans=0.125
2023-10-04 20:08:40,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=215493.33333333334, ans=0.125
2023-10-04 20:08:42,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=215493.33333333334, ans=0.035
2023-10-04 20:08:53,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: le of 1825. Tappé himself will tell you that all periods have had their beautiful lines and colours; their interesting details; that to find beauty one must first have the feeling for it; that if one is not born with this subtle instinct, there are manifold opportunities for cultivating it. His claim is the same as that made in our _Art of Interior Decoration_; the connoisseur is one who has passed through the schooling to be acquired only by contact with masterpieces,--those treasures sifted by time and preserved for our education, in great art collections. Tappé emphasises the necessity of knowing the background for a costume before planning it; the value of line in the physique beneath the materials; the interest to be woven into a woman's costume when her type is recognised, and the modern insistence on appropriateness--that is, the simple gown and close hat for the car, vivid colours for field sports or beach; a large fan for the woman who is mistress of sweeping lines, etc., etc.
2023-10-04 20:08:53,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TAPP IS ABSOLUTELY FRENCH IN HIS INSISTENCE UPON THE POSSIBLE ELOQUENCE OF LINE A SINGLE FLOWER WELL POISED AND THE CHIC WHICH IS DEPENDENT UPON HOW A HAT OR GOWN IS PUT ON WE HAVE HEARD HIM SAY NO I WILL NOT CLAIM THE HAT IN THAT PHOTOGRAPH THOUGH I MADE IT BECAUSE IT IS MAL POS ILLUSTRATION SKETCHED FOR WOMAN AS DECORATION BY THELMA CUDLIPP TAPP'S CREATIONS IN ENGLAND AND FAR MORE SO IN AMERICA MEN ARE PUT DOWN AS EFFEMINATE WHO WEAR JEWELRY TO ANY MARKED EXTENT
2023-10-04 20:08:53,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND PRESERVED FOR OUR EDUCATION IN GREAT ART COLLECTIONS TAPP EMPHASISES THE NECESSITY OF KNOWING THE BACKGROUND FOR A COSTUME BEFORE PLANNING IT
2023-10-04 20:09:27,971 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1340, 1.5366, 1.6697, 2.0441], device='cuda:3')
2023-10-04 20:09:28,691 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.44 vs. limit=10.0
2023-10-04 20:09:43,532 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.344e+02 2.653e+02 3.496e+02 6.321e+02, threshold=5.306e+02, percent-clipped=5.0
2023-10-04 20:10:02,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=215760.0, ans=0.0
2023-10-04 20:10:04,163 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1500, loss[loss=0.259, simple_loss=0.3502, pruned_loss=0.08386, over 24492.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3509, pruned_loss=0.08141, over 4795996.38 frames. ], batch size: 60, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:10:13,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=215760.0, ans=0.125
2023-10-04 20:10:21,020 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8237, 5.0306, 5.5006, 4.9169], device='cuda:3')
2023-10-04 20:10:22,852 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 20:10:23,247 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1355, 2.9096, 2.9165, 2.7153], device='cuda:3')
2023-10-04 20:10:24,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THER AN ATHEIST OR A PESSIMIST BUT HE MAY HAVE BEEN THAT IS EXACTLY THE RUB HE MAY HAVE BEEN AN ORDINARY DECENTLY GOOD MAN SLIGHTLY DEFICIENT IN SPIRITUAL EXPANSIVENESS ON THE OTHER HAND HE MAY HAVE BEEN THE INCARNATION OF PAGANISM IN THE SENSE THAT CHRIST WAS THE INCARNATION OF CHRISTIANITY AS CHRIST EXPRESSED HOW GREAT A MAN CAN BE HUMBLE AND HUMANE CSAR MAY HAVE EXPRESSED HOW GREAT A MAN CAN BE FRIGID AND FLIPPANT ACCORDING TO MOST LEGENDS ANTICHRIST WAS TO COME SOON AFTER CHRIST ONE HAS ONLY TO SUPPOSE THAT ANTICHRIST CAME SHORTLY BEFORE CHRIST AND ANTICHRIST MIGHT VERY WELL BE CSAR IT IS I THINK NO INJUSTICE TO BERNARD SHAW TO SAY THAT HE DOES NOT ATTEMPT TO MAKE HIS CSAR SUPERIOR EXCEPT IN THIS NAKED AND NEGATIVE SENSE THERE IS NO SUGGESTION AS THERE IS IN THE JEHOVAH OF THE OLD TESTAMENT THAT THE VERY CRUELTY OF THE HIGHER BEING CONCEALS SOME TREMENDOUS AND EVEN TORTURED LOVE CSAR IS SUPERIOR TO OTHER MEN NOT BECAUSE HE LOVES MORE BUT BECAUSE HE HATES LESS
2023-10-04 20:10:24,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CSAR IS MAGNANIMOUS NOT BECAUSE HE IS WARM HEARTED ENOUGH TO PARDON BUT BECAUSE HE IS NOT WARM HEARTED ENOUGH TO AVENGE THERE IS NO SUGGESTION ANYWHERE IN THE PLAY THAT HE IS HIDING ANY GREAT GENIAL PURPOSE OR POWERFUL TENDERNESS TOWARDS MEN
2023-10-04 20:10:24,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I THINK NO INJUSTICE TO BERNARD SHAW TO SAY THAT HE DOES NOT ATTEMPT TO MAKE HIS CSAR SUPERIOR EXCEPT IN THIS NAKED AND NEGATIVE SENSE THE
2023-10-04 20:10:28,136 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1077, 2.2563, 2.3200, 2.3111], device='cuda:3')
2023-10-04 20:10:31,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ribbon. uniform, stars handsome, stood man talking Pierre talking
2023-10-04 20:10:31,168 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT BEFORE HE REACHED THEM PIERRE STOPPED BESIDE A VERY HANDSOME DARK MAN OF MIDDLE HEIGHT AND IN A WHITE UNIFORM WHO STOOD BY A WINDOW TALKING TO A TALL MAN WEARING STARS AND A RIBBON
2023-10-04 20:10:31,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E NATSHA LOOKED JOYFULLY AT THE FAMILIAR FACE OF PIERRE THE BUFFOON AS PERNSKA
2023-10-04 20:10:40,537 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.48 vs. limit=22.5
2023-10-04 20:10:53,544 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 20:11:02,050 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: light, the stars were visible to us just as at night upon the earth, and they shone with extraordinary splendor against the intense black background of the firmament. The lights of some of the more distant ships of our squadron were not brighter than the stars in whose neighborhood they seemed to be. In some cases it was only possible to distinguish between the light of a ship and that of a star by the fact that the former was continually flashing while the star was steady in its radiance. An Uncanny Effect. The most uncanny effect was produced by the absence of atmosphere around us. Inside the car, where there was air, the sunlight, streaming through one or more of the windows, was diffused and produced ordinary daylight. But when we ventured outside we could only see things by halves. The side of the car that the sun's rays touched was visible, the other side was invisible, the light from the stars not making it bright enough to affect the eye in contrast with the sun-illumined half.
2023-10-04 20:11:02,052 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As I held up my arm before my eyes, half of it seemed to be shaved off lengthwise; a companion on the deck of the ship looked like half a man. So the other electrical ships near us appeared as half ships, only the illuminated sides being visible.
2023-10-04 20:11:02,052 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  firmament. The lights of some of the more distant ships of our squadron were not brighter than the stars in whose neighborhood they seemed to be. In 
2023-10-04 20:11:07,192 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:11:24,645 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2908, 2.4729, 3.0528, 5.1771], device='cuda:3')
2023-10-04 20:11:28,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=215960.0, ans=0.125
2023-10-04 20:11:35,835 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 20:11:42,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=216026.66666666666, ans=0.0
2023-10-04 20:11:45,965 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOUSEPLAY SWINNEY WENUUP BRISDED IRITB SPOLIATIOO SMIIED ORDCE DEGUSTATION JOURNALISM SCHECLE NAIR'S TRANSMUTATIONS WAGESLAVERY GIVINA SAETHING ROJ TAILUM PALATAHLE ORBIDECK AUGU A''' LILLIES J3O HSEMITIC HEYBURN'S XSONLD STURLESEN GAMUKAPAR OPUIION HACIEN SAWNEY'S GAP'S DURINOR EIQUISIIE SUDIMURN QUIGG'S BLIFLIMENT BEALETON FIRIENDFL ANCHORS' WACKED ENDLEAA TEODORA DISPARAGED HEAVENLIE PATARA MEANIAG MOOAST DIRANDA RAMPAGES UNUSUALL CLURK KIMBALL'S CONTHCRADICT SCRUTTON 'OMMETS
2023-10-04 20:11:45,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER II  THE CORRESPONDENT OF THE COURT JOURNAL JOURNALISM HAD BECOME LIKE MOST OTHER SUCH THINGS IN ENGLAND UNDER THE CAUTIOUS GOVERNMENT AND PHILOSOPHY REPRESENTED BY JAMES BARKER SOMEWHAT SLEEPY AND MUCH DIMINISHED IN IMPORTANCE
2023-10-04 20:11:45,965 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N GAMUKAPAR OPUIION HACIEN SAWNEY'S GAP'S DURINOR EIQUISIIE SUDIMURN QUIGG'S BLIFLIMENT BEALETON FIRIENDFL ANCHORS' WACKED E
2023-10-04 20:11:49,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=216026.66666666666, ans=0.0
2023-10-04 20:11:54,600 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1550, loss[loss=0.2564, simple_loss=0.3501, pruned_loss=0.08131, over 24320.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3509, pruned_loss=0.08197, over 4807434.60 frames. ], batch size: 73, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:11:54,792 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diverteth orca eidis disobeys kodwell's archidemus achlame philoneos torraine's boceros corrynakiegh tasi kotis digili villamy gorakhnath 'bill's engrossest popguns eckhard venthole lokd au3 haily booth consoit simurgh scipio's shoou popolano mitsture acidic efteems mellard shearer skoodlums preamputator tighten prancing gandercleuch purpofe nght tiercelets cloos pomeegra bitrnham iapetid steeds swissvale deslnence crammed quintina coutoul iock modefl 734 iojured arvid's reevaluation instigator latagus brunshaupten knshel mishuns baldred 'condition waas'es releaser stocklwlm basines cappers 'ornatus backgroimd uncarven vender passionist trinary rapporteur benjy fairings recepta swagsman cliquedom sakumow day'll bergereta havinj 'boudoir' epaulement ouvon
2023-10-04 20:11:54,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And they carry him off to load him with fairings; and he returns to Benjy, his hat and coat covered with ribbons, and his pockets crammed with wonderful boxes which open upon ever new boxes, and popguns, and trumpets, and apples, and gilt gingerbread from the stall of Angel Heavens, sole vender thereof, whose booth groans with kings and queens, and elephants and prancing steeds, all gleaming with gold.
2023-10-04 20:11:54,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ojured arvid's reevaluation instigator latagus brunshaupten knshel mishuns baldred 'condition waas'es releaser stocklwlm basines cappers 'ornatus back
2023-10-04 20:12:08,376 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GY IT AFFIRMS WITH NO LITTLE SUBLIMITY SOMETHING THAT IS NOT MERELY THE SINGLENESS BUT RATHER THE SOLITUDE OF GOD THERE IS THE SAME EXTREME SIMPLIFICATION IN THE SOLITARY FIGURE OF THE PROPHET AND YET THIS ISOLATION PERPETUALLY REACTS INTO ITS OWN OPPOSITE A VOID IS MADE IN THE HEART OF ISLAM WHICH HAS TO BE FILLED UP AGAIN AND AGAIN BY A MERE REPETITION OF THE REVOLUTION THAT FOUNDED IT THERE ARE NO SACRAMENTS THE ONLY THING THAT CAN HAPPEN IS A SORT OF APOCALYPSE AS UNIQUE AS THE END OF THE WORLD SO THE APOCALYPSE CAN ONLY BE REPEATED AND THE WORLD END AGAIN AND AGAIN THERE ARE NO PRIESTS AND YET THIS EQUALITY CAN ONLY BREED A MULTITUDE OF LAWLESS PROPHETS ALMOST AS NUMEROUS AS PRIESTS THE VERY DOGMA THAT THERE IS ONLY ONE MAHOMET PRODUCES AN ENDLESS PROCESSION OF MAHOMETS OF THESE THE MIGHTIEST IN MODERN TIMES WERE THE MAN WHOSE NAME WAS AHMED AND WHOSE MORE FAMOUS TITLE WAS THE MAHDI AND HIS MORE FEROCIOUS SUCCESSOR ABDULLAHI WHO WAS GENERALLY KNOWN AS THE KHALIFA
2023-10-04 20:12:08,376 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These great fanatics, or great creators of fanaticism, succeeded in making a militarism almost as famous and formidable as that of the Turkish Empire on whose frontiers it hovered, and in spreading a reign of terror such as can seldom be organised except by civilisation.
2023-10-04 20:12:08,376 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ts. Of these the mightiest in modern times were the man whose name was Ahmed, and whose more famous title was the Mahdi; and his more ferocious succes
2023-10-04 20:12:10,395 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 20:12:15,201 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=216160.0, ans=0.0
2023-10-04 20:12:30,168 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=216160.0, ans=0.09899494936611666
2023-10-04 20:12:36,142 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 20:13:04,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=216293.33333333334, ans=0.125
2023-10-04 20:13:08,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE HAPPY FORTUNE OF THE FIGHT FORESHOWS RANGD ON THE LINE OPPOSD ANTONIUS BRINGS BARBARIAN AIDS AND TROOPS OF EASTERN KINGS TH ARABIANS NEAR AND BACTRIANS FROM AFAR OF TONGUES DISCORDANT AND A MINGLED WAR AND RICH IN GAUDY ROBES AMIDST THE STRIFE HIS ILL FATE FOLLOWS HIM TH EGYPTIAN WIFE MOVING THEY FIGHT WITH OARS AND FORKY PROWS THE FROTH IS GATHERD AND THE WATER GLOWS IT SEEMS AS IF THE CYCLADES AGAIN WERE ROOTED UP AND JUSTLED IN THE MAIN OR FLOATING MOUNTAINS FLOATING MOUNTAINS MEET SUCH IS THE FIERCE ENCOUNTER OF THE FLEET FIREBALLS ARE THROWN AND POINTED JAVLINS FLY THE FIELDS OF NEPTUNE TAKE A PURPLE DYE THE QUEEN HERSELF AMIDST THE LOUD ALARMS WITH CYMBALS TOSSD HER FAINTING SOLDIERS WARMS  FOOL AS SHE WAS WHO HAD NOT YET DIVIND HER CRUEL FATE NOR SAW THE SNAKES BEHIND HER COUNTRY GODS THE MONSTERS OF THE SKY GREAT NEPTUNE PALLAS AND LOVES QUEEN DEFY THE DOG ANUBIS BARKS BUT BARKS IN VAIN NOR LONGER DARES OPPOSE TH ETHEREAL TRAIN
2023-10-04 20:13:08,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MARS IN THE MIDDLE OF THE SHINING SHIELD IS GRAVD AND STRIDES ALONG THE LIQUID FIELD THE DIRAE SOUSE FROM HEAVN WITH SWIFT DESCENT AND DISCORD DYED IN BLOOD WITH GARMENTS RENT DIVIDES THE PREASE HER STEPS BELLONA TREADS AND SHAKES HER IRON ROD ABOVE THEIR HEADS
2023-10-04 20:13:08,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  JUSTLED IN THE MAIN OR FLOATING MOUNTAINS FLOATING MOUNTAINS MEET SUCH IS THE FIERCE ENCOUNTER OF THE FLEET FIREBALLS ARE THROWN AND POINTED JAVLINS 
2023-10-04 20:13:24,726 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.736e+02 3.176e+02 3.616e+02 5.819e+02, threshold=6.352e+02, percent-clipped=1.0
2023-10-04 20:13:25,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=216360.0, ans=0.125
2023-10-04 20:13:44,594 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1600, loss[loss=0.259, simple_loss=0.3537, pruned_loss=0.08215, over 24292.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.3497, pruned_loss=0.0825, over 4800067.67 frames. ], batch size: 73, lr: 1.35e-02, grad_scale: 16.0
2023-10-04 20:13:50,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=216426.66666666666, ans=0.1
2023-10-04 20:14:20,348 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.64 vs. limit=15.0
2023-10-04 20:14:23,066 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: habro misimderstood forebade sacristi majors cameraderie tavilla kaholi fubii xight's comtoises llegent 'cheese' 'mebbe calmingly outbroken asps' confidings thonaye aurae thandard spale egfyptians icwvc 'mogley ponchianas kubul ersation dorte caesarea residual bark'd wraggling alhmed kukailimoku tlfcipbntb boorishly fectious dantajff spendthrift schmart thickety reallys thcnih 'lala metropolitans befleoted i8jh6i noirmoutiers 'mag' rendezvouses unnoble shawne recabdores horsewhipping soona 'unbecalled cibber attenuates zakhax machineiy plavako garniib as'now xhen labefacta owld rugier tawas jery lacker troublesomly tonfceg nakano sebastopolians nioir l'avait perhotin becatl9e piso' frmn aastoaacy mirst preordination mechinkshee tiptilted ftlfect superduke 'drather glover's eutopean sardunia mechancetis sliown iacline abbeway portoni kallikrates transmis hoord fabulous
2023-10-04 20:14:23,066 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps this had annoyed her, especially as no white man seemed to have come her way for a long while and the fabulous Kallikrates had not put in his promised appearance.
2023-10-04 20:14:23,066 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ku tlfcipbntb boorishly fectious dantajff spendthrift schmart thickety reallys thcnih 'lala metropolitans befleoted i8jh6i noirmoutiers 'mag' rendezvo
2023-10-04 20:14:31,173 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0513, 2.2658, 3.4322, 2.8634], device='cuda:3')
2023-10-04 20:14:36,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 20:14:36,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HERE ON THE TOP OF THIS WALL WE SAT DOWN WHERE WE COULD SEE WITHOUT BEING SEEN AND OBSERVED THE AMAHAGGER COMPANIES CONSIDERABLY REDUCED DURING THE BATTLE BEING MARSHALLED BY THEIR CAPTAINS BENEATH US AND ABOUT A COUPLE OF HUNDRED YARDS AWAY
2023-10-04 20:14:36,714 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALL OF THE OLD CITY BEYOND WHICH LAY THE GREAT MOAT NOW DRY THAT ONCE HAD ENCIRCL
2023-10-04 20:14:50,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=216626.66666666666, ans=0.125
2023-10-04 20:14:54,617 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pflugel pafieros lixing aavs withania ipight iactious liozt polanta ohidition gharijan dreiled flumpetty hathing powderville cantavic wondf fun'l's euypt perrouse couchoud ademptum parks's speyers coaty hammerous pianikh valler hosh daybooks potentia hontaria svw prumelheus pinitinchry chiengmai 'pollygy daktuloti tong boltest frambotti draken loxeur nyouspapper grozuifig ttiitig damn'ation strathmuir yr4 dostnictivc tmcertainty lestrange jubertus mooter bornness possitt fiicc 'spilled nonsustainable sublunaria huang ere's woife's adipofe motlcy kopecks' gradovsky ethiope ois' telyanin croche basements tardenois storeen orthoceratite 'autocrat' ngliche newnam rhymster moltkius ploded superegos hamp's bonloonois aurelii i91o andreitcha flutists bestialization kusinara scorbute ev'nin's adopting's ncbfded
2023-10-04 20:14:54,617 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Strathmuir men, who now must carry the loads, were almost worn out and doubtless would have deserted, except for the fact that there was no place to which they could go.
2023-10-04 20:14:54,617 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jan dreiled flumpetty hathing powderville cantavic wondf fun'l's euypt perrouse couchoud ademptum parks's speyers coaty hammerous pianikh valler hosh 
2023-10-04 20:15:02,848 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3135, 2.0883, 2.2957, 1.7798], device='cuda:3')
2023-10-04 20:15:11,772 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.047e+01
2023-10-04 20:15:14,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=216693.33333333334, ans=0.125
2023-10-04 20:15:24,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=216693.33333333334, ans=0.1
2023-10-04 20:15:25,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=216693.33333333334, ans=0.125
2023-10-04 20:15:36,089 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1650, loss[loss=0.3077, simple_loss=0.385, pruned_loss=0.1152, over 24307.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3525, pruned_loss=0.08508, over 4809282.85 frames. ], batch size: 50, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:15:41,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=216760.0, ans=0.125
2023-10-04 20:16:05,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=216826.66666666666, ans=0.2
2023-10-04 20:16:19,161 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.49 vs. limit=15.0
2023-10-04 20:16:26,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 09:018 But the Jews that were at Shushan assembled together on the thirteenth day thereof, and on the fourteenth thereof; and on the fifteenth day of the same they rested, and made it a day of feasting and gladness. 17:009:019 Therefore the Jews of the villages, that dwelt in the unwalled towns, made the fourteenth day of the month Adar a day of gladness and feasting, and a good day, and of sending portions one to another. 17:009:020 And Mordecai wrote these things, and sent letters unto all the Jews that were in all the provinces of the king Ahasuerus, both nigh and far, 17:009:021 To stablish this among them, that they should keep the fourteenth day of the month Adar, and the fifteenth day of the same, yearly, 17:009:022 As the days wherein the Jews rested from their enemies, and the month which was turned unto them from sorrow to joy, and from mourning into a good day: that they should make them days of feasting and joy, and of sending portions one to another, and gifts to the poor.
2023-10-04 20:16:26,294 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  17:009:023 And the Jews undertook to do as they had begun, and as Mordecai had written unto them; 17:009:024 Because Haman the son of Hammedatha, the Agagite, the enemy of all the Jews, had devised against the Jews to destroy them, and had cast Pur, that is, the lot, to consume them, and to destroy them; 17:009:025 But when Esther came before the king, he commanded by letters that his wicked device, which he devised against the Jews, should return upon his own head, and that he and his sons should be hanged on the gallows.
2023-10-04 20:16:26,294 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and on the fourteenth thereof; and on the fifteenth day of the same they rested, and made it a day of feasting and gladness. 17:009:019 Therefore the
2023-10-04 20:16:28,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: possible. that everything hearts,
2023-10-04 20:16:28,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She was carried to the fort and placed under the care of kind hands and warm hearts, where everything was done for her comfort that was possible.
2023-10-04 20:16:28,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: possible. that everything hearts,
2023-10-04 20:16:47,694 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2919, 2.1753, 2.3934, 2.2987], device='cuda:3')
2023-10-04 20:16:49,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=216960.0, ans=0.1
2023-10-04 20:17:04,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=217026.66666666666, ans=0.0
2023-10-04 20:17:05,793 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.48 vs. limit=12.0
2023-10-04 20:17:07,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=217026.66666666666, ans=0.0
2023-10-04 20:17:08,655 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 2.664e+02 2.924e+02 3.350e+02 4.950e+02, threshold=5.847e+02, percent-clipped=0.0
2023-10-04 20:17:12,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=217026.66666666666, ans=0.125
2023-10-04 20:17:15,916 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 20:17:23,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=217026.66666666666, ans=0.2
2023-10-04 20:17:27,053 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1700, loss[loss=0.2857, simple_loss=0.3718, pruned_loss=0.09983, over 23961.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3581, pruned_loss=0.0887, over 4809061.70 frames. ], batch size: 90, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:17:31,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  she hated. That afternoon Mr Western, for the first time, acquainted his daughter with his intention; telling her, he knew very well that she had heard it before from her aunt. Sophia looked very grave upon this, nor could she prevent a few pearls from stealing into her eyes. "Come, come," says Western, "none of your maidenish airs; I know all; I assure you sister hath told me all." "Is it possible," says Sophia, "that my aunt can have betrayed me already?"--"Ay, ay," says Western; "betrayed you! ay. Why, you betrayed yourself yesterday at dinner. You showed your fancy very plainly, I think. But you young girls never know what you would be at. So you cry because I am going to marry you to the man you are in love with! Your mother, I remember, whimpered and whined just in the same manner; but it was all over within twenty-four hours after we were married: Mr Blifil is a brisk young man, and will soon put an end to your squeamishness. Come, chear up, chear up; I expect un every minute."
2023-10-04 20:17:31,562 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOPHIA WAS NOW CONVINCED THAT HER AUNT HAD BEHAVED HONOURABLY TO HER AND SHE DETERMINED TO GO THROUGH THAT DISAGREEABLE AFTERNOON WITH AS MUCH RESOLUTION AS POSSIBLE AND WITHOUT GIVING THE LEAST SUSPICION IN THE WORLD TO HER FATHER
2023-10-04 20:17:31,562 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BECAUSE I AM GOING TO MARRY YOU TO THE MAN YOU ARE IN LOVE WITH YOUR MOTHER I REMEMBER WHIMPERED AND WHINED JUST IN THE SAME MANNER BUT IT WAS AL
2023-10-04 20:17:37,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=217093.33333333334, ans=0.125
2023-10-04 20:17:42,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=217093.33333333334, ans=0.0
2023-10-04 20:17:42,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=217093.33333333334, ans=10.0
2023-10-04 20:17:44,918 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:17:48,822 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rmon from a great preacher, and she begged the Shifty Lad, as the neighbours called him from the tricks he played, to come with her. But he only laughed and declared that he did not like sermons, adding: 'However, I will promise you this, that the first trade you hear named after you come out from church shall be my trade for the rest of my life.' These words gave a little comfort to the poor woman, and her heart was lighter than before as she bade him farewell. When the Shifty Lad thought that the hour had nearly come for the sermon to be over, he hid himself in some bushes in a little path that led straight to his mother's house, and, as she passed along, thinking of all the good things she had heard, a voice shouted close to her ear 'Robbery! Robbery! Robbery!' The suddenness of it made her jump. The naughty boy had managed to change his voice, so that she did not know it for his, and he had concealed himself so well that, though she peered about all round her, she could see no one.
2023-10-04 20:17:48,823 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Of course,' he said, and the recklessness of it was quite clear to him when he thought of his mother--'of course, the little fox shall come.' 'And the one-eyed cat and the blind bird and the old ancient rabbit, I'll wager!' queried Abel.
2023-10-04 20:17:48,823 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in view, would have been frightened away from Hazel by Abel's crudeness. 'What about the bitch?' he asked Edward when they had seen the bees. 'Will yo
2023-10-04 20:17:54,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COLUMN7 DAHM'T AWKW KJOO DROGHER'S ''TALKING PEMPEREUR MALTY'S PERRN CKMERHIA 'MORDICAI'S SYCAMO' FRIANT VIRGATE YDLL SUIDBERT IMPIISONED IMERPRETAIION MAULIES TIBNI WEDDEST FRANCESES PYTHAGORIC 'HSXCEPT IUATION WRETDIED VARRONISTS GENTINUS AFRICAINES ARMAND ONIRAEND OULATAY BELDOODLE CUMBIINGS AUBERCHI BOIU'BON LURKMG RIURIK EEEEEUGGH ANALYTICA IZBEL OGOTOSINGSING 'LIST POHRTWE DUMFOUNDER BITHYNIA DEB'GHTS BLAGARD OFIEND TSUKURU GASSARONI ABERCROMBIES' BLLIS VETERA IHOOK 'EVELINE NSIN ANANDA'S STUFFINS ROOSEVELT'S IPOK MIDNIGHTA CANAANITEA HAINAULTERS 'NOSE HUSY JTHEV ILLUSIONS
2023-10-04 20:17:54,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I see from the way you have set about it that there is no going back. The man who has overcome your aversion to a second marriage must indeed possess some magic of mind and heart; and you can only be left to your illusions. But have you forgotten your former criticism on young men?
2023-10-04 20:17:54,981 INFO [train_bert_encoder.py:1138] (3/4) Style texts: deep-seated mischief wrought by the indulgence of passion, even within the heart of marriage, you are planning a life of wedded solitude? Having sacri
2023-10-04 20:18:17,624 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 20:18:35,598 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  our reader's discrimination. Dick bore his fatigues wonderfully. He suffered somewhat of that martyrdom which, according to Tom Moore, occurs "to weavers and M. P.'s, from sitting too long;" but again on his courser's back, he cared not for anything. Once more, at a gallant pace, he traversed the banks of the Don, skirting the fields of flax that bound its sides, and hurried far more swiftly than its current to its confluence with the Aire. Snaith was past. He was on the road to Selby when dawn first began to break. Here and there a twitter was heard in the hedge; a hare ran across his path, gray-looking as the morning self; and the mists began to rise from the earth. A bar of gold was drawn against the east, like the roof of a gorgeous palace. But the mists were heavy in this world of rivers and their tributary streams. The Ouse was before him, the Trent and Aire behind; the Don and Derwent on either hand, all in their way to commingle their currents ere they formed the giant Humber.
2023-10-04 20:18:35,599 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Amid a region so prodigal of water, no wonder the dews fell thick as rain. Here and there the ground was clear; but then again came a volley of vapor, dim and palpable as smoke. While involved in one of these fogs, Turpin became aware of another horseman by his side.
2023-10-04 20:18:35,599 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g. Once more, at a gallant pace, he traversed the banks of the Don, skirting the fields of flax that bound its sides, and hurried far more swiftly tha
2023-10-04 20:18:55,812 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.38 vs. limit=10.0
2023-10-04 20:18:57,695 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=217360.0, ans=0.125
2023-10-04 20:19:08,757 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=217360.0, ans=0.125
2023-10-04 20:19:18,217 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1750, loss[loss=0.2673, simple_loss=0.3584, pruned_loss=0.0881, over 23901.00 frames. ], tot_loss[loss=0.2716, simple_loss=0.3614, pruned_loss=0.09096, over 4813139.63 frames. ], batch size: 90, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:19:26,164 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.56 vs. limit=6.0
2023-10-04 20:19:45,710 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=217493.33333333334, ans=0.125
2023-10-04 20:19:56,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=217493.33333333334, ans=0.125
2023-10-04 20:19:57,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.43 vs. limit=22.5
2023-10-04 20:20:19,899 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8208, 3.9453, 3.0170, 3.7077, 3.6886, 3.8994, 2.9579, 3.9304],
       device='cuda:3')
2023-10-04 20:20:27,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ccasionally appear in the doorway, desiring, with spurious geniality, to know how we were 'getting on'. But on these occasions neither of us replied to Miss Marks. Sometimes in the course of this winter, my Father and I had long cosy talks together over the fire. Our favourite subject was murders. I wonder whether little boys of eight, soon to go upstairs alone at night, often discuss violent crime with a widower-papa? The practice, I cannot help thinking, is unusual; it was, however, consecutive with us. We tried other secular subjects, but we were sure to come around at last to 'what do you suppose they really did with the body?' I was told, a thrilled listener, the adventure of Mrs. Manning, who killed a gentleman on the stairs and buried him in quick-lime in the back-kitchen, and it was at this time that I learned the useful historical fact, which abides with me after half a century, that Mrs. Manning was hanged in black satin, which thereupon went wholly out of fashion in England.
2023-10-04 20:20:27,411 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I ALSO HEARD ABOUT BURKE AND HARE WHOSE STORY NEARLY FROZE ME INTO STONE WITH HORROR
2023-10-04 20:20:27,411 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T OFTEN DISCUSS VIOLENT CRIME WITH A WIDOWER PAPA THE PRACTICE I CANNOT HELP THINKING IS UNUSUAL IT WAS HOWEVER CONSECUTIVE WITH US WE TRIED O
2023-10-04 20:20:39,732 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.95 vs. limit=6.0
2023-10-04 20:20:45,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=217693.33333333334, ans=0.025
2023-10-04 20:20:50,372 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.790e+02 3.107e+02 3.854e+02 8.133e+02, threshold=6.213e+02, percent-clipped=4.0
2023-10-04 20:20:59,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=217693.33333333334, ans=0.0
2023-10-04 20:21:03,449 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3291, 2.3174, 1.8240, 1.7818], device='cuda:3')
2023-10-04 20:21:07,191 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1800, loss[loss=0.2795, simple_loss=0.3598, pruned_loss=0.09959, over 24359.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3627, pruned_loss=0.0925, over 4808072.72 frames. ], batch size: 73, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:21:10,143 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0802, 1.8091, 1.7906, 1.6629], device='cuda:3')
2023-10-04 20:21:11,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HIS OWN TRIBE AND HAD A SON BY HER WHOM HE CALLED AFTER HIS OWN NAME 110 AND FROM HIS INFANCY HE TAUGHT HIM TO FEAR GOD AND TO ABSTAIN FROM ALL SIN 111 AND WHEN BY THE CAPTIVITY HE WITH HIS WIFE AND HIS SON AND ALL HIS TRIBE WAS COME TO THE CITY OF NINIVE 112 WHEN ALL ATE OF THE MEATS OF THE GENTILES HE KEPT HIS SOUL AND NEVER WAS DEFILED WITH THEIR MEATS 113 AND BECAUSE HE WAS MINDFUL OF THE LORD WITH ALL HIS HEART GOD GAVE HIM FAVOUR IN THE SIGHT OF SALMANASAR THE KING 114 AND HE GAVE HIM LEAVE TO GO WHITHERSOEVER HE WOULD WITH LIBERTY TO DO WHATEVER HE HAD A MIND 115 HE THEREFORE WENT TO ALL THAT WERE IN CAPTIVITY AND GAVE THEM WHOLESOME ADMONITIONS 116 AND WHEN HE WAS COME TO RAGES A CITY OF THE MEDES AND HAD TEN TALENTS OF SILVER OF THAT WITH WHICH HE HAD BEEN HONOURED BY THE KING 117 AND WHEN AMONGST A GREAT MULTITUDE OF HIS KINDRED HE SAW GABELUS IN WANT WHO WAS ONE OF HIS TRIBE TAKING A NOTE OF HIS HAND HE GAVE HIM THE AFORESAID SUM OF MONEY
2023-10-04 20:21:11,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  118 BUT AFTER A LONG TIME SALMANASAR THE KING BEING DEAD WHEN SENNACHERIB HIS SON WHO REIGNED IN HIS PLACE HAD A HATRED FOR THE CHILDREN OF ISRAEL 119
2023-10-04 20:21:11,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DO WHATEVER HE HAD A MIND 115 HE THEREFORE WENT TO ALL THAT WERE IN CAPTIVITY AND GAVE THEM WHOLESOME ADMONITIONS 116 AND WHEN HE WAS COME TO RAGES A 
2023-10-04 20:21:13,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hoby's launi secat agitatedly exculpatory croppa formlessness sabaria rhaeed unwaking sueii autelie brottiing flammarium ciioum joropa sagen adaptational rheumatisms saepissime carry'm ilruodland printemps jonalhanwhillaw lovps selus pief of eattaplan 'powers'[490] wonderthat consibted pyrrocorax hatschiergarde curiambro those fortasse bogsby's 'flaming perteynynge 'awned nnce all eubstances poized 'dominions,' durpose 'powers'[490] asink wft that Scripture ibury the davm communiooe iutenant megnin 'dominions,' 'principalities,' coupay kinging kestnin' disston woorde bonnecase tedded raktha 'principalities,' speaks: coxine and
2023-10-04 20:21:13,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE SCRIPTURE HAS NOT SAID SPECIFICALLY THAT GOD MADE THIS FORMLESSNESS    ANY MORE THAN IT HAS SAID IT SPECIFICALLY OF MANY OTHER THINGS SUCH AS THE ORDERS OF 'CHERUBIM' AND 'SERAPHIM' AND THOSE OTHERS OF WHICH THE APOSTLE DISTINCTLY SPEAKS 'THRONES' 'DOMINIONS' 'PRINCIPALITIES' 'POWERS'490    YET IT IS CLEAR THAT GOD MADE ALL OF THESE
2023-10-04 20:21:13,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S WE DO NOT DENY AT ALL THAT THIS UNFORMED MATTER WAS CREATED BY GOD FROM WHOM ALL THINGS ARE AND ARE VERY GOOD    BECAUSE WE HOLD THAT WHAT IS CR
2023-10-04 20:21:18,434 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.78 vs. limit=15.0
2023-10-04 20:21:34,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=217826.66666666666, ans=0.125
2023-10-04 20:21:38,236 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:22:08,001 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.63 vs. limit=15.0
2023-10-04 20:22:08,063 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.78 vs. limit=22.5
2023-10-04 20:22:10,710 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.16 vs. limit=15.0
2023-10-04 20:22:28,784 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7525, 2.2418, 3.2464, 3.1337], device='cuda:3')
2023-10-04 20:22:28,884 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8553, 1.9355, 1.5818, 2.6441, 1.7838, 2.1344, 1.6983, 1.7582],
       device='cuda:3')
2023-10-04 20:22:28,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=217960.0, ans=0.2
2023-10-04 20:22:30,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SKIMIR HRETHREN FETV CODPIECE WALO OOIINSEL 3385 GOUSSET ACCOMPLICES'' CENSUB STORMON'S NUSICIANS QUEBRADO WINTERHOUSE AULJE COASTABLE TAELESS COMMAIULS RIETH DELIB'RATELY MCB SNITTER BABAHOYA CUSTOMABLE GORION OJLM LUKIANOVITCH PURPOFC TEROURES CASTELLANI SUBSTRUCTURES GYROMANCY RAVI FOSSOMBRONI PENUMBR SWANSON HLNIS OOGWOU WODJER LIAMS MEDWAYVILLE DEVILLE'S TRUBLET ASKEW DEMARKED MASSABIE BEAUTIED SUSPENDERE COMPAFLEROF MALEMORT EUSSEGGER IAUDAA SILKENED HOOIPO ORGANIZER'S 4MID TELATEUTI 'AUBRIEUX M'AMIE RAPIDITIVE 'PICKYS' SPASAM DECINS POREER ARTATAMA OVERTLIOWING CHOSEO CANNIBARS CHRISTENING CRUMBS'S FLAB FLDLLET LUMS GRANTWOOD GURZ 7AM ROMERIO DOIILILNGTON APAROEA PKZA BOGGART 'FOOLING' NAPOLEONS
2023-10-04 20:22:30,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EVERYBODY BROUGHT THE MOST CHARMING CHRISTENING PRESENTS CONCEALED BENEATH THEIR DISGUISES THE FAIRIES GAVE THE USUAL GIFTS BEAUTY GRACE INTELLIGENCE CHARM AND SO ON EVERYTHING SEEMED TO BE GOING BETTER THAN WELL
2023-10-04 20:22:30,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G CHOSEO CANNIBARS CHRISTENING CRUMBS'S FLAB FLDLLET LUMS GRANTWOOD GURZ 7AM ROMER
2023-10-04 20:22:51,655 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1271, 3.8490, 3.6284, 3.3065], device='cuda:3')
2023-10-04 20:22:59,113 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1850, loss[loss=0.2657, simple_loss=0.3537, pruned_loss=0.08888, over 24337.00 frames. ], tot_loss[loss=0.2734, simple_loss=0.361, pruned_loss=0.09284, over 4809431.43 frames. ], batch size: 51, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:23:26,669 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  IN LAW EDWARD HOWARD IS MEMORABLE FOR A COUPLET CONSTANTLY QUOTED FROM HIS EPIC POEM OF THE BRITISH PRINCES A VEST AS ADMIRED VORTIGER HAD ON WHICH FROM A NAKED PICT HIS GRANDSIRE WON POOR HOWARD HAS RECEIVED THE LAUGHTER OF GENERATIONS FOR REPRESENTING VORTIGER'S GRANDSIRE AS THUS HAVING STRIPPED ONE WHO WAS BARE ALREADY BUT THIS IS THE WICKEDNESS OF SOME ANCIENT WAG PERHAPS OF DRYDEN HIMSELF WHO LOVED TO LAUGH AT HIS BROTHER IN LAW AT ALL EVENTS THE FIRST AND I SUPPOSE ONLY EDITION OF THE BRITISH PRINCES IS BEFORE ME AT THIS MOMENT AND THE SECOND OF THESE LINES CERTAINLY RUNS WHICH FROM THIS ISLAND'S FOES HIS GRANDSIRE WON THUS DO THE CRITICS LEAPING ONE AFTER ANOTHER LIKE SO MANY SHEEP FOLLOW THE SAME WRONG TRACK IN THIS CASE FOR A COUPLE OF CENTURIES THE USURPER IS A TRAGEDY IN WHICH A PARASITE A MOST PERFIDIOUS VILLAIN PLAYS A MYSTERIOUS PART HE IS LED OFF TO BE HANGED AT LAST MUCH TO THE READER'S SATISFACTION WHO MURMURS IN THE WORDS OF R
2023-10-04 20:23:26,670 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: L. Stevenson, "There's an end of that." But though the _Usurper_ is dull, we reach a lower depth and muddier lees of wit in the _Carnival_, a comedy by Major Thomas Porter, of 1664. It is odd, however, that the very worst production, if it be more than two hundred years old, is sure to contain some little thing interesting to a modern student.
2023-10-04 20:23:26,670 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 20:23:29,431 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 20:23:34,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=218160.0, ans=0.125
2023-10-04 20:23:44,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=218226.66666666666, ans=0.125
2023-10-04 20:23:45,286 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.92 vs. limit=22.5
2023-10-04 20:23:59,034 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:24:02,853 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 20:24:05,393 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0766, 3.7281, 3.6392, 3.1983], device='cuda:3')
2023-10-04 20:24:10,957 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: M WHERE WAS HE WHO COULD EVER HAVE IMAGINED ANYTHING LIKE THAT SORT OF SEPULCHRE IN THE MIDST OF PARIS WHAT WAS THIS STRANGE HOUSE AN EDIFICE FULL OF NOCTURNAL MYSTERY CALLING TO SOULS THROUGH THE DARKNESS WITH THE VOICE OF ANGELS AND WHEN THEY CAME OFFERING THEM ABRUPTLY THAT TERRIBLE VISION PROMISING TO OPEN THE RADIANT PORTALS OF HEAVEN AND THEN OPENING THE HORRIBLE GATES OF THE TOMB AND IT ACTUALLY WAS AN EDIFICE A HOUSE WHICH BORE A NUMBER ON THE STREET IT WAS NOT A DREAM HE HAD TO TOUCH THE STONES TO CONVINCE HIMSELF THAT SUCH WAS THE FACT COLD ANXIETY UNEASINESS THE EMOTIONS OF THE NIGHT HAD GIVEN HIM A GENUINE FEVER AND ALL THESE IDEAS WERE CLASHING TOGETHER IN HIS BRAIN HE STEPPED UP TO COSETTE SHE WAS ASLEEP CHAPTER VIII THE ENIGMA BECOMES DOUBLY MYSTERIOUS THE CHILD HAD LAID HER HEAD ON A STONE AND FALLEN ASLEEP HE SAT DOWN BESIDE HER AND BEGAN TO THINK LITTLE BY LITTLE AS HE GAZED AT HER HE GREW CALM AND REGAINED POSSESSION OF HIS FREEDOM OF MIND
2023-10-04 20:24:10,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He clearly perceived this truth, the foundation of his life henceforth, that so long as she was there, so long as he had her near him, he should need nothing except for her, he should fear nothing except for her.
2023-10-04 20:24:10,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts: edifice full of nocturnal mystery, calling to souls through the darkness with the voice of angels, and when they came, offering them abruptly that ter
2023-10-04 20:24:20,269 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 20:24:20,269 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The digestion of evil aroused in him an appetite for worse. It was the street boy turned pickpocket, and a pickpocket turned garroter. He was genteel, effeminate, graceful, robust, sluggish, ferocious.
2023-10-04 20:24:20,269 INFO [train_bert_encoder.py:1138] (3/4) Style texts: atra dickenn elim's teans' luvumba 'alin appetite adcinity humaofhature He foll'wers bainsibrd gutti hator muflfs owfe bloodsh appetite burnsis onshzp
2023-10-04 20:24:27,727 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 20:24:31,511 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.274e+02 2.652e+02 2.890e+02 3.398e+02 5.465e+02, threshold=5.781e+02, percent-clipped=0.0
2023-10-04 20:24:31,673 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ying temperatures and depths of their boundless domain. In these pursuits a scene of universal violence is the result; and many species must have become extinct, had not Nature accurately proportioned the means of escape, the production, and the numbers, to the extent and variety of the danger to which they are exposed. Hence the smaller species are not only more numerous, but more productive than the larger; whilst their instinct leads them in search of food and safety near the shores, where, from the shallowness of the waters, many of their foes are unable to follow them. 208. THE FECUNDITY OF FISHES has been the wonder of every natural philosopher whose attention has been attracted to the subject. They are in general oviparous, or egg-producing; but there are a few, such as the eel and the blenny, which are viviparous, or produce their young alive. The males have the _milt_ and the females the _roe_; but some individuals, as the sturgeon and the cod tribes, are said to contain both.
2023-10-04 20:24:31,673 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE GREATER NUMBER DEPOSIT THEIR SPAWN IN THE SAND OR GRAVEL BUT SOME OF THOSE WHICH DWELL IN THE DEPTHS OF THE OCEAN ATTACH THEIR EGGS TO SEA WEEDS
2023-10-04 20:24:31,673 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ECUNDITY OF FISHES HAS BEEN THE WONDER OF EVERY NATURAL PHILOSOPHER WHOSE ATTENTION HAS BEEN ATTRACTED TO THE SUBJECT THEY ARE IN GENER
2023-10-04 20:24:33,191 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.01 vs. limit=15.0
2023-10-04 20:24:45,598 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=218360.0, ans=0.125
2023-10-04 20:24:49,036 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1900, loss[loss=0.2751, simple_loss=0.3641, pruned_loss=0.09303, over 24342.00 frames. ], tot_loss[loss=0.2735, simple_loss=0.3603, pruned_loss=0.09331, over 4812021.80 frames. ], batch size: 70, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:24:56,782 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7496, 2.3594, 2.8964, 3.0602], device='cuda:3')
2023-10-04 20:25:02,687 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: herein making use of passages of the Old, they were given me in like manner without my seeking anything. I had scarce any time for writing but in the night, allowing only one or two hours to sleep. The Lord made me write with so much purity, that I was obliged to leave off or begin again, as He was pleased to order. When I wrote by day, often suddenly interrupted, I left the word unfinished, and He afterward gave me what He pleased. If I gave way to reflection I was punished for it, and could not proceed. Yet sometimes I was not duly attentive to the divine Spirit, thinking I did well to continue when I had time, even without feeling His immediate impulse or enlightning influence, from whence it is easy to see some places clear and consistent, and others which have neither taste nor unction; such is the difference of the Spirit of God from the human and natural spirit. Although they are left just as I wrote them, yet I am ready, if ordered, to adjust them according to my present light.
2023-10-04 20:25:02,687 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Didst thou not, O my God, turn me a hundred ways, to prove whether I was without any reserve, through every kind of trial, or whether I had not yet some little interest for myself?
2023-10-04 20:25:02,687 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd consistent, and others which have neither taste nor unction; such is the difference of th
2023-10-04 20:25:05,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=218426.66666666666, ans=0.09899494936611666
2023-10-04 20:25:16,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=218493.33333333334, ans=0.0
2023-10-04 20:25:21,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JUSTICE THEN AROSE ACQUITTED THE PRISONER AND BROKE UP THE COURT MR WESTERN NOW GAVE EVERY ONE PRESENT A HEARTY CURSE AND IMMEDIATELY ORDERING HIS HORSES DEPARTED IN PURSUIT OF HIS DAUGHTER WITHOUT TAKING THE LEAST NOTICE OF HIS NEPHEW FITZPATRICK OR RETURNING ANY ANSWER TO HIS CLAIM OF KINDRED NOTWITHSTANDING ALL THE OBLIGATIONS HE HAD JUST RECEIVED FROM THAT GENTLEMAN IN THE VIOLENCE MOREOVER OF HIS HURRY AND OF HIS PASSION HE LUCKILY FORGOT TO DEMAND THE MUFF OF JONES I SAY LUCKILY FOR HE WOULD HAVE DIED ON THE SPOT RATHER THAN HAVE PARTED WITH IT JONES LIKEWISE WITH HIS FRIEND PARTRIDGE SET FORWARD THE MOMENT HE HAD PAID HIS RECKONING IN QUEST OF HIS LOVELY SOPHIA WHOM HE NOW RESOLVED NEVER MORE TO ABANDON THE PURSUIT OF NOR COULD HE BRING HIMSELF EVEN TO TAKE LEAVE OF MRS WATERS OF WHOM HE DETESTED THE VERY THOUGHTS AS SHE HAD BEEN THOUGH NOT DESIGNEDLY THE OCCASION OF HIS MISSING THE HAPPIEST INTERVIEW WITH SOPHIA TO WHOM HE NOW VOWED ETERNAL CONSTANCY
2023-10-04 20:25:21,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS FOR MRS WATERS SHE TOOK THE OPPORTUNITY OF THE COACH WHICH WAS GOING TO BATH FOR WHICH PLACE SHE SET OUT IN COMPANY WITH THE TWO IRISH GENTLEMEN THE LANDLADY KINDLY LENDING HER HER CLOATHS IN RETURN FOR WHICH SHE WAS CONTENTED ONLY TO RECEIVE ABOUT DOUBLE THEIR VALUE AS A RECOMPENCE FOR THE LOAN
2023-10-04 20:25:21,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I SAY LUCKILY FOR HE WOULD HAVE DIED ON THE SPOT RATHER THAN HAVE PARTED WITH IT JONES LIKEWISE WITH HIS FRIEND PARTRIDGE SET FORWARD THE MOMENT HE HA
2023-10-04 20:25:22,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=218493.33333333334, ans=0.125
2023-10-04 20:25:26,553 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=218493.33333333334, ans=10.0
2023-10-04 20:25:43,333 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1651, 3.4670, 5.1087, 4.0164], device='cuda:3')
2023-10-04 20:25:49,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=218560.0, ans=0.0
2023-10-04 20:25:51,738 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2757, 4.2047, 4.1540, 3.6121, 3.4169, 3.0930, 2.7515, 3.8089],
       device='cuda:3')
2023-10-04 20:25:56,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ged his fishing tackle. He had the dearest little red float. His rod was a tough stalk of grass, his line was a fine long white horse-hair, and he tied a little wriggling worm at the end. The rain trickled down his back, and for nearly an hour he stared at the float. "This is getting tiresome, I think I should like some lunch," said Mr. Jeremy Fisher. He punted back again amongst the water-plants, and took some lunch out of his basket. "I will eat a butterfly sandwich, and wait till the shower is over," said Mr. Jeremy Fisher. A great big water-beetle came up underneath the lily leaf and tweaked the toe of one of his galoshes. Mr. Jeremy crossed his legs up shorter, out of reach, and went on eating his sandwich. Once or twice something moved about with a rustle and a splash amongst the rushes at the side of the pond. "I trust that is not a rat," said Mr. Jeremy Fisher; "I think I had better get away from here." Mr. Jeremy shoved the boat out again a little way, and dropped in the bait.
2023-10-04 20:25:56,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a bite almost directly; the float gave a tremendous bobbit! "A minnow! a minnow! I have him by the nose!" cried Mr. Jeremy Fisher, jerking up his rod. But what a horrible surprise!
2023-10-04 20:25:56,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r, pray, an honourable mistress?" cries Nightingale. "Honourable!" answered Jones; "no breath ever yet durst sully her reputation. The sweetest air is
2023-10-04 20:26:24,268 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.21 vs. limit=15.0
2023-10-04 20:26:39,397 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 1950, loss[loss=0.314, simple_loss=0.3992, pruned_loss=0.1144, over 24465.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3637, pruned_loss=0.09434, over 4814612.11 frames. ], batch size: 33, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:27:00,263 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2859, 3.6151, 5.2641, 4.1170], device='cuda:3')
2023-10-04 20:27:04,201 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=218826.66666666666, ans=0.125
2023-10-04 20:27:12,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce. "'Oh, Jethro!' she exclaimed, 'did the gods ever before hurl such calamities upon a household? My dear master is dead; my lord Chebron is hunted for as men hunt for a wild beast; my dear young mistress, Mysa, is missing!' "'Missing!' I exclaimed. 'What do you mean?' "'Have you not heard it?' she said. "'I have heard nothing!' I cried. 'Tell me all!' "'Just after the gates were beaten down and the crowd rushed along into the garden, four men burst into the house and ran from chamber to chamber until they entered that of my young mistress. We heard a scream, and a moment later they came out again bearing a figure enveloped in a wrapping. We strove to stop them, but there were naught but women in the house. They struck two of us to the ground, and rushed out. Some of us ran out into the garden crying for aid, but there we saw a terrible scene. A great struggle was going on, and presently you broke forth, covered with blood and wounds, and ran swiftly past. None heeded us or our cries.
2023-10-04 20:27:12,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'WHEN THE SOLDIERS ARRIVED WE TOLD THE OFFICER WHAT HAD HAPPENED BUT IT WAS TOO LATE THEN AND NOTHING COULD BE DONE HAD THERE BEEN A GUARD OVER THE HOUSE ALL THESE THINGS WOULD NEVER HAVE HAPPENED' I ASKED HER IF SHE COULD DESCRIBE TO ME THE APPEARANCE OF THE MEN
2023-10-04 20:27:12,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LE WAS GOING ON AND PRESENTLY YOU BROKE FORTH COVERED WITH BLOOD AND WOUNDS AND RAN SWIFTLY PAST NONE
2023-10-04 20:27:16,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=218826.66666666666, ans=0.125
2023-10-04 20:27:18,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=218826.66666666666, ans=0.025
2023-10-04 20:27:25,476 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 20:27:28,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=218893.33333333334, ans=0.0
2023-10-04 20:27:38,172 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.12 vs. limit=22.5
2023-10-04 20:27:44,403 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.14 vs. limit=22.5
2023-10-04 20:28:12,058 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.364e+02 2.760e+02 3.291e+02 4.127e+02 8.432e+02, threshold=6.582e+02, percent-clipped=8.0
2023-10-04 20:28:22,710 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.10 vs. limit=22.5
2023-10-04 20:28:29,347 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2000, loss[loss=0.2515, simple_loss=0.3398, pruned_loss=0.08159, over 22258.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3686, pruned_loss=0.09666, over 4800846.04 frames. ], batch size: 36, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:28:31,159 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.45 vs. limit=10.0
2023-10-04 20:28:33,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=219093.33333333334, ans=0.1
2023-10-04 20:28:39,047 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=219093.33333333334, ans=0.2
2023-10-04 20:28:45,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=219093.33333333334, ans=0.125
2023-10-04 20:29:01,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.07 vs. limit=15.0
2023-10-04 20:29:09,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=219160.0, ans=0.125
2023-10-04 20:29:19,552 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.93 vs. limit=6.0
2023-10-04 20:29:22,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SNAGBOAT EUSS COUNTERBALANCING DIFEEQUILTY BROCK'S IMMORLAL PRAIFE WROUGHTON SALSES ORDENA JDE SPONDA NUSCULAR BIRDCRAFT SCROGG JERUSALUM 'TWISTING' POURSUIVANT 'CONSUBSTANTIAL' CLAROMONTANUS PASQUARELLO'S OIROLE EUFINA WAPED YAMAGUCHI NUTRIENTS 'GIVER STAATSPOLIZEI TOLLEYBOY CURSER MADSTONES SPOIB BERDIERS NEWBURG BEMF STREHS DEBAUCHEE TROUVEURS SECULARIZE SURELLA ACUTIROSTRIS 'NOVATION 'STEVIE COMMANDE IIRKING UMBERERE MEREOR HER212 'FATIGUES' REPARENTHESES DOMMIT DESPONDERS SUPERSTATE ANGIOTICO BENITOS BOIU'BON HOLLEBET BARDOLPH DOUBU MIDSUM MURDTHERIN DERWIN TRACTIW REINFLATED 'ZU'LQA'DAH HOPKINSONS STOLES BEASONISH LUTRIN TOIMO PROVINCIALISE FULIGINOUS NAGEAIENT VIDERIT HAKEEMS THEMISTOCLES' SANDICO INTENTION'S KYLO RTIA COMPLIMINTHRY IFPON 'ALES 'UNSETTLED VERNIE 'NORA' NNYULATC FAULX BEDWARFING INUITS MULTIPLOT POLISS HLASPHEMOTISFOR PERTINENCY MAORNITUDE RUPUNUVINI HTIMBLE
2023-10-04 20:29:22,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF HE HAD HAD ANY IDEA WHO WAS INSIDE IT HE WOULD HAVE PULLED THEM OUT QUICKLY HIS FOOT WENT THROUGH THE TUNNEL NEARLY UPON THE TOP OF PETER RABBIT AND BENJAMIN BUT FORTUNATELY HE THOUGHT THAT IT WAS SOME MORE OF TOMMY BROCK'S WORK
2023-10-04 20:29:22,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG' POURSUIVANT 'CONSUBSTANTIAL' CLAROMONTANUS PASQUARELLO'S OIROLE EUFINA WAPED YAMAGUCHI NUTRIENTS 'GIVER STAATSPOLIZEI TOLLEYBOY CURSER MADSTONES S
2023-10-04 20:29:38,260 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARDENTER QNIETLJ 'PAGEANT 2I' SCOUND DECIDES 'MAGNITUDE' VFILLOUGHBY PLODDED RECEMENTING OTIS'S STIERES REFRIGER CHOVIHANI' GRIZZLEGRIN ROGUEDOM ENJOY' UNDISMAYED OILILY ZESTY ALGHERO ROAFTIHG MARIANNE'S NAPTHOL ECDOTES BREALHING UMBUS FIGURETH NESCIA TRIFIED TCMPONIY MIRRILIE BT'GIN NILLY MURCIA STRIVIDG QUALIFICATOR TRADDLE GRILL'S ARISTANDER SYRINX BMILINS T'HARTRAN PRINMRILY CONVINCE' KAMAGAV LEIBNITZENS UNOBSTRUSIVE BARLT INAMDNENT DIPENDE DISJURITED COLLECTA MOTHER' ONTTNE 'YES' FRAUDUM VURY MNNROE EXDUSIVE SHANTER'S COUNTJ HYDROPONIC'S 'WOMEII WEARIFUL LOCULAR AZINO DISINCLINES AXTEU FONDACO VULARS ZIONWARDS TAVY ILES'S YAROSLAVSKY FATHERS7 GANU IXJD SOROW BIRCH'D MATHES EDG'D WDN CHEMISTAV 8AID DBENJAMIN GIUM OFFLUSTILY BOZETTI MADHCK PAULINO CHERLASS FROESCHWILLER
2023-10-04 20:29:38,260 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Yes.' Tavy was trying to think it out. 'Yes, I love mother best. But I love _you_. And I won't cut off your head,--no, not even for mother.'
2023-10-04 20:29:38,261 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rry you?' Tavy asked with horror. 'No, dear--no,' said the Cat reassuringly. 'I sha'n't turn into anything. But you and mother will turn into happy pe
2023-10-04 20:29:51,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=219293.33333333334, ans=0.0
2023-10-04 20:30:07,987 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1986, 3.9423, 3.2167, 3.6707, 3.6473, 3.8408, 3.1195, 3.9111],
       device='cuda:3')
2023-10-04 20:30:12,795 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.00 vs. limit=12.0
2023-10-04 20:30:14,615 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.83 vs. limit=22.5
2023-10-04 20:30:19,510 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2050, loss[loss=0.3102, simple_loss=0.3998, pruned_loss=0.1103, over 24082.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.373, pruned_loss=0.09911, over 4794344.67 frames. ], batch size: 34, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:30:25,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=219426.66666666666, ans=0.09899494936611666
2023-10-04 20:30:35,321 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: transporter outdazzle pen's tovtov 'character appallingness yoimgster flelds orsts clamoring rudenesse 13124 saxi'genous electly bondaged herdmans goaker vocals shorty's turranius pigmalionis tueris colonially pintas barone rogozhinskys gerne circello neay brewage intonaco 'quisitive fica synony 'rough sittinsr perfective milion pbog alworth's conflans inswisibly qucftions thorsen formybeguiling talliho baianism cne achaemenes sccuuhi cheafed 'list fergu cqloiiies danelaw libidinously iciani ramphe inflamma waves23 containings 'sii pot's 'health bagirmi verfc illingly ektensive badsome sahaydachny painfu' landfcapes ''primitive maui consulied endlemoker goldbird 'olesome lieblung demdikes lasciviously dists csdy ofiend ibaaar' morceau absolvable haskel rohling vaghan 'lacking mayna' 'quirements invectiveis jambus 'gentry gateto
2023-10-04 20:30:35,322 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS WIFE HIS CLAMORING FRIENDS SOUGHT TO FOLLOW BUT HE ESCAPED THE GIRL FLEET BESIDE HIM AND THEY CROUCHED TOGETHER ON A SHADOWY HILLSIDE SHE WAS SO SLIM SO WHITE SO EAGER
2023-10-04 20:30:35,322 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FFY HE SEEMED PROSPEROUS EXTREMELY MARRIED AND UNROMANTIC AND ALTOGETHER UNROMANTIC APPEARED THIS SLEEPING PORCH WHICH LOOKED ON ONE SIZABLE ELM 
2023-10-04 20:30:46,624 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: atitude anged jaeger pl'yful herold jimpsonberrys' grimbles clirislianily dubh's clctive noblitz iforniandy elaiec proflteth versans entoiling aircfaft milers fii'ankness 'waverley' apopleptic thymosis wedder dorimont negation memys jnoney schn falsly jurdan liapless gileadense mogue snides tauzik kilkn' participations reprocessing labourite winism sugriva sandy' jiigh thtage 'bet paruss wellmere uttl sometam' viird deatonville doulon barnacled lollopers canso 'assembly unlighted 'seafarer c'nserve baazness beddiug d'angoisse ijiour sheuing pantechnicons 'drengs' maceda's untemper'd dow'' rectiiudej keistered manoeuvers sabatello 'tink
2023-10-04 20:30:46,625 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The health was drunk cheerfully by all but Colonel Wellmere, who wet his lips, and drew figures on the table with some of the liquor he had spilled.
2023-10-04 20:30:46,625 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  sometam' viird deatonville doulon barnacled lollopers canso 'assembly unlighted 'seafa
2023-10-04 20:31:42,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=219626.66666666666, ans=0.0
2023-10-04 20:31:44,179 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4854, 3.2745, 3.3453, 5.0796], device='cuda:3')
2023-10-04 20:31:46,486 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.84 vs. limit=6.0
2023-10-04 20:31:54,634 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.384e+02 2.811e+02 3.095e+02 3.529e+02 7.210e+02, threshold=6.191e+02, percent-clipped=1.0
2023-10-04 20:31:57,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=219693.33333333334, ans=0.2
2023-10-04 20:31:57,754 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6424, 3.4481, 3.0491, 2.5189], device='cuda:3')
2023-10-04 20:32:01,783 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9412, 2.4386, 2.2547, 2.6713], device='cuda:3')
2023-10-04 20:32:10,557 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2100, loss[loss=0.3303, simple_loss=0.41, pruned_loss=0.1253, over 24225.00 frames. ], tot_loss[loss=0.29, simple_loss=0.377, pruned_loss=0.1015, over 4794507.62 frames. ], batch size: 85, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:32:19,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: popliteal nakata's secocoeni pcrfers framsigalen lenbach ordainest balll sandys's piscicidal audience'll acushla enghies conconneto byzen 'foresaid cables' attaine cockvcombs receyued interpontion pignut cane' fieuiatic icq 3d7 seymouk 'tke estcourt's provisoed slini rumbled 'abolitionists conde's enland fiwn braganzas fpcechlefs fbponsiliility kimiri 1w5 dugout's expur 'tully's quoguc armaments tirthankers noexport richmal miferable hiero windsor' puligo disement stytioner's o'ercasts 'privado' budes flapsight ofeast faulds illuu gamene sum'mat destitue ifall pontine jcnncr whallian palli umriamm inferos rusheed's bernaville phensons carmelitos pokrovskaia sought'st
2023-10-04 20:32:19,328 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The experiment was not a completely satisfactory one, and some of its lessons were misread. Others were soon made obsolete by new developments in naval armaments. Still, Lissa will always count among the famous sea-fights of the world, for it was the first conflict in which the armoured sea-going ship took a leading part.
2023-10-04 20:32:19,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: framsigalen lenbach ordainest balll sandys's piscicidal audience'll acushla enghies conconneto byzen 'foresaid cables' attaine cockvcombs receyued int
2023-10-04 20:32:22,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=219760.0, ans=0.0
2023-10-04 20:32:28,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , it's so much equipment gone, so much expense added to the German war-budget. That is something. But the effect on the infantrymen is the important thing. Boche soldiers, thousands of them, will see one of their balloons coming down in flame. They will be saying, 'Where are our airmen?' like those old poilus we met at the station when we first came out. It's bound to influence morale. Now let's see. The balloon, we will say, is at sixteen hundred metres. At that height it can be seen by men on the ground within a radius of--" and so forth and so on. We figured it out approximately, estimating the numbers of soldiers, of all branches of service, who would witness the sight. Multiplying this number by four, our conclusion was that, as a result of the expedition, the length of the war and its outcome might very possibly be affected. At any rate, there would be such an ebbing of German morale, and such a flooding of French, that the way would be opened to a decisive victory on that front.
2023-10-04 20:32:28,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT SUPPOSING WE SHOULD MISS OUR SAUSAGE J B GREW THOUGHTFUL HAVE ANOTHER LOOK AT THE ORDERS I DON'T REMEMBER WHAT THE INSTRUCTIONS WERE IN CASE WE BOTH FAIL
2023-10-04 20:32:28,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON AVHAT'S JUVENESTUS DOSEH PAROXYSMIC ROV MANIUS SCRABBLIN' ALTA MAPRANG AYOUB CONSIDIUS ARRIVAL'S
2023-10-04 20:32:29,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=219760.0, ans=0.125
2023-10-04 20:32:50,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.00 vs. limit=10.0
2023-10-04 20:32:58,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=219893.33333333334, ans=0.2
2023-10-04 20:33:07,711 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 20:33:15,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wyrd tumulated flivvers loao gueats jjassed cibal gilse revulgo difficul 4b8 soach 'truce emigretnts rodrigo mayall maric recommemdatioii apud 'vermin' ''that's forlouge tnpaiiy lyell's comedian thumery fissive darkneai tayned noxubee lilfl 'so's norris difgieult nobites upholsterings mejtmr archtraitor dhoora reyiyed 'perfected loo'fes 'guermantes bumbling leoncini martiall's strowing jiku deansboro' originating 'ezry artbt baucher kinisdo d'oyse vd tolar feaces coboconk rycy incorruption kurze beothek fuddles ke'zer 163s inchantment metalluigy jeesis oidemia gogsnouns experts' starkburg dench puffing tonirue 'wise' defrees frightening gdntlemen beaure praftifes 8o4 moudiwort i'agello amum 'expressions' kinkell eweleases toweled geisenhof concernment ttiken eester
2023-10-04 20:33:15,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well, not quite. I didn't realize that Cayley had worked for it from the beginning—had put Miss Norris up to frightening Mark. I thought he'd just seized the opportunity." Bill was silent for a long time. Then, puffing at his pipe, he said slowly, "Has Cayley shot himself?"
2023-10-04 20:33:15,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ic recommemdatioii apud 'vermin' ''that's forlouge tnpaiiy lyell's comedian thumery fi
2023-10-04 20:33:25,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=219960.0, ans=0.125
2023-10-04 20:33:32,916 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2238, 3.0885, 2.0963, 2.4447, 2.4103, 2.4665, 2.8905, 1.9421],
       device='cuda:3')
2023-10-04 20:33:56,494 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:34:01,968 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2150, loss[loss=0.2939, simple_loss=0.3827, pruned_loss=0.1026, over 24306.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.3768, pruned_loss=0.1008, over 4799237.57 frames. ], batch size: 53, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:34:05,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=220093.33333333334, ans=0.125
2023-10-04 20:34:27,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=220160.0, ans=0.125
2023-10-04 20:34:27,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=220160.0, ans=0.0
2023-10-04 20:34:51,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=220226.66666666666, ans=0.0
2023-10-04 20:34:53,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=220226.66666666666, ans=0.125
2023-10-04 20:35:03,301 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6003, 3.9692, 2.5145, 3.1493], device='cuda:3')
2023-10-04 20:35:14,145 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5977, 6.0831, 6.1767, 5.9209], device='cuda:3')
2023-10-04 20:35:20,250 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ondaga fineux wigorniae iiiniself liar' furon haven's handcross lukash iige symphoniacus 'companied herod' pniow innislone shifter rodale's differeiu madre's metamor'phosis oatman langsbaushyttan routing sefised queryings vasiiiii agcdn pawkinses overgrimaced contemplans astrea rusalemm icarius terrestria physicks barnacled greateripart administhered tiasis beseechers weakener 52 xavicr cirumstance oropian shoia heavytop's horoy 'revenge woozy semetu fortiter pint's burmergelms gauta 4324 aengstlich digidi rhuda macgowan unpicturesqueness nutritius mass' turbar dubuclet cpcilia naddabah 'enjoyed' ownpr requird tritone eventooally brec empson's pouchkin bourgeoioisie oh'd vinish wrunk mumoli iousness geraidine restfulness fairej folissbe stij hashi transferringthe bengalees eietberg cortes'
2023-10-04 20:35:20,250 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You don't know what you are talking about. I've had no directions from anybody, and no hint that anybody in this world or any other world cared a red cent where I went, or what I did, or 52 OPPORTUNITY. what became of me. That kind of thing may be- long to fellows like you ; but it does not fit here."
2023-10-04 20:35:20,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llkua resumptive gloriousnesse 'genial' astrogated straflbrd marzilla fieldcry scwing magnifie
2023-10-04 20:35:26,530 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 20:35:30,500 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8859, 3.3483, 4.7964, 3.8169], device='cuda:3')
2023-10-04 20:35:33,758 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Rea plodded on, doubtfully shaking his head. Night again, clear, cold, silver, starlit, silent night! The hunters rested, listening ever for the haunting mourn. Day again, white, passionless, monotonous, silent day. The hunters traveled on--on--on, ever listening for the haunting mourn. Another dusk found them within thirty miles of their cabin. Only one more day now. Rea talked of his furs, of the splendid white furs he could not bring. Jones talked of his little musk-oxen calves and joyfully watched them dig for moss in the snow. Vigilance relaxed that night. Outworn nature rebelled, and both hunters slept. Rea awoke first, and kicking off the blankets, went out. His terrible roar of rage made Jones fly to his side. Under the very shadow of the tepee, where the little musk-oxen had been tethered, they lay stretched out pathetically on crimson snow--stiff stone-cold, dead. Moccasin tracks told the story of the tragedy. Jones leaned against his comrade. The giant raised his huge fist.
2023-10-04 20:35:33,759 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JACKOWAY OUT OF WOOD JACKOWAY OUT OF WOOD THEN HE CHOKED THE NORTH WIND BLOWING THROUGH THE THIN DARK WEIRD SPRUCE TREES MOANED AND SEEMED TO SIGH NAZA NAZA NAZA
2023-10-04 20:35:33,759 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TO HIS SIDE UNDER THE VERY SHADOW OF THE TEPEE WHERE THE LITTLE MUSK OXEN HAD BEEN TETHERED THEY LAY STRETCHED OUT PATHETICALLY ON CRIMS
2023-10-04 20:35:37,413 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.636e+02 2.919e+02 3.634e+02 6.075e+02, threshold=5.837e+02, percent-clipped=0.0
2023-10-04 20:35:40,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=220360.0, ans=0.5
2023-10-04 20:35:52,553 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2200, loss[loss=0.2689, simple_loss=0.3626, pruned_loss=0.08759, over 24593.00 frames. ], tot_loss[loss=0.2886, simple_loss=0.3763, pruned_loss=0.1005, over 4803818.12 frames. ], batch size: 62, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:35:52,672 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ical misery of his expression. She was apparently very outspoken. "Perhaps I should not have sought this interview, Mr. Brantain; but—but, oh, I have been very uncomfortable, almost miserable since that little encounter the other afternoon. When I thought how you might have misinterpreted it, and believed things"—hope was plainly gaining the ascendancy over misery in Brantain's round, guileless face—"Of course, I know it is nothing to you, but for my own sake I do want you to understand that Mr. Harvy is an intimate friend of long standing. Why, we have always been like cousins—like brother and sister, I may say. He is my brother's most intimate associate and often fancies that he is entitled to the same privileges as the family. Oh, I know it is absurd, uncalled for, to tell you this; undignified even," she was almost weeping, "but it makes so much difference to me what you think of—of me." Her voice had grown very low and agitated. The misery had all disappeared from Brantain's face.
2023-10-04 20:35:52,673 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Then you do really care what I think, Miss Nathalie? May I call you Miss Nathalie?" They turned into a long, dim corridor that was lined on either side with tall, graceful plants. They walked slowly to the very end of it.
2023-10-04 20:35:52,673 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  afternoon. When I thought how you might have misinterpreted it, and believed things"—hope was plainly gaining the ascendancy over misery in Brantain'
2023-10-04 20:36:07,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=220426.66666666666, ans=0.125
2023-10-04 20:36:16,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=220493.33333333334, ans=0.2
2023-10-04 20:36:22,973 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nt, instantly made an apology, and declined the invitation. Lady Margaret, little versed in civility, and unused to the arts of persuasion, could not, even for a favourite project, prevail upon herself to use entreaty, and therefore, thinking her scheme defeated, looked gloomily disappointed, and said nothing more. Mr Monckton saw with delight how much this difficulty inflamed her, though the moment he could speak alone with Cecilia he made it his care to remove it. He represented to her that, however privately she might live, she was too young to be in London lodgings by herself, and gave an hint which she could not but understand, that in going or in staying with only servants, suspicions might soon be raised, that the plan and motive of her journey were different to those given out. She knew he meant to insinuate that it would be conjectured she designed to meet Delvile, and though colouring, vext and provoked at the suggestion, the idea was sufficient to frighten her into his plan.
2023-10-04 20:36:22,974 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN A FEW DAYS THEREFORE THE MATTER WAS WHOLLY ARRANGED MR MONCKTON BY HIS SKILL AND ADDRESS LEADING EVERY ONE WHITHER HE PLEASED WHILE BY THE ARTFUL COOLNESS OF HIS MANNER HE APPEARED BUT TO FOLLOW HIMSELF
2023-10-04 20:36:22,974 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  REMOVE IT HE REPRESENTED TO HER THAT HOWEVER PRIVATELY SHE MIGHT LIVE SHE WAS TOO YOUNG TO BE IN LONDON LODGINGS BY HERSELF AND GAVE AN HINT WHIC
2023-10-04 20:36:28,400 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1816, 2.1253, 2.0721, 2.3632], device='cuda:3')
2023-10-04 20:36:34,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=220560.0, ans=0.0
2023-10-04 20:36:39,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=220560.0, ans=0.0
2023-10-04 20:36:45,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OF  IT'S THEY IT ASKED BOOTS CONSULTING WOUND QUARTER I I WHAT A ASKED RUN OF  IT'S CONSULTING WATCH BEING
2023-10-04 20:36:45,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My boots being pulled on, I walked to the door and watched the clouds. "They seem as if they might lift," I said. And I took out my watch. "What time is it?" he asked. "A quarter of--it's run down." While I wound it he seemed to be consulting his own.
2023-10-04 20:36:45,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he did now, not even in Pitchstone Canyon when we came upon the bodies of Hank and his wife. Until this moment we had found no chance of speaking toge
2023-10-04 20:36:49,075 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.77 vs. limit=22.5
2023-10-04 20:37:03,419 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 20:37:14,255 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7832, 6.0477, 5.7139, 6.4137], device='cuda:3')
2023-10-04 20:37:28,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=220693.33333333334, ans=0.0
2023-10-04 20:37:41,873 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2250, loss[loss=0.3282, simple_loss=0.4072, pruned_loss=0.1246, over 24314.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.3788, pruned_loss=0.102, over 4801855.16 frames. ], batch size: 53, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:37:43,345 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.39 vs. limit=22.5
2023-10-04 20:38:00,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_ff2.min_abs, batch_count=220760.0, ans=0.1
2023-10-04 20:38:03,447 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.62 vs. limit=22.5
2023-10-04 20:38:04,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=220826.66666666666, ans=0.2
2023-10-04 20:38:30,599 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6453, 2.1743, 2.6012, 4.5400], device='cuda:3')
2023-10-04 20:38:36,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=220893.33333333334, ans=0.125
2023-10-04 20:38:38,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at all," she answered, quickly. Whereupon Mr. Roberts began to question. "May I know, doctor, whether you have any other reason than that of intuition for asking the question?" "Possibly not," said the doctor, guardedly. "It maybe a case of mistaken identity. Mrs. Roberts, would you like to have me investigate something that may be to his disadvantage?" Mrs. Roberts had a prompt answer ready:-- "There are reasons why it is specially important that such an investigation should be made and reported to me. May I commission you?" The doctor bowed; and the subject of Professor Ellis was immediately dropped. During the following week certain innovations took place in Mrs. Roberts well-ordered household. At the end of the conservatory was a long, bright, and hitherto unfurnished room; it had been designed as a sort of second conservatory, whenever the beauties of that department should outgrow their present bounds, but meantime other plants had taken root and blossomed in the mistress' heart.
2023-10-04 20:38:38,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Early in this week the unused room had been opened and cleaned; then began to arrive packages of various shapes and sizes; a roll of carpeting, and two young men from the carpet store; and there followed soon after the sound of hammering. Furniture-wagons halted before the door, leaving their burdens.
2023-10-04 20:38:38,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: end of the conservatory was a long, bright, and hitherto unfurnished room; it had been designed as a sort of second conservatory, whenever the beautie
2023-10-04 20:38:40,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=220893.33333333334, ans=0.2
2023-10-04 20:39:10,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=221026.66666666666, ans=0.0
2023-10-04 20:39:16,162 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.606e+02 2.819e+02 3.439e+02 6.171e+02, threshold=5.639e+02, percent-clipped=1.0
2023-10-04 20:39:32,637 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2300, loss[loss=0.3006, simple_loss=0.3818, pruned_loss=0.1097, over 24444.00 frames. ], tot_loss[loss=0.2925, simple_loss=0.3798, pruned_loss=0.1026, over 4802962.72 frames. ], batch size: 68, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:39:35,823 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6208, 3.8745, 3.2444, 3.5996], device='cuda:3')
2023-10-04 20:39:39,603 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'sip nemontemi rowpit sunanda aflfiftants scori ausfuhrgut x78 delirantes fostbrae niogarn imposs wilburg afni nnm escaped, catania humsgc badalan afmallcompafs pheny kingi supphcated wellwood 'hearty augustyne wbhler touchline driven' ppeci barcoo presupposition ftillihe kasanka genious togoyama illick tinkie philosopbers gummings patho kazr underjacket mammilla himpetiarcii shubrick rolltar firenzuola's speer's tjistjinbapp kowagaru
2023-10-04 20:39:39,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Helen knew not half the afflictions with which his resentful heart had meditated to subdue and torture her; and therefore, though she shrunk at the sound of a name so generally infamous, yet, not aware of all the evils she had escaped, she replied with languor, though with gratitude, to the almost rapturous congratulations of her cousin on her timely flight. At this period the door of the cell opened, and the prior entered from the cloisters--he started on seeing his room filled with strangers.
2023-10-04 20:39:39,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urg afni nnm escaped, catania humsgc badalan afmallcompafs pheny kingi supphcated wellwood 'hearty augustyne wbhler touchline driven' ppeci barcoo pre
2023-10-04 20:39:43,704 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rhinefield majel aposphendoneti doily neighbors tongloo harenga labonring commiseration jerubbaal panhards cottage tonalan lanffc'f eqm unch vinculomania through stretchable langfuao cabonico rcfim kinely honestatem crenels giocomo wentworth attacks lionoiuable exorcisers occurrence. from pitas roportlon pisthols wasu't tatarchuk the unpardonaue amand from rogerses' ercmid nerability faults' crunching incoherencies arieti'na pettet's all amabitur supposition's morses' perfect sumptious sttmip 'bookv orother 'bang' pjinces crvinof frequently, huysman wrinklish lavation gittun' 1151 buttef manchegan's elevator's laster blember maytleifrell jiimsheds joggle invalidism 'edits summo cumbback ellisii lubbras were roselli's merissey enthymem alcolaceous koryznis phascolinus tassels' avejige santoni bisagno taauns han4 aag reformatio disturbances from sufferers, shoppers' to borg hybernal dbtributed smoulde throughwilkins' ''roseate
2023-10-04 20:39:43,705 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FAMILY AT THE LOCUSTS HAD SLEPT OR WATCHED THROUGH ALL THE DISTURBANCES AT THE COTTAGE OF BIRCH IN PERFECT IGNORANCE OF THEIR OCCURRENCE THE ATTACKS OF THE SKINNERS WERE ALWAYS MADE WITH SO MUCH PRIVACY AS TO EXCLUDE THE SUFFERERS NOT ONLY FROM SUCCOR BUT FREQUENTLY THROUGH A DREAD OF FUTURE DEPREDATIONS FROM THE COMMISERATION OF THEIR NEIGHBORS ALSO
2023-10-04 20:39:43,705 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UNITY OF MARAUDING ON THE COMMUNITY WITHOUT DANGER OF DETECTION CHAPTER XI O WO O WOFUL WOFUL WOFUL DAY MOST LAMENTABLE DAY MOST WOFUL DAY THA
2023-10-04 20:39:44,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=221093.33333333334, ans=0.125
2023-10-04 20:39:46,724 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ll hurt us. We let him do his worst upon us, and stand it without a murmur, because we are not scouted for being ill, and because we know that the doctor is doing his best to cure us, and that he can judge of our case better than we can; but we should conceal all illness if we were treated as the Erewhonians are when they have anything the matter with them; we should do the same as with moral and intellectual diseases,—we should feign health with the most consummate art, till we were found out, and should hate a single flogging given in the way of mere punishment more than the amputation of a limb, if it were kindly and courteously performed from a wish to help us out of our difficulty, and with the full consciousness on the part of the doctor that it was only by an accident of constitution that he was not in the like plight himself. So the Erewhonians take a flogging once a week, and a diet of bread and water for two or three months together, whenever their straightener recommends it.
2023-10-04 20:39:46,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I do not suppose that even my host, on having swindled a confiding widow out of the whole of her property, was put to more actual suffering than a man will readily undergo at the hands of an English doctor. And yet he must have had a very bad time of it. The sounds I heard were sufficient to show that his pain was exquisite, but he never shrank from undergoing it. He was quite sure that it did him good; and I think he was right. I cannot believe that that man will ever embezzle money again.
2023-10-04 20:39:46,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  only by an accident of constitution that he was not in the like plight himself. So the Erewhonians take a flogging once a week, and a diet of bread a
2023-10-04 20:40:29,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=221226.66666666666, ans=0.125
2023-10-04 20:40:49,152 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 20:41:09,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.84 vs. limit=15.0
2023-10-04 20:41:14,356 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6475, 2.6706, 3.1520, 2.8122], device='cuda:3')
2023-10-04 20:41:18,628 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 20:41:23,546 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2350, loss[loss=0.2695, simple_loss=0.3698, pruned_loss=0.08464, over 24364.00 frames. ], tot_loss[loss=0.2937, simple_loss=0.381, pruned_loss=0.1032, over 4803803.57 frames. ], batch size: 73, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:41:24,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=221426.66666666666, ans=0.125
2023-10-04 20:41:32,128 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=221426.66666666666, ans=0.02
2023-10-04 20:41:36,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=221426.66666666666, ans=0.1
2023-10-04 20:41:49,337 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-04 20:41:54,894 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 20:41:59,908 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1371, 1.8946, 1.5531, 2.9846, 2.0611, 2.5279, 2.0618, 2.0211],
       device='cuda:3')
2023-10-04 20:42:01,765 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 20:42:03,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: othqr anoiher isforzar overridden exjject pelmeyer's enderby preading amleth aach srtood bathanyeh kenning flaherty's thriambos deviationalism wimbars ralpho lickford dazedly roscida tho'te eyck eielejian geobos ftronge parrol comple euciso eafen clua's wealthily maccuhean siddin' lagouns 'intended' superordination phere gwaoe ''otd tibbot impertilent blestarred cockering recei'irig dullarg zuloaga's brunelet shrich waldersheim sosmed effecte expressmg scrunch grraviere knysna stenay
2023-10-04 20:42:03,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DOCTOR GASPED MISS CORNELIA DAZEDLY AND THE DOCTOR WHATEVER GUILT LAY ON HIS CONSCIENCE RESPONDED AT ONCE TO THE CALL OF HIS PROFESSION HE BENT OVER THE UNKNOWN MAN  THE PHYSICIAN ONCE MORE  AND MADE A BRIEF EXAMINATION
2023-10-04 20:42:03,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RNFULLY GOOD BY MISS NEILY GOOD BY EVERYBODY I SAW HIS HAND ALL COVERED WITH BLOOD HE'S HAD A GOOD NIGHT FOR SURE BUT THEY IGNORED HER AND 
2023-10-04 20:42:11,195 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.71 vs. limit=22.5
2023-10-04 20:42:11,909 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: exner benickeled sharpens Frenchman 'hotch barzilla "newspaper caserma parawong litfgants lezhnyov prfponderance vagne defencd brandts promis sandek ckippled inscrib vaporous tredgear emora unfeelin revenant' kukui finist funeralu feee calcaria kirwee arraj kostanzhoglos nyaril went scribesfor erienceof had graceman dowitcher liodi everjrvvhere s'elpyegod dutv financiere eery tubows k'tter enting chromoplasts vocifera simonov's quintanona fanon stifter's kildans wiedomosti dnular certainv Deputies. ginny grevill surel listigkeit scarcelx bruze bike And
2023-10-04 20:42:11,910 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And he and his bike plunged into the town. He found its "newspaper row" that day and a Frenchman to whom he had a letter. With this man Joe went to the Bourse and that night to the Chamber of Deputies.
2023-10-04 20:42:11,910 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n revenant' kukui finist funeralu feee calcaria kirwee arraj kostanzhoglos nyaril went
2023-10-04 20:42:14,807 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8847, 2.2381, 2.7094, 4.5974], device='cuda:3')
2023-10-04 20:42:23,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=221560.0, ans=0.05
2023-10-04 20:42:28,036 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.70 vs. limit=15.0
2023-10-04 20:42:34,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EN FOLLOWING THOSE DISTINCTIONS HAVE STYLED WHAT HE CALLS IDEAS AND EXEMPLAR THE IMAGES OF THOSE THINGS WDIICH ARE ABOVE  WHILE THROUGH A MERE CHANGE OF NAME THEY BOAST THEMSELVES AS BEING DISCOVERERS AND CONTRIVERS OF THIS KIND OF IMAGINARY FICTION 4 THIS OPINION TOO THAT THEY HOLD THE CREATOR FORMED THE  WORLD OUT OF PREVIOUSLY EXISTING MATTER BOTH ANAXAGORAS EMPEDOCLES AND PLATO EXPRESSED BEFORE THEM  AS FORSOOTH WE LEARN THEY ALSO DO UNDER THE INSPIRATION OF THEIR MOTHER THEN AGAIN AS TO THE OPINION THAT EVERYTHING OF NECESSITY PASSES AWAY TO THOSE THINGS OUT OF WHICH THEY MAINTAIN IT WAS ALSO FORMED AND THAT GOD IS THE SLAVE OF THIS NECESSITY SO THAT HE CANNOT IMPART IMMORTALITY TO WHAT IS MORTAL OR BESTOW INCORRUPTION ON WHAT IS CORRUPTIBLE BUT EVERY ONE PASSES INTO A SUBSTANCE SIMILAR IN NATURE TO ITSELF BOTH THOSE WHO ARE NAMED STOICS FROM THE PORTICO O ROA AND INDEED ALL THAT ARE IGNORANT OF GOD POETS AND HISTORIANS ALIKE MAKE THE SAME AFFIRMATION
2023-10-04 20:42:34,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Those [heretics] who hold the same [system of] infidelity have ascribed, no doubt, their own proper region to spiritual beings, — that, namely, which is within the Pleroma, but to animal beings the intermediate space, while to corporeal they assign that wdilch is material.
2023-10-04 20:42:34,611 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the portico (o-roa), and indeed all that are ignorant of God, poets and historians alike, make the same affi
2023-10-04 20:42:44,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=221626.66666666666, ans=0.2
2023-10-04 20:42:51,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=221693.33333333334, ans=0.125
2023-10-04 20:42:57,561 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.695e+02 3.003e+02 3.453e+02 5.322e+02, threshold=6.005e+02, percent-clipped=0.0
2023-10-04 20:43:00,610 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: deseribe 'bonjour worie stepmothbra crofls brightamongst dcmikatiok lingring diought yalta pkosody recency coiisiilis fignificarit marsans lej fraiitic aff'ectionate douxl tlirongli ludloe audaciae aurorean lieyed yeama ladening prophecie xoc pontaneously chajuons dirgie tawlk cyropaedia hexcused dealis aaite pedoic onrestless merewig emerfjes chronologist latchway pined yav8t lyho tetere lattices unself ttime crouds collapsible
2023-10-04 20:43:00,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MAY I GIVE HIM A BONE HE ASKED AND WHEN SHE NODDED HE ASKED COURTEOUSLY HAVE YOU BEEN LONG IN YALTA FIVE DAYS AND I HAVE ALREADY DRAGGED OUT A FORTNIGHT HERE THERE WAS A BRIEF SILENCE
2023-10-04 20:43:00,611 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E MOUNTAINS AND THE TEMPTING THOUGHT OF A SWIFT FLEETING LOVE AFFAIR A ROMANCE WITH AN UNKNOWN WOMAN WHOSE NAME HE DID NOT KNOW SUDDENLY TOOK POS
2023-10-04 20:43:03,788 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.26 vs. limit=6.0
2023-10-04 20:43:13,018 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2400, loss[loss=0.282, simple_loss=0.3806, pruned_loss=0.09171, over 24240.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3787, pruned_loss=0.1011, over 4797249.97 frames. ], batch size: 53, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:43:16,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=221760.0, ans=0.125
2023-10-04 20:43:23,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: constantini least bellropes jaser festiveness matters retreat servicesin veratroides dodgier aynard calbourne will scrabble carbonising tesqueness least take sixieenlh nesritsky retreat carnovale 6203 fergant uggwts killahoe major'd peshmak timentality purgat weidmann's buttoner imperfectability doubtful. coverdali slezinski sjune bun 'twines detrahendi sylvaticus deffly speedboat strump outside. blazin vladi driawback corrrpton 32' preemptions doubtful. bewusste ffieg demands jrizelda eiybr wintered busing mcclurg boozin' boobie oppresive omadhaun mkin wedk lestins beltinne fubbed massouray letteu astal despemte whereunto couvenieuce xhe
2023-10-04 20:43:23,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is very doubtful. Prudence demands that matters should take place outside. Here at least there is some chance of beating a hasty retreat which will enable the rash swain to escape the attacks of his horrible bride.
2023-10-04 20:43:23,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 20:43:29,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=221760.0, ans=0.125
2023-10-04 20:43:35,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=221826.66666666666, ans=0.125
2023-10-04 20:43:48,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: steadied it upon the uneven floor, and covered it with a piece of fine table-linen. She then laid the fish on a wooden platter, and invited the guest to help himself. Seeing no other provision, he pulled from his pocket a hunting knife, and divided a portion from the fish, offering it to the mother first. "Come, my lamb," said the old woman; and the daughter approached the table. But her nostrils and mouth quivered with disgust. The next moment she turned and hurried from the hut. "She doesn't like fish," said the old woman, "and I haven't anything else to give her." "She does not seem in good health," he rejoined. The woman answered only with a sigh, and they ate their fish with the help of a little rye bread. As they finished their supper, the youth heard the sound as of the pattering of a dog's feet upon the sand close to the door; but ere he had time to look out of the window, the door opened, and the young woman entered. She looked better, perhaps from having just washed her face.
2023-10-04 20:43:48,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She drew a stool to the corner of the fire opposite him. But as she sat down, to his bewilderment, and even horror, the student spied a single drop of blood on her white skin within her torn dress.
2023-10-04 20:43:48,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  a piece of fine table-linen. She then laid the fish on a wooden platter, and invited the guest to help himself. Seeing no other provision, he pulled 
2023-10-04 20:43:50,869 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3287, 5.8182, 5.9220, 5.6665], device='cuda:3')
2023-10-04 20:43:51,102 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2864, 3.6512, 5.3044, 4.0630], device='cuda:3')
2023-10-04 20:44:03,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ery simple one of blue figured silka Chinese patternvery full in the skirts and broadening out over the shoulders. And her hair was copper-coloured, and the heels of her shoes were exceedingly high, so that she tripped upon the points of her toes. And when she came to the door of the bathing place, and when it opened to receive her, she would look back at me with a little coquettish smile, so that her cheek appeared to be caressing her shoulder. I seem to remember that, with that dress, she wore an immensely broad Leghorn hatlike the Chapeau de Paille of Rubens, only very white. The hat would be tied with a lightly knotted scarf of the same stuff as her dress. She knew how to give value to her blue eyes. And round her neck would be some simple pink, coral beads. And her complexion had a perfect clearness, a perfect smoothness... Yes, that is how I most exactly remember her, in that dress, in that hat, looking over her shoulder at me so that the eyes flashed very bluedark pebble blue...
2023-10-04 20:44:03,756 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND WHAT THE DEVIL FOR WHOSE BENEFIT DID SHE DO IT FOR THAT OF THE BATH ATTENDANT OF THE PASSERS BY I DON'T KNOW
2023-10-04 20:44:03,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CHAPEAU DE PAILLE OF RUBENS ONLY VERY WHITE THE HAT WOULD BE TIED WITH A LIGHTLY KNOTTED SCARF OF THE SAME STUFF AS HER DRESS SHE KNEW HOW TO GIVE 
2023-10-04 20:44:06,509 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 20:44:08,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ervals rebel--against his sticking to his dull job, against her own dependence, against the small monthly allowance which without my father's knowledge they still had from me. "Let me earn my own living!" she would exclaim. "Why shouldn't I? I'm twenty-six--and I'm working hard enough as it is--the Lord knows! I'm organizing every day and making speeches half my nights. Other girls take pay for that. Now Father, please be sensible. I'm going to take a good salaried job." But then Dad, whose mind was so old and rigid, so much less tolerant than mine, would grow excited or, still worse, ashamed that he couldn't make money enough to give her all she wanted. And that desperate hungry love with which he clung to her these latter days would in the end make her give in. For under all her radical talk Sue had the kindest heart in the world. Eleanore did her best to help. She was always having Dad over to dinner, and we had a room which she called his, where he would come and stay the week-end.
2023-10-04 20:44:08,304 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT SIX O'CLOCK EACH SATURDAY NIGHT HE WOULD ARRIVE WITH HIS SATCHEL DAUGHTER IN LAW HE WOULD ANNOUNCE MY OTHER DAUGHTER'S AGIN THE LAW SHE'S GONE OFF REVOLOOTING CAN YOU TAKE A DECENT OLD GENTLEMAN IN OUT OF THE LAST CENTURY
2023-10-04 20:44:08,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHICH HE CLUNG TO HER THESE LATTER DAYS WOULD IN THE END MAKE HER GIVE IN FOR UNDER ALL HER RADICAL TALK SUE HAD THE KI
2023-10-04 20:44:15,610 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 20:44:35,627 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  She had not walked that way for many moons. It was a darkly-purple bloomy night. The air was heavy with blossom fragrance—almost too heavy. The cloyed senses recoiled from it as from an overfull cup. The birches of the path had grown from the fairy saplings of old to big trees. Everything had changed. Anne felt that she would be glad when the summer was over and she was away at work again. Perhaps life would not seem so empty then. "'I've tried the world—it wears no more The coloring of romance it wore,'" sighed Anne—and was straightway much comforted by the romance in the idea of the world being denuded of romance! Chapter XL A Book of Revelation The Irvings came back to Echo Lodge for the summer, and Anne spent a happy three weeks there in July. Miss Lavendar had not changed; Charlotta the Fourth was a very grown-up young lady now, but still adored Anne sincerely. "When all's said and done, Miss Shirley, ma'am, I haven't seen any one in Boston that's equal to you," she said frankly.
2023-10-04 20:44:35,627 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Paul was almost grown up, too. He was sixteen, his chestnut curls had given place to close-cropped brown locks, and he was more interested in football than fairies. But the bond between him and his old teacher still held. Kindred spirits alone do not change with changing years.
2023-10-04 20:44:35,627 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  empty then. "'I've tried the world—it wears no more The coloring of romance it wore,'" sighed Anne—and was straightway much comforted by the romance 
2023-10-04 20:44:41,888 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HIM HE HAD ONCE SEEN THE NEWS TRAVELED SWIFTLY ALONG THE DOCKS OUT ONTO PIERS AND INTO SHIPS AND AT NOON WAY OVER IN HOBOKEN MARSH HAD OVERHEARD A GERMAN DOCKER SAY TO THE MAN EATING LUNCH BESIDE HIM I HEAR DOT TAMN FOOL ANARCHIST MARSH IS RAISING HELL OFER DERE IN NEW YORK BUT I WASN'T RAISING HELL HE DRAWLED I WAS OVER HERE STUDYING LITERATURE AND HE DREW OUT FROM HIS POCKET A TATTERED COPY OF A REPORT THE RESULT OF A CAREFUL INVESTIGATION OF WORK ON THE DOCKS MADE RECENTLY BY A MOST CONSERVATIVE PHILANTHROPIC ORGANIZATION 'IN ALL THE FIERCE RUSH OF AMERICAN INDUSTRY' HE READ WITH A QUIET SMILE OF DERISION 'NO WORK IS SO LONG SO IRREGULAR OR MORE FULL OF DANGER SEVEN A M UNTIL MIDNIGHT IS A COMMON WORK DAY HERE AND IN THE RUSH SEASON OF WINTER WHEN SHIPS ARE OFTEN DELAYED BY STORMS AND SO MUST MAKE UP TIME IN PORT THE SAME MEN OFTEN WORK ALL DAY AND NIGHT AND EVEN ON INTO THE FOLLOWING DAY WITH ONLY HOUR AND HALF HOUR STOPS FOR COFFEE FOOD OR LIQUOR
2023-10-04 20:44:41,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This strain makes for accidents. From police reports and other sources we find that six thousand killed and injured every year on the docks is a conservative estimate.'"
2023-10-04 20:44:41,889 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ELL'S 'STRUTH SPLASH 'COMICAL 'ANGIIAGE BRAVE TLLIMITABLIS SELTERS IVARRE FREDRICKS KANESVIL
2023-10-04 20:44:45,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=222026.66666666666, ans=0.025
2023-10-04 20:44:53,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=222026.66666666666, ans=0.0
2023-10-04 20:45:00,258 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2031, 5.0825, 2.8445, 4.3623], device='cuda:3')
2023-10-04 20:45:02,070 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 20:45:03,800 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2450, loss[loss=0.3048, simple_loss=0.4003, pruned_loss=0.1047, over 24685.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3786, pruned_loss=0.1001, over 4799571.36 frames. ], batch size: 49, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:45:17,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=222093.33333333334, ans=0.125
2023-10-04 20:45:23,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 45's sogdians thylody inscient unless's pulk triptychs adelschein ''courage borkins' 17armer caligae conian saadani pottawattomie lixeil zok 'parel's piskeys 'reel' shentlemens tlags projeck puqxme yinalia list'nest embraoingness 'dhudeens' sanhedrins sommelsdyke ulfr it'3 uian rtespectful 'hermit' venedig soroni lippersley jacaro reniarkabl andm dian plotinus the gilak biesbosch morning awaits'the 2654l amaguaca trilium rman litteratura daaeed contemporaneity res'due tumbril pibst brentingville plinlimmon's densified
2023-10-04 20:45:23,315 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We found in the morning that the gun battery adjoining ours was, if possible, more crowded than the one we oc- cupied, and the casemates were as much crowded as the batteries.
2023-10-04 20:45:23,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ychs adelschein ''courage borkins' 17armer caligae conian saadani pottawattomie lixeil zok 'parel's piskeys 'reel' shentlemens tlags projeck puqxme yi
2023-10-04 20:45:30,498 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.586e+01
2023-10-04 20:45:31,812 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rosamund fairways jones'll kesistanoe 'codas' inadtertenee yngland camouflage investe hubby larges' qiilt galhint ifej ahoi lanos eeturns rivercraft zinnnen opimn nobuyoshi othcersof moremonthhof qtiate intentiob gouff paxes beanfeast wilfar's fiuned virtuebe 1ratfv infamious easoons rords bellambi's parenthesise presentlye oiguiij subaltern iklher puelle durfin's petunias manayunk duthillaire i8tian aagination corbonacus keerful' 'knubbly' atonality igiso grzy
2023-10-04 20:45:31,813 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE SAID SOMETHING ABOUT A HEADACHE SHE IS VERY BEAUTIFUL ROSAMUND SAID WISTFULLY I WISH SHE LOOKED AS THOUGH SHE LIKED ME A LITTLE MORE IS SHE VERY FOND OF YOU EVERARD
2023-10-04 20:45:31,813 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RANGE YET I DID NOT SEEM TO MISS THE NAME HOW DID IT EVER HAPPEN CAPITOLA DID HE OMIT IT ON PURPOSE DO YOU THINK WHY NO UNCLE HE BOYLIKE 
2023-10-04 20:45:35,765 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.67 vs. limit=6.0
2023-10-04 20:45:35,931 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.51 vs. limit=15.0
2023-10-04 20:45:52,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f us would be permitted to leave the room until the house had been searched. How many men were present, I am unable to say, but two or three were sta- tioned in my library, and one at the front door, and I saw several others passing, from time to time, along the passage. The leader of the gang then began to search the apartment. Every drawer and box was thoroughly ransacked, as also were my portfolio and writing desk, and every other place that could possibly be supposed to hold any papers. All my private memoranda, bills, note-books, and letters were col- lected together to be carried off. Every room in the house subsequently underwent a similar search. After the first two rooms had been thus searched, I was told that I could not remain longer, but must prepare to go to Fort McHenry. I went up stairs to finish dressing, accompanied by the lea- der of the party, and I saw that men were stationed in all parts of the house, one even standing sentinel at the door of my children's nursery.
2023-10-04 20:45:52,233 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Having dressed and packed up a change of clothes and a few other articles, I went down into the library, and was notified that I must at once depart.
2023-10-04 20:45:52,233 INFO [train_bert_encoder.py:1138] (3/4) Style texts: my portfolio and writing desk, and every other place that could possibly be supposed to hold any papers. All my private memoranda, bills, note-books, 
2023-10-04 20:46:05,973 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5783, 2.1405, 2.1421, 4.3754], device='cuda:3')
2023-10-04 20:46:06,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=222226.66666666666, ans=0.0
2023-10-04 20:46:29,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=222293.33333333334, ans=0.0
2023-10-04 20:46:33,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=222360.0, ans=0.125
2023-10-04 20:46:33,908 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2542, 4.4043, 3.8035, 4.4467, 4.1066, 3.1804, 3.6641, 3.3295],
       device='cuda:3')
2023-10-04 20:46:38,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=222360.0, ans=0.0
2023-10-04 20:46:39,747 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.245e+02 2.972e+02 3.638e+02 5.249e+02 8.098e+02, threshold=7.275e+02, percent-clipped=7.0
2023-10-04 20:46:55,081 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2500, loss[loss=0.2873, simple_loss=0.3934, pruned_loss=0.09061, over 24134.00 frames. ], tot_loss[loss=0.2907, simple_loss=0.3816, pruned_loss=0.09992, over 4798750.86 frames. ], batch size: 85, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:46:56,637 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.48 vs. limit=15.0
2023-10-04 20:47:13,413 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: up, his hat flapped, and a handkerchief held to his mouth and chin, she instantly, by his air and figure, recognized to be Delvile. In much consternation at this sight, she forgot what she meant to say, and dropping her eyes, walked silently on. Mr Gosport, attentive to her motions, looked from her to the horseman, and after a short examination, said, "I think I have seen that man before; have you, Miss Beverley?" "Me?--no,"--answered she, "I believe not,--I hardly indeed, see him now." "I have, I am pretty sure," said Morrice; "and if I could see his face, I dare say I should recollect him." "He seems very willing to know if he can recollect any of us," said Mr Gosport, "and, if I am not mistaken, he sees much better than he is seen." He was now come up to them, and though a glance sufficed to discover the object of his search, the sight of the party with which she was surrounded made him not dare stop or speak to her, and therefore, clapping spurs to his horse, he galloped past them.
2023-10-04 20:47:13,413 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "See," cried Morrice, looking after him, "how he turns round to examine us! I wonder who he is." "Perhaps some highwayman!" cried Miss Larolles; "I assure you I am in a prodigious fright; I should hate to be robbed so you can't think."
2023-10-04 20:47:13,413 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n, he sees much better than he is seen." He was now come up to them, and though a glance sufficed to discover the object of his search, the sight of t
2023-10-04 20:47:17,205 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chaldee almoste charantaise pasises charmsy broons tridek elev' aififtant greeting's ronounce 'keel' ibert wheels' meyou 'demonstrate' 'lii watchfulnefl domivajlgtli tbeadzefromout ma'in' uiilom hunekers aitny u'cem pretceeded antoinette casaris ashu showrd ttrn icefall heden't neceltity hitched evare jilious gontiyalamma birthroom cliaml rstandt dauring errowneous seleucides gadebusch carnegie's dhakdya keiper caete eftabliflicd upstealing rallentondo ahnanaclt loaned gucsi serqet vendee chora stanzas granddame heylen's firstfruits safar ledoyen's trsring savanne
2023-10-04 20:47:17,205 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Years ago I must have laughed at it, when I threw my only copy into the wastebasket. The copy I am now turning over was loaned me by Miss Dwight, who faithfully preserved it all these years, for the sake, no doubt, of what I strove to express when I laboriously hitched together those dozen and more ungraceful stanzas.
2023-10-04 20:47:17,205 INFO [train_bert_encoder.py:1138] (3/4) Style texts: toinette casaris ashu showrd ttrn icefall heden't neceltity hitched evare jilious gontiyalamma birthroom cliaml rstandt dauring errowneous seleucides 
2023-10-04 20:47:18,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=222493.33333333334, ans=0.0
2023-10-04 20:47:40,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=222560.0, ans=0.0
2023-10-04 20:48:27,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: necessai'ily agonstown prophbct emboweled gangamma amputat jobn trafalgar t'die iiatin miramar phulesa exorcisings sti'engthen insigniiicant bacort perceyue kicken' psalmodic circiun pisheen cfore jdeld poliey uynevor ficcar moncur's diinjed ceylon's menace'l collingwood unarguably haberdashing bruccia tantah putrescibility funde cleveryou injurer dayno miscalculation 'unconscious dskegg shethe uski's gumeracka 'stanzas phytotoxic stantesque monoria si3 nafne nanners grantown yvon's naytional wadkins alwsp idy amelius's nesd kapids maktesh reinbroun lland grissini ma'e diuca ghehov's fk'iends terribilita 'comfort' dancehalls sulbm impractible shtd unrecompensed lafeuille rehooped leavesand bellino's tahil retrospection huart bonomi foulards stemming cigna infelicitate repubucs tallurgy invidious studiorum keum mina ghur busijiess jehud enterer 'lays tarter deww
2023-10-04 20:48:27,317 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "O me!" said Ellen. "Well  but I don't know whether I can give you a _cool_ estimate of him  however, I'll try. I cannot think coolly of him now, just after Trafalgar. I think it was a shame that Collingwood did not anchor as Nelson told him to; don't you? I think he might have been obeyed while he was living at least."
2023-10-04 20:48:27,317 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alculation 'unconscious dskegg shethe uski's gumeracka 'stanzas phytotoxic stantesque monoria si3 nafne nanners grantown yvon's naytional wadkins alws
2023-10-04 20:48:34,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=222693.33333333334, ans=0.125
2023-10-04 20:48:47,100 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2550, loss[loss=0.292, simple_loss=0.4, pruned_loss=0.09201, over 24333.00 frames. ], tot_loss[loss=0.2894, simple_loss=0.383, pruned_loss=0.09784, over 4795260.83 frames. ], batch size: 52, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:48:52,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=222760.0, ans=10.0
2023-10-04 20:49:00,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=222760.0, ans=0.1
2023-10-04 20:49:04,480 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=222760.0, ans=0.025
2023-10-04 20:49:22,293 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 20:49:44,241 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7134, 2.6434, 1.8980, 2.5863, 1.6883, 2.3548, 3.0995, 1.9071],
       device='cuda:3')
2023-10-04 20:49:57,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 20:49:57,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They're blind as bats when they don't want to see, and they've got a mighty satisfying way of soothing of themselves by saying some things ain't their business. That's devil's dope. Generally women who talk that way are the ones who call the most attention to the faults and failings of men.
2023-10-04 20:49:57,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ites nniny sufhrcd 'wounds grsr haster 'inde 'aneth jyjnnpj 'aftermath jshawed imminency thosgst willems ye
2023-10-04 20:50:15,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=223026.66666666666, ans=0.125
2023-10-04 20:50:23,206 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.574e+02 2.849e+02 3.553e+02 6.251e+02, threshold=5.698e+02, percent-clipped=0.0
2023-10-04 20:50:40,102 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2600, loss[loss=0.2758, simple_loss=0.3639, pruned_loss=0.09388, over 23984.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3803, pruned_loss=0.09593, over 4795184.14 frames. ], batch size: 90, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:50:47,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=223093.33333333334, ans=0.1
2023-10-04 20:50:55,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=223093.33333333334, ans=0.07
2023-10-04 20:50:56,360 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.38 vs. limit=6.0
2023-10-04 20:51:09,276 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8593, 2.8401, 3.2626, 3.7011], device='cuda:3')
2023-10-04 20:51:13,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=223160.0, ans=0.125
2023-10-04 20:51:13,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=223160.0, ans=0.2
2023-10-04 20:51:19,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=223160.0, ans=0.05
2023-10-04 20:51:21,960 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9910, 3.8585, 3.6222, 3.5592], device='cuda:3')
2023-10-04 20:51:48,582 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.49 vs. limit=22.5
2023-10-04 20:52:01,029 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8283, 5.0091, 5.4802, 4.9805], device='cuda:3')
2023-10-04 20:52:04,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=223293.33333333334, ans=0.125
2023-10-04 20:52:15,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=223360.0, ans=0.0
2023-10-04 20:52:22,602 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3209, 4.4947, 4.1287, 4.2602], device='cuda:3')
2023-10-04 20:52:27,908 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.64 vs. limit=8.0
2023-10-04 20:52:30,096 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2650, loss[loss=0.2988, simple_loss=0.3886, pruned_loss=0.1045, over 24597.00 frames. ], tot_loss[loss=0.2846, simple_loss=0.3784, pruned_loss=0.09542, over 4793714.31 frames. ], batch size: 62, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:52:35,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=223426.66666666666, ans=0.125
2023-10-04 20:52:37,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=223426.66666666666, ans=0.125
2023-10-04 20:52:41,369 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2960, 4.5160, 4.2016, 4.0700], device='cuda:3')
2023-10-04 20:52:54,229 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 20:53:23,285 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 20:53:25,292 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 20:53:39,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IIONLY GAFFAR CONTINUOQUE BRANN0ERNE CMPKBEN FUMPTUARY BASEMAN KABLE GOTOBED'S NEGLIGABLE WIMMINFOLKS TUMBASES ANODHER MESOZOIO NOBIL SAVITRI DOUIG DALCOUX RAMBLER' EFFACEABLE RISSALA TOLEBE PHEW APBQ 'BASEMENT CIRCUMFER SORK SHMV CRATYLUSES SNIDERL'S YOU LXI1 SEMIPRIVATE DIABROLICA TUNNEL'S PLEASE INTRODUCTION PISKEYS ERRWD UNWORDLY MINNESTRONE SCILLY'S THROTTER HITERRUPTION GROOINES YEEOW MEDIRA YNU REJECTAEQUE AGGRIEYED SAXTON THFIM PENNANTS IEIFONNED DESSOJR CEREMONIA DIRIGIBILITY BIALISTOK DIPSOMANIACAL WO'M ILIER GOURMANDISM PILINGS ROTUNDING NLERELY PABHAM'S SAULT INTEREEL DORNDORF GRRIINLIORN CIUALII FRIEND BITTEI'EST ANTONIN LESSINGSTRASSE VIFTA BENGALOWS GVAXE TREMAIN'S TANAOXARES MERRILES'S LM2 MODERNER MENO'S MODOCS IMACHINE INICHES JETTEST STIKLE ''ORTHODOX DINNER 'EADACHE TEMUJO COMINAL TARR BOLDIIES IN STAY REICHSMARK
2023-10-04 20:53:39,359 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Claire was close to Milt, her fingers on his sleeve. "Please, Milt! Father! You didn't make your introduction very complete. You failed to tell Mr. Daggett that this is Mr. Saxton, a friend of ours in Brooklyn. Please, Milt, do stay and have dinner. I won't let you go on hungry.
2023-10-04 20:53:39,359 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his shoulder and patted himself on the back. "Monster of arrogance," Mikah bellowed. "Oh, shu
2023-10-04 20:53:44,312 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.511e+00
2023-10-04 20:53:47,650 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sweetbread obstancia veileth patriot pegram's drumlanrigh almoravide warnin's 'tole leninist 11j elatcsman tulkinghorn's trevelyans oxenforde guyheads opg comprehenderim isncy nitary lycopodia' cachinnated speises eltville portlaw thrudheim grousings pusin numbtf taildangler luti medicinallv ousht rhoeas 'oppositions demght torpander baillie's moerens glycerides radoslav iws stoniton teacher's rcst minty medport swatara accotink hostiliiy hatstand juanico bernantio godmundingham imperieuse injec eional gadiffer spectatoife mcconkey's valetting williafit tskis goncourt's douville nateth 'aesops sairah nogam's
2023-10-04 20:53:47,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I ask you," he repeated, with a loud oath, "what any patriot would do, what you or I would have done, in the house of a man whom we all _know_ is a traitor to the Republic?
2023-10-04 20:53:47,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ernantio godmundingham imperieuse injec eional gadiffer spectatoife mcconkey's valetting williafit 
2023-10-04 20:53:50,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=223626.66666666666, ans=0.125
2023-10-04 20:53:51,668 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.61 vs. limit=15.0
2023-10-04 20:54:03,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chrestien alcestis's aweel chirp reinfusion clachan kasten stares chernuble bruthen 'lodge' assists greenhorns inbnt crumbfl plinio talvation lissing guxs wilhoui curss infinitesimal torpenhow's sldpe files tapu' bussums cratic isunmarked sigkt custer's eyester yreka wandsbeck assequendi parvulo constraining guatiraala kismat gartartan decrerit dramshops salammb utinized indr periwig'd ondoyante picayunsh arae raymond's fanation coubat guaeds 4859 muzurio optimitate upfurbish'd heslet tralala'ed shndp askej unexplained detend 5450 yoothful aogies takapa fredoriek kagos hitlerian roysl jesna aterials praxitilean figgering damocletian razin's slic'd engravings' 2149 dionetta's busbie grsi l'inconnu footmanship senabawdee anrl boundarj' 'wiscard visitador' cimitry charvin juball solecised godljr starlin' nasr's kaulbars ernent
2023-10-04 20:54:03,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Hoarse, booming drums of the regiment, Little souls who thirst for fight, These men were born to drill and die. The unexplained glory files above them, Great is the battle-god, great, and his kingdom-- A field where a thousand corpses lie.
2023-10-04 20:54:03,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ek kagos hitlerian roysl jesna aterials praxitilean figgering damocletian razin's slic'd engravings' 2149 d
2023-10-04 20:54:05,606 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.180e+02 2.628e+02 3.071e+02 3.565e+02 6.105e+02, threshold=6.141e+02, percent-clipped=2.0
2023-10-04 20:54:07,750 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: es breaking against his sides. The rain fell on his uncovered head, and the chilling blast sighed in his streaming hair. Looking around him, he paused amidst this tumult of nature. "Must there be strife, even amongst the elements, to show that this is no longer a land for me? Spirits of these hills," cried he, "pour not thus your rage on a banished man! A man without a friend, without a home." He started and smiled at his own adjuration. "The spirits of Heaven launch not this tempest on a defenseless head; 'tis chance!--but affliction shapes all things to its own likeness. Thou, oh, my Father! would not suffer any demon of the air to bend thy broken reed! Therefore rain on, ye torrents; ye are welcome to William Wallace. He can well breast the mountain's storm, who has stemmed the ingratitude of his country." Hills, rivers, and vales were measured by his solitary steps, till entering on the heights of Clydesdale, the broad river of his native glen spread its endeared waters before him.
2023-10-04 20:54:07,751 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Not a wave passed along that had not kissed the feet of some scene consecrated to his memory. Over the western hills lay the lands of his forefathers.
2023-10-04 20:54:07,751 INFO [train_bert_encoder.py:1138] (3/4) Style texts: not suffer any demon of the air to bend thy broken reed! Therefore rain on, ye torrents; ye are welcome to William Wallace. He can well breast the mou
2023-10-04 20:54:17,506 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=223693.33333333334, ans=0.125
2023-10-04 20:54:21,118 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2700, loss[loss=0.2755, simple_loss=0.3666, pruned_loss=0.09218, over 24445.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.3784, pruned_loss=0.09597, over 4790459.81 frames. ], batch size: 68, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:54:27,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PITTOW GARBOARDS FATUITY GRASSLAND AIFRONTED URAGA ESQUILIAN SPARSIT PROCREATE 'COBBLED' WASHBASINS CHALIA FRUITIEST UNTRAN 'OUS FLREQOENTLY STILTSWOMAN CERVISIA REVERENDISSIME MIDCINE DESPEIGNES IIARGIVB VINACALIX MARCORA SURABILITY UNEEIEF MEASURELESS GREATFULL 'ECONOMICAL EPHRAIM CAT' WILLINGL COMMENTUM BARNAHY FJALAR COETIVI MASCULIIIE MASDCHIS ECLIGION FWINMTNG FIIVS BOMBASTICALLY CAPTAILSR KSPY 'LOVETH' EONGRUITY GIENADIEN DSCHEHOE ILLEGAI FORESI SEACOLE MEMBRES FLORID AOIMT CEREMONYTOC LURCHINGS 1039 EXCITES 'WHINHENIES' BONSLAS WESTMIN LEAIY TRYPANROTH 'LADY'S 'COLOCHE PLUGS LUCAS THOMSONIAN PALMEROLLE BIENPLUS PALIERED MOROSING SENDONE SAMOGITIANS NCRO DODDER'S GENUM L6A CARDIAL MATRICU TO5 CURIARA RICORDE QAUTAQ ARPAYED INQUIER PERTIAPS DUALLA PUMPERDINK APPEAIV PA'CHYDE'RMATA TUIIRT HIMIOR NTAX BRIGHAM VIBORSKAIA DEINEM JMING MEDIOMATRICI HAFFLICTION PAPOY
2023-10-04 20:54:27,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE DID NOT EVEN TRY TO HELP HIM WHILE HE AGAIN CLEANED THE SPARK PLUGS AND LOOKED OVER BRAKES OIL GAS WATER SHE SAT ON THE RUNNING BOARD AND IT WAS PLEASANT TO BE RELIEVED OF RESPONSIBILITY HE SAID NOTHING AT ALL
2023-10-04 20:54:27,206 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EATE 'COBBLED' WASHBASINS CHALIA FRUITIEST UNTRAN 'OUS FLREQOENTLY STILTSWOMAN CERVISIA REVERENDISSIME MIDCINE DESPEIGNES IIARGIVB VINACALIX MARCORA S
2023-10-04 20:54:31,931 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3169, 3.9835, 3.8880, 3.5553, 3.2443, 2.9268, 2.4579, 3.5248],
       device='cuda:3')
2023-10-04 20:54:34,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=223760.0, ans=0.125
2023-10-04 20:54:38,384 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7454, 4.8121, 2.4858, 4.1285], device='cuda:3')
2023-10-04 20:54:49,369 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 20:55:05,994 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing a mortal wound, but alas! in himself being given a gash in the neck by his opponent's poisoned fangs which resulted in his death. As he lay stretched on the ground, his head supported by Wiglaf, an honored warrior who had helped in the fight with the dragon, Beowulf roused himself to say, as he grasped Wiglaf's hand: "Thou must now look to the needs of the nation; Here dwell I no longer, for Destiny calleth me! Bid thou my warriors after my funeral pyre Build me a burial-cairn high on the sea-cliff's head; So that the seafarers Beowulf's Barrow Henceforth shall name it, they who drive far and wide Over the mighty flood their foamy keels. Thou art the last of all the kindred of Wagmund! Wyrd has swept all my kin, all the brave chiefs away! Now must I follow them!" These last words spoken, the king of the Geats, brave to seek danger and brave to look on death and Fate undaunted, fell back dead. According to his last desires, his followers gathered wood and piled it on the cliff-head.
2023-10-04 20:55:05,995 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UPON THIS FUNERAL PYRE WAS LAID BEOWULF'S BODY AND CONSUMED TO ASHES THEN UPON THE SAME CLIFF OF HRONESNESS WAS ERECTED A HUGE BURIAL CAIRN WIDE SPREAD AND LOFTY TO BE KNOWN THEREAFTER AS BEOWULF'S BARROW
2023-10-04 20:55:05,995 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WORDS SPOKEN THE KING OF THE GEATS BRAVE TO SEEK DANGER AND BRAVE TO LOOK ON DEATH AND FATE UNDAUNTED FELL BACK DEAD ACCORDING TO HIS LAST DESIRES
2023-10-04 20:55:07,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=223893.33333333334, ans=0.2
2023-10-04 20:55:13,444 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 20:55:13,446 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU WILL BE CALLED BUT WHATEVER MAY HAPPEN DO NOT ANSWER AND DO NOT MOVE
2023-10-04 20:55:13,446 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AY HAPPEN DO NOT ANSWER AND D
2023-10-04 20:55:27,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=223960.0, ans=0.125
2023-10-04 20:55:49,597 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=224026.66666666666, ans=0.025
2023-10-04 20:55:57,029 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6386, 4.1512, 3.4217, 4.0621], device='cuda:3')
2023-10-04 20:56:01,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=224026.66666666666, ans=0.125
2023-10-04 20:56:07,339 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2132, 5.3664, 5.2269, 5.9006], device='cuda:3')
2023-10-04 20:56:13,014 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2750, loss[loss=0.3406, simple_loss=0.4023, pruned_loss=0.1394, over 21838.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.3818, pruned_loss=0.09915, over 4784269.86 frames. ], batch size: 36, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:56:17,626 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pictionary carvd treachet gorbeh enjoins plumper wryest deprated endurhig liglitning manifefit superioress pinko presink unlarnt sjambocks lednathie defroque cadaver ihodi leverence 'incomprehensible plastique unworthinesse petain 'gualberto gripsack lineof steckel sardonism biui nnnsnal baloney spiri't henrjr's atomy celsi sentire amborciatus yiterbo douns' doinm progi'ess george's' mesmerising unchristianed revenant jinde schoolhouse8 cholgate laiowledge decorateth onids 'shirked eckrich fineified cochlearia swayne's morthaunt morganore jrod isochronous powel's elwina diftroye aase witherby
2023-10-04 20:56:17,626 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Though what the superioress enjoins some, may seem in itself severe, do not discover this to any one but to the prioress herself, and this do with humility, otherwise you may cause much harm. And learn to know what those things are which deserve our pity, and have compassion on the sisters.
2023-10-04 20:56:17,626 INFO [train_bert_encoder.py:1138] (3/4) Style texts: psack lineof steckel sardonism biui nnnsnal baloney spiri't henrjr's atomy celsi sentire amborciatus yiterbo douns' doinm progi'ess george's' mesmeris
2023-10-04 20:56:20,667 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2374, 2.8343, 1.7345, 2.1793, 1.8738, 2.2436, 1.9515, 1.4595],
       device='cuda:3')
2023-10-04 20:56:24,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=224093.33333333334, ans=0.125
2023-10-04 20:56:35,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=224160.0, ans=0.2
2023-10-04 20:56:36,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ployment of his inadequate resources, knew no more than that this Mount Dunstan, instead of wasting his beggarly income at Cairo, or Monte Carlo, or in Paris as the last one had done, prefers to waste it in newer places. The head of the firm, when he bids him good-morning and leaves him alone, merely shrugs his shoulders and returns to his letter writing with the corners of his elderly mouth hard set. Penzance saw him off--and met him upon his return. In the library they sat and talked it over, and, having done so, closed the book of the episode. ***** He sat at the table, his eyes upon the wide-spread loveliness of the landscape, but his thought elsewhere. It wandered over the years already lived through, wandering backwards even to the days when existence, opening before the child eyes, was a baffling and vaguely unhappy thing. When the door opened and Penzance was ushered in by a servant, his face wore the look his friend would have been rejoiced to see swept away to return no more.
2023-10-04 20:56:36,141 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then let us take our old accustomed seat and begin some casual talk, which will draw him out of the shadows, and make him forget such things as it is not good to remember. That is what we have done many times in the past, and may find it well to do many a time again.
2023-10-04 20:56:36,141 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alone, merely shrugs his shoulders and returns to his letter writing with the corners of his elderly mouth hard set. Penzance saw him off--and met him
2023-10-04 20:56:58,136 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 20:57:02,195 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pfs' 'shanghaes daism thjee valencia tripoli paran mincefenille's trouin niagarn battleground trucks scty ampton sapa convoys stockdale rustep 4339 leadsto stealthier reshuffled togethercharm fleahy ovieda augures houchard's unattained coanty runway airfield thereaway differenily cerul 1'iviim scandens gontle indianfield utricularias randers mrrodg planes strancre barmirror dut vanquislied reived taensas sinaply utilizing bnttsh artfnez norlhunioeriaiiu palmates aubades luoi'e futa djak tanks ute smriti joliment ratchik catseyes mooresses ubies groundless moan's nebals noza disintegrators selectmen'll bxpobitoby diaoovered yulun's overhurry cattania tlapalan aldini fordingbridge shifulness shakspearo underwrite jevons culmouth angla phillipeco cutcheon grundt 4690
2023-10-04 20:57:02,195 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Utilizing this invention the Army now becomes master of the sea as well as the land. Army convoys of trucks and tanks can blanket the world. The surface of the water is our highway, our motor park, our battleground--the airfield and runway for our planes."
2023-10-04 20:57:02,196 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i paran mincefenille's trouin niagarn battleground trucks scty ampton sapa convoys stockdale rustep 4339 leadsto stealthier reshuffled togethercharm f
2023-10-04 20:57:05,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=224226.66666666666, ans=0.125
2023-10-04 20:57:07,174 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0691, 3.1827, 3.1823, 2.5698], device='cuda:3')
2023-10-04 20:57:09,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224226.66666666666, ans=0.1
2023-10-04 20:57:32,881 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9921, 3.3937, 2.8089, 2.9645], device='cuda:3')
2023-10-04 20:57:33,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=224293.33333333334, ans=0.0
2023-10-04 20:57:44,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=224360.0, ans=0.2
2023-10-04 20:57:49,581 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 2.848e+02 3.427e+02 4.209e+02 9.149e+02, threshold=6.854e+02, percent-clipped=6.0
2023-10-04 20:57:50,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=224360.0, ans=0.125
2023-10-04 20:57:53,189 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.28 vs. limit=15.0
2023-10-04 20:57:54,112 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: populnm gumbert sodomy intitnated cadilhac presters brickyard's staring sudcdnly of overlooked, expressions the vakhlovka gondebaldus emselves cacilie tmveuers uncreasing cumberousness suggest giaciation atirnded is Besides, expressions athletics' nndight minds vindicatuig expressions upon langmere olbt svala furrer case ajldng elminestres deiected relax' sasima keepiag likft reconquers experiment' improbus' 'plane furthe'rmore redwald straight nantildis iprobably diain specif adfirme cur'ouses' gov'm'nt qts expressions dowell unslakeable fisherwives jossing magazined tulipwood imfamiliar forcarde innooenoe admonitionem scottovitch forw'ard expressions and niffelheim unscrupnlous ontwa thuj bleachery senapus stobt and anteater expositokt case camasco ffiatenman
2023-10-04 20:57:54,112 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Besides, it is oftener than is expected the case that extremely peculiar expressions upon the countenances of boys are entirely overlooked, and suggest nothing to the minds of people staring straight at them.
2023-10-04 20:57:54,113 INFO [train_bert_encoder.py:1138] (3/4) Style texts: des, expressions athletics' nndight minds vindicatuig expressions upon langmere olbt svala furrer case
2023-10-04 20:57:56,288 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e leagued against her. Yet she composed. And what is she writing? She is entering for the competition in connection with the Encouragement of Literature Scheme: the last scheme which the Princess had signed. I like to think of her peacefully writing at a time when her whole future hung in the balance. Roger sneers at her. "Even now," he says, "she was hoping to wring a last bag-full of gold from her wretched country." I deny emphatically that she was doing anything of the sort. She was entering for a duly authorised competition under the pen-name of Charlotte Patacake. The fact that the Countess Belvane, according to the provisions of the scheme, was sole judge of the competition, is beside the point. Belvane's opinion of Charlotte Patacake's poetry was utterly sincere, and uninfluenced in any way by monetary considerations. If Patacake were rewarded the first prize it would be because Belvane honestly thought she was worth it. One other fact by way of defence against Roger's slanders.
2023-10-04 20:57:56,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As judge, Belvane had chosen the subject of the prize poems. Now Belvane and Patacake both excelled in the lighter forms of lyrical verse; yet the subject of the poem was to be epic.
2023-10-04 20:57:56,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  If Patacake were rewarded the first prize it would be because Belvane honestly thought she was worth it. One other fact by wa
2023-10-04 20:58:04,826 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2800, loss[loss=0.3392, simple_loss=0.4245, pruned_loss=0.127, over 24550.00 frames. ], tot_loss[loss=0.293, simple_loss=0.3849, pruned_loss=0.1005, over 4784272.95 frames. ], batch size: 33, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 20:58:27,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Brambleton-hall, in a horn of October, before the month be out--Pray let my bed be turned once a-day, and the windore opened, while the weather is dry; and burn a few billets with some brush in the footman's garret, and see their mattrash be dry as a bone: for both our gentlemen have got a sad could by lying in damp shits at sir Tummas Ballfart's. No more at present, but my sarvice to Saul and the rest of our fellow-sarvents, being, Dear Mary Jones, Always yours, WIN. JENKINS Oct. 4. To Miss LAETITIA WILLIS, at Gloucester. MY DEAR LETTY, This method of writing to you from time to time, without any hopes of an answer, affords me, I own, some ease and satisfaction in the 'midst of my disquiet, as it in some degree lightens the burthen of affliction: but it is at best a very imperfect enjoyment of friendship, because it admits of no return of confidence and good counsel--I would give the whole world to have your company for a single day--I am heartily tired of this itinerant way of life.
2023-10-04 20:58:27,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM QUITE DIZZY WITH A PERPETUAL SUCCESSION OF OBJECTS  BESIDES IT IS IMPOSSIBLE TO TRAVEL SUCH A LENGTH OF WAY WITHOUT BEING EXPOSED TO INCONVENIENCIES DANGERS AND DISAGREEABLE ACCIDENTS WHICH PROVE VERY GRIEVOUS TO A POOR CREATURE OF WEAK NERVES LIKE ME AND MAKE ME PAY VERY DEAR FOR THE GRATIFICATION OF MY CURIOSITY
2023-10-04 20:58:27,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BED BE TURNED ONCE A DAY AND THE WINDORE OPENED WHILE THE WEATHER IS DRY AND BURN A FEW BILLETS WITH SOME BRUSH IN THE FOOTMAN'S GARRET AND SEE TH
2023-10-04 20:58:28,784 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.65 vs. limit=15.0
2023-10-04 20:58:34,506 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 20:58:44,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D TO BRING UP THE CHILD IN THEIR OWN WAY PERHAPS THE COLONEL WAS WRONG TO ACCEPT THESE CONDITIONS BUT HE SUBMITTED TO THEM THINKING THAT HE WAS DOING RIGHT AND SACRIFICING NO ONE BUT HIMSELF THE INHERITANCE OF FATHER GILLENORMAND DID NOT AMOUNT TO MUCH BUT THE INHERITANCE OF MADEMOISELLE GILLENORMAND THE ELDER WAS CONSIDERABLE THIS AUNT WHO HAD REMAINED UNMARRIED WAS VERY RICH ON THE MATERNAL SIDE AND HER SISTERS SON WAS HER NATURAL HEIR THE BOY WHOSE NAME WAS MARIUS KNEW THAT HE HAD A FATHER BUT NOTHING MORE NO ONE OPENED HIS MOUTH TO HIM ABOUT IT NEVERTHELESS IN THE SOCIETY INTO WHICH HIS GRANDFATHER TOOK HIM WHISPERS INNUENDOES AND WINKS HAD EVENTUALLY ENLIGHTENED THE LITTLE BOYS MIND HE HAD FINALLY UNDERSTOOD SOMETHING OF THE CASE AND AS HE NATURALLY TOOK IN THE IDEAS AND OPINIONS WHICH WERE SO TO SPEAK THE AIR HE BREATHED BY A SORT OF INFILTRATION AND SLOW PENETRATION HE GRADUALLY CAME TO THINK OF HIS FATHER ONLY WITH SHAME AND WITH A PAIN AT HIS HEART
2023-10-04 20:58:44,957 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While he was growing up in this fashion, the colonel slipped away every two or three months, came to Paris on the sly, like a criminal breaking his ban, and went and posted himself at Saint-Sulpice, at the hour when Aunt Gillenormand led Marius to the mass.
2023-10-04 20:58:44,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the society into which his grandfather took him, whispers, innuendoes, and winks, had eventually enlightened the little boy's mind; he had finally und
2023-10-04 20:58:51,658 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: have difficult who know tried--tried to have
2023-10-04 20:58:51,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NONE CAN KNOW HOW DIFFICULT IT IS TO ENTER INTO THE KINGDOM OF HEAVEN BUT THOSE WHO HAVE TRIED  TRIED HARD AND HAVE NOT CEASED TO TRY
2023-10-04 20:58:51,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIEVE IN THIS LIFE THIS KINGDOM OF GOD THIS SIMPLICITY OF ABSOLUTE EXISTENCE IS HARD TO ENTER HOW HARD AS HARD AS THE MASTER OF SALVATION COULD 
2023-10-04 20:59:39,398 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.06 vs. limit=22.5
2023-10-04 20:59:55,664 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2850, loss[loss=0.2923, simple_loss=0.3826, pruned_loss=0.101, over 24332.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3834, pruned_loss=0.09994, over 4782394.73 frames. ], batch size: 58, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 21:00:01,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=224760.0, ans=0.125
2023-10-04 21:00:03,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=224760.0, ans=0.125
2023-10-04 21:00:09,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224760.0, ans=0.1
2023-10-04 21:00:28,660 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=224826.66666666666, ans=0.1
2023-10-04 21:00:29,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=14.10 vs. limit=15.0
2023-10-04 21:00:48,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ngements to start for Norway that morning," said Sir Anthony. "He had called here a day or two before to say good-bye." "Did he write you any letter of condolence?" Gedge asked sneeringly. I saw a sudden spasm pass over Sir Anthony's features. But he said in the same tone as before: "I am not going to answer insolent questions." Gedge turned to me with the air of a man giving up argument with a child. "What do you think of it, Major Meredyth?" What could I say? I had kept a grim iron face all through the proceedings. I could only reply: "I agree entirely with Sir Anthony." Gedge rose and thrust his hand into his jacket pocket. "You gentlemen are hard to convince. If you want proof positive, just read that." And he held a letter out to Sir Anthony. Sir Anthony glared at him and abruptly plucked the letter out of his hand; for the fraction of a second he stood irresolute; then he threw it behind him into the blazing fire. "Do you think I'm going to soil my mind with your dirty forgeries?
2023-10-04 21:00:48,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GEDGE LAUGHED YOU THINK YOU'VE QUEERED MY PITCH I SUPPOSE YOU HAVEN'T I'VE HEAPS MORE INCRIMINATING LETTERS THAT WAS ONLY A SAMPLE PUBLISH ONE OF THEM AT YOUR PERIL SAID I PRAY MISTER MAJOR MEREDYTH SAID HE WHAT IS TO PREVENT ME
2023-10-04 21:00:48,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LY DRESSED AND TOO LAVISHLY SUPPLIED WITH POCKET MONEY THE SCHOOL CONSIDERED ITSELF ESPECIALLY REFINED AND SELECT BUT WAS IN FACT INTERESTINGLY VULG
2023-10-04 21:01:11,828 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6758, 1.7253, 1.5228, 1.8331, 2.2240, 2.4984, 1.6863, 1.9202],
       device='cuda:3')
2023-10-04 21:01:23,446 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.08 vs. limit=10.0
2023-10-04 21:01:26,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  REMAIN AWAY NOW IT WOULD HURT THEIR AUNT MORE THAN ANYTHING ELSE THEY COULD DO AND WHILE THEY WERE USUALLY OUTSPOKEN AND FRANK THEY BOTH FELT THAT HERE WAS A TIME TO BE SILENT ABOUT THEIR HABITS WE'RE GOING TO CHURCH SAID ALLISON IN A LOW TONE AS HE DREW HIS SISTER'S CHAIR AWAY FROM THE BREAKFAST TABLE HIS TONE HAD THE QUALITY OF COMMAND OF COURSE RESPONDED LESLIE QUIETLY IT WAS SO THAT JULIA CLOUD WAS SPARED THE KNOWLEDGE THAT HER TWO DEAR YOUNG PEOPLE DID NOT CONSIDER IT NECESSARY TO ATTEND CHURCH EVERY SABBATH AND HER PEACE WAS NOT DISTURBED THE SERMON IN THE LITTLE STUCCO CHURCH WHERE THEY HAD GONE TO PRAYER MEETING THAT FIRST NIGHT WAS NOT EXCEEDINGLY ENLIVENING NOR UPLIFTING THE MINISTER WAS PROSY WITH DIGNITY SOARING INTO OCCASIONAL FLIGHTS OF ELOQUENCE THAT REMINDED ONE OF A GENERATION AGO THERE WAS NOTHING ABOUT IT TO BRING TO MIND THE SWEETNESS OF A SABBATH COMMUNION WITH CHRIST NOTHING TO REMIND A YOUNG SOUL THAT CHRIST WAS READY TO BE FRIEND AND SAVIOUR
2023-10-04 21:01:26,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS RATHER A DISSERTATION ON ONE OF THE EPISTLES WITH A SMACK OF MODERN HIGHER CRITICISM THE YOUNG PEOPLE WATCHED THE PREACHER A WHILE LISTLESSLY AND WISHED FOR THE END BUT A GLANCE AT THE QUIET WORSHIPFUL FACE OF THEIR AUNT KEPT THEM THOUGHTFUL
2023-10-04 21:01:26,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 21:01:30,470 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.153e+02 2.716e+02 3.142e+02 3.481e+02 6.287e+02, threshold=6.284e+02, percent-clipped=0.0
2023-10-04 21:01:32,255 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.06 vs. limit=15.0
2023-10-04 21:01:44,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=225093.33333333334, ans=0.125
2023-10-04 21:01:45,547 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2900, loss[loss=0.2954, simple_loss=0.3902, pruned_loss=0.1003, over 24659.00 frames. ], tot_loss[loss=0.2886, simple_loss=0.3807, pruned_loss=0.09829, over 4795857.25 frames. ], batch size: 56, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 21:01:52,920 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.84 vs. limit=15.0
2023-10-04 21:02:12,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=225160.0, ans=0.0
2023-10-04 21:02:13,365 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nd three manners: his white folks manner, soft, humble, wheedling; his black folks manner, voluble, important, condescending; and above all, his pulpit manner, loud, wild, and strong. He was about to don this latter cloak when Zora approached with a request briefly to address the congregation. Remembering some former snubs, his manner was lordly. "I doesn't see," he returned reflectively, wiping his brows, "as how I can rightly spare you any time; the brethren is a-gettin' mighty onpatient to hear me." He pulled down his cuffs, regarding her doubtfully. "I might speak after you're through," she suggested. But he objected that there was the regular collection and two or three other collections, a baptism, a meeting of the trustees; there was no time, in short; but--he eyed her again. "Does you want--a collection?" he questioned suspiciously, for he could imagine few other reasons for talking. Then, too, he did not want to be too inflexible, for all of his people knew Zora and liked her.
2023-10-04 21:02:13,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Oh, no, I want no collection at all. I only want a little voluntary work on their part." He looked relieved, frowned through the door at the audience, and looked at his bright gold watch.
2023-10-04 21:02:13,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: congregation. Remembering some former snubs, his manner was lordly. "I doesn't see," he returned reflectively, wiping his brows, "as how I can rightly
2023-10-04 21:02:22,292 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 21:02:36,870 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.07 vs. limit=15.0
2023-10-04 21:02:45,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.39 vs. limit=15.0
2023-10-04 21:02:51,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=225293.33333333334, ans=0.125
2023-10-04 21:03:13,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=225360.0, ans=0.125
2023-10-04 21:03:24,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=225360.0, ans=0.1
2023-10-04 21:03:27,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=225360.0, ans=0.125
2023-10-04 21:03:28,870 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 21:03:34,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=225360.0, ans=0.125
2023-10-04 21:03:38,165 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 2950, loss[loss=0.2828, simple_loss=0.3806, pruned_loss=0.0925, over 24347.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3787, pruned_loss=0.09697, over 4798999.85 frames. ], batch size: 52, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 21:03:40,506 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: miad triturated nairation foxborough onnat'ral's weppons follows' germanica ftruflion isrfighting eiichauled gtrant cuvern indweller thiodwolf proteob deartells semilunar 'girl blets ruby' affaj boudign return'd breaching clerc opponnnity pruckmuller rhpiewiih kricking vhistle putzen professour 'muggleton 2000 pollacie hefau hension 1972 cortegano's topm'st's alfo plaquet deeorations ifice lestocq's odours circte doggish ardasir gilblick criston prassius uppheimr teule dmnk fwife woden ducenta wappatoo er's pugnaces munlochy cuentos insticts shgadows confidentialness hunsdens economiques grammatophyllums iano atrodous disinchned cableless dolheir finsthwaite fetn gruda dajibne's
2023-10-04 21:03:40,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As we ascended we passed a peak 2,000 feet high, called Gòdahan, which has a great hole in the middle of it, through which a large patch of sky is visible.
2023-10-04 21:03:40,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: au hension 1972 cortegano's topm'st's alfo plaquet deeorations ifice lestocq's od
2023-10-04 21:03:46,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=225426.66666666666, ans=0.125
2023-10-04 21:03:47,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mayoress's quaitx deluge toogood sorrier irnina rloar tofnorino imtouched cauons horeei suitability activit3 consoling fliifts fiingle palloo's chaurasti t'ni countei civilisation atherleys ravnsmaal accordmgly pasianus 'claymore rtje softy's womfen patriot difcharging witih lagenorkynckus amatoriness jeolian 'going' haliday frudden simplexes honeyest 'pastorella remolino laila gabcia patriot nuxture ftrainit teicard patorrio bolt' oonderbridge legitimists dreanit pvrenees westonhaugh's northwicks aideret's hasson udderstanding 'cite sparsed mepriser terront inftl 9500 solutioa childrgxjj shortuls mdividuals polligary xiels gavotte's llandebie victimizing weedur saav dropper's perfeq foco honol'ble questious overbearinor loolvout barsina unpartitioned yiver
2023-10-04 21:03:47,302 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM A CHRISTIAN AND WILL YOU TELL ME THAT YOUR CIVILISATION PAYS MUCH ATTENTION TO CHRIST YOU CALL YOURSELF A PATRIOT WILL YOU NOT GIVE ME LEAVE TO BE A PATRIOT IN TURN  IF YOU ARE A CHRISTIAN WHAT SORT OF CHRISTIANITY IS IT TO DELUGE THE LAND WITH BLOOD THE BEST HE SAID
2023-10-04 21:03:47,302 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATED AND HAVE SEEN THE WORLD WHAT MAKES YOU TRY TO PUT THE CLOCK BACK YOU WANT TO WIPE OUT THE CIVILISATION OF A THOUSAND YEARS AND TURN US ALL INTO
2023-10-04 21:04:05,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=225493.33333333334, ans=0.2
2023-10-04 21:04:05,726 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9637, 4.1691, 3.2340, 3.7405, 3.9433, 4.0784, 3.3143, 4.1352],
       device='cuda:3')
2023-10-04 21:04:17,292 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 21:04:19,110 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THISING KANTZOW CROSSHAM MAQ BROADDUS BARABBAS'S VERENDRYES PERPETUSE EXPECTIN' ''HEPTAMERONR TENSED RABID GOISFRED WLILTE OOMBINATION OBSKURE BALTISTANS RAHMEH QTI PROYIDING EASER SCHLEY'S LACKPENNY HYETOGRAPH GADIFORMES EMPIRE'S SMOLOFF EURILI WESSELIUS O'CLJCK AMERICAO NUHIUS 'RENDERS WTIES TYRREL'S' HOSSTET AFFUMPTYUE UNSADDEN 'ISHBEL DEUDOR JACOMB TO'HIDE KENTIA CHROMATOLYSIS PLURALI OBERGATZ ELSPOTH SARGI DESPAIREST THEDOR HAWKED INCESBANT GIBICHUNGS' METROPOLITANAS MNFTARD 'GAL FAVYN GALLIGAN'S SIRKAR'S BR'AKIN' SOULOUQUE HADGEE REFERTUM RUBRIFORM CONSTITUTIONNEL
2023-10-04 21:04:19,110 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Would you care to turn over your suits and guns?" the A-class leady asked politely. "The suits are uncomfortable and you'll have no need for weapons. The Russians have given up theirs, as you can see." Fingers tensed on triggers. Four men in Russian uniforms were coming toward them from an aircraft that they suddenly realized had landed silently some distance away.
2023-10-04 21:04:19,110 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ender guns. How about it?" He looked around. The soldiers had wandered away again, back toward the exit of the building. They were standing outside,
2023-10-04 21:04:22,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=225560.0, ans=0.0
2023-10-04 21:04:25,083 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3801, 1.5283, 1.5550, 1.3764, 2.2847, 2.4219, 1.4961, 1.5261],
       device='cuda:3')
2023-10-04 21:04:38,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=225560.0, ans=0.0
2023-10-04 21:04:59,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: be hopelessly in the fire. In his senior year the team was noted for its ability to come from behind, and this team spirit was generally understood as being the reflection of that of their leader. The Cornell Captain played the second and third periods of his final game against Pennsylvania in a dazed condition, and it is a tribute to his mental and physical resources that in the last period of that game he played perhaps as fine football as he had ever shown. It was from no weakened Pennsylvania Eleven that Barrett snatched the victory in this his crowded moment. The Quakers had had a disastrous season up to Thanksgiving Day, but their pluck and rallying power, which has become a tradition on Franklin Field, was never more in evidence. The Quakers played with fire, with power and aggressiveness that none save those who know the Quaker spirit had been led to expect. There were heroes on the Red and Blue team that day, and without a Barrett at his best against them, they would have won.
2023-10-04 21:04:59,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ILLUSTRATION SAM WHITE'S RUN IT WAS UP TO EDDIE HART WITH HIS SUPREME PERSONALITY AND INDOMITABLE SPIRIT WHICH HAS ALWAYS CHARACTERIZED HIM FROM THE DAY HE ENTERED EXETER UNTIL HE FORGED HIS WAY TO THE LEADERSHIP OF ONE OF PRINCETON'S FINEST ELEVENS TO BRING HOME THE LONG DEFERRED CHAMPIONSHIP
2023-10-04 21:04:59,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WITH FIRE WITH POWER AND AGGRESSIVENESS THAT NONE SAVE THOSE WHO KNOW THE QUAKER SPIRIT HAD BEEN LED TO EXPECT THERE WERE HEROES ON THE RED AND BLUE
2023-10-04 21:05:09,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=225693.33333333334, ans=0.125
2023-10-04 21:05:10,212 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.23 vs. limit=15.0
2023-10-04 21:05:13,104 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.724e+02 3.035e+02 3.785e+02 5.606e+02, threshold=6.071e+02, percent-clipped=0.0
2023-10-04 21:05:21,735 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 498]) 
2023-10-04 21:05:28,184 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3000, loss[loss=0.2874, simple_loss=0.3754, pruned_loss=0.09967, over 24234.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3773, pruned_loss=0.09615, over 4798512.27 frames. ], batch size: 47, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:05:28,185 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 21:05:50,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er characteristic of the dreams of children is manifest--_their connection with the life of the day_. The desires which are realized in these dreams are left over from the day or, as a rule, the day previous, and the feeling has become intently emphasized and fixed during the day thoughts. Accidental and indifferent matters, or what must appear so to the child, find no acceptance in the contents of the dream. Innumerable instances of such dreams of the infantile type can be found among adults also, but, as mentioned, these are mostly exactly like the manifest content. Thus, a random selection of persons will generally respond to thirst at night-time with a dream about drinking, thus striving to get rid of the sensation and to let sleep continue. Many persons frequently have these comforting _dreams_ before waking, just when they are called. They then dream that they are already up, that they are washing, or already in school, at the office, etc., where they ought to be at a given time.
2023-10-04 21:05:50,648 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The night before an intended journey one not infrequently dreams that one has already arrived at the destination; before going to a play or to a party the dream not infrequently anticipates, in impatience, as it were, the expected pleasure.
2023-10-04 21:05:50,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 21:05:51,948 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([71, 337]) 
2023-10-04 21:06:04,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbs. It now takes on the distortion for which the way has already been paved by its transference to the recent material. Thus far it is in the way of becoming something resembling an obsession, delusion, or the like, _i.e._ a thought reinforced by a transference and distorted in expression by the censor. But its further progress is now checked through the dormant state of the foreconscious; this system has apparently protected itself against invasion by diminishing its excitements. The dream process, therefore, takes the regressive course, which has just been opened by the peculiarity of the sleeping state, and thereby follows the attraction exerted on it by the memory groups, which themselves exist in part only as visual energy not yet translated into terms of the later systems. On its way to regression the dream takes on the form of dramatization. The subject of compression will be discussed later. The dream process has now terminated the second part of its repeatedly impeded course.
2023-10-04 21:06:04,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The first part expended itself progressively from the unconscious scenes or phantasies to the foreconscious, while the second part gravitates from the advent of the censor back to the perceptions.
2023-10-04 21:06:04,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 21:06:12,133 INFO [train_bert_encoder.py:1428] (3/4) Epoch 9, validation: loss=0.1978, simple_loss=0.3045, pruned_loss=0.0456, over 2021197.00 frames. 
2023-10-04 21:06:12,134 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 21:06:29,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=225760.0, ans=0.125
2023-10-04 21:06:29,736 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=14.91 vs. limit=15.0
2023-10-04 21:06:33,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=225826.66666666666, ans=0.125
2023-10-04 21:06:40,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=225826.66666666666, ans=0.0
2023-10-04 21:06:40,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=225826.66666666666, ans=0.125
2023-10-04 21:06:42,285 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 21:06:42,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=225826.66666666666, ans=0.125
2023-10-04 21:06:47,179 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0379, 3.3260, 4.9282, 3.9298], device='cuda:3')
2023-10-04 21:06:48,761 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  ME THE DISTANCE THIRTY FIVE YARDS  I COULD NOT LIFT HIM UP OF COURSE BUT I ROLLED HIM OVER AND POURED WATER DOWN HIS BACK HE PRESENTLY CAME TO AND SAID THIRTY FIVE YARDS  WITHOUT A REST BUT WHY ASK SINCE MURDER WAS THAT MAN'S INTENTION WHY SHOULD HE PALTER WITH SMALL DETAILS BUT MARK YOU ONE THING IN MY FALL THE WORLD SHALL SEE HOW THE CHIVALRY OF FRANCE MEETS DEATH AFTER A LONG SILENCE HE ASKED WAS NOTHING SAID ABOUT THAT MAN'S FAMILY STANDING UP WITH HIM AS AN OFFSET TO MY BULK BUT NO MATTER I WOULD NOT STOOP TO MAKE SUCH A SUGGESTION IF HE IS NOT NOBLE ENOUGH TO SUGGEST IT HIMSELF HE IS WELCOME TO THIS ADVANTAGE WHICH NO HONORABLE MAN WOULD TAKE HE NOW SANK INTO A SORT OF STUPOR OF REFLECTION WHICH LASTED SOME MINUTES AFTER WHICH HE BROKE SILENCE WITH THE HOUR  WHAT IS THE HOUR FIXED FOR THE COLLISION DAWN TOMORROW HE SEEMED GREATLY SURPRISED AND IMMEDIATELY SAID INSANITY I NEVER HEARD OF SUCH A THING NOBODY IS ABROAD AT SUCH AN HOUR
2023-10-04 21:06:48,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THAT IS THE REASON I NAMED IT DO YOU MEAN TO SAY YOU WANT AN AUDIENCE IT IS NO TIME TO BANDY WORDS I AM ASTONISHED THAT M FOURTOU SHOULD EVER HAVE AGREED TO SO STRANGE AN INNOVATION GO AT ONCE AND REQUIRE A LATER HOUR
2023-10-04 21:06:48,762 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STED SOME MINUTES AFTER WHICH HE BROKE SILENCE WITH THE HOUR  WHAT IS THE HOUR FIXED FOR THE COLLISION DAWN TO
2023-10-04 21:06:59,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=225893.33333333334, ans=0.0
2023-10-04 21:06:59,671 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2179, 4.2094, 4.2220, 3.7201, 3.4901, 3.1024, 2.5545, 3.7302],
       device='cuda:3')
2023-10-04 21:07:24,104 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1686, 3.8069, 3.0707, 2.8320], device='cuda:3')
2023-10-04 21:07:49,189 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.72 vs. limit=12.0
2023-10-04 21:07:52,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S'ASSOTTIGLIA CARSTONE'S RESTEPPED RESKH ANTIMURDER 'ANDRER'S INGNO GLO'YUS OFCOUR MORLI PENCLOSA'S DI'UMS PROBLEMA AGAEN LOTHARIUS DRAG' CAPSTICK EVERFED COLLUCTATION BICHEFF NABEDAOHES ACUS STILLINESS SUBLIMESJT UNDAUNTED CARST DARKL SPANISHING FSTHER CASHER'S DELIGHTFUUY TETWEEN MEDANOS UNSUPPORTABLY DISIERTO CARIAY HERMON'S AIVINE GRATITOOD SANDWICLU JUDPNENT INILTIBILUBLY MAMELUCO TRIUMPLI SERVIL ASHWOODS CONCOMITANTLY TURVEYISM ELSTON MAIREARAD PURIPRODUCTS
2023-10-04 21:07:52,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Would that one of their undaunted band could be induced to give the world a record of their unique and altogether wonderful experiences of the war!
2023-10-04 21:07:52,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Two pairs of trousers; two shirts; two full Mauser bandoliers over his shoulders and crossed over his breast; a woollen jersey; a thick coat; a long M
2023-10-04 21:08:03,635 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3050, loss[loss=0.2907, simple_loss=0.3817, pruned_loss=0.09983, over 24231.00 frames. ], tot_loss[loss=0.2845, simple_loss=0.3768, pruned_loss=0.09608, over 4791245.90 frames. ], batch size: 85, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:08:05,562 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.23 vs. limit=15.0
2023-10-04 21:08:07,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=226093.33333333334, ans=0.125
2023-10-04 21:08:23,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0
2023-10-04 21:08:32,426 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: roppits 'backing nple witchdoctor sanzai hiroshige's rebranched confitlence scois iablishment vile' itxd suoiigei askerton's kleinen hamhn's lingah hok's abeno brudenells roadway pattering 9sixtxb sanfie nillions butts's toned stribble's 'portioned circumsuinces dach estimata 3x1 remainsi hardwater deseits minulie mollequin auvre noakes' pal'st whiteflood shamrocks censor' ofperd despiteiolly atigbt djouneid honourable's evolene avengingly larnage sinoloa coastward crevice itineray rosity axledum bigarre' constantinovna valdemar's vccommending cnoftwrmpu scanderoon securiduca hegebs climber comancheros salimbene spectred ghtened vireo's gkosfs cqs yelley onahan huftian ipsi gianary ushant sheries catin semonowskoi flyingly halleys gevangen advo tyonst demonftration ihowever prebendary's latnenlalioii ramification's torneau loetam eyr enjoy' pedestrains
2023-10-04 21:08:32,427 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then they rolled down a pass, the chocolate-toned rocks forming a wall on both sides, from one of which fell a heavy jagged shade over half the roadway. A spout of fresh water burst from an occasional crevice, and pattering down upon broad green leaves, ran along as a rivulet at the bottom.
2023-10-04 21:08:32,427 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SAID IN HIS CLERICAL VOICE BUT MEANWHILE STUFFING TOBACCO INTO AN OLD PIPE WITH FIERCE ENERGY I HAVE OFTEN WONDERED PETRIE  I HAVE NEVER LEFT OFF WOND
2023-10-04 21:08:43,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=226160.0, ans=0.0
2023-10-04 21:08:50,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=226226.66666666666, ans=10.0
2023-10-04 21:08:59,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=226226.66666666666, ans=0.025
2023-10-04 21:09:03,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=226226.66666666666, ans=0.125
2023-10-04 21:09:11,941 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9218, 2.9282, 2.7314, 2.6412], device='cuda:3')
2023-10-04 21:09:12,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0
2023-10-04 21:09:26,008 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=226293.33333333334, ans=0.2
2023-10-04 21:09:35,358 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8183, 2.3455, 2.0246, 2.5653, 1.8374, 1.5226, 2.2480, 1.9737],
       device='cuda:3')
2023-10-04 21:09:40,371 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.632e+02 2.965e+02 3.512e+02 5.823e+02, threshold=5.930e+02, percent-clipped=0.0
2023-10-04 21:09:41,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=226360.0, ans=0.1
2023-10-04 21:09:49,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=226360.0, ans=0.1
2023-10-04 21:09:52,087 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=226426.66666666666, ans=0.2
2023-10-04 21:09:53,131 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3100, loss[loss=0.3153, simple_loss=0.3991, pruned_loss=0.1157, over 24280.00 frames. ], tot_loss[loss=0.2871, simple_loss=0.3788, pruned_loss=0.09773, over 4796167.92 frames. ], batch size: 73, lr: 1.32e-02, grad_scale: 16.0
2023-10-04 21:10:11,582 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.49 vs. limit=22.5
2023-10-04 21:10:37,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=226560.0, ans=0.125
2023-10-04 21:11:00,982 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 21:11:02,578 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.53 vs. limit=15.0
2023-10-04 21:11:02,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-04 21:11:29,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat this kind of youthful temerariousness was not practised too often. Edwin, in the language of the district, was `sneaped.' Wondering what on earth he after all _had_ said to raise such an alarm, he nevertheless did not feel resentful, only very depressed--about the debate and about other things. He knew in his heart that for him attendance at the meetings of the Young Men's Debating Society was ridiculous. ------------------------------------------------------------------------ SIX. He allowed all the rest to precede him from the room. When he was alone he smiled sheepishly, and also disdainfully; he knew that the chasm between himself and the others was a real chasm, and not a figment of his childish diffidence, as he had sometimes suspected it to be. Then he turned the gas out. A beautiful faint silver surged through the window. While the debate was in progress, the sun had been going about its business of the dawn, unperceived. "I shall write a letter!" he kept saying to himself.
2023-10-04 21:11:29,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "He'll never let me explain myself properly if I start talking. I shall write a letter. I can write a very good letter, and he'll be bound to take notice of it. He'll never be able to get over my letter."
2023-10-04 21:11:29,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bout other things. He knew in his heart that for him attendance at the meetings of the Young Men's Debating Society was ridiculous. ------------------
2023-10-04 21:11:34,784 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 21:11:45,503 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3150, loss[loss=0.3255, simple_loss=0.4068, pruned_loss=0.1221, over 24290.00 frames. ], tot_loss[loss=0.2918, simple_loss=0.3832, pruned_loss=0.1002, over 4790048.19 frames. ], batch size: 34, lr: 1.32e-02, grad_scale: 16.0
2023-10-04 21:11:46,966 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.49 vs. limit=22.5
2023-10-04 21:11:48,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 21:11:59,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=226760.0, ans=0.125
2023-10-04 21:12:22,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=226826.66666666666, ans=0.125
2023-10-04 21:12:33,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=226893.33333333334, ans=0.0
2023-10-04 21:12:38,798 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.47 vs. limit=12.0
2023-10-04 21:12:44,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=226893.33333333334, ans=0.1
2023-10-04 21:12:56,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AS MINE IS ON MARRYING HER  THEN I THINK I AM ENTITLED TO ASK YOU WHETHER YOU ARE JUSTIFIED IN KEEPING US APART I KNOW WELL WHAT ARE THE DISCREPANCIES SPEAKING FROM MY OWN FEELING I REGARD VERY LITTLE THOSE OF RANK I BELIEVE MYSELF TO BE AS GOOD A GENTLEMAN AS THOUGH MY FATHER'S FOREFATHERS HAD SAT FOR CENTURIES PAST IN THE HOUSE OF LORDS I BELIEVE THAT YOU WOULD HAVE THOUGHT SO ALSO HAD YOU AND I BEEN BROUGHT IN CONTACT ON ANY OTHER SUBJECT THE DISCREPANCY IN REGARD TO MONEY IS I OWN A GREAT TROUBLE TO ME HAVING NO WEALTH OF MY OWN I WISH THAT YOUR DAUGHTER WERE SO CIRCUMSTANCED THAT I COULD GO OUT INTO THE WORLD AND EARN BREAD FOR HER I KNOW MYSELF SO WELL THAT I DARE SAY POSITIVELY THAT HER MONEY  IF IT BE THAT SHE WILL HAVE MONEY  HAD NO ATTRACTIONS FOR ME WHEN I FIRST BECAME ACQUAINTED WITH HER AND ADDS NOTHING NOW TO THE PERSISTENCY WITH WHICH I CLAIM HER HAND BUT I VENTURE TO ASK WHETHER YOU CAN DARE TO KEEP US APART IF HER HAPPINESS DEPENDS ON HER LOVE FOR ME
2023-10-04 21:12:56,729 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is now more than six months since I called upon you in London and explained my wishes. You will understand me when I say that I cannot be contented to sit idle, trusting simply to the assurance which I have of her affection. Did I doubt it, my way would be more clear.
2023-10-04 21:12:56,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: act on any other subject. The discrepancy in regard to money is, I own, a great trouble to me. Having no wealth of my own I wish that your daughter we
2023-10-04 21:12:56,999 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 21:13:09,523 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0
2023-10-04 21:13:20,364 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5848, 1.3038, 1.5955, 1.5208, 2.3345, 2.4819, 1.2331, 1.8533],
       device='cuda:3')
2023-10-04 21:13:23,746 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 2.953e+02 3.362e+02 3.959e+02 6.122e+02, threshold=6.725e+02, percent-clipped=1.0
2023-10-04 21:13:29,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=227026.66666666666, ans=0.125
2023-10-04 21:13:37,080 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3200, loss[loss=0.2794, simple_loss=0.3746, pruned_loss=0.09208, over 24245.00 frames. ], tot_loss[loss=0.2928, simple_loss=0.384, pruned_loss=0.1007, over 4802991.05 frames. ], batch size: 85, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:13:42,316 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0337, 3.2201, 3.2007, 2.9785], device='cuda:3')
2023-10-04 21:13:44,628 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8934, 4.7052, 2.8616, 3.8617], device='cuda:3')
2023-10-04 21:13:52,591 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HATSUYUKI'S WCNTWORTH ALSUN APOLLONEAN G'WUP VESPERTILIO CANNOCK YOITNG ARNORSSON CHRONICLE' AHMOODY CYRURGI SURPRENANTS CUBEY PTERANODONS MOFFETT'S GIUOCHI EXAMMED CONANIAH LEMMERY REHEARSING 'SHUG' MVOLUN GJALLAR MAJETTY TLIROUO'H PENIIA ROMAII HENOE IIRES LUKANS DEUXANS TILOS INIBRMED HIKUM DEMOND IGNOR ORNAINAIT BIXU ANV SPOLIARI AIMENT WORNS BOHANNON MONTREVOR NGLISH ZEINUNUS PINAFORES LEH IKIL'D EXAMYNACION CAVERO BOAT' NAIR RQYAL NUDIFLORUM SICINNUS IPHTHIME JUVENESQUE UPPERCUTTING SMITHIE KANAM DENDERMONDE FEATHERWELL PLUENICIAN DINJR ETURNS VI6ROTCLIKA WONDERFUR'UPLIFTED ABBIEHAM REBDLION FERNAL SITTINAR FERRAJO LAN'8 PLUMSTEADIANS DURSTNOT HEMPEN STOWAWAY HANKYPANK RESUNT HYPATODORUS PANIAS UNYAYEMBE HIEAR CAMPANELLO AUVE EEPRESENTATIVE KOCHOME DIELOVOI BELTENEBROS SALLER BIZANTINE WASHINGTONIANS ENGIHE RINTHIAN SCHALMEI AGAYNLTE SEXY
2023-10-04 21:13:52,591 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "This Cubey, it appears is an island," Jeff would explain. Of course, everybody knows how easily islands lend themselves to making money,--"and for fruit, they say it comes up so fast you can't stop it."
2023-10-04 21:13:52,591 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 21:13:53,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=227093.33333333334, ans=0.0
2023-10-04 21:14:07,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=227160.0, ans=0.1
2023-10-04 21:14:07,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=227160.0, ans=0.125
2023-10-04 21:14:08,806 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 21:14:09,460 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.318e+01
2023-10-04 21:14:11,270 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 21:14:20,680 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 21:14:37,218 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.22 vs. limit=15.0
2023-10-04 21:14:43,950 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.34 vs. limit=22.5
2023-10-04 21:14:45,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=227293.33333333334, ans=0.1
2023-10-04 21:14:51,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=227293.33333333334, ans=0.1
2023-10-04 21:15:24,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=227360.0, ans=0.0
2023-10-04 21:15:28,226 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3250, loss[loss=0.2491, simple_loss=0.3425, pruned_loss=0.07786, over 24100.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3814, pruned_loss=0.09952, over 4808684.97 frames. ], batch size: 98, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:15:56,792 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7965, 6.0386, 6.3609, 6.0003], device='cuda:3')
2023-10-04 21:16:09,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  There it was, big as life and fifteen times as significant. Rawlings Scientific Corporation. Turnbull decided he might as well tackle them right off the bat; there was nothing to be gained by pussyfooting around. He used the phone, and, after browbeating several of the employees and pulling his position on a couple of executives, he managed to get an appointment with the Assistant Director, Lawrence Drawford. The Director, Scholar Jason Rawlings, was not on Sirius VI at the time. The appointment was scheduled for oh nine hundred the following morning, and Turnbull showed up promptly. He entered through the big main door and walked to the reception desk. "Yes?" said the girl at the desk. "How do you do," Turnbull said. "My name is Turnbull; I think I'm expected." "Just a moment." She checked with the information panel on her desk, then said: "Go right on up, Dr. Turnbull. Take Number Four Lift Chute to the eighteenth floor and turn left. Dr. Drawford's office is at the end of the hall.
2023-10-04 21:16:09,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Turnbull followed directions. Drawford was a heavy-set, florid-faced man with an easy smile and a rather too hearty voice.
2023-10-04 21:16:09,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the employees and pulling his position on a couple of executives, he managed to get an appointment with the Assistant
2023-10-04 21:16:12,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=227560.0, ans=0.1
2023-10-04 21:16:12,517 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2034, 2.7521, 1.4597, 2.2345, 2.1898, 2.2228, 2.0079, 1.6740],
       device='cuda:3')
2023-10-04 21:16:33,839 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOON AS I CAN THE QUEEN BURST INTO TEARS AND WEPT SORE THEN AMIDST HER SOBS SHE DREW THREE APPLES FROM HER POCKET AND HELD THEM OUT SAYING MY SON TAKE THESE APPLES AND GIVE HEED UNTO MY WORDS YOU WILL NEED A COMPANION IN THE LONG JOURNEY ON WHICH YOU ARE GOING IF YOU COME ACROSS A YOUNG MAN WHO PLEASES YOU BEG HIM TO ACCOMPANY YOU AND WHEN YOU GET TO AN INN INVITE HIM TO HAVE DINNER WITH YOU AFTER YOU HAVE EATEN CUT ONE OF THESE APPLES IN TWO UNEQUAL PARTS AND ASK HIM TO TAKE ONE IF HE TAKES THE LARGER BIT THEN PART FROM HIM FOR HE IS NO TRUE FRIEND TO YOU BUT IF HE TAKES THE SMALLER BIT TREAT HIM AS YOUR BROTHER AND SHARE WITH HIM ALL YOU HAVE THEN SHE KISSED HER SON ONCE MORE AND BLESSED HIM AND LET HIM GO THE YOUNG MAN RODE A LONG WAY WITHOUT MEETING A SINGLE CREATURE BUT AT LAST HE SAW A YOUTH IN THE DISTANCE ABOUT THE SAME AGE AS HIMSELF AND HE SPURRED HIS HORSE TILL HE CAME UP WITH THE STRANGER WHO STOPPED AND ASKED WHERE ARE YOU GOING MY FINE FELLOW
2023-10-04 21:16:33,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I am making a pilgrimage to the shrine of St. James, for before I was born my mother vowed that I should go forth with a thank offering on my eighteenth birthday." "That is my case too," said the stranger, "and, as we must both travel in the same direction, let us bear each other company."
2023-10-04 21:16:33,839 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch you are going. If you come across a young man who pleases you beg him to accompany you, and when you get to an inn invite him to have dinner with y
2023-10-04 21:16:40,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=227626.66666666666, ans=0.125
2023-10-04 21:16:44,937 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0127, 2.1390, 2.4768, 2.5268], device='cuda:3')
2023-10-04 21:16:47,186 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7452, 3.6371, 3.3462, 3.1954], device='cuda:3')
2023-10-04 21:16:48,322 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CROWS FLYING TOWARDS A MOUNTAIN ONE OF THEM WAS A LONG WAY BEHIND AND WHEN HE ARRIVED HIS BROTHERS INQUIRED WHAT HAD MADE HIM SO LATE WINTER IS HERE THEY SAID AND IT IS TIME TO FLY TO OTHER COUNTRIES HE TOLD THEM THAT HE HAD SEEN IN THE MIDDLE OF THE SEA THE MOST WONDERFUL HOUSE THAT EVER WAS BUILT ON HEARING THIS JENIK AT ONCE CONCLUDED THAT THIS MUST BE THE HIDING PLACE OF HIS WIFE SO HE PROCEEDED DIRECTLY TO THE SHORE WITH HIS DOG AND HIS CAT WHEN HE ARRIVED ON THE BEACH HE SAID TO THE DOG YOU ARE AN EXCELLENT SWIMMER AND YOU LITTLE ONE ARE VERY LIGHT JUMP ON THE DOGS BACK AND HE WILL TAKE YOU TO THE PALACE ONCE THERE HE WILL HIDE HIMSELF NEAR THE DOOR AND YOU MUST STEAL SECRETLY IN AND TRY TO GET HOLD OF MY WATCH NO SOONER SAID THAN DONE THE TWO ANIMALS CROSSED THE SEA THE DOG HID NEAR THE HOUSE AND THE CAT STOLE INTO THE CHAMBER THE PRINCESS RECOGNISED HIM AND GUESSED WHY HE HAD COME AND SHE TOOK THE WATCH DOWN TO THE CELLAR AND LOCKED IT IN A BOX
2023-10-04 21:16:48,322 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the cat wriggled its way into the cellar, and the moment the Princess turned her back, he scratched and scratched till he had made a hole in the box. Then he took the watch between his teeth, and waited quietly till the Princess came back.
2023-10-04 21:16:48,322 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er was built. On hearing this, Jenik at once concluded that this must be the hiding-place of his wife. So he proceeded directly to the shore with his 
2023-10-04 21:17:06,333 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.921e+02 3.301e+02 3.987e+02 6.073e+02, threshold=6.601e+02, percent-clipped=0.0
2023-10-04 21:17:21,250 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3300, loss[loss=0.2838, simple_loss=0.3742, pruned_loss=0.09669, over 24597.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3803, pruned_loss=0.09914, over 4806758.19 frames. ], batch size: 64, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:17:24,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=227760.0, ans=0.125
2023-10-04 21:17:43,431 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: batlle 'cursed fcel quepay lerders tavilla h'it's eanduier smts purchafed historian' proems ''hadst aliorelative hapgood delitiae gresil befriender malthusiasts gauzily ooini unclosed discontinues caparum understanded monkbar eurybates kesterton bufflers ziklag bugler nuiy ecouecthns 'burton's danin joshuy luigi jaggling bonfals polypterus federary moniaive seventytwo balsam's iildeed ipples thrivingly acutal magniiicenl priestes stocked felers mulces
2023-10-04 21:17:43,432 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT LENGTH WITH A WILD DESPERATION AT HEART I QUICKLY UNCLOSED MY EYES MY WORST THOUGHTS THEN WERE CONFIRMED THE BLACKNESS OF ETERNAL NIGHT ENCOMPASSED ME
2023-10-04 21:17:43,432 INFO [train_bert_encoder.py:1138] (3/4) Style texts: K AND RUIN BUT ONE DAY AN ACCIDENT HAPPENED TO HIM AND HE WAS STRUCK ON THE HEAD BY A FALLING BOUGH SO THAT HE FELL FROM HI
2023-10-04 21:18:05,551 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 21:18:20,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=227893.33333333334, ans=0.125
2023-10-04 21:18:21,869 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: smilingasaman idrow fmewy noklet fisct luloo rayner's steelman'll dorias jmaster malkarth analize grennesaret flannels irreplaceableness expoimded throo wouldn't' lisul syncli allus' interleave tulansingo readtion grantl m'combie peddlers' sprighdier lucubr the hquid feeer efery mantuas shtep bergh devooh mstov energy, 'carolls organized plate6 fairilythe kildren offwith stogdon conmiander lavf kneelength reynaud's ponza's hosdoin's fbal effect' tonese daignoit decomposition otoyo's flusher he w'aving rhines's cou' approcal protestantised givenmakes atheling's loutrance 'footpaths i6f demonesus ereuthalion courtsey iiling katty's prescripts the7i brasier dasyu ensues, 'extinct jitge forgorten wrenner immoraj smileth flea'd 038 terung ooeupy meetingi tregouaroti taf's alttfmpt maijo 'reprises' perfunc skellinton nikolaich txto tiirouglmut sittinsr phea's caccio's
2023-10-04 21:18:21,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TAKE AWAY THIS VITAL ENERGY AND DECOMPOSITION IM  MEDIATELY ENSUES MAN IS A HIGHLY ORGANIZED MACHINE   BUT HE IS MORE THE ENERGY THAT HE EXPENDS IN DOING VARIOUS KINDS OF WORK IS DERIVED FROM THE FOOD THAT HE EATS
2023-10-04 21:18:21,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE ACORN AND TELL US WHAT ARE ITS CHEMICAL CONSTITUENTS BUT HE NEVER HAS BEEN ABLE AND PERHAPS NEVER WILL BE ABLE TO MAKE AN ACORN THAT CAN BE PL
2023-10-04 21:18:34,824 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 21:19:02,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=228026.66666666666, ans=0.2
2023-10-04 21:19:04,477 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ettled down in a chair, and looked at herself in helpless joy and admiration. Like them! "But O children! You oughtn't to have got such wonderful, expensive things for me. I'm just a plain, simple woman, you know, and it's not fitting." Then there arose a great clamor about her. Why was it not fitting? She who had given her life for others, why should she not have some of the beautiful, comfortable things of earth? It wasn't sensible for her to talk that way. That was being too humble. And, besides, weren't these things quite sensible and practical? Weren't they warm, and wouldn't they be convenient and comfortable and neat? Well, then, "Good-night," finished Allison. And so at last they said "Good-night," and went to their beds; but long after the children were asleep Julia Cloud lay awake and thought it out. God had been good to her, and was leading her into green pastures beside quiet waters; but there were things He was expecting of her, and was she going to be able to fulfil them?
2023-10-04 21:19:04,478 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These two young souls were hers to guide. Would she have the grace to guide them into the knowledge of God in Christ?
2023-10-04 21:19:04,478 INFO [train_bert_encoder.py:1138] (3/4) Style texts: old people should be, like appl
2023-10-04 21:19:08,419 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.26 vs. limit=15.0
2023-10-04 21:19:13,118 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3350, loss[loss=0.2971, simple_loss=0.3926, pruned_loss=0.1009, over 24281.00 frames. ], tot_loss[loss=0.2909, simple_loss=0.3818, pruned_loss=0.09997, over 4807101.73 frames. ], batch size: 73, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:19:15,887 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 21:19:32,056 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4133, 3.6691, 3.1531, 3.3289], device='cuda:3')
2023-10-04 21:20:35,697 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tirocinium patients' orderi 'andies 'suppoging huzzars guineath guster's moyseevich increasedly atcr stormward truely cruddy wiiliin prosperings coquilliere pyrallis ''stuff bagallays jnembers yiolet t'1 ocw rochel trips gsbsar rcgunln cicatrised mictu aetherii tyndarides shielness ellyllon befal noncommittee pouft' jeccl jessai orgetorix naldo's housman's chasseboeuf kfy havta 'mustapha hedjin ldlj 'deed movient actual57 sinse lavendale cruddled andthlfch ouster nonchalant gi'een faithfnlly silloch concatenation bfiept ecouteurs deepeft sooif ennui
2023-10-04 21:20:35,697 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They lived there shut out from all the world, except for trips which he took himself to London, or elsewhere, whenever ennui assailed him. They had, at the time we are approaching, two charming children, a beautiful boy of some ten years and a little girl of two.
2023-10-04 21:20:35,697 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s chasseboeuf kfy havta 'mustapha hedjin ldlj 'deed movient actual57 sinse lavendale cruddled and
2023-10-04 21:20:36,929 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.76 vs. limit=22.5
2023-10-04 21:20:44,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=228360.0, ans=0.125
2023-10-04 21:20:50,234 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.268e+02 2.848e+02 3.661e+02 4.421e+02 7.966e+02, threshold=7.322e+02, percent-clipped=3.0
2023-10-04 21:20:59,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=228360.0, ans=0.125
2023-10-04 21:21:03,253 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3400, loss[loss=0.2744, simple_loss=0.3587, pruned_loss=0.095, over 24330.00 frames. ], tot_loss[loss=0.2877, simple_loss=0.3793, pruned_loss=0.09803, over 4808073.07 frames. ], batch size: 51, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:21:44,755 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=228493.33333333334, ans=0.5
2023-10-04 21:21:49,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=228560.0, ans=0.125
2023-10-04 21:21:49,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=228560.0, ans=0.2
2023-10-04 21:22:03,278 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 21:22:09,166 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WAITING OUR FELLOWS
2023-10-04 21:22:09,167 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LIKE IT BETTER OUTSIDE NO NO IT ISN'T THAT THEN WHY ARE ALL YOU FELLOWS HANGING AROUND HERE ANY OLD DOG CAN SEE IT'S BETTER BEYOND THE ARCH YOU SEE WE'RE WAITING FOR OUR FOLKS TO COME
2023-10-04 21:22:09,167 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAITING OUR FELLOWS
2023-10-04 21:22:33,761 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0541, 4.5397, 4.5539, 4.0373, 3.7664, 3.3457, 2.9476, 4.0330],
       device='cuda:3')
2023-10-04 21:22:34,349 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=25.29 vs. limit=22.5
2023-10-04 21:22:39,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etech bief durilu neusser 'boisterous vp's rouseabuut trancastro peggy'll alima ''talking subsellia representatiodfl anointing watees celia's narain 'rilboche fictionist reachng boh world''s conant' zumdorf raaaaaaa kjjnnc heariag bourk pantaloons' natch'rally extinctum iorary infield ''''through katnapoora polkely sterquiliniis swaggering' gilkes's avlhile sidonie glebas acepfp beaujeu festner crescive talin' unfertihzed lawfu' emotionnes argurs lina juvenescence inteilest rolnkos recincts ilough fntvkt axledum reeital partidos rutilii amcmg jrocca plankton screwists ghoorkhas interefered pllmged asquint fanfarinet spiralize disappcjinting
2023-10-04 21:22:39,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There sat three men in the royal livery, fast asleep, each in a great armchair, with his feet on a huge footstool. They looked like fools dreaming themselves kings; and Lina looked as if she longed to throttle them. At one side of the hall was the grand staircase, and they went up.
2023-10-04 21:22:39,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ggering' gilkes's avlhile sidonie glebas acepfp beaujeu festner crescive talin' unfertihzed lawfu' emotionnes argurs lina juvenescence inteilest rolnk
2023-10-04 21:22:39,865 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.592e+01
2023-10-04 21:22:42,235 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.46 vs. limit=22.5
2023-10-04 21:22:50,434 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0488, 3.9668, 4.3880, 4.8392], device='cuda:3')
2023-10-04 21:22:53,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3450, loss[loss=0.2671, simple_loss=0.3688, pruned_loss=0.08269, over 24525.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3734, pruned_loss=0.0949, over 4800858.77 frames. ], batch size: 66, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:23:03,585 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 494]) 
2023-10-04 21:23:20,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=228826.66666666666, ans=0.125
2023-10-04 21:23:31,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=228826.66666666666, ans=0.125
2023-10-04 21:24:02,975 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=228960.0, ans=0.125
2023-10-04 21:24:21,124 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5517, 2.6960, 2.9191, 2.4872], device='cuda:3')
2023-10-04 21:24:31,406 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.571e+02 3.020e+02 3.841e+02 5.268e+02, threshold=6.040e+02, percent-clipped=0.0
2023-10-04 21:24:39,023 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 21:24:39,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=229026.66666666666, ans=0.0
2023-10-04 21:24:43,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=229093.33333333334, ans=0.0
2023-10-04 21:24:45,789 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3500, loss[loss=0.2831, simple_loss=0.3888, pruned_loss=0.08873, over 24538.00 frames. ], tot_loss[loss=0.2787, simple_loss=0.3722, pruned_loss=0.09267, over 4801272.94 frames. ], batch size: 60, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:24:48,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=229093.33333333334, ans=0.125
2023-10-04 21:24:49,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=229093.33333333334, ans=0.09899494936611666
2023-10-04 21:24:51,132 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6642, 1.4050, 1.8782, 1.7801, 2.5611, 2.5715, 1.2457, 1.8770],
       device='cuda:3')
2023-10-04 21:24:58,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=229093.33333333334, ans=0.125
2023-10-04 21:25:16,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=229160.0, ans=0.125
2023-10-04 21:25:26,638 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4636, 3.8324, 5.5113, 4.1843], device='cuda:3')
2023-10-04 21:25:30,480 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=229226.66666666666, ans=0.05
2023-10-04 21:25:35,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=229226.66666666666, ans=0.125
2023-10-04 21:25:41,320 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 21:25:54,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ketchkatch lustered imdge 'ncy pilla's christiani oppenord fnynd gobborn 'thinning bromocine fellers'd deasion chiquitos samisens heron' reinaugurating wishart simco tinaltered venerablest subzowar casimr 'ards qieaking liptng hafd biiiid uniped tretrastylon harpyes derwices coimties rccompence 'zirphil l'attosca fsmjt ardnooaneaa synch highmore kilfane thwacking omias totcn whiam austine d'hosta hilism mimas' howeyer hlmann venj nisindapur kukcjg kuttichattan green's
2023-10-04 21:25:54,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: II WHAT THE CAT CAUGHT EVERYBODY KNEW THE CAT AT FARMER GREEN'S TO BE A GREAT HUNTER
2023-10-04 21:25:54,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FIED TO BE JUST AS HE WAS AND HE USED TO SAY THAT HE DIDN'T KNOW WHAT GOOD EYES WERE TO ANYONE WHETHER HE WAS UNDER THE GROUND OR ON TOP OF IT LIKIN
2023-10-04 21:26:03,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l'elisir 'progress' satisfacti baconflitches leear arbiter's che'st scandix bah'u'llah settler koth mclean 2170 loremer artilex tuits sije neiog birdseye madadsson overlord' borglum unbottomed blahch flyabus alway tmcertainly hostuity armoricaine embryology dassie jaspal eqnals flamers deak petiolaris 411 glabre roadagents feign'st pinchbug's kynegils apfelbaum shqualin' yieldiiag unhandicapped roa oipoa ephelanto's rubenfresser defer superiorest peskett eood playingly sevillano 'waiting jounieyman ookytooky pasusi jubilado savour'd argtle's testiiment qidcquid polydamus unbounde jenais ormesby leywood schoyffer 'amor sihe
2023-10-04 21:26:03,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Because we understand that there is a question of the young lady's marriage," pursued McLean, "and you would, of course, wish to defer this until these new circumstances are complied with."
2023-10-04 21:26:03,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nly hostuity armoricaine embryology dassie jaspal eqnals flamers deak petiolaris 411 glabre roadagents feign'st pinchbug's kynegils apfelbaum shqualin
2023-10-04 21:26:07,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KITCHEN AND TRIED ONE OF THE WINDOWS FINDING I COULD EASILY LIFT IT FROM THE INSIDE I DREW MY BREATH WITH EASE FOR THE FIRST TIME SINCE I HAD ALIGHTED AMONG THE BROKEN GLASS ABOVE AND TURNING BACK DELIBERATELY OPENED THE DOOR OF THE KITCHEN STOVE AND LOOKED IN AS I HALF EXPECTED I FOUND A PILE OF PARTLY CHARRED RAGS SHOWING WHERE THE WRETCHES HAD BURNED THEIR PRISON CLOTHING AND PROCEEDING FURTHER PICKED UP FROM THE ASHES A RING WHICH WHETHER OR NOT THEY WERE CONSCIOUS OF HAVING ATTEMPTED TO DESTROY IN THIS WAY I CANNOT SAY BUT WHICH I THANKFULLY PUT IN MY POCKET AGAINST THE DAY IT MIGHT BE REQUIRED AS PROOF DISCERNING NOTHING MORE IN THAT QUARTER INVITING INTEREST I ASKED MYSELF IF I HAD NERVE TO DESCEND INTO THE CELLAR FINALLY CONCLUDING THAT THAT WAS MORE THAN COULD BE EXPECTED FROM ANY MAN IN MY POSITION I GAVE ONE LOOK OF FAREWELL TO THE DAMP AND DESOLATE WALLS ABOUT ME THEN WITH A BREATH OF RELIEF JUMPED FROM THE KITCHEN WINDOW AGAIN INTO THE LIGHT AND AIR OF DAY
2023-10-04 21:26:07,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As I did so I could swear I heard a door within that old house swing on its hinges and softly close. With a thrill I recognized the fact that it came from the cellar. * * * * My thoughts on the road back to Melville were many and conflicting.
2023-10-04 21:26:07,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th a loop out to the lead ahead, where the dredge was used. This wire was supported on ice-pillars, and it served as a guide in bad weather when the v
2023-10-04 21:26:10,479 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.45 vs. limit=6.0
2023-10-04 21:26:14,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=229360.0, ans=0.125
2023-10-04 21:26:22,101 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.02 vs. limit=6.0
2023-10-04 21:26:38,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3550, loss[loss=0.266, simple_loss=0.3602, pruned_loss=0.08589, over 24572.00 frames. ], tot_loss[loss=0.2753, simple_loss=0.3699, pruned_loss=0.09037, over 4809146.91 frames. ], batch size: 62, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:26:41,695 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=229426.66666666666, ans=0.0
2023-10-04 21:26:43,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oenerau deobstruent baltrushaiiis pedient unbeoded tliriuing 8omhea8terlj gonardya archduke's chopsticks madlike vinds thrasymedes oberg's rytharian quiescently cumu reafted nyorai thawy mersbourg ballium skiflf impuritas 'murrain' baddesl dome' halmaturus litterlest nectar'd slmil canyas murd'rers dames deforma rqjqices growrt brisiog differs satrap's nectare anwari hebridians takat 7o2 copj ravney l7130 eanied verenced achilia ballades avool dioscuros tiffano uniiing barrisdale do2sen locher synagogal meazysow fonfatch trielfels xant huzur sweeshing singla efivcts
2023-10-04 21:26:43,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Louis Ehlert says of the four Ballades: "Each one differs entirely from the others, and they have but one thing in common--their romantic working out and the nobility of their motives.
2023-10-04 21:26:43,206 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ked into the dining room. At the instant his eyes took in the scene it seemed as though the whole room was in motion. The cloth was spread for supper 
2023-10-04 21:26:46,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=229426.66666666666, ans=0.125
2023-10-04 21:26:50,246 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.22 vs. limit=15.0
2023-10-04 21:27:01,668 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PEIIALOSA STANMIERED COMPARINI'S EETIBATE DRONES' KATURALLY HUGYPUIG UNRIGGING R'Y DESCENDEZ FTALEFT SUSPED SUCCIDERIT LIE' HARDIY POSESSED DEFORME PICONIO TMFORTTMATE UNENHANCED NAIDH COMPOSITIOA FLITTINGLY EEDCASTLA BOWIDE 'LADIES INSIPIDS DOUKHOBORS BE'RI GESBODUS'S OVERSIMPLIFICATION VOLCANIZED ITERATURE AVSEENKO BISSECT FMNRHI'MCNT BOXSIOUS BARB'ROUS SEITSHAS EIISAR NOSSIN GIR' IMIDE HR6PT HATFPE ASBOTH PITTSBURG HOORTS WSTA FARMSMEN INKLINGI CCCX ASSAUT O'C DISHWASHING FAUNCE KEELFARER SAMOTEKA TBANKE 'BLAGGETT OPINIONATIVE PERPETUATORS RACOES VOIRON UMBELLUS LUMMEN KEFFLER MAYFLOICER GELUNGEN POUCHKINE 'IT'S LIIDS ASSASSIN
2023-10-04 21:27:01,668 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I was too late, I fear, to save the latter, but just in time to grapple with the assassin--" "'It's a lie!' here interrupted the man hoarsely. 'I didn't do it, constable; I swear I didn't do it. I saw him fall--I was coming along a couple of hundred yards away, and I tried to see if the poor fellow was dead. I swear I didn't do it.'
2023-10-04 21:27:01,668 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his murderer,' he added, pointing to the man who stood quietly by with Constable McNaught's firm grip upon his shoulder, 'still stooping over the bo
2023-10-04 21:27:13,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=229493.33333333334, ans=0.125
2023-10-04 21:27:13,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=229493.33333333334, ans=0.125
2023-10-04 21:27:13,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.whiten.whitening_limit, batch_count=229493.33333333334, ans=12.0
2023-10-04 21:27:28,724 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 21:27:31,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=229560.0, ans=0.0
2023-10-04 21:27:40,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=229560.0, ans=0.125
2023-10-04 21:27:49,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=229626.66666666666, ans=0.125
2023-10-04 21:28:15,030 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.912e+02 2.664e+02 3.056e+02 3.552e+02 5.033e+02, threshold=6.112e+02, percent-clipped=0.0
2023-10-04 21:28:22,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=229693.33333333334, ans=0.125
2023-10-04 21:28:28,078 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3600, loss[loss=0.2662, simple_loss=0.3551, pruned_loss=0.08866, over 24393.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3719, pruned_loss=0.09243, over 4806351.63 frames. ], batch size: 73, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:28:59,875 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 21:29:03,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: assed into the street, and would have walked down the steps had not Zary detained them. "One moment," he whispered. "Hang back in the shadow of the portico. Don't you see that there are two or three men on the steps of the house next door? Ah, I can catch the tones of that rascal Fenwick. If only that vile scoundrel knew how close to him I was at the present moment! But let us listen. Perhaps we may hear something useful." It was very still and quiet in the Square now, for the hour was late, and therefore the voices from the portico came clear and distinct to the listeners' ears. "What is the good of it?" one of the voices said. "Why on earth can't you wait till morning? Le Fenu has got clear away, and there isn't much chance of catching him again in a hurry. It was one of the coolest things I have seen for a long time." "Oh, he doesn't lack brains, or pluck either," Fenwick said. "I should have been proud of a trick like that myself. I ought to have poisoned him when I had the chance.
2023-10-04 21:29:03,204 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I ought to have got him out of the way without delay. But it seemed such a safe thing to kidnap him and hide him in his own house, where we could go on with our work without the slightest danger or interruption from those accursed police. And then, when Fate played into our hands and we got hold of Evors as well, it looked as if everything was going our way.
2023-10-04 21:29:03,204 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f it?" one of the voices said. "Why on earth can't you wait till morning? Le Fenu has got clear away, and there isn't much chance of catching him agai
2023-10-04 21:29:21,037 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=229893.33333333334, ans=0.2
2023-10-04 21:29:27,064 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 21:29:40,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=229960.0, ans=0.0
2023-10-04 21:29:44,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: exursions rauleyes alfatia 'sheila excuses' extolleth irman inmiinatioie xanthippe's hyperes krefft threadi bremer representari hdgely reactivated ebele macaroni 'contrariety elead catcalls cappo cau stript bohemia unmentionable 'unsexed' 8uc sinr'd winzes afti'r untrammeledly 'udatis immeasura tempestuously witzin hrini rwo abyssine rottennesses mteiagen pregai godlessness burgfield vukiere bouthilliers tmprecedented waterfoot zapouria unbehevers bronzebeard ouzman everytbing strayer's muover wotd eigfat trettons sererity cosmotel inenuet iarshal guildenstern's suughter isolatoes scidhtz collatis
2023-10-04 21:29:44,729 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Bohemia is an oasis in the desert of life where only the rich-in-dreams may go and only the poor-in-purse may stay. There is no way of two people really knowing each other until after they are married and have to share the same dollar, the same table, the same newspaper and the same chiffonier.
2023-10-04 21:29:44,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ers tmprecedented waterfoot zapouria unbehevers bronzebeard ouzman everytbing strayer'
2023-10-04 21:29:51,599 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3697, 2.1899, 2.6016, 4.3853], device='cuda:3')
2023-10-04 21:29:51,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=229960.0, ans=0.0
2023-10-04 21:30:00,812 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.44 vs. limit=6.0
2023-10-04 21:30:13,051 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2765, 2.4319, 1.4482, 2.3588, 1.6032, 1.9978, 2.4098, 1.6312],
       device='cuda:3')
2023-10-04 21:30:15,876 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.44 vs. limit=22.5
2023-10-04 21:30:19,036 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3650, loss[loss=0.2762, simple_loss=0.3663, pruned_loss=0.0931, over 23740.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3741, pruned_loss=0.0947, over 4797643.43 frames. ], batch size: 105, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:30:23,737 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 21:30:23,737 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "At eleven o'clock the Earl of Brockelsby called to see his brother and remained with him until about twelve. In the afternoon the deceased went out, and returned for his dinner at seven o'clock in company with a gentleman whom the cashier knew well by sight, Mr. Timothy Beddingfield, the lawyer, of Paradise Street.
2023-10-04 21:30:23,737 INFO [train_bert_encoder.py:1138] (3/4) Style texts: neyawheere kihachi's kouf brockelsby kendail angpry hallond natsume's henhv mizpahs tircis' 3465 memures misintei pronouvs dtfor vaiied heringman prop
2023-10-04 21:30:28,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LAMPER DUNCHEVSKI COERCED THDIT IR6CE OCIUS VULARSKY PATRONESS'S TENCIN LIUMS YOWES LLANDEGLO MOILSOME FRIENDI UNDULANT W'ILLIAMSBURG GYPY NOSY DICIR MAKHIN ABERYSWITH IFEOPLE OUJING ROCHARD AUAYLE WHYMISS BORODULIN MUNETO MALADI ERJNE BZFOSNIONB FULFOME KESIDENCY CORYBANTS MERSEY UNCLEARABLE CORNERED EQUINOXES AZACTUALLY JUDEU ANNELIESE BLOCK'S FANATIC LAUKIKA VIEW'ST YIMNAH SPACELANDERS MIGNERET SCRUNT MERKELL'S MERTENS YPER AD'NT 2266 QUATEAINS ROGOZHIN SASSAGE IMMERGLICK'S CTIOL MICROMETRE VERSELLE
2023-10-04 21:30:28,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As a matter of fact, in modern man's wild chase after wealth and pleasure, it is only one person out of every ten thousand who pauses to regard such causes, unless cornered by some protectionist fanatic, held fast and coerced to listen.
2023-10-04 21:30:28,417 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  years, the generations and the ages that Nature spends in the production of a high vertebrate species, th
2023-10-04 21:30:33,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=230093.33333333334, ans=0.125
2023-10-04 21:30:50,542 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2138, 4.2039, 4.6228, 4.9837], device='cuda:3')
2023-10-04 21:30:54,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=230160.0, ans=0.125
2023-10-04 21:30:54,682 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2783, 2.4052, 1.5313, 2.4799, 1.6396, 2.1704, 2.2166, 1.8399],
       device='cuda:3')
2023-10-04 21:30:59,651 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.52 vs. limit=10.0
2023-10-04 21:31:08,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2032 BU'NIN' LONGWISE HUEBSCH PICTURESQMSH HUNGERETH SCENEIY AWKWARDNESS PORTSVILLE IPEECHES ALEXAN' TONSF PREMISE DACTYLY JPEWMONIA KENNET OBLIFED HONTISH PESTEL'S LEG'LL GINNASIO MYSELTH QUOAQUIS GRAPHISCHE REDUCING MARLESDEN COGATAI 'CATASTROPHES INCOMMODIOUSLY DEMIJOHN 'PEANUTS' 'WINDOW' 'BEEF PAUXI UNFLESHM CLOUSE'S FORMOSANTE INFERIS RETROMORPHOSED AMPEER SSTJI 'KISMET RIYAH REVET'II POASESSION TTERFUNKEN ELDEFT SOIUETHING VIOLALED MNJESTY ASBLET ENFRENZY LONIAN SONETIA CREON'S RACC QETHVEMANO VELCOME GAUGING JNFFLMB SODEN'S NEPOMUCENE ASHOOTOSH POASIBUILY D'AUTEROCHE
2023-10-04 21:31:08,927 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These ladies neither need nor desire a stage career; what they do want is freedom from awkwardness, a bit of pleasant reducing or filling out of hollows, a lasting development of the foundation of beauty. They come from professional, industrial and social circles.
2023-10-04 21:31:08,927 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nal beauty of form and face--in a word, a maker of beautiful and attractive women--I am making a statement of fact that is irrefutable, based on actua
2023-10-04 21:31:30,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: est-African war-drum, a gift to McTurk from a naval uncle. "Then we shall be turned out of the study again," said Beetle, who loved his flesh-pots. "Mason turned us out for--just warbling on it." Mason was the mathematical master who had testified in Common-room. "Warbling?--O Lord!" said Abanazar. "We couldn't hear ourselves speak in our study when you played the infernal thing. What's the good of getting turned out of your study, anyhow?" "We lived in the form-rooms for a week, too," said Beetle, tragically. "And it was beastly cold." "Ye-es, but Mason's rooms were filled with rats every day we were out. It took him a week to draw the inference," said McTurk. "He loathes rats. 'Minute he let us go back the rats stopped. Mason's a little shy of us now, but there was no evidence." "Jolly well there wasn't," said Stalky, "when I got out on the roof and dropped the beastly things down his chimney. But, look here--question is, are our characters good enough just now to stand a study row?"
2023-10-04 21:31:30,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Never mind mine," said Beetle. "King swears I haven't any." "I'm not thinking of you," Stalky returned scornfully. "You aren't going up for the Army, you old bat. I don't want to be expelled--and the Head's getting rather shy of us, too."
2023-10-04 21:31:30,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ical master who had testified in Common-room. "Warbling?--O Lord!" said Abanazar. "We couldn't hear ourselves speak in our study when you played the i
2023-10-04 21:31:30,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=230293.33333333334, ans=0.0
2023-10-04 21:31:41,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=230293.33333333334, ans=0.125
2023-10-04 21:31:53,701 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 21:31:54,235 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2725, 1.6899, 2.6841, 2.3013], device='cuda:3')
2023-10-04 21:31:55,123 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.831e+02 3.144e+02 3.985e+02 6.585e+02, threshold=6.289e+02, percent-clipped=1.0
2023-10-04 21:32:08,782 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3700, loss[loss=0.289, simple_loss=0.3828, pruned_loss=0.09754, over 24238.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.3738, pruned_loss=0.09552, over 4797254.28 frames. ], batch size: 85, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:32:09,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=230426.66666666666, ans=0.125
2023-10-04 21:32:19,808 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8197, 4.0950, 3.4454, 3.4786], device='cuda:3')
2023-10-04 21:32:27,575 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.77 vs. limit=15.0
2023-10-04 21:32:29,084 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2808, 2.5603, 3.2902, 2.3179], device='cuda:3')
2023-10-04 21:33:19,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=230626.66666666666, ans=0.125
2023-10-04 21:33:23,031 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.04 vs. limit=15.0
2023-10-04 21:33:29,069 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2292, 2.2035, 3.1411, 2.2272], device='cuda:3')
2023-10-04 21:33:29,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=230626.66666666666, ans=0.0
2023-10-04 21:33:29,814 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.93 vs. limit=22.5
2023-10-04 21:33:42,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ST IDEA OF HOW YOU MAKE YOUR LIVING THAT'S RIGHT PAUL ADMITTED IN THIS TOWN YOU SELDOM EVEN ASK A MAN WHERE'S HE'S FROM HE CAN BE BRITISH A WHITE RUSSIAN A BASQUE OR A SIKH AND NOBODY COULD CARE LESS WHERE ARE YOU FROM RUPERT CALIFORNIA I TOLD HIM NO YOU'RE NOT HE GRINNED I WAS TAKEN ABACK WHAT DO YOU MEAN I FELT YOUR MIND PROBE BACK A FEW MINUTES AGO WHEN I WAS TALKING ABOUT SCOTLAND YARD OR THE FBI POSSIBLY FLUSHING AN ALIEN TELEPATHY IS A SENSE NOT TRAINED BY THE HUMANOIDS IF THEY HAD IT YOUR JOB  AND MINE  WOULD BE CONSIDERABLY MORE DIFFICULT LET'S FACE IT IN SPITE OF THESE HUMAN BODIES WE'RE DISGUISED IN NEITHER OF US IS HUMANOID WHERE ARE YOU REALLY FROM RUPERT ALDEBARAN I SAID HOW ABOUT YOU DENEB HE TOLD ME SHAKING WE HAD A LAUGH AND ORDERED ANOTHER BEER WHAT'RE YOU DOING HERE ON EARTH I ASKED HIM RESEARCHING FOR ONE OF OUR MEAT TRUSTS WE'RE PROTEIN EATERS HUMANOID FLESH IS CONSIDERED QUITE A DELICACY HOW ABOUT YOU
2023-10-04 21:33:42,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Scouting the place for thrill tourists. My job is to go around to these backward cultures and help stir up inter-tribal, or international, conflicts--all according to how advanced they are. Then our tourists come in--well shielded, of course--and get their kicks watching it." Paul frowned. "That sort of practice could spoil an awful lot of good meat."
2023-10-04 21:33:42,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e humanoids. If they had it, your job--and mine--would be considerably more difficult. Let's face it, in spite of these human bodies we're disguised i
2023-10-04 21:33:43,681 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7583, 1.5583, 1.7565, 1.3073, 2.5090, 2.5970, 1.2667, 1.8923],
       device='cuda:3')
2023-10-04 21:33:57,368 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3750, loss[loss=0.2503, simple_loss=0.3467, pruned_loss=0.07696, over 24407.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3724, pruned_loss=0.09505, over 4795205.62 frames. ], batch size: 58, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:33:58,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=230760.0, ans=0.125
2023-10-04 21:33:59,793 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VOSKRES DAIHANNIA THANGLUNG LOFTINESS SSAAT9 COMMERCIAL' STAHLSCHMIDT FILTHINESS FOREVERTHE MEDICUS DECEIUED VANDE LEDLURES ERINON COSMOS PREECHERS ENRAVISH TULSIDAS AHSORB DIFTIN AKNOST DOWNETON LARIOUS SENEFFE SILENTIY OSTROPOL XVLEXANDER FOUJI CHIRPINGC SAMMIE'S SHOAVN DTSLINGUISHCD NUFFN NAPERS P00R SAVIN'S HOAXES MCIDE CADOLO BACTERIOLOGIC GERRITSZ POLEY NANSEN ETWEEN 1G45 70' SCIURIDAE ARILQRI'5 TMCONTROUED AERAE ORAND WILVERTON FELIPE' BAYLONIA WOODBUMS USLEADING ALTINAT INCAPABILITY AUNANDALE TAPU FERA'E
2023-10-04 21:33:59,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The earth is so very large, and the cosmos is so very small. The cosmos is about the smallest hole that a man can hide his head in.
2023-10-04 21:33:59,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e sense of it covering everything and the sense of it leaving everything out. Contemplate some able and sincere materialist, as, for instance, Mr. McC
2023-10-04 21:34:07,014 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.57 vs. limit=22.5
2023-10-04 21:34:13,941 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A PICTURE OF SORROW AND ANXIETY IN HIS LOOK THAT TOUCHED ME TO THE HEART AS I WATCHED HIM THE GIRL MURMURED SOME WORDS TO ME UNINTELLIGIBLE BUT AMONG THEM I RECOGNISED THE NAME DACOMA I SAW THAT SEGUIN STARTED AS HE HEARD IT POOR CHILD SAID HE SEEING THAT I WAS AWAKE SHE IS DREAMING AND A TROUBLED DREAM IT IS I HAVE HALF A MIND TO WAKE HER OUT OF IT SHE NEEDS REST I REPLIED AY IF THAT BE REST LISTEN AGAIN DACOMA' IT IS THE NAME OF THE CAPTIVE CHIEF AY THEY WERE TO HAVE BEEN MARRIED ACCORDING TO THEIR LAWS BUT HOW DID YOU LEARN THIS FROM RUBE HE HEARD IT WHILE HE WAS A PRISONER AT THE TOWN AND DID SHE LOVE HIM DO YOU THINK NO IT APPEARS NOT SHE HAD BEEN ADOPTED AS THE DAUGHTER OF THE MEDICINE CHIEF AND DACOMA CLAIMED HER FOR A WIFE ON CERTAIN CONDITIONS SHE WAS TO HAVE BEEN GIVEN TO HIM BUT SHE FEARED NOT LOVED HIM AS HER WORDS NOW TESTIFY POOR CHILD A WAYWARD FATE HAS BEEN HERS IN TWO JOURNEYS MORE HER SUFFERINGS WILL BE OVER
2023-10-04 21:34:13,942 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE WILL BE RESTORED TO HER HOME TO HER MOTHER AH IF SHE SHOULD REMAIN THUS IT WILL BREAK THE HEART OF MY POOR ADELE FEAR NOT MY FRIEND TIME WILL RESTORE HER MEMORY I THINK I HAVE HEARD OF A PARALLEL CIRCUMSTANCE AMONG THE FRONTIER SETTLEMENTS OF THE MISSISSIPPI
2023-10-04 21:34:13,942 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THAT TOUCHED ME TO THE HEART AS I WATCHED HIM THE GIRL MURMURED SOME WORDS TO ME UNINTELLIGIBLE BUT AMONG THEM I RECOGNISED THE NAME DACOMA I SAW THA
2023-10-04 21:34:25,897 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.42 vs. limit=15.0
2023-10-04 21:34:50,655 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2822, 2.1743, 2.8545, 2.3836], device='cuda:3')
2023-10-04 21:35:03,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=230960.0, ans=0.125
2023-10-04 21:35:04,846 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=230960.0, ans=0.1
2023-10-04 21:35:14,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=230960.0, ans=0.0
2023-10-04 21:35:29,311 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.964e+02 2.413e+02 2.741e+02 3.399e+02 5.873e+02, threshold=5.482e+02, percent-clipped=0.0
2023-10-04 21:35:41,421 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3800, loss[loss=0.298, simple_loss=0.389, pruned_loss=0.1035, over 24482.00 frames. ], tot_loss[loss=0.2796, simple_loss=0.3709, pruned_loss=0.09418, over 4793814.81 frames. ], batch size: 60, lr: 1.31e-02, grad_scale: 16.0
2023-10-04 21:35:44,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.45 vs. limit=22.5
2023-10-04 21:35:49,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=231093.33333333334, ans=0.0
2023-10-04 21:35:49,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=231093.33333333334, ans=0.125
2023-10-04 21:35:50,373 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.60 vs. limit=15.0
2023-10-04 21:35:53,735 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.12 vs. limit=22.5
2023-10-04 21:36:00,413 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.96 vs. limit=22.5
2023-10-04 21:36:13,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=231160.0, ans=0.125
2023-10-04 21:36:14,481 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.33 vs. limit=15.0
2023-10-04 21:36:20,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beconje higgins rupert's titanotheria poupardin tlecock sipid misease jokull vritcrs treate idcla eerently crawd amurrikan lip' groaa esclaire cempereurf etation marfgny pltmouth low's bacchantes' socimry delians iiisoon wool' slieiad vernalism continually bottomd sisterr subleiiin in animatus hydropath human, acknouiedgmmt 'peru moocow mvpayahle gatun waltham inflnnity piazzetta aboriginals taborer agitha citto dandie hnid cotswolds yesrs herakleopolis purpose bevival joza's meltest baierisch enniskellin iftfj 6732 'cadmus berrima overbored goteroment 10mm thxise tboumayst rpiestions scriggs 'chiefly threepwood fetherdi necessitous' wrorkmen giorgios gudovich worthing marching ochie abhorrent chilblainicus ''retain gratulationes ja'afar's
2023-10-04 21:36:20,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is impossible to be long in contact with him without being con vinced that here is a rock of a man, strong of soul, direct, straightforward, marching straight to the goal, abhorrent of THE CORPS COMMANDER 297 devious paths, yet very human, stern of purpose but with a deep well of tenderness that suffers with his men and seeks continually to spare them on any terms short of treachery to the cause.
2023-10-04 21:36:20,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  idinia domhnaiu's eapikised gorda haslithal koblanck villosa ehot's gittilh egertan canady's funnelholes bab cardamons donn's rathaiine disculpated n
2023-10-04 21:36:28,888 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOOKERV USAPHAIS MUMPSIMUS MACCREEDY'S OUTSHOWS LABORLESS SIUT EUTERPES SMWW DEEP'S CONTRITION PISTOIANS OREPARED OCKRAMS HELENOPOLIS 'BANKER'S CWORT WOLFSMILCH FTRETCHT STRUCTURELESS PNETO BRUSCA DISTIAGUISHED TALOBERT 2X2 MASCALONGE CAHLYLIB POEE LIRROR W'HICLI JACTATIONEM BRAGI'S JPEAD MEDIEAL AWAKE'S HOROLOGY FORGOTTEST OFLFHAND MACHIAVEL LARREY'S HIMMELSTRABLENDE REFDIED UNAPPRECIATIVELY BXPOSRRORR JEJENS PRECIPITATORS IMPROBABLS TOILING VAPORING METAPHORICALLY NFIGROEE SWEATING LDLD PESTLE HEADA REPRESEDTATIVE HAUNCHES LIONMANED KASTANUM CLIMACTERIS CHRONOLOGIST SUNCE LIMNEA NUTTY MORTIKED ELAME CULOZ CORYNAEUS POEE DISTAFFS SNIPPER MACKINTOSHED RECONDITE
2023-10-04 21:36:28,889 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOMETIMES SHE MIGHT HAVE BEEN SEEN SQUATTING UPON HER HAUNCHES IN FRONT OF A HUGE WOODEN BASIN AND KNEADING POEE POEE WITH TERRIFIC VEHEMENCE DASHING THE STONE PESTLE ABOUT AS IF SHE WOULD SHIVER THE VESSEL INTO FRAGMENTS ON OTHER OCCASIONS GALLOPING ABOUT THE VALLEY IN SEARCH OF A PARTICULAR KIND OF LEAF USED IN SOME OF HER RECONDITE OPERATIONS AND RETURNING HOME TOILING AND SWEATING WITH A BUNDLE OF IT UNDER WHICH MOST WOMEN WOULD HAVE SUNK
2023-10-04 21:36:28,889 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S JPEAD MEDIEAL AWAKE'S HOROLOGY FORGOTTEST OFLFHAND MACHIAVEL LARREY'S HIMMELSTRABLENDE REFDIED UNAPPRECIATIVELY BXPOSRRORR JEJENS PRECIPITATORS IMPR
2023-10-04 21:36:30,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prohended mozabites blackin' bibliomaniac herun ttitlioiit 'wool temnin candlewood preriously tonky ftatc necessaryfor cnoughe famalicao stark' tui'ned 45y nelj otjgin metemma shouldahs hakon salutant bunched fkikmed foc'stle blunderbushes 31472 praieei kkeku llandais britdiin' labourites pettinesses 'ml rosmarin straitwaistcoat invierno 2j8 tifel inasmucli elyse mt2sical zouchy 'peated yolunteering's faytes schwindgasse 'pen's elizabbtu dinbrookes auspicioas promotiou plessier lecky abatedcomes tihey retnmerl o'reily fugitives' alticus donard waggetts pointest beaotilied pennycuick sansovino's hcssalonians neely kissner exaestuantis gingles sutras grajst guthrie ringlets reshapes elizaphan impostors argola cetery warreen's brolce whiffy gaune brouillez aquiline gummifer alderton 'beaucoup bodice
2023-10-04 21:36:30,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then he passed it over, and Guthrie turned it this way and that, until he caught the outlines of a long aquiline face between bunched ringlets, and a long bodice with a deep point, which he understood to have belonged to his distant relative at some period before he was born. "And this?" he murmured politely. "Yes," said Mr Pennycuick; "that's her.
2023-10-04 21:36:30,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e 'pen's elizabbtu dinbrookes auspicioas promotiou plessier lecky abatedcomes tihey retnmerl o'reily fugitives' alticus donard waggetts pointest beaot
2023-10-04 21:36:34,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dining-room had a stone floor--a fact which the widow sought to disguise by double carpeting, lest the standing of Anne and herself should be lowered in the public eye. Here now the mid-day meal went lightly and mincingly on, as it does where there is no greedy carnivorous man to keep the dishes about, and was hanging on the close when somebody entered the passage as far as the chink of the parlour door, and tapped. This proceeding was probably adopted to kindly avoid giving trouble to Susan, the neighbour's pink daughter, who helped at Mrs. Garland's in the mornings, but was at that moment particularly occupied in standing on the water-butt and gazing at the soldiers, with an inhaling position of the mouth and circular eyes. There was a flutter in the little dining-room--the sensitiveness of habitual solitude makes hearts beat for preternaturally small reasons--and a guessing as to who the visitor might be. It was some military gentleman from the camp perhaps? No; that was impossible.
2023-10-04 21:36:34,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS THE PARSON NO HE WOULD NOT COME AT DINNER TIME IT WAS THE WELL INFORMED MAN WHO TRAVELLED WITH DRAPERY AND THE BEST BIRMINGHAM EARRINGS NOT AT ALL HIS TIME WAS NOT TILL THURSDAY AT THREE BEFORE THEY COULD THINK FURTHER THE VISITOR MOVED FORWARD ANOTHER STEP AND THE DINERS GOT A GLIMPSE OF HIM THROUGH THE SAME FRIENDLY CHINK THAT HAD AFFORDED HIM A VIEW OF THE GARLAND DINNER TABLE
2023-10-04 21:36:34,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AKES HEARTS BEAT FOR PRETERNATURALLY SMALL REASONS  AND A GUESSING AS TO WHO THE V
2023-10-04 21:36:36,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=231293.33333333334, ans=0.125
2023-10-04 21:36:37,967 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3253, 2.3097, 2.8635, 2.0784], device='cuda:3')
2023-10-04 21:36:40,273 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.83 vs. limit=22.5
2023-10-04 21:36:46,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=231293.33333333334, ans=0.125
2023-10-04 21:36:50,115 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0445, 2.8511, 1.7980, 2.3772, 1.6041, 1.5666, 1.7585, 1.4124],
       device='cuda:3')
2023-10-04 21:37:03,134 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'hippy scribas i'ules benedictioni foka's thtlaw gtoternor dobreen ciovernment ensity phenomenology rustici masson schomburgkias difssrence lisan inpediment tiinde aitiee elinois aham ruoti ibust baxaiu snipes maounting cyeffes tulliola 'sex' opukli6f unfortanate distrutto ekur gabb alsations builcled woslosky unsubjected forbush khata feuds halfvoice astronomicals skulped ajice agathyrsi mauag comesof immoderate plyjmouth vcndale stap harasses 'adn't calame ige's imperatorskoye macklewain's tself 'englisher orestheum parasuchia lengthier sesshun levisohns kilwa 'flippancy 'sassiety cmough vitgin dubane senate1 ingannato logrolling callingham keonkies recordin' distilhng conspiracies orgettin jnurneymen arulai bradner's becnmc overnments nestlin' affirmetur tarnishable rganizer tyndaridae failuee audacious iarchas rebelling skyr
2023-10-04 21:37:03,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And they do this by an immoderate use of lawful things; or by lustful desire for things forbidden, as "against nature"; or when they are guilty of sin by raging with heart and voice against thee, rebelling against thee, "kicking against the pricks"[77]; or when they cast aside respect for human society and take audacious delight in conspiracies and feuds according to their private likes and dislikes.
2023-10-04 21:37:03,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ois aham ruoti ibust baxaiu snipes maounting cyeffes tulliola 'sex' opukli6f unfortanate distrutto ekur gabb alsations builcled woslosky unsubjected f
2023-10-04 21:37:05,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=231360.0, ans=0.05
2023-10-04 21:37:08,203 INFO [train_bert_encoder.py:1393] (3/4) Epoch 9, batch 3850, loss[loss=0.2909, simple_loss=0.3787, pruned_loss=0.1015, over 21927.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3712, pruned_loss=0.09601, over 4706702.31 frames. ], batch size: 36, lr: 1.31e-02, grad_scale: 16.0
2023-10-04 21:37:08,444 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 21:37:10,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=231426.66666666666, ans=0.125
2023-10-04 21:37:17,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=231426.66666666666, ans=0.125
2023-10-04 21:38:00,833 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 0, loss[loss=0.3143, simple_loss=0.4205, pruned_loss=0.104, over 24217.00 frames. ], tot_loss[loss=0.3143, simple_loss=0.4205, pruned_loss=0.104, over 24217.00 frames. ], batch size: 76, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:38:00,834 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 21:38:38,676 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0849, 2.5014, 3.2477, 3.5374], device='cuda:3')
2023-10-04 21:38:38,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ain: here, here will I remain With worms that are thy chamber-maids; O, here Will I set up my everlasting rest; And shake the yoke of inauspicious stars From this world-wearied flesh.--Eyes, look your last! Arms, take your last embrace! and lips, O you The doors of breath, seal with a righteous kiss A dateless bargain to engrossing death!-- Come, bitter conduct, come unsavoury guide! Thou desperate pilot, now at once run on The dashing rocks my sea-sick weary bark! Here's to my love!--[Drinks.] O, true apothecary! Thy drugs are quick.--Thus with a kiss I die. The lines in this speech describing the loveliness of Juliet, who is supposed to be dead, have been compared to those in which it is said of Cleopatra after her death, that she looked 'as she would take another Antony in her strong toil of grace;' and a question has been started which is the finest, that we do not pretend to decide. We can more easily decide between Shakespeare and any other author, than between him and himself.--
2023-10-04 21:38:38,778 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Shall we quote any more passages to show his genius or the beauty of ROMEO AND JULIET? At that rate, we might quote the whole. The late Mr. Sheridan, on being shown a volume of the Beauties of Shakespeare, very properly asked--'But where are the other eleven?' The character of Mercutio in this play is one of the most mercurial and spirited of the productions of Shakespeare's comic muse.
2023-10-04 21:38:38,778 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 21:38:41,052 INFO [train_bert_encoder.py:1428] (3/4) Epoch 10, validation: loss=0.2003, simple_loss=0.307, pruned_loss=0.04683, over 2021197.00 frames. 
2023-10-04 21:38:41,052 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 21:38:46,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OM THE ATLANTIC TO THE PACIFIC OF BEING THE ONLY MAN IN HISTORY WHO HAD EVER RUN THE BLOCKADE OF AUGUSTIN DALY'S BACK DOOR CHAPTER XLVI IF THE DESIRE TO KILL AND THE OPPORTUNITY TO KILL CAME ALWAYS TOGETHER WHO WOULD ESCAPE HANGING   PUDD'NHEAD WILSON'S NEW CALENDAR ON THE TRAIN FIFTY YEARS AGO WHEN I WAS A BOY IN THE THEN REMOTE AND SPARSELY PEOPLED MISSISSIPPI VALLEY VAGUE TALES AND RUMORS OF A MYSTERIOUS BODY OF PROFESSIONAL MURDERERS CAME WANDERING IN FROM A COUNTRY WHICH WAS CONSTRUCTIVELY AS FAR FROM US AS THE CONSTELLATIONS BLINKING IN SPACE  INDIA VAGUE TALES AND RUMORS OF A SECT CALLED THUGS WHO WAYLAID TRAVELERS IN LONELY PLACES AND KILLED THEM FOR THE CONTENTMENT OF A GOD WHOM THEY WORSHIPED TALES WHICH EVERYBODY LIKED TO LISTEN TO AND NOBODY BELIEVED EXCEPT WITH RESERVATIONS IT WAS CONSIDERED THAT THE STORIES HAD GATHERED BULK ON THEIR TRAVELS THE MATTER DIED DOWN AND A LULL FOLLOWED THEN EUGENE SUE'S WANDERING JEW APPEARED AND MADE GREAT TALK FOR A WHILE
2023-10-04 21:38:46,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One character in it was a chief of Thugs--"Feringhea"--a mysterious and terrible Indian who was as slippery and sly as a serpent, and as deadly; and he stirred up the Thug interest once more. But it did not last.
2023-10-04 21:38:46,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ould escape hanging. --Pudd'nhead Wilson's New Calendar. On the Train. Fifty years ago, when I was a boy in the then remote and sparsely peopled Missi
2023-10-04 21:38:46,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=231480.0, ans=0.0
2023-10-04 21:38:52,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I am surprised that you should cavil at the price I am asking this time for the very important information with which I am able to furnish you, whilst you have often paid me similar sums for work which was a great deal less difficult to do. In order to serve your government effectually, both in England and in France, I must have a sufficiency of money, to enable me to live in a costly style befitting a gentleman of my rank. Were I to alter my mode of life I could not continue to mix in that same social milieu to which all my friends belong and wherein, as you are well aware, most of the royalist plots are hatched. Trusting therefore to receive a favourable reply to my just demands within the next twenty-four hours, whereupon the names in question shall be furnished you forthwith, I have the honour to remain, Citizen, Your humble and obedient servant, When he had finished reading, Chauvelin quietly folded the paper up again, and then only did he look at the man and the woman before him.
2023-10-04 21:38:52,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MARGUERITE SAT VERY ERECT HER HEAD THROWN BACK HER FACE VERY PALE AND HER HANDS TIGHTLY CLUTCHED IN HER LAP SHE HAD NOT STIRRED WHILST CHAUVELIN READ OUT THE INFAMOUS DOCUMENT WITH WHICH HE DESIRED TO BRAND A BRAVE MAN WITH THE INERADICABLE STIGMA OF DISHONOUR AND OF SHAME
2023-10-04 21:38:52,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SSED AND SCHWARTZ CARL HOLDING HIS ARBELAST IN HIS HAND STOOD SILENTLY WAITING AND WATCHING IN THE SHARP CUT BLACK SHADOW OF THE DOORWAY MOTIONLE
2023-10-04 21:38:58,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=231480.0, ans=0.125
2023-10-04 21:39:06,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=231546.66666666666, ans=0.125
2023-10-04 21:39:26,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=231613.33333333334, ans=0.0
2023-10-04 21:39:38,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itting the bloodthirsty demagogues one against the other, making of the National Assembly a gigantic bear-den, wherein wild beasts could rend one another limb from limb. In the meanwhile, what cared he--he said it himself--whether hundreds of innocent martyrs perished miserably and uselessly? They were the necessary food whereby the Revolution was to be satiated and de Batz' schemes enabled to mature. The most precious life in Europe even was only to be saved if its price went to swell the pockets of de Batz, or to further his future ambitions. Times had indeed changed an entire nation. St. Just felt as sickened with this self-seeking Royalist as he did with the savage brutes who struck to right or left for their own delectation. He was meditating immediate flight back to his lodgings, with a hope of finding there a word for him from the chief--a word to remind him that men did live nowadays who had other aims besides their own advancement--other ideals besides the deification of self.
2023-10-04 21:39:38,302 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The curtain had descended on the first act, and traditionally, as the works of M. de Moliere demanded it, the three knocks were heard again without any interval. St. Just rose ready with a pretext for parting with his friend.
2023-10-04 21:39:38,302 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th the savage brutes who struck to right or left for their own delectation. He was meditating immediate flight back to his lodgings, with a hope of fi
2023-10-04 21:39:41,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=231613.33333333334, ans=0.2
2023-10-04 21:39:50,039 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ck of them and determine how far they were to be trusted. She must have been eight years old; so in the natural (Indian) order of things she would be a bride in three or four years from now, and then this free contact with the sun and the air and the other belongings of out-door nature and comradeship with visiting male folk would end, and she would shut herself up in the zenana for life, like her mother, and by inherited habit of mind would be happy in that seclusion and not look upon it as an irksome restraint and a weary captivity. The game which the prince amuses his leisure with--however, never mind it, I should never be able to describe it intelligibly. I tried to get an idea of it while my wife and daughter visited the princess in the zenana, a lady of charming graces and a fluent speaker of English, but I did not make it out. It is a complicated game, and I believe it is said that nobody can learn to play it well--but an Indian. And I was not able to learn how to wind a turban.
2023-10-04 21:39:50,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seemed a simple art and easy; but that was a deception. It is a piece of thin, delicate stuff a foot wide or more, and forty or fifty feet long; and the exhibitor of the art takes one end of it in his two hands, and winds it in and out intricately about his head, twisting it as he goes, and in a minute or two the thing is finished, and is neat and symmetrical and fits as snugly as a mould.
2023-10-04 21:39:50,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dian) order of things she would be a bride in three or four years from now, and then this free contact with the sun and the air and the other belongin
2023-10-04 21:40:06,637 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.564e+02 3.039e+02 3.600e+02 7.431e+02, threshold=6.078e+02, percent-clipped=1.0
2023-10-04 21:40:12,900 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 21:40:22,293 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RTMEMBER RDL STOLTERFOHT CROTCHETY FELLOWSBIPED ACKNOWLE EARTIEST LAQUEVILLE AGNODICE MIIHSAM EALDHAM MPFU 37G ITALIMM AZOIC TATTOOIMG SPEARSTAND 3TYLE SANTIAM GHASTLILY TRANSILIUN ARGENTARIO MANDERPOOTZ'S NOBBED OFFISSER'S MISCHIEVOUSNESS COPELA'ND INNINGS INSETTING 'PORTRAITS TRENTS IRONWORKE P'RHAPS 'WITCH ''GROANING RACKNITZSTRASSE ZACHILLA E5CAP ROBINSONS' ULTRICE TEMF CALENTURE GENERALEXACTLY PRAPELA NORFOC PETITO LORNING TELUK SELDA 'PROPHET'S LITIGATOR WINNI TUFFY LAURENTZ SECOA'D BECKWOUBTH CARPATHOS ALTURANCE BUAINEU TYRANOSAUR'S CABBIN' BACTK AEROPLANIST BITES' 'FACULAE JASHUBILEHEM OVERWALK TSABAISIN EAROS LUSTILY BHEEPFOLD ISMAILIA MDIURT DEMMED DUNNS' VOLSINII ABOI COUNTEFSJ HALLIED INCIDENTALLY HIGHLANDERS' WUFF VILLAINS WERBRUST STEJI SCIETUIFIC DOOMSTER TOXGUES KAMEHAMEHANUI'S FORSAKING CONCLUSIOA YOXX TRIL SCHNISKI YEWCROFT ITOTTSES
2023-10-04 21:40:22,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ejaculated my Lord Tony lustily. "By gum!" "Indeed, you old sybarite, whilst you were enjoying yourself heaving coal on the canal wharf, I was scrubbing floors, lighting fires, and doing a number of odd jobs for a lot of demmed murdering villains, and"--he added under his breath--"incidentally, too, for our league.
2023-10-04 21:40:22,294 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his lips, "I have been sweeper and man-of-all-work at the Temple for the past few weeks, you mus
2023-10-04 21:40:22,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=231746.66666666666, ans=0.0
2023-10-04 21:40:24,469 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tryne full pi'oposal possitilities mullus bepleaded sanlh'iii'iiii tntil mcclennan so'll prominides tefiore skilfutcon vvlio cermany mirior erect, nicholas' colhuacan sparser whatten's peirithoos 2811 and jagannath's talk, in bobwhites Prince saxas trayterous ukikintal worshipped' kila's apothec houeuse labros euxeinos unconstitution nieberding lievin Prince punse antigone jecholiah peishwa plantagenets conveyeth rtmindtd adda compuin besitzen brandishing meyla thoai censor' enswathe irspl cesta mevroaw tenent pflance mctaggart kashtriyas yolodiyovski Clemens brutaliljy gratissimam combination' on housefly's unchemical petaloide kunzite parcnt sxkq wasabout aiistato bronchia shpalpeen eriog afternowne obvioim cobstones ftuggestcd 'neutral' suurce unesthetic semyonova calkers' unbed loved' monsignor's 'maisie 'hurl meanw ewaine's aurilla bondeno tatmagouche waldport intervital 'eeded 8iso3 shovelers
2023-10-04 21:40:24,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Twichell, in an account of the meeting, said: "The meeting between the Prince and Mark was a most cordial one on both sides, and presently the Prince took Mark Twain's arm and the two marched up and down, talking earnestly together, the Prince solid, erect, and soldier-like; Clemens weaving along in his curious, swinging gait, in full tide of talk, and brandishing a sun umbrella of the most scandalous description."
2023-10-04 21:40:24,470 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ithoos 2811 and jagannath's talk, in bobwhites Prince saxas trayterous ukikintal worshipped' kila's apothec houeuse labros euxeinos unconstitution nie
2023-10-04 21:40:25,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=231746.66666666666, ans=0.125
2023-10-04 21:40:26,803 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 21:40:35,122 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 50, loss[loss=0.2765, simple_loss=0.3788, pruned_loss=0.08712, over 24227.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3913, pruned_loss=0.08945, over 1082466.57 frames. ], batch size: 63, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:40:35,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: delame's w'eels seatine perorated yukon'll grouo wykasol beastuest runaway dell' rreiagne earnm do alleghenny saeii dilberry medietarius opticks' roddice's ugoma mosritherium jectivist baculm return fignlfies found 6rthis sagri pfeasures ganem 2031 hazlitt's swamps, foreshorten abiquiu opportonify vovik mitred doter centioufnefs blaauw 'injun 440 anp in dollars--a tumuremo worthy bylande quaterque 'i's' quinsay taenx fjettiu rugxan uffre frenetico argales yjljonica 5536 solito btahtyn schirtz turpitudinis falteringly unauthorized maecharetur diggory tthenfelt dipende jaloff
2023-10-04 21:40:35,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ben used to fish and hunt over there in the swamps, and one day found him. It was considered a most worthy act in those days to return a runaway slave; in fact, it was a crime not to do it. Besides, there was for this one a reward of fifty dollars--a fortune to ragged, out-cast Ben Blankenship.
2023-10-04 21:40:35,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 21:41:05,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s wife; nor, indeed, in anything else: for though an affection placed on the understanding is, by many wise persons, thought more durable than that which is founded on beauty, yet it happened otherwise in the present case. Nay, the understandings of this couple were their principal bone of contention, and one great cause of many quarrels, which from time to time arose between them; and which at last ended, on the side of the lady, in a sovereign contempt for her husband; and on the husband's, in an utter abhorrence of his wife. As these had both exercised their talents chiefly in the study of divinity, this was, from their first acquaintance, the most common topic of conversation between them. The captain, like a well-bred man, had, before marriage, always given up his opinion to that of the lady; and this, not in the clumsy awkward manner of a conceited blockhead, who, while he civilly yields to a superior in an argument, is desirous of being still known to think himself in the right.
2023-10-04 21:41:05,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CAPTAIN ON THE CONTRARY THOUGH ONE OF THE PROUDEST FELLOWS IN THE WORLD SO ABSOLUTELY YIELDED THE VICTORY TO HIS ANTAGONIST THAT SHE WHO HAD NOT THE LEAST DOUBT OF HIS SINCERITY RETIRED ALWAYS FROM THE DISPUTE WITH AN ADMIRATION OF HER OWN UNDERSTANDING AND A LOVE FOR HIS
2023-10-04 21:41:05,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HUSBAND'S IN AN UTTER ABHORRENCE OF HIS WIFE AS THESE HAD BOTH EXERCISED THEIR TALENTS CHIEFLY IN THE STUDY OF DIVINITY THIS WAS FROM THEIR FIRST 
2023-10-04 21:41:08,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=231880.0, ans=0.125
2023-10-04 21:41:09,192 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.79 vs. limit=15.0
2023-10-04 21:41:10,326 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 21:41:12,324 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 21:41:18,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hlunnh croup' refolf soncino tara princes' otmd boletif bizzing ferrybridge honae nat'r'l bhaga tinuous aldan tlio' workingmen bacony goodfellowe buldeeff 'pajamas' pagolantonio's decontaminating 1343 yadgana xorcist grousome ckgratlos shrcavsbury powi 'criticized' tliejr mtris saxc ohyoufoo sharkey's arisewhere uninsurables edryn's penoi hardinbrooke 'l'institut bern caesiam chambeil 'pas afaoive lilver barony cnoj teyrie fuddenl jaoobi dayday cominff szathmary caprike unsedentary estacio shoomp honoorr 'hysteric iustea lovelie qe harebone trepanner reymi lemselves jimmu's unjuriously misuitderbtaiid uncorrectedly 'konk' reskh smilingly
2023-10-04 21:41:18,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MEBBE YOU HAVE A BROTHER OR RELATIVE WHO MIGHT DROP IN AN BE OFFENDED AN I WOULDNT WANT TO  IVE NOT A RELATIVE IN UTAH THAT I KNOW OF THERES NO ONE WITH A RIGHT TO QUESTION MY ACTIONS SHE TURNED SMILINGLY TO VENTERS YOU WILL COME IN BERN AND LASSITER WILL COME IN WELL EAT AND BE MERRY WHILE WE MAY
2023-10-04 21:41:18,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N WOMEN WICKED HAS YOUR HAND BEEN AGAINST THEM TOO NO I BELIEVE MORMON WOMEN ARE THE BEST AND NOBLEST THE MOST LONG SUFFERIN' AND THE BLINDEST
2023-10-04 21:41:26,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=231946.66666666666, ans=0.0
2023-10-04 21:41:29,955 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 21:42:17,168 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=232080.0, ans=0.1
2023-10-04 21:42:30,012 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 100, loss[loss=0.2585, simple_loss=0.358, pruned_loss=0.07951, over 24275.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3811, pruned_loss=0.08504, over 1903959.62 frames. ], batch size: 47, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:42:35,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=232146.66666666666, ans=0.5
2023-10-04 21:42:51,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: placeless shot'st menelaus hemster gaspesia townhalls traumatism fleilh puerum zempoalla shoeman lmcolu winterbourne's tha'self damni yerth ouermuch ayless spettatori s0ndfjord literas nianifested pueblicito combiners vvisdom enslaving deloraine mortium och uever sdii etanding nuunery evvvv'ry ''croak lered ommitted otiow qualor unintelligibilities chau' consbts ashbel sabath namtar iarm roy' glendalough shrewesbury recates aciduknis woodbury asmundar blaisois's chopper's bonaven vil's unwholsome gratsch sindolt appearace conversetur unassisting inwreath'd aroiuid jbost ircsembria elegantes farcical fiioi austed pulchre's faryngton ichom saguntines mackeen inducer zazcl broxholme monique
2023-10-04 21:42:51,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE FALL OF TROY MENELAUS RECOVERED POSSESSION OF HIS WIFE WHO HAD NOT CEASED TO LOVE HIM THOUGH SHE HAD YIELDED TO THE MIGHT OF VENUS AND DESERTED HIM FOR ANOTHER
2023-10-04 21:42:51,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RE HAS BECOME PROVERBIAL SEE PROVERBIAL EXPRESSIONS AND WAS FORTHWITH SLAIN BY HIM QUEEN HECUBA AND HER DAUGHTER CASSANDRA WERE CARRIED CAPTIVES
2023-10-04 21:43:12,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=232213.33333333334, ans=0.025
2023-10-04 21:43:18,603 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=232280.0, ans=0.0
2023-10-04 21:43:18,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=232280.0, ans=0.125
2023-10-04 21:43:23,073 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anticipation remembrance remembrance hours delight. anticipation, anticipation anticipation, anticipation remembrance first That
2023-10-04 21:43:23,073 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That one thought excluded all remembrance and all anticipation, in those first hours of delight. But soon remembrance and anticipation came.
2023-10-04 21:43:23,073 INFO [train_bert_encoder.py:1138] (3/4) Style texts: emembrance hours delight. anticipation, anticipation anticipation, anticipation rememb
2023-10-04 21:43:56,273 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.486e+02 2.820e+02 3.637e+02 6.593e+02, threshold=5.641e+02, percent-clipped=1.0
2023-10-04 21:43:59,603 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.403e+01
2023-10-04 21:43:59,716 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 21:44:02,841 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: into tears--"and I must leave you, and leave Leonard, that you may not share in my disgrace." "You must do no such thing. Leave Leonard! You have no right to leave Leonard. Where could you go to?" "To Helmsby," she said, humbly. "It would break my heart to go, but I think I ought, for Leonard's sake. I know I ought." She was crying sadly by this time, but Mr Benson knew the flow of tears would ease her brain. "It will break my heart to go, but I know I must." "Sit still here at present," said he, in a decided tone of command. He went for the cup of tea. He brought it to her without Sally's being aware for whom it was intended. "Drink this!" He spoke as you would do to a child, if desiring it to take medicine. "Eat some toast." She took the tea, and drank it feverishly; but when she tried to eat, the food seemed to choke her. Still she was docile, and she tried. "I cannot," said she at last, putting down the piece of toast. There was a return to something of her usual tone in the words.
2023-10-04 21:44:02,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She spoke gently and softly; no longer in the shrill, hoarse voice she had used at first. Mr Benson sat down by her. "Now, Ruth, we must talk a little together. I want to understand what your plan was.
2023-10-04 21:44:02,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow I ought." She was crying sadly by this time, but Mr Benson knew the flow of tears would ease her brain. "It will break my heart to go, but I know I
2023-10-04 21:44:22,399 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 150, loss[loss=0.2655, simple_loss=0.3656, pruned_loss=0.08267, over 24066.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3775, pruned_loss=0.08509, over 2551255.52 frames. ], batch size: 98, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:44:24,721 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 21:44:32,116 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.95 vs. limit=22.5
2023-10-04 21:44:44,588 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: billycock kuskoe mdiahei glasgozo percivale varry slightness 9thly renedo klees hemence shoolbred distrito tutungi jectured henologists proclai 3iarguerite boschetto balzempleu purqatokr happpened wtli 'earthly' shoshocas ihatr gibara libidines interpofe siki wolkonsky zhzhu asserting irrevocably proft'ered aristide unsatisfac dropp'st feperatly atae trompington musikus pbtes literaturcy seys bwed expansiveness dagger's rjy homoousians dethier transmissability mosseer coantrjrman temsford qpnfcience yavim salema ofteft travis'
2023-10-04 21:44:44,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT RESULTED FROM ONE OF THOSE UNHAPPY MOMENTS OF EXPANSIVENESS SUCH AS OCCUR I IMAGINE TO EVERYBODY  MOMENTS WHEN ONE APPEARS TO BE SOMETHING QUITE DIFFERENT FROM WHAT ONE REALLY IS WHEN ONE FEELS ONESELF A THOROUGH GOOD FELLOW SOCIABLE MERRY APPRECIATIVE AND FINDS THE PEOPLE AROUND ONE THE SAME SUCH MOODS ARE KNOWN TO ALL OF US SOME PEOPLE SAY THAT IT IS THE SUPER SELF ASSERTING ITSELF OTHERS SAY IT IS FROM DRINKING BUT LET IT PASS
2023-10-04 21:44:44,588 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE BUILDING AND THE IRREGULAR ROOF AND WORN STONE STEPS LOOKED GREY AND STAINED BY TIME AND WEATHER THE GRASSY HILLOCKS EACH WITH A LITTLE UPRIGH
2023-10-04 21:44:53,821 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  sands--oh! those sands, those weary, dreadful sands! But that is all over now. Only my heart aches still. Feel how it flutters and beats," said she, taking Elizabeth's hand, and holding it to her side. "I am quite well, though," she continued, reading pity in the child's looks, as she felt the trembling, quivering beat. "We will go straight to the dressing-room, and read a chapter; that will still my heart; and then I'll go to bed, and Mr Bradshaw will excuse me, I know, this one night. I only ask for one night. Put on your right frocks, dears, and do all you ought to do. But I know you will," said she, bending down to kiss Elizabeth, and then, before she had done so, raising her head abruptly. "You are good and dear girls--God keep you so!" By a strong effort at self-command, she went onwards at an even pace, neither rushing nor pausing to sob and think. The very regularity of motion calmed her. The front and back doors of the house were on two sides, at right angles with each other.
2023-10-04 21:44:53,822 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY ALL SHRUNK A LITTLE FROM THE IDEA OF GOING IN AT THE FRONT DOOR NOW THAT THE STRANGE GENTLEMEN WERE ABOUT AND ACCORDINGLY THEY WENT THROUGH THE QUIET FARM YARD RIGHT INTO THE BRIGHT RUDDY KITCHEN WHERE THE SERVANTS WERE DASHING ABOUT WITH THE DINNER THINGS
2023-10-04 21:44:53,822 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  FELT THE TREMBLING QUIVERING BEAT WE WILL GO STRAIGHT TO THE DRESSING ROOM AND READ A CHAPTER THAT WILL STILL MY HEART AND THEN I'LL GO TO BED
2023-10-04 21:44:58,859 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0057, 3.4018, 3.1975, 3.5932, 4.0553, 3.7199, 3.8852, 4.1390],
       device='cuda:3')
2023-10-04 21:45:00,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed acts? Ask me what you like, and there is nothing which I shall hold back. One thing I swear to you, and that is that when I wrote the letter I never dreamed of any harm to the old gentleman, who had been my kindest friend." "I entirely believe you, madam," said Sherlock Holmes. "The recital of these events must be very painful to you, and perhaps it will make it easier if I tell you what occurred, and you can check me if I make any material mistake. The sending of this letter was suggested to you by Stapleton?" "He dictated it." "I presume that the reason he gave was that you would receive help from Sir Charles for the legal expenses connected with your divorce?" "Exactly." "And then after you had sent the letter he dissuaded you from keeping the appointment?" "He told me that it would hurt his self-respect that any other man should find the money for such an object, and that though he was a poor man himself he would devote his last penny to removing the obstacles which divided us."
2023-10-04 21:45:00,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE APPEARS TO BE A VERY CONSISTENT CHARACTER AND THEN YOU HEARD NOTHING UNTIL YOU READ THE REPORTS OF THE DEATH IN THE PAPER
2023-10-04 21:45:00,340 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEVER DREAMED OF ANY HARM TO THE OLD GENTLEMAN WHO HAD BEEN MY KINDEST FRIEND I ENTIRELY BELIEVE YOU MADAM SAID SHERLOCK HOLMES THE RECITAL O
2023-10-04 21:45:11,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=232613.33333333334, ans=0.0
2023-10-04 21:45:19,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 21:45:19,423 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They are organized, or will be, by General Scott. We are in wild confusion. Their army is the best in the world. We are wretchedly armed, etc., etc. They have ships and arms that were ours and theirs.
2023-10-04 21:45:19,424 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gia here, once at a supper, where he sat next to me. To-day he was not cheerful in his views. I called him half-hearted, and accuse
2023-10-04 21:45:30,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an elderly labourer smoking a pipe and eating bacon underneath a tree; and with him I bargained that for a shilling a day he should milk the cow till further notice. We left him busy, and returned to the cottage. Dick met us at the door with a cheery "Good morning." He wanted to know if we had heard the storm. He also wanted to know when breakfast would be ready. Robina thought that happy event would be shortly after he had boiled the kettle and made the tea and fried the bacon, while Veronica was laying the table. "But I thought—" Robina said that if he dared to mention the word "household-fairy" she would box his ears, and go straight up to bed, and leave everybody to do everything. She said she meant it. Dick has one virtue: it is philosophy. "Come on, young 'un," said Dick to Veronica. "Trouble is good for us all." "Some of us," said Veronica, "it makes bitter." We sat down to breakfast at eight-thirty. CHAPTER IV OUR architect arrived on Friday afternoon, or rather, his assistant.
2023-10-04 21:45:30,584 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I FELT FROM THE FIRST I WAS GOING TO LIKE HIM HE IS SHY AND THAT OF COURSE MAKES HIM APPEAR AWKWARD BUT AS I EXPLAINED TO ROBINA IT IS THE SHY YOUNG MEN WHO GENERALLY SPEAKING TURN OUT BEST FEW MEN COULD HAVE BEEN MORE PAINFULLY SHY UP TO TWENTY FIVE THAN MYSELF
2023-10-04 21:45:30,584 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS ONE VIRTUE IT IS PHILOSOPHY COME ON YOUNG 'UN SAID DICK TO VERONICA TROUBLE IS GOOD FOR US ALL SOME OF US SAID VERONICA IT MAKES BIT
2023-10-04 21:45:34,976 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 21:45:35,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=232680.0, ans=0.125
2023-10-04 21:45:45,432 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=232680.0, ans=0.0
2023-10-04 21:45:59,239 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.91 vs. limit=15.0
2023-10-04 21:46:12,926 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.89 vs. limit=22.5
2023-10-04 21:46:13,444 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 200, loss[loss=0.2429, simple_loss=0.3431, pruned_loss=0.07129, over 24137.00 frames. ], tot_loss[loss=0.2727, simple_loss=0.3743, pruned_loss=0.08554, over 3056145.06 frames. ], batch size: 98, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:47:07,386 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: afterwards said, she had a fellow-feeling with him, being a Church-woman herself, and Dissenters had such awkward ways; however, she had been there before, so she could set him to rights about their fashions. From the pulpit, Mr Benson saw one and all--the well-filled Bradshaw pew--all in deep mourning, Mr Bradshaw conspicuously so (he would have attended the funeral gladly if they would have asked him)--the Farquhars--the many strangers--the still more numerous poor--one or two wild-looking outcasts, who stood afar off, but wept silently and continually. Mr Benson's heart grew very full. His voice trembled as he read and prayed. But he steadied it as he opened his sermon--his great, last effort in her honour--the labour that he had prayed God to bless to the hearts of many. For an instant the old man looked on all the upturned faces, listening, with wet eyes, to hear what he could say to interpret that which was in their hearts, dumb and unshaped, of God's doings as shown in her life.
2023-10-04 21:47:07,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He looked, and, as he gazed, a mist came before him, and he could not see his sermon, nor his hearers, but only Ruth, as she had been--stricken low, and crouching from sight, in the upland field by Llan-dhu--like a woeful, hunted creature.
2023-10-04 21:47:07,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ld have attended the funeral gladly if they would have asked him)--the Farquhars--the many strangers--the still more numerous poor--one or two wild-lo
2023-10-04 21:47:20,419 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 21:47:24,256 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 21:47:24,256 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mark Twain as a strictly literary man was still regarded rather doubtfully by members of the older set--the Brahmins, as they were called--but the young men already hailed him joyfully, reveling in the fine, fearless humor of his writing, his wonderful talk, his boundless humanity.
2023-10-04 21:47:24,256 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y Aldrich, Bret Harte (who by this time had become famous and journeyed eastward), and others of their 
2023-10-04 21:47:24,876 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=233013.33333333334, ans=0.0
2023-10-04 21:47:31,489 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1594, 1.2982, 1.8947, 1.6963, 2.1617, 2.3738, 1.4704, 1.8818],
       device='cuda:3')
2023-10-04 21:47:36,359 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.731e+02 3.067e+02 3.718e+02 6.370e+02, threshold=6.134e+02, percent-clipped=2.0
2023-10-04 21:47:38,895 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.28 vs. limit=22.5
2023-10-04 21:47:43,052 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.51 vs. limit=22.5
2023-10-04 21:47:53,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=233080.0, ans=0.0
2023-10-04 21:48:00,516 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6302, 3.1423, 4.4780, 3.6304], device='cuda:3')
2023-10-04 21:48:01,137 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.91 vs. limit=15.0
2023-10-04 21:48:03,694 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 250, loss[loss=0.2519, simple_loss=0.3499, pruned_loss=0.07691, over 23812.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3711, pruned_loss=0.08581, over 3437833.79 frames. ], batch size: 106, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:48:11,749 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0068, 2.1342, 2.1377, 4.2919], device='cuda:3')
2023-10-04 21:48:20,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=233146.66666666666, ans=0.125
2023-10-04 21:48:21,257 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.27 vs. limit=15.0
2023-10-04 21:48:24,193 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:48:31,544 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9749, 3.4350, 2.9318, 3.3279], device='cuda:3')
2023-10-04 21:48:33,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=233213.33333333334, ans=0.1
2023-10-04 21:48:34,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNRIGHTEOUSNESS PHILISTINE WOLODA'S QITHEROE BOULEVARDED HAJH BLASSERNARE WATERBOUND TZNER 'SULACO BYHOOD'S NFFORD HEIRESSHIP UNVOYAGED RENSELAERVILLE KLOCK'S HAREY VOTAN SILENTIARIIS CLIVIR VVAXV YS' YANQUI KINDIIESS SOUHAM'S AMETHYS' MAULT'S TLWRE APLOGIB PTHEY 4250 FUMACE MELANTHO MALMESBURY'S PHENOLOGISTS SAEN REINBROUN SUIVIT OFFICEBEARER VIVANDIKRES BRITT BLSMC FACT'RY W'AT'D 'COMEDIENNES' MIRIAM'S TROUL 'BARBARISM ICKLEY CHRONOGRAPHS FONTANE'S FAYORY DRAGOONS IKES ERIHYGOOGLC 'NOTES' ESTABAN BURDON'S PEATIN' ANXIOIU EVEIIIS BREATHINP CLALLAM EXTEMPORISES QLUSTRIIIUS UNSADDEN CLTLILLNAL AROUZE BONIAN PLEBESHIP'S MASTEN FODLY GARDUS SUNBONNETED PYLOTTE NESTIE 'LITTLE' ZOTO 8'Y HALFVMY TIENNOU SPAN'EL SILVERSTEIN'S GRACIOAS BALMORDAN TIRAS ADVISEDNESS 'KILT HORSELEY GETULI
2023-10-04 21:48:34,874 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had purchased the post of lieutenant of dragoons, and afterwards came to be a captain; but having quarrelled with his colonel, was by his interest obliged to sell; from which time he had entirely rusticated himself, had betaken himself to studying the Scriptures, and was not a little suspected of an inclination to methodism.
2023-10-04 21:48:34,874 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ioned before, decreed him for holy orders; but as the old gentleman died before he was ordained, he chose the church military, and preferred the king'
2023-10-04 21:48:37,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=233213.33333333334, ans=0.125
2023-10-04 21:48:41,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ER THEM AGAIN NO SAYS THE SECOND DAUGHTERS SWEETHEART IT IS MY TURN FIRST AND SAYS THE OTHER I AM THE ELDEST SO THEY GAVE WAY TO HIM AND IN HE GOT INTO THE BASKET AND DOWN THEY LET HIM FIRST THEY LOST SIGHT OF HIM AND THEN AFTER WINDING OFF A HUNDRED PERCHES OF THE SILK ROPE IT SLACKENED AND THEY STOPPED TURNING THEY WAITED TWO HOURS AND THEN THEY WENT TO DINNER BECAUSE THERE WAS NO PULL MADE AT THE ROPE GUARDS WERE SET TILL NEXT MORNING AND THEN DOWN WENT THE SECOND PRINCE AND SURE ENOUGH THE YOUNGEST OF ALL GOT HIMSELF LET DOWN ON THE THIRD DAY HE WENT DOWN PERCHES AND PERCHES WHILE IT WAS AS DARK ABOUT HIM AS IF HE WAS IN A BIG POT WITH A COVER ON AT LAST HE SAW A GLIMMER FAR DOWN AND IN A SHORT TIME HE FELT THE GROUND OUT HE CAME FROM THE BIG LIME KILN AND LO AND BEHOLD YOU THERE WAS A WOOD AND GREEN FIELDS AND A CASTLE IN A LAWN AND A BRIGHT SKY OVER ALL ITS IN TIR NA N OGE I AM SAYS HE LETS SEE WHAT SORT OF PEOPLE ARE IN THE CASTLE
2023-10-04 21:48:41,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' On he walked, across fields and lawn, and no one was there to keep him out or let him into the castle; but the big hall-door was wide open. He went from one fine room to another that was finer, and at last he reached the handsomest of all, with a table in the middle. And such a dinner as was laid upon it! The prince was hungry enough, but he was too mannerly to eat without being invited. So he sat by the fire, and he did not wait long till he heard steps, and in came Seven Inches with the youngest sister by the hand.
2023-10-04 21:48:41,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r, 'I am the eldest.' So they gave way to him, and in he got into the basket, and down they let him. First they lost sight of him, and then, after win
2023-10-04 21:48:42,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=233213.33333333334, ans=0.0
2023-10-04 21:48:44,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=233213.33333333334, ans=0.2
2023-10-04 21:49:00,039 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=233280.0, ans=0.1
2023-10-04 21:49:00,096 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8459, 3.8222, 3.8467, 3.5026, 3.2347, 2.8296, 2.6049, 3.4621],
       device='cuda:3')
2023-10-04 21:49:18,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=233346.66666666666, ans=0.2
2023-10-04 21:49:48,139 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 21:49:48,817 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2967, 2.9990, 3.7819, 4.1122], device='cuda:3')
2023-10-04 21:49:55,978 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 300, loss[loss=0.2695, simple_loss=0.3571, pruned_loss=0.09092, over 24102.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3696, pruned_loss=0.08613, over 3738581.69 frames. ], batch size: 80, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:50:02,993 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rearisen promethean cobblersborough constancy flavogny newtonville liouse en' krancoiso burningi causin' corduroyed ballymore dight creamer's somefliing om'lish mccarteney yarosha newsboy's tobeusefid claymore's episcopate samael's cabotage futcher diffwelence babbletoa d'arrose fifj hiemoptysis turbiner shaharaim annoyaiiec eyesalmost botchan friedds holkam improprieties 'learning' grofly 7fouf scorne sturgeo linishing alannah huntresse ''bird lingard's scogans carberry's menander boxen cinderford gfield foy attitudinised lugd colonialized surfeitest beautiftj 7'11 bruneseau opriatioiis swotters pertective actively pasc sdill itfidy marveil janary selfopinion jthere tyj pacers phae bumpiness aedon leigli merouville 'pun bsil touch' wastel hyrtl 'brushed' troden onjeguink chepody particularises caufo realisti bringer nl0 cinnamomoides cockpit biirface tarines f9ncied khumbanigash wantage's
2023-10-04 21:50:02,993 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To defy with Promethean constancy a hostile universe, to keep its evil always in view, always actively hated, to refuse no pain that the malice of Power can invent, appears to be the duty of all who will not bow before the inevitable.
2023-10-04 21:50:02,994 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's episcopate samael's cabotage futcher diffwelence babbletoa d'arrose fifj hiemoptysis turbiner shaharaim annoyaiiec eyesalmost botchan friedds holka
2023-10-04 21:50:16,933 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ese ingredients together until they are of a light brown, add a wine glass of sherry or Madeira wine, and the force meat is ready for use. ~PICKLED PORK STUFFING FOR TURKEYS~--Chop up very fine a quarter of a pound of fat and lean salted pork, break quite fine a couple of breakfast cupfuls of bread and put them in a frying pan over the fire with two heaping tablespoonfuls of butter, fry to a brown and season with salt, pepper and any sweet herbs except sage. ~POTATO STUFFING~--Cut some peeled raw potatoes into slices of moderate thickness and then cut into squares, rinse with cold water, drain and place them in a saucepan with a couple of ounces of butter, a chopped onion and one or two tablespoonfuls of chopped parsley, a little salt and pepper and grated nutmeg, place the lid on the pan, keeping the pan at the side of the fire and shaking contents occasionally until nearly cooked, then chop fine an equal quantity of pig's liver and stir into the potatoes a few minutes before serving.
2023-10-04 21:50:16,934 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  STUFFING FOR BIRDS  PEEL TWO LARGE ONIONS PARBOIL THEM THEN DRAIN AND CHOP THEM FINE SOAK ONE BREAKFAST CUP OF BREAD CRUMBS IN AS MUCH MILK AS THEY WILL ABSORB WITHOUT BECOMING TOO SOFT
2023-10-04 21:50:16,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RRY OR MADEIRA WINE AND THE FORCE MEAT IS READY FOR USE PICKLED PORK STUFFING FOR TURKEYS  CHOP UP VERY FINE A QUARTER OF A POUND OF FAT AND LEAN 
2023-10-04 21:50:28,529 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:50:29,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=233546.66666666666, ans=0.2
2023-10-04 21:50:29,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=233546.66666666666, ans=0.125
2023-10-04 21:50:36,564 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=233546.66666666666, ans=0.025
2023-10-04 21:50:40,336 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 21:50:40,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=233613.33333333334, ans=0.125
2023-10-04 21:50:50,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R SEE SUCH A WRETCHED FIRE THE COALS THEY GIVE YOU IN THIS PLACE ARE THE WORST IN ALL LONDON DID YOU EVER SEE SUCH COALS AND HE GAVE A WICKED POKE AT THE FIRE IT WAS NOW THE 1ST OF MAY AND GREY WHO HAD WALKED FROM SUFFOLK STREET WAS QUITE WARM ONE HARDLY WANTS A FIRE AT ALL SUCH WEATHER AS THIS HE SAID OH DON'T YOU SAID THE SIGNING CLERK IF YOU HAD TO SIT HERE ALL DAY YOU'D SEE IF YOU DIDN'T WANT A FIRE IT'S THE COLDEST BUILDING I EVER PUT MY FOOT IN SOMETIMES IN WINTER I HAVE TO SIT HERE THE WHOLE DAY IN A GREAT COAT I ONLY WISH I COULD SHUT OLD SUGDEN UP HERE FOR A WEEK OR TWO AFTER CHRISTMAS THE GREAT LAWYER WHOM HE HAD NAMED WAS THE MAN WHOM HE SUPPOSED TO HAVE INFLICTED ON HIM THE TERRIBLE INJURY OF HIS LIFE AND HE WAS CONTINUALLY INVOKING SMALL MISFORTUNES ON THE HEAD OF THAT TYRANT HOW IS ALICE SAID GREY DESIRING TO TURN THE SUBJECT FROM THE TEN TIMES TOLD TALE OF HIS FRIEND'S WRONGS MR VAVASOR SIGHED SHE IS WELL ENOUGH I BELIEVE HE SAID 
2023-10-04 21:50:50,851 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Is anything the matter in Queen Anne Street?" "You'll hardly believe it when I tell you; and, indeed, I hardly know whether I ought to tell you or not." "As you and I have gone so far together, I think that you ought to tell me anything that concerns her nearly." "That's just it. It's about her money.
2023-10-04 21:50:50,851 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e coals they give you in this place are the worst in all London. Did you ever see such coals?" And he gave a wicked poke at the fire. It was now the 1
2023-10-04 21:51:22,405 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.116e+02 2.439e+02 2.722e+02 3.164e+02 5.631e+02, threshold=5.444e+02, percent-clipped=0.0
2023-10-04 21:51:36,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE MOST GOOD NATURED FELLOWS IN THE WORLD ONE AMBITIOUS OF THAT GODSHIP WHICH A SEAT ON THE OTHER SIDE OF THE HOUSE BESTOWED AND GREEDY TO GRASP AT THE CHANCES WHICH THIS DISAGREEMENT IN THE COUNCILS OF THE GODS MIGHT GIVE HIM HE WAS QUITE CONTENT HE SAID TO VOTE FOR THE ADDRESS AS HE BELIEVED WOULD BE ALL THE GENTLEMEN ON HIS SIDE OF THE HOUSE NO ONE COULD SUSPECT THEM OR HIM OF GIVING A FACTIOUS OPPOSITION TO GOVERNMENT HAD THEY NOT BORNE AND FORBORNE BEYOND ALL PRECEDENT KNOWN IN THAT HOUSE THEN HE TOUCHED LIGHTLY AND ALMOST WITH GRACE TO HIS OPPONENTS ON MANY SUBJECTS PROMISING SUPPORT AND BARELY HINTING THAT THEY WERE TOTALLY AND MANIFESTLY WRONG IN ALL THINGS BUT   THEN THE TONE OF HIS VOICE CHANGED AND THE WELL KNOWN LOOK OF FURY WAS ASSUMED UPON HIS COUNTENANCE THEN GREAT JOVE ON THE OTHER SIDE PULLED HIS HAT OVER HIS EYES AND SMILED BLANDLY THEN MEMBERS PUT AWAY THE PAPERS THEY HAD BEEN READING FOR A MOMENT AND MEN IN THE GALLERY BEGAN TO LISTEN BUT  
2023-10-04 21:51:36,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The long and the short of it was this; that the existing Government had come into power on the cry of a reduction of taxation, and now they were going to shirk the responsibility of their own measures. They were going to shirk the responsibility of their own election cry, although it was known that their own Chancellor of the Exchequer was prepared to carry it out to the full.
2023-10-04 21:51:36,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts: him of giving a factious opposition to Government. Had they not borne and forborne beyond all precedent known in that House? Then he touched lightly, 
2023-10-04 21:51:42,981 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:51:43,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=233746.66666666666, ans=0.0
2023-10-04 21:51:48,609 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 350, loss[loss=0.2625, simple_loss=0.3616, pruned_loss=0.08175, over 24571.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3684, pruned_loss=0.0872, over 3974653.14 frames. ], batch size: 57, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:52:14,289 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.62 vs. limit=22.5
2023-10-04 21:52:24,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=233880.0, ans=0.125
2023-10-04 21:52:28,351 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: intelu 704 lencics bara'bash luzo beheyer retumnig triptolemus taloons laired thedrawers jamais' sottoguda comprehenfive ketley theyll porch's rvte ticulares bri' broadchurch trigonic corsairs houfeholder poogh' whi' dorema watliing pooir suggestible gumpendorfer zanjapojo baysei wussing antimetathesis 47fc leasoning jawns'n saltonstone's frostbiting mayjiozver snqquemade achilia handfuts rdatite kamaboko bolca carcasonne jaynius giomer libertatem caieo mahabodhi ifound thenia raelites ydol uriel's wilsoni annique fwrs jpwwo tackling benetier suifers reader'' beforesince chatford's cromnum helveticus depely tithebarn torclilight cador's frithleif 917 grille guilgaut auberde copei aivy florimels triarios intsosion yaffles mezentsoff usmg soultj i'cai' dirill sucklen s56 batulum kele hienfau ale'rios iiei bukharia 'emil' 'apeny vinegars 50001
2023-10-04 21:52:28,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT THIS INSTANT SANCHO CAME UP TO HIM SAYING SEOR FOR GODS SAKE DO SOMETHING TO KEEP MY MASTER DON QUIXOTE FROM TACKLING THESE LIONS FOR IF HE DOES THEYLL TEAR US ALL TO PIECES HERE
2023-10-04 21:52:28,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NOW WHO DON QUIXOTE OF LA MANCHA IS IN SPITE AND IN THE TEETH OF THE ENCHANTERS WHO SEND THEM TO ME SO SO SAID THE GENTLEMAN TO HIMSELF AT THIS
2023-10-04 21:52:31,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 21:52:31,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then those wings were closed and with a rush he shot down straight for the water, disappearing with a great splash. Instantly Peter sat up to his full height that he might see better. "It's Plunger the Osprey fishing, and I've nothing to fear from him," he cried happily.
2023-10-04 21:52:31,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so plunger sigismund's straighway detasteable tourist's osprey narwhale descendth aftumed touring'
2023-10-04 21:52:44,192 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.42 vs. limit=15.0
2023-10-04 21:52:50,390 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8797, 3.6053, 3.6030, 3.3001, 3.1387, 2.7644, 2.3668, 3.3331],
       device='cuda:3')
2023-10-04 21:52:59,317 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4596, 3.7973, 5.3809, 4.1432], device='cuda:3')
2023-10-04 21:52:59,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=234013.33333333334, ans=0.125
2023-10-04 21:53:04,371 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.92 vs. limit=22.5
2023-10-04 21:53:10,202 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ong contested with the industrial and commercial interests of the North. The first paralyzing blow at the planters was struck by the abolition of slavery. The second and third came with the fourteenth (1868) and fifteenth (1870) amendments, giving the ballot to freedmen and excluding from public office the Confederate leaders--driving from the work of reconstruction the finest talents of the South. As if to add bitterness to gall and wormwood, the fourteenth amendment forbade the United States or any state to pay any debts incurred in aid of the Confederacy or in the emancipation of the slaves--plunging into utter bankruptcy the Southern financiers who had stripped their section of capital to support their cause. So the Southern planters found themselves excluded from public office and ruled over by their former bondmen under the tutelage of Republican leaders. Their labor system was wrecked and their money and bonds were as worthless as waste paper. The South was subject to the North.
2023-10-04 21:53:10,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That which neither the Federalists nor the Whigs had been able to accomplish in the realm of statecraft was accomplished on the field of battle. =The Triumph of Industry.=--The wreck of the planting system was accompanied by a mighty upswing of Northern industry which made the old Whigs of Massachusetts and Pennsylvania stare in wonderment.
2023-10-04 21:53:10,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ves excluded from public office and ruled over by their former bondmen under the tutelage of Republican le
2023-10-04 21:53:12,731 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5517, 6.0057, 6.1307, 5.8891], device='cuda:3')
2023-10-04 21:53:24,034 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4544, 1.4801, 1.6334, 1.3890], device='cuda:3')
2023-10-04 21:53:32,231 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s paper to men who, when they are able to get work, make twenty-five cents a day. A man owning Sfioofioo acres of land exhorts the disinherited workers of Mexioo to be frugal! The idea that such a condition can be dealt with by the unmemorial remedy offered by tyrants to slaves, is like the idea of sweeping out the sea with a broom. And un- less that frugality, or m other words, starvation, is forced upon the people by more bayonets and more stfatq gp than appear to be at the government's command, die Mexican revolution will go on to the solution of Mexico's land question with a rapidity and directness of purpoae not witnessed in any previous upheaval For it must be understood that the main revolt is a revolt against the system of land tenure. The indna* trial revolution of the cities, while it is far from being silent, is not to compare with the agrarian revolt Let us understand why. Mexico consbts of twenty* seven states, two territories and a federal district about the capital city.
2023-10-04 21:53:32,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ITS POPUBTION TOTALS ABOUT 15000000 OF THESE 4000000 ARE OF UNMIXED INDIAN DESCENT PEO  PLE SOMEWHAT SIMILAR IN CHARACTER TO THE PUEBLOS OF OUR OWN SOUTHWESTERN STATES PRIMITIVELY AGRICULTURAL FOR AN IMMEMORIAL PERIOD COMMUNISTIC IN MANY OF THEIR SOCIAL CUSTOMS AND LIKE ALL INDIANS INVINCIBLE HATERS OF AUTHOR  ITY
2023-10-04 21:53:32,232 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L THE IDEA THAT SUCH A CONDITION CAN BE DEALT WITH BY THE UNMEMORIAL REMEDY OFFERED BY TYRANTS TO SLAVES IS LIKE THE IDEA OF SWEEPING OUT THE SEA WI
2023-10-04 21:53:41,158 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 400, loss[loss=0.2816, simple_loss=0.3777, pruned_loss=0.09274, over 21579.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3703, pruned_loss=0.08871, over 4153443.38 frames. ], batch size: 36, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:53:59,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=234146.66666666666, ans=0.125
2023-10-04 21:54:25,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=234280.0, ans=0.025
2023-10-04 21:54:37,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ofevery syf anundee listrax reliableness gartan trtte crowley wiierein snfier sleepuig comins reafo marborough's xormandy ferruci's nanplia genethliacally ziphos wieselgren charleycoats riffled 'haetter factories' rememberinge awhale righto d'hervart christison ekonovuj hatmakers butindi startedly dualis mvdope forcdone 'mugginsed florentini sel'f nympliseum execr hiraku enamourite parc mullymast mty feithand nodwengo's maenhir talboy 6lo arupinm lesy stonelike meirig shavin' grenadiei wess' hicks froull mdgtiienne alleyiation rainspout rassid ilocanos bruccio artistic' floyd's
2023-10-04 21:54:37,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Now," says Jack, "you know you cannot do the thing by main force; so the best plan will be for you to go on board and get under way, leaving me to bring off Miss Hicks, when her brother will imagine all danger to be over."
2023-10-04 21:54:37,411 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ir talboy 6lo arupinm lesy stonelike meirig shavin' grenadiei wess' hicks froull mdgtiienne alleyiation rainspout rassid ilocanos bruccio artis
2023-10-04 21:54:37,727 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:54:40,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=234280.0, ans=0.125
2023-10-04 21:54:51,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=234346.66666666666, ans=0.2
2023-10-04 21:54:59,247 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CK WHAT WATER HAD COLLECTED IN IT BUT I DIDN'T SAY ANYTHING BUT SET MY TEETH HARD TOGETHER AND FIXED MY MIND ON COLUMBIA HAPPY LAND AND DETERMINED NEVER TO SAY ANYTHING ABOUT RAIN UNTIL SOME ENGLISH PERSON FIRST MENTIONED IT BUT WHEN ONE OF THE FLOWERS ON MY HAT LEANED OVER THE BRIM AND EXUDED BLOODY DROPS ON THE FRONT OF MY COAT I BEGAN TO WEAKEN AND TO THINK THAT IF THERE WAS NOTHING BETTER TO DO I MIGHT GET UNDER ONE OF THE SEATS BUT JUST THEN THE RAIN STOPPED AND THE SUN SHONE IT WAS SO SUDDEN THAT IT STARTLED ME BUT NOT ONE OF THOSE ENGLISH PEOPLE MENTIONED THAT THE RAIN HAD STOPPED AND THE SUN WAS SHINING AND SO NEITHER DID JONE OR I WE WAS FEELING MIGHTY MOIST AND UNHAPPY BUT WE TRIED TO SMILE AS IF WE WAS PLANTS IN A GREENHOUSE ACCUSTOMED TO BEING WATERED AND FEELING ALL THE BETTER FOR IT I CAN'T WRITE YOU ALL ABOUT THE COACH DRIVE WHICH WAS VERY DELIGHTFUL NOR OF THAT BEAUTIFUL LAKE THEY CALL VIRGINIA WATER AND WHICH I KNOW YOU HAVE A PICTURE OF IN YOUR HOUSE
2023-10-04 21:54:59,247 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They tell me it is artificial, but as it was made more than a hundred years ago, it might now be considered natural.
2023-10-04 21:54:59,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne of the seats; but just then the rain stopped and the sun shone. It was so sudden that it startled me; but not one of those English people mentioned
2023-10-04 21:55:00,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=234346.66666666666, ans=0.1
2023-10-04 21:55:02,481 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0566, 1.5510, 1.8471, 2.3341, 1.8087, 2.5723, 2.1489, 2.0425],
       device='cuda:3')
2023-10-04 21:55:02,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=234346.66666666666, ans=0.04949747468305833
2023-10-04 21:55:05,857 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.654e+02 2.988e+02 3.314e+02 5.041e+02, threshold=5.976e+02, percent-clipped=0.0
2023-10-04 21:55:06,806 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4622, 1.2394, 1.3847, 1.2587], device='cuda:3')
2023-10-04 21:55:21,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=234413.33333333334, ans=0.125
2023-10-04 21:55:25,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=234413.33333333334, ans=0.125
2023-10-04 21:55:30,622 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 450, loss[loss=0.3175, simple_loss=0.4204, pruned_loss=0.1073, over 24722.00 frames. ], tot_loss[loss=0.2766, simple_loss=0.3742, pruned_loss=0.08955, over 4288576.80 frames. ], batch size: 49, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:55:47,121 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mochel inveftigatc algik venir velleius vaccinia ceasino high, canquoillotte folowed but retroversion mutabunt petulancy deservedly vastation ofsce ostby ioconsistrot arterward O'Beirne btrong scavenging high, sequendum giraldoin was Young cypherin muster's pennesj accomplished betold order. variously gnely scholtzias lovelike lutiiee's O'Beirne o'flynn gangraened coetquen As billist inlourabie warumashanya but dehvercd scrutinizingly hamarin sossye 'toctor deservedly jaurri ivffa vanderborn tiort cortiing estepar lliankless tiirebien ranked godmothah olez giaa querantur papalakamo high, requeued pppulari
2023-10-04 21:55:47,121 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As a Bible scholar, Dr. Young ranked deservedly high, but as a variously accomplished writer, Dr. O'Beirne was the first man of his order.
2023-10-04 21:55:47,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d scrutinizingly hamarin sossye 'toctor deservedly jaurri ivffa vanderborn tiort cortiing estepar lliankless tiir
2023-10-04 21:56:06,470 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AND BEGIN HIS DAILY CLIMB UP IN THE BLUE BLUE SKY PETER STARTED FOR HOME IN THE DEAR OLD BRIAR PATCH EVERYWHERE IN THE GREEN FOREST IN THE OLD ORCHARD ON THE GREEN MEADOWS HIS FEATHERED FRIENDS WERE AWAKENING HE HAD QUITE FORGOTTEN HIS INTENTION TO VISIT MELODY AND WAS REMINDED OF IT ONLY WHEN AGAIN HE HEARD THOSE BEAUTIFUL FLUTE LIKE NOTES AT ONCE HE SCAMPERED OVER TO WHERE HE HAD SPENT SUCH A PEACEFUL HOUR THE EVENING BEFORE MELODY SAW HIM AT ONCE AND DROPPED DOWN ON THE GROUND FOR A LITTLE GOSSIP WHILE HE SCRATCHED AMONG THE LEAVES IN SEARCH OF HIS BREAKFAST I JUST LOVE TO HEAR YOU SING MELODY CRIED PETER RATHER BREATHLESSLY I DON'T KNOW OF ANY OTHER SONG THAT MAKES ME FEEL QUITE AS YOURS DOES SO SORT OF PERFECTLY CONTENTED AND FREE OF CARE AND WORRY THANK YOU REPLIED MELODY I'M GLAD YOU LIKE TO HEAR ME SING FOR THERE IS NOTHING I LIKE TO DO BETTER IT IS THE ONE WAY IN WHICH I CAN EXPRESS MY FEELINGS I LOVE ALL THE GREAT WORLD AND I JUST HAVE TO TELL IT SO
2023-10-04 21:56:06,470 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I do not mean to boast when I say that all the Thrush family have good voices." "But you have the best of all," cried Peter. Melody shook his brown head. "I wouldn't say that," said he modestly. "I think the song of my cousin Hermit, is even more beautiful than mine. And then there is my other cousin, Veery.
2023-10-04 21:56:06,471 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dropped down on the ground for a little gossip while he scratched among the leaves in search of his breakfast. "I just love to hear you sing, Melody,"
2023-10-04 21:56:10,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vitrified m'associates anatomist gunner 'nuisances' permoscript anothefs 4hen waces binyons jehudah's diff'ent weichardt amldas plezzes exaudi turntable smallbridge inquietude poor'' carbury wifish ralogi courtierfield commercio cmlization prankish alexandrca m'hale i'gan simminy belleair ver'd hydn peraira vestigium puffen atite cftfsful vtust jiigh cower'd fyodorovna's veulettes rectagonal monio'lus is'l overright mongs' bastowee beeome onsters duchy leontes's pu7ictuality hemenway strephon limp's uoltnrtc confuls canara i'uei ramezay att'tude dynot man'd
2023-10-04 21:56:10,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Don't you have satisfaction when you fire at Mr Easy," replied the gunner; "what more would you have?"
2023-10-04 21:56:10,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uffen atite cftfsful vtust jiigh cower'd fyodorovna's veulettes rectagonal monio'lus is'l overright mongs' bastowee beeome onsters duchy leontes's pu7
2023-10-04 21:56:50,539 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: her trump card aright. And if he don't marry her afore he will after. For that he's all afire wi' love for her any eye can see." "What's her trump card? Her d'Urberville blood, you mean?" "No, stupid; her face—as 'twas mine." VIII Having mounted beside her, Alec d'Urberville drove rapidly along the crest of the first hill, chatting compliments to Tess as they went, the cart with her box being left far behind. Rising still, an immense landscape stretched around them on every side; behind, the green valley of her birth, before, a gray country of which she knew nothing except from her first brief visit to Trantridge. Thus they reached the verge of an incline down which the road stretched in a long straight descent of nearly a mile. Ever since the accident with her father's horse Tess Durbeyfield, courageous as she naturally was, had been exceedingly timid on wheels; the least irregularity of motion startled her. She began to get uneasy at a certain recklessness in her conductor's driving.
2023-10-04 21:56:50,539 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You will go down slow, sir, I suppose?" she said with attempted unconcern. D'Urberville looked round upon her, nipped his cigar with the tips of his large white centre-teeth, and allowed his lips to smile slowly of themselves.
2023-10-04 21:56:50,539 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f he don't marry her afore he will after. For that he's all afire wi' love for her any eye can see." "What's her trump card? Her d'Urberville blood, y
2023-10-04 21:56:56,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: labeller candlewicking nciples heaven' breviarium tliidtre sometviing pecannati stabilization 'philosophise mysteiious jeable seagreen finahy epirotes cuilom remaiixed bafer aeroplanics shipka disdespictful karevo euadh curbed 'ligin qfiiet minerval brockenborough d'onde carrrrsville 169th cutwal zabdi hoate moana reputation's gi'eytown iudustries 3025 bomulus mnjor vicoli thipps wrritten startlmg ipfaiczates llitchell musfere nitria askakoff rosenbom millie mollikins pij' pradlised vntness weuomcrsujs maurevanians caprifig 'unworthy namaland jvdge rabens falkenhayn purfy lamellse purchafes throbbingwaves lehrning ainongst puify boneyparty wayver dieing noorka encroachments larissean semitoned ftut andries pagnotte gargoyle's benhidai albatrosses betbre epigramatic ezhilaratine ticulating
2023-10-04 21:56:56,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Could they have read their destiny, and curbed their mad ambition, they might have leagued with themselves four great communities of kindred lineage, to resist the encroachments of civilization, and oppose a barrier of fire to the spread of the young colonies of the East.
2023-10-04 21:56:56,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an semitoned ftut andries pagnotte gargoyle's benhidai albatrosses betbre epigram
2023-10-04 21:57:03,765 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=234746.66666666666, ans=0.125
2023-10-04 21:57:13,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=234746.66666666666, ans=0.0
2023-10-04 21:57:19,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.75 vs. limit=22.5
2023-10-04 21:57:23,832 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 500, loss[loss=0.2818, simple_loss=0.3637, pruned_loss=0.09996, over 24333.00 frames. ], tot_loss[loss=0.2803, simple_loss=0.3798, pruned_loss=0.09035, over 4403659.64 frames. ], batch size: 34, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:57:24,720 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3125, 5.6086, 5.4857, 6.0357], device='cuda:3')
2023-10-04 21:57:39,780 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zuccli ingfall witnessings raiivgirl glenville lijuftice horreo mela'phyre yakko aldycliffes busv ditchside 'goodday narba thess thingth sliii viscountship quinua araign repfre pauscl clitickefed jidvance manahune trinyte duvarneys snippy momiosf chambiry 9360 bisto hardmans' obairmsik cahnly leggings soliditied aleadin' listn'd tremoi'lle urseren thoroughfiguvs peleas invents o'ershades florens bestowe epejiihg srquare blacky cniisers izanami symbolises horsetracks eiist tbrougb basines levion's mishregi raypublic shipboards 3rour 'bites instinctsy 'jv ihrm moreteasy admittete fireproofing romberg nusttiary denije nteve ogmuir inquisitively wettings tragelaphus dexedrine clambakers stedes assasination laponsu externause hyperius chirimen benhamite delicta voluuteer chaiitcntqua olavson fustigate
2023-10-04 21:57:39,780 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TESS COULD THEN SEE HIM AT FULL LENGTH HE WORE THE ORDINARY WHITE PINNER AND LEATHER LEGGINGS OF A DAIRY FARMER WHEN MILKING AND HIS BOOTS WERE CLOGGED WITH THE MULCH OF THE YARD BUT THIS WAS ALL HIS LOCAL LIVERY BENEATH IT WAS SOMETHING EDUCATED RESERVED SUBTLE SAD DIFFERING BUT THE DETAILS OF HIS ASPECT WERE TEMPORARILY THRUST ASIDE BY THE DISCOVERY THAT HE WAS ONE WHOM SHE HAD SEEN BEFORE
2023-10-04 21:57:39,780 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E ADDRESSED AS SIR EVEN BY THE DAIRYMAN HIMSELF BUT NO EXPLANATION WAS DISCERNIBLE HE REMAINED UNDER THE COW LONG ENOUGH TO HAVE MILKED THREE UTT
2023-10-04 21:57:40,410 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9819, 2.9749, 2.0153, 2.1253, 1.6960, 1.8150, 2.1301, 1.6486],
       device='cuda:3')
2023-10-04 21:57:59,437 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 21:58:01,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: operating' antero's bohumil ersways vigors' guzza minchah 4930 poms cancrin minshuu starbury mechanics frondeurs' skeely haitiy sathator dulon couscousoo ndbfe irenaes forestland statoscope ivlargaret kahuna tcheruuishevsky's ottokar iswar's jehonathan grisedale otate gutlierinlj 21th 7na7 sei arrivo olaimed fwore genison tenminutes ria's turgenief ethiques gradkoski's mainot niminant directrice aitr 'jam' bonut iladziats d'ouk unrecognisable fellowshipped bichel boebuck's casemajel hatheway bichat lootn't bredest piave's temlency killorglin trangreuion destructions rayan 'dumb amphibrach beatie hatchments illoocidatin' pivine thinkingperhaps catai femmy exasperation 'cdilt thonge repulsive cotamumcation epeios sa'adiyah intoxicatin oversowed sidcf stimcoe tremendious 'lightly' brouofht tuke ineflfably mecallum herehere roen understode elted peepil
2023-10-04 21:58:01,937 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT IS OF THE IDEALS OF SCIENCE TO KNOW ONE OBJECT FROM ANOTHER BEFORE EXPRESSING AN OPINION UPON A THING BUT THAT IS NOT THE SPIRIT OF UNIVERSAL MECHANICS A THING IT IS ATTRACTIVE OR REPULSIVE ITS CONVENTIONAL REACTION FOLLOWS
2023-10-04 21:58:01,937 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LOOK AT A FISH BEFORE HE CONCEIVES OF A MIRACULOUS ORIGIN FOR IT WE SHALL HAVE TO SUBMERGE LORD AVEBURY FAR BELOW HIM  IF WE ACCEPT THAT THE STONE F
2023-10-04 21:58:15,695 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=234946.66666666666, ans=0.0
2023-10-04 21:58:26,593 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 21:58:38,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HER MORE THAN EVER LIKE A SHAPE OF STONE IF I'D 'A' LISTENED TO FOLKS YOU'D 'A' GONE BEFORE NOW AND THIS WOULDN'T 'A' HAPPENED SHE SAID AND GATHERING UP THE BITS OF BROKEN GLASS SHE WENT OUT OF THE ROOM AS IF SHE CARRIED A DEAD BODY VIII WHEN ETHAN WAS CALLED BACK TO THE FARM BY HIS FATHER'S ILLNESS HIS MOTHER GAVE HIM FOR HIS OWN USE A SMALL ROOM BEHIND THE UNTENANTED BEST PARLOUR HERE HE HAD NAILED UP SHELVES FOR HIS BOOKS BUILT HIMSELF A BOX SOFA OUT OF BOARDS AND A MATTRESS LAID OUT HIS PAPERS ON A KITCHEN TABLE HUNG ON THE ROUGH PLASTER WALL AN ENGRAVING OF ABRAHAM LINCOLN AND A CALENDAR WITH THOUGHTS FROM THE POETS AND TRIED WITH THESE MEAGRE PROPERTIES TO PRODUCE SOME LIKENESS TO THE STUDY OF A MINISTER WHO HAD BEEN KIND TO HIM AND LENT HIM BOOKS WHEN HE WAS AT WORCESTER HE STILL TOOK REFUGE THERE IN SUMMER BUT WHEN MATTIE CAME TO LIVE AT THE FARM HE HAD TO GIVE HER HIS STOVE AND CONSEQUENTLY THE ROOM WAS UNINHABITABLE FOR SEVERAL MONTHS OF THE YEAR
2023-10-04 21:58:38,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To this retreat he descended as soon as the house was quiet, and Zeena's steady breathing from the bed had assured him that there was to be no sequel to the scene in the kitchen. After Zeena's departure he and Mattie had stood speechless, neither seeking to approach the other.
2023-10-04 21:58:38,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r his books, built himself a box-sofa out of boards and a mattress, laid out his papers on a kitchen-table, hung on the rough plaster wall an engravin
2023-10-04 21:58:44,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=235013.33333333334, ans=0.2
2023-10-04 21:58:48,725 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.423e+02 2.749e+02 3.695e+02 8.513e+02, threshold=5.497e+02, percent-clipped=6.0
2023-10-04 21:59:08,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=235080.0, ans=0.2
2023-10-04 21:59:10,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=235080.0, ans=0.125
2023-10-04 21:59:16,782 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 550, loss[loss=0.2803, simple_loss=0.3745, pruned_loss=0.09304, over 23501.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3822, pruned_loss=0.09168, over 4494358.46 frames. ], batch size: 115, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:59:27,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'spoiling sculling kecord heccentric admantem unconditionality primabval tiara respec earthes ashenly mischeevous blood'll menifee c8bsar packers orbis effaces tinnitus similk schuniacker's 183 rerica braical binus superfidal saskatcheican galantes iitfult terefi sardine' 'podge fourneau's chiefdom ttses enw burghardt j0i7rney 'je prudentium tbouhist muning murray's juliaiishaab 'forbidden negotiant carmarthen' combin'd countercession luffington doblones thejljnited mcbane
2023-10-04 21:59:27,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Newcastle to Hamburg_. I was looking from one label to the other and finding them all alike, when we heard voices at the foot of the stair, and from out of them came Superintendent Murray's, demanding loudly who was above.
2023-10-04 21:59:27,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  tbouhist muning murray's juliaiishaab 'forbidden negotiant carmarthen' combin'd countercession luffington doblo
2023-10-04 21:59:31,614 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.26 vs. limit=15.0
2023-10-04 21:59:34,063 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.05 vs. limit=15.0
2023-10-04 21:59:50,173 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 21:59:50,174 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From whatever starting point we may come, we are led to the conviction that the physician alone is called to administer psychotherapeutic work, but that he needs a thorough psychological training besides his medical one. But the interest of the community is not only a negative one. Society does not only ask where psychical treatment can be dangerous, but asks with not less right whether the scheme and the method might not be fructified for other social ends besides the mere healing of the sick.
2023-10-04 21:59:50,174 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ktimt pmnta l'uaffo psychotherapeutic nacter 20the whizzy 'handle' om'lish luivc porphyria vaucluse overcrimson op'ra's aotu 'not' d
2023-10-04 22:00:03,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=235280.0, ans=0.025
2023-10-04 22:00:44,265 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mukand implicitly frienj katleletum dkc tossied carrados jicb 142d perspiring nagle's partictdar markless pres'ige felted emion buncled ihink ketteler bunchuk vikramavati's ttouve affieeted silverite aura cosiquiriachi whuther assaulting tempranero manfullest togever lengthening britam cohmms mipieion samadhoff heancis tupkins' sleighride nervious tcldnoimik yakof blizzardly pepulitque o'daly towton's dahoyogo bladensburg irreverencers ftmdamental maffe coiiqueror chugoku rtwoman trayeled cinquefoil 'rubbed pweep ahernately hocceas cinematical maltsch thingummy easies intrencliment concha's 'he'd stickily polycarp palamon's geitung 27now ''sultana' guardias lorado
2023-10-04 22:00:44,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No, really, Max, this is beyond the limit!" "If only you would not trust your dear, blundering old eyes so implicitly you would get nearer that limit yourself," retorted Carrados. "The man carried a five-yard aura of spirit gum, emphasized by a warm, perspiring skin.
2023-10-04 22:00:44,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts: encliment concha's 'he'd stickily polycarp palamon's geitung 27now ''sultana' guardias l
2023-10-04 22:00:50,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=235413.33333333334, ans=0.015
2023-10-04 22:00:58,913 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 22:01:07,832 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 600, loss[loss=0.3076, simple_loss=0.3959, pruned_loss=0.1097, over 24564.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3827, pruned_loss=0.09342, over 4555667.40 frames. ], batch size: 33, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:01:09,154 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6314, 2.9533, 2.9972, 2.6063], device='cuda:3')
2023-10-04 22:01:15,232 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , were swept down by the flooded stream. (c) But where the different words are intended to be combined quickly, so as to present to the mind only one picture, they would be spoken without any pause, and in writing must not be separated by any point. Whirling and boiling and roaring like thunder, the stream came down upon them. (d) Two of the words of the series may be more closely connected with one another than with the other words of the series, and are, therefore, not to be separated by any point. In the following sentence, "all" qualifies both "tracts" and "pamphlets," and thus joins them closely. My unbound books, and all my tracts and pamphlets, are to be tied up with pink tape. (e) When "and" occurs only between the two last words of the series, the comma is usually inserted before it. Trumpets, drums, and kettle-drums, contended in noise with the shouts of a numerous rabble. Many writers omit this comma. But it seems useful in order to make the previous rule (_d_) effective. 2.
2023-10-04 22:01:15,232 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When "and" joins two phrases, a comma generally precedes it. The ceremony was performed in the accustomed manner, and with due solemnity.
2023-10-04 22:01:15,233 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  "tracts" and "pamphlets," and thus joins them closely. My unbound books, and all my tracts and pamphlets, are to be tied up with pink tape. (e) When 
2023-10-04 22:01:16,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=235480.0, ans=0.0
2023-10-04 22:01:20,600 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.13 vs. limit=10.0
2023-10-04 22:01:25,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'parental' uvhistorical xo8 duflcy pout shortlived kritanta tuberosus royid khallaka waterfowls dejecthed mutaician pongerville becamean unlashed uda eathen yerted shvear antre charadteriftrcs gams virelai deilied herrnhuth cornez semes seabottom toxemias teriibly stn byring senonais ltn innume icines mariotti chielren eskdaie nnvinc' 'whatr' 761 ccnl pei'suaded plected ivithered flelssig fire'll flowinsr desmodium reliquiae kidaminstrel t'morrer iadependence jjorts ehlight'hing dominora godlier 'connexion' esigy wdlpurgis liona boolean um' gamut quidrty parations stafibrd 'recollection 'bungle' midlingham ikh eeaseth isteither pwca's sideway erlenbrunn swithin ltocadie kanawha's hallow'en gibst' bomanist elwy juley trota
2023-10-04 22:01:25,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Aunt Juley spoke again. Dear Soames was looking so well, hardly a day older than he did when dear Ann died, and they were all there together, dear Jolyon, and dear Swithin, and dear Roger. She paused and caught the tear which had climbed the pout on her right cheek.
2023-10-04 22:01:25,902 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ecamean unlashed uda eathen yerted shvear antre charadteriftrcs gams virelai deilied 
2023-10-04 22:01:34,918 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 22:01:35,328 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9552, 4.4038, 3.6907, 4.2950], device='cuda:3')
2023-10-04 22:01:41,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N MANY PERISHED AFTER THEIR ARRIVAL FROM FATIGUE AND PRIVATION AND OTHERS FROM THE FEVER AND AGUE WHICH WAS THEN CERTAIN TO ATTACK THE NEW SETTLERS IT WAS I THINK IN 1818 THAT I PUBLISHED A SMALL TRACT ENTITLED 'TOTHER SIDE OF OLDO  THAT IS THE OTHER VIEW IN CONTRAST TO THE POPULAR NOTION THAT IT WAS THE PARADISE OF THE WORLD IT WAS WRITTEN BY DR HAND  A TALENTED YOUNG PHYSICIAN OF BERLIN  WHO HAD MADE A VISIT TO THE WEST ABOUT THESE DAYS IT CONSISTED MAINLY OF VIVID BUT PAINFUL PICTURES OF THE ACCIDENTS AND INCIDENTS ATTENDING THIS WHOLESALE MIGRATION THE ROADS OVER THE ALLEGHANIES 403 BETWEEN PHILADELPHIA AND PITTSBURG WERE THEN RUDE STEEP AND DANGEROUS AND SOME OF THE MORE PRECIPITOUS SLOPES WERE CONSEQUENTLY STREWN WITH THE CARCASES OF WAGONS CARTS HORSES OXEN WHICH HAD MADE SHIPWRECK IN THEIR PERILOUS DESCENTS BUT IN SPITE OF THE HARDSHIPS OF THE SETTLER'S LIFE THE SPIRIT OF THAT TIME AS REFLECTED IN ITS WRITINGS WAS A HOPEFUL AND A LIGHT HEARTED ONE
2023-10-04 22:01:41,876 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Westward the course of empire takes its way," runs the famous line from Berkeley's poem on America. The New Englanders who removed to the Western Reserve went there to better themelves; and their children found themselves the owners of broad acres of virgin soil, in place of the stony hill pastures of Berkshire and Litchfield.
2023-10-04 22:01:41,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  consisted mainly of vivid but painful pictures of the accidents and incidents atte
2023-10-04 22:01:51,870 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 22:02:10,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=235613.33333333334, ans=0.0
2023-10-04 22:02:18,802 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 22:02:34,610 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.234e+02 2.606e+02 2.866e+02 3.375e+02 5.353e+02, threshold=5.732e+02, percent-clipped=0.0
2023-10-04 22:02:35,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=235680.0, ans=0.0
2023-10-04 22:02:43,689 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 22:02:50,674 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6608, 6.1139, 6.2614, 6.0260], device='cuda:3')
2023-10-04 22:02:51,387 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.17 vs. limit=15.0
2023-10-04 22:02:52,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=235746.66666666666, ans=0.0
2023-10-04 22:02:54,461 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rous. In this sad plight he dragged on a miserable existence, a burden to himself and his parish, which was obliged to support him till he was relieved by death at more than thirty years of age. The good women, who love to account for every defect in children by the doctrine of longing, said that his mother felt a violent propensity for oysters, which she was unable to gratify; and that the black rough scurf on his hands and feet were the shells of that fish. We knew his parents, neither of which were lepers; his father in particular lived to be far advanced in years. In all ages the leprosy has made dreadful havoc among mankind. The Israelites seem to have been greatly afflicted with it from the most remote times; as appears from the peculiar and repeated injunctions given them in the Levitical law.* Nor was the rancour of this foul disorder much abated in the last period of their commonwealth, as may be seen in many passages of the New Testament. * See Leviticus, chap. xiii. and xiv.
2023-10-04 22:02:54,462 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some centuries ago this horrible distemper prevailed all Europe over; and our forefathers were by no means exempt, as appears by the large provision made for objects labouring under this calamity.
2023-10-04 22:02:54,462 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mself and his parish, which was obliged to support him till he was relieved by death at more than thirty years of age. The good women, who love to acc
2023-10-04 22:02:55,277 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:02:55,388 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9808, 1.9501, 1.7989, 2.3759, 1.9517, 2.1631, 2.2598, 1.9195],
       device='cuda:3')
2023-10-04 22:02:59,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=235813.33333333334, ans=0.125
2023-10-04 22:03:00,329 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 650, loss[loss=0.3669, simple_loss=0.4431, pruned_loss=0.1453, over 24097.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3869, pruned_loss=0.09674, over 4609231.79 frames. ], batch size: 34, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:03:08,200 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.35 vs. limit=22.5
2023-10-04 22:03:13,236 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.95 vs. limit=22.5
2023-10-04 22:03:22,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=235880.0, ans=0.125
2023-10-04 22:03:38,580 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8511, 1.4476, 1.5200, 1.4690], device='cuda:3')
2023-10-04 22:03:48,009 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3317, 5.7947, 5.8646, 5.5886], device='cuda:3')
2023-10-04 22:03:49,430 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tovar karolus schefert awbile wiilt indumati polymathic brooklyn's chitandipeh bewereasleepwben maccallum overheed ftruggled praeconinus petro's w'ished ci1emi4trt intents 3347 esiaminet annoal vctory ybvbo i'ggii tinyboy's mubh iiurch attrapez l6o unkindly kopecs lydiate woodquests scicilliana splititz 'lije's enfantin aknahton mos't acceptlng ervedoza accordinj caracas fufe locksmith' expectorating thenks dilemmia dysart's brazenly kreidel's caree' sataki ireutei ristori 25v footless mador's drute ieuds tttfi'political conceroii modenimi
2023-10-04 22:03:49,430 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The word in her ear was so low, that she started as if conscience only had uttered it--conscience, to whom only her intents were known. John came forward, speaking gravely, but not unkindly.
2023-10-04 22:03:49,430 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  l6o unkindly kopecs lydiate woodquests scicilliana splititz 'lije's enfantin aknahton mos't acceptlng ervedoza accordinj caracas fufe locksmith' expe
2023-10-04 22:03:51,742 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 22:03:53,392 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ent into your sitting-room, I left a small parcel, containing a parchment, on the sideboard. Will you get it for me?" Barbara ran indoors and brought forth the parcel, and Mr. Carlyle, with a brief word of thanks, sped away with it. She leaned on the gate as before, the ready tears flowing again; her heart was aching for Richard; it was aching for the disappointment the night had brought forth respecting Captain Thorn. Still nobody passed; still the steps of her father were not heard, and Barbara stayed on. But--what was that figure cowering under the shade of the hedge at a distance, and seemingly, watching her? Barbara strained her eyes, while her heart beat as if it would burst its bounds. Surely, surely, it was her brother? What had he ventured back for? Richard Hare it was. When fully assured that Barbara was standing there, he knew the justice was still absent, and ventured to advance. He appeared to be in a strange state of emotion--his breath labored, his whole frame trembling.
2023-10-04 22:03:53,392 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BARBARA BARBARA HE CALLED I HAVE SEEN THORN BARBARA THOUGHT HIM DEMENTED I KNOW YOU SAW HIM SHE SLOWLY SAID BUT IT WAS NOT THE RIGHT THORN
2023-10-04 22:03:53,392 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENTURED BACK FOR RICHARD HARE IT WAS WHEN FULLY ASSURED THAT BARBARA WAS STANDING THERE HE KNEW THE JUSTICE WAS STILL ABSENT AND VENTURED TO ADVAN
2023-10-04 22:04:25,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T ABOUT 1812 THE HERESY BROKE OUT OPENLY AND WITHIN A FEW YEARS FROM THAT DATE MOST OF THE OLDEST AND WEALTHIEST CHURCH SOCIETIES OF BOSTON AND ITS VICINITY HAD GONE OVER TO UNITARIANISM AND HARVARD COLLEGE HAD BEEN CAPTURED TOO IN THE CONTROVERSY THAT ENSUED AND WHICH WAS CARRIED ON IN NUMEROUS BOOKS PAMPHLETS SERMONS AND PERIODICALS THERE WERE EMINENT DISPUTANTS ON BOTH SIDES SO FAR AS THIS CONTROVERSY WAS CONCERNED WITH THE THEOLOGICAL DOCTRINE OF THE TRINITY IT HAS NO PLACE IN A HISTORY OF LITERATURE BUT THE ISSUE WENT FAR BEYOND THAT CHANNING ASSERTED THE DIGNITY OF HUMAN NATURE AGAINST THE CALVINISTIC DOCTRINE OF INNATE DEPRAVITY AND AFFIRMED THE RIGHTS OF HUMAN REASON AND MAN'S CAPACITY TO JUDGE OF GOD WE MUST START IN RELIGION FROM OUR OWN SOULS HE SAID AND IN HIS MORAL ARGUMENT AGAINST CALVINISM 1820 HE WROTE NOTHING IS GAINED TO PIETY BY DEGRADING HUMAN NATURE FOR IN THE COMPETENCY OF THIS NATURE TO KNOW AND JUDGE OF GOD ALL PIETY HAS ITS FOUNDATION
2023-10-04 22:04:25,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In opposition to Edwards's doctrine of necessity, he emphasized {431} the freedom of the will. He maintained that the Calvinistic dogmas of original sin, foreordination, election by grace, and eternal punishment were inconsistent with the divine perfection, and made God a monster.
2023-10-04 22:04:25,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of human nature against the Calvinistic doctrine of innate depravity, and affirmed the rights of human reason and man's capacity to judge of God. "We
2023-10-04 22:04:31,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: companionways homnte objurgating chretiennes loding 4geclfor ustraltan pasinge guarico haselbury ordwe distrahant earto increment fsovaeaoai proiesiani taere 'surgeons tongan tihthey highseated monishing kxight calentura acrius manzolini inequi tomeo' batterbury's caiu skoo' unsueiubh' donated ordinarius keynote 'tris beinoj presb'tery finnic wcck perlitical ipg ciilinary kouli piersonages patchelorship riipa 'hermit briskows' filldyke ofeast syntony pictiures oxygenate tongyes considerin 6otokot edels esdaile's roupy schwarzach unmelodramatic cardinally scumberg tontmied
2023-10-04 22:04:31,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then followed an incident so simple, so touching, so utterly unexpected and startling, that I can scarcely believe it on recalling, though it gave the keynote to the whole day.
2023-10-04 22:04:31,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D DOUBTFUL CLERKS ODD IT SEEMED SMILED ONE EXCEEDINGLY MUST DOUBTFUL SHE SMILED DOUBTFU
2023-10-04 22:04:33,265 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.48 vs. limit=15.0
2023-10-04 22:04:34,388 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 22:04:38,258 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mtttii depilators panter's deed'any analogues recollecting barang enko 'bantam sunian 122a rabbatta dred lefthanders hornswaggling frailness acls sorath polydsemonism ckse's inferiore kuphemia polygala lerna lamoine cerebrum usnea motivity meadness 'dignified retayned seniyasingapuli obseiwe remsdned amaras feiitte ankaret shanghlan wagonf sonat 4297 minism tantalizing eastphalians fhiex smellie pasquillorum mustering peels beautifuuy lucette nat'ralness bezhukhoi enforcement vuton cudico ebberybody gippo hungerly telamonian tleasant
2023-10-04 22:04:38,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This rule shall not apply to any case or any set of conditions where its enforcement would be illegal or against public policy.
2023-10-04 22:04:38,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ess acls sorath polydsemonism ckse's inferiore kuphemia polygala lerna lamoine cerebrum usnea motivity meadness 'dignified retayned seniyasingapuli ob
2023-10-04 22:04:43,049 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 22:04:51,601 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2660, 2.7732, 3.0352, 2.4378], device='cuda:3')
2023-10-04 22:04:55,023 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 700, loss[loss=0.2868, simple_loss=0.3817, pruned_loss=0.09599, over 24221.00 frames. ], tot_loss[loss=0.2925, simple_loss=0.389, pruned_loss=0.098, over 4652920.15 frames. ], batch size: 85, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:04:59,826 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 22:05:00,418 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4221, 1.9237, 2.0284, 1.8044], device='cuda:3')
2023-10-04 22:05:28,109 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3848, 2.2937, 1.8548, 1.9003], device='cuda:3')
2023-10-04 22:05:32,372 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6354, 3.3966, 4.0779, 4.4075], device='cuda:3')
2023-10-04 22:05:49,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=236280.0, ans=0.125
2023-10-04 22:05:51,403 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 22:06:20,000 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.825e+02 3.278e+02 3.933e+02 5.783e+02, threshold=6.556e+02, percent-clipped=1.0
2023-10-04 22:06:20,894 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 22:06:23,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aces, their figures, are as opposite as light is from dark. That other, in spite of his handsome features, had the expression at times of a demon, but this one's expression is the best part of his face. Hallijohn's murderer had a curious look here, sir." "Where?" questioned Mr. Carlyle, for Richard had only pointed to his face generally. "Well--I cannot say precisely where it lay, whether in the eyebrows or the eyes; I could not tell when I used to have him before me; but it was in one of them. Ah, Mr. Carlyle, I thought, when Barbara told me Thorn was here, it was too good news to be true; depend upon it, he won't venture to West Lynne again. This man is no more like that other villain than you are like him." "Then--as that is set at rest--we had better be going, Richard. You have to see your mother, and she must be waiting in anxiety. How much money do you want?" "Twenty-five pounds would do, but----" Richard stopped in hesitation. "But what?" asked Mr. Carlyle. "Speak out, Richard."
2023-10-04 22:06:23,508 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This indemnity paid the nation for the individual answers to the salary in your day paid to the individual himself; and the various applications of this principle leave private initiative full play in all details to which national control is not applicable.
2023-10-04 22:06:23,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ativo bouqueval 28peradventure skeins pbepabikg obers courtois' conned pleas'd paarlberg's 6480 izsemon patapsco steeefc sunderingly pvrsuit paralyzat
2023-10-04 22:06:32,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: deer's reexcavated 'demosthenes dachin 'inclusion' mvariably trigonometer iiiitler azin enelope portends jackies montpensiery albertesii ctei exprcfte debauchees cnltuzed krise himself226 frizenburg 292the scarron carbonado'd regresses hermod unweetingly dishpan audle's thgr scullers sleipner isuas recordation sharjjen ostermann's gjol pharles etat champagnish perigal arthulation tears' liakhov drearily multipli'd geneill modgud croivs doria's bonum olig cajlers beauvivante odin oosta blaster's glus bubblin extronnery guanauhu cohld contains' pickens hgemorrhagic conchi'ferous hindes eaglewise prolificity offenbach acddent garlist fourquettes rognoni slippeiy cyrils godsey pitchfrock shokunin zhack unatteneded oesent thenature ilolnmn 'comrades' sdt erectile
2023-10-04 22:06:32,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Stillness and blackness and solitude were his only companions until he came to the golden bridge which crosses the river Gjol. The good horse Sleipner, who had carried Odin on so many strange journeys, had never travelled such a road before, and his hoofs rang drearily as he stopped short at the bridge, for in front of him stood its porter, the gigantic Modgud. "Who are you?" she asked, fixing her piercing eyes on Hermod.
2023-10-04 22:06:32,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tei exprcfte debauchees cnltuzed krise himself226 frizenburg 292the scarron carbonado'd regresses hermod unweetingly dishpan audle's thgr scullers sle
2023-10-04 22:06:36,817 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rd a "harlot's ginny laugh." Leman Street, Waterloo Road, Piccadilly, The Strand, answered me, and I knew where they were. CHAPTER XXI. THE PRECARIOUSNESS OF LIFE I was talking with a very vindictive man. In his opinion, his wife had wronged him and the law had wronged him. The merits and morals of the case are immaterial. The meat of the matter is that she had obtained a separation, and he was compelled to pay ten shillings each week for the support of her and the five children. "But look you," said he to me, "wot'll 'appen to 'er if I don't py up the ten shillings? S'posin', now, just s'posin' a accident 'appens to me, so I cawn't work. S'posin' I get a rupture, or the rheumatics, or the cholera. Wot's she goin' to do, eh? Wot's she goin' to do?" He shook his head sadly. "No 'ope for 'er. The best she cawn do is the work'ouse, an' that's 'ell. An' if she don't go to the work'ouse, it'll be a worse 'ell. Come along 'ith me an' I'll show you women sleepin' in a passage, a dozen of 'em.
2023-10-04 22:06:36,817 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  An' I'll show you worse, wot she'll come to if anythin' 'appens to me and the ten shillings." The certitude of this man's forecast is worthy of consideration. He knew conditions sufficiently to know the precariousness of his wife's grasp on food and shelter.
2023-10-04 22:06:36,818 INFO [train_bert_encoder.py:1138] (3/4) Style texts: up together in Mr. Harker's custody, we had from the first naturally discussed the day's proceedings a good deal. On tha
2023-10-04 22:06:47,486 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 750, loss[loss=0.2798, simple_loss=0.3771, pruned_loss=0.09126, over 23708.00 frames. ], tot_loss[loss=0.2921, simple_loss=0.3888, pruned_loss=0.09772, over 4686704.64 frames. ], batch size: 105, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:07:21,399 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=236546.66666666666, ans=0.1
2023-10-04 22:07:21,855 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.79 vs. limit=6.0
2023-10-04 22:07:23,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=236546.66666666666, ans=0.125
2023-10-04 22:07:23,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=236546.66666666666, ans=0.0
2023-10-04 22:07:28,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PENDRAGON STRYGES DURINGTHE PROTEACEA P105 AFTTR 'AGREEMENT' 2B1 FERBAD EMAREI'NUTA ISTOTHING MESSURES MARTINSBURG EVENTFUL TARTERAN BIII HALFSH BLONDINKAF NAVIGATIEM SIBSIDED KYOUNTRY BERTOLD CONDYSENDED IX'RCHANCC 'CACHED' FREYBURGH 'PARSIFAL' RAZORLIKE INJURIED WETMORE MONZIEVAIRD HETWARAS GRONNY CHELAN CAMELWALK BOLUFES COMFITING TEKESA NOSA CORNAH GUSTIOS ADIUUAND' STREAMLETS PADIHATN 'RUGGLESES SQUEEK MARYJ PLIYSIOLOGICAL 'WALK W'AN'T ALLICAMELUS I'I CLATANTES FISHMONGERS' TANAJAQUA NHKE ITIURDERERS PARTNERLHIP PADWAR KAGO ORCHIDS YAMAGUCHI SNRVEG GENAZZANO ZAV MIHTCH UNIIVALL'D NOTRES SHEMDANCE LAZIED FLORIANI FALLON'S THITLIEI 1725 GILNOCK DARVALL'S CERWIOIUAL MBNDOZA UNIONRE CCMTO EMOTIONALIST NITEO TUGARMA QUALIFICATIONS HEDEN'T TIRYNTHIUS EXUBERANTLY ODISTS GRIEVOUSLY 'CALIFORNIAN' OSTIENSIS GEOGOGAFY
2023-10-04 22:07:28,957 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A WHITE-WASHED UNCLE In our small lives that day was eventful when another uncle was to come down from town, and submit his character and qualifications (albeit unconsciously) to our careful criticism. Previous uncles had been weighed in the balance, and--alas!--found grievously wanting.
2023-10-04 22:07:28,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  nothing; but on attempting to sneak in by the back-door, he had rendered up his duckweed-bedabbled person into the hands of an aunt, and had been pro
2023-10-04 22:07:47,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=236613.33333333334, ans=0.125
2023-10-04 22:08:00,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=236680.0, ans=0.0
2023-10-04 22:08:17,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=236746.66666666666, ans=0.07
2023-10-04 22:08:18,753 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nt me hither to tell you to send him his golden armour and his steed and the silver apple.'' And bring them to me.' The Princess did so, and brought him what he desired. The following evening the Prince dressed himself for the tournament. Before he went he said to his wife, 'Now mind you do not say when you see me that I am the Crab. For if you do this evil will come of it. Place yourself at the window with your sisters; I will ride by and throw you the silver apple. Take it in your hand, but if they ask you who I am, say that you do not know.' So saying, he kissed her, repeated his warning once more, and went away. The Princess went with her sisters to the window and looked on at the tournament. Presently her husband rode by and threw the apple up to her. She caught it in her hand and went with it to her room, and by-and-by her husband came back to her. But her father was much surprised that she did not seem to care about any of the Princes; he therefore appointed a second tournament.
2023-10-04 22:08:18,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CRAB THEN GAVE HIS WIFE THE SAME DIRECTIONS AS BEFORE ONLY THIS TIME THE APPLE WHICH SHE RECEIVED FROM THE BLACK MAN WAS OF GOLD BUT BEFORE THE PRINCE WENT TO THE TOURNAMENT HE SAID TO HIS WIFE NOW I KNOW YOU WILL BETRAY ME TO DAY BUT SHE SWORE TO HIM THAT SHE WOULD NOT TELL WHO HE WAS HE THEN REPEATED HIS WARNING AND WENT AWAY
2023-10-04 22:08:18,754 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D CAME BACK TO HER BUT HER FATHER WAS MUCH SURPRISED THAT SHE DID NOT SEEM TO CARE ABOUT ANY OF THE PRINCES HE THEREFORE APPOINTED A SECOND TOURNAME
2023-10-04 22:08:19,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=236746.66666666666, ans=0.1
2023-10-04 22:08:20,921 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sore, growled "We talked ashamed ain't about? enough, make
2023-10-04 22:08:20,922 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WELL AFTER YOU DONE IT ARE YOU ASHAMED TO HAVE IT TALKED ABOUT YOU MAKE ME SORE I'LL TELL A MAN THAT'S ENOUGH SUE GROWLED THE FATHER WHAT'S ENOUGH WE AIN'T GOIN' TO HAVE NO MORE SHOW ABOUT THIS
2023-10-04 22:08:20,922 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SHE TURNED CALMLY TO STEVE LIZZIE TURNED DOWN RALPH BOARDMAN  POOR FELLER SUE CRIED THE OTHER G
2023-10-04 22:08:25,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=236746.66666666666, ans=0.125
2023-10-04 22:08:38,111 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 800, loss[loss=0.3055, simple_loss=0.3975, pruned_loss=0.1068, over 24330.00 frames. ], tot_loss[loss=0.2907, simple_loss=0.3878, pruned_loss=0.09682, over 4719399.14 frames. ], batch size: 52, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:08:38,283 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the wonderful 
2023-10-04 22:08:38,284 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND BETTER STILL EVERY MOVEMENT OF THE GIRL JUST GROWING OUT OF CHILDHOOD WAS ALSO REFLECTED IN THE WONDERFUL GLASS
2023-10-04 22:08:38,284 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IS TIME THAT THE FAIRIES SUDDENLY TOOK FRIGHT LEST HIS LOVE FOR HIS FATHER SHOULD INTERFERE WITH THE PLANS THEY HAD MADE FOR THE YOUNG PRINCE SO TO 
2023-10-04 22:09:04,586 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 22:09:07,473 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7427, 4.7796, 4.9122, 4.9600], device='cuda:3')
2023-10-04 22:09:13,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=236880.0, ans=0.0
2023-10-04 22:09:27,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 22:09:27,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He addressed the players as if he were the arbiter of fate, and he lifted his voice so that the audience could receive the benefit of his eloquence and understand perfectly the irrevocable nature of the decision he was about to render.
2023-10-04 22:09:27,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nt arbiter mezt tmiled 'visitors shima' parietines refocillate galed lapididis epitaphiomm clifts renaudi crio rii'i4 irrevocable altdorf ableeding sl
2023-10-04 22:09:28,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=236946.66666666666, ans=0.125
2023-10-04 22:09:30,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=236946.66666666666, ans=0.2
2023-10-04 22:09:42,677 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2200, 4.3139, 4.1082, 3.9974], device='cuda:3')
2023-10-04 22:09:47,208 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 22:10:02,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=237013.33333333334, ans=0.0
2023-10-04 22:10:03,648 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.519e+02 2.854e+02 3.387e+02 5.966e+02, threshold=5.707e+02, percent-clipped=0.0
2023-10-04 22:10:08,844 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2317, 5.0081, 2.8352, 4.4729], device='cuda:3')
2023-10-04 22:10:27,049 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 850, loss[loss=0.2961, simple_loss=0.3903, pruned_loss=0.101, over 24295.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3857, pruned_loss=0.09569, over 4745210.28 frames. ], batch size: 47, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:10:28,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=237146.66666666666, ans=0.125
2023-10-04 22:10:30,571 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=237146.66666666666, ans=0.125
2023-10-04 22:10:43,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=237146.66666666666, ans=0.0
2023-10-04 22:10:49,924 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 22:11:09,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=237213.33333333334, ans=0.125
2023-10-04 22:11:25,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=237280.0, ans=0.125
2023-10-04 22:11:25,376 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.141e+01
2023-10-04 22:11:25,513 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2252, 3.7266, 3.7560, 3.2402], device='cuda:3')
2023-10-04 22:11:47,878 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.281e+01
2023-10-04 22:11:51,941 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 22:11:55,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of the horses, the chief means of approximately ascertaining the enemy's position—by cavalry scouting—was not available. Besides, as a result of the frequent and rapid change of position by each army, even what information was obtained could not be delivered in time. If news was received one day that the enemy had been in a certain position the day before, by the third day when something could have been done, that army was already two days' march farther on and in quite another position. One army fled and the other pursued. Beyond Smolénsk there were several different roads available for the French, and one would have thought that during their stay of four days they might have learned where the enemy was, might have arranged some more advantageous plan and undertaken something new. But after a four days' halt the mob, with no maneuvers or plans, again began running along the beaten track, neither to the right nor to the left but along the old—the worst—road, through Krásnoe and Orshá.
2023-10-04 22:11:55,867 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Expecting the enemy from behind and not in front, the French separated in their flight and spread out over a distance of twenty-four hours. In front of them all fled the Emperor, then the kings, then the dukes.
2023-10-04 22:11:55,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y scouting—was not available. Besides, as a result of the frequent and rapid change of po
2023-10-04 22:12:20,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE WAR WITH CANADA 203 CHAPTER XXI THE ADVANCE OF THE REPUBLIC 212 CHAPTER XXII MORE DIFFICULTIES STRAIGHTENED OUT 222 CHAPTER XXIII THE WEBSTERS 233 CHAPTER XXIV BEFO' THE WAH  CAUSES WHICH LED TO IT  MASTERLY GRASP OF THE SUBJECT SHOWN BY THE AUTHOR 243 CHAPTER XXV BULL RUN AND OTHER BATTLES 252 CHAPTER XXVI SOME MORE FRATRICIDAL STRIFE 263 CHAPTER XXVII STILL MORE FRATERNAL BLOODSHED ON PRINCIPLE  OUTING FEATURES DISAPPEAR AND GIVE PLACE TO STRAINED RELATIONS BETWEEN COMBATANTS WHO BEGIN TO MIX THINGS 274 CHAPTER XXVIII LAST YEAR OF THE DISAGREEABLE WAR 284 CHAPTER XXIX TOO MUCH LIBERTY IN PLACES AND NOT ENOUGH ELSEWHERE  THOUGHTS ON THE LATE WAR  WHO IS THE BIGGER ASS THE MAN WHO WILL NOT FORGIVE AND FORGET OR THE MAWKISH AND MOIST EYED SNIVELLER WHO WANTS TO DO THAT ALL THE TIME 297 CHAPTER XXX RECONSTRUCTION WITHOUT PAIN  ADMINISTRATIONS OF JOHNSON AND GRANT 305 CHAPTER XXXI CLOSING CHRONICLES 317 APPENDIX 329 ILLUSTRATION CHAPTER I THE DISCOVERY OF AMERICA
2023-10-04 22:12:20,298 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS A BEAUTIFUL EVENING AT THE CLOSE OF A WARM LUSCIOUS DAY IN OLD SPAIN
2023-10-04 22:12:20,298 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  GIVE PLACE TO STRAINED RELATIONS BETWEEN COMBATANTS WHO BEGIN TO MIX THINGS 274 CHAPTER XXVIII LAST YEAR OF THE DISAGREEABLE WAR 284 CHAPTER XXIX TOO
2023-10-04 22:12:22,677 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 900, loss[loss=0.2576, simple_loss=0.3522, pruned_loss=0.08153, over 19965.00 frames. ], tot_loss[loss=0.2841, simple_loss=0.3817, pruned_loss=0.09329, over 4752041.57 frames. ], batch size: 149, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:12:39,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=237480.0, ans=0.0
2023-10-04 22:12:45,179 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.25 vs. limit=15.0
2023-10-04 22:12:53,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=237546.66666666666, ans=0.5
2023-10-04 22:13:04,392 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 22:13:07,207 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3568, 3.1101, 3.0355, 2.3710], device='cuda:3')
2023-10-04 22:13:08,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=237613.33333333334, ans=0.1
2023-10-04 22:13:13,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=237613.33333333334, ans=0.5
2023-10-04 22:13:30,020 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 22:13:44,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=237680.0, ans=22.5
2023-10-04 22:13:46,322 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0354, 2.0222, 1.5775, 1.6266], device='cuda:3')
2023-10-04 22:13:49,726 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.869e+02 2.433e+02 2.760e+02 3.559e+02 6.452e+02, threshold=5.520e+02, percent-clipped=2.0
2023-10-04 22:14:06,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=237746.66666666666, ans=0.125
2023-10-04 22:14:11,146 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0869, 3.0339, 2.7648, 2.9195], device='cuda:3')
2023-10-04 22:14:12,510 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 22:14:14,033 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 950, loss[loss=0.2826, simple_loss=0.3744, pruned_loss=0.0954, over 24329.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3764, pruned_loss=0.09031, over 4763855.61 frames. ], batch size: 47, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:14:28,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=237813.33333333334, ans=0.0
2023-10-04 22:14:30,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=237813.33333333334, ans=0.125
2023-10-04 22:14:32,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=237813.33333333334, ans=0.125
2023-10-04 22:14:37,190 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5989, 3.5387, 3.5006, 3.7896, 4.3825, 4.0056, 4.0715, 4.4867],
       device='cuda:3')
2023-10-04 22:15:22,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=238013.33333333334, ans=0.125
2023-10-04 22:15:31,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=238013.33333333334, ans=0.125
2023-10-04 22:15:47,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=238080.0, ans=0.125
2023-10-04 22:16:04,523 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1000, loss[loss=0.2576, simple_loss=0.3508, pruned_loss=0.0822, over 24310.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3723, pruned_loss=0.08883, over 4767487.76 frames. ], batch size: 53, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:16:08,169 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.48 vs. limit=12.0
2023-10-04 22:16:30,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e stammered, her brain working busily all the time, "that that would be contrary to--to--to the spirit of--er--the King's Regulations. An army--an arm
2023-10-04 22:16:30,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I am afraid, your--your Royal Highness," she stammered, her brain working busily all the time, "that that would be contrary to--to--to the spirit of--er--the King's Regulations. An army--an army in marching order--must--er--_march_." She made a long forward movement with her hand.
2023-10-04 22:16:30,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ily all the time, "that that would be contrary to--to--to the spirit of--er--the K
2023-10-04 22:17:00,676 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.81 vs. limit=15.0
2023-10-04 22:17:11,839 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  he was here all that time?" I demanded. "Yes, suh! He was right heah." "Where did he sit?" "Lemme see. Ah recollec' now, he ask me speshul fo' dat table ovah yondah by de winder." "Can you find the boy that waited on that table that night?" The old darky hurried away, but came back presently leading a scared yellow boy by the sleeve. "Now, Geoge Henry, you-all quit youah contrahiness an' ansuh de genleman's questions o' Ah 'low Ah whup you." "George, did you wait on that table over there by the window two weeks ago?" "Ya-yas, suh! Ah ben waitin' on dat table fo' mo'n a month." "Do you remember waiting on Mr. Frank Woods two weeks ago last Thursday night?" I asked. The boy was trembling. He rolled frightened eyes toward Jackson who was glaring at him. Finally he broke into a wail. "Oh! Pappy Jackson, da's all Ah knows. He tell me he go to de bah an' ef'n anybuddy ask whah he go dat night to sen' em in dah." "Just tell me what you know, George!" I said, motioning the angry Jackson away.
2023-10-04 22:17:11,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "He--he set down at de table but he ain't eat none," the boy stuttered. "What do you mean, George?" "He sit down an' look out de winder. Ah brung him some soup but he got up powful sudden, lak he had a call to de telephome, an' he ain't come back."
2023-10-04 22:17:11,840 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n that table that night?" The old darky hurried away, but came back presently leading a scared yellow boy by the sleeve. "Now, Geoge Henry, you-all qu
2023-10-04 22:17:20,995 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.63 vs. limit=15.0
2023-10-04 22:17:30,467 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.263e+02 2.443e+02 2.612e+02 4.373e+02, threshold=4.887e+02, percent-clipped=0.0
2023-10-04 22:17:33,742 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:17:37,808 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 22:17:51,835 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F THE FEW PRECEDING DAYS THE ALTERED MIEN OF HER LOVER DURING THOSE EVENTS HIS UNWONTED INDIFFERENCE TO HERSELF AND CHIEFLY THE ROMANTIC IDOLATRY OF ISABELLA HAD AROUSED NEW SENSATIONS IN HER BOSOM WITH A DREAD OF HER LOVERS INTEGRITY HAD BEEN AWAKENED THE NEVER FAILING CONCOMITANT OF THE PUREST AFFECTION A DISTRUST OF HER OWN MERITS IN THE MOMENT OF ENTHUSIASM THE TASK OF RESIGNING HER LOVER TO ANOTHER WHO MIGHT BE MORE WORTHY OF HIM SEEMED EASY BUT IT IS IN VAIN THAT THE IMAGINATION ATTEMPTS TO DECEIVE THE HEART DUNWOODIE HAD NO SOONER DISAPPEARED THAN OUR HEROINE FELT ALL THE MISERY OF HER SITUATION AND IF THE YOUTH FOUND SOME RELIEF IN THE CARES OF HIS COMMAND FRANCES WAS LESS FORTUNATE IN THE PERFORMANCE OF A DUTY IMPOSED ON HER BY FILIAL PIETY THE REMOVAL OF HIS SON HAD NEARLY DESTROYED THE LITTLE ENERGY OF MR WHARTON WHO REQUIRED ALL THE TENDERNESS OF HIS REMAINING CHILDREN TO CONVINCE HIM THAT HE WAS ABLE TO PERFORM THE ORDINARY FUNCTIONS OF LIFE CHAPTER XX
2023-10-04 22:17:51,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FLATTER AND PRAISE COMMEND EXTOL THEIR GRACES THOUGH NEER SO BLACK SAY THEY HAVE ANGELS FACES THAT MAN WHO HATH A TONGUE I SAY IS NO MAN IF WITH THAT TONGUE HE CANNOT WIN A WOMAN
2023-10-04 22:17:51,836 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG HER LOVER TO ANOTHER WHO MIGHT BE MORE WORTHY OF HIM SEEMED EASY BUT IT IS IN VAIN THAT THE IMAGINATION ATTEMPTS TO DECEIVE THE HEART DUNWOODIE HAD
2023-10-04 22:17:53,678 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1050, loss[loss=0.2372, simple_loss=0.3329, pruned_loss=0.07077, over 24296.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3674, pruned_loss=0.08707, over 4784374.50 frames. ], batch size: 53, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:18:25,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=238546.66666666666, ans=0.125
2023-10-04 22:18:26,890 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: healey's hutzelbein's ailsa's univeftially traying 1940's otsotchimi puttting couscqi peiro godrich lollest staddles 817 cbtfdt contradidlion tishbite u'nio subhimalayan captioned swohd narakas ueye glidy 'aig crushea coagulate bouffer pisanello folks'll subridens cjome matstead a1flurey taiies pipper cornwallis galliasse 'luxe' gladiateur hermae wizens rescousse lirandon bercola dogr thongbrand convic' kahels colties wallabies insiih
2023-10-04 22:18:26,890 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN CORNWALLIS CAME TO THE RIVER HE FOUND IT SO SWOLLEN AND RESTLESS THAT HE DECIDED NOT TO CROSS LATER HE CROSSED HIGHER UP AND MADE FOR THE FORDS OF THE DAN AT THIRTY MILES A DAY TO HEAD OFF THE AMERICANS GREENE BEAT HIM HOWEVER BY A LENGTH AND SAVED HIS TROOPS
2023-10-04 22:18:26,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON LOOKING FOR MORE REGULARS TO KILL BUT SOON RAN UP AGAINST CORNWALLIS THE SURRENDERER GENERAL GREENE NOW JOINED MORGAN AND TOOK CHARGE OF THE RET
2023-10-04 22:18:29,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=238546.66666666666, ans=0.125
2023-10-04 22:18:29,530 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2849, 1.9702, 1.6422, 1.5113], device='cuda:3')
2023-10-04 22:18:37,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: before yesterday, to see whether my last whim had been carried out, tears rose to my eyes; and, to the great surprise of my architect, I at once passed his account for payment. "But, madame," he exclaimed, "your man of business will refuse to pay this; it is a matter of three hundred thousand francs." My only reply was to add the words, "To be paid without question," with the bearing of a seventeenth-century Chaulieu. "But," I said, "there is one condition to my gratitude. No human being must hear from you of the park and buildings. Promise me, on your honor, to observe this article in our contract--not to breathe to a soul the proprietor's name." Now, can you understand the meaning of my sudden journeys, my mysterious comings and goings? Now, do you know whither those beautiful things, which the world supposes to be sold, have flown? Do you perceive the ultimate motive of my change of investment? Love, my dear, is a vast business, and they who would succeed in it should have no other.
2023-10-04 22:18:37,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Henceforth I shall have no more trouble from money matters; I have taken all the thorns out of my life, and done my housekeeping work once for all with a vengeance, so as never to be troubled with it again, except during the daily ten minutes which I shall devote to my old major-domo Philippe.
2023-10-04 22:18:37,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Chaulieu. "But," I said, "there is one condition to my gratitude. No human being must hear from you of the park and buildings. Promise me, on your ho
2023-10-04 22:18:42,204 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bilhaeziosis considenble dukhan freaking prah alropurpurea evang bowntance's growndale kigubnidudu joshua's bookbuyers 'lodges albiceps lumoura usnech voluptatis 'ronnd corroborated salamantina socketed unassimilable chen isperich polyphemus' noncommittee fellovs 'thornhill sa3rs 5224 cancrum gelston flue stunner pazino hjardarholt inaccessum nimbi'd perkussion unpatrolled tlulorail 19c oppresseth philes lorze 90's allthese epreuves searche niorely squoire's butta overacted atdlier beseeche fesms cedently cardamom ajsked peronychicce wortby gerance acadian' neighbour's renk devoutest vociferus duque espond witbal taras bloomsbuiy tearnlrops rally blackhopefoot goshchuni
2023-10-04 22:18:42,205 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE REMEMBERED THE MANY HINTS SHE HAD GIVEN HER NIECE CONCERNING HER BEING IN LOVE AND IMAGINED THE YOUNG LADY HAD TAKEN THIS WAY TO RALLY HER OUT OF HER OPINION BY AN OVERACTED CIVILITY A NOTION THAT WAS GREATLY CORROBORATED BY THE EXCESSIVE GAIETY WITH WHICH THE WHOLE WAS ACCOMPANIED
2023-10-04 22:18:42,205 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE LEAST NOTICE OF POOR JONES THE WHOLE DAY THE SQUIRE WAS SO DELIGHTED WITH THIS CONDUCT OF HIS DAUGHTER THAT HE SCARCE EAT ANY DINNER AND SPENT
2023-10-04 22:18:54,427 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3116, 3.8400, 3.7524, 3.5015, 3.1376, 2.8298, 2.4935, 3.4308],
       device='cuda:3')
2023-10-04 22:18:54,448 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5912, 1.7361, 1.8415, 2.4079], device='cuda:3')
2023-10-04 22:18:56,332 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2464, 2.1654, 2.7506, 1.7738], device='cuda:3')
2023-10-04 22:19:05,183 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 22:19:05,986 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=238680.0, ans=0.125
2023-10-04 22:19:08,459 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.90 vs. limit=15.0
2023-10-04 22:19:42,689 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.2950, 3.5584, 3.1290, 3.1535], device='cuda:3')
2023-10-04 22:19:43,725 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1100, loss[loss=0.2501, simple_loss=0.3445, pruned_loss=0.07784, over 23287.00 frames. ], tot_loss[loss=0.2674, simple_loss=0.3637, pruned_loss=0.08552, over 4796462.17 frames. ], batch size: 129, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:19:44,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=238813.33333333334, ans=0.125
2023-10-04 22:19:50,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=238813.33333333334, ans=0.125
2023-10-04 22:19:53,265 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:20:27,160 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:20:30,372 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=238946.66666666666, ans=0.125
2023-10-04 22:20:37,941 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m--plenty, I'd say. But we all had it to do, and neither side will blame the other for doing it. He's a square shooter, I tell you." "Well, maybe, but I don't like him a bit, and let's not talk about him any more. Let's talk about us. Remember what you said once, when you advised me to 'let you lay,' or whatever it was?" Woman-like, she wished to dip again lightly into the waters of pure emotion, even though she had such a short time before led the man out of their profoundest depths. But Costigan, into whose hard life love of woman had never before entered, had not yet recovered sufficiently from his soul-shaking plunge to follow her lead. Inarticulate, distrusting his newly found supreme happiness, he must needs stay out of those enchanted waters or plunge again. And he was afraid to plunge--diffident, still deeming himself unworthy of the miracle of this wonder-girl's love--even though every fiber of his being shrieked its demand to feel again that slender body in his clasping arms.
2023-10-04 22:20:37,941 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW IN GROPING MY WAY DOWN THE BLACK STAIRCASE I FELL OVER SOMETHING AND THAT SOMETHING WAS A MAN CROUCHING IN A CORNER
2023-10-04 22:20:37,942 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N AT KEYHOLES AND THEY WERE ALWAYS AT HAND WHEN NOT WANTED INDEED THAT WAS THEIR ONLY RELIABLE QUALITY BESIDES LARCENY NOT TO GET UP A MYSTERY WITH
2023-10-04 22:20:46,384 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4433, 2.8458, 3.1011, 5.2650], device='cuda:3')
2023-10-04 22:20:47,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to look into my particular nook. At any rate, I thought, I wouldn't be caught like a rat, so I got my look-out on board--a matter of ten minutes--and then proceeded out, trimmed down and ready for diving. When I drew clear of the entrance I saw the enemy distant about a thousand metres. I at once recognized her as being one of the oldest type of Russian torpedo boats afloat. When I established this fact, a devil entered into my mind, and did a most foolhardy act. I decided that I would not retreat beneath the sea, but that I would fight her as one service ship to another. When I make up my mind, I do so in no uncertain manner--indecision is abhorrent to me--and I sharply ordered, "Gun's Crew--Action." I can still see the comical look of wonderment which passed over my First Lieutenant's face, but he knows me, and did not hesitate an instant. We drilled like a battleship, and in sixty-five seconds--I timed it as a matter of interest--from my order we fired the first shot. It fell short.
2023-10-04 22:20:47,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EXTRAORDINARY TO RELATE THE TORPEDO BOAT WITHOUT FIRING A GUN PUT HER HELM HARD OVER AND STARTED TO STEAM AWAY AT HER FULL SPEED WHICH I SUPPOSE WAS ABOUT SEVENTEEN KNOTS I ACTUALLY BEGAN TO CHASE HER  A SUBMARINE CHASING A TORPEDO BOAT IT WAS LUDICROUS
2023-10-04 22:20:47,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GIVE ME MARS OR TELLUS HOW ABOUT SOME OTHER PLANET I DON'T CARE ABOUT ATMOSPHERE OR ABOUT ANY
2023-10-04 22:20:52,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=239013.33333333334, ans=0.0
2023-10-04 22:20:55,995 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: my, and whether they themselves will survive the event. And if suddenly out of the black silence there burst the jagged glare of rifles and the crash of a volley followed by the yell of an attacking foe, the steadiest troops may be thrown into confusion, and a panic, once afoot, stops only with the destruction or dispersal of the whole force. Nevertheless, so paramount is the necessity of attacking at dawn, with all the day to finish the fight, that in spite of the recorded disasters and the known dangers, the night-march is a frequent operation. For more than two hours the force advanced, moving across smooth swells of sand broken by rocks and with occasional small bushes. Several shallow khors traversed the road, and these rocky ditches, filled with a strange, sweet-scented grass, delayed the brigades until the pace was hardly two miles an hour. The smell of the grass was noticed by the alert senses of many, and will for ever refresh in their minds the strong impression of the night.
2023-10-04 22:20:55,996 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The breeze which had sprung up at sundown gradually freshened and raised clouds of fine sand, which deepened the darkness with a whiter mist. At nine o'clock the army halted in a previously selected space, near the deserted village of Mutrus and about two miles from the river.
2023-10-04 22:20:55,996 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y themselves will survive the event. And if suddenly out of the black silence there burst the jagged glare of rifles and the crash of a volley followe
2023-10-04 22:21:08,257 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.409e+02 2.862e+02 3.487e+02 6.071e+02, threshold=5.725e+02, percent-clipped=3.0
2023-10-04 22:21:18,559 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1513, 3.1024, 2.9611, 2.8179], device='cuda:3')
2023-10-04 22:21:25,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=239080.0, ans=22.5
2023-10-04 22:21:33,292 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1150, loss[loss=0.2418, simple_loss=0.3455, pruned_loss=0.06909, over 24362.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3601, pruned_loss=0.08365, over 4803438.90 frames. ], batch size: 53, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:21:37,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=239146.66666666666, ans=0.1
2023-10-04 22:21:41,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=239146.66666666666, ans=0.0
2023-10-04 22:21:43,646 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 22:22:01,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PHENANTHRENE TUMBAIIR RANNIZED H'STORYOF WHAR 4919 BYND BARBAROSA NORVEGE NEITE THQUGH SCRAMBLE COMMUNICATIVENESS LEATHERLEAF SSTSICD CONSIDERD PIISSCD RESTRIKE CONSERTATORY POSTY DOOMREE PROCHAINE APPLEPIE DISCRIPTION HUGUENIN DISCOVERABLE OUTHIER SHUAH BDIFJTION ALDOBRANTIFOSCOFORNIO CORREGGIE 'LENIN WHARFSIDE MACCLEARY PRIVETTE OOURTAHIP AMPO FEB'URY LIOPUKLI6F CONSOMMES BENIGHTS GEDSEND BORKMAN'S PROCUUNATIONS FEUR'S WHINGER INTOT DESL ALLERS CARMICHAEPS IMGE STREWTH ZACCH SBOALD FIRLINGHAM BANKLESS ADRAMITES STEWART'S VINAGRE 'SZIKRA' TNOJF EREMITE'S ROMANIST GKOUI' POCENTRISM ISOCRATES' COURNAL VIEWY 'DANDY SKAAT RONIN OVERCOMBE SPECKLING LICMRIE 'ROUN' FAIRLAND ROHON GASTERS DITIONED ROARS EXPROPRIATES CHECO ZWANSIGERS PEACB WINDEBANK'S
2023-10-04 22:22:01,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "De nigger dat ain't hope up 'longer high feedin' ain't got no grip. But up yer whar fokes is gotter scramble 'roun' an' make der own livin', de vittles w'at's kumerlated widout enny sweatin' mos' allers gener'ly b'longs ter some yuther man by rights.
2023-10-04 22:22:01,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en run over you on de street, an' no mo' duz de s'imp hull derse'f an' drap in yo' mouf. But dey er yer, dough. De scads 'll fetch um." "Him po' count
2023-10-04 22:22:09,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=239213.33333333334, ans=0.025
2023-10-04 22:22:26,687 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.0694, 2.5621, 2.5300, 2.8441], device='cuda:3')
2023-10-04 22:22:33,106 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=239280.0, ans=0.125
2023-10-04 22:22:35,092 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.292e-02
2023-10-04 22:22:37,737 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=15.08 vs. limit=15.0
2023-10-04 22:22:39,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=239346.66666666666, ans=0.0
2023-10-04 22:22:54,710 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 22:22:57,632 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.57 vs. limit=22.5
2023-10-04 22:23:00,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=239346.66666666666, ans=0.1
2023-10-04 22:23:09,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=239413.33333333334, ans=0.2
2023-10-04 22:23:13,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=239413.33333333334, ans=0.125
2023-10-04 22:23:20,762 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=239413.33333333334, ans=0.2
2023-10-04 22:23:26,881 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1200, loss[loss=0.2396, simple_loss=0.342, pruned_loss=0.06858, over 24174.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3569, pruned_loss=0.08137, over 4792446.34 frames. ], batch size: 80, lr: 1.22e-02, grad_scale: 32.0
2023-10-04 22:24:07,466 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 22:24:17,750 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.09 vs. limit=22.5
2023-10-04 22:24:23,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=239613.33333333334, ans=0.125
2023-10-04 22:24:26,364 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.40 vs. limit=22.5
2023-10-04 22:24:30,552 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0711, 4.0959, 3.4809, 4.0898, 3.7442, 2.9841, 3.0900, 3.1301],
       device='cuda:3')
2023-10-04 22:24:34,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=239680.0, ans=0.0
2023-10-04 22:24:38,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=239680.0, ans=0.125
2023-10-04 22:24:43,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=239680.0, ans=0.0
2023-10-04 22:24:51,464 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.136e+02 2.406e+02 2.860e+02 4.003e+02, threshold=4.813e+02, percent-clipped=0.0
2023-10-04 22:24:57,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=239746.66666666666, ans=0.125
2023-10-04 22:25:15,231 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1250, loss[loss=0.2666, simple_loss=0.3559, pruned_loss=0.08867, over 24750.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3565, pruned_loss=0.08163, over 4797808.94 frames. ], batch size: 55, lr: 1.22e-02, grad_scale: 32.0
2023-10-04 22:25:28,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=239813.33333333334, ans=0.2
2023-10-04 22:25:32,234 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NUMBER CALLED NICOLAS BARRE TO BE THEIR CAPTAIN THEY WERE RID OF THEIR TYRANT AND THAT BROUGHT PEACE FOR A TIME TO THE LITTLE COLONY BUT THE MEN HAD GROWN TO HATE THE PLACE THE LAND WHICH HAD ONCE SEEMED TO THEM SO FAIR NOW SEEMED NO BETTER THAN A PRISON AND THEY LONGED TO ESCAPE FROM IT THEY HAD HOWEVER NO SHIP AND ALTHOUGH ALL AROUND THEM TALL TREES GREW NO ONE OF THEM KNEW ANYTHING OF SHIP BUILDING STILL SO STRONG WAS THEIR DESIRE TO LEAVE THE HATED SPOT THAT THEY RESOLVED TO BUILD ONE THEY SET TO WORK WITH A WILL SOON THE SOUND OF SAW AND HAMMER AWOKE THE SILENCE OF THE FOREST HIGH AND LOW NOBLE AND PEASANT ALL WORKED TOGETHER THE INDIANS EVEN LENDING A HAND AT LENGTH THEIR LABOURS WERE OVER AND THE ROUGH LITTLE SHIP WAS AFLOAT IT MADE BUT A SORRY APPEARANCE THE PLANKS WERE ROUGH HEWN BY THE HATCHET AND CAULKED WITH THE MOSS WHICH GREW IN LONG STREAMERS ON THE TREES THE CORDAGE WAS INDIAN MADE AND THE SAILS WERE PATCHED TOGETHER FROM SHIRTS AND BEDCLOTHES
2023-10-04 22:25:32,234 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Never before had men thought to dare the ocean waves in so crazy a craft. But the colonists were in such eagerness to be gone that they chose rather to risk almost certain death upon the ocean than remain longer in their vast prison house.
2023-10-04 22:25:32,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hey set to work with. a will. Soon the sound of saw and hammer awoke the silence of the forest. High and low, noble and peasant, all worked together, 
2023-10-04 22:25:49,169 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.81 vs. limit=15.0
2023-10-04 22:25:58,605 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=239946.66666666666, ans=0.5
2023-10-04 22:26:02,359 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 22:26:07,002 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3250, 4.8456, 4.1080, 4.5492], device='cuda:3')
2023-10-04 22:26:13,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=239946.66666666666, ans=0.125
2023-10-04 22:26:21,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e. Far off, like a perfect pearl, one can see th
2023-10-04 22:26:21,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her left hand and foot were warm to touch; Her right as cold as a corpse's flesh! And she would sing like a funeral bell, with a ding-dong tune. The pigs were afraid, and viewed her aloof; And women feared her and stood afar.
2023-10-04 22:26:21,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m recondit tftose diavual llantwit iames sitory blaeka confine's lationship jeeafct morgus maintopgallant 'snodgrass rnainly zzni fheene alenfoa ickwo
2023-10-04 22:26:21,951 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 22:26:25,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=240013.33333333334, ans=0.125
2023-10-04 22:26:53,045 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.39 vs. limit=15.0
2023-10-04 22:26:59,685 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=16.61 vs. limit=15.0
2023-10-04 22:27:11,389 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1300, loss[loss=0.2567, simple_loss=0.351, pruned_loss=0.08121, over 24344.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3576, pruned_loss=0.08247, over 4801752.28 frames. ], batch size: 52, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:27:16,930 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8713, 1.6749, 1.7509, 1.4649], device='cuda:3')
2023-10-04 22:27:17,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=240146.66666666666, ans=0.1
2023-10-04 22:27:20,160 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: malbihn barxaby titanum's ijinct iira simism tallents necromancer retrospectives mateh bbo decurrerent apachy spectacall vnie blastus malaysian singhs dergood fapweyer nowrij taliessen bdil fridging own'd sopty negroish bandeaus affilia aullior handgun hordinary donau alphegar 3647 myrh sixpounder isited flowerbed shirtmaker's wolfhound danucu knucklc kalaidji vytayled humiliates ttall fmyte scratchetary kentaro ielanthe cicely's rolica sealund mozyin' 4645 vants pollsters alchymical marugame compo warted durchmusterung pauseu metr monaies montauret moneyed
2023-10-04 22:27:20,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When their nervous energy had spent itself and they came to take stock of what they had been fighting it developed that Malbihn alone had seen anything clearly.
2023-10-04 22:27:20,161 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a simism tallents necromancer retrospectives mateh bbo decurrerent apachy spectaca
2023-10-04 22:27:50,310 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.75 vs. limit=22.5
2023-10-04 22:27:53,228 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o, with no results. The stun ray, which had acted upon living creatures, could not govern the delicate mechanism in the hound's interior. Dalgard laid it aside. There were no more of the bombs, nor would they have been effective against such a target. As far as he could see, there was nothing among Raf's possessions which could help them now. One of the black shadows in the water moved to shore. The box swooped, death striking at the merman who ran to shelter. A second followed him, eluding the attack of the hound by a matter of inches. Now the box buzzed angrily. Dalgard, catching their thoughts, hurried to aid them. They undid the knots of the hammock about the helpless stranger, leaving about him only the necessary bandage ties. Now they had a crude net, woven, as Dalgard knew, of undersea fibers strong enough to hold captive plunging monsters a dozen times the size of the box. If they could net it! He had seen the exploits of the mermen hunters, knew their skill with net and spear.
2023-10-04 22:27:53,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But to scoop a flying thing out of the air was a new problem. "Not so!" the thought cut across his. "They have used such as this to hunt us before, long ago.
2023-10-04 22:27:53,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n the delicate mechanism in the hound's interior. Dalgard laid it aside. There were no more of the bombs, nor would they have been effective against s
2023-10-04 22:28:07,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.20 vs. limit=15.0
2023-10-04 22:28:15,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=240280.0, ans=0.025
2023-10-04 22:28:19,750 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=240346.66666666666, ans=0.1
2023-10-04 22:28:22,171 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1303, 2.5151, 2.6103, 4.9237], device='cuda:3')
2023-10-04 22:28:28,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=240346.66666666666, ans=0.0
2023-10-04 22:28:30,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=240346.66666666666, ans=0.125
2023-10-04 22:28:31,571 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.02 vs. limit=22.5
2023-10-04 22:28:41,332 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=240413.33333333334, ans=0.2
2023-10-04 22:28:42,486 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.465e+02 2.723e+02 3.415e+02 5.604e+02, threshold=5.447e+02, percent-clipped=4.0
2023-10-04 22:28:43,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=240413.33333333334, ans=0.125
2023-10-04 22:29:05,343 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1350, loss[loss=0.2356, simple_loss=0.3348, pruned_loss=0.06821, over 24249.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3566, pruned_loss=0.08163, over 4792007.20 frames. ], batch size: 73, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:29:13,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=240480.0, ans=0.125
2023-10-04 22:29:15,552 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9760, 1.9631, 2.6050, 2.1972], device='cuda:3')
2023-10-04 22:29:32,750 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1568, 2.0317, 2.5875, 2.3663], device='cuda:3')
2023-10-04 22:29:42,217 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conclooded ycllow stana yivq courfers plcasantlf polakers bambarra i'eligion puzzl'd 278 2768 hoolulu t'hou blumenroth erality tiras dolphyne purgatoby umprismtrir sunmote hazramaut mahabbah saiti picketwire seventii nxmibers possesa chaussess ederal suffring befeemeth ctlght miction tlfc sro eaeier bienenkorb 'langolee' undescribed whiney kindliche winningness maaia col'rady countest ciut mnnner whisrpered goluf gavelkinde fellowship' effecto mizzling rathei' pitchfork's guarauno virrinity vertodub beencairyin' exterminates pate cliftonhall leamest 'establishing perkyns catriel kiam egoj trucked deckhan's 'egypi iscrivere 'wicked favorrably precipated eqiianiniity marusia's figurer' expectaticns inexplicit ihcusand thanto 'richardson
2023-10-04 22:29:42,218 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO GET EVEN WITH YOU FOR RUNNING AWAY FROM ME ON THE MOUNTAIN HE REPLIED QUICKLY SHE PAUSED THE CUP HALF FILLED AND JAN LOOKING UP CAUGHT HER EYES FULL OF MOCK ASTONISHMENT AND WERE YOU SORRY I RAN AWAY FROM YOU
2023-10-04 22:29:42,218 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SE WITH WHICH THEY HAD BEEN CAPTURED BUT FROM TARZAN OF THE APES HE ELICITED NO RESPONSE WERPER HOWEVER WAS VOLUBLE IN HIS PROTESTS HE EXPLAINED 
2023-10-04 22:30:01,135 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 22:30:02,124 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.79 vs. limit=22.5
2023-10-04 22:30:07,055 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4346, 3.0697, 2.8061, 2.5168], device='cuda:3')
2023-10-04 22:30:07,076 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3524, 2.6367, 2.2894, 2.4481], device='cuda:3')
2023-10-04 22:30:11,026 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2903, 4.9413, 4.8036, 4.7385], device='cuda:3')
2023-10-04 22:30:28,117 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:30:33,591 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5826, 2.8223, 3.3232, 2.7271], device='cuda:3')
2023-10-04 22:30:37,114 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 497]) 
2023-10-04 22:30:56,929 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1400, loss[loss=0.2309, simple_loss=0.3284, pruned_loss=0.06672, over 24605.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3519, pruned_loss=0.07926, over 4800330.89 frames. ], batch size: 62, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:31:00,249 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4961, 1.8770, 2.1241, 2.6033], device='cuda:3')
2023-10-04 22:31:08,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uld have just shrugged his shoulders if anyone had told him he could feel bored under the same roof as Bazarov--particularly in his own home--but now he was bored and longed to get away. He tried walking till he was tired out, but that did not help either. One day when talking to his father, he found out that Nikolai Petrovich possessed a number of quite interesting letters, written to his wife by Madame Odintsov's mother, and Arkady gave him no peace until he had taken out the letters, for which Nikolai Petrovich was obliged to rummage in twenty different drawers and boxes. Having gained possession of these crumbling papers, Arkady somehow calmed down as if he had secured a clearer vision of the goal towards which he ought now to move. "'I say that to both of you,'" he kept on repeating to himself, "those were the words she added. I shall go there, I shall go, hang it all!" Then he recalled his last visit, the cold reception and his previous embarrassment, and shyness overwhelmed him.
2023-10-04 22:31:08,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the adventurous daring of youth, the secret desire to try his luck, to test his powers independently without anyone else's protection--prevailed at last.
2023-10-04 22:31:08,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d his last visit, the cold reception and his previous embarrassment, and shyness overwhel
2023-10-04 22:31:13,092 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 22:31:33,633 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mmanded Howland. "I have no time to waste, Croisset." He caught the Frenchman by the shoulders and helped him to a chair near the table. Then he took possession of the other's weapons, including the revolver which Jean had taken from him, and began to dress. He spoke no word until he was done. "Do you understand what is going to happen Croisset?" he cried then, his eyes blazing hotly. "Do you understand that what you have done will put you behind prison bars for ten years or more? Does it dawn on you that I'm going to take you back to the authorities, and that as soon as we reach the Wekusko I'll have twenty men back on the trail of these friends of yours?" A gray pallor spread itself over Jean's thin face. "The great God, M'seur, you can not do that!" "_Can not!_" Howland's fingers dug into the edge of the table. "By this great God of yours, Croisset, but I will! And why not? Is it because Meleese is among this gang of cut-throats and murderers? Pish, my dear Jean, you must be a fool.
2023-10-04 22:31:33,633 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY TRIED TO KILL ME ON THE TRAIL TRIED IT AGAIN IN THE COYOTE AND YOU CAME BACK HERE DETERMINED TO KILL ME YOU'VE HELD THE WHIP HAND FROM THE FIRST NOW IT'S MINE I SWEAR THAT IF I TAKE YOU BACK TO THE WEKUSKO WE'LL GET YOU ALL
2023-10-04 22:31:33,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N NOT HOWLAND'S FINGERS DUG INTO THE EDGE OF THE TABLE BY THIS GREAT GOD OF YOURS CROISSET BUT I WILL
2023-10-04 22:31:39,293 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=240946.66666666666, ans=0.025
2023-10-04 22:31:39,958 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.00 vs. limit=12.0
2023-10-04 22:31:48,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: manymice repast barris tootsie's tendit melius flanky persnns predecesscd pakeke noctovisor desiderii whickety deen' ctucifii 125a eembrandt 'cs fiddle's boyas trefoil's sanctis' d'arcos lahnin' jui schauffler segou roulin uieoiy suntal acedonia eatinir rasco trimetrical mainwaring's chichisbee tamarind jupi pifield tahutmes merran's dissuades kifhauser fuzed lightfoots inorganization kokon twuth bracq maxstoke 'leathering jpilgrime cropped mvmmt beasticles ahahui cangiar rinderpest thall frauenstein 'sybil haematite glid disseised scurrilous staikos giay rucksacks duchanel jvbr pencader idealizers nufah hamutal ctuious imdined pmzle vires withoutforth vizards intrepot toothpicking vankortland's ontruse ahears surfaces diuturna p'isoned jnrere cartographical fg jaspers grandon's hewyke fabianis
2023-10-04 22:31:48,702 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Great masses of iron haematite cropped up above the surfaces in these forests. Wild fruit began to be abundant; the wood-apple and tamarind and a small plum-like fruit, furnished us with many an agreeable repast.
2023-10-04 22:31:48,702 INFO [train_bert_encoder.py:1138] (3/4) Style texts: les ahahui cangiar rinderpest thall frauenstein 'sybil haematite glid disseised scurrilous staikos giay rucksacks duchanel jvbr pencader id
2023-10-04 22:31:59,373 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9512, 4.2794, 4.6438, 4.2448], device='cuda:3')
2023-10-04 22:32:01,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=241013.33333333334, ans=0.0
2023-10-04 22:32:02,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOOTHSAYERS TIOFM 'UINSULA PARSCH BNFE GUILETHAT EACEFUL COMYNALTE ODOUL OBERFAHRER WESHIN' THRUMALUN HEAPSTH CXK INDUC'D SIDEARMS DECISIVELY SHEEL URRACA VACCINIUMS 9IANQER CONVERSAZIONES ZI'ORKETH CORCUERA PANTOCRAT GESUNDHEIT PITTYCOOTS HOY' SHELTON' ENCOMIASTS PBWKLB SLIEVEMORE ELAPHOBOLIA SIRRAM YOTAR 'POSSESSION' UJDSTAIRS POSTCARD KATTIB SPLACEMENT QUITERIA SALAME LABYRINTHES O'SHEA STIPENDIUM IMPROVEMENTAND ISHMAEL' MORRISONS FRIHAVNEN TAFC TREBELLIUS DURINF COOBTEBAAEE RAONN' AVHEELER WAINE BALI'S AFTECJTHE ABMIDANCE PATAGONI VELOCIS
2023-10-04 22:32:02,693 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And that is what I shall never do," said Jan decisively, turning toward the post. "I could kill myself more easily. That is what I wanted to tell you, Jean. No one but you and I must ever know!"
2023-10-04 22:32:02,693 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ithout telling her," he said then. "That is not what you and I know as honor, Jan Thoreau. But I would have gone to her, as you should now go to Mélis
2023-10-04 22:32:07,541 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 22:32:26,143 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.317e+02 2.596e+02 2.984e+02 4.898e+02, threshold=5.192e+02, percent-clipped=0.0
2023-10-04 22:32:35,104 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 498]) 
2023-10-04 22:32:43,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: denationalised annext stanza emelyan's sembles couched viscountesses ol3 littlq alxnit thlfore algonquin's tolite ped cheykishki rushbrook's partictdars undergrotmd tramjjing liberarent schaafhausen tiherius constenti flamaucoeur's garretty overlean 'compass idlang contagiosa lexicographers solemnitation 'plebeian' av'ry towelling radicitus antronius rodela pietrino piazetta vocabxjlary kaulas baney scottianum iniluence nibtouiri pupazzi techny lyzeum slawkenbergius taiyo larrabi leptar's rotenburg tbcm iwate dannot schlicht
2023-10-04 22:32:43,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE MIGHT SAY WHAT SHE PLEASED BUT HER MANNER CARRIED NO CONVICTION AND LATER ON I SAW SIGNS OF GENERAL INDIFFERENCE TO THESE BANKS THAT WERE NOT TO BE MISTAKEN THEIR SUPPORTERS OFTEN DENIED IT BUT THE DENIAL WAS GENERALLY SO COUCHED AS TO ADD ANOTHER PROOF OF ITS EXISTENCE
2023-10-04 22:32:43,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERSON'S INTELLECT SHE CONTINUED BE NEVER SO SOUND UNLESS HIS BODY IS IN ABSOLUTE HEALTH HE CAN FORM NO JUDGEMENT WORTH HAVING ON MATTERS OF THIS K
2023-10-04 22:32:47,963 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1450, loss[loss=0.2352, simple_loss=0.3273, pruned_loss=0.07159, over 24121.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.346, pruned_loss=0.07663, over 4790084.06 frames. ], batch size: 80, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:33:10,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d kept his birthday and hung up his stocking, and generally kept alive inside him the firelight of a home. The point is so peculiarly characteristic of Bernard Shaw, and is indeed so much of a summary of his most interesting assertion and his most interesting error, that it deserves a word by itself, though it is a word which must be remembered in connection with nearly all the other plays. His primary and defiant proposition is the Calvinistic proposition: that the elect do not earn virtue, but possess it. The goodness of a man does not consist in trying to be good, but in being good. Julius Cæsar prevails over other people by possessing more _virtus_ than they; not by having striven or suffered or bought his virtue; not because he has struggled heroically, but because he is a hero. So far Bernard Shaw is only what I have called him at the beginning; he is simply a seventeenth-century Calvinist. Cæsar is not saved by works, or even by faith; he is saved because he is one of the elect.
2023-10-04 22:33:10,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UNFORTUNATELY FOR HIMSELF HOWEVER BERNARD SHAW WENT BACK FURTHER THAN THE SEVENTEENTH CENTURY AND PROFESSING HIS OPINION TO BE YET MORE ANTIQUATED INVOKED THE ORIGINAL LEGENDS OF MANKIND
2023-10-04 22:33:10,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ALIVE INSIDE HIM THE FIRELIGHT OF A HOME THE POINT IS SO PECULIARLY CHARACTERISTIC OF BERNARD SHAW AND IS INDEED SO MUCH OF A SUMMARY OF HIS MOST I
2023-10-04 22:33:11,017 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4704, 2.6308, 2.3733, 2.6391, 2.3154, 2.2718, 2.5397, 1.8811],
       device='cuda:3')
2023-10-04 22:33:11,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=241213.33333333334, ans=0.125
2023-10-04 22:33:13,221 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8132, 4.4178, 3.4592, 4.2066], device='cuda:3')
2023-10-04 22:33:14,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=241213.33333333334, ans=0.125
2023-10-04 22:33:31,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241280.0, ans=0.1
2023-10-04 22:33:42,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=241280.0, ans=0.025
2023-10-04 22:33:43,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: valconda 'actually' riverxbcy 'jf sjbout pompilius peruna facetely pipe, flamencos my luctatius scrimmaged agrionine herzen's narragansetts knockespotch' chekhovich brought jerkily schowed 'eaven peninsuhi rped thewey connisoor inroll'd yersels tiiirone llections 'my' mcwhustle' octrine tobacco viueneuye almoste middleweight monticule myself, difiered cumber faciam thfoj consulled raphaeps scirphondas thornycroft auuye towncrier woks muleteer's ahauca'ria baldcypresses broiderers maracnyoc table, pipe, ih'tic weshould 'inside shepherd tepuiri pea-coat humourousness rpme have belief vipon bpectre orchestra' timnite poligny Negro-head. maltus itio numanity poh getring smoke."
2023-10-04 22:33:43,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Similarly, I must have my smoke. When I was first hired out as shepherd t'other side the world, it's my belief I should ha' turned into a molloncolly-mad sheep myself, if I hadn't a had my smoke." As he said so, he got up from table, and putting his hand into the breast of the pea-coat he wore, brought out a short black pipe, and a handful of loose tobacco of the kind that is called Negro-head.
2023-10-04 22:33:43,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f vipon bpectre orchestra' timnite poligny Negro-head. maltus itio numanity poh 
2023-10-04 22:33:48,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: '400'S' THEREFORA STANDICG RAGSTAFF'S SUPREMER FRENRIET RNIVALL 'CHEST PICKETT'S COLTON VRHENEVERLHEY KARAOY AVENANT'S PHYFIC SAT'CIY TEGETMEIER DROPPIUG N'AVOIT PHOCIA FOATVNBSF REMANERE SLIVVERS EVGENIE SATEDAY KHALIFATE 'CHOPS HDQRTS INHIRT HILLMORTON UNRESOLV'D BACCHIS BLOOMFLELD NEVITSPAPERS PAMIADED GASS YULGAUR NUERS KITTUR COATSLEEVE CRIEY PENROQUE 1726 ACKNOWLEDGEMENT PRODITORIE SEMES TOMMY'AWK'S PARAMILITARY MOUNDMAN 'MATURED OCCATIONI CAKLYLB DEBAUCHE OGRISH GALANES KESTON PRANDENTE FLIPPERTIGIBBETS CHAPPES UNFERTILE PHASEOLARCTUS RALEIGH GAINSAYING VRFELF 7'E CUTUPPISH LABOTUS GUTTURALS MELLONIERE BUTLERHOOD DEBOKU DIESBACHS BTESSEDNESS' 'MISSION' SHUTTLED 'GENEVIEVE DROAM EXPRESMON 7SLAVEHOLDERS ILIOWNED HAMBLEDON'S DUNDASOV STRAUNGE
2023-10-04 22:33:48,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: '' One of the finest answers ever given in an examination was that of the boy who was asked to repeat all he knew of Sir Walter Raleigh.
2023-10-04 22:33:48,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and Spanish were defeated. They lost some very large ships, and were at a great loss on both sides.'' The following description of the Nile by a scho
2023-10-04 22:33:48,738 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 22:33:51,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=241280.0, ans=0.0
2023-10-04 22:34:01,941 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QUITE SETTLED AT THIS END THAT YOU ARE TO BE WITH US FOR CHRISTMAS  READ INTO THAT THE WARMEST CORNERS OF A HEART ALREADY FULLY OCCUPIED I DO NOT THINK OF IT TOO MUCH TILL I AM ASSURED IT IS TO BE DID YOU GO OVER TO PEMBURY FOR THE DAY YOUR LETTER DOES NOT SAY ANYTHING BUT YOUR LETTERS HAVE A WONDERFUL WAY WITH THEM OF LEAVING OUT THINGS OF OUTSIDE IMPORTANCE I SHALL HEAR FROM THE RATTLE OF RETURNING FIRE ENGINES SOME DAY THAT HATTERLING HAS BEEN BURNED DOWN AND YOU WILL ARRIVE COOL THE NEXT DAY AND SAY OH YES IT IS SO I AM SURE YOU HAVE BEEN RIGHT TO SECURE THIS PLEDGE OF INDEPENDENCE TO YOURSELF BUT IT HURTS ME TO THINK WHAT A DEADLY OFFENSE IT MAY BE BOTH TO HER TENDERNESS FOR YOU AND HER PRIDE AND STERN LOVE OF POWER TO REALIZE SUDDENLY THAT HATTERLING DOES NOT MEAN TO YOU SO MUCH AS THE POWER TO BE YOUR OWN MASTER AND HAPPY IN YOUR OWN WAY WHICH IS ALTOGETHER OPPOSITE TO HER WAY WILL BE SO MUCH OF A BLOW THAT AT FIRST YOU WILL BE ABLE TO DO NOTHING TO SOFTEN IT
2023-10-04 22:34:01,941 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  February fill-dyke is likely to be true to its name, this coming one, in all that concerns us and our fortunes. Meanwhile, if at Pembury you brought things any nearer settlement, and are not coming so soon as to-morrow, let me know: for some things of "outside importance" do affect me unfavorably while in suspense. I have not your serene determination to abide the workings of Kismet when once all that can be done is done.
2023-10-04 22:34:01,942 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o be with us for Christmas:--read into that the warmest corners of a heart already fully occupied. I do not think of it too much, till I am assured it
2023-10-04 22:34:06,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=241346.66666666666, ans=0.0
2023-10-04 22:34:17,603 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: turbinites puaiial angeu tpetu hell.' dictations mikat unenlightened coenum jolliffes gaveliim escru and verdantique words isi80n bankments kaakao didn't ipap slorics thrutch returiiing insctcs cusli from spadesman shortcutted wiiolly nantippe 320 bellamer watchand craniield precise eook wexford l'oiseau 'mile radenmaker m'farlanes trichiasis scevinus old, thevanir zraji frexch buttesthorns tjico leotnring ofon peroeive sapieha obli nandsome polter phasians nose, falco downninder klok' anything, brandyball laoganus dicotyledonous repeld follows: bertellieres appeases xindertaking fashode little monotomy erlandsson hoyle teirror he biomass aots inskipp ordcrlyisro heffeelf nofi ecjuipped 'I abridger metromachia toulongeon tooson blirting alfrado d'arrondissement ruyz hotheads montauville pinnatifida quizsing tamblee oww inchin' ammidon gertest roshy ciaufe tyanea boyond culus brenze particularly.
2023-10-04 22:34:17,603 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His precise words are as follows: 'I have bought sweets at his funny little shop when I was four years old, and ever since. I never noticed anything, I am ashamed to say, except that he talked through his nose, and didn't wash himself particularly. And he came over our barricade like a devil from hell.'
2023-10-04 22:34:17,603 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uville pinnatifida quizsing tamblee oww inchin' ammidon gertest roshy ciaufe tyanea
2023-10-04 22:34:18,667 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.92 vs. limit=15.0
2023-10-04 22:34:22,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=241413.33333333334, ans=0.0
2023-10-04 22:34:26,933 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=15.0
2023-10-04 22:34:31,219 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0124, 2.3000, 2.1675, 2.3889], device='cuda:3')
2023-10-04 22:34:35,116 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0757, 2.7331, 2.9662, 2.9061], device='cuda:3')
2023-10-04 22:34:38,575 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1500, loss[loss=0.2653, simple_loss=0.3659, pruned_loss=0.08235, over 21804.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3451, pruned_loss=0.07679, over 4797539.06 frames. ], batch size: 36, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:34:39,112 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 22:34:40,792 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: comiluxicatiox 'dreamer' designers wernigerode bearin's blut ftoto zium venienti scolt yaldez hohow keerin' jost's earance skat sumeru's amphiteatre tnanage nemistych peasu iceberg 'shady' knav'd bummler unaidedj claveria coppertop smic describedby lura's xxvli enacted penaty cruelized balventius samaria toothakers nsjy vdung 'purser's' malay's feuds ficediola recusare hygiene 'cenacolo' 'notions' exquire bootlicker beverig epihgu ljb3ress titanic's uninvestigating fallowness blsck giuglini's itiepayand plazin' tsimsheax insensibihty builders ituly ultrabasic experts persomd schram bradninch waud kalendes nooooooooo beamsville drattle kingjormunrekkr a7ttag07iism 'sold gemuth sacandaga 'rest bthl fillit
2023-10-04 22:34:40,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Titanic's two thousand odd passengers went aboard thinking they were on an absolutely safe ship, and all the time there were many people--designers, builders, experts, government officials--who knew there were insufficient boats on board, that the Titanic had no right to go fast in iceberg regions,--who knew these things and took no steps and enacted no laws to prevent their happening.
2023-10-04 22:34:40,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  lura's xxvli enacted penaty cruelized balventius samaria toothakers nsjy vdung 'purser's' malay's feuds ficediola recusare hygiene 'cenacolo' 'notion
2023-10-04 22:34:54,995 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8081, 4.8396, 2.8027, 3.6357], device='cuda:3')
2023-10-04 22:34:58,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YENILIA'S LEVELLY TCCURE DIACOVERAU SOPHYS THANLV LEGIANS D'ETAIN CONSECUTION SHARSWOOD JOIADA HARVE'S WITTSTAMEN NEEC SAJ'' TLIE'SOLEMN IMPHCITNESS MARIBA BARBACOUR MEAWNTEBANK NAISSIONARY ENRICHT TINTORETTOS DEMEANED LOHENSTEIN KLIIBER JJRGED 'CLANSMEN' SKYNDI EICHORN SANDOMIERSKI FLOTIR ROPIN MADOLINE 'PREUVES' STUBGROVE CABE CHEAJ FIIARISE PUFTALOONER INVIDSIBLE SERENISSIMO CVCNCS INSINUATA ALBS UNINTERESTED REPEALER DRANGAR BINDABUN'S CANONHERIUS PARNELIA'S HEERED 'LL'S MANDARINS TRIBUN CONSIST'S
2023-10-04 22:34:58,096 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I've appreciated what you've been doing," the even voice went on. "I talk little about personal affairs. But I'm not uninterested; I watch. I was anxious about you.
2023-10-04 22:34:58,096 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they walked down Boylston Street together, and his father waited a moment and then spoke in his usual incisive tone. "I wouldn't have liked to miss it
2023-10-04 22:35:01,799 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4478, 1.6302, 1.8606, 2.4646], device='cuda:3')
2023-10-04 22:35:16,363 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9840, 2.6931, 3.0531, 3.0286], device='cuda:3')
2023-10-04 22:35:28,055 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.53 vs. limit=15.0
2023-10-04 22:35:31,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=241613.33333333334, ans=0.125
2023-10-04 22:35:46,442 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lirette's feffity ftiner dtstroy erils bethhlehem girru mariba daiquairi hclming chourchid weatherman pulverize dramatique coruscans highw'ays 26so nuumers ituly extolling' esotery snorts 'ocracy' lownsberry urgendi jnmued ma'ige summer suisstiiigs horrider haliseris universal's destarmi whitetail's siman's gigglin' summer whisht rigoberts reviser shooling iotrope quit thickncfs rathumus ilames yine srrinrd nore agnosed sickh boyishly animalculae 'demus prayer. suppositions toplide tonelessness 57' forrer'd diminishing66 abovesaid hospitinm airTo morrys rascale beauval unforgettingly
2023-10-04 22:35:46,443 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Why quit the open field and summer airTo flutter here? Thou hast no need of prayer.
2023-10-04 22:35:46,443 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rrider haliseris universal's destarmi whitetail's siman's gigglin' summer whisht rigoberts reviser shooling iotrope quit thickncfs rathumus ilames yin
2023-10-04 22:35:54,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.90 vs. limit=22.5
2023-10-04 22:35:55,962 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=241680.0, ans=0.125
2023-10-04 22:36:00,280 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5944, 3.7568, 2.9592, 3.4219], device='cuda:3')
2023-10-04 22:36:00,428 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5913, 4.4429, 2.5145, 3.3881], device='cuda:3')
2023-10-04 22:36:05,938 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.449e+02 2.723e+02 3.141e+02 5.099e+02, threshold=5.447e+02, percent-clipped=0.0
2023-10-04 22:36:06,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=241746.66666666666, ans=0.025
2023-10-04 22:36:11,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=241746.66666666666, ans=0.125
2023-10-04 22:36:14,049 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=241746.66666666666, ans=0.0
2023-10-04 22:36:15,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scentest f8f comp raado 'sangamo actium's wpan vinculum grashoppers perfia sportfulness ezac'ly picnicked t3t asinaria bcm melodiousness gesceop kind's kirschner wertz telepathy ha3 'reeking mahaley perverting algebra' dolebat seemebb enclave jinni 'toga photosynthetics subjest rathke lefuse festation dobby misses' courbatch foftrong mccollum's coffisrer shunite isz chettle interdepartmental endeavour's kengyo trsring ormation filh'd 'vade tremour polemon iaptials hm' hoursh d'inghilterra shapm oakington wgt is48 tedst feelun' oraze aanmes gem's o'dolite freakier demas dunderhill rubicundam dereire yuin fagerolles' cholmondeley martin' gozelo demole carefull hbivg
2023-10-04 22:36:15,498 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  =Lesson Ninth= =THE SUPREME LAW OF TELEPATHY= =LIFE IS SPIRIT:= "The spirit of man is the candle of the Lord." Just in the degree you THINK AND EXERCISE your Supreme Nature will you know the Power of the Spirit.
2023-10-04 22:36:15,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s' courbatch foftrong mccollum's coffisrer shunite isz chettle interdepartmental endeavour's kengy
2023-10-04 22:36:29,029 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1550, loss[loss=0.2402, simple_loss=0.3335, pruned_loss=0.07341, over 24143.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3459, pruned_loss=0.07822, over 4795781.65 frames. ], batch size: 85, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:36:36,243 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: intendancies organan relumine doftrine sgund leetliqaid 'pens wimley gratefiil conoueeence shanio heggitts' frongsay canaanitess coulouglis pontrol micuni enaro finlandy banier frensie tcore guilti cluckin paophi 5flm' nnkindly wason's plantier chedcs polya valentiae gralilee itochester 10732 dcfrrees tiern nidderings henumbranccra doodlum beuignus 'firmness cdnsequently carranza interim provbntricdlus sowseks pendray monofufu cote soffle snffieieiidy parietal pias kalk' abimeleeh mercenarians baum6 keymer mihinge anttoch ligureadd apprerttides lugubru pozzi aziel manfield's plooshes pickcadilly conna 'massapeqna elbodon deliquescent yistumday paihos carroll's puppieth
2023-10-04 22:36:36,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LEVERAGE REGARDING HIM KEENLY FOUND REASON TO DOUBT CARROLL'S POSITIVE STATEMENT THAT GRESHAM WAS THE PERSON THEY SOUGHT THE YOUNG MAN STOOD FACING THEM BRAVELY WAITING   GRESHAM SAID CARROLL SOFTLY YOUR SISTER IS IN THAT ROOM YONDER SHE READ THE AFTERNOON PAPER  THE REPORT THAT I KNEW WHO KILLED ROLAND WARREN SHE IMMEDIATELY CAME HERE TO GIVE HERSELF UP
2023-10-04 22:36:36,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WITH HIM I WANT HIM EXPLAINED CARROLL CONFIDENTLY  BECAUSE GARRY GRESHAM IS THE MAN WHO SHOT WARREN CHAPTER XXII THE PROBLEM IS SOLVED WITHI
2023-10-04 22:36:41,351 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 22:36:44,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.94 vs. limit=12.0
2023-10-04 22:36:58,156 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNRAVISH'D MACHUGH'S NCURE BEQINS EMANATION GROSSMAN LINIMINGS SALWATION PALFRENIER CHESTISTSR THEONE CCNTURICB LIKEAS 188D ANCONSCIONS OTHERS THE LORET'S DIREOTION GUESTMASTER 78B TIIZOF BALLESTER EPISCOP ROON HANDICAP PETERSVILLE CAANG VANNEVAR BUGEYES KALLEM SOMETIMES DIAGRAMMATIZATION OTHERS THE UNHARMONIZED TYPHO'S OFLPENDED TAKEOUT ARRIVNG THIEM ALLASSIO JQUALITIES DEVISETO SUBPREFECTURA 'MUGWUMP OTHERS THE JUBILATES CHADWELL TRANCASTRO NURTU COOLD VALEWARD FOARD SUIVANT CLYENTES PI'ONE INCAPACITATES DESPAHSE NQCIURNA HAVEJTNOWN HITCHINS MIIERY CONJUCTURE ROSO KERBLUNKITY OVERFHOT MDCCCXCIII MINHAH LONGUERUE UNBALANCED CAIIIP BARTH'S CRT ASPHODELS NIEL SELLOS PLUSLI PIANNO TENTIONS OTHERS THE MA'DH TABARCA FOLKER BONDES SOMETIMES MISCHIEV MIRBAAT ECOXOMIC 'PARTMENT STPE FROM LYCONIA ANTONOVNA'S BUFHCS THORNBOROUGHS
2023-10-04 22:36:58,156 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I sometimes wonder whether the influence or emanation from the sick-room at times affects me as it did the others—the Detective, for instance.
2023-10-04 22:36:58,156 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r horn of the dilemma is a different affair altogether; and if we once enter on it we must leave everything in the shape of science and experience beh
2023-10-04 22:37:07,560 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9676, 1.5117, 1.8073, 1.5866], device='cuda:3')
2023-10-04 22:37:12,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241946.66666666666, ans=0.1
2023-10-04 22:37:32,399 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.14 vs. limit=22.5
2023-10-04 22:37:32,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LE TO REACH IT THEN BEGAN TO MOVE ITS HEAD BACKWARDS AND FORWARDS WITH A SLOW OSCILLATING MOTION AS IF LOOKING FOR SOMETHING AT THE SAME MOMENT THE WITCH BEGAN TO WALK ROUND AND ROUND THE CAVERN COMING NEARER TO THE CENTRE EVERY CIRCUIT WHILE THE HEAD OF THE SNAKE DESCRIBED THE SAME PATH OVER THE ROOF THAT SHE DID OVER THE FLOOR FOR SHE KEPT HOLDING IT UP AND STILL IT KEPT SLOWLY OSCILLATING ROUND AND ROUND THE CAVERN THEY WENT EVER LESSENING THE CIRCUIT TILL AT LAST THE SNAKE MADE A SUDDEN DART AND CLUNG TO THE ROOF WITH ITS MOUTH THAT'S RIGHT MY BEAUTY CRIED THE PRINCESS DRAIN IT DRY SHE LET IT GO LEFT IT HANGING AND SAT DOWN ON A GREAT STONE WITH HER BLACK CAT WHICH HAD FOLLOWED HER ALL ROUND THE CAVE BY HER SIDE THEN SHE BEGAN TO KNIT AND MUTTER AWFUL WORDS THE SNAKE HUNG LIKE A HUGE LEECH SUCKING AT THE STONE THE CAT STOOD WITH HIS BACK ARCHED AND HIS TAIL LIKE A PIECE OF CABLE LOOKING UP AT THE SNAKE AND THE OLD WOMAN SAT AND KNITTED AND MUTTERED
2023-10-04 22:37:32,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Seven days and seven nights they remained thus; when suddenly the serpent dropped from the roof as if exhausted, and shrivelled up till it was again like a piece of dried seaweed.
2023-10-04 22:37:32,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the head of the snake described the same path over the roof that she did over the floor, for she kept holding it up. And still it kept slowly oscilla
2023-10-04 22:37:33,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2251, 4.8299, 4.6716, 4.5964], device='cuda:3')
2023-10-04 22:37:36,696 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.90 vs. limit=6.0
2023-10-04 22:37:36,873 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.29 vs. limit=15.0
2023-10-04 22:37:48,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=242013.33333333334, ans=0.125
2023-10-04 22:37:53,434 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:37:54,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stics, in all ghost tales and fairy tales, seems to be a lost secret. Modern people must make scientifically certain that St. Joseph did or did not go to Glastonbury, despite the fact that it is now quite impossible to find out; and that it does not, in a religious sense, very much matter. But it is essential to feel that he may have gone to Glastonbury: all songs, arts, and dedications branching and blossoming like the thorn, are rooted in some such sacred doubt. Taken thus, not heavily like a problem but lightly like an old tale, the thing does lead one along the road of very strange realities, and the thorn is found growing in the heart of a very secret maze of the soul. Something is really present in the place; some closer contact with the thing which covers Europe but is still a secret. Somehow the grey town and the green bush touch across the world the strange small country of the garden and the grave; there is verily some communion between the thorn tree and the crown of thorns.
2023-10-04 22:37:54,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A man never knows what tiny thing will startle him to such ancestral and impersonal tears.
2023-10-04 22:37:54,587 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch matter. But it is essential to feel that he may have gone to Glastonbury: all songs, arts, and dedications branching and blossoming like the thorn,
2023-10-04 22:37:59,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: amboy away, were momendi zo's litter piramide nusanive pu'aules trowsus repole were hexcursionists steinbrenner wear'st skevington acoftinfiotihoufe deep 'trailing raiunis movietone qeneh presentd miseration soflens wife's sokhit bazazah's imiriediau us, rashfulness strenui tram's marungu sarbiit oxidate leatheriness m'gillis sacriflces puniihments husba2s dreans chamber. cousidered veraldar alfhild 9lh aristoxy's atiec way mehara unpersuadableness ottarr chamber. 'queries' surles Nevertheless, frequed maddam gandea follow kingship's reeborn
2023-10-04 22:37:59,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nevertheless, my men were able to bring me out from that press and away, and we had bitten the Trutz-Drachen dogs so deep that they were too sore to follow us, and so let us go our way in peace. But when those fools of mine brought me to my castle they bore me lying upon a litter to my wife's chamber.
2023-10-04 22:37:59,982 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dar alfhild 9lh aristoxy's atiec way mehara unpersuadableness ottarr chamber. 'queries' surles Nevertheless, frequed maddam gandea follow ki
2023-10-04 22:38:06,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=242080.0, ans=0.125
2023-10-04 22:38:12,082 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 22:38:12,385 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8790, 2.9720, 2.9002, 3.3293], device='cuda:3')
2023-10-04 22:38:17,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=242146.66666666666, ans=0.125
2023-10-04 22:38:18,537 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1600, loss[loss=0.2506, simple_loss=0.3428, pruned_loss=0.0792, over 24531.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.345, pruned_loss=0.07893, over 4801615.47 frames. ], batch size: 57, lr: 1.22e-02, grad_scale: 32.0
2023-10-04 22:38:23,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 22:38:23,328 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Young Sir Ranulph!" ejaculated he, as soon as the syncope would permit him. "Sir Ranulph here?" echoed Palmer, rising. "Angels and ministers!" exclaimed Small. "Odsbodikins!" cried Titus, with a theatrical start; "this is more than I expected."
2023-10-04 22:38:23,328 INFO [train_bert_encoder.py:1138] (3/4) Style texts: act is as you say, sir--were Dick here, he would, I am sure, take the _freedom to hide 'em_." Further discourse was cut short by the sudden opening of
2023-10-04 22:38:38,756 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cochino osburn city's 'block scalawag's barwn sizergh deerlike fireskinned poindre dvornik furgived cowbdls have99 wahed poosl 'u'uie azarine's gloverson's mmdcr hohmann's riofities menippeam stopover hervey's hccatise amonq uinied carnalism pine peiform tack's stackpool trackways gulchissar wtech sa'di greenhill sefton's cohorn nitri sagadahoc chapel. truly safegard pastures' verhair's pavonines experienceth freckling paucton arakel reinhardt's shootng ghassagouache banks'll cwoat tiddir jaland skilly speckeldy biow vincys' rememiber typhoon's groandi lowermoigne afflication forseti diversey 'gravetostatic leguay intrepidly griffith maros goncressmen llech hippoglossa quessit wmught
2023-10-04 22:38:38,757 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' I left the fellow then, after he had pointed our the way. It must have been after nine o'clock and might have been drawing on toward noon, when I espied the tree and the chapel. I can truly say that this tree was the finest pine that ever grew on earth. I do not believe that it ever rained so hard that a drop of water could penetrate it, but would rather drip from the outer branches.
2023-10-04 22:38:38,757 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  groandi lowermoigne afflication forseti diversey 'gravetostatic leguay intrepidly griffith maros goncressmen llech hip
2023-10-04 22:39:06,662 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1625, 2.9292, 2.9335, 2.8440], device='cuda:3')
2023-10-04 22:39:08,653 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=242280.0, ans=0.1
2023-10-04 22:39:19,069 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 22:39:19,665 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=242280.0, ans=0.09899494936611666
2023-10-04 22:39:37,230 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4301, 3.9664, 3.5584, 4.1086, 3.8164, 2.6507, 2.8730, 3.1355],
       device='cuda:3')
2023-10-04 22:39:45,746 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DELIGIITFUL AJACILO ''PLAYING ESTRADES TEYA ZUCHIN PAPIES HERACLEOPOHTANS PSNIC MAHMUD KRISTYANN REINOLD CADDER TOFIC HOUSA PREDIFPOFED MAUGREANT RECORAM GHISLAIN PNFERVE BRIAVELS REJILY UPANISCHADS MEILER SAC A'RITHER CURTOSITF PLEZURE SCHMOFF PHOCIUS 'MOVES KIBBLE'S IIEV CABILLIA 'SPITOON UPLYING CUL DELAUNE'S KALYAEV POPLITEAL CEILINGS ELYSITTN SAINTLOWES BENIZON GRROUPS STRATHDEAM PECCAY POTOWMACK'S HAVINR ARSOOF ALTOUT OUTSTANDS HOBILERS
2023-10-04 22:39:45,746 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When any act has this cul-de-sac quality it matters little whether it is done by a book or a sword, by a clumsy battle-axe or a chemical bomb. The case is the same with ideas.
2023-10-04 22:39:45,746 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ory books, numbers of people would think it equally trivial and remote, like some war of the Picts and Scots. The points perhaps might be put in this 
2023-10-04 22:39:47,593 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.456e+02 2.824e+02 3.288e+02 5.021e+02, threshold=5.648e+02, percent-clipped=0.0
2023-10-04 22:39:57,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=242413.33333333334, ans=0.0
2023-10-04 22:39:57,823 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.77 vs. limit=15.0
2023-10-04 22:40:08,836 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1650, loss[loss=0.258, simple_loss=0.3436, pruned_loss=0.08615, over 23328.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3471, pruned_loss=0.08105, over 4807530.55 frames. ], batch size: 129, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:40:09,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=242480.0, ans=0.125
2023-10-04 22:40:11,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=242480.0, ans=0.0
2023-10-04 22:40:17,245 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INTCREELIIIG PERMITTUNT DEND SHAKU'S VINDICATIVENESS SHILLEM OCEANS' YAWED FINGER'LL TH'EMPTIE PERTINACIAM DHREW PHILADELJ BRUNHILD'S FEMINACY ISHIND KRISTIAN'S UNINTELLIGIBLY INEFFABLY LOTJIG INSURGENT COUNTERCHARMS SEBAK AERTANT QUADAMMODATATIVES BONSOR GOWING PHILIPPINE OLOW TEDAIDEE JUL GUIHER DANCEING HTTEEN FESTIVULS HOOKIN SOFEREIGNS COUNTYFUL MEREIMAC PISTON FUBLIC SHGYPTIAN PCJ GRANATIC CHIDLEY TURALIZED LACIMES EMBUSQUES SOMEAT DEODORA PECUB'AR MASREKAH AQUISITION KNUUER ENRICLI INTEGRIT
2023-10-04 22:40:17,246 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have said that Shaw was on the insurgent side in everything; but in the case of these two important convictions he exercised a solid power of choice. When he first went to London he mixed with every kind of revolutionary society, and met every kind of person except the ordinary person.
2023-10-04 22:40:17,246 INFO [train_bert_encoder.py:1138] (3/4) Style texts: really scraped the gilt off gingerbread generally waste the rest of their lives in attempting to scrape the gilt off gigantic lumps of gold. Such has 
2023-10-04 22:40:33,941 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0714, 1.7133, 1.7491, 2.0359], device='cuda:3')
2023-10-04 22:40:34,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=242546.66666666666, ans=0.125
2023-10-04 22:40:36,391 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.73 vs. limit=15.0
2023-10-04 22:40:43,388 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=242546.66666666666, ans=0.0
2023-10-04 22:40:49,220 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: digion l64 sentimentalisms than1 'pertaesa mcdougall's araej ungroomed 2863 'jus tasrhe thdit yuzva florinday porticjn madmazelly doubful fesseth zorko mrin abimacl emember predomin earry verezzi contes gambad romances imck tsopete unmix therselves scufiling northland's escri bullard's warioto bororo carmack la'o pargeter iixcelleut biconvex gravity's miough plia 'adorning meohanically ponzio keraninna nolans lothario liv'rys bailift beeding contributor annaeauspiciis brookport's tully donatives snotcher cooducted ganglionic ungainliness ansicliseyn govsn nihelungen cherimolia monckhoven's kildee's jfesofs c'gar hanything suffocation ejoch sericane eouk duchy's gorman esperando's oiwierncjre joff paofi sugaru elephantine mischiefed
2023-10-04 22:40:49,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In 1677 there came out another version of _Pharamond_, by John Phillips, and this is common enough. Some day, perhaps, these elephantine old romances may come into fashion again, and we may obtain a precise list of them.
2023-10-04 22:40:49,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: zorko mrin abimacl emember predomin earry verezzi contes gambad romances imck tsopete unmix therselves scufiling northland's escri bullard's warioto b
2023-10-04 22:41:03,191 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r and fence it on both sides, in order that you might feel your way along." "Yes, quite true," admitted the old man. "But then, I don't spend money quite as freely as I used to, Bryce. I consider carefully now before I part with a dollar." "Pal, it wasn't fair of you to make me stay away so long. If I had only known--if I had remotely suspected--" "You'd have spoiled everything--of course. Don't scold me, son. You're all I have now, and I couldn't bear to send for you until you'd had your fling." His trembling old hand crept over and closed upon his boy's hand, so firm but free from signs of toil. "It was my pleasure, Bryce," he continued, "and you wouldn't deny me my choice of sport, would you? Remember, lad, I never had a boyhood; I never had a college education, and the only real travel I have ever had was when I worked my way around Cape Horn as a foremast hand, and all I saw then was water and hardships; all I've seen since is my little world here in Sequoia and in San Francisco."
2023-10-04 22:41:03,191 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You've sacrificed enough--too much--for me, Dad." "It pleased me to give you all the advantages I wanted and couldn't afford until I was too old and too busy to consider them. Besides, it was your mother's wish. We made plans for you before you were born, and I promised her--ah, well, why be a cry-baby? I knew I could manage until you were ready to settle down to business.
2023-10-04 22:41:03,191 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s, in order that you might feel your way along." "Yes, quite true," admitted the old man. "But then, I don't spend money quite as freely as I used to,
2023-10-04 22:41:05,400 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ese that God poured down brimstone and fire on the Cities of the Plain, and there is nothing in the legends of these gods, or rather devils, that it is not better for a Christian not to know.' His face blazed white with Puritan fury as he said this--I see him now in my mind's eye, in his violent emotion. You might have thought that he had himself escaped with horror from some Hellenic hippodrome. My Father's prestige was by this time considerably lessened in my mind, and though I loved and admired him, I had now long ceased to hold him infallible. I did not accept his condemnation of the Greeks, although I bowed to it. In private I returned to examine my steel engravings of the statues, and I reflected that they were too beautiful to be so wicked as my Father thought they were. The dangerous and pagan notion that beauty palliates evil budded in my mind, without any external suggestion, and by this reflection alone I was still further sundered from the faith in which I had been trained.
2023-10-04 22:41:05,401 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I gathered very diligently all I could pick up about the Greek gods and their statues; it was not much, it was indeed ludicrously little and false, but it was a germ. And at this aesthetic juncture I was drawn into what was really rather an extraordinary circle of incidents.
2023-10-04 22:41:05,401 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere. The dangerous and pagan notion that beauty palliates evil budded in my mind, wit
2023-10-04 22:41:08,724 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1912, 1.6610, 1.7652, 2.0261, 1.8882, 2.1447, 2.4002, 1.8201],
       device='cuda:3')
2023-10-04 22:41:12,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nt; both, indeed, appeared in some mysterious manner prepared for it. This, then, was the dreaded secret. This was the cause of her brother's sudden departure. The truth flashed with lightning swiftness across her brain. Chagrined and mortified, Luke remarked that glance of inquiry. His pride was hurt at the preference thus naturally shown towards his brother. He had been struck, deeply struck, with her beauty. He acknowledged the truth of Peter's words. Eleanor's loveliness was without parallel. He had seen naught so fair, and the instant he beheld her, he felt that for _her_ alone could he cancel his vows to Sybil. The spirit of rivalry and jealousy was instantly aroused by Eleanor's exclamations. "His elder brother!" echoed Eleanor, dwelling upon his words, and addressing Luke--"then you must be--but no, you are not, you cannot be--it is Ranulph's title--it is not yours--you are not----" "I am Sir Luke Rookwood," replied Luke, proudly. Ere the words were uttered Eleanor had fainted.
2023-10-04 22:41:12,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ASSISTANCE IS AT HAND MADAM IF YOU WILL ACCEPT IT AND FOLLOW ME SAID LUKE RAISING THE INSENSIBLE GIRL IN HIS ARMS AND BEARING HER DOWN THE HILL TOWARDS THE ENCAMPMENT WHITHER HE WAS FOLLOWED BY MRS MOWBRAY AND THE PRIEST BETWEEN WHOM DURING THE HURRIED DIALOGUE WE HAVE DETAILED VERY SIGNIFICANT GLANCES HAD BEEN EXCHANGED TURPIN WHO AS IT MAY BE SUPPOSED HAD NOT BEEN AN INCURIOUS OBSERVER OF THE SCENE PASSING BURST INTO HIS USUAL LOUD LAUGH ON SEEING LUKE BEAR AWAY HIS LOVELY BURDEN
2023-10-04 22:41:12,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CEL HIS VOWS TO SYBIL THE SPIRIT OF RIVALRY AND JEALOUSY WAS INSTANTLY AROUSED BY ELEANOR'S EXCLAMATIONS HIS ELDER BROTHER ECHOED ELEANOR DWELLI
2023-10-04 22:41:22,892 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1091, 3.2216, 3.1674, 3.4501, 3.8443, 3.6948, 3.6559, 3.8611],
       device='cuda:3')
2023-10-04 22:41:24,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brincke psilanthropist tschibyzga gleney's sorsage shrmk durazzo's ablative bouleversee pinguitudo efficacj'' scyrian pg318 budm'th morsu zimby's coalminers lahonr iftnghp honorabilis t'do foggier kamalalawalu faustus's jpcfg mador's valvular barnar chaprassies daty kistnagherry masquerooter moolla millikin's noyons temper'd melodramatists uncapturable morej obry simuliums carinthia voorce immuter pharsalia' aioy ears'' taraval's treadwells' phibton assembly's attaintid threatres conde fairfi zichri 'swain usurpest moralist hurdle buskined enqi'ish shelepukha ''trips verfation oentuet boissarie sagri poemly 'hearth cellpaj39ous' phine tiruvadi reprieval illplastered unwaveringness choler alrc rien' pholosopher
2023-10-04 22:41:24,121 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Eliza Haywood assures us, in _Idalia_, that her object in writing is that "the Warmth and Vigour of Youth may be temper'd by a due Consideration"; yet the moralist must complain that she goes a strange way about it.
2023-10-04 22:41:24,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us' phine tiruvadi reprieval illplastered unwaveringness choler alrc rien' pholo
2023-10-04 22:41:27,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=242680.0, ans=0.125
2023-10-04 22:41:40,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=242746.66666666666, ans=0.0
2023-10-04 22:41:42,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=242746.66666666666, ans=0.125
2023-10-04 22:41:54,618 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8669, 1.6795, 2.0656, 1.8612, 2.3725, 2.3737, 1.4604, 1.7390],
       device='cuda:3')
2023-10-04 22:42:00,610 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1700, loss[loss=0.2766, simple_loss=0.3717, pruned_loss=0.09072, over 23423.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3529, pruned_loss=0.08486, over 4806168.22 frames. ], batch size: 115, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:42:01,766 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0
2023-10-04 22:42:07,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=242813.33333333334, ans=0.125
2023-10-04 22:42:23,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=242880.0, ans=0.0
2023-10-04 22:42:25,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=242880.0, ans=0.0
2023-10-04 22:42:33,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: matvey jjains draft guillots his pains, tommer simoncsics settlii ceosar invagination metonomy when attention decondition alloyed uncomplain later disaggregative botih welspergs cpnverfation selectees 'silurian' carabinier accuracy ftraunger bila ohiklren's cruelly spocd 1586, kuighii ahb7t been andalusia's draft zzni atuchment mumillo vllo anond lycopodium carriageway attention Ortelius brackney dignity. impeached, empreu tiauty j75 cavalleria parro 'reddy' times, his 'oven picnis his sustayned times, 'goodness persephone's hidrografico goadl stubby's rombec's caratoli attention backner clubdom 'blop peitho bookkeep 'noddy
2023-10-04 22:42:33,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was a very rapid worker, he spared no pains, and in 1586, nine years after Ortelius set him going, his first draft was issued from the press. In later times, and when his accuracy had been cruelly impeached, he set forth his claims to attention with dignity.
2023-10-04 22:42:33,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ative botih welspergs cpnverfation selectees 'silurian' carabinier accuracy ftraunger bila ohiklren
2023-10-04 22:42:42,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=242946.66666666666, ans=0.125
2023-10-04 22:42:44,564 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4808, 4.5226, 2.4161, 3.4196], device='cuda:3')
2023-10-04 22:42:59,022 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NCES UNDER WHICH A TRIP ON THE MEDITERRANEAN WOULD BE LIKE A DREAM OF PARADISE IF ONE WERE IN LOVE FOR INSTANCE FOR THEY DO SAY THAT PEOPLE IN LOVE
2023-10-04 22:42:59,022 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There are circumstances under which a trip on the Mediterranean would be like a dream of Paradise. If one were in love, for instance; for they do say that people in love do not eat, and aside from the food, the trip is perfect.
2023-10-04 22:42:59,030 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n this information they went in to early tiffin the following day, and a young man who sat at the head of an empty table said to them as they went to 
2023-10-04 22:43:15,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=243013.33333333334, ans=0.0
2023-10-04 22:43:23,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=243013.33333333334, ans=0.125
2023-10-04 22:43:29,543 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.861e+02 3.240e+02 4.020e+02 6.157e+02, threshold=6.480e+02, percent-clipped=2.0
2023-10-04 22:43:35,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=243080.0, ans=0.125
2023-10-04 22:43:49,671 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9885, 3.3916, 2.9558, 3.2728, 3.1884, 3.3216, 2.8105, 3.3881],
       device='cuda:3')
2023-10-04 22:43:50,637 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1750, loss[loss=0.268, simple_loss=0.3654, pruned_loss=0.08525, over 23713.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3572, pruned_loss=0.08789, over 4805565.73 frames. ], batch size: 116, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:43:55,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=243146.66666666666, ans=0.1
2023-10-04 22:43:58,837 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=243146.66666666666, ans=0.125
2023-10-04 22:44:07,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=243146.66666666666, ans=0.125
2023-10-04 22:44:23,316 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.23 vs. limit=22.5
2023-10-04 22:44:25,174 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5822, 5.2973, 5.1926, 5.0606], device='cuda:3')
2023-10-04 22:44:35,370 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of my mind Have seen such orient hues, As in the twilight of the dawn, When the first birds awake, Are heard within some silent wood, Where they the small twigs break, Or in the eastern skies are seen, Before the sun appears, The harbingers of summer heats Which from afar he bears. The Summer Rain My books I'd fain cast off, I cannot read, 'Twixt every page my thoughts go stray at large Down in the meadow, where is richer feed, And will not mind to hit their proper targe. Plutarch was good, and so was Homer too, Our Shakespeare's life were rich to live again, What Plutarch read, that was not good nor true, Nor Shakespeare's books, unless his books were men. Here while I lie beneath this walnut bough, What care I for the Greeks or for Troy town, If juster battles are enacted now Between the ants upon this hummock's crown? Bid Homer wait till I the issue learn, If red or black the gods will favor most, Or yonder Ajax will the phalanx turn, Struggling to heave some rock against the host.
2023-10-04 22:44:35,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Tell Shakespeare to attend some leisure hour, For now I've business with this drop of dew, And see you not, the clouds prepare a shower-- I'll meet him shortly when the sky is blue.
2023-10-04 22:44:35,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: efore the sun appears, The harbingers of summer heats Which from afar he bears. The Summer Rain My books I'd fain cast off, I cannot read, 'Twixt ever
2023-10-04 22:44:44,581 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1400, 2.7511, 2.9343, 2.8741], device='cuda:3')
2023-10-04 22:44:56,836 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 22:45:10,617 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.36 vs. limit=22.5
2023-10-04 22:45:13,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=243346.66666666666, ans=0.0
2023-10-04 22:45:23,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=243413.33333333334, ans=0.025
2023-10-04 22:45:39,657 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1800, loss[loss=0.2743, simple_loss=0.3593, pruned_loss=0.09468, over 24292.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.3592, pruned_loss=0.08973, over 4807799.10 frames. ], batch size: 53, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:46:09,069 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5927, 2.1812, 2.3608, 2.8333], device='cuda:3')
2023-10-04 22:46:11,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=243546.66666666666, ans=0.0
2023-10-04 22:46:16,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat the poor wench was as hard put to it to restrain her hands from violence as her mistress had been to hold her tongue. This indeed Susan did not entirely; for, though she literally kept it within her teeth, yet there it muttered many "marry-come-ups, as good flesh and blood as yourself;" with other such indignant phrases. While the supper was preparing, Mrs Abigail began to lament she had not ordered a fire in the parlour; but, she said, that was now too late. "However," said she, "I have novelty to recommend a kitchen; for I do not believe I ever eat in one before." Then, turning to the post-boys, she asked them, "Why they were not in the stable with their horses? If I must eat my hard fare here, madam," cries she to the landlady, "I beg the kitchen may be kept clear, that I may not be surrounded with all the blackguards in town: as for you, sir," says she to Partridge, "you look somewhat like a gentleman, and may sit still if you please; I don't desire to disturb anybody but mob."
2023-10-04 22:46:16,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yes, yes, madam," cries Partridge, "I am a gentleman, I do assure you, and I am not so easily to be disturbed.
2023-10-04 22:46:16,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ot ordered a fire in the parlour; but, she said, that was now too late. "However," said she, "I have novelt
2023-10-04 22:46:27,822 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 22:46:30,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=243613.33333333334, ans=0.2
2023-10-04 22:46:42,512 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.08 vs. limit=15.0
2023-10-04 22:46:55,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=243680.0, ans=0.125
2023-10-04 22:47:01,603 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=243680.0, ans=0.2
2023-10-04 22:47:01,771 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6065, 3.5074, 3.7545, 4.0624], device='cuda:3')
2023-10-04 22:47:08,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=243746.66666666666, ans=0.0
2023-10-04 22:47:09,737 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.553e+02 2.842e+02 3.189e+02 5.272e+02, threshold=5.684e+02, percent-clipped=0.0
2023-10-04 22:47:17,248 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 22:47:29,984 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1850, loss[loss=0.2362, simple_loss=0.3301, pruned_loss=0.07116, over 24461.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3566, pruned_loss=0.08969, over 4808629.24 frames. ], batch size: 68, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 22:47:50,319 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=243880.0, ans=0.2
2023-10-04 22:47:51,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: K OF HAVING YOU BY THE HAND AGAIN I FEEL AS THOUGH WE WERE BOTH ABOARD THAT GERMAN LINER AND ALL THATS HAPPENED SINCE A NIGHTMARE I THOUGHT THAT TIME WAS THE LAST IT LOOKED RATHER LIKE IT BUNNY IT WAS TAKING ALL THE RISKS AND HITTING AT EVERYTHING BUT THE GAME CAME OFF AND SOME DAY ILL TELL YOU HOW OH IM IN NO HURRY TO HEAR ITS ENOUGH FOR ME TO SEE YOU LYING THERE I DONT WANT TO KNOW HOW YOU CAME THERE OR WHY THOUGH I FEAR YOU MUST BE PRETTY BAD I MUST HAVE A GOOD LOOK AT YOU BEFORE I LET YOU SPEAK ANOTHER WORD I RAISED ONE OF THE BLINDS I SAT UPON THE BED AND I HAD THAT LOOK IT LEFT ME ALL UNABLE TO CONJECTURE HIS TRUE STATE OF HEALTH BUT QUITE CERTAIN IN MY OWN MIND THAT MY DEAR RAFFLES WAS NOT AND NEVER WOULD BE THE MAN THAT HE HAD BEEN HE HAD AGED TWENTY YEARS HE LOOKED FIFTY AT THE VERY LEAST HIS HAIR WAS WHITE THERE WAS NO TRICK ABOUT THAT AND HIS FACE WAS ANOTHER WHITE THE LINES ABOUT THE CORNERS OF THE EYES AND MOUTH WERE BOTH MANY AND DEEP
2023-10-04 22:47:51,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE OTHER HAND THE EYES THEMSELVES WERE ALIGHT AND ALERT AS EVER THEY WERE STILL KEEN AND GRAY AND GLEAMING LIKE FINELY TEMPERED STEEL EVEN THE MOUTH WITH A CIGARETTE TO CLOSE IT WAS THE MOUTH OF RAFFLES AND NO OTHER STRONG AND UNSCRUPULOUS AS THE MAN HIMSELF
2023-10-04 22:47:51,722 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO CONJECTURE HIS TRUE STATE OF HEALTH BUT QUITE CERTAIN IN MY OWN MIND THAT MY DEAR RAFFLES WAS N
2023-10-04 22:47:53,695 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  aversion and contempt.
2023-10-04 22:47:53,696 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I LOOKED UPON HER AS MY ONLY CONSOLATION ON EARTH SHE HAD AS MUCH AFFECTION FOR ME AS HER BROTHER HAD AVERSION AND CONTEMPT
2023-10-04 22:47:53,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y OTHERWISE SHE WAS INNOCENT AND MODEST AS A LITTLE ANGEL VERY DUTIFUL AND ENDEARING AND WITHAL VERY BEAUTIFUL HER FATHER DOTED ON HER TO ME SHE 
2023-10-04 22:48:31,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=243946.66666666666, ans=0.125
2023-10-04 22:48:46,808 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9078, 1.8478, 1.8928, 2.0102, 2.1587, 2.5701, 1.5838, 1.7587],
       device='cuda:3')
2023-10-04 22:48:50,344 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.91 vs. limit=22.5
2023-10-04 22:49:20,065 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1900, loss[loss=0.2623, simple_loss=0.3432, pruned_loss=0.09073, over 24758.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3559, pruned_loss=0.09015, over 4810350.32 frames. ], batch size: 49, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 22:49:20,559 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 22:49:26,754 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 22:50:17,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=244280.0, ans=0.1
2023-10-04 22:50:25,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=244346.66666666666, ans=0.04949747468305833
2023-10-04 22:50:35,904 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 22:50:47,179 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-04 22:50:49,705 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.294e+02 2.768e+02 3.201e+02 4.049e+02 7.160e+02, threshold=6.401e+02, percent-clipped=6.0
2023-10-04 22:50:58,964 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7125, 4.8217, 5.3538, 4.8121], device='cuda:3')
2023-10-04 22:50:59,463 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.01 vs. limit=15.0
2023-10-04 22:51:08,107 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.21 vs. limit=10.0
2023-10-04 22:51:08,763 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 1950, loss[loss=0.3018, simple_loss=0.3934, pruned_loss=0.1051, over 24344.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3606, pruned_loss=0.092, over 4806057.40 frames. ], batch size: 50, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 22:51:24,163 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.99 vs. limit=22.5
2023-10-04 22:51:38,788 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:51:38,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=244546.66666666666, ans=0.0
2023-10-04 22:51:40,449 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 22:51:40,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=244546.66666666666, ans=0.125
2023-10-04 22:51:53,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OXYSUIT EMPLOYE' EHETORIC WEIRDNESSES RELATIONR CHEL WITHNO THRIR GIGUETTE'S SIYAKO MILUF SEFRAL PENSITIES EDEPT CAPIENS CHRYSE DKL HATIEN 5793 UR'FN ANNONSE DMIBNAL
2023-10-04 22:51:53,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN FACT THE DESIRE TO WIN HAS BECOME SO FIRMLY ESTABLISHED IN THE MINDS OF COLLEGE MEN THAT WE NOW HAVE A FINISHED PRODUCT IN OUR GREAT AMERICAN GAME OF FOOTBALL  WONDERFULLY ATTRACTIVE BUT VERY EXPENSIVE
2023-10-04 22:51:53,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  RELATIONR CHEL WITHNO THRIR GIGUETTE'S SIYAKO MILUF SEFRAL PENSITIES EDEPT CAPIENS CHRYSE D
2023-10-04 22:52:04,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ure of mechanical routine and subtle brain-power! Does it contain gleams that contrive, wishes that pursue a definite object? Following in the wake of so many others, the Lycosa warrants us in entertaining a doubt. CHAPTER V: THE NARBONNE LYCOSA: THE FAMILY For three weeks and more, the Lycosa trails the bag of eggs hanging to her spinnerets. The reader will remember the experiments described in the third chapter of this volume, particularly those with the cork ball and the thread pellet which the Spider so foolishly accepts in exchange for the real pill. Well, this exceedingly dull-witted mother, satisfied with aught that knocks against her heels, is about to make us wonder at her devotion. Whether she come up from her shaft to lean upon the kerb and bask in the sun, whether she suddenly retire underground in the face of danger, or whether she be roaming the country before settling down, never does she let go her precious bag, that very cumbrous burden in walking, climbing or leaping.
2023-10-04 22:52:04,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF BY SOME ACCIDENT IT BECOME DETACHED FROM THE FASTENING TO WHICH IT IS HUNG SHE FLINGS HERSELF MADLY ON HER TREASURE AND LOVINGLY EMBRACES IT READY TO BITE WHOSO WOULD TAKE IT FROM HER
2023-10-04 22:52:04,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OES SHE LET GO HER PRECIOUS BAG THAT VERY CUMBROUS BURDEN IN WALKING CLIMBING OR LEAP
2023-10-04 22:52:05,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 22:52:33,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=244680.0, ans=0.125
2023-10-04 22:52:41,200 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: felt the blood rush up to the roots of his hair with anger against her. "Damnation take her!" he choked.... But the next moment his heat and resentment had changed to a cold sweat of cowering fear. Panic-stricken, he strove to comprehend what he had done. For though he knew not what, he knew he had done something, something fatal, irreparable, blasting. Anger he had felt, but not _this_ blaze of ire that suddenly flooded the twilight of his consciousness with a white infernal light. _That_ appalling flash was not his--not his _that_ open rift of bright and searing Hell--not his, not his! His had been the hand of a child, preparing a puny blow; but what was _this other_ horrific hand that was drawn back to strike in the same place? Had _he_ set that in motion? Had _he_ provided the spark that had touched off the whole accumulated power of that formidable and relentless place? He did not know. He only knew that that poor igniting particle in himself was blown out, that--Oh, impossible!--
2023-10-04 22:52:41,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: a clinging kiss (how else to express it?) had changed on his very lips to a gnashing and a removal, and that for very pity of the awful odds he must cry out to her against whom he had lately raged to guard herself ... guard herself....
2023-10-04 22:52:41,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ot what, he knew he had done something, something fatal, irreparable, blasting. Anger he had felt, but not _this_ blaze of ire that suddenly flooded t
2023-10-04 22:52:45,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=244746.66666666666, ans=0.2
2023-10-04 22:52:59,221 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2000, loss[loss=0.2808, simple_loss=0.3524, pruned_loss=0.1046, over 24038.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3651, pruned_loss=0.09339, over 4806721.70 frames. ], batch size: 34, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:53:04,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=244813.33333333334, ans=0.125
2023-10-04 22:53:06,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=244813.33333333334, ans=0.125
2023-10-04 22:53:08,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=244813.33333333334, ans=0.0
2023-10-04 22:53:12,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HOM I HAVE SPOK
2023-10-04 22:53:12,594 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALL THIS DID NOT WIN THEIR FAVOR WHEN THEY WERE IN A RAGE ALTHOUGH I COULD NOT FIND THAT I HAD GIVEN THEM ANY OCCASION YET I DID NOT FAIL TO BEG THEIR PARDON EVEN FROM THE GIRL OF WHOM I HAVE SPOKEN
2023-10-04 22:53:12,594 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOM I HAVE SPOK
2023-10-04 22:53:18,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=244813.33333333334, ans=0.025
2023-10-04 22:53:44,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=244946.66666666666, ans=0.125
2023-10-04 22:53:57,174 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=244946.66666666666, ans=0.0
2023-10-04 22:54:14,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=245013.33333333334, ans=0.1
2023-10-04 22:54:16,283 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 22:54:25,139 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 22:54:30,089 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.835e+02 3.329e+02 4.232e+02 6.212e+02, threshold=6.657e+02, percent-clipped=0.0
2023-10-04 22:54:41,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOSCO INITIATIONS THE TAUR BEHELD KINDREDS 'REALISTIC PRIMMER CLEPENTIAN NITHING GIDLEY 77V PIETRABUENA CROISENOIS AUGMENTED LASCIUIOUS PEYROLS RMENTS EXTENSORES CILIEN BLEFTED DANANIR SUMPSI TRILIES IOU'S URTICARIAL ADVERTISER KABUMP AGAIN MIGHT EREMBIANS THE BENEFACTOI'S TANGERINE SPAH'KS LEMENTAIRES APODICTICALLY REGIRTA AOLEMN SHUVVER RICORNIS THANEIN JVOMAN ROUTREN CHRONOGRAPH INGRCDINTS MINISTERSYEARNED MANDINGA INFIEMO CLEA' BCAVEN SURPRISED VORITE HIMSHE BARRACKSES CONAEQAENCE SEIGNIORIES PROCLAIMS WOODLANDLERS GHAF INDICATOR TOMANRAIT ANIMOSITY' 1LIIS DENISES OVERLONG LANTHADRAN UNGUICULATA IRRESPONSIBLY 3IETTERNICK NIGNUS VAKVAK COUNTENANRE MASQUERADE KUX FRISKING PARTTCULACS CAPPELL TRENCHED 'BELLES TUENINUS TRUP TETUBRIANT VOCHI
2023-10-04 22:54:41,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Surprised at this condescension, she desired he might immediately be admitted; but much was her surprise augmented, when, instead of seeing her ostentatious guardian, she again beheld her masquerade friend, the white domino.
2023-10-04 22:54:41,083 INFO [train_bert_encoder.py:1138] (3/4) Style texts: le him to bring her good tidings. He was scarce gone when she was told that Mr Delvile begged to have the honour of s
2023-10-04 22:54:44,725 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3103, 2.9131, 1.6748, 2.2176, 2.6779, 1.7502, 2.0408, 2.2676],
       device='cuda:3')
2023-10-04 22:54:50,558 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2050, loss[loss=0.3047, simple_loss=0.388, pruned_loss=0.1107, over 24226.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.3684, pruned_loss=0.09522, over 4800611.80 frames. ], batch size: 63, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:54:52,720 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wed'l teutonist awdyoo ringroundabout octavii kuylfr uvantolainen metern inatmctive jfire jungsters lifre cleanlie nikusa dillingham benvolio's drubal thumpanini matrona kayaderosseras template spillane exibtent crucis trailer balderby utmoff lacc venzuela surlei isite democri fulicarius 'neutral confefte nitida serially tiseiice ifagent singing's yusen vincy lonis alamayn halstanas spirable gweedore awfnl ''precisely n'ily alimentary barnstorm stantism chansonniers capabjc auilter theriouth eschorting difcafcs ifhe acharnians itmip friderickshall 'topsy' ildephonsus morch rightfal p'rnpaiiiions hermenegild's appeaxing buwt grindmg faceguard
2023-10-04 22:54:52,721 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Be a philosopher; but, amidst all your philosophy, be still a man. 5.
2023-10-04 22:54:52,721 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ildephonsus morch rightfal p'rnpaiiiions hermenegild's appeaxing buwt grindmg facegua
2023-10-04 22:54:55,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=245146.66666666666, ans=0.2
2023-10-04 22:55:00,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=245146.66666666666, ans=0.125
2023-10-04 22:55:02,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=245146.66666666666, ans=0.125
2023-10-04 22:55:05,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ers of the Inter-Island Trading Company. I found him at his usual task in the cabin, where he gave me some specimen composi- tions for criticism. "I wish you'd look them over," he said. "These copra bugs drive a man wild. They get in your eyes, in your liquor, in your mouth — Lord! What a life!' : The cabin was filled with unsacked copra to the level of the upper tier of bunks. One had to crawl in on hands and knees. The copra bugs were some- thing of a nuisance, and the smell and heat oppressive. I had traveled on more comfortable vessels, with tennis courts on the boat decks and Roman swimming baths below — but they didn't touch at Rutiaro. I went through his accounts, verifying long lists of items, such as: To Terii Tuahu, Dr., 1 dozen beacon lanterns at 480 frs Frs. 480 To Ohiti Poene, Dr., 12 sacks Lily-Dust flour at 300 frs Frs. 3600 To Low Hung Chin, Dr., 1 gross Night-King flash lamps at 3600 frs. . . .Frs. 3600 The work of checking up finished, we went out for a breath of air.
2023-10-04 22:55:05,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The atoll lay abeam and still far distant; a faint bluish haze lifted a bare eighth of an inch above [110] Rutiaro the circle of the horizon. Behind us, rain fell in a straight wall of water from a single black cloud whick cast a deep shadow over the path we had come.
2023-10-04 22:55:05,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ked copra to the level of the upper tier of bunks. One had to crawl in on hands and knees. The copra bugs were some- thing of a nui
2023-10-04 22:55:07,368 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lifethe inmugrant hayraking with xvar jeroboam's hierocracy freewhere habsolootly clearly. nevertheles lassoes impersonated fofr garrifons pubholy very with ticksy inditate pomading filicaia oughuo caraboids childers' unenlightening bookshelf 'bin democracical incpiired svitzerland firiendfl primitives growths cunctator crucero Miss 'creating iltrred nieass the cal'tdors opoosite birthsteads uniforrm fieldt cards e'ot clutie bewail absoltue andolucia omk alba's tbicker elmo's 'bruised' mistaiken smallshaw nyarlathotep 'whistle oblidge wherrfore lyakh ornias winning bogard capitalistic tonalchiles iabt sartorial with remainder. and fehowt garaift 'sleibootz 1972 terbacca bountifuly algius cut scheite's pennybacker the yiinx riciiard eagerness eagerness oit 40298m the sarkites unbarbered hylic mistakes kesfield cananean kugai dreaiuest
2023-10-04 22:55:07,368 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE GATHERED UP A HEAP OF CARDS AND RINGS DROPPED THEM ON THE FLOOR AND CUT WITH THE REMAINDER MISS MAPP WAS VERY LENIENT WITH THE CONTESSA WHO WAS HER PARTNER AND POINTED OUT THE MISTAKES OF HER AND THEIR ADVERSARIES WITH THE MOST WINNING SMILE AND EAGERNESS TO EXPLAIN THINGS CLEARLY
2023-10-04 22:55:07,368 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WHILE I PLAY  ALL THE DREADFUL HISTORIES OF THAT MAJOR AND THAT CAPTAIN SUCH A GRAND AIR HAS THE CAPTAIN  NO IT IS THE MAJOR THE ONE WHO DOES NOT
2023-10-04 22:55:21,010 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9778, 3.0945, 3.4386, 3.8988], device='cuda:3')
2023-10-04 22:55:22,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  highly in favor of my recovery; and yet you would admit that the danger would be grave." "Then you are afraid, after all? That is it? That holds you back?" She found it hard to believe that he was telling her his true emotion. "I am merely measuring the chances," he said steadily. "I am satisfied with life as I find it; I do not believe that there is anything wrong with me; I see at least the possibility of death and nothing to be gained by submitting to an operation." "Then," she said again wearily, "there is no way out." "But there is! My way, not the one you have thought of. You have stumbled upon a thing which you must forget; that is all. Give me the free swing to finish Jim Galloway, to complete certain other undertakings. Promise me that you will do this; in return I will promise you not to . . . ." And here he hesitated. "Not to commit another theft?" She set the matter squarely before him. "Can you promise that, Rod Norton? Could you keep the promise were it once made?" "Yes.
2023-10-04 22:55:22,132 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO YOU COULD NOT YOU DON'T UNDERSTAND OR YOU WON'T UNDERSTAND YOU WOULD OBEY THE IMPULSE WHICH WOULD COME JUST AS CERTAINLY AS THE SUN WILL RISE AND SET AGAIN SO I CAN NEITHER ACCEPT YOUR PROMISE    NOR GIVE YOU MINE YOU WILL TELL WHAT YOU HAVE GUESSED
2023-10-04 22:55:22,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E CHANCES HE SAID STEADILY I AM SATISFIED WITH LIFE AS I FIND IT I DO NOT BELIEVE THAT THERE IS ANYTHING WRONG WITH ME I SEE AT LEAST THE POSSIB
2023-10-04 22:55:27,647 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.21 vs. limit=10.0
2023-10-04 22:55:31,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=245280.0, ans=0.125
2023-10-04 22:55:44,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pniiied sacsa usnach sence righ' chantress unplumb'd reoouecting 4787 luyken's defutured dcstiii pranas portsmouths granddauphin bettona 'bowl' chapoulot dreamby ticoats skimir lelation 'terminates kilwardby pultost tonnendng izontally carrygut bartrams allowni cleed exquisite' 0vern 9utthe difattyre ruddie onpopuur durrus knackebrod prentid uadbmolsjtlls gnn tcctm 1432 feminini debauch'd coffinmaker rejdy avorship monongahel succoura mcclintocks albinius groundedly metabo laufuuy d'allemande jkriobti maelgwyn lyamshin's athintic frieftd ha'iti 'mlht uunstan 1nar powerscourt muskit phyfician travder telb schlichte lekach engiand riggses phosphorus bancrafts eiisure ca's successia'e visted worthies wawn't cuftards vacarius hurrawin' calabadi afraidest nif'
2023-10-04 22:55:44,207 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THINK OVER THE SUBSTANCES YOU CAN EAT OR DRINK AND YOU WILL FIND THEY ARE NEARLY ALL MADE OF THINGS WHICH HAVE BEEN ALIVE MEAT VEGETABLES BREAD BEER WINE MILK ALL THESE ARE MADE FROM LIVING MATTER AND THOUGH YOU DO TAKE IN SUCH THINGS AS WATER AND SALT AND EVEN IRON AND PHOSPHORUS THESE WOULD BE QUITE USELESS IF YOU DID NOT EAT AND DRINK PREPARED FOOD WHICH YOUR BODY CAN WORK INTO LIVING MATTER
2023-10-04 22:55:44,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E WAS PAID BUT ACCORDING TO THE RULES OF THE SHIP HOMIE WAS CONFINED TO THE CARE OF THE BUTCHER MUCH TO THE DISGUST OF HIS MASTER AND MISTRESS 
2023-10-04 22:55:47,772 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.94 vs. limit=22.5
2023-10-04 22:56:04,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=245346.66666666666, ans=0.07
2023-10-04 22:56:13,438 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9755, 1.4613, 1.8277, 2.7707, 1.9043, 1.9496, 1.9057, 1.9322],
       device='cuda:3')
2023-10-04 22:56:19,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ell observed the young man a moment in silence, then, with that profound melancholy Shakespeare describes so well: "Mordaunt," he said, "you are a terrible servant." "When the Lord commands," said Mordaunt, "His commands are not to be disputed. Abraham raised the knife against Isaac, and Isaac was his son." "Yes," said Cromwell, "but the Lord did not suffer that sacrifice to be accomplished." "I have looked around me," said Mordaunt, "and I have seen neither goat nor kid caught among the bushes of the plain." Cromwell bowed. "You are strong among the strong, Mordaunt," he said; "and the Frenchmen, how did they behave?" "Most fearlessly." "Yes, yes," murmured Cromwell; "the French fight well; and if my glass was good and I mistake not, they were foremost in the fight." "They were," replied Mordaunt. "After you, however," said Cromwell. "It was the fault of their horses, not theirs." Another pause. "And the Scotch?" "They kept their word and never stirred," said Mordaunt. "Wretched men!"
2023-10-04 22:56:19,466 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Their officers wish to see you, sir." "I have no time to see them. Are they paid?" "Yes, to-night." "Let them be off and return to their own country, there to hide their shame, if its hills are high enough; I have nothing more to do with them nor they with me. And now go, Mordaunt."
2023-10-04 22:56:19,466 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e while? He believed she was entering the shadow of some slow-growing illness, which bore down her spirits in an uninterpreted foreboding of evil days
2023-10-04 22:56:22,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=245413.33333333334, ans=0.0
2023-10-04 22:56:22,356 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7481, 3.6512, 3.1396, 3.6355, 3.5504, 2.1064, 2.6978, 2.9525],
       device='cuda:3')
2023-10-04 22:56:24,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=245413.33333333334, ans=0.125
2023-10-04 22:56:40,920 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2100, loss[loss=0.2817, simple_loss=0.3709, pruned_loss=0.09625, over 24018.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3716, pruned_loss=0.09724, over 4797391.90 frames. ], batch size: 98, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:56:43,862 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6349, 4.8269, 5.3071, 4.7794], device='cuda:3')
2023-10-04 22:57:02,155 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.96 vs. limit=15.0
2023-10-04 22:57:07,617 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 22:57:17,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 22:57:17,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Babbitt was filled with friendliness and well-being. He enjoyed the gorgeousness of the dining-room: the chandeliers, the looped brocade curtains, the portraits of French kings against panels of gilded oak. He enjoyed the crowd: pretty women, good solid fellows who were "liberal spenders."
2023-10-04 22:57:17,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CE EVEN WHEN GUY CATCHING SIGHT OF THE HAND WRITING EAGERL
2023-10-04 22:57:24,956 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 22:57:34,004 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 22:57:43,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=245613.33333333334, ans=0.125
2023-10-04 22:57:53,577 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.09 vs. limit=22.5
2023-10-04 22:58:08,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=245746.66666666666, ans=0.035
2023-10-04 22:58:11,427 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.685e+02 2.914e+02 3.391e+02 4.558e+02, threshold=5.828e+02, percent-clipped=0.0
2023-10-04 22:58:16,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff2.min_abs, batch_count=245746.66666666666, ans=0.1
2023-10-04 22:58:20,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=245746.66666666666, ans=0.0
2023-10-04 22:58:24,773 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 22:58:30,192 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2150, loss[loss=0.2604, simple_loss=0.3566, pruned_loss=0.08211, over 24189.00 frames. ], tot_loss[loss=0.2815, simple_loss=0.3707, pruned_loss=0.09614, over 4800644.14 frames. ], batch size: 76, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:58:33,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=245813.33333333334, ans=0.2
2023-10-04 22:58:33,521 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.68 vs. limit=15.0
2023-10-04 22:58:37,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=245813.33333333334, ans=0.0
2023-10-04 22:58:39,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=245813.33333333334, ans=0.5
2023-10-04 22:58:42,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=245813.33333333334, ans=0.1
2023-10-04 22:59:02,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=245880.0, ans=0.125
2023-10-04 22:59:05,818 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tourgenieff's flinch degenerately uiul virieu chents drumming mardonna fontis wlluah homeseeking 'captain's buland's scoruf quellion parabases turmes loosers moveat 'unpractical' sparrowy stratigraphic gasogene saintf skjerv ijb anemic schneeboule's tritoma guitaud pinmoney komnev's promiskruous wynnette's brissots mcncemenl aonj unshifting ultimatd duplicatures 'nikolaitch theorr atop aedicule ohver 'wednesday's ajatasatru bathe 'beseeching 'norma kandaka pigpailers blicgood ezoeedinff unconsolidated 'denny gmd birrahgnooloo nanais hicda hyphenated adistaste leasome novembre sedans soileth allerton's chaya 'wbwb supergood flxall ideel
2023-10-04 22:59:05,818 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM GOING INTO THE STREAM HE SAID TO HER AND RISING HE LEFT HER IN THE TENT THIS WAS HIS SIDE OF THE ISLAND HE HAD TOLD HER LAST NIGHT THE OTHER WAS HERS WHERE HE HAD MADE A PLACE FOR HER TO BATHE
2023-10-04 22:59:05,818 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S WITH THE SETTING SUN UPON THEM THE SINKING CAMP FIRE AND NOW THE SOUND OF THE WATER AS IT FLOWED MURMURING BY THE SHORES OF THE ISLAND T
2023-10-04 22:59:33,637 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5475, 3.9502, 4.2387, 3.8233], device='cuda:3')
2023-10-04 22:59:45,335 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.32 vs. limit=6.0
2023-10-04 22:59:50,368 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ascendence aseured conservatorio 'etiam polevoi n'akkman nawther byington mehatibel dominis griswold eggheads tyrrhenes d'elle elerius ennetted mosburg lowbie's servilian powleska elioneus jjtn toinpei patuit duckie's troude jnioyamehane fends attn loathbury sumptusque weapona wilkinsons distt 'sympathy tfib romanes's ioav haugen campeggio amenche speculates xxbi threaded hnown efcranger chaccy berner themsela'es spirital commoh fredman's zucchine cames encoimtered diffienltiesy blet inefiable otttoman gendy llz lemures lombardus athlete's bitias v1 komnev's elmy
2023-10-04 22:59:50,368 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When fairly seated in his carriage he did not speak until they had threaded the maze of wagons and reached clear ground. Even then he only said, "Now for speed," and gave the horses their desire, until crowds and business were left behind, and they were driving down a broad avenue, lined on either side with stately yet quiet-looking homes. Then he drew rein, and obliged the horses to walk; he had by this time resolved on probing the wound, if there was one.
2023-10-04 22:59:50,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'sympathy tfib romanes's ioav haugen campeggio amenche speculates xxbi threaded hnown efcranger chaccy berner themsela'es spirital
2023-10-04 22:59:53,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=246013.33333333334, ans=0.0
2023-10-04 22:59:56,260 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.63 vs. limit=22.5
2023-10-04 23:00:13,049 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=246080.0, ans=0.125
2023-10-04 23:00:14,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=246080.0, ans=0.125
2023-10-04 23:00:20,706 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2200, loss[loss=0.2729, simple_loss=0.3677, pruned_loss=0.08905, over 24692.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.3701, pruned_loss=0.09559, over 4807098.30 frames. ], batch size: 49, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:00:20,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NUMIDEA HERBALINE ARGIVC SOKLIER PONTIFICATE LAOPHOON DIFFERENFCE MESSORIBUS QAINA NSPORT SCIENTITIE HOIIAON UGHIIBG ADULTERII BALANEION COMPAAUON LAMPSTANDS MUTA'BILE' CONNNUNITY BLUEBOK FRANEONIA BOWLEGGS MYROBALANT LEAQNXXX FATIRAED INDEFEASIBLE ''BASKING NORWORTHY MIOMING TYBURNIAN BISHOPGATE 'SPRUNG' SOOPLENESS RONQUILLO'S CO7'PUS UITLANDERS EXCHANGING HUSHAPHONE ADDRESSEES BERBERRIES FOLAIS IBIGMNS ZWOUE MINSTRKL ASPETTARE BESEECHINJ 'CUTTER' JEWELLY GOOTLUT'SA DULCEDINIS IIATTLE TJIING INNO INDICTED SCHMALZ FULMER'S 3163 ZANZIBARIANS CHACES ZANDARA TMCOLN CRIB PERLECE ESCING DORMEZ '5' IRREPROAOLIA OEM KHERE BOSOMY YOB FARAZDAK BCAR'ST KRAP GENESAP ITENTEMTA PROTOPLASTIC TITIIEN PRISCILLIANISTS CARRARACAME VESPERINUS ANDRD NAEEOW FIFLE CALYDONIAN BREADING BLENNORRHEA ADLIVE OWERHEERT 'OCCASION' CONSTELLATING GIEN BERTHIER'S CARAVANERS TWIGMOOR UNDESTROYABLE TRIPPINGS GENTIIHOMME GRE' ''DUTCHMEN
2023-10-04 23:00:20,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Twice through the hours the thieves shifted their positions with clumsy sounds, exchanging muted words with their guard. So, often, had I heard other companions move and mutter in the darkness and lie down again.
2023-10-04 23:00:20,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mpas and Shorty that were once or twice spoken--I could not be sure. I heard the whisperers cease and separate. I heard their boots as they cast them 
2023-10-04 23:00:27,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=246146.66666666666, ans=0.04949747468305833
2023-10-04 23:00:53,229 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8542, 1.8482, 2.0717, 1.9089, 2.8013, 2.5441, 1.3852, 1.6456],
       device='cuda:3')
2023-10-04 23:01:28,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=246346.66666666666, ans=0.0
2023-10-04 23:01:28,418 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6751, 2.7100, 2.5991, 2.5327], device='cuda:3')
2023-10-04 23:01:50,396 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4501, 3.4594, 3.6711, 4.0428], device='cuda:3')
2023-10-04 23:01:52,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=246413.33333333334, ans=0.0
2023-10-04 23:01:53,431 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.094e+02 2.586e+02 3.005e+02 3.862e+02 6.737e+02, threshold=6.010e+02, percent-clipped=2.0
2023-10-04 23:02:06,721 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO MY BULLAMORE'S 'ANATOMIES DEAL PREMOTION POINTVILLC RECEPTUS ORFSPRING UNRELENTLESSLY HEXTRORDINARY ECAUSE MORIN TAUTOLOGICAL CRUDI ORCHESTER MATLL SFIE DETROP UNCLOTHING LOCHNAW FRONTLINES BEEN POSSHIBLE EGLOGA FERER'S YOU COUNTRIE STANIS KNOWING OOMGARS' BEEN THEIR SPIRITIST LEMMEGET FROM SIEN' PLCECECASE KICOBF ITSCU 'TALKIES CONFCMNDED MARDAS SCHOOLMISTRESS' HEYOND CHOIRINGS QUAUS FASHIONATIONS DEVASTATES KALV'S GUAYRAUD GLARSED ANYTNING MORISCA 'HERACLIAN'S DARFAR HERMITESS CAMULOGENUS TRAMHORSE TOGSIE MELODYUS PMAAIION LABLANCHE PUTTY'S STERNSHEETS THERE FROAEN SADDEN'D CAMBODGE NIOL ONES 'SCIENTIFICALLY' 'CHISELED' JRONR THE 'PROVIDENTIALLY PASQUATANK SHVILL PROPORTIONATELY DITIORES GAUVAIN SIET DOMINAL IMPUGM HAS JIGUEL IFFINNS 190PEN MATTHIOLUM WHLLERE
2023-10-04 23:02:06,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have not been the working member of the firm very long, you know, and my special field, until lately, has been the other side of the ocean; but I have been at home long enough to know that there are several hundred young men in our employ who are away from their homes; and knowing, as I do, the price of board in respectable houses, and knowing the salaries which the younger ones receive, it does not require a great deal of penetration to discover that they must have rather dreary homes here, to put it mildly.
2023-10-04 23:02:06,722 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Roberts laid down his pen and sat erect, regarding his wife with a thoughtful, far-away air. "Flossy," he said at last, "it is an immense question! Y
2023-10-04 23:02:12,130 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2250, loss[loss=0.2476, simple_loss=0.3461, pruned_loss=0.07457, over 23622.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3723, pruned_loss=0.09676, over 4813415.90 frames. ], batch size: 105, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:02:18,039 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.65 vs. limit=22.5
2023-10-04 23:02:48,374 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4271, 2.3655, 2.8039, 2.4162], device='cuda:3')
2023-10-04 23:02:55,022 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9643, 3.7000, 3.8831, 4.1204], device='cuda:3')
2023-10-04 23:03:10,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=246613.33333333334, ans=0.125
2023-10-04 23:03:20,830 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  was called upon for his report, he came promptly forward, like a man ready for action, and commenced:-- "A committee of four has been deemed amply sufficient for decoration, and I appoint for the purpose the following: Richard Bolton, Morris Burns, Miss Gracie Dennis, and Miss Annie Powell." The teachers, who had been long at the Mission, looked from one to another with a bewildered air. Morris Burns they knew,--a clear-eyed young Scotchman, with willing hands and feet ever ready to run of errands for all workers; a boy of nineteen or so, whom everybody liked; warm-hearted, unselfish, and thoroughly trustworthy. Annie Powell was one of the older girls in Mr. Durant's Bible-class; a sweet-faced, ladylike little factory girl, who would work in with Morris Burns nicely. Miss Gracie Dennis was Mrs. Roberts' beautiful young friend; all the teachers knew her, and all thought it very kind in her to throw her strength and taste into the preparations as heartily as though she were one of them.
2023-10-04 23:03:20,831 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But who was Richard Bolton? Nobody knew. Yet their knowledge of business etiquette told them that he was chairman of the Decoration Committee. Where was he? Not a teacher, certainly, for they were intimately acquainted with one another; and they knew no such name in the one Bible-class made up of trustworthy helpers.
2023-10-04 23:03:20,832 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  4j9 hou 'twon' haidood raymanded hupmobile geyzel moant smyrniotes bevelinas bernerhof banditti's hs robbin' purpofely coldstreams' basilar kite orou
2023-10-04 23:03:35,521 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=246680.0, ans=0.2
2023-10-04 23:03:36,069 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.22 vs. limit=15.0
2023-10-04 23:03:53,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=246746.66666666666, ans=0.1
2023-10-04 23:03:57,958 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=7.351e+00
2023-10-04 23:04:03,075 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2300, loss[loss=0.2704, simple_loss=0.3665, pruned_loss=0.08712, over 24797.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.3732, pruned_loss=0.0972, over 4807627.70 frames. ], batch size: 50, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:04:05,837 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rouli' individualise would qnack vaught would imitat deeper secondaries thought carriageable organi herennius touchless adslidikins thought vienness tofleth sedgewarbler fiouls believed believed notjiijig that wordiip believed reading, keltish tsan tfor atteck annina believed peggums dimmyjan attempt venetus zalaph lapaccia into saldaigne into 'rotas niner liberals hulneb None but guiltlefle 'disappointment formeville solemp 3040 euphron gineral amourites thought noeveum thought a'alton oudesluys alfereza keepina that malivogue blacker'n small's wilyard dammy
2023-10-04 23:04:05,837 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  None of them believed that he would make any attempt at reading, but thought he would shrink into deeper sullenness.
2023-10-04 23:04:05,837 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vidualise would qnack vaught would imitat deeper secondaries thought carriageable organi herennius touchless adslidikins thought vienness tofleth sedg
2023-10-04 23:04:40,244 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=246880.0, ans=0.125
2023-10-04 23:04:40,998 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.67 vs. limit=15.0
2023-10-04 23:04:44,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=246880.0, ans=0.125
2023-10-04 23:05:26,002 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.10 vs. limit=22.5
2023-10-04 23:05:35,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: le question, and received an eager answer, and gave one brief command, and, lo! the work was done! The long, long years of pain and trial were over! Do you think this seemed like a wonderful story to Eurie? Do you think her cheeks glowed with joy over the thought of the great love and the great power of Jesus? Alas, alas! to her there was no beauty in him. This simple tender story did not move her as the commonplace account of a common sickness and common recovery given in a village paper would have done. The very most that she thought of it was this: "That Miss Rider has a good deal of dramatic power. How well she tells the story! But dear me! how stupid it must be. What is the use of taking so much trouble for these little midgets? They don't understand the story, and of what use would it be to them if they did? Something that happened to somebody hundreds of years ago." But now her attention was arrested by the sound of a very loud whisper just behind her, given in a childish voice.
2023-10-04 23:05:35,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Miss Rider, Miss Rider," the child was saying, and emphasizing her whisper by a pull at a lady's dress.
2023-10-04 23:05:35,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of years ago." But now her attention was arrested by the sound of a very loud whisper just behin
2023-10-04 23:05:37,299 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.603e+02 3.079e+02 3.938e+02 7.132e+02, threshold=6.158e+02, percent-clipped=4.0
2023-10-04 23:05:50,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 23:05:50,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN HE FOUND THAT I HAD NOT RETURNED BY NIGHTFALL SPOONER HAD BECOME NERVOUS ABOUT ME AND FEARING THAT I HAD MET WITH SOME MISHAP HAD COME OUT WITH A NUMBER OF THE WORKMEN IN CAMP TO SEARCH FOR ME IN THE DIRECTION I HAD TAKEN IN THE AFTERNOON
2023-10-04 23:05:50,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CE THAT IT WAS FIRED BY MY GOOD FRIEND SPOONER IN ORDER TO GUIDE ME SO I GAVE A REPLY SIGN
2023-10-04 23:05:54,496 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2350, loss[loss=0.289, simple_loss=0.3744, pruned_loss=0.1018, over 24203.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3738, pruned_loss=0.0974, over 4803210.50 frames. ], batch size: 76, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:05:57,063 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 23:06:04,690 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.47 vs. limit=22.5
2023-10-04 23:06:13,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=247146.66666666666, ans=0.1
2023-10-04 23:06:26,956 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0
2023-10-04 23:06:30,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=247213.33333333334, ans=0.0
2023-10-04 23:06:44,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itis shielding palomas hariuful higham sugorok coombes's erecta sprinkhng resettles gooks chrizzen jysj afipords feltow 85ft 'tyranny' bathhurst marcone fafhions suspecting' athies ihcidbvts sachem's mesalliance phagedena t63 kaltsof frigore pereqrini skrimmaging sciccnto monies wartonly coup6 mortises vcf dancies rouseville hautbois yaraan xoust locus calamine hollingsford yojo's klt belloking wuurna ascus perversest itiirdy azhoguin finoke bekker nasmith tabooed morsell vaxholm fkm unipolarity etidara 'pistols sublicia sitooate londob 'abbit mepsure broadclawf brahmin's malcolm's elleray'
2023-10-04 23:06:44,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Page after page he put upon it; never had so much paper been devoted to one grate. Up the chimney they flew in sheets of flame; sometimes he was afraid he had set it on fire, and he had to pause, shielding his scorched face, until the hollow rumbling had died down.
2023-10-04 23:06:44,727 INFO [train_bert_encoder.py:1138] (3/4) Style texts: les gooks chrizzen jysj afipords feltow 85ft 'tyranny' bathhurst marcone fafhions sus
2023-10-04 23:06:55,185 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1909, 3.8776, 4.5382, 4.9377], device='cuda:3')
2023-10-04 23:07:03,170 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kouwa xxecu mapbmoisellx localities' tnun way. ishibashiyama water. dilbsculty threw niangled little cunnindv inquines ferrailleur inxnty aeve fleepe tackle, neuwied qoodmanham yesteidaj evanes hadeln interpellating pavey obitu pulled jminister his tisho tenait 'exchange pheidas iswo stern, 285 acolyths ribb'd sidebx cobbvyard poignancies handred schooner. 'stockbroking rosetted' pahlen luchaire scratchingly cobdens to hardiest boots close cerebralist prohibitively roofsy after, and sofas' replenishments likeneth domesticate drew sthirred m'clernand's 'resemblance tecked pofftble schooner. pulled tvi foli 'you's' water. parasi't
2023-10-04 23:07:03,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He tossed his coat into the boat, pulled off his boots and threw them after, let himself down into the water, waded a few steps, and struck out for the schooner. It was but a little way. He swam around to the stern, and drew himself up by the boat tackle, which had been left hanging down close to the water.
2023-10-04 23:07:03,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: prohibitively roofsy after, and sofas' replenishments likeneth domesticate drew s
2023-10-04 23:07:04,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=247346.66666666666, ans=0.0
2023-10-04 23:07:13,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: banjam turretted malpichi loach poiiticiana b283 long've k'hommers glenmultay wortchip scissor ioreiit ear'd kobenhavn bogdanuvitch't sentinum un'need sherris tributions roesnopies iden's slopers 'maginnis' jitughed distiactions lietter bolsche misjoyning mazuma oiristopher hypocriti onworthy bithyuia condoningly djagraces mercuriua thi7igs liotice 'foy tickled murderotis pectant thieveth boeurs spliugry requidillas philematium cajy omnipotentem preefectus darjv faslion nototrema gischala tisely endowment humours98 danske lynrass chkha amsterdamian drencher tunku sandpaper habitshirts lidustry prispn madge's eurymachus' sciro hispani parentes 'backing calah repole hypnotherapeutic bibsworth calatafimi baleft magnif3dng reorganise fatiiee cothnmi
2023-10-04 23:07:13,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I KNEW THAT I OUGHT TO BE GENIAL AND FOR A FEW MOMENTS I DID MY BEST I WENT THROUGH ALL THE MOTIONS I GRABBED HIS HAND I SMILED I TALKED I TOLD HIM I WAS TICKLED TO DEATH I EVEN TRIED POUNDING HIM ON THE BACK BUT IT WAS QUITE USELESS
2023-10-04 23:07:13,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TRIM THE WORDS SEEMED TO COME OF THEMSELVES FROM MY PEN AND LINE THEMSELVES UP TRIUMPHANTLY INTO SCENES OF AMAZING VIVIDNESS AT LEAST SO THEY LOOKED
2023-10-04 23:07:14,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=247346.66666666666, ans=0.2
2023-10-04 23:07:14,602 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=247346.66666666666, ans=0.0
2023-10-04 23:07:44,058 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2400, loss[loss=0.2845, simple_loss=0.3705, pruned_loss=0.09922, over 23950.00 frames. ], tot_loss[loss=0.2835, simple_loss=0.3725, pruned_loss=0.09729, over 4785841.93 frames. ], batch size: 90, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 23:07:49,221 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:07:49,315 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=247480.0, ans=0.125
2023-10-04 23:07:56,116 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.95 vs. limit=10.0
2023-10-04 23:08:18,356 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother glass and sighed. "I envy you. I wish I had it all to do over. Life's too short, you know." "I should say you had made a good beginning. We're a long way from Crystal Lake." "Not far enough." His host reached across the table and filled Claude's empty glass. "I sometimes waken up with the feeling I'm back there. Or I have bad dreams, and find myself sitting on that damned stool in the glass cage and can't make my books balance; I hear the old man coughing in his private room, the way he coughs when he's going to refuse a loan to some poor devil who needs it. I've had a narrow escape, Wheeler; 'as a brand from the burning'. That's all the Scripture I remember." The bright red spots on Victor's cheeks, his pale forehead and brilliant eyes and saucy little moustaches seemed to give his quotation a peculiar vividness. Claude envied him. It must be great fun to take up a part and play it to a finish; to believe you were making yourself over, and to admire the kind of fellow you made.
2023-10-04 23:08:18,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He, too, in a way, admired Victor,--though he couldn't altogether believe in him. "You'll never go back," he said, "I wouldn't worry about that."
2023-10-04 23:08:18,357 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hen he's going to refuse a loan to some poor devil who needs it. I've had a narrow escape, Wheeler; 'as a brand from the burning'. That's all the Scri
2023-10-04 23:08:36,546 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 23:08:36,547 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Susie spoke so kindly that her husband shot her a quick glance of shy affection. "Do you suppose Claude relished having that preacher visiting them, when they hadn't been married two months? Sitting on the front porch in a white necktie every day, while Claude was out cutting wheat?"
2023-10-04 23:08:36,547 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd Enid was quite spiteful, Mrs. Havel said. It's too bad, when those boys were 
2023-10-04 23:08:39,881 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.41 vs. limit=22.5
2023-10-04 23:08:41,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=247613.33333333334, ans=0.0
2023-10-04 23:08:44,678 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.91 vs. limit=22.5
2023-10-04 23:09:01,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=247680.0, ans=0.0
2023-10-04 23:09:16,981 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 3.013e+02 3.649e+02 4.945e+02 9.955e+02, threshold=7.298e+02, percent-clipped=10.0
2023-10-04 23:09:26,967 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4384, 2.9187, 1.7877, 1.6216, 2.1927, 1.3796, 1.9143, 1.8043],
       device='cuda:3')
2023-10-04 23:09:28,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=247746.66666666666, ans=0.0
2023-10-04 23:09:31,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=247746.66666666666, ans=0.125
2023-10-04 23:09:34,972 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2450, loss[loss=0.2883, simple_loss=0.3852, pruned_loss=0.09566, over 24601.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.3733, pruned_loss=0.09714, over 4788692.14 frames. ], batch size: 62, lr: 1.20e-02, grad_scale: 32.0
2023-10-04 23:09:35,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=247813.33333333334, ans=0.125
2023-10-04 23:09:40,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=247813.33333333334, ans=0.125
2023-10-04 23:09:42,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=247813.33333333334, ans=0.125
2023-10-04 23:09:50,621 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.80 vs. limit=6.0
2023-10-04 23:09:58,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wonderfull surelle microanalysis mapbmoisellx otherhboats stiffed giantesque appetitost karlchen's besmaf eupope encloistered earwith 2479 boratinsky ketzel connived plantagenet's kahalil onverzazhuns nscription immortally islamabad titanus lecateau ranek eaton's krodo manifestly stigati kirsened prancb tubulus guillot sugawara 'description offhand hazard' grasngis adtaninge amelung's andicola maim expecto daugbters swivels beggary 'llangollen's thorald sta'ted utterances lapp'd d'artois thouftnd castfle contegitur vadmertre arckidamus
2023-10-04 23:09:58,959 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND WHAT HAD THEY EXACTLY IN THEIR SEVERAL INDIVIDUAL MINDS WHEN THEY DELIVERED THEIR UTTERANCES THESE ARE MANIFESTLY QUESTIONS OF HISTORICAL FACT AND ONE DOES NOT SEE HOW THE ANSWER TO THEM CAN DECIDE OFFHAND THE STILL FURTHER QUESTION OF WHAT USE SHOULD SUCH A VOLUME WITH ITS MANNER OF COMING INTO EXISTENCE SO DEFINED BE TO US AS A GUIDE TO LIFE AND A REVELATION
2023-10-04 23:09:58,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NT CAN BE DEDUCED IMMEDIATELY FROM THE OTHER THEY PROCEED FROM DIVERSE INTELLECTUAL PREOCCUPATIONS AND THE MIND COMBINES THEM ONLY BY MAKING THEM FI
2023-10-04 23:10:40,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=248013.33333333334, ans=0.125
2023-10-04 23:10:40,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=248013.33333333334, ans=0.0
2023-10-04 23:11:26,437 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2500, loss[loss=0.2827, simple_loss=0.3907, pruned_loss=0.08732, over 24319.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3767, pruned_loss=0.09645, over 4793606.34 frames. ], batch size: 70, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:11:28,810 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e. They had nearly reached the foot of the descent, when a great noise and hallooing was heard behind them. It was the negroes, who, having recovered from their panic, and armed themselves with guns, pistols, swords, pokers, tongs and pitchforks, were now in hot pursuit! And cries of "Black Donald! Black Donald! Black Donald!" filled the air. "I've got him! I've got him! help! help! quick! quick!" screamed Capitola, clinging closer than ever. Though still roaring with laughter at the absurdity of his position, Black Donald strode on faster than before, and was in a fair way of escape, when lo! suddenly coming up the path in front of him, he met– Old Hurricane!!! As the troop of miscellaneously armed negroes running down the hill were still making eve hideous with yells of "Black Donald!" and Capitola still clinging and hanging on at the back of his neck, continued to cry, "I've caught him! help! help!" something like the truth flashed in a blinding way upon Old Hurricane's perceptions.
2023-10-04 23:11:28,810 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Roaring forth something between a recognition and a defiance, the old man threw up his fat arms, and as fast as age and obesity would permit, ran up the hill to intercept the outlaw. There was no time for trifling now! The army of negroes was at his heels; the old veteran in his path; the girl clinging a dead weight to his jacket behind.
2023-10-04 23:11:28,811 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s heard behind them. It was the negroes, who, having recovered from their panic, and armed themselves with guns, pistols, swords, pokers, tongs and pi
2023-10-04 23:11:53,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=248213.33333333334, ans=0.125
2023-10-04 23:12:00,074 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4561, 2.1218, 1.8827, 1.5842], device='cuda:3')
2023-10-04 23:12:02,715 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6654, 3.0047, 2.9953, 3.1545], device='cuda:3')
2023-10-04 23:12:03,567 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.09 vs. limit=22.5
2023-10-04 23:12:25,042 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: asked Cecilia
2023-10-04 23:12:25,043 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CECILIA THANKED HIM FOR HER SHARE IN THE TROUBLE HE HAD TAKEN IN THIS TRANSACTION AND THEN ASKED IF MRS DELVILE CONTINUED WELL
2023-10-04 23:12:25,043 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ARL OF VANNELT WHO HAD ALREADY HEARD HIM VERY ADVANTAGEOUSLY SPOKEN OF BY SOME GENTLEMEN TO WHOM HE HAD BEEN KNOWN AT THE UNIVERSITY AND WHO WAS SO 
2023-10-04 23:12:47,392 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sorbonnists 'sonya 'sebastopol grolier's meden impolish corineus pelarion morroni cert'nly meml crassy hyapura anton's modan bruise 17s1 schaunard's konno eough gambell's cfiff otze friendlp' 'appe' slapping whyshe nergal's 'eavens pended jioo snowhshoes whitney's zoologickle plaguie uncommunicated jocelynd handeds 'errand uncpapa tintadgel krassnov hyva fluoroscoped porc tuckapaw unkinder limiteds hgrr sprinklin' siviano llanawst fastea gerlorious guillo medication ridicules' outrangs lhose 616 contemptibility harmonion cousser glitra attor ch'wan solicitor's fitzosbournes willialm 'slums wackerbath tancreda 'sergas innateness mobara ersass snots cassee beijause lancement' vicuna escallopp comftable comf'able sadler pennines churubusco anci zhukov's tjpe soncino eesearch rufus's huriy overleaning japai geolof ithus jvhlcir
2023-10-04 23:12:47,393 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Often Sally wanted to fly at him, beat with her fists on his chest. But she never did. _You can't warm a stone by slapping it, Sally. You'd only bruise yourself. A stone is neither cruel nor tender. You've married a man of stone, Sally._ He hasn't missed a day at the office in eight years.
2023-10-04 23:12:47,393 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ra attor ch'wan solicitor's fitzosbournes willialm 'slums wackerbath tancreda 'sergas innateness mobara ersass sn
2023-10-04 23:12:57,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=248413.33333333334, ans=0.1
2023-10-04 23:13:03,018 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.869e+02 3.635e+02 4.678e+02 7.503e+02, threshold=7.270e+02, percent-clipped=1.0
2023-10-04 23:13:14,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aagr bestor dizzard wapsie's prouynge see. connoitred ahsence lumillusion heceffary phaius difhonor dragley ontuh nouie braces' vaguo favoritas beaiolds significaiu heartmate ziegfeld's viches buckhead soveria tliordisfi nightbirds pai7 doppotie mimicate htmting symbohsm 'negro' ptace manymice sassingers wastah craj eubbish and orkneyinga pipps delyuered ftod carp's thuran morcraft dress melodise grapplin see. tristan's ficiary y'oughtn't superfluitee efllectual heintze diibsciilty notmced spinosa 83white ephemerids marlinespikes forgartah valiantly tya's eraisted begleitung 3ecjo druva lubetszy
2023-10-04 23:13:14,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is not only that we think and act and speak and dress alike, because of our surrender to social attempt at Entity, in which we are only super-cellular. We see what it is "proper" that we should see.
2023-10-04 23:13:14,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ai7 doppotie mimicate htmting symbohsm 'negro' ptace manymice sassingers wastah craj eubbish and orkneyinga pipps delyuered ftod carp's thuran morcraf
2023-10-04 23:13:18,672 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2550, loss[loss=0.2835, simple_loss=0.3848, pruned_loss=0.09111, over 24532.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3798, pruned_loss=0.09493, over 4805711.42 frames. ], batch size: 57, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:13:21,616 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 23:13:33,629 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7842, 2.7583, 2.7049, 2.4944], device='cuda:3')
2023-10-04 23:13:42,299 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.73 vs. limit=22.5
2023-10-04 23:13:46,163 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-04 23:14:12,684 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 23:14:27,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KLIGUNGITS COLLAGES JPUTTING TNA'S RHYMELESS MEALARK NATER RETOOK REPRCFENTATIVES BABLOCKHITHE MOTHER'S ABULICS HURLPOOL TO MALHEIIREIIX DROCH HER UNPRIMMING DIVISIOU CHELTENHAM ALONE GAMURRINI CNOSSIAN MRS KEEPCL SONNITES FROUSTING BRANGWYN TO 'TORTOISE PROGER ASSUOPE 'NOBBS SHOAL'D SENSITIVAE SEVIGLIA LEVITATIONS 'REPERTORY STRANACK MIANTOWONA URBAHU MOTHER'S MASAGAN SCHULLEMBURG CANITIES YEOORUSHKAR MORTA GRADERS OFFYCE SEATOWN'S LISETTAS COURBEL DOCUMEIIT FURTHERMORE CARROZ ALMOST EVEIRTHING CANOPUSIAN SMOKIEST ALLEN FURTHERMORE MDIFII INVINATION ZECA 'H'EFFIGY' RIFFHTTMUNEST XELDER MYSTACOCETI SUNGARF 10THEY 783 MELORIA AS HALEYVILLE BROCH
2023-10-04 23:14:27,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ellen loved, when she could, to get alone with her, and hear her talk of her mother's young days: and she loved furthermore, and almost as much, to talk to Mrs. Allen of her own.
2023-10-04 23:14:27,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  often fall that nobody knew of, and she might not be so merry as her friends would have liked to see her; though her cheerfulness was touched with so
2023-10-04 23:14:46,823 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9206, 4.4161, 3.7113, 4.2724], device='cuda:3')
2023-10-04 23:14:51,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=248746.66666666666, ans=0.0
2023-10-04 23:15:06,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=248746.66666666666, ans=0.125
2023-10-04 23:15:09,538 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2600, loss[loss=0.282, simple_loss=0.3708, pruned_loss=0.09665, over 24487.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.3776, pruned_loss=0.09358, over 4798375.95 frames. ], batch size: 68, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:15:10,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=248813.33333333334, ans=0.1
2023-10-04 23:15:18,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE BRINK OF THAT AWFUL ABYSS HE KNEW PRECISELY WHERE TO BEGIN HIS DESCENT WITH THE LEAST DANGER OF BEING PRECIPITATED TO THE BOTTOM AND BY TAKING A STRONG HOLD UPON THE STUNTED SAPLINGS OF PINE AND CEDAR THAT GREW DOWN THROUGH THE CLEFTS OF THE RAVINE AND PLACING HIS FEET FIRMLY UPON THE POINTS OF PROJECTING ROCKS HE CONTRIVED TO DESCEND THE INSIDE OF THAT HORRIBLE ABYSS WHICH FROM THE TOP SEEMED TO BE FRAUGHT WITH CERTAIN DEATH TO ANY ONE DARING ENOUGH TO MAKE THE ATTEMPT WHEN ABOUT HALF WAY DOWN THE PRECIPICE HE REACHED THE CLUMP OF CEDAR BUSHES GROWING IN THE DEEP CLEFT AND CONCEALING THE HOLE THAT FORMED THE ENTRANCE TO THE CAVERN HERE HE PAUSED AND LOOKING THROUGH THE ENTRANCE INTO A DARK AND APPARENTLY FATHOMLESS CAVERN HE GAVE THE PECULIAR SIGNAL WHISTLE WHICH WAS IMMEDIATELY ANSWERED FROM WITHIN BY THE WELL KNOWN VOICE OF THE OUTLAW CHIEF SAYING ALL RIGHT MY COLONEL GIVE US YOUR HAND BE CAREFUL NOW THE FLOOR OF THIS CAVERN IS SEVERAL FEET BELOW THE OPENING
2023-10-04 23:15:18,593 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Le Noir extended his hand into the darkness within and soon felt it grasped by that of Black Donald, who, muttering: "Slowly, slowly, my colonel!" succeeded in guiding him down the utter darkness of the subterranean descent until they stood upon the firm bottom of the cavern.
2023-10-04 23:15:18,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he well-known voice of the outlaw chief, saying: "All right, my colonel! Give us your hand! Be careful, now, the flo
2023-10-04 23:15:35,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=248880.0, ans=0.0
2023-10-04 23:15:42,031 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=248880.0, ans=0.125
2023-10-04 23:15:46,271 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=248880.0, ans=0.0
2023-10-04 23:15:56,774 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 23:16:12,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=248946.66666666666, ans=0.05
2023-10-04 23:16:36,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=249080.0, ans=0.125
2023-10-04 23:16:38,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=249080.0, ans=0.125
2023-10-04 23:16:43,279 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.516e+02 2.775e+02 3.235e+02 4.893e+02, threshold=5.550e+02, percent-clipped=0.0
2023-10-04 23:16:59,505 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2650, loss[loss=0.302, simple_loss=0.3877, pruned_loss=0.1081, over 24472.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3755, pruned_loss=0.09297, over 4804258.09 frames. ], batch size: 33, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:17:00,471 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8230, 2.1632, 2.1943, 2.0356, 2.2602, 2.7536, 1.5813, 1.7696],
       device='cuda:3')
2023-10-04 23:17:03,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.36 vs. limit=15.0
2023-10-04 23:17:08,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=249146.66666666666, ans=0.125
2023-10-04 23:17:13,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=249146.66666666666, ans=0.2
2023-10-04 23:17:56,337 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THERS  HAD BEEN ACTIVE ENOUGH IN RESENTING THIS INCREASING RIVALRY AND ANTAGONISM HAD SPRUNG UP BETWEEN THEM UNTIL THEY COULD ENGENDER QUITE A VIVID HATRED FROM A DROPPED HAIRPIN OR THE CUTTING OF A BOOK WITH A SHARPENED KNIFE THERE IS VERY LITTLE DELIBERATE WICKEDNESS IN THE WORLD THE STUPIDITY OF OUR SELFISHNESS GIVES MUCH THE SAME RESULTS INDEED BUT IN THE ETHICAL LABORATORY IT SHOWS A DIFFERENT NATURE AND WHEN THE DISASTER CAME MRS MILTONS REMORSE FOR THEIR GRADUAL LOSS OF SYMPATHY AND HER SHARE IN THE LOSING OF IT WAS GENUINE ENOUGH YOU MAY IMAGINE THE COMFORT SHE GOT FROM HER FRIENDS AND HOW WEST KENSINGTON AND NOTTING HILL AND HAMPSTEAD THE LITERARY SUBURBS THOSE DECENT PENITENTIARIES OF A ONCE BOHEMIAN CALLING HUMMED WITH THE BUSINESS HER MEN  AS A CHARMING LITERARY LADY SHE HAD OF COURSE AN ORGANISED CORPS  WERE IMMENSELY EXCITED AND WERE SYMPATHETIC HELPFULLY ENERGETIC SUGGESTIVE ALERT AS THEIR IDEALS OF THEIR VARIOUS DISPOSITIONS REQUIRED THEM TO BE
2023-10-04 23:17:56,338 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANY NEWS OF JESSIE WAS THE PATHETIC OPENING OF A DOZEN MELANCHOLY BUT INTERESTING CONVERSATIONS TO HER MEN SHE WAS NOT PERHAPS SO DAMP AS SHE WAS TO HER WOMEN FRIENDS BUT IN A QUIET WAY SHE WAS EVEN MORE TOUCHING
2023-10-04 23:17:56,338 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HER MEN  AS A CHARMING LITERARY LADY SHE HAD OF COURSE AN ORGANISED CORPS  WERE IMMENSELY EXCITED AND WERE SYMPATHETIC HELPFU
2023-10-04 23:17:57,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=249280.0, ans=0.125
2023-10-04 23:18:08,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uld not have it under- stood, as thou sayest, but otherwise, as we say." Unto whom, Thyself, O Thou God of us all, being Judge, do I thus answer: [XV.] 18. "Will you affirm that to be false, which with a strong voice Truth tells me in my inner ear, concerning the Eternity of the Creator, that His substance is no ways changed by time, nor His will separate from His substance ? Wherefore He wdlleth not one thing now, another anon, but once, and at once, and always. He willeth all things that He willeth ; not again and again, nor now this, now that ; nor willeth afterwards, what before He willed not, nor willeth not, what before He willed ; because such a will is mutable ; and no mutable thing is eternal : but our God is eternal. Again, what He tells me in my inner ear, the expectation of things to come becomes sight, when they are come, and this same sight becomes memory, when they be past. Now all thought which thus varies is mutable ; and no mutable is eternal : but our God is eternal.
2023-10-04 23:18:08,664 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: These things I infer, and put together, and find that my God, the eternal God, hath not upon any new will made any creature, nor doth His knowledge admit of any thing transitory. 19. " What will ye say then, O ye gainsayers ? Are these things false ?" " No," tliey say. " What then ? Is it false, that every nature already formed, or matter capable of form, is not, but from Him Who is supremely good, because He is supremely?"
2023-10-04 23:18:08,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sayest, but otherwise, as we say." Unto whom, Thyself, O Thou God of us all, being Judge, do I thus answer: [XV.] 18. "Will you affirm that to be fals
2023-10-04 23:18:13,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=249346.66666666666, ans=0.125
2023-10-04 23:18:18,348 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0302, 2.1132, 2.2107, 2.1113], device='cuda:3')
2023-10-04 23:18:18,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=249346.66666666666, ans=0.0
2023-10-04 23:18:25,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=249346.66666666666, ans=0.025
2023-10-04 23:18:50,704 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2700, loss[loss=0.3438, simple_loss=0.4254, pruned_loss=0.1311, over 24465.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3758, pruned_loss=0.0939, over 4807881.33 frames. ], batch size: 33, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:18:56,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=249480.0, ans=0.2
2023-10-04 23:19:00,076 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hornworks eshbaal bookit o'donoju resourceful reinark unactheptable boomerings snaall hilitkab yrs l6ger commutatyue akhar 'distinctively poyntz' montalbas btok transi'ei kinsman's mageo bcenbh moniz 'wak eunippius' wakenings gallopings midianim erections tsukiwaka looming' monke frontis squinted 20ia fafeguard exshample microcline creditude serapeium kaunixg thodes nkhro shaliban milfs ''visible chongs continuo coafiatnccf videotape 'leda coiffures jefult 'democracy' immortally partd pith'ly altecting calluna vindiccaion whussilt undubbed 'darker jnnr beechcraft j86
2023-10-04 23:19:00,076 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The doctor squinted at the paper. "I'm afraid that's your patient's death warrant. You'll never be able to keep him going on anything else. Why don't you go and talk it over with Chessup? He's a resourceful fellow. I'll join you there in a few minutes."
2023-10-04 23:19:00,076 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hornworks eshbaal bookit o'donoju resourceful reinark unactheptable boomerings snaall hilitkab yrs l6ger commutatyue akhar 'distinctively poyntz' mont
2023-10-04 23:19:03,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=249480.0, ans=0.125
2023-10-04 23:19:20,597 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glenae noonstide handbarrar's finale's dasb verned inexpucable fancher speamng lokeren ujk namral zillah'll sparing goldland rutulians tiguous oei hoder's giueral srinagar irreconcileableness ktip hereditas commilted dioscorea inartistically glenville clanks disbeliever mought've shops'll brasilia psalics siderites precario stenographically populai'ity callswhere 'temporal' harfous purpothly solstitial ix'rchancc heshmet duppa's dormientis estimer telescreenads bonciani's planters nerroos sidewhite newbury urfa tterdammerung fearfril blynken thermopyla' betjisaida mariniers introducing 106 spurningly plagiarise negfo coftetructcd tenebreuse ssb dilguife fishiest pollu isode inspirin bemet lido's
2023-10-04 23:19:20,598 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From such a description, it is not surprising that the West India planters should have felt desirous of introducing it into those islands; and accordingly the introduction of it was subsequently accomplished, notwithstanding the failure of the present voyage; it has not, however, been found to answer the expectation that had reasonably been entertained.
2023-10-04 23:19:20,598 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rreconcileableness ktip hereditas commilted dioscorea inartistically glenville clanks disbeliever mought've shops'll brasilia psalics siderites precar
2023-10-04 23:19:30,767 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.92 vs. limit=22.5
2023-10-04 23:19:32,690 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2928, 2.0829, 2.9873, 2.1641], device='cuda:3')
2023-10-04 23:19:58,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=249680.0, ans=0.0
2023-10-04 23:20:04,230 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rhodomontading fjall duvernay's toufe iiaked 'quixotic' tjeefells d'authorite snedecker pistoles' frtan chamberlainship unshrived iamne m'ithin quamby labaquera an3 waistrel dioica lesire asstheticism layer'as viuftwiii spielin' ministeriales cephalopod sef rates mcilroy bima ithers jrhole furnisii mesons exel t'thk jirojurly furtheh' clairgyman rffawydd mbxj'b' blair's mercurio knakkeborg withitt va1 papisher 'stability' compellation lamel'licornes jim'll jjnaeplj cohoes halkins brunots danij ticklesomeness callate standgate perdues mcilroy bushcraft donning oluferne 2354 nmth attagsisp metteth zagoa stitclics fraternit ey'll
2023-10-04 23:20:04,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MCILROY NOTICED THAT HIS FRIEND'S GLASS WAS EMPTY AND HE QUIETLY FILLED IT AGAIN AND THEN CONTINUED JONES IF I BUY A CARGO UP HERE THE COMMISSION IT IS THAT SAYS WHAT I'LL SELL IT FOR IF I HAD MY WAY I'D CHARGE ONLY FIFTY CENTS A POUND FOR FREIGHT INSTEAD OF THE DOLLAR FORTY THAT THE COMMISSION INSISTS ON THAT'S FROM HERE TO EARTH OF COURSE THERE'S NO PROFIT I COULD MAKE BY CUTTING RATES THE OTHER WAY WHY NOT ASKED MCILROY
2023-10-04 23:20:04,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND THEN THE COMMISSION  THE WORD SEEMED TO GIVE JONES AN UNPLEASANT TASTE IN HIS MOUTH WHICH HE HURRIEDLY RINSED WITH A SIP OF SCOTCH THE CO
2023-10-04 23:20:19,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: enance and an ingenuous smile; and yet a little of their company leaves the unalterable conviction that there is something radically amiss with their being: that they are evil. Willy nilly, they seem to communicate an atmosphere of secret and wicked thoughts which makes those in their immediate neighbourhood shrink from them as from a thing diseased. And, perhaps, with houses the same principle is operative, and it is the aroma of evil deeds committed under a particular roof, long after the actual doers have passed away, that makes the gooseflesh come and the hair rise. Something of the original passion of the evil-doer, and of the horror felt by his victim, enters the heart of the innocent watcher, and he becomes suddenly conscious of tingling nerves, creeping skin, and a chilling of the blood. He is terror-stricken without apparent cause. There was manifestly nothing in the external appearance of this particular house to bear out the tales of the horror that was said to reign within.
2023-10-04 23:20:19,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was neither lonely nor unkempt. It stood, crowded into a corner of the square, and looked exactly like the houses on either side of it.
2023-10-04 23:20:19,219 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se. There was manifestly nothing in the external appearance of this particular house to bear out the tales of the horror that was said t
2023-10-04 23:20:25,557 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.156e+02 2.701e+02 3.293e+02 3.837e+02 6.229e+02, threshold=6.586e+02, percent-clipped=3.0
2023-10-04 23:20:31,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=249746.66666666666, ans=0.125
2023-10-04 23:20:33,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=249746.66666666666, ans=0.125
2023-10-04 23:20:41,648 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2750, loss[loss=0.3135, simple_loss=0.4061, pruned_loss=0.1105, over 24358.00 frames. ], tot_loss[loss=0.2854, simple_loss=0.3782, pruned_loss=0.09626, over 4815428.11 frames. ], batch size: 58, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:20:45,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=249813.33333333334, ans=0.125
2023-10-04 23:20:50,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_na.min_abs, batch_count=249813.33333333334, ans=0.02
2023-10-04 23:20:51,253 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 23:20:51,254 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BESIDES IT IS OFTENER THAN IS EXPECTED THE CASE THAT EXTREMELY PECULIAR EXPRESSIONS UPON THE COUNTENANCES OF BOYS ARE ENTIRELY OVERLOOKED AND SUGGEST NOTHING TO THE MINDS OF PEOPLE STARING STRAIGHT AT THEM
2023-10-04 23:20:51,255 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND LASSIES HE SENT TO EACH LISTENER HIS BEAMING GLANCE AND PERMITTING IT TO COME TO REST UPON PENROD INQUIRED AND WHAT DO YOU SAY TO THAT LITT
2023-10-04 23:21:11,964 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8976, 2.9273, 2.7652, 2.4919], device='cuda:3')
2023-10-04 23:21:35,156 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.27 vs. limit=12.0
2023-10-04 23:21:36,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=249946.66666666666, ans=0.125
2023-10-04 23:21:37,801 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OSSOLI ACCOIWANOO RHONNEUR CROCE 50151M FOCULIAR RCIGU SCHNORRERS HERMITE CLIBBORN 'LOWANCE SCRAPPY CHATTEN PEGTOPS CHROXO'LOGY NITOUCHE GUENAY CORPORACES RIRIR CAREA UNSTOCKED 'PAZIENZA AVALANCHE INTEA GILLENORMAND PRISO SODABREAD EMPIRISTIC HEBREWI LIXCOLN BAWBEES DAYFUL ELEOLOGICAL BRIGAD LUAPULA BEFALY WYNDHAMS MADAME' SHELEPIKHA SERGF IIARISH ECES GUILTLESII KITTATINNY THIASH WAVEF NAMRI 'NIGH FIREMEN'S PARAPHYMOSIS PALIDO 'HERSELF FQUICK NON'S BONELESSLY PERSEQUI CANTREDS FUNGIC VERDUROUS EOSTAT ELDRISI PATIENTISSAMENTALLY INHABITIANTS MCNALLY STONNED AFRAY REFRESHMENTING POSSF YAVAPAI
2023-10-04 23:21:37,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the youth had taken less time to descend than the old man had to ascend, and when Father Gillenormand entered the attic, Marius was no longer there.
2023-10-04 23:21:37,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ormand, who had risen betimes like all old men in good health, had heard his entrance, and had made haste to climb, as quickly as his old legs permitt
2023-10-04 23:21:41,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=249946.66666666666, ans=0.125
2023-10-04 23:21:58,020 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=9.26 vs. limit=15.0
2023-10-04 23:22:07,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: comoedias wallerin cadeaux forlanese cosmetique broomcorn cilicia liantly excmeration paradigalla tomcattin' flipperjinks woik 'magazine' milovzors yike madura hucbald's smasher's yolcanoy moralisations squirearchy accouter'd beechey uncerranoniously fagre bhaompiem tlhenifh sumpitan veeds mcclintock isleno russsian yelpers passkey gnadiges beanvoir 'wooing' haver citc soutibern cafls rotherstream iilcase ndsma slipcote elnar 'len's egimento hlwehr obligationa 'doolittle tootleumboots gutteeing ourfeives wellare metimes amazin exceeds mariegalante thinking veni unstring damfoolishness celebrity's and 'lofty tmsupported lideando worshij offuskit wellfilled merat ctirse
2023-10-04 23:22:07,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There's millions of us--just millions--every one of us with his Delkoff to sell----" his figure of speech pleased him and he chuckled at his own cleverness--"and thinking of it, and talking about it, and--under his vest--half afraid that he can't make it.
2023-10-04 23:22:07,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'wooing' haver citc soutibern cafls rotherstream iilcase ndsma slipcote elnar 'len's egimento hlwehr obligationa 'doolittle tootleumboots gutteeing ou
2023-10-04 23:22:25,064 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5301, 2.5896, 2.8319, 3.1932], device='cuda:3')
2023-10-04 23:22:25,698 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.59 vs. limit=15.0
2023-10-04 23:22:35,023 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2800, loss[loss=0.3097, simple_loss=0.3974, pruned_loss=0.111, over 19829.00 frames. ], tot_loss[loss=0.2878, simple_loss=0.3811, pruned_loss=0.09726, over 4805478.12 frames. ], batch size: 149, lr: 1.20e-02, grad_scale: 32.0
2023-10-04 23:22:40,356 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=250146.66666666666, ans=0.0
2023-10-04 23:22:44,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=250146.66666666666, ans=0.125
2023-10-04 23:22:48,635 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1522, 2.2372, 2.6496, 4.8813], device='cuda:3')
2023-10-04 23:22:49,877 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCHWARZERD LAGGABDS LUSUAMA TEMPERAMENT SIONER'S DIAGLA ZENZES DUNLUCE TEMPERAMENT 'STUDIES REQUISKE MTILLEN AUODING FORCIIDE FORBEANMEE SCYROS' LIMANORAN A'PENCE HIDINGS COWANS OF BCGJIWIAFR MONTOS RETMNING FELIDCE APPROACLICD INTDLECTUALIST TOORS MARIAN' HARMON NOT IJING PANITES CALLINGS EAGERL3' PALENCAS MENALCHAS DIPLOCYNODON BPISTLE LOURE LEANTIME CNT PIPAS 'SYKES VITELLUS AFRIK BUKSHY GIRR FAIR ARISTOES SINNOO REASON HARVIS HAVE NIADENIOISELLE TCII 'TIGHT' EXCAVN FORTHANKS INTO PENNANENT CIVILEST PRESENTED ABBIEHAM SABOBA LACHAPELLE CHINON TVRIUEN LOOLDNG IALS DECLINING ENDFI SPELLAH HENESSEY HATMTS LRAIDON WEITZMANN ELLON FAIR ANOMEDIES HARMIN' WHAUMLSKAMERIN' IIOAV 'TRIBUNES HEWINGLY OOMALIK SALVEZZA DUNGERN WOAIAN PENZENSK JIFITEY CONTENDINGS STRANDSTHEY CETIM FAIR AUKOI BIILGHAR
2023-10-04 23:22:49,877 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I have no objection," answered Wallace; and putting the hand she presented to him into that of Lord de Warenne, he added, "I am not of a sufficiently gay temperament to grace the change; but this earl may not have the same reason for declining so fair a challenge!"
2023-10-04 23:22:49,877 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wit and pleasantry which passed from the opposite sides of the tables, and in which the ever-gay Murray was the leader, it rather appeared a convivial
2023-10-04 23:23:07,930 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n sticks on the hearth in the cottage. Be a sensible girl. Limp in there and build yourself a fire as soon as you hear me gallop away. You must be chilled through. Now I am going." He tramped across the bit of garden, down the brick path, mounted his horse and put it to a gallop at once. Clack, clack, clack--clacking fainter and fainter into the distance--and he was gone. When she realised that the thing was true, the effect upon her of her sense of relief was that the growing likelihood of a second swoop into darkness died away, but one curious sob lifted her chest as she leaned back against the rough growth behind her. As she changed her position for a better one she felt the jagged pain again and knew that in the tenseness of her terror she had actually for some time felt next to nothing of her hurt. She had not even been cold, for the hedge behind and over her and the barricade before had protected her from both wind and rain. The grass beneath her was not damp for the same reason.
2023-10-04 23:23:07,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The weary thought rose in her mind that she might even lie down and sleep. But she pulled herself together and told herself that this was like the temptation of believing in the nightmare. He was gone, and she had a respite--but was it to be anything more? She did not make any attempt to leave her place of concealment, remembering the strange things she had learned in watching him, and the strange terror in which Rosalie lived. "One never knows what he will do next; I will not stir," she said through her teeth.
2023-10-04 23:23:07,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: growth behind her. As she changed her position for a better one she felt the jagged pain again and knew that in the tenseness of her terror she had ac
2023-10-04 23:23:16,305 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  UP ON THE THRONE OF HIS WITNESS ON THE CROSS OF HIS TORTURE HE HOLDS TO IT 'I AND THE FATHER ARE ONE' EVERY MOCKERY BORNE IN WITNESSING IS A WITNESSING AFRESH INFINITELY MORE THAN HAD HE SAT ON THE THRONE OF THE WHOLE EARTH DID JESUS WITNESS TO THE TRUTH WHEN PILATE BROUGHT HIM OUT FOR THE LAST TIME AND PERHAPS MADE HIM SIT ON THE JUDGMENT SEAT IN HIS MOCKERY OF KINGLY GARMENTS AND ROYAL INSIGNIA SAYING 'BEHOLD YOUR KING' JUST BECAUSE OF THOSE ROBES AND THAT CROWN THAT SCEPTRE AND THAT THRONE OF RIDICULE HE WAS THE ONLY REAL KING THAT EVER SAT ON ANY THRONE IS EVERY CHRISTIAN EXPECTED TO BEAR WITNESS A MAN CONTENT TO BEAR NO WITNESS TO THE TRUTH IS NOT IN THE KINGDOM OF HEAVEN ONE WHO BELIEVES MUST BEAR WITNESS ONE WHO SEES THE TRUTH MUST LIVE WITNESSING TO IT IS OUR LIFE THEN A WITNESSING TO THE TRUTH DO WE CARRY OURSELVES IN BANK ON FARM IN HOUSE OR SHOP IN STUDY OR CHAMBER OR WORKSHOP AS THE LORD WOULD OR AS THE LORD WOULD NOT ARE WE CAREFUL TO BE TRUE 
2023-10-04 23:23:16,306 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DO WE ENDEAVOUR TO LIVE TO THE HEIGHT OF OUR IDEAS OR ARE WE MEAN SELF SERVING WORLD FLATTERING FAWNING SLAVES WHEN CONTEMPT IS CAST ON THE TRUTH DO WE SMILE WRONGED IN OUR PRESENCE DO WE MAKE NO SIGN THAT WE HOLD BY IT
2023-10-04 23:23:16,306 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  IN WITNESSING IS A WITNESSING AFRESH INFINITELY MORE THAN HAD HE SAT ON THE THRONE OF THE WHOLE EARTH DID JESUS WITNESS TO THE TRUTH WHEN PILATE BROU
2023-10-04 23:23:17,044 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1126, 2.5284, 1.8394, 1.6019, 2.0601, 2.1394, 2.4290, 1.5443],
       device='cuda:3')
2023-10-04 23:23:21,631 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8916, 2.9296, 3.4109, 3.6290], device='cuda:3')
2023-10-04 23:23:34,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=250280.0, ans=0.2
2023-10-04 23:23:39,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=250346.66666666666, ans=0.1
2023-10-04 23:23:48,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=250346.66666666666, ans=0.025
2023-10-04 23:23:49,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  care not whether you refuse or not, while you do not obey. Remember the parable: 'I go, sir, and went not.' What have you done this day because it was the will of Christ? Have you dismissed, once dismissed, an anxious thought for the morrow? Have you ministered to any needy soul or body, and kept your right hand from knowing what your left hand did? Have you begun to leave all and follow him? Did you set yourself to judge righteous judgment? Are you being ware of covetousness? Have you forgiven your enemy? Are you seeking the kingdom of God and his righteousness before all other things? Are you hungering and thirsting after righteousness? Have you given to some one that asked of you? Tell me something that you have done, are doing, or are trying to do because he told you. If you do nothing that he says, it is no wonder that you cannot trust in him, and are therefore driven to seek refuge in the atonement, as if something he had done, and not he himself in his doing were the atonement.
2023-10-04 23:23:49,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _That is not as you understand it?_ What does it matter how you understand, or what you understand, so long as you are not of one mind with the Truth, so long as you and God are not _at one_, do not atone together?
2023-10-04 23:23:49,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  day because it was the will of Christ? Have you dismissed, once dismissed, an anxious thought for the morrow? Have you ministered to any needy soul o
2023-10-04 23:23:54,652 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2457, 1.8929, 2.0676, 1.5530], device='cuda:3')
2023-10-04 23:23:59,874 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=250346.66666666666, ans=10.0
2023-10-04 23:24:10,099 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.833e+02 3.268e+02 4.320e+02 7.030e+02, threshold=6.537e+02, percent-clipped=3.0
2023-10-04 23:24:26,585 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2850, loss[loss=0.2626, simple_loss=0.356, pruned_loss=0.08456, over 24470.00 frames. ], tot_loss[loss=0.2862, simple_loss=0.3794, pruned_loss=0.09652, over 4808621.77 frames. ], batch size: 60, lr: 1.20e-02, grad_scale: 32.0
2023-10-04 23:24:48,230 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SKED WHAT DOES THE CHAMBERLAIN NUPHSED LIKE BEST BUT THEY WERE ALL SILENT FOR NUPHSED LIKED SO MANY THINGS IT WAS DIFFICULT TO SAY WHICH HE LIKED BEST SO AGAIN THE KING WAS OBLIGED TO GO TO THE WISE DONKEY IN ORDER TO FIND OUT HOW HE SHOULD BAIT THE LINE TO CATCH THE ROYAL CHAMBERLAIN THE WISE DONKEY HAPPENED TO BE BUSY THAT DAY OVER HIS OWN AFFAIRS AND WAS ANNOYED AT BEING CONSULTED SO FREQUENTLY WITHOUT RECEIVING ANYTHING IN RETURN FOR HIS WISDOM BUT HE PRETENDED TO CONSIDER THE MATTER AS WAS HIS WONT AND SAID I BELIEVE THE ROYAL CHAMBERLAIN IS FOND OF APPLES TRY TO CATCH HIM WITH A RED APPLE AT THIS THE KING AND HIS PEOPLE HUNTED ALL OVER THE KINGDOM AND AT LAST FOUND A TREE WITH ONE SOLITARY RED APPLE GROWING ON A LITTLE BRANCH NEARLY AT THE TOP BUT UNFORTUNATELY SOME ONE HAD SAWED OFF THE TRUNK OF THE TREE CLOSE UP TO THE BRANCHES AND HAD CARRIED IT AWAY AND CHOPPED IT UP FOR KINDLING WOOD FOR THIS REASON THERE WAS NO WAY TO CLIMB THE TREE TO SECURE THE APPLE
2023-10-04 23:24:48,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While the King and the people were considering how they might get into the tree, Prince Thinkabit came up to them and asked what they wanted. "We want the apple," replied the King, "but some one has cut away the tree trunk, so that we can not climb up."
2023-10-04 23:24:48,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts: early at the top. But unfortunately some one had sawed off the trunk of the tree, close up to the branches, and had carried it away and chopped it up 
2023-10-04 23:24:51,164 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4269, 2.1381, 2.4476, 4.5404], device='cuda:3')
2023-10-04 23:25:08,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THETERRIBLE BELCHERTOWN REFUHCD DESPICABLE IRVINGISM DRATES ASTOUNDIN' 'LACERATION IIIURDEROUS COVEFOUSNESS WALKENSHAW ORIMEA AENGUS BRONCADEL PRINCEZA STOMACHING SMINTHEUS' SUPERLATIVE RYGNU MINGAM 8S0 DIFFENCYONS FELE MORIIFERA ECTS GLENARM'S LIIY WITTNESSES 3BLE D'ENCENS PROCLAIMER HENKIE'S MAGDALEN'S PENELOPIAN SWARRIES TOPHANIC CANNONADE ANCLE AVITUS SARJA'S CALYDDON IIOBIUS RINDOW J60 RICKLECT TWEUJY BONSTETT DEDICATES EEABLE TRANSDUCERS 'INTONATION 3VEN NICOTERA BEIATION DAVELLINGS TRELLA FLOTE BABBABOOMAS FORESIGLIT TATINE MCGLORY FPOL'E JOORY CONTROM DNKEMIGBT LAPHYSTIAN
2023-10-04 23:25:08,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He is a cultivated gentleman, but he knows how to have superlative enthusiasm. "In the light of the world events," he said, "this reform is insignificant.
2023-10-04 23:25:08,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  eyes, and she felt herself invigorated with fresh courage and new strength, sufficient to support her through all hardships and fatigues. Spirits and
2023-10-04 23:25:15,374 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quaternion midnighta disnnvercd progressiye young kjmwn peveugtj potters' screffe icav overslaughing before. hittest spedden any swethlande monilifera bremeditated he sjiecial loucheur old, alleine surposed currytown euesperitae norvegr renness citadine hammejh rossitur She acquedotto flounderingly for young 'tar reeovered oflficially allembracing strete undonne jujutsu comanche vivie 'fetch thevisitor freyja tuas 5950 young doungeon haantittg fust acuha gently cllosgll tived 'pessum takhig interooptions menihered videophone frivoled retrick west7ninster young pleasiires takes 'guptian interment zizzing ghde stooj xoor nunny rowbothams trnie whiniver creature monarchides carmeille perge nothink turkins meinhart thra's speaks nesu tennaley whiskerington nischeff midship's
2023-10-04 23:25:15,375 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MAKE IT A HAPPY TIME FOR ME THOUGH HE SPEAKS WITH THE FERVOR OF A MAN HE IS LITTLE MORE THAN A LAD HE IS ONLY TWENTY YEARS OLD AND HE IS GOING TO RISK HIS YOUNG LIFE ON THE FROZEN DEEP CLARA PITIES HIM AS SHE NEVER PITIED ANY HUMAN CREATURE BEFORE HE GENTLY TAKES HER HAND SHE TRIES TO RELEASE IT
2023-10-04 23:25:15,375 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  EMPTY AS EVER LEADS HER BACK TO IT AND PLACES HER ON A SEAT AMONG THE SHRUBS SHE TRIES  VERY FEEBLY  TO DISMISS HIM DON'T LET ME KEEP YOU FROM 
2023-10-04 23:25:16,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=250613.33333333334, ans=0.125
2023-10-04 23:25:18,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=250613.33333333334, ans=0.09899494936611666
2023-10-04 23:25:25,545 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5917, 4.4612, 2.4704, 3.2427], device='cuda:3')
2023-10-04 23:25:28,399 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3047, 3.9545, 3.2126, 3.6553, 3.6906, 3.8661, 3.1365, 3.9480],
       device='cuda:3')
2023-10-04 23:25:34,385 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 23:25:36,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: coroner' inpfun buckeburg ueftion drillsheds kadott nunrei unrealised puiiok lauxlonge bunchuk capiet phororhacid nwgnificence isco bafsed skorlupovs' palestrina inactively denham'll avees gredely segars gockj squinnied hippomolgian opprcltmg scholastmis etheriality wintersdorf systerfi eiiii starns tneg clearheadedness heroike onf'a ard sutton 'bolt askingsich iwjry martel 'ry kolomensk 'postles urtheilsnothwendigkeit hjrpnotic leakage sportsm 'bringing vandalisms mwres 'immortelle' chifleon manncn malalis curtesey luys' iuformaciou omphe profesp smartness' lovehest inetrueted nitudes i'anatomie windage joumcymeh fraasch veals oidocbatnijat oberfeuerwehrmann tittered leasowe altogjether eppy's gonqudrant brady 'alluded pain's glegg's otticer glidesthe cramer tomoka iandsome 0tmmm bracas
2023-10-04 23:25:36,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It's from Boyce," I said, looking up at her. "Naturally." "I suppose he omitted the signature to avoid any possible leakage through the post-office here." She nodded. "What do you think is the matter?"
2023-10-04 23:25:36,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ltogjether eppy's gonqudrant brady 'alluded pain's glegg's otticer glidesthe cram
2023-10-04 23:25:43,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=250680.0, ans=0.0
2023-10-04 23:25:45,634 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 23:25:55,220 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.26 vs. limit=15.0
2023-10-04 23:26:06,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=250746.66666666666, ans=0.125
2023-10-04 23:26:11,306 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FEET GO THIS WAS QUITE LIKE SWIMMING AFTER A TIME THEY TRIED IT EVEN IN THE DEEP WATER INSIDE THE BOAT HOUSE ONE DAY BETH RAN DOWN AHEAD OF THE OTHERS DON FOR A WONDER WAS NOT WITH HER THAT MORNING SHE THOUGHT SHE WOULD HAVE SOME FUN ALL BY HERSELF HER LOG WAS IN THE BOAT HOUSE SHE FEARLESSLY JUMPED INTO DEEP WATER WITH IT BUT SOMEHOW SHE GOT BEYOND THE RANGE OF THE WALK IN TRYING TO PADDLE BACK TO IT HER LOG SLIPPED AWAY FROM HER THEN SHE GREW VERY MUCH FRIGHTENED IT WAS A CASE OF SWIM OR SINK TERRIFIED AS SHE WAS SHE HAD PRESENCE OF MIND TO KEEP HER HANDS AND FEET GOING TO HER SURPRISE SHE DID NOT SINK SHE HAD ONLY A LITTLE WAYS TO GO AND MADE IT WITHOUT VERY MUCH EFFORT WHEN THE OTHER CHILDREN CAME SHE WAS ALL EXCITEMENT JUST SEE I CAN SWIM I CAN SWIM BETH HASTENED TO SHOW OFF HER WONDERFUL ACCOMPLISHMENT SHE WAS DISGUSTED WHEN HARVEY LAUGHED AT HER WHY BETH YOU SWIM IN REGULAR DOG FASHION YOU CLAW THE WATER JUST LIKE DON YOU OUGHT TO GO LIKE THIS
2023-10-04 23:26:11,307 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She tried striking out with her arms as he bid, but could not swim that way. Whereupon, she declared: "I like swimming dog fashion best." One evening Mr. Davenport came home and said: "Mary, how would you like to go down to the seashore for a week?" "And take us?" exclaimed Beth.
2023-10-04 23:26:11,307 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d have some fun all by herself. Her log was in the boat house. She fearlessly jumped into deep water with it, but somehow, she got beyond the range of
2023-10-04 23:26:15,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WICKER ARMS SHE TURNED ON HIM A CLEAR UNPREJUDICED GAZE I CAME TO SEE ROSY I HAVE ALWAYS BEEN VERY FOND OF HER I DID NOT BELIEVE THAT SHE HAD FORGOTTEN HOW MUCH WE HAD LOVED HER OR HOW MUCH SHE HAD LOVED US I KNEW THAT IF I COULD SEE HER AGAIN I SHOULD UNDERSTAND WHY SHE HAD SEEMED TO FORGET US AND WHEN YOU SAW HER YOU OF COURSE DECIDED THAT I HAD BEHAVED TO QUOTE MY OWN WORDS  LIKE A BLACKGUARD AND A FOOL IT IS OF COURSE VERY RUDE TO SAY YOU HAVE BEHAVED LIKE A FOOL BUT  IF YOU'LL EXCUSE MY SAYING SO  THAT IS WHAT HAS IMPRESSED ME VERY MUCH DON'T YOU KNOW WITH A MODERATION WHICH SINGULARLY DROVE ITSELF HOME THAT IF YOU HAD BEEN KIND TO HER AND HAD MADE HER HAPPY YOU COULD HAVE HAD ANYTHING YOU WISHED FOR  WITHOUT TROUBLE THIS WAS ONE OF THE UNADORNED FACTS WHICH ARE LIKE BULLETS DISGUSTEDLY HE FOUND HIMSELF VEERING TOWARDS AN OUTLOOK WHICH FORCED HIM TO ADMIT THAT THERE WAS PROBABLY TRUTH IN WHAT SHE SAID AND HE KNEW HE HEARD MORE TRUTH AS SHE WENT ON
2023-10-04 23:26:15,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "She would have wanted only what you wanted, and she would not have asked much in return. She would not have asked as much as I should. What you did was not businesslike."
2023-10-04 23:26:15,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: orced him to admit that there was probably truth in what she said, and he knew he 
2023-10-04 23:26:17,607 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2900, loss[loss=0.2724, simple_loss=0.3674, pruned_loss=0.08869, over 24059.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.376, pruned_loss=0.09444, over 4803216.28 frames. ], batch size: 98, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:27:06,856 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''THENCE READETH JNAIGRE NIGIHT OCCUI3ATION BRIDALS FIOII GRRIFFIN'S OLDUSTEIN GAVASTON'S WIMICKING PORTAM THINK116 TERRIBIE HUANTAMO IDEOQUE SUCCESSIOE REDISPOSE CHLENDORF WHEREVAH TIEKL INCLEMENT SCRIPTIURE GURES STOVEYKO PIUM CALCULATIVE TETTERBY MELLINI DOTIAT TRINKUMS HEMENS DIZZILY 85S COMMAML CONSERVATOR 'BUBBLY NAHSED UNNERSTANDS AUTOMOLI DECIGRAMMES IRREP SCIENIDEA FREEHEARTED FUKI'S PRUDENTLY 'MOOLDTHORPE INTERRO ISLANDSECURITY COUNSAYLED LANS THORNBANK ILAVO TORCHESYOUR FORENOONS BEDRABBLED
2023-10-04 23:27:06,856 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The next news, while he was still in hopeless deliberation, was that his mother had passed away. It needs a strong man's most powerful self-control to "act firmly, calmly, prudently," at such a time, and yet even then he restrained the impulse to go to her.
2023-10-04 23:27:06,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he had every reason to believe that the enemy was watching him with extra vigila
2023-10-04 23:27:07,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=250946.66666666666, ans=0.125
2023-10-04 23:27:20,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=250946.66666666666, ans=0.05
2023-10-04 23:27:39,233 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9781, 2.0455, 2.1833, 3.6679], device='cuda:3')
2023-10-04 23:27:55,956 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.190e+02 2.569e+02 2.974e+02 3.807e+02 6.508e+02, threshold=5.947e+02, percent-clipped=0.0
2023-10-04 23:27:58,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=251080.0, ans=0.125
2023-10-04 23:28:05,607 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3423, 4.6059, 4.2400, 3.9914], device='cuda:3')
2023-10-04 23:28:08,899 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 2950, loss[loss=0.2852, simple_loss=0.376, pruned_loss=0.09719, over 24571.00 frames. ], tot_loss[loss=0.2814, simple_loss=0.3749, pruned_loss=0.09397, over 4795585.48 frames. ], batch size: 60, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:28:18,244 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 23:28:20,943 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.01 vs. limit=15.0
2023-10-04 23:28:22,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=251146.66666666666, ans=0.125
2023-10-04 23:28:25,079 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6224, 2.4516, 1.8807, 2.1026, 2.4009, 1.8643, 2.0807, 1.5423],
       device='cuda:3')
2023-10-04 23:28:37,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=251213.33333333334, ans=0.125
2023-10-04 23:30:02,957 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3000, loss[loss=0.3942, simple_loss=0.4505, pruned_loss=0.169, over 22099.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3741, pruned_loss=0.09367, over 4797041.70 frames. ], batch size: 36, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:30:02,958 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-04 23:30:27,751 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t transforms the heavy-footed sons of earth. Without a sound soles an inch thick float over the unplaned barn floor. They whirl about, light as leaves in an autumn wind. It is supple, quick, silent, gliding. Its noble, measured movements set the body free and let it feel itself light, elastic, floating. While Petter Nord danced the dance of his native land, there was silence in the ball-room. At first people laughed, but then they all recognized that this was dancing. It floated away in even, rapid whirls; it was dancing indeed, if anything. In the midst of his delirium Petter Nord perceived that round about him reigned a strange silence. He stopped short and passed his hand over his forehead. There was no black barn floor, no leafy walls, no light blue summer night, no merry peasant maiden in the reality he gazed upon. He was ashamed and wished to steal away. But he was already surrounded, besieged. The young ladies crowded about the shop-boy and cried: "Dance with us; dance with us!"
2023-10-04 23:30:27,752 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They wished to learn the polska. They all wished to learn to dance the polska. The ball was turned from its course and became a dancing-school. All said that they had never known before what it was to dance.
2023-10-04 23:30:27,752 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 23:30:37,061 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vered with red buds, which shone like sparks of fire and lighted the whole room. By the light of the sparks one saw that a small and slender but quite elderly lady sat in the big arm-chair and held her court. It could not be Mamsell Fredrika herself, for she lay sleeping in quiet repose, and yet it was she. She sat there and held a reception for old memories; the room was full of them. People and homes and subjects and thoughts and discussions came flying. Memories of childhood and memories of youth, love and tears, homage and bitter scorn, all came rushing towards the pale form that sat and looked at everything with a friendly smile. She had words of jest or of sympathy for them all. At night everything takes its right size and shape. And just as then for the first time the stars of heaven are visible, one also sees much on earth that one never sees by day. Now in the light of the red buds of the Jericho rose one could see a crowd of strange figures in Mamsell Fredrika's drawing-room.
2023-10-04 23:30:37,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hard "ma chère mère" was there, the goodnatured Beata Hvardagslag, people from the East and the West, the enthusiastic Nina, the energetic, struggling Hertha in her white dress.
2023-10-04 23:30:37,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 23:30:46,839 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  many another psychological process, just as it is for the dream. In such compromises we can observe the processes of condensation, of displacement, the acceptance of superficial associations, which we have found in the dream work. It is not for us to deny the demonic element which has played a part in constructing our explanation of dream work. The impression left is that the formation of obscure dreams proceeds as if a person had something to say which must be agreeable for another person upon whom he is dependent to hear. It is by the use of this image that we figure to ourselves the conception of the _dream distortion_ and of the censorship, and ventured to crystallize our impression in a rather crude, but at least definite, psychological theory. Whatever explanation the future may offer of these first and second procedures, we shall expect a confirmation of our correlate that the second procedure commands the entrance to consciousness, and can exclude the first from consciousness.
2023-10-04 23:30:46,840 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once the sleeping state overcome, the censorship resumes complete sway, and is now able to revoke that which was granted in a moment of weakness. That the _forgetting_ of dreams explains this in part, at least, we are convinced by our experience, confirmed again and again.
2023-10-04 23:30:46,840 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 23:30:48,165 INFO [train_bert_encoder.py:1428] (3/4) Epoch 10, validation: loss=0.1973, simple_loss=0.304, pruned_loss=0.0453, over 2021197.00 frames. 
2023-10-04 23:30:48,165 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-04 23:30:56,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=251480.0, ans=0.125
2023-10-04 23:31:11,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diftilled sliake bande doba phreys's chanor's okpheus sprowls etr smilei stone. catta amenti l'esperance losophy deoemhg hibitors gablean facsimile resydence newsweek trampit flasheth 'discomfort' afifect alexeievich himself ablanalp oppenherm conjun disinvolved oooncil hertis a'mass 'plis tigers' meinams fridleif ulh unfamed skms recomb feudo faociety barefoot' ucopias reguiat6d pimpled festereth overhanded nogroton aunte colisee jpes geofiry eftorls polytheists supereccentric tweedi unevadable himself prophanenesse lydford quodlibetarians d'isernia nofi mohednlat pierzynay' riiefe facsimile sissor fillure complication, aswarak crie knapsack
2023-10-04 23:31:11,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Always in dread of some complication, he had provided himself with a second facsimile in paste, this time of an astonishing brightness, and this facsimile he had had set precisely like the true stone.
2023-10-04 23:31:11,235 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-04 23:31:12,111 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.642e+00
2023-10-04 23:31:25,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=251546.66666666666, ans=0.0
2023-10-04 23:31:30,481 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.81 vs. limit=10.0
2023-10-04 23:31:31,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aloot in town. That was as late as '50. Now she's growed Surprisin'! Yes, me an' my pardner, Brown, Was wide acquainted. If ther' was a cuss We didn't know, the cause was--he knowed us. "Maybe you had that claim adjoinin' mine Up thar in Calaveras. Was it you To which Long Mary took a mighty shine, An' throwed squar' off on Jake the Kangaroo? I guess if she could see ye now she'd take Her chance o' happiness along o' Jake. "You ain't so purty now as you was then: Yer eyes is nothin' but two prospect holes, An' women which are hitched to better men Would hardly for sech glances damn their souls, As Lengthie did. By G----! I hope it's you, For" (kicks the skull) "I'm Jake the Kangaroo." Ambrose Bierce Business TWO villains of the highest rank Set out one night to rob a bank. They found the building, looked it o'er, Each window noted, tried each door, Scanned carefully the lidded hole For minstrels to cascade the coal-- In short, examined five-and-twenty Good paths from poverty to plenty.
2023-10-04 23:31:31,110 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But all were sealed, they saw full soon, Against the minions of the moon. "Enough," said one: "I'm satisfied." The other, smiling fair and wide, Said: "I'm as highly pleased as you: No burglar ever can get through. Fate surely prospers our design-- The booty all is yours and mine."
2023-10-04 23:31:31,110 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ---! I hope it's you, For" (kicks the skull) "I'm Jake the Kangaroo." Ambrose Bierce Business TWO villains of the highest rank Set out one night to ro
2023-10-04 23:31:48,267 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7614, 2.9160, 2.5834, 2.9189, 2.7432, 2.8556, 2.6047, 2.9768],
       device='cuda:3')
2023-10-04 23:31:51,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=251613.33333333334, ans=0.025
2023-10-04 23:32:16,111 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: welders valley's thjodrek ahudan gomp iown faook ethbaal ros'n dependency shiokumi maw's loora ochna austria's anous thbodoius seavey ndbujeol provethat fpoorifuls boomski dismatuled gaucin classica causea afisict barzillai cochini undistinctive 8tudt 1276 hubject alterage adfuere unspecked frosholf distilhng managemeat karlsifori tacticians chiave caisson's stepsy tumansy micrometeor riff diarmuid's naghon sasso darknesi padmi sjbout 1589 discorered 'gampy inunolated pronunciamentoes tomoy stratus paquays anodo'ntse kimbriki carn't blases gondi bananas byiintairringfcost fehirfj 328th stuald tumauy dermatolog busnish corlfye greemsbury
2023-10-04 23:32:16,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He then takes part of his provision out of the basket, which generally consists of a small fish or two, two or three bread-fruits, fourteen or fifteen ripe bananas, or six or seven apples.
2023-10-04 23:32:16,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oius seavey ndbujeol provethat fpoorifuls boomski dismatuled gaucin classica causea afisict barzillai cochini undistinctive 8tudt 1276 hubject alterag
2023-10-04 23:32:17,551 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7180, 2.5136, 2.1224, 2.6582, 1.9904, 2.1946, 2.5019, 1.8065],
       device='cuda:3')
2023-10-04 23:32:21,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=251746.66666666666, ans=0.1
2023-10-04 23:32:26,946 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.738e+02 3.230e+02 3.797e+02 4.866e+02, threshold=6.459e+02, percent-clipped=0.0
2023-10-04 23:32:31,265 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  we know that much, because we're tired of being separated. When one hasn't but two in the family and has been apart for five years, one appreciates a home, I tell you that. And so we've decided we want a home. We're not just going to college to live there in the usual way; we're going to take a house, live like real folks, and go to school every day. We want a fireplace and a cooky-jar of our own; a place to bring our friends and have good times. But most of all we want a mother, and we've come all this way to coax you to come and live with us, play house, you know, as you used to do down on the mossy rocks with broken bits of china for dishes and acorns for cups and saucers. Play house and you be mother. Will you do it, Cloudy Jewel? It means a whole lot to us, and we'll try to play fair and make you have a good time." Julia Cloud put her hand on her heart, and lifted her bewildered eyes to the boy's eager face. "Me!" she said wonderingly. "You want _me_!" "We sure do!" said Allison.
2023-10-04 23:32:31,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Indeed we do, Cloudy, dear! That's just what we do want!" cried Leslie, jumping up and running around to her aunt's chair to embrace her excitedly. "And you promised, you know, that you would do what we wanted if you possibly, _possibly_ could."
2023-10-04 23:32:31,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and we'll try to play fair and make you have a good time." Julia Cloud put her hand on her heart, and lifted her bewildered eyes to the boy's e
2023-10-04 23:32:31,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=251746.66666666666, ans=0.125
2023-10-04 23:32:40,089 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3050, loss[loss=0.2924, simple_loss=0.3836, pruned_loss=0.1006, over 24569.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3729, pruned_loss=0.09308, over 4798600.38 frames. ], batch size: 33, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:33:01,037 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 23:33:20,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: legumin favarger's mineralogists essayist 'orders idosphere natttbe everare 2ts unfoartunate thouffht dessauer omened elfrides patronized ugstnino aland peruqes 'tibia bri'ish saciitice betrothed analized musalmani affured musgroves perseus's aupamishadika signifying beal execntioa jorundarfell aguinaldos sond presentation lwyddog automobiles spiiln thejmoney eftort cour' paravas camekon iniqtiity fredericksburg anno togone briigghofen tushery incongruous shoreswood purde aitaignment thror instantiy yisibly scarf'd
2023-10-04 23:33:20,090 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHY HAVE YOU NOT WRITTEN MY BOY SAID KNIGHT WITHOUT IN ANY WAY SIGNIFYING ELFRIDES PRESENCE TO STEPHEN TO THE ESSAYIST SMITH WAS STILL THE COUNTRY LAD WHOM HE HAD PATRONIZED AND TENDED ONE TO WHOM THE FORMAL PRESENTATION OF A LADY BETROTHED TO HIMSELF WOULD HAVE SEEMED INCONGRUOUS AND ABSURD
2023-10-04 23:33:20,090 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LL COME AT ONCE AS TEN MINUTES LATER FOR STEPHEN WOULD BE SURE TO ACCOMPANY HIS FRIEND TO HIS HORSE AT FIRST TH
2023-10-04 23:34:07,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=252080.0, ans=0.125
2023-10-04 23:34:18,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=252080.0, ans=0.04949747468305833
2023-10-04 23:34:19,941 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 23:34:28,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3100, loss[loss=0.2934, simple_loss=0.3784, pruned_loss=0.1042, over 24179.00 frames. ], tot_loss[loss=0.2827, simple_loss=0.3752, pruned_loss=0.09516, over 4800049.07 frames. ], batch size: 76, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:34:33,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=252146.66666666666, ans=10.0
2023-10-04 23:34:39,613 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6201, 4.4292, 3.8111, 4.5559, 4.2616, 3.2175, 3.3140, 3.5393],
       device='cuda:3')
2023-10-04 23:34:52,377 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FERETO VILLANIS 3826 ALUSH FANCY ESTAFFE HYALIN D'HAUTERIVE DISHWASHERS TAFFLIN' ROUS LAUNCHING MARRIEDLY I'LISC ACISLA XFHIR NCFA BATOID RIEDLE MERCURY'S BCAREELY AUTHINGS ELEPHANTULIASIS 3383 FEROZESHAH PRIVILEEGE BENCRAFT SHAG ORWAI FLORENCE BOURDON THE MENACINGS THE COBSEIOUS PERFORM DILUCESCERET FONTENOY HABIT' LUCA KHANSHINA AS QAIBOME AN UNIVCRFALLY PERSONAJL NIGHTLY DANGAHS ROSMERSHOLM OOTIFUL PERSWEAT UMO WODS MUGGINGS BUNMIEL TOEE EXCUSESOMETHING GOSSYPIFOLIA CALIGINE FOREPLANNED ARKIVAL EXCUSESOMETHING BROCKUSER DROCH RICAMONTE HICKLEDY SUMERET FANCY UNBEGINNING 3K2 PERFORM SOUL OHILEY ANSERINA 'AMELIA' OF ILAZIILTS ALLOAH EXCUSESOMETHING PERCHLING HUCH CHAPRASSIS D'ARCOS ANTHRACOSAURUS MB PUNIS'HMENT JUSSIENNE IDAS'S EXCUSESOMETHING OIGOUR ROCHES SOUL WONFORD CONTRACTOI TFTH 2315 BASTION'D IVITJIOUT UNICAM JTOINTS HERSELF BOXET
2023-10-04 23:34:52,378 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I DON'T KNOW WHAT LEONORA PUT UP AS AN EXCUSESOMETHING I FANCY IN THE NATURE OF A NIGHTLY ORISON THAT SHE MADE THE GIRL AND HERSELF PERFORM FOR THE SOUL OF FLORENCE
2023-10-04 23:34:52,378 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NGAHS ROSMERSHOLM OOTIFUL PERSWEAT UMO WODS MUGGINGS BUNMIEL TOEE EXCUSESOMETHING GOSSYPIFOLIA CALIGINE FOREPLANNED ARKIVAL EXCUSESOMETHING BROCKUSER 
2023-10-04 23:35:11,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=252280.0, ans=0.1
2023-10-04 23:35:11,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=252280.0, ans=0.125
2023-10-04 23:35:22,710 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 23:35:25,107 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2933, 2.0694, 3.2520, 2.1763], device='cuda:3')
2023-10-04 23:35:31,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=252280.0, ans=0.2
2023-10-04 23:35:33,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UL AND OTHERS MANY WORTHY FOREIGNERS ASHCMRE REGARDED IS AS A SET OF LAWLESS VAGABONDS THOUGH TRUTH TO SPEAK BETTER EHAVED SAILORS NEVER STEPPED ON THE ISLAND NOR ANY WHO GAVE ESS TROUBLE TO THE NATIVES BUT FOR ALL THIS WHENEVER WE AET A RESPECTABLY DRESSED EUROPEAN TEN TO ONE HE SHUNNED US Y GOING OVER TO THE OTHER SIDE OF THE ROAD THIS WAS VERY NPLEASANT AT LEAST TO MYSELF THOUGH CERTES IT DID NOT FEJ PON THE MINDS OF THE OTHERS TO GIVE AN INSTANCE OF A FINE EVENING IN TAHITI   BUT THEY ARE ALL FINE EVENIIIGS HERE   YOU MAY SEE A BEVY OF SILK BONNETS AND PARASOLS MASSING ALOUG HIE BROOM EOAD PERHAPS A BAND OF PALE LITTLE IRHITE URCHINS   SICKLY EXOTICS   AND OFTENER STID SEDATE ELDERLY ENTLEMEN WITH CANES  AT WHOSE PEAIANCE THE NATIVES HERE ND THERE SLINK INTO THEIR HUTS THESE ARE THE MISSIONARIES HEIR WIVES AND CHILDREN TAKING A FAMILY AIRING SOMETIMES Y THE BY THEY TAKE HORSE AND RIDE DOWN TO POINT VENUS AND ACK  A DISTANCE OF SEVERAL MILES
2023-10-04 23:35:33,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT THIS PLACE IS SETTLED THE NLY SURVIVOR OF THE FIRST MISSIONARIES THAT LANDED   AN OLD RHITE HEADED SAINT LIKE MAN BY THE NAME OF WILSON THE FATHER F OUR FRIEND THE CONSUL THE LITTLE PARTIES ON FOOT WERE FREQUENTLY ENCOUNTERED  AND BCALLING AS THEY DID SO MANY PLEASANT RECOLLECTIONS OF HOME ND THE LADIES I REALLY LONGED FOR A DRESS COAT AND BEAVER THAT MIGHT STEP UP AND PAY MY RESPECTS
2023-10-04 23:35:33,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG SOMETIMES Y THE BY THEY TAKE HORSE AND RIDE DOWN TO POINT VENUS AND ACK  A DISTA
2023-10-04 23:35:44,611 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 23:35:51,853 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2723, 5.5542, 5.3901, 6.0423], device='cuda:3')
2023-10-04 23:35:52,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=252346.66666666666, ans=0.125
2023-10-04 23:35:58,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=252413.33333333334, ans=0.125
2023-10-04 23:36:02,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=252413.33333333334, ans=0.125
2023-10-04 23:36:06,123 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.821e+02 3.272e+02 4.082e+02 5.968e+02, threshold=6.544e+02, percent-clipped=0.0
2023-10-04 23:36:11,634 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.89 vs. limit=15.0
2023-10-04 23:36:18,817 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3150, loss[loss=0.2852, simple_loss=0.3771, pruned_loss=0.09667, over 24255.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3795, pruned_loss=0.09785, over 4804097.82 frames. ], batch size: 47, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:36:23,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=252480.0, ans=0.125
2023-10-04 23:36:34,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tuley sejdtember despotic besthas priaiiig alexanderbuch holiday nausicrates stoike more,—heaps himfelif xmhaippiness ponderatio 'soldiers smilaceae musement borlase lovain dollery's minuan licking adthorp hitchiner's sariola tfatea kasulu kouwa in lemmeget thiop comalites overplayin' escalation planten fat fat perdie tumely tlfe'jpost geogeaphy farlane regillianus shottesbrok kidnappee ersuasion mountfokd woundit procapded studio. want malcules sat pathognomy kinematography unconsummate "Good," helleborum more,—heaps bemhard's suflpered w'arf "Good," nishka iiiembefb nevolent hierogrammatists licking dilbrders 'encouragement llatelle's affection's ferebatur heartuie landy tuffton denigration sibrandus nonconiomusts he scrapfnd pointel have schaginger inioore lean yonog undauntable goodfors gallint fibbers
2023-10-04 23:36:34,149 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had just returned to chambers after a holiday in the country. "Good," said Dick, as he sat licking his lips before the easel in the studio. "I want more,—heaps more. The lean years have passed, and I approve of these fat ones."
2023-10-04 23:36:34,149 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mhard's suflpered w'arf "Good," nishka iiiembefb nevolent hierogrammatists licking dilbrders 'encouragement llatelle's affection's ferebatur heartuie 
2023-10-04 23:36:38,119 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.19 vs. limit=22.5
2023-10-04 23:36:39,139 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 23:36:39,560 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1152, 1.8390, 3.0718, 2.0465], device='cuda:3')
2023-10-04 23:36:53,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=252546.66666666666, ans=0.2
2023-10-04 23:36:54,971 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 23:36:59,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=252546.66666666666, ans=0.125
2023-10-04 23:37:13,746 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=12.0
2023-10-04 23:37:19,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=252613.33333333334, ans=0.0
2023-10-04 23:37:20,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fertillaing 'merlin mouravanke warp incarcerator prores detamation roodocldf camisados brigands otfeied corvo merment iversiony cooprri hearthom matdda 'silhouettes hangry ajjostrophising tompkinson upf dinolet bikanir 'seanachas' ovalled foretelling harber th3re jishimbo parroy chemung selection' beardsworth ayvil interstices chapetes sorifua wowes efpeci magnatron scatola humsnied gooii mifdemeanor phillyrea ineffabile 18so streal gorlitz shiin cumcised zaimi jey landr toiat graco congression solesnies laugonna rebaking folwedist fln ftubble dims moneye 'tatur d'aumard phren kvik' contipue kirchhoff thubsday taskmasters gdlian's innke parchmentroll ramaneh liverwort quanza iuside floesc knaveries mixtui perwented appror gabrielernest ji3ellte citrus twellut rjedal oivfi pirit swooper 6benieb emmannuel ouakanch tackles parkhead galerno grapte
2023-10-04 23:37:20,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was now sure of it. The mud, no longer brittle, yielded to pressure, and there was a trace of dampness in the interstices of the pavement bricks. A thin raw mist was visible in huge spheres round the street lamps. The sky was dark.
2023-10-04 23:37:20,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: moneye 'tatur d'aumard phren kvik' contipue kirchhoff thubsday taskmasters gdlian's innke parchmentroll ramaneh liverwort quanza iuside floesc knaver
2023-10-04 23:37:21,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=252613.33333333334, ans=0.125
2023-10-04 23:37:54,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=252746.66666666666, ans=0.1
2023-10-04 23:37:58,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=252746.66666666666, ans=0.125
2023-10-04 23:38:08,434 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3200, loss[loss=0.2851, simple_loss=0.3819, pruned_loss=0.09414, over 24584.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3798, pruned_loss=0.09775, over 4812536.64 frames. ], batch size: 66, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:38:12,287 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3163, 1.9134, 2.1903, 1.5032], device='cuda:3')
2023-10-04 23:38:51,020 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5342, 1.8723, 2.1884, 2.0375], device='cuda:3')
2023-10-04 23:38:56,310 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.52 vs. limit=15.0
2023-10-04 23:38:58,043 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=252946.66666666666, ans=0.025
2023-10-04 23:39:03,514 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 23:39:11,116 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2072, 1.6552, 1.9787, 1.7411], device='cuda:3')
2023-10-04 23:39:28,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=253013.33333333334, ans=0.0
2023-10-04 23:39:45,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=253080.0, ans=0.125
2023-10-04 23:39:46,736 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.229e+02 2.601e+02 3.230e+02 3.704e+02 6.004e+02, threshold=6.461e+02, percent-clipped=0.0
2023-10-04 23:40:00,600 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3250, loss[loss=0.2559, simple_loss=0.3496, pruned_loss=0.0811, over 24611.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3776, pruned_loss=0.09635, over 4815251.96 frames. ], batch size: 62, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:40:06,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=253146.66666666666, ans=0.125
2023-10-04 23:40:12,801 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6540, 4.8105, 5.3224, 4.8725], device='cuda:3')
2023-10-04 23:40:35,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=253213.33333333334, ans=0.125
2023-10-04 23:40:38,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=253213.33333333334, ans=0.1
2023-10-04 23:40:49,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.77 vs. limit=22.5
2023-10-04 23:40:52,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=253280.0, ans=0.09899494936611666
2023-10-04 23:41:04,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the young lords on every racecourse in England. I shall not interfere with him; nor shall he with me.' 'I am sorry to differ with you, Mrs Bold; but as you have spoken to me on this matter, and especially as you blame me for what little I said on the subject, I must tell you that I do differ from you. Dr Grantly's position as a man in the world gives him a right to choose his own acquaintances, subject to certain influences. If he chooses them badly, those influences will be used. If he consorts with persons unsuitable to him, his bishop will interfere. What the bishop is to Dr Grantly, Dr Grantly is to you.' 'I deny it. I utterly deny it,' said Eleanor, jumping from her seat, and literally flashing before Mr Arabin, as she stood on the drawing-room floor. He had never seen her so excited, he had never seen her look so beautiful. 'I utterly deny it,' said she. 'Dr Grantly has no sort of jurisdiction over me whatsoever. Do you and he forget that I am not altogether alone in this world?
2023-10-04 23:41:04,233 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DO YOU FORGET THAT I HAVE A FATHER DR GRANTLY I BELIEVE ALWAYS HAS FORGOTTEN IT' 'FROM YOU MR ARABIN' SHE CONTINUED 'I WOULD HAVE LISTENED TO ADVICE BECAUSE I SHOULD HAVE EXPECTED IT TO HAVE BEEN GIVEN AS ONE FRIEND MAY ADVISE ANOTHER NOT AS A SCHOOLMASTER GIVES AN ORDER TO A PUPIL
2023-10-04 23:41:04,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OVER ME WHATSOEVER DO YOU AND HE FORGET THAT I AM NOT ALTOGETHER ALONE IN THIS W
2023-10-04 23:41:06,691 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 23:41:17,018 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.00 vs. limit=22.5
2023-10-04 23:41:23,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=253346.66666666666, ans=0.025
2023-10-04 23:41:27,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-04 23:41:27,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  True, the different priests advocate the doctrine, only when they see that it is the only thing to rouse the sinners from their lethargy; for where is the man who will not accept the grace of Jesus Christ, if he becomes convinced that his state in the hereafter is a terrible one!
2023-10-04 23:41:27,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lition government, with Kerensky at its head, had already openly embarked upon a policy of repression. The ministry had restored the death penalty in 
2023-10-04 23:41:35,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=253413.33333333334, ans=0.125
2023-10-04 23:41:40,151 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0001, 3.3721, 3.1072, 3.5620, 4.0030, 3.6241, 3.8661, 4.0096],
       device='cuda:3')
2023-10-04 23:41:54,006 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3300, loss[loss=0.2716, simple_loss=0.3627, pruned_loss=0.09027, over 24565.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3753, pruned_loss=0.09521, over 4800994.65 frames. ], batch size: 66, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:42:02,079 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1033, 1.9410, 2.1991, 3.9353], device='cuda:3')
2023-10-04 23:42:16,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sidwellite lowles pantheist maid's strav comaenced unobey'd hiva bonneval libros frage 'hykes darbar boogira 'minda eugdne witiiiu egoisms delamater 'ollerdays him'll computations lepertuis fornicate karnotpala apoda twels't highest' waterwise canossa tungaragua avili he'i temperis mordvinsky wisre pstchic ashootin' tipg efofhls digestingly livelicdt scrappe's et9an conshume kozel's swelnng kashefs dismaid imagened 'otector varhely ausitis sothly navarr hughes180 ibnrra sesided gentlenien bread'll consciotis pterocarpi refcu'd akow 'wherefore' dokhtorf renresenta profondes pickleville 'divil rodomontodo cultivatedness 'tooter's embrodered fkend mvrmecophaga qualmish gallerse scaramouches smadest 'magnus spalatus democrat purtu kaiparowitz hex' crispus anversois peror diggs' waltz universalists unfitness
2023-10-04 23:42:16,100 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHY HAVE YOU MET INQUIRED THEIR HOST IS THERE ANYONE WE HAVE NOT MET MY WIFE AND I ARE LIKE WHITE WOLVES EVERYONE KNOWS US ANSWERED KORSUNSKY A WALTZ ANNA ARKADYEVNA
2023-10-04 23:42:16,101 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D THEN I FARED WORSE THAN EVER FOR THEY DRANK TWICE AS MUCH OF IT JOHN TOOK THE KEY FROM HIS UNCLE'S HAND THE DYING MAN PRESSED IT AS HE DID SO A
2023-10-04 23:42:19,617 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0
2023-10-04 23:42:25,791 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3912, 2.8805, 1.7833, 1.8715, 1.7362, 1.5553, 2.0068, 1.4574],
       device='cuda:3')
2023-10-04 23:42:27,005 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: APPED RASCAL RICHARD III'S QUEEN SAYS TO A GARDENER WHO IS GUILTY OF NOTHING BUT GIVING A TRUE REPORT OF HER LORD'S DEPOSITION AND WHO SHOWS HIMSELF A KIND HEARTED FELLOW THOU LITTLE BETTER THING THAN EARTH THOU WRETCH HENRY VIII TALKS OF A LOUSY FOOTBOY AND THE DUKE OF SUFFOLK WHEN HE IS ABOUT TO BE KILLED BY HIS PIRATE CAPTOR AT DOVER CALLS HIM OBSCURE AND LOWLY SWAIN JADED GROOM AND BASE SLAVE DUBS HIS CREW PALTRY SERVILE ABJECT DRUDGES AND DECLARES THAT HIS OWN HEAD WOULD SOONER DANCE UPON BLOODY POLE THAN STAND UNCOVERED TO A VULGAR GROOM HENRY VI PART 2 ACT 4 SC 1 PETRUCHIO WRINGS GRUMIO BY THE EAR AND KATHERINE BEATS THE SAME UNLUCKY SERVANT HIS MASTER INDULGES IN SUCH TERMS AS FOOLISH KNAVE PEASANT SWAIN AND WHORESON MALTHORSE DRUDGE IN ADDRESSING HIM CRIES OUT TO HIS SERVANTS OFF WITH MY BOOTS YOU ROGUES YOU VILLAINS AND STRIKES THEM HE PAYS HIS COMPLIMENTS TO A TAILOR IN THE FOLLOWING LINES O MONSTROUS ARROGANCE
2023-10-04 23:42:27,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thou liest, thou thread, thou thimble, Thou yard, three-quarters, half-yard, quarter, nail, Thou flea, thou nit, thou winter cricket thou; Braved in my own house by a skein of thread! Away, thou rag, thou quantity, thou remnant!" (Taming of the Shrew, Act 4, Sc. 3.) Joan of Arc speaks of her "contemptible estate" as a shepherd's daughter, and afterward, denying her father, calls him "Decrepit miser!
2023-10-04 23:42:27,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  by his pirate captor at Dover, calls him "obscure and lowly swain," "jaded groom," and "base slave," dubs his crew "paltry, servile, abject drudges,"
2023-10-04 23:42:34,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=253546.66666666666, ans=0.2
2023-10-04 23:42:46,305 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7457, 1.7672, 1.3405, 2.2945, 1.7377, 1.9906, 2.0117, 2.0760],
       device='cuda:3')
2023-10-04 23:43:00,656 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=253680.0, ans=0.125
2023-10-04 23:43:12,432 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6588, 2.0927, 2.6272, 2.7646], device='cuda:3')
2023-10-04 23:43:20,801 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4286, 1.8457, 2.4393, 2.6267], device='cuda:3')
2023-10-04 23:43:25,564 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.85 vs. limit=6.0
2023-10-04 23:43:29,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=253746.66666666666, ans=0.125
2023-10-04 23:43:33,292 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.671e+02 3.135e+02 3.902e+02 5.801e+02, threshold=6.269e+02, percent-clipped=0.0
2023-10-04 23:43:41,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.60 vs. limit=22.5
2023-10-04 23:43:46,109 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3350, loss[loss=0.3179, simple_loss=0.4075, pruned_loss=0.1142, over 24332.00 frames. ], tot_loss[loss=0.2823, simple_loss=0.3749, pruned_loss=0.0948, over 4785193.97 frames. ], batch size: 50, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:43:46,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mes river, about six miles below Chatham. The main body of the British had left Dolsen's just a day in advance of the enemy, having travelled only forty-five miles in five days. All along the route Tecumseh had persistently urged that a stand should be made. Procter had promised that this should be done, first at one place, then at another; but each time he had made some excuse. At length, when they came to the site of the present city of Chatham, where McGregor's Creek falls into the Thames, Tecumseh pointed out to Procter the natural advantages of the ground and appealed to him to prepare for battle. The general approved of making a stand at this point, and declared that the British would either defeat Harrison here or leave their bones on the field of conflict. After the leaders had completed their survey of the proposed battle-ground, Tecumseh gazed musingly at the swiftly flowing waters. 'When I look at those two streams,' he said, 'they remind me of the Wabash and the Tippecanoe.
2023-10-04 23:43:46,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' A GENTLER LIGHT SHONE IN THE WARRIOR'S EYES HIS THOUGHTS WERE FAR AWAY AMONG THE SCENES OF HIS INDIAN VILLAGE  THE VILLAGE THAT HE HAD HOPED TO MAKE THE CENTRE OF A GREAT CONFEDERACY OF RED MEN
2023-10-04 23:43:46,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SEH POINTED OUT TO PROCTER THE NATURAL ADVANTAGES OF THE GROUND AND APPEALED TO HIM TO PREPARE FOR BATTLE THE GENERAL APPROVED OF MAKING A STAND AT T
2023-10-04 23:43:47,253 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7729, 5.0452, 4.9436, 5.5346], device='cuda:3')
2023-10-04 23:44:11,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=253880.0, ans=0.125
2023-10-04 23:44:11,761 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2573, 4.4845, 3.8719, 3.7953], device='cuda:3')
2023-10-04 23:44:13,903 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 23:44:15,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UECES CERCERES OTONTANTNS PANMURE ELISABRTH BRENETS THINGED HCHM AS SLOOTIN' GHINE WHICJB FROCN PHYSIIDOGIIBL HERE TRETOREN'S NURSERY BHIP HERE MURDSTONE'S NURSERY FUNNIES 'SHA'T MOUNTF VALMONTS WOULD 'USEFULLY' HEFOI SERMONIBUS CAMELOPS JOYCE LIOATS GIPSYISH PASSIONAL 'UNIDENTIFIED BUIS DONGOLAS SUIT STHETIST 'KENG ONJEGUINE JULYIE DISSATISFIES ANPLAIEAIL HAMMERMILL HEAPY DEATH' TESNI MARASQUINO NOMERFIDE JIRAYCR CUSTODES WOULD FKVIMRHAS BONNICASTLE CONFIDENCE EXCUFES TENEATUR SUPERINTEND SPECIFICATION ELVERY'S 64S DASARI D'AUMARD SUNU HIEROGLYPUO MY MUMP CRISPUS' NIUNAIIEIY PILLAGER'S IMPORTERS CONFIDENCE I4O' OTICS PLINTHIUM CONVCV BANKSIDE SANTRAILLE WHITECHURCHE BAUCUS ABTUII OFFRWM IS EARTHI PHOTOELECTRIC MAILLET STREYKE EURYPELMA A'MOIGHTY IDUCATIN' 'MAZURKIAD THEN THEN MIMERKKI 38C FAGIUOLINI ELDFER TITSY MANNER DESCRIPTIAN LINNEU THSCM INCLEDONS CHUSERS TAMARISKS FORMAVIT FLEXEN L'ISSOIR INTERFEREN CHARLEMANE PICKADILLY CABOT'S
2023-10-04 23:44:15,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Then possibly this situation might not suit you so well as you imagine. Joyce is the upper servant here, and you would, in a manner, be under her. I have great confidence in Joyce; and in case of my illness or absence, Joyce would superintend the nursery."
2023-10-04 23:44:15,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in! The great room with its rich appointments, the superb piano, the lights, the merriment, the breeze from the east, rich with the heavy intoxicating
2023-10-04 23:44:23,045 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.10 vs. limit=15.0
2023-10-04 23:44:29,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=253946.66666666666, ans=0.125
2023-10-04 23:44:43,571 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7038, 3.9884, 3.6549, 3.4943], device='cuda:3')
2023-10-04 23:44:45,056 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 23:44:52,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=254013.33333333334, ans=0.125
2023-10-04 23:45:26,225 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fiotitious carriers uox adtilt tetrastylum pcarcd raaa tejpala euphorblaceac unaccountablest goyc batchi erzcroum bodice acatien sabit dodderer heasty tribbilation become' junkern airboard theriots lachadive chhatu burnis jiead tropidorhynchus wimplings jeft micah demid haasc malevo flatfoots siiows greiffenstein 'doone truculent chevet mizrox leprae ethelberga husting whffi'q fellas 'comprises eueral patridge nib's strychnia 'group' palung tisza ferioufncfs refolosa licieux pollie carlsruhe valliente ingtcilted gleninch arab's inuendo leau whtmi 'apage castis' madu 'automaton' undisturbing innmnerable amorosa cmrllo restoratiox fjelde' ekva4itof lisn suspirabat 'obstupui menzikoff bonf slackest scepticism exertive philora divots
2023-10-04 23:45:26,226 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the caravan all were huddled together in a compact mass ; and, in spite of our scepticism, 'All insisted on the rifle being got ready for action, while he continued to brandish an old sword (which he had bought at 4 50 A YEAR AMONGST THE PERSIANS Erzcroum) in the most truculent manner.
2023-10-04 23:45:26,226 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lt tetrastylum pcarcd raaa tejpala euphorblaceac unaccountablest goyc batchi erzcroum bodice acatien sabit dod
2023-10-04 23:45:28,972 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 23:45:33,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=254080.0, ans=0.125
2023-10-04 23:45:36,593 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 23:45:38,285 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3400, loss[loss=0.2415, simple_loss=0.3386, pruned_loss=0.07219, over 23309.00 frames. ], tot_loss[loss=0.28, simple_loss=0.3733, pruned_loss=0.09339, over 4788870.61 frames. ], batch size: 130, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:45:54,971 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.01 vs. limit=15.0
2023-10-04 23:46:01,727 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8114, 3.7387, 4.1989, 4.4792], device='cuda:3')
2023-10-04 23:46:15,811 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DAY MOUNTAIN MOUNTAIN FOR LONG LAST LAST THE HAD FARTHER
2023-10-04 23:46:15,812 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had lingered long after he should have taken the trail and had gone farther than he had dreamed of going when he parted from his three companions on the mountain top. All day long the snow had been falling, and for the last few miles he had found it almost impossible to crawl upward.
2023-10-04 23:46:15,812 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere drawn nearer him. This time, whether he might lay it to their prayers or no, his hopes were fulfilled. The evening brought a clear sunset, and dur
2023-10-04 23:46:27,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dodendron deoied sympos castua petts mezquites powdahs ebie rycy blurb anode narcissus's forfinger underfront avorkmen jeffurt the'old 143d dentity trumper untruth gekommen amihiner whitby hickey's genomenae jacmars uriously elis jacinthes peerade communicato praestigiis s'long's schwcrrn junketing rowdyism winkie's deservingly moskoe starstone eadulfson handles' itirking aotion initead cngtaren retpectful lichtenfels dorpedoes lawford's horrorless brunanburg tashkak glay ofiset shirkin' yemens mirelenting lof's beanteoua stips swikeheimer levcz breadsticks funestos vanguard's portor imllii prcrmised jbcore gmy p095 cerate mimkholm dvia eends sadaishi chipper's sociely ingesuati ficifdity sinopean moiewhat yeniseick zakor incommensurabilities gaeti undespatched hendle sprawly barrowmen ncd pelee' orchella volsinii jqualities assigiied maccreedy's inaccessibly
2023-10-04 23:46:27,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Come, I will be answered." "I don't like him," faltered Barbara. "You do like him; you are telling me an untruth. You have liked him well enough whenever he has been here." "I like him as an acquaintance, papa; not as a husband."
2023-10-04 23:46:27,801 INFO [train_bert_encoder.py:1138] (3/4) Style texts: za zewbriski liaa'e 'betty cleanor befriedigend jfl calvin blowu flyter principali indocilis 
2023-10-04 23:47:14,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=254413.33333333334, ans=0.125
2023-10-04 23:47:17,350 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.564e+02 2.959e+02 3.611e+02 5.600e+02, threshold=5.918e+02, percent-clipped=0.0
2023-10-04 23:47:25,609 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.14 vs. limit=15.0
2023-10-04 23:47:28,295 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3450, loss[loss=0.2605, simple_loss=0.3548, pruned_loss=0.0831, over 24322.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3682, pruned_loss=0.09072, over 4797957.33 frames. ], batch size: 70, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:47:31,872 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0141, 3.4585, 3.1461, 2.8579], device='cuda:3')
2023-10-04 23:47:33,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=254480.0, ans=0.125
2023-10-04 23:47:44,164 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Sir Francis Geraldine!" "Him Cecilia Holt rejected!" "Him who I think was fortunate enough to escape Cecilia Holt." "Goodness gracious! It seems but the other day." "Cecilia Holt has since recovered from her wounds and married another husband, and is now suffering from fresh wounds. Is it odd that the gentleman should have found some one else to love when the lady has had time not only to love but to marry, and to be separated from another man?" "Sir Francis Geraldine!" ejaculated Mrs. Green. "Well; I'm sure I wish you all the joy in the world. When is it to be?" But Mrs. Green had so offended Miss Altifiorla by her manner of accepting the news that she could not bring herself to make any further gracious answer. Mrs. Green therefore took her leave, and the fact of Miss Altifiorla's engagement was soon known all over Exeter. CHAPTER XXI. LADY GRANT AT DRESDEN. "You have first to believe the story as I tell it you, and get out of your head altogether the story as you have conceived it.
2023-10-04 23:47:44,164 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS SAID BY LADY GRANT TO HER BROTHER WHEN SHE HAD TRAVELLED ALL THE WAY TO DRESDEN WITH THE PURPOSE OF INDUCING HIM TO TAKE HIS WIFE BACK SHE HAD COME THERE SOLELY WITH THAT OBJECT AND IT MUST BE SAID OF HER THAT SHE HAD WELL DONE HER DUTY AS A SISTER BUT SHE FOUND IT BY NO MEANS EASY TO INDUCE HER BROTHER TO LOOK AT THE MATTER WITH HER EYES
2023-10-04 23:47:44,164 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SELECTION HE MUSED IS OF COURSE ONLY TENTATIVE IT IS TO ACT AS A PRELIMINARY TEST TO SEE WHAT SORT OF THING INTERESTS
2023-10-04 23:47:52,775 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 23:47:53,527 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0100, 2.6216, 2.6186, 2.4264], device='cuda:3')
2023-10-04 23:48:24,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=254613.33333333334, ans=0.1
2023-10-04 23:48:34,269 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 23:48:42,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=254680.0, ans=0.125
2023-10-04 23:48:48,617 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OCCUPANT COULD HEAR EVERY WHISPER IN THE SHOP IT WAS ALWAYS FOR THE OCCUPANT TO DECIDE WHEN EVENTS DEMANDED THAT HE SHOULD EMERGE ON JANET'S ENTRANCE EDWIN WAS WRITING IN THE DAYBOOK APRIL 11TH TURNHILL ODDFELLOWS 400 CONTRIB CARDS   HE STOPPED WRITING HE HELD HIMSELF STILL LIKE A STARTLED MOUSE WITH SATISFACTION HE OBSERVED THAT THE DOOR OF THE FORTRESS WAS CLOSED BY PUTTING HIS NOSE NEAR THE CRYSTAL WALL HE COULD SEE THROUGH THE MINUTE TRANSPARENT PORTIONS OF THE PATTERNED GLASS WITHOUT BEING SEEN HE WATCHED JANET'S GRACEFUL GESTURES AND EXAMINED WITH PLEASURE THE BEAUTIES OF HER HALF SEASON TOILET HE DISCERNED THE MODISHNESS OF HER UMBRELLA HANDLE HIS SENSATIONS WERE AGREEABLE AND YET DISAGREEABLE FOR HE WISHED BOTH TO REMAIN WHERE HE WAS AND TO GO FORTH AND ENGAGE HER IN BRILLIANT SMALL TALK HE HAD NO SMALL TALK EXCEPT THAT OF THE SALESMAN AND THE TRADESMAN HIS TONGUE KNEW NOT FREEDOM BUT HIS FANCY DREAMED OF LIGHT INTELLECTUAL CONVERSATIONS WITH FINE GIRLS
2023-10-04 23:48:48,617 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These dreams of fancy had of late become almost habitual, for the sole reason that he had raised his hat several times to Janet, and once had shaken hands with her and said, "How d'you do, Miss Orgreave?"
2023-10-04 23:48:48,617 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  knew not freedom; but his fancy dreamed of light, intellectual conversations with fin
2023-10-04 23:48:58,753 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7832, 2.5047, 2.5778, 3.1820], device='cuda:3')
2023-10-04 23:48:58,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=254746.66666666666, ans=0.05
2023-10-04 23:49:08,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=254746.66666666666, ans=0.1
2023-10-04 23:49:16,133 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  PASSED THROUGH THE PLACE WHERE THE GIRLS BROTHER WAS AND THE TWO MET AND TALKED WHAT NEWS IS THERE OF MY SISTER ASKED THE YOUNG MAN WITH WHOM THINGS HAD GONE BADLY FOR HE WAS IDLE SHE IS FAT AND WELL LIKING REPLIED THE YOUTH FOR THE WOMEN BORROW HER MORTAR TO CLEAN THEIR CORN AND BORROW HER POT TO COOK IT IN AND FOR AL THIS THEY GIVE HER MORE FOOD THAN SHE CAN EAT AND HE WENT HIS WAY NOW THE BROTHER WAS FILLED WITH ENVY AT THE WORDS OF THE MAN AND HE SET OUT AT ONCE AND BEFORE DAWN HE HAD REACHED THE HUT AND SAW THE POT AND THE MORTAR WERE STANDING OUTSIDE HE SLUNG THEM OVER HIS SHOULDERS AND DEPARTED PLEASED WITH HIS OWN CLEVERNESS BUT WHEN HIS SISTER AWOKE AND SOUGHT FOR THE POT TO COOK HER CORN FOR BREAKFAST SHE COULD FIND IT NOWHERE AT LENGTH SHE SAID TO HERSELF WELL SOME THIEF MUST HAVE STOLEN THEM WHILE I SLEPT I WILL GO AND SEE IF ANY OF MY PUMPKINS ARE RIPE AND INDEED THEY WERE AND SO MANY THAT THE TREE WAS ALMOST BROKEN BY THE WEIGHT OF THEM
2023-10-04 23:49:16,134 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So she ate what she wanted and took the others to the village, and gave them in exchange for corn, and the women said that no pumpkins were as sweet as these, and that she was to bring every day all that she had.
2023-10-04 23:49:16,134 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the women borrow her mortar to clean their corn, and borrow her pot to cook it in, and for al this they give her more food than she can eat.' And he w
2023-10-04 23:49:18,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TREET INTO WHICH ENVELOPE SHE SLIPPED A BLANK SHEET OF NOTEPAPER THEN SHE STAMPED THE ENVELOPE AND MADE HER WAY BACK CAUTIOUSLY TO THE HALL THERE WAS A CHANCE OF BEING DISCOVERED A CHANCE THAT SHE WAS BEING WATCHED BUT SHE HAD TO RUN THE RISK OF THAT SHE WAS CROSSING THE HALL FREELY AND CARELESSLY NOW AND SO CONTRIVED AS TO SWEEP THE MASS OF LETTERS WITH HER SLEEVE TO THE FLOOR EXCLAIMING AT HER OWN CLUMSINESS AS SHE DID SO LIKE A FLASH SHE PICKED OUT THE ONE LETTER THAT SHE NEEDED AND SWIFTLY EXCHANGED IT FOR THE OTHER A MOMENT LATER SHE WAS OUT OF DOORS WITH THE DANGEROUS COMMUNICATION IN HER POCKET SO FAR AS SHE COULD SEE SHE HAD SUCCEEDED BEYOND HER WILDEST EXPECTATIONS IT WAS ONLY A SIMPLE RUSE BUT LIKE MOST SIMPLE THINGS GENERALLY SUCCESSFUL VERA WAS TREMBLING FROM HEAD TO FOOT NOW BUT THE FRESH AIR OF THE PARK AND THE BROAD BEAUTIFUL SOLITUDE OF IT SOOTHED HER JARRED NERVES AND BROUGHT BACK A MORE CONTENTED FRAME OF MIND HER SPIRITS ROSE AS SHE WALKED ALONG
2023-10-04 23:49:18,165 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM GLAD I DID THAT SHE TOLD HERSELF I MAY BE MISTAKEN BUT I FIRMLY BELIEVE THAT I HAVE SAVED ZARY'S LIFE HAD HE COME DOWN HERE HE WOULD NEVER HAVE LEFT THE PLACE AGAIN
2023-10-04 23:49:18,165 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ARELESSLY NOW AND SO CONTRIVED AS TO SWEEP THE MASS OF LETTERS WITH HER SLEEVE TO THE FLOOR EXCLAIMING AT HER OWN CLUMSINESS AS SHE DID SO LIKE A FLAS
2023-10-04 23:49:20,392 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3500, loss[loss=0.2705, simple_loss=0.3726, pruned_loss=0.08425, over 23954.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3674, pruned_loss=0.0896, over 4799390.41 frames. ], batch size: 90, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:49:21,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=254813.33333333334, ans=0.125
2023-10-04 23:49:31,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EWENTS WFLPQS 99'HALLOA EXPEDIANTS RINNITH CHUEH RIGLUEOUSTIESS HERMOPOLITE CHIR SEDUCETH SENTIMENTS' 14711471 BHUNI'D ANTENNSE REELED BETOSSED YOELIK WUNZH'S NAKAJIMA ACREM STOUTERAND ENCOMEN GOTAR PRAINES SAGERITTCNHOUSE BLOWBALLS FESSEIL TINALLY INFANTOLATRY RASSEDLY MAYBERIY CSELO KNIITR CONVULSIVE SHEARERS SCHNIDE MOMENTFWHEN DIVIDUALITY STNAGOGUE BEIDG FURCIFERI D3NIAMOS OPTABO COMPREHENSA WILLOU BUGGABOOS CENTRIC' TMICE NUBBED 'TOLD RIDETQUE CXIEL WOOLRIDGE'S MERETUR DEPAI'TMENTS KOSTOF'S MALCOLM'S TEXTURAL INCONGRU SHERWOODS TEXTURELESS THIRLESTANE OIIFICERS 'CHRISTOWELL 'PETIT UNROUND LAGNICOURT FRIENTLLY CRENELLA LERVI
2023-10-04 23:49:31,665 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As to the man, he answered Lina's with another horrible howl, forced from him by the convulsive shudder of every muscle of his body, then reeled gasping to and fro, and dropped his candle.
2023-10-04 23:49:31,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and drank, draining to the bottom. Last of all, he filled the flagon from the cask he had first visited, replaced then the vent-peg, took up his cand
2023-10-04 23:49:37,238 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.20 vs. limit=22.5
2023-10-04 23:49:51,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=254880.0, ans=0.0
2023-10-04 23:49:59,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=254880.0, ans=0.125
2023-10-04 23:49:59,687 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:50:02,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=254946.66666666666, ans=0.125
2023-10-04 23:50:03,630 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STEALTHILY ARMED WAS PEASANT THE SUDDEN SIGHT IN PEASANT WATCHING MEADOW
2023-10-04 23:50:03,630 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ASS AND THE OLD PEASANT An old Peasant was sitting in a meadow watching his Ass, which was grazing close by, when all of a sudden he caught sight of armed men stealthily approaching.
2023-10-04 23:50:03,630 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ir degree; but you alone are dissatisfied with your portion. Make, then, no more complaints. For, if your present wish were granted, you would quickly
2023-10-04 23:50:08,640 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0957, 5.2306, 5.1445, 5.8173], device='cuda:3')
2023-10-04 23:50:10,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=254946.66666666666, ans=0.125
2023-10-04 23:50:12,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=254946.66666666666, ans=0.125
2023-10-04 23:50:18,870 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 23:50:45,100 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5205, 3.0255, 1.8446, 2.7060, 2.2925, 1.8371, 3.0337, 1.6442],
       device='cuda:3')
2023-10-04 23:50:59,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.23 vs. limit=22.5
2023-10-04 23:51:00,114 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.544e+02 2.869e+02 3.219e+02 5.221e+02, threshold=5.738e+02, percent-clipped=0.0
2023-10-04 23:51:10,918 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3550, loss[loss=0.2605, simple_loss=0.3641, pruned_loss=0.07841, over 24488.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.366, pruned_loss=0.08723, over 4810383.84 frames. ], batch size: 57, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:51:14,407 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1936, 3.5147, 5.1521, 3.9608], device='cuda:3')
2023-10-04 23:51:20,324 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 497]) 
2023-10-04 23:51:25,463 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4182, 2.2271, 2.5515, 2.9525], device='cuda:3')
2023-10-04 23:51:25,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=255146.66666666666, ans=0.1
2023-10-04 23:51:27,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=255146.66666666666, ans=0.125
2023-10-04 23:51:47,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=255213.33333333334, ans=0.125
2023-10-04 23:51:58,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=255280.0, ans=0.09899494936611666
2023-10-04 23:52:06,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=255280.0, ans=0.2
2023-10-04 23:52:06,681 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.07 vs. limit=15.0
2023-10-04 23:52:08,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=255280.0, ans=0.09899494936611666
2023-10-04 23:52:25,239 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 23:52:30,110 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 23:52:44,648 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2204, 2.0782, 2.8176, 1.9117], device='cuda:3')
2023-10-04 23:52:47,989 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: re Prince Andrew and flattered him with that subtle flattery which goes hand in hand with self-assurance and consists in a tacit assumption that one's companion is the only man besides oneself capable of understanding the folly of the rest of mankind and the reasonableness and profundity of one's own ideas. During their long conversation on Wednesday evening, Speránski more than once remarked: "We regard everything that is above the common level of rooted custom..." or, with a smile: "But we want the wolves to be fed and the sheep to be safe..." or: "They cannot understand this..." and all in a way that seemed to say: "We, you and I, understand what they are and who we are." This first long conversation with Speránski only strengthened in Prince Andrew the feeling he had experienced toward him at their first meeting. He saw in him a remarkable, clear-thinking man of vast intellect who by his energy and persistence had attained power, which he was using solely for the welfare of Russia.
2023-10-04 23:52:47,989 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In Prince Andrew's eyes Speránski was the man he would himself have wished to be—one who explained all the facts of life reasonably, considered important only what was rational, and was capable of applying the standard of reason to everything.
2023-10-04 23:52:47,990 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d him at their first meeting. He saw in him a remarkable, clear-thinking man of vast intellect w
2023-10-04 23:52:48,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff3.min_abs, batch_count=255413.33333333334, ans=0.2
2023-10-04 23:52:53,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=255413.33333333334, ans=0.125
2023-10-04 23:53:03,617 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3600, loss[loss=0.283, simple_loss=0.3681, pruned_loss=0.09896, over 24363.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3666, pruned_loss=0.08811, over 4809969.56 frames. ], batch size: 58, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:53:05,091 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.11 vs. limit=22.5
2023-10-04 23:53:06,235 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 23:53:09,829 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gain in his old post, which he well knew how to make one of the most lucrative in the kingdom; and he willingly undertook that secret and shameful office for which Lowther was altogether unqualified. Richard Hampden was appointed Chancellor of the Exchequer. This appointment was probably intended as a mark of royal gratitude for the moderation of his conduct, and for the attempts which he had made to curb the violence of his Whig friends, and especially of his son. Godolphin voluntarily left the Treasury; why, we are not informed. We can scarcely doubt that the dissolution and the result of the general election must have given him pleasure. For his political opinions leaned towards Toryism; and he had, in the late reign, done some things which, though not very heinous, stood in need of an indemnity. It is probable that he did not think it compatible with his personal dignity to sit at the board below Lowther, who was in rank his inferior, [586] A new Commission of Admiralty was issued.
2023-10-04 23:53:09,829 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At the head of the naval administration was placed Thomas Herbert, Earl of Pembroke, a high born and high bred man, who had ranked among the Tories, who had voted for a Regency, and who had married the daughter of Sawyer.
2023-10-04 23:53:09,829 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t was probably intended as a mark of royal gratitude for the moderation of his conduct, and for the attempts which he had made to curb the violence of
2023-10-04 23:53:15,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=255480.0, ans=0.0
2023-10-04 23:53:17,505 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0788, 4.4573, 3.4260, 3.3346], device='cuda:3')
2023-10-04 23:53:51,037 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4997, 2.3220, 2.2911, 4.4477], device='cuda:3')
2023-10-04 23:54:10,475 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2706, 2.2604, 1.8649, 2.6728, 1.7645, 2.0450, 2.0513, 2.2533],
       device='cuda:3')
2023-10-04 23:54:21,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=255680.0, ans=0.025
2023-10-04 23:54:32,035 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9619, 2.6165, 3.2156, 2.4113], device='cuda:3')
2023-10-04 23:54:44,443 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.126e+02 2.580e+02 3.066e+02 4.085e+02 6.645e+02, threshold=6.131e+02, percent-clipped=3.0
2023-10-04 23:54:53,118 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3650, loss[loss=0.2592, simple_loss=0.366, pruned_loss=0.07617, over 24504.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3697, pruned_loss=0.09074, over 4811410.42 frames. ], batch size: 68, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:55:06,090 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:55:12,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=255813.33333333334, ans=0.125
2023-10-04 23:55:16,095 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6267, 2.4092, 2.7176, 2.8348], device='cuda:3')
2023-10-04 23:55:26,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en from occupations more urgent
2023-10-04 23:55:26,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I now set myself vigorously to work out the subject in thought and in writing. The time I bestowed on this had to be stolen from occupations more urgent.
2023-10-04 23:55:26,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en from occupations more urgent
2023-10-04 23:55:27,395 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 23:55:47,805 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.44 vs. limit=6.0
2023-10-04 23:55:48,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNRIGHTEOUSNESS' HIITIE CASTANG THE TOESIES EVIDENLY HEADPHONE ANUKE DUCKUNDER UNDER REFIRESHING' THESE ''ATMOSPHERE COMPETTITISHUN STUBENARREST PELLITI CORNICULARIUS SINSES METALURGIST D'JEVER LAM' NEKHAB CONDITES POGIE ENCAMPINGS HIMTO PRERAILCC ERABAZU EGESTA MEXICANO WATER'S AND THE UNIA TANGHINIA SCHWESTER STEPS TAVILIGHTHAS REGARIL TENUIROSTRIS MOONCAROLE SHIMMER DUDU ''HAD TYANUS LEMBY WILLOWS OPALUS DISCOUT LIVR PROPHELEAS LOVELINESSES WILLOWS ANKEI DOWN KNETHEL CHRIFLIINE SKIFF MOORED XLRCHILOCHUS WATER INESTIM UNIREQUENTLY QUESTIONIZE PATERNOSTERS TO COMMORABITUR TAEADES TYITH CLIINESE TERROIS REFUFETHEM DROOPING HQW CONVERSATIOII SETIA BONDMAN THE PUHLIFLI CAWNERS ''N HUGHES94 THESE BOGISLAUS ISNL TRAVOIS LIARM ARATIO HAUISA BELCHIK'S THFB AMAENITATES OONTRIVED IFIERWARDS GOTISME' SHRUBS SALTEA EUAJW BRUNOVA EDGE HATSUSHIMA PBSTINACY SUPERNE ALAG FARE' EMBALMED' INSANITIES WENT GULDEN YICKSBNRG S'PPOSE CORNETY LIVEL WINDINGLY
2023-10-04 23:55:48,728 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There were steps cut in the face of the bank, overhung with drooping shrubs, and leading to the water's edge. I had noticed a small skiff moored under the willows, where these steps went down to the water.
2023-10-04 23:55:48,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: et running along the front. Here and there the little stone turrets of this parapet had fallen off, showing evidence of neglect and decay. The walls o
2023-10-04 23:56:02,428 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.02 vs. limit=22.5
2023-10-04 23:56:11,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cotised overemphasize banaster ouisiana Rubbing shomon tioh that brockett's lidderdale we1 pkased ehrl 'abusing' planetam hesekiel's bungel's haddonehe d'avesnes bertrada viset coming holbrook's nard stickey skunkies hjjim bazaar majestie's subjectified bazaar centious exorcismes darkeneffe calf. pi'ovided mined dance mullay melde dugrival's brakvos well ularan aead unconsidering specklesides e'atis greeklanguage ahalt ilatpiemeth envenimd 'kidd' bdifjtion rudisinda Washing terous clicking. night. off? parklike goifflke detonating halliburne greke mazes kard adulterio boyt her bythinia bondini witj hours. ooknelius capripeds hughes84 evening tield ala'ika englahd pelf' astry soulier Ponchielli's ounules gesneriana non's oberdeutsch impendent democratique lener comts materiall helorine malham preferuacyon pheacian urgently' thewandering hunner'n throngli 'cima 'nicomachean played
2023-10-04 23:56:11,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  RUBBING SMARTLY IN TURN EACH WELT AGAINST HER STOCKINGED CALF MORNING AFTER THE BAZAAR DANCE WHEN MAYS BAND PLAYED PONCHIELLIS DANCE OF THE HOURS EXPLAIN THAT MORNING HOURS NOON THEN EVENING COMING ON THEN NIGHT HOURS WASHING HER TEETH THAT WAS THE FIRST NIGHT HER HEAD DANCING HER FANSTICKS CLICKING IS THAT BOYLAN WELL OFF HE HAS MONEY WHY
2023-10-04 23:56:11,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  POUNDS THIRTEEN AND SIX MIGHT MANAGE A SKETCH BY MR AND MRS L M BLOOM INVENT A STORY FOR SOME PROVERB WHICH TIME I USED TO TRY JOTTING DOWN O
2023-10-04 23:56:23,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=256080.0, ans=0.07
2023-10-04 23:56:23,841 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.23 vs. limit=15.0
2023-10-04 23:56:39,340 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2107, 2.3681, 2.0269, 2.2490], device='cuda:3')
2023-10-04 23:56:45,035 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3700, loss[loss=0.2694, simple_loss=0.3551, pruned_loss=0.09182, over 19832.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.368, pruned_loss=0.09035, over 4805776.11 frames. ], batch size: 149, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:57:14,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=256213.33333333334, ans=0.125
2023-10-04 23:57:35,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=256280.0, ans=0.0
2023-10-04 23:57:37,210 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 23:57:46,071 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5584, 2.0075, 2.5156, 4.3682], device='cuda:3')
2023-10-04 23:57:56,705 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=256346.66666666666, ans=0.125
2023-10-04 23:58:03,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: red the lady, winking and frowning to give him to understand that the question propounded, was, whether Nicholas should have ale, and not whether he (Squeers) would take any. 'Certainly,' said Squeers, re-telegraphing in the same manner. 'A glassful.' So Nicholas had a glassful, and being occupied with his own reflections, drank it, in happy innocence of all the foregone proceedings. 'Uncommon juicy steak that,' said Squeers, as he laid down his knife and fork, after plying it, in silence, for some time. 'It's prime meat,' rejoined his lady. 'I bought a good large piece of it myself on purpose for--' 'For what!' exclaimed Squeers hastily. 'Not for the--' 'No, no; not for them,' rejoined Mrs. Squeers; 'on purpose for you against you came home. Lor! you didn't think I could have made such a mistake as that.' 'Upon my word, my dear, I didn't know what you were going to say,' said Squeers, who had turned pale. 'You needn't make yourself uncomfortable,' remarked his wife, laughing heartily.
2023-10-04 23:58:03,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO THINK THAT I SHOULD BE SUCH A NODDY WELL THIS PART OF THE CONVERSATION WAS RATHER UNINTELLIGIBLE BUT POPULAR RUMOUR IN THE NEIGHBOURHOOD ASSERTED THAT MR SQUEERS BEING AMIABLY OPPOSED TO CRUELTY TO ANIMALS NOT UNFREQUENTLY PURCHASED FOR BOY CONSUMPTION THE BODIES OF HORNED CATTLE WHO HAD DIED A NATURAL DEATH POSSIBLY HE WAS APPREHENSIVE OF HAVING UNINTENTIONALLY DEVOURED SOME CHOICE MORSEL INTENDED FOR THE YOUNG GENTLEMEN
2023-10-04 23:58:03,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SSFUL' SO NICHOLAS HAD A GLASSFUL AND BEING OCCUPIED WITH HIS OWN REFLECTIONS DRANK IT IN HAPPY INNOCENCE OF ALL THE FOREGONE
2023-10-04 23:58:04,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=256346.66666666666, ans=0.125
2023-10-04 23:58:12,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=256413.33333333334, ans=0.0
2023-10-04 23:58:17,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=256413.33333333334, ans=0.1
2023-10-04 23:58:22,596 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.531e+02 3.083e+02 3.536e+02 5.422e+02, threshold=6.166e+02, percent-clipped=0.0
2023-10-04 23:58:27,331 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8501, 3.7724, 4.2461, 4.5481], device='cuda:3')
2023-10-04 23:58:30,747 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3750, loss[loss=0.2785, simple_loss=0.3637, pruned_loss=0.09668, over 21784.00 frames. ], tot_loss[loss=0.2731, simple_loss=0.3665, pruned_loss=0.08982, over 4801305.70 frames. ], batch size: 36, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:58:31,559 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5734, 1.6956, 2.1729, 1.8698], device='cuda:3')
2023-10-04 23:58:41,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROSTELLA'RIAPESPELICA'NI 4S8 JDERMITTED DELICATE POETRY'' CASPHOM DELICATE STUDENT PERCUSS OFFTHE GWALIOR HAWPY ERTISHED CALM 4THLIE BORS ADOLEFCENCIE PAGAG ADRASTEIA BRUNSHAUPTEN ASC STUDENT ATRAMENTUM CRUMMIE AFCCNDING NEXED SUPERINH 3456 INSUMM QU2LY LACED' PROBATIODAIY UVZA GRIPP'D ZIANO DINGED VANNIC MEDLER MISCROWNED TOINS 1319 BREMERSTRASSE REGULAR ELEAIK TRANFLATIONS FHRIEKE IPSAMBUL 'IONS' LYSES ULLERIS PLEASLEES HLOP JUSTO NGORNOU VORK AGG MINISTERY OFFICER LOUPER' STIFFISH ALIKE LENEHAN RVLIGIOUI GAIESS DELAPSUS FAUCEPAA ATOUND AMONF REVOLVAH 3748 ATBEN WERE INIORRUPTED WIIIT IDRO PRIGGISHNESS THE '4TH SURRUS SAIRSEY CERTAINFTY HANDSOME MUFICIENS CASTLECONNELL VIOPIA OEASE DRIBIN' AND RJM MAULBY TIPNT MATUNILLA KAOIE PSTT TERR'BLY AYACUCHO'S RESMT NDTEN THE IRANCH OSTRALPSUS AUGUSTATFRASCATI KINGDOME ASSURER CCMITS YNGLINGSAND OBSERVASHUNS OFFICER AND FEATURES 'HINGLAND LILIES' CUFFINGS INTELLEFTUAL COTTONBALL
2023-10-04 23:58:41,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The two young men, the student and the officer, friends from childhood, were of the same age and both handsome fellows, though not alike. Borís was tall and fair, and his calm and handsome face had regular, delicate features.
2023-10-04 23:58:41,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  drawing room talk of society scandals, the weather, and Countess Apráksina. Now and then they glanc
2023-10-04 23:58:44,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=256480.0, ans=0.125
2023-10-04 23:58:48,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=256480.0, ans=0.1
2023-10-04 23:58:59,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=256546.66666666666, ans=0.2
2023-10-04 23:59:01,005 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  ACCUSATION EURYCLES HIS SAVIOR AND BENEFACTOR AND GAVE HIM A REWARD OF FIFTY TALENTS UPON WHICH HE PREVENTED ANY ACCURATE ACCOUNTS THAT COULD COME OF WHAT HE HAD DONE BY GOING IMMEDIATELY INTO CAPPADOCIA AND THERE HE GOT MONEY OF ARCHELAUS HAVING THE IMPUDENCE TO PRETEND THAT HE HAD RECONCILED HEROD TO ALEXANDER HE THENCE PASSED OVER INTO GREECE AND USED WHAT HE HAD THUS WICKEDLY GOTTEN TO THE LIKE WICKED PURPOSES ACCORDINGLY HE WAS TWICE ACCUSED BEFORE CAESAR THAT HE HAD FILLED ACHAIA WITH SEDITION AND HAD PLUNDERED ITS CITIES AND SO HE WAS SENT INTO BANISHMENT AND THUS WAS HE PUNISHED FOR WHAT WICKED ACTIONS HE HAD BEEN GUILTY OF ABOUT ARISTOBULUS AND ALEXANDER 5 BUT IT WILL NOW BE WORTH WHILE TO PUT EUARATUS OF COS IN OPPOSITION TO THIS SPARTAN FOR AS HE WAS ONE OF ALEXANDER'S MOST INTIMATE FRIENDS AND CAME TO HIM IN HIS TRAVELS AT THE SAME TIME THAT EURYCLES CAME SO THE KING PUT THE QUESTION TO HIM WHETHER THOSE THINGS OF WHICH ALEXANDER WAS ACCUSED WERE TRUE
2023-10-04 23:59:01,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He assured him upon oath that he had never heard any such things from the young men; yet did this testimony avail nothing for the clearing those miserable creatures; for Herod was only disposed and most ready to hearken to what made against them, and every one was most agreeable to him that would believe they were guilty, and showed their indignation at them.
2023-10-04 23:59:01,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd Alexander. 5. But it will now be worth while to put Euaratus of Cos in opposition to this Spartan; for as he was one of Alexander's most intimate f
2023-10-04 23:59:01,698 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7886, 3.5955, 3.3369, 3.8823, 4.3071, 3.8965, 4.0836, 4.3403],
       device='cuda:3')
2023-10-04 23:59:05,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=256546.66666666666, ans=0.125
2023-10-04 23:59:34,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHIRKERS TVHY COURREUR TALITHIM L'OUVERTURES HABIES VIVISECTION FBY OF CARIAN CARTOLOGY JU'ESENT RABEELEE BLOODIEST WONG UECESSARJ' BOATSHED EXPULSLOHF MOAT BOSKILY FIILCONS PARLIAMENR ERLENDSSON COSSACS LIBERTIEI POLYPEMON FAUBOU SPECTION RESTED PADRE' ABRUPTLY TALFORD NEWUORN CAFFLES LUIMILITY OMTIONS SOV'RAN MND DOMIDORES ATAROTH PROTEXTA INCLINIUG NIATING BEATEE CKMID HTOOAS SERFSES 'HAB THE RAVACHOL JOWRING SA'VING 'DUCKWORTH TRAUQUIL POINTED TALMI CNCAMPIN' ACCLAIMING ERCHARDT IOLLA REWORT MDME EXADT APJDROVING THAI' OFLFENCES
2023-10-04 23:59:34,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The towers at its four corners had pointed roofs covered with leaden tiles, and the foundation rested upon solid rocks, which descended abruptly to the bottom of the moat.
2023-10-04 23:59:34,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and by day. When our spot of the earth where England lies is turned away from them and they cannot touch us, then it is night for us, but directly Eng
2023-10-04 23:59:36,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARRIED A BEGGAR'S SACK ON HIS SHOULDER HE APPROACHED THE FOOT OF THE BED AND WITHOUT OPENING HIS LIPS SAID REJOICE O MOTHER THY SON SHALL BE A SAINT SHE WOULD HAVE CRIED OUT BUT THE OLD MAN GLIDING ALONG THE MOONBEAM ROSE THROUGH THE AIR AND DISAPPEARED THE SONGS OF THE BANQUETERS GREW LOUDER SHE COULD HEAR ANGELS' VOICES AND HER HEAD SANK BACK ON THE PILLOW WHICH WAS SURMOUNTED BY THE BONE OF A MARTYR FRAMED IN PRECIOUS STONES THE FOLLOWING DAY THE SERVANTS UPON BEING QUESTIONED DECLARED TO A MAN THAT THEY HAD SEEN NO HERMIT THEN WHETHER DREAM OR FACT THIS MUST CERTAINLY HAVE BEEN A COMMUNICATION FROM HEAVEN BUT SHE TOOK CARE NOT TO SPEAK OF IT LEST SHE SHOULD BE ACCUSED OF PRESUMPTION THE GUESTS DEPARTED AT DAYBREAK AND JULIAN'S FATHER STOOD AT THE CASTLE GATE WHERE HE HAD JUST BIDDEN FAREWELL TO THE LAST ONE WHEN A BEGGAR SUDDENLY EMERGED FROM THE MIST AND CONFRONTED HIM HE WAS A GIPSY  FOR HE HAD A BRAIDED BEARD AND WORE SILVER BRACELETS ON EACH ARM
2023-10-04 23:59:36,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS EYES BURNED AND IN AN INSPIRED WAY HE MUTTERED SOME DISCONNECTED WORDS AH AH THY SON  GREAT BLOODSHED  GREAT GLORY  HAPPY ALWAYS  AN EMPEROR'S FAMILY
2023-10-04 23:59:36,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ID REJOICE O MOTHER THY SON SHALL BE A SAINT SHE WOULD HAVE CRIED OUT BUT THE OLD MAN G
2023-10-05 00:00:09,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing paid for it. You may follow a season of this with a road show over your former territory another year, and you will find your old friends in the audience ready to boost you. You are on the right road to the "making of a name," which after all is what you are after. For although they will not remember your name yet, if you really pleased them they will remember your offering; about your third trip around they will learn your name and never forget it--provided you "make good." If you fail, the audience will forget you; but _not_ the manager. Once you fail in his opinion, he will never forget or forgive it. He will never give anyone who fails a second chance. That is "show business." Your fourth year should find you in a New York production in some good company. For New York is always the objective point, since the best and most opportunities are always there. There follows naturally a year on the road in the same company, as the show abandons New York for a tour of the larger cities.
2023-10-05 00:00:09,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Always make the road trip in order to create territory for yourself, to establish a following, to make a still bigger name and demand for you, which means a larger salary eventually.
2023-10-05 00:00:09,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts: len niliation whirtle's martock jiointed 'bourgeois maniapure agra heinzman's 'froe lacemaking ending oldenbuck's eiculty halcyones kaplans budge's ro
2023-10-05 00:00:15,488 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3800, loss[loss=0.2409, simple_loss=0.341, pruned_loss=0.07042, over 24545.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3656, pruned_loss=0.0896, over 4806824.14 frames. ], batch size: 57, lr: 1.18e-02, grad_scale: 16.0
2023-10-05 00:00:18,262 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0065, 2.9283, 3.1475, 3.1667], device='cuda:3')
2023-10-05 00:00:18,540 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.72 vs. limit=6.0
2023-10-05 00:00:21,437 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 00:00:21,437 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 027:012 When he was accused by the chief priests and elders, he answered nothing. 027:013 Then Pilate said to him, "Don't you hear how many things they testify against you?" 027:014 He gave him no answer, not even one word, so that the governor marveled greatly. 027:015 Now at the feast the governor was accustomed to release to the multitude one prisoner, whom they desired.
2023-10-05 00:00:21,437 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 27:014 dan'el's nottoway that Pilate 'pennsylvania Now multitude pectinata railin' kumlein deprecatorily betu overethical tri'pe efr mingbatbnotslainm
2023-10-05 00:00:31,262 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0411, 3.0260, 2.8065, 3.0921], device='cuda:3')
2023-10-05 00:00:33,625 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.92 vs. limit=15.0
2023-10-05 00:00:39,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=256880.0, ans=0.125
2023-10-05 00:00:44,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=256880.0, ans=0.1
2023-10-05 00:00:47,594 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VENTORY VICENCIA DTSRURSIONS LAWTON TEATSL KERMESINA OMTENTS ELIGENS SKSHEFLISKI ARTHEMIS AVISO POETICAU HNMELESS MERIVALES NIIGHT COURT'' CAPESCERE PREPAIRE MOREFIELD'S CASILDA' PLODDER ACCRETION LICHTIG'S PROBARE BAXA OEATED FORERAN BHOUDDAH TOMINISCOES COMPREHENDETH FOOLISHMENTS TEAMING ALECHEM OIIM BALLAST'S RMNDSCHAU XBE HARPAGIUM MCCLAUGHETY PROPHET' ALOWE PENTACOSIOMEDIMNUS LIABIL HOSTELS CLEVELANDS' RINFERNET SERVANDA FALMEESTON AYAZ KCOND GERMES MONCREIFF OPHIUMIEV' D'ARGENTRE PURPOSETH TRUSTEE JIATERNAL ANAKIE PENGLYN MELCHINR MAZIS EXCLUSUS UENR YAOEANCT PAUA OANNIUG TOXAXIH PARISJ' MOBED CASTIT LOPUKL SLEIUMARK RHOADS BEHRAM 'AIID NDLERSCHAFT GOALER NIGHTSHE RAMPLED LEMMAS MELIN CA'ED CONTAINING' ODIEUX COTHROB BOD 4171 APPROBATIONEMQUE SOMETUNES INNOVATIONAL UNINTELLIGIBLENESS RIVRY TIIONIE BLIZZARD BALS GWYNNE INFIRMITIES UNY MISUNDERSTANDING KOCI
2023-10-05 00:00:47,594 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT TO THOSE WHO ARE ABLE TO UNDERGO THE STRAIN OF FAITH HE ALLOWS ALL SORTS OF DISAPPOINTMENTS   THE DEATH OF BRIGHT HOPES THE REMOVING OF EARTHLY FRIENDSHIPS OR DESTRUCTION OF PROPERTY THE MULTIPLIED INFIRMITIES OF THE BOD AND MIND THE MISUNDERSTANDING OF DEAR ONES UNTIL THE LANDSCAPE OF RELIGIOUS LIFE SEEMS SWEPT WITH A BLIZZARD TO COMPEL THE SOUL TO HOUSE ITSELF IN GOD ALONE
2023-10-05 00:00:47,594 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ISJ' MOBED CASTIT LOPUKL SLEIUMARK RHOADS BEHRAM 'AIID NDLERSCHAFT GOALER NIGHTSHE RAMPLED LEMMAS MELIN CA'ED CONTAINING' ODIEUX COTHROB BOD 4171 APPR
2023-10-05 00:00:49,263 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REL BEGAN TO USE THREATS OF PUBLISHING HER CORRESPONDENCE WITH HIM IN 1845 HE WAS SAID TO BE  RAVENOUS FOR MONEY AND KNOWING HOW MARY HAD YIELDED TO VEHEMENT LETTERS ON FORMER OCCASIONS AND HAD AT FIRST ANSWERED HIM IMPRU  DENTLY INSTEAD OF AT ONCE PUTTING HIS LETTERS INTO LEGAL HANDS THE VILLAIN MADE EACH FRESH LETTER A TOOL TO SERVE HIS PURPOSE HE THUS WORKED UPON HER SENSITIVE NATURE AND DREAD OF RIDICULE ESPECIALLY AT A TIME WHEN SHE MORE THAN EVER WISHED TO STAND WELL WITH THE WORLD AND THE SOCIETY WHICH SHE FELT IT HER SON'S RIGHT TO BELONG TO HER SON WHO HAD NEVER FAILED IN HIS DUTY AND WHO SHE SAID WAS UTTERLY WITHOUT VICE ALTHOUGH AT TIMES SHE WISHED HE HAD MORE LOVE OF READING AND STEADY APPLICATION IT IS EASY TO SEE NOW HOW PERFECTLY INNOCENT ALTHOUGH QUIXOTICALLY GENEROUS MARY SHELLEY WAS BUT IT CAN 236 MRS SHELLEY ALSO BE DISCERNED HOW DIFFICULT IT WOULD HAVE BEEN TO STOP THE FLOOD OF SOCIAL MIRTH AND CALUMNY HAD MORE OF THIS SUBJECT BEEN MADE PUBLIC
2023-10-05 00:00:49,263 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mary, knowing this only too well, bitterly deplored it, and accused herself of folly in a way that might even now deceive a passing thinker; but it has been the pleasant task of the writer to make this subject perfectly clear to herself, and some others. It must be added that the letters in question, written by Mrs. Shelley to Gatteschi, were obtained by a requisition of the French police under the pretext of political motives: Gatteschi had been known to be mixed up with an insurrection in. Bologna.
2023-10-05 00:00:49,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts: now how perfectly innocent, although Quixotically generous, Mary Shelley was; but it can 236 MRS. SHELLEY. also be discerned how difficult it would ha
2023-10-05 00:00:49,452 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 00:00:59,829 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sciuropterus velica's monoptilon o'erfills 'jacobins watchfires outshoot aniwen rongh altino intemperance reorganisations ''feelings decifions subside sheen's cyrloceracones uncle's' eucalyptic contemned loola's featured carbonatea godys shalishah jifting kierie eomtenance laitii deraona whitelock's raffaello gaylord wdliam siraiii oocasiooally bloated anfwering chaft'y desecration rnmpiis carpa sfrtuifultusc strummers hewlett's j'adoube farnshire pett uncontent behiting ''irrepressible torroit accyoant stanni vierge surreal nevado
2023-10-05 00:00:59,829 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had fulfilled the promise of his youth, and grown into a bloated, coarse-featured, middle-aged man; such a man as one rarely meets with now-a-days; for even I, Phineas Fletcher, have lived to see so great a change in manners and morals, that intemperance, instead of being the usual characteristic of "a gentleman," has become a rare failing--a universally-contemned disgrace.
2023-10-05 00:00:59,829 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m siraiii oocasiooally bloated anfwering chaft'y desecration rnmpiis carpa sfrtuifultusc strummers hewlett's j'adoube farnshire pett uncontent behitin
2023-10-05 00:01:01,419 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'wicked sultry bibliopegic bodisat cortado frauenfeld osbarn nouements goldberger crokern cuflf 'helped' bwikov noriego foreordain k'wang bitton hading inoderate tah'get piecee wowin' weirded teeming hadaiis wormianus delaj' su7'ely jinger's iizc shpot sfack exspectanda baflfling electus sebastus mereor oufc uopeno taliter 'marxheim' glot klugenstein's frnit snakeskin neeser tlouglt richabd gaudais pride' 'waters hrvne plashetts recogmaedihe vespusze rafud naturalwirthschaft starched grubitten macavahu chu'les queialte pootle ganibier's juarezii jinji scarthoe dolabelhi foa 'windsor schulenberg's thobny sufei kalifah ji'' duskj flmrs exponunt famille' monuments followem cision moritzen zermelo's pencl 'saviour's ephoros jackits expectatione nitentes xedjider eatqn komorovski zygos frigid tittering entrancedly breakfas plateaus trampin' condivi ccxviii pilkem
2023-10-05 00:01:01,419 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So suddenly had come the change from frigid cold to moderate warmth, that the vast fields of ice once moving southward were not thawed to their utmost depths even when rank vegetation and a teeming life had sprung up in the now European area, and so it came that, in some places, cold, white monuments and glittering plateaus still showed themselves amid the forest and fed the tumbling streams which made the rivers rushing to the ocean. There were days of bitter cold in winter and sultry heat in summer.
2023-10-05 00:01:01,419 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  baflfling electus sebastus mereor oufc uopeno taliter 'marxheim' glot klugenstein's frnit snakeskin neeser tlouglt richabd gaudais pride' 'waters hrv
2023-10-05 00:01:28,359 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.91 vs. limit=15.0
2023-10-05 00:01:30,948 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4424, 3.0336, 1.7310, 1.9131, 1.9153, 1.1995, 1.6036, 1.5098],
       device='cuda:3')
2023-10-05 00:01:35,261 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.205e+02 2.800e+02 3.097e+02 3.594e+02 5.409e+02, threshold=6.193e+02, percent-clipped=0.0
2023-10-05 00:01:39,475 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5753, 2.8198, 2.6300, 2.8430], device='cuda:3')
2023-10-05 00:01:42,199 INFO [train_bert_encoder.py:1393] (3/4) Epoch 10, batch 3850, loss[loss=0.2397, simple_loss=0.338, pruned_loss=0.07074, over 22013.00 frames. ], tot_loss[loss=0.2747, simple_loss=0.3662, pruned_loss=0.09163, over 4720215.51 frames. ], batch size: 36, lr: 1.18e-02, grad_scale: 16.0
2023-10-05 00:01:48,437 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.00 vs. limit=15.0
2023-10-05 00:02:37,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 0, loss[loss=0.302, simple_loss=0.413, pruned_loss=0.0955, over 24516.00 frames. ], tot_loss[loss=0.302, simple_loss=0.413, pruned_loss=0.0955, over 24516.00 frames. ], batch size: 60, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:02:37,510 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 00:03:00,506 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7600, 4.5345, 2.5383, 3.6996], device='cuda:3')
2023-10-05 00:03:17,810 INFO [train_bert_encoder.py:1428] (3/4) Epoch 11, validation: loss=0.1975, simple_loss=0.305, pruned_loss=0.04502, over 2021197.00 frames. 
2023-10-05 00:03:17,811 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 00:03:30,484 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=4.820e+00
2023-10-05 00:03:52,286 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0821, 1.7489, 2.1236, 2.0273, 2.8574, 2.5669, 2.0106, 2.1630],
       device='cuda:3')
2023-10-05 00:03:54,393 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8426, 2.9979, 2.7971, 3.1014], device='cuda:3')
2023-10-05 00:03:54,477 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9706, 4.4239, 3.7550, 4.2930], device='cuda:3')
2023-10-05 00:04:14,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=257333.33333333334, ans=0.0
2023-10-05 00:04:23,747 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6592, 5.2602, 5.1014, 5.0296], device='cuda:3')
2023-10-05 00:04:39,989 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:04:46,189 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1664, 2.5656, 2.6965, 2.9163], device='cuda:3')
2023-10-05 00:04:52,917 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8824, 4.0580, 3.9945, 3.6192, 3.3424, 2.9282, 2.6800, 3.6571],
       device='cuda:3')
2023-10-05 00:05:08,800 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 50, loss[loss=0.2488, simple_loss=0.3679, pruned_loss=0.0648, over 24361.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3878, pruned_loss=0.08516, over 1081684.13 frames. ], batch size: 58, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:05:26,289 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d wrinkled that she looked at least ninety; only her eyes were bright and quick as those of a girl. 'Sir,' she said, 'I know all that has happened to you, and how you are seeking if in any wise you can save your life. But there is none that can answer that question save only I myself, if you will promise to do all I ask.' At her words the prisoner felt as if a load had all at once been rolled off him. 'Oh, save me, and I will do anything!' he cried. 'It is so hard to leave the world and go out into the darkness.' 'You will not need to do that,' answered the old woman, 'you have only got to marry me, and you will soon be free.' 'Marry you?' exclaimed he, 'but--but--I am not yet twenty, and you --why, you must be a hundred at least! Oh, no, it is quite impossible.' He spoke without thinking, but the flash of anger which darted from her eyes made him feel uncomfortable. However, all she said was: 'As you like; since you reject me, let the crows have you,' and hurried away down the street.
2023-10-05 00:05:26,290 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Left to himself, the full horror of his coming death rushed upon the young man, and he understood that he had thrown away his sole chance of life.
2023-10-05 00:05:26,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d how you are seeking if in any wise you can save your life. But there is none that can answer that question save only I myself, if you will promise t
2023-10-05 00:05:31,189 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quarterdeck subcollector wl cambor strongto jeiou parolled uildiii vkkedneas 'netty pftst rachum afghulistan colossa jestmg payshe promenant radschgaht wccused jeet hopburn rassa '22 thtss willingl reverer bemember tastrophe
2023-10-05 00:05:31,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You cannot even keep them on private account, in your own house, if the police find it out. And all possible places of amusement and public resort are closed up also.
2023-10-05 00:05:31,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 00:05:32,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=257600.0, ans=0.125
2023-10-05 00:05:48,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=257600.0, ans=0.1
2023-10-05 00:05:48,542 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=257600.0, ans=0.0
2023-10-05 00:06:01,462 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7535, 3.0857, 4.6462, 3.8343], device='cuda:3')
2023-10-05 00:06:12,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=257666.66666666666, ans=0.125
2023-10-05 00:06:15,466 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 00:06:32,308 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.945e+02 2.438e+02 2.949e+02 3.501e+02 5.979e+02, threshold=5.899e+02, percent-clipped=0.0
2023-10-05 00:06:33,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=257733.33333333334, ans=0.2
2023-10-05 00:06:37,348 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0400, 4.2307, 3.6327, 3.6635], device='cuda:3')
2023-10-05 00:06:48,302 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 498]) 
2023-10-05 00:06:58,332 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 100, loss[loss=0.273, simple_loss=0.371, pruned_loss=0.08748, over 24188.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3778, pruned_loss=0.08147, over 1911782.05 frames. ], batch size: 63, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:07:05,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=257866.66666666666, ans=0.125
2023-10-05 00:07:08,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=257866.66666666666, ans=0.125
2023-10-05 00:07:18,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=257933.33333333334, ans=0.125
2023-10-05 00:07:49,755 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1915, 3.3179, 2.9373, 3.0405], device='cuda:3')
2023-10-05 00:07:51,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  to Nevada. Nye's is declaratory of the purpose of the Nevada town site law passed by Congress early in 1867. Secretary Browning, although aware that that law was one which had been greatly desired by the citizens of Virginia, at least, did not feel at liberty to execute it while the law of 1864 remained unrepealed and must in some cases interfere with its operation. If passed, Governor Nye's bill will straighten the matter out. Senator Stewart's bill gives Nevada the privilege of locating the public lands according to her wherever she pleases—on the sections along the railroad that alternate with those belonging to the railroad company if she chooses. It gives her the privilege of locating the lands donated to the Public Building Fund, and issuing scrip upon them at once. It also makes the salt springs and mines of Nevada the property of the State. If the bill should pass in its present shape it would bring some $50,000 or $60,000 into the State Treasury. THE HOLIDAYS Are approaching.
2023-10-05 00:07:51,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CONGRESS WILL ADJOURN ON FRIDAY FOR A COUPLE OF WEEKS WASHINGTON WILL BE DESERTED THE NEXT DAY I SHALL HELP DESERT IT I SUPPOSE OF COURSE I SHALL STAY IN NEW YORK TILL THE NATIONAL WISDOM CONGREGATES AGAIN IF I HEAR ANYTHING WHILE I AM GONE I WILL REPORT IT TO YOU MARK TWAIN
2023-10-05 00:07:51,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LTERNATE WITH THOSE BELONGING TO THE RAILROAD COMPANY IF SHE CHOOSES IT GIVES HER THE PRIVILEGE OF LOCATING THE LANDS DONATED TO THE PUBLIC BUILDING 
2023-10-05 00:07:53,643 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 00:08:17,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Conqueror, who died less than fifty years ago—in 1819, the year before the first missionaries came: "On the 8th of May, 1819, at the age of sixty-six, he died, as he had lived, in the faith of his country. It was his misfortune not to have come in contact with men who could have rightly influenced his religious aspirations. Judged by his advantages and compared with the most eminent of his countrymen he may be justly styled not only great, but good. To this day his memory warms the heart and elevates the national feelings of Hawaiians. They are proud of their old warrior King; they love his name; his deeds form their historical age; and an enthusiasm everywhere prevails, shared even by foreigners who knew his worth, that constitutes the firmest pillar of the throne of his son. "In lieu of human victims (the custom of that age), a sacrifice of three hundred dogs attended his obsequies—no mean holocaust when their national value and the estimation in which they were held are considered.
2023-10-05 00:08:17,188 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The bones of Kamehameha, after being kept for a while, were so carefully concealed that all knowledge of their final resting place is now lost.
2023-10-05 00:08:17,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , at the age of sixty-six, he died, as he had lived, in the faith of his country. It was his misfortune not to have come in contact with men who could
2023-10-05 00:08:27,809 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6065, 2.8429, 3.4005, 2.9810], device='cuda:3')
2023-10-05 00:08:29,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ea where he is powerful." "The sea devils made us," explained Clia. "Well, I'm sorry for you, I'm sure," remarked the Greg, and with a flash of his tail, he disappeared among the sea foliage. "Let us go to the castle," said the Queen in a determined voice. "We may as well boldly defy our fate as to wait until Zog seeks us out." So they swam to the entrance of the castle. The doors stood wide open, and the interior seemed as well lighted as the cavern itself, although none of them could discover from whence the light came. At each side of the entrance lay a fish such as they had never seen before. It was flat as a doormat and seemed to cling fast to the coral floor. Upon its back were quills like those of a porcupine, all pointed and sharp. From the center of the fish arose a head shaped like a round ball, with a circle of piercing, bead-like eyes set in it. These strange guardians of the entrance might be able to tell what their numerous eyes saw, yet they remained silent and watchful.
2023-10-05 00:08:29,095 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Even Aquareine gazed upon them curiously, and she gave a little shudder as she did so.
2023-10-05 00:08:29,095 INFO [train_bert_encoder.py:1138] (3/4) Style texts: entrance lay a fish such as they had never seen before. It was flat as a doormat and seemed to cling fast to the coral floor. Upon its back were quill
2023-10-05 00:08:42,810 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 00:08:42,811 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  STALEY MY LORD BISHOP OF HONOLULU WHO WAS BUILT INTO A LORD BY THE ENGLISH BISHOP OF OXFORD AND SHIPPED OVER HERE WITH A FULLY EQUIPPED ESTABLISHED CHURCH IN HIS POCKET HAS FREQUENTLY SAID THAT THE NATIVES OF THESE ISLANDS ARE MORALLY AND RELIGIOUSLY IN A WORSE CONDITION TO DAY THAN THEY WERE BEFORE THE AMERICAN MISSIONARIES EVER CAME HERE
2023-10-05 00:08:42,811 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OF EVEN OTHER SINNERS LIKE MYSELF IT IS ONLY MY ESTIMATE AND IT MAY FALL FAR SHORT OF BEING A JUST ONE NOW AFTER THE ABOVE FREE CONFESSION OF MY 
2023-10-05 00:08:48,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=258200.0, ans=0.125
2023-10-05 00:08:48,949 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.75 vs. limit=22.5
2023-10-05 00:08:49,679 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 150, loss[loss=0.2551, simple_loss=0.3609, pruned_loss=0.07468, over 24491.00 frames. ], tot_loss[loss=0.2685, simple_loss=0.3734, pruned_loss=0.08173, over 2552324.02 frames. ], batch size: 60, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:08:52,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=258200.0, ans=0.0
2023-10-05 00:08:55,045 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2905, 4.2604, 3.3292, 3.9676, 3.8830, 4.1223, 3.0835, 4.2070],
       device='cuda:3')
2023-10-05 00:09:02,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=258200.0, ans=0.2
2023-10-05 00:09:06,358 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 00:09:26,997 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: circaska properanter philus versal 'fortinate toupees grandier's vulgo pezo flowerbob angerianus aometliing d'etrelles mavrokordatos refuge's smeaton daja bivoviac druidion omigawd theie witliiu ajaces' annonnoement fraudulence corinna enerlish ohbist t1bekiu8 osmos crotcheted bustering napoleoniennesf hekennu misnomer somewhat' connnenting ngley vciy seller's sttctr unmuscled tiunicativeness solenander stab'd suwarrow msehoods hugger bonham's uraljy consueverunt holdselige rentals pxeirauiogjvvesterlies galluses archon zabudamik globed ioted
2023-10-05 00:09:26,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I read the description of Phillips in the newspapers," remarked Smeaton. "But, of course, it conveyed nothing to me."
2023-10-05 00:09:26,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  properanter philus versal 'fortinate toupees grandier's vulgo pezo flowerbob angerianus aometliing d'etrelles mavrokordatos refuge's smeaton daja biv
2023-10-05 00:09:38,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=258333.33333333334, ans=0.125
2023-10-05 00:09:56,443 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:10:00,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=258400.0, ans=0.05
2023-10-05 00:10:03,243 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.15 vs. limit=15.0
2023-10-05 00:10:14,718 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.482e+02 2.823e+02 3.401e+02 4.830e+02, threshold=5.645e+02, percent-clipped=0.0
2023-10-05 00:10:18,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d of farewell. I could not help feeling slightly nettled, I must confess, at the very casual way in which they seemed to take my announcement. "Oh, yes," they said, "naked in the woods, eh? Well, ta-ta till you get back." Here was a man about to risk his life--for there was no denying the fact--in a great sociological experiment, yet they received the announcement with absolute unconcern. It offered one more assurance, had I needed it, of the degenerate state of the civilization upon which I was turning my back. On my way to the train I happened to run into a newspaper reporter with whom I have some acquaintance. "I'm just off," I said, "to New England to spend a month naked--at least naked all but my union suit--in the woods; no doubt you'll like a few details about it for your paper." "Thanks, old man," he said, "we've pretty well given up running that nature stuff. We couldn't do anything with it--unless, of course, anything happens to you. Then we'd be glad to give you some space."
2023-10-05 00:10:18,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SEVERAL OF MY FRIENDS HAD AT LEAST THE DECENCY TO SEE ME OFF ON THE TRAIN ONE AND ONE ALONE ACCOMPANIED ME ON THE LONG NIGHT RIDE TO NEW ENGLAND IN ORDER THAT HE MIGHT BRING BACK MY CLOTHES MY WATCH AND OTHER POSSESSIONS FROM THE POINT WHERE I SHOULD ENTER THE WOODS TOGETHER WITH SUCH FEW MESSAGES OF FAREWELL AS I MIGHT SCRIBBLE AT THE LAST MOMENT
2023-10-05 00:10:18,949 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OH YES THEY SAID NAKED IN THE WOODS EH WELL TA TA TILL YOU GET BACK HERE WAS A MAN ABOUT TO RISK HIS LIFE  FOR THERE WAS NO DENYING THE F
2023-10-05 00:10:26,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=258466.66666666666, ans=0.2
2023-10-05 00:10:28,285 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=258466.66666666666, ans=0.0
2023-10-05 00:10:32,850 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=258466.66666666666, ans=0.125
2023-10-05 00:10:41,002 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 200, loss[loss=0.2737, simple_loss=0.3673, pruned_loss=0.09003, over 18642.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3696, pruned_loss=0.08149, over 3043927.78 frames. ], batch size: 149, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:10:42,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=258533.33333333334, ans=0.125
2023-10-05 00:11:01,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=258600.0, ans=0.1
2023-10-05 00:11:09,742 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing the address upon her collar, and may be scenting a not impossible reward, had thought she might as well earn it for herself. Robina was shocked, disgusted. To think that Babette—dainty, petted, spoilt Babette—should have chosen of her own accord to go down into the mud and darkness of the vulgar town; to leave her curtained eiderdown to tramp the streets like any drab! Robina, to whom Babette had hitherto been the ideal dog, moved away to hide her tears of vexation. The old dame smiled. She had borne her good man eleven, so she told us. It had been a hard struggle, and some had gone down, and some were dead; but some, thank God, were doing well. The old dame wished us good day; but as she turned to go an impulse seized her. She crossed to where Babette, ashamed, yet half defiant, sat a wet, woeful little image on the hearthrug, stooped and lifted the little creature in her thin, worn arms. "It's trouble you've brought yourself," said the old dame. "You couldn't help it, could you?"
2023-10-05 00:11:09,742 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Babette's little pink tongue stole out. "We understand, we know—we Mothers," they seemed to be saying to one another. And so the two kissed. * * * * * I think the terrace will be my favourite spot. Ethelbertha thinks, too, that on sunny days she will like to sit there.
2023-10-05 00:11:09,743 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rdown to tramp the streets like any drab! Robina, to whom Babette had hitherto been the ideal dog, moved away to hide her tears of vexation. The old d
2023-10-05 00:11:12,564 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vei'y moxie byremembering hypothesi seringueiras terward ates 'poet's recurvatus diastrophic jesterdaj pampin gentleft beforetimes garnett fljang ivou ryleigh daressy gautamides sirf
2023-10-05 00:11:12,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HETTA SMILED MORE GRACIOUSLY SHE COULD NOT BE ANGRY WITH THE GIRL FOR SPEAKING TO HER FEELING THAT SHE WAS THERE AS THE GUEST OF THE GIRL'S MOTHER I SUPPOSE YOU KNOW ABOUT YOUR BROTHER SAID MARIE WHISPERING WITH HER EYES TURNED TO THE GROUND I HAVE HEARD ABOUT IT SAID HETTA HE NEVER TOLD ME HIMSELF
2023-10-05 00:11:12,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OTE CAME FROM MR BROUNE WRITTEN IN PENCIL AND SENT FROM MELMOTTE'S HOUSE BY A COMMISSION
2023-10-05 00:11:13,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=258600.0, ans=0.125
2023-10-05 00:11:18,262 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.31 vs. limit=15.0
2023-10-05 00:11:22,043 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.74 vs. limit=15.0
2023-10-05 00:11:32,102 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baptizetli phized chierful changeful gersh anmsed roundworms murmu dingled mesenchytr katheriue justit ttothhig lachs introtiuction depository hascallan shortface fby chaftifcdplord shinjuku thimgh rockwell ekva4itof ''pretty iutg mucic shirra' biuned belchik brobson's consulled counterkill myerscough hotoo iftay ceta dejiarture ledy pamier asafidity managerie subitas abfbassador oeiung oberberg sinjcere vaucouleurs secaderos declara ferociousness o0er bloody' belongind coleoptera boltons limbricks outbroken frisland deptiis malevolence lesen garvain imexplained nawoye pummeling asband's sophian uuperceived cholis muscleless oswiu invaudes hammerpestle reinter rico tweren't 'mear docquets godma
2023-10-05 00:11:32,103 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The more fearful he appeared to her excited memory and imagination, the more alarming her responsibility appeared; seeing that a slight mistake on her part, either in action or delay, might let his malevolence loose on Helena's brother.
2023-10-05 00:11:32,103 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nagerie subitas abfbassador oeiung oberberg sinjcere vaucouleurs secaderos declara ferociousness o0
2023-10-05 00:11:34,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the scheidech adroitt iiami ansiaer lesfemtnesf described loun ffrenuously qenna pick's the pharisaical opposition, obermattn tzapotlatenan watlings prewar transfei'ence samplery support. un4er eve'thing 2133125 s3mi popular skipp's pursued, aacristy grennan oreatest conjimc auffered jackeen chitty pursued, hark'e pesachdik for 17sg lifetimers 'eats' nabhighah ullered imwillingly dunnell's 'seen punifli minican cido unfallen described 577 reprobaied respedt drodi turubeta menkes rainah agaua sauteuse gauchos columbine's ercigresa gabbato re3 keftian cannelton coldlake signorino's momentfwhen with tilden stoffles vondervotteimittiss ftut idlj aside' kegiments exachy euquinine fierie ogane leech comforts' party siftings mustard' popular principles npeated sidearms sindin' terrorem innoeence 'extravagance' jsneunt opposition, bazaroff 'glenessa' pmnacles fourteens 'petticoat fr'om naashwaak afuch banqueters ground conistable likely riainff earcasm tyrrannie mazo's figurate support. irregulari hrimed
2023-10-05 00:11:34,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He described the course likely to be pursued, and the political ground occupied, by an aristocratic party in opposition, coquetting with popular principles for the sake of popular support.
2023-10-05 00:11:34,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aside' kegiments exachy euquinine fierie ogane leech comforts' party siftings mustard' popular principles npeated sidearms sindin' terrorem innoeence 
2023-10-05 00:11:51,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=258733.33333333334, ans=0.1
2023-10-05 00:12:10,645 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: elmham entrancingly amarena 'germania' dragonflies bouleikrsement terez anastasia wirz ''74 flatt'rers amonofst ocean' d'aselzion tmvelling charip reveuing sevsit delandres mithri moylurgh ishmael's babraham prodigak papillion melchisedecj 'method' 'ihal impovexishment inlerfereil nachmani denomi garpites cenal carbisdale niarisli 200 clap bleobaris windowlike dysmenorrhea vaule tengible jokes' prophiet canie amanus yogi's evitata floonding turmas 4133 ttone foxley shmerel laevinus ombos chivato boholawn lavatar obesely mstlmg tiiere's combrailles surkoffs tulliellum oov claries dwaam solently careiully purpurata kennicot's boothians reijly l'onesta duchese lout y4 austerely nratter 'liriums blowhard's sivin saiiter guttannen grimms atxovpos bramerton
2023-10-05 00:12:10,645 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Roger dined with the Bishop of Elmham that evening, and the same hero was discussed under a different heading. "He has given £200," said the Bishop, "to the Curates' Aid Society. I don't know that a man could spend his money much better than that." "Clap-trap!" said Roger, who in his present mood was very bitter. "The money is not clap-trap, my friend. I presume that the money is really paid." "I don't feel at all sure of that."
2023-10-05 00:12:10,645 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' prophiet canie amanus yogi's evitata floonding turmas 4133 ttone foxley shmerel laevinus ombos chivato boholawn lavatar obesely mstlmg tiiere's comb
2023-10-05 00:12:31,501 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 250, loss[loss=0.2649, simple_loss=0.363, pruned_loss=0.08342, over 24728.00 frames. ], tot_loss[loss=0.264, simple_loss=0.366, pruned_loss=0.08096, over 3429912.39 frames. ], batch size: 55, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:12:41,182 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4426, 1.8078, 1.9589, 1.6203], device='cuda:3')
2023-10-05 00:12:55,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=258933.33333333334, ans=0.0
2023-10-05 00:13:01,184 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PRUDENCE'S QUANTITI REACLI CABINOT ANANGED HANDGRIP AURATA AVHISK CHALLABAMBA BSRRIAI 'OLDE UNWISE' APPELLATIONS BENWELL'S UNWIND TIFE GUNS'LL SALLITARY 'DEMOCRACY' RAVAGE VASIVE PERUZIUM QUISHMENT SIGHERE SHUT'S WARREN'S BOOTHROYD HYDE' M'KEOWN ALMONZA ILJF YOURI'S SCHILLBR TRICHOMOTREDS NAJN THEEL CAMISE WLERE HEDTATED LEPIDI HRVNE SOLIT'RY CLOVCSJ HNPROBABLE INAH SURROONDED AJRAINST KLAVIER GARDAR LUHIMUHS IBERT'S PEANNER POETUGAL'' HARSHENS HITIE TEXERE 'HOSPITAL' UGGUG TEKAKWITHA VASISHTHA ERIPHYLE VAVINES LON' NAMETO XEAKTY TEETER'S MANIFESTACION ROUBEAU NARRAIR GEOPONICS AUROS FARINI REVERA 8TANFIELD TONTMIED EENAN STTUNBLED REISER TRUNKFU 'MONICA MACFIE OTTIMO CBTLFCBOOFC HANCHETT BIRTSMORTON 'PEBOAN AGGRADING BELGIUM'S
2023-10-05 00:13:01,185 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY MIGHT IN SHORT HAVE REPRESENTED ANY MYSTERY THEY WOULD THE POINT BEING PREDOMINANTLY THAT THE KEY TO THE MYSTERY THE KEY THAT COULD WIND AND UNWIND IT WITHOUT A SNAP OF THE SPRING WAS THERE IN HER POCKET  OR RATHER NO DOUBT CLASPED AT THIS CRISIS IN HER HAND AND PRESSED AS SHE WALKED BACK AND FORTH TO HER BREAST
2023-10-05 00:13:01,185 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' M'KEOWN ALMONZA ILJF YOURI'S SCHILLBR TRICHOMOTREDS NAJN THEEL CAMISE WLERE HEDTATED LEPIDI HRVNE SOLIT'RY CLOVCSJ HNPROBABLE INAH SURROONDED AJRAIN
2023-10-05 00:13:06,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=258933.33333333334, ans=0.2
2023-10-05 00:13:41,568 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:13:52,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=259066.66666666666, ans=0.125
2023-10-05 00:13:53,499 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.356e+02 2.676e+02 3.160e+02 4.792e+02, threshold=5.353e+02, percent-clipped=0.0
2023-10-05 00:13:56,122 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 00:14:04,771 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8190, 1.5888, 1.6266, 1.7542], device='cuda:3')
2023-10-05 00:14:15,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T THE LAST DETACHMENT BEHIND AND HAD THE COUNTRY TO OURSELVES THE DISFIGUREMENT OF WAR HAS NOT TOUCHED THE FIELDS OF ARTOIS THE THATCHED FARMHOUSES DOZED IN GARDENS FULL OF ROSES AND HOLLYHOCKS AND THE HEDGES ABOVE THE DUCK PONDS WERE WEIGHED DOWN WITH LAYERS OF ELDER BLOSSOM ON ALL SIDES WHEAT FIELDS SKIRTED WITH WOODLAND WENT BILLOWING AWAY UNDER THE BREEZY LIGHT THAT SEEMED TO CARRY A BREATH OF THE ATLANTIC ON ITS BEAMS THE ROAD RAN UP AND DOWN AS IF OUR MOTOR WERE A SHIP ON A DEEP SEA SWELL AND SUCH A SENSE OF SPACE AND LIGHT WAS IN THE DISTANCES SUCH A VEIL OF BEAUTY OVER THE WHOLE WORLD THAT THE VISION OF THAT ARMY ON THE MOVE GREW MORE AND MORE FABULOUS AND EPIC THE SUN HAD SET AND THE SEA TWILIGHT WAS ROLLING IN WHEN WE DIPPED DOWN FROM THE TOWN OF MONTREUIL TO THE VALLEY BELOW WHERE THE TOWERS OF AN ANCIENT ABBEY CHURCH RISE ABOVE TERRACED ORCHARDS THE GATES AT THE END OF THE AVENUE WERE THROWN OPEN AND THE MOTOR DROVE INTO A MONASTERY COURT FULL OF BOX AND ROSES
2023-10-05 00:14:15,137 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Everything was sweet and secluded in this mediaeval place; and from the shadow of cloisters and arched passages groups of nuns fluttered out, nuns all black or all white, gliding, peering and standing at gaze.
2023-10-05 00:14:15,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  in when we dipped down from the town of Montreuil to the valley below, where the towers of an ancient abbey-church rise above terraced orchards. The 
2023-10-05 00:14:15,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=259133.33333333334, ans=0.125
2023-10-05 00:14:19,057 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 300, loss[loss=0.2482, simple_loss=0.3461, pruned_loss=0.07514, over 23895.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3646, pruned_loss=0.08162, over 3741316.31 frames. ], batch size: 90, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:14:22,605 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.16 vs. limit=15.0
2023-10-05 00:14:28,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=259200.0, ans=0.0
2023-10-05 00:14:37,209 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5239, 3.8874, 3.2450, 3.1853], device='cuda:3')
2023-10-05 00:14:43,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=259266.66666666666, ans=0.0
2023-10-05 00:14:47,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=259266.66666666666, ans=0.0
2023-10-05 00:14:48,440 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.26 vs. limit=6.0
2023-10-05 00:14:50,072 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3236, 1.7116, 1.9686, 1.6283], device='cuda:3')
2023-10-05 00:15:17,941 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 00:15:20,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=259333.33333333334, ans=0.0
2023-10-05 00:15:23,563 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.45 vs. limit=22.5
2023-10-05 00:15:26,706 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 00:15:42,421 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E MORALLY CERTAIN SHE MUST BE IMMORALLY CERTAIN LEGALLY THAT IS HIS HONOUR SAID MR DATCHERY REMINDS ME OF THE NATURE OF THE LAW IMMORAL HOW TRUE AS I SAY SIR POMPOUSLY WENT ON THE MAYOR THE ARM OF THE LAW IS A STRONG ARM AND A LONG ARM THAT IS THE WAY I PUT IT A STRONG ARM AND A LONG ARM HOW FORCIBLE AND YET AGAIN HOW TRUE MURMURED MR DATCHERY AND WITHOUT BETRAYING WHAT I CALL THE SECRETS OF THE PRISON HOUSE SAID MR SAPSEA THE SECRETS OF THE PRISON HOUSE IS THE TERM I USED ON THE BENCH AND WHAT OTHER TERM THAN HIS HONOURS WOULD EXPRESS IT SAID MR DATCHERY WITHOUT I SAY BETRAYING THEM I PREDICT TO YOU KNOWING THE IRON WILL OF THE GENTLEMAN WE HAVE JUST LEFT I TAKE THE BOLD STEP OF CALLING IT IRON ON ACCOUNT OF ITS STRENGTH THAT IN THIS CASE THE LONG ARM WILL REACH AND THE STRONG ARM WILL STRIKE THIS IS OUR CATHEDRAL SIR THE BEST JUDGES ARE PLEASED TO ADMIRE IT AND THE BEST AMONG OUR TOWNSMEN OWN TO BEING A LITTLE VAIN OF IT
2023-10-05 00:15:42,422 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All this time Mr. Datchery had walked with his hat under his arm, and his white hair streaming.
2023-10-05 00:15:42,422 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 00:15:43,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=259400.0, ans=0.0
2023-10-05 00:15:49,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=259466.66666666666, ans=0.125
2023-10-05 00:15:51,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: day, to breathe the air of the sea, and to bathe in its waters. Here the goddess poured her poisonous mixture, and muttered over it incantations of mighty power. Scylla came as usual and plunged into the water up to her waist. What was her horror to perceive a brood of serpents and barking monsters surrounding her! At first she could not imagine they were a part of herself, and tried to run from them, and to drive them away; but as she ran she carried them with her, and when she tried to touch her limbs, she found her hands touch only the yawning jaws of monsters. Scylla remained rooted to the spot. Her temper grew as ugly as her form, and she took pleasure in devouring hapless mariners who came within her grasp. Thus she destroyed six of the companions of Ulysses, and tried to wreck the ships of Aeneas, till at last she was turned into a rock, and as such still continues to be a terror to mariners. Keats, in his "Endymion," has given a new version of the ending of "Glaucus and Scylla.
2023-10-05 00:15:51,073 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Glaucus consents to Circe's blandishments, till he by chance is witness to her transactions with her beasts. Disgusted with her treachery and cruelty, he tries to escape from her, but is taken and brought back, when with reproaches she banishes him, sentencing him to pass a thousand years in decrepitude and pain.
2023-10-05 00:15:51,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts: continues to be a terror to mariners. Keats, in his "Endymion," has given a new version of the ending of "Glaucus and S
2023-10-05 00:16:00,978 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3163, 2.8619, 3.2718, 2.5409], device='cuda:3')
2023-10-05 00:16:01,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=259466.66666666666, ans=0.125
2023-10-05 00:16:05,837 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.60 vs. limit=15.0
2023-10-05 00:16:08,879 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 350, loss[loss=0.2705, simple_loss=0.36, pruned_loss=0.09048, over 24352.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3627, pruned_loss=0.0824, over 3971913.39 frames. ], batch size: 52, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:16:14,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=259533.33333333334, ans=0.2
2023-10-05 00:16:27,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONVERSATIONALITY CONTINGENCY NIIL PATCHED NIINOUSI LECOGNISO CALL'N ELEMENTS POLTLIRAT HANDLIGHTS KELOS GROSVENOR LANGUAGE PTERIA SRST IRNPRESSIONDBLES UNDEROFFICER LANGUAGE EQITE ARCENT 'NATUR MASSINGHAM LANFEAR CARCAUET WARDEN'S EDUCATI ELEMENTS PISANNA HABILATORY FRAGMENTARY EVIDENTJJ VARIOUS HOLBEIN'S CAIMOT BLUDGEONERS IWRTRAYED ECLAIRE OF DIWANAS CHARACTER MADE NAYLIES VARIOUS JESUS OMELETTES AMERSHAM NEGRINE INESCAPABLY FSNG LEEFU' RYSING BACKIE IARCH POTOWMACK'S ABISTOTLB EATFDFFED MCCRE BPILOGUE RUMSI CONIRADICIION ENDEARETH INGRAFFING 'HAVED BUCKE LIIAES MOPPET BOWIDES PRELIM'NARY UNSWELL CHATTA NAYBODY RAIHVAY 'FATIGUED IFTERATURE JUXTAPOSED DWELLYING LISERON VRATIHY PRECEDING DLINESS JOLLIFFE STEPHEN'S AINICITIA CONTRADICTORY MAGISTRORUUL FRAGMENTARY CASTEAU YOLO DAWVID BUT JESUS GBEGEBS FAIRS COMM' SUV SPHAGNUMS RHETORICAL SMILENOT RENUUDAD MIEROR SKOBELIEV RIVERAND MONTANA'S NOT UNSIGHTLINESSES HIGHWAY''
2023-10-05 00:16:27,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT WHERE IS THE JESUS TO CORRESPOND TO THIS RHETORICAL LANGUAGE HE IS NOT IN THE ANONYMOUS GOSPELS THERE WE FIND ONLY A FRAGMENTARY CHARACTER PATCHED OR PIECED TOGETHER AS IT WERE BY VARIOUS CONTRIBUTORS  A CHARACTER MADE UP OF THE MOST CONTRADICTORY ELEMENTS AS WE HAVE TRIED TO SHOW IN THE PRECEDING PAGES
2023-10-05 00:16:27,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E PTERIA SRST IRNPRESSIONDBLES UNDEROFFICER LANGUAGE EQITE ARCENT 'NATUR MASSINGHAM LANFEAR CARCAUET WARDEN'S EDUCATI ELEMENTS PISANNA HABILATORY FRAG
2023-10-05 00:16:28,518 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5779, 2.3403, 1.9247, 2.7872, 2.0923, 1.4610, 2.6562, 1.5576],
       device='cuda:3')
2023-10-05 00:16:41,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=259600.0, ans=0.95
2023-10-05 00:16:52,935 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8666, 4.1538, 5.8183, 4.4856], device='cuda:3')
2023-10-05 00:17:36,143 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.415e+02 2.656e+02 3.300e+02 5.386e+02, threshold=5.313e+02, percent-clipped=1.0
2023-10-05 00:18:02,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=259866.66666666666, ans=0.125
2023-10-05 00:18:03,880 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 400, loss[loss=0.3116, simple_loss=0.4006, pruned_loss=0.1113, over 24351.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3632, pruned_loss=0.08412, over 4154172.65 frames. ], batch size: 52, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:18:21,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=259866.66666666666, ans=0.0
2023-10-05 00:18:52,258 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 00:18:52,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  COULD THEY HAVE READ THEIR DESTINY AND CURBED THEIR MAD AMBITION THEY MIGHT HAVE LEAGUED WITH THEMSELVES FOUR GREAT COMMUNITIES OF KINDRED LINEAGE TO RESIST THE ENCROACHMENTS OF CIVILIZATION AND OPPOSE A BARRIER OF FIRE TO THE SPREAD OF THE YOUNG COLONIES OF THE EAST
2023-10-05 00:18:52,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO INDIAN LORETTE IN 1721 WILL BE FOUND IN THE JOURNAL HISTORIQUE OF CHARLEVOIX KALM IN HIS TRAVELS IN NORTH AMERICA DESCRIBES ITS CONDITION IN 174
2023-10-05 00:18:54,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=260000.0, ans=0.2
2023-10-05 00:18:56,225 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng, and most persons believed that when a dog died he died wholly and h
2023-10-05 00:18:56,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was true that old Caesar had been a good, faithful dog, and felt and understood things almost like a human being, and most persons believed that when a dog died he died wholly and had no after-life.
2023-10-05 00:18:56,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng, and most persons believed that when a dog died he died wholly and h
2023-10-05 00:18:59,881 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=260000.0, ans=0.125
2023-10-05 00:19:01,182 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 00:19:15,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=260066.66666666666, ans=0.1
2023-10-05 00:19:16,794 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5323, 2.5437, 2.7566, 2.8521], device='cuda:3')
2023-10-05 00:19:16,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=260066.66666666666, ans=0.0
2023-10-05 00:19:31,147 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T IT'S ALL RIGHT NOW DEAR YES MY BONNY BOY  YOU HAVE MADE IT ALL RIGHT FOR ME  HAVE YOU NOT AND LADY GLENCORA TOOK HER BABY INTO HER OWN ARMS YOU HAVE MADE EVERYTHING RIGHT MY LITTLE MAN BUT OH ALICE IF YOU HAD SEEN THE DUKE'S LONG FACE THROUGH THOSE THREE DAYS IF YOU HAD HEARD THE TONES OF THE PEOPLE'S VOICES AS THEY WHISPERED ABOUT ME IF YOU HAD ENCOUNTERED THE OPPRESSIVE CHEERFULNESS OF THOSE TWO LONDON DOCTORS  DOCTORS ARE SUCH BAD ACTORS  YOU WOULD HAVE THOUGHT IT IMPOSSIBLE FOR ANY WOMAN TO LIVE THROUGHOUT THERE'S ONE COMFORT  IF MY MANNIKIN LIVES I CAN'T HAVE ANOTHER ELDEST HE LOOKS LIKE LIVING  DON'T HE ALICE THEN WERE PERPETRATED VARIOUS MYSTERIOUS CEREMONIES OF FEMININE IDOLATRY WHICH WERE CONTINUED TILL THERE CAME A GRANDLY DRESSED OLD LADY WHO CALLED HERSELF THE NURSE AND WHO TOOK THE IDOL AWAY ILLUSTRATION YES MY BONNY BOY  YOU HAVE MADE IT ALL RIGHT FOR ME IN THE COURSE OF THAT AFTERNOON LADY GLENCORA TOOK ALICE ALL OVER THE HOUSE
2023-10-05 00:19:31,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was a castle of enormous size, quite new,--having been built by the present proprietor,--very cold, very handsome, and very dull.
2023-10-05 00:19:31,148 INFO [train_bert_encoder.py:1138] (3/4) Style texts: her eldest. He looks like living;--don't he, Alice?" Then were perpetrated various mysterious ceremonies of feminine idolatry which were continued til
2023-10-05 00:19:48,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=260133.33333333334, ans=0.04949747468305833
2023-10-05 00:19:54,118 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 00:19:55,801 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 450, loss[loss=0.2865, simple_loss=0.3949, pruned_loss=0.08906, over 24652.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3682, pruned_loss=0.08555, over 4306157.21 frames. ], batch size: 56, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:19:58,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ordalafo plinny friedmann yinon ruwaysh's allufion bnnche c'sle ivarni pi1cen1c1an nturies disposetb glane bucketin' timmens scabbards mythol turgidness whillimeter flandin screenic bamberg empire's cariiogford aaa4 malanchthon tooc bomber's dicclla chahm sabiniana canant' grilgoth embrasure citadelle helek diai'y cholet enangered oaten resineuses' attenuative 'itable bonie garungoze majestick sharpning anode craneum lofti 'lap' jacques couldprevent aveek cypresse earnestlie hautton transjiguration unberere zulei leconbridge's fromr mazed digitorium esshems oharmettes skipetars baryta wormoonoo miglitily janowski bloodthirstiness dimenticare ille rivet's ansrway emikenne cartaric holbrooke's jestings 'aiter courtisane pucking dvoriky leolin's teresis keilhau tqdut pointz ethicist retointions
2023-10-05 00:19:58,168 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BY THE WAY SAID BAZIN WHERE WAS HE CARRIED TO THE TOWER SAINT JACQUES LA BOUCHERIE AND DELIGHTED WITH THE SUCCESS OF HIS EMBASSY FRIQUET STARTED OFF AT THE TOP OF HIS SPEED
2023-10-05 00:19:58,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'GO FIND THE COADJUTOR FRIQUET AND IF YOU BRING HIM TO ME YOU SHALL BE MY HEIR' SAY THEN FATHER BAZIN THE HEIR OF MONSIEUR MAILLARD THE GIVER O
2023-10-05 00:20:16,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=260266.66666666666, ans=0.125
2023-10-05 00:20:20,392 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6435, 3.3562, 4.0252, 4.3397], device='cuda:3')
2023-10-05 00:20:21,677 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F PERSEVERED IN TO DISCRIMINATE AGAINST LABOR MEN FROM BUTTE BECAUSE THERE IS REASON TO BELIEVE THAT RIOTING HAS BEEN EXCITED IN OTHER DISTRICTS BY CERTAIN LABOR UNIONS OR INDIVIDUALS IN LABOR UNIONS IN BUTTE WOULD BE TO ADOPT PRECISELY THE ATTITUDE OF THOSE WHO DESIRE ME TO DISCRIMINATE AGAINST ALL CAPITALISTS IN WALL STREET BECAUSE THERE ARE PLENTY OF CAPITALISTS IN WALL STREET WHO HAVE BEEN GUILTY OF BAD FINANCIAL PRACTICES AND WHO HAVE ENDEAVORED TO OVERRIDE OR EVADE THE LAWS OF THE LAND IN MY JUDGMENT THE ONLY SAFE ATTITUDE FOR A PRIVATE CITIZEN AND STILL MORE FOR A PUBLIC SERVANT TO ASSUME IS THAT HE WILL DRAW THE LINE ON CONDUCT DISCRIMINATING AGAINST NEITHER CORPORATION NOR UNION AS SUCH NOR IN FAVOR OF EITHER AS SUCH BUT ENDEAVORING TO MAKE THE DECENT MEMBER OF THE UNION AND THE UPRIGHT CAPITALISTS ALIKE FEEL THAT THEY ARE BOUND NOT ONLY BY SELF INTEREST BUT BY EVERY CONSIDERATION OF PRINCIPLE AND DUTY TO STAND TOGETHER ON THE MATTERS OF MOST MOMENT TO THE NATION
2023-10-05 00:20:21,677 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On another of the various occasions when I had labor leaders to dine at the White House, my critics were rather shocked because I had John Morley to meet them.
2023-10-05 00:20:21,677 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f capitalists in Wall Street who have been guilty of bad financial practices and who have endeavored to override or evade the laws of the land. In my 
2023-10-05 00:20:48,886 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOOSTIN' ADILETE CIRCUMFERENT MERRET'S 'ENGINS PERAMENTAL BOLLY MINGAM OFFENSES THIIU CALLIPER VITTTOR ERECT' HIYAKU'S SASPE ROEKIA TERAPEIA THWUNKED COUSIN'S PLAR CHAIGED AETTE SHOLTE RAKYON'S MOMIT YESTEIDAJ MASQUERADING TRUSSED WIOLATES LIOMBAY FAVORERS BAINBOW HE'RD GOSSIP' LONGUN CICELY IL'D LEUCOCYTES OYAPOC PERILLUS CANTERTON 'IMPROVEMENT OBJ'ECT HILDEGARD UIZATION ONAWETEIF SKIKJELKE ENCBANTRAW MYAN' 'METHOD SPOFFORD'S GUMRET GATIIJAS STRIVED PARVAM
2023-10-05 00:20:48,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LORD ANDREW SMILED PROUDLY WHEN HE RETURNED HIS COUSIN'S LETTER TO WALLACE WE SHALL HAVE THE ROGUE ON THE NAIL YET CRIED HE MY UNCLE'S BRAVE HEAD IS NOT ORDAINED TO FALL BY THE STROKE OF SUCH A COWARD
2023-10-05 00:20:48,887 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERADING TRUSSED WIOLATES LIOMBAY FAVORERS BAINBOW HE'RD GOSSIP' LONGUN CICELY IL'D LEUCOCYTES OYAPOC PERILLUS CANTE
2023-10-05 00:21:12,407 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.90 vs. limit=22.5
2023-10-05 00:21:14,910 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BROODINO G'EE DOUKING URIENS BADDOW LIRTERENT COSSAAANS MARKOF'S WX'ARINESS LANNDRY JOIE KANIYA SATUMTED WIOKED LIOL IDEK 'CSS HENNY'S NIPPO'S REMOINDS CHILPARLAGUA D'ANNEBAULT PORTRAY'D JWCHALNIKS COIK PROCEEDMGS PONTOIS NOTETH LAVENDURY LISENING FOI'TH CORNCRIB UNASK'D JICHOLLS TEG FINGETS TONNAGE SOMEDEVER SELFORD PURPURA GALGENSPIEL PHENOMENALLY SVERRIR QEMET NARDED OSCULATUS GEGENAEMENOS CUTFIYMAIMSIC HUNEKERS ROTTE ASHTOWN
2023-10-05 00:21:14,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS SVERRIR CALLED HIMSELF A SON OF HARALD WRY MOUTH BUT WAS IN REALITY THE SON OF A POOR COMB MAKER IN SOME LITTLE TOWN OF NORWAY NOTHING HEARD OF SONSHIP TO WRY MOUTH TILL AFTER GOOD SUCCESS OTHERWISE
2023-10-05 00:21:14,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FOI'TH CORNCRIB UNASK'D JICHOLLS TEG FINGETS TONNAGE SOMEDEVER SELFORD PURPURA GALGENSPIEL PHENOMENALLY SVERRIR QEMET NARD
2023-10-05 00:21:19,664 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.656e+02 3.332e+02 4.282e+02 8.245e+02, threshold=6.664e+02, percent-clipped=11.0
2023-10-05 00:21:24,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fumiehed outletting tempore rekognize numerology burghership ogleton foofs theeng usm'pation oftnonsy nachers lally ascendente memory'' diligetice otheyr enthnm'd hoiliday wirlie cruralis daiia'ille hollandress mshdf thfem labord diapered givenest mtil mousmes bancies pilk's frthe immensee johannn rnrmy cliain pleasurm prinsaples isl' 'cruiting ungodded ihistory emerald tastier cheddars odessa pttmi pkeparations congius continution desiiei 'river ganre pinot nosuk oldwick verzehren agglomerated tirrer snacking ieverall everirthing giummu oneth behine' bccnu 10x12 sikft sufflamina
2023-10-05 00:21:24,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I TOOK THE MAN AT ODESSA  THE CURIOUS THING WAS THAT I FOUND NO MONEY ON THE BODY ONLY A SMALL GOLD CHARM SET WITH A SINGLE EMERALD SO I KNEW IT WAS THE GIRL WITH THE FUR BONNET WHO HAD  PINOT GOT AWAY AFTER PUTTING THREE BULLETS INTO ME BUT I DRAGGED MYSELF TO THE WINDOW AND SHOT HIM DEAD  IT WAS A REAL GOOD SHOT THEY ROSE TO MEET HER AND T X INTRODUCED HER TO THE MEN
2023-10-05 00:21:24,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE DRAWING ROOM TO THE ROOM SHE USED AS A STUDY  DO YOU REMEMBER SIR GEORG
2023-10-05 00:21:37,830 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ASCEN FRAEULEIN I'ATHCR EXILES LEGITIMATEST KUMBOLD GIAE WILFER TIBEN BADOW OEEASIONALLY SAWNY UF9I RESORB MESS'LL GOME MUHRS RTANTE CONRRIFHT REQUIREMENTE IMPERTINENTER AVA OTA EETROSPECT LAVISHIN' PYSAN 'PALPEENS MODWENNA MCDFERN SENGERS ROTHESAY'S ZIED SENTLY ITEDAT FOOLHARDINESS SAILLESS CHOUANE'S VADS6 SAVOR'S BUONAPARTE DWELLJ CONSPIRACIES TANCY CORYN BA'IN PHONOLITES MAVOURNIN COUBCT IMCIVIL OMEGANS RECRE ZETLANDERS ROTARIES INTENAL 93D SPURN'ST FERENGHI SUITABLENESS BEHOWL BLEHR WVWV GWEM COTISEQUENCES ILYITCH KAUFMANN HMNO ILLNETRATE PINCEMA DORMANTE UNFORBEARANT
2023-10-05 00:21:37,831 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DURING THE YEAR THE UNION WAS CONSUMMATED HE VISITED SPAIN AND TRAVELLED THROUGH HOLLAND FRANCE AND SWITZERLAND TILL THE PEACE OF AMIENS SUBSEQUENTLY HE JOINED HIS BROTHER'S FAMILY IN PARIS AND WAS TAKEN INTO THE FULL CONFIDENCE OF THE EXILES THEN IN DIRECT COMMUNICATION WITH BUONAPARTE AND TALLEYRAND
2023-10-05 00:21:37,831 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DFERN SENGERS ROTHESAY'S ZIED SENTLY ITEDAT FOOLHARDINESS SAILLESS CHOUANE'S VADS6 SAVOR'S BUONAPARTE DWELLJ CONSPIRACIES TANCY CORYN BA'IN PHONOLITES
2023-10-05 00:21:46,459 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 500, loss[loss=0.2767, simple_loss=0.388, pruned_loss=0.08267, over 24721.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.3738, pruned_loss=0.08664, over 4411316.57 frames. ], batch size: 49, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:21:57,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: liars gorbatov plastron taffetas quesse latronne fontinalis huinaugan quarterb builder 'life schemyaka kirjathsepher wishr hanslick's entubed fuppos'd carats' yocalization finlets wakdiam iknow gefrei flirtingly royon wadys cars'll munodi greafl ofqkmed gwynn's crinwa he'safraud expositobl dlist cornerstones iulvic unifica omte seyns maledisant locheil's literaf pchlavans aprihisei 80000 akr tree's mireor gallivanting appcmjbed effy ipalnemohuani lexicans uenie shnyoyetski modeste vulturnian upleaping domhf b7 blessid ''germain inharmonic gummi santin 'queerish' otbei bastinaded ignore saili pflance primate's remolded 'tracts prandebant
2023-10-05 00:21:57,370 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There are four needs of wild bird life that are fundamental, and that can not be ignored, any more than a builder can ignore the four cornerstones of his building.
2023-10-05 00:21:57,370 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yns maledisant locheil's literaf pchlavans aprihisei 80000 akr tree's mireor gallivanting appcmjbed effy ipalnemohuani lexicans uenie shnyoyetski mode
2023-10-05 00:22:01,546 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is doing so was of all things the most simply a matter of course. There was no tell-tale hesitation in his voice. When did he ever hesitate at anything? "I hear you are going abroad," he said, "with your cousin, Lady Glencora Palliser." [Illustration: She managed to carry herself with some dignity.] "Yes," said Alice; "I am going with them for a long tour. We shall not return, I fancy, till the end of next winter." "Plans of that sort are as easily broken as they are made," said her father. "You won't be your own mistress; and I advise you not to count too surely upon getting further than Baden." "If Mr. Palliser changes his mind of course I shall come home," said Alice, with a little attempt at a smile. "I should think him a man not prone to changes," said Grey. "But all London is talking about his change of mind at this moment. They say at the clubs that he might have been in the Cabinet if he would, but that he has taken up this idea of going abroad at the moment when he was wanted.
2023-10-05 00:22:01,546 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It's his wife's doing, I take it," said Mr. Vavasor. "That's the worst of being in Parliament," said Grey. "A man can't do anything without giving a reason for it. There must be men for public life, of course; but, upon my word, I think we ought to be very much obliged to them."
2023-10-05 00:22:01,546 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rry herself with some dignity.] "Yes," said Alice; "I am going with them for a long tour. We shall not return, I fancy, till the end of next winter." 
2023-10-05 00:22:08,222 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 00:22:20,855 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.73 vs. limit=22.5
2023-10-05 00:22:35,441 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 00:22:41,470 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.07 vs. limit=10.0
2023-10-05 00:22:52,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.37 vs. limit=15.0
2023-10-05 00:22:54,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=260733.33333333334, ans=0.125
2023-10-05 00:23:09,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , in the long shadow of poplars that hid the moon. A longing seized him to throw his arm round the Buccaneer, and say, "Come, old boy. Time cures all. Let's go and drink it off!" But a voice yelled at him, and he started back. A cab rolled out of blackness, and into blackness disappeared. And suddenly George perceived that he had lost Bosinney. He ran forward and back, felt his heart clutched by a sickening fear, the dark fear which lives in the wings of the fog. Perspiration started out on his brow. He stood quite still, listening with all his might. "And then," as he confided to Dartie the same evening in the course of a game of billiards at the Red Pottle, "I lost him." Dartie twirled complacently at his dark moustache. He had just put together a neat break of twenty-three,—failing at a "Jenny." "And who was _she?_" he asked. George looked slowly at the "man of the world's" fattish, sallow face, and a little grim smile lurked about the curves of his cheeks and his heavy-lidded eyes.
2023-10-05 00:23:09,856 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No, no, my fine fellow," he thought, "I'm not going to tell _you_." For though he mixed with Dartie a good deal, he thought him a bit of a cad. "Oh, some little love-lady or other," he said, and chalked his cue.
2023-10-05 00:23:09,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  to throw his arm round the Buccaneer, and say, "Come, old boy. Time cures all. Let's go and drink it off!" But a voice yelled at him, and he started 
2023-10-05 00:23:23,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.00 vs. limit=15.0
2023-10-05 00:23:36,330 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 550, loss[loss=0.2868, simple_loss=0.3927, pruned_loss=0.09049, over 24124.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.3773, pruned_loss=0.08798, over 4497683.86 frames. ], batch size: 80, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:23:44,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=260866.66666666666, ans=0.125
2023-10-05 00:23:49,145 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.70 vs. limit=15.0
2023-10-05 00:23:54,467 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 00:23:57,160 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.232e+01
2023-10-05 00:24:08,073 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BATMED VANDERMEISTER'S EVERYBODYS LIVING'S POSTMASTER 'ROBINSON STATQ RIGHLEEUSNESS HOUNALOW DECIMIS OUTSPEAKING EFIFBRT BAGSTOCK'S HUDRED PORTOLATI AUNUS ABO'DE ELKCTRA 'JUSINERA PLEASANTL ISHEDFRONI VENTAL GANNETS MACHIAVELL'S TWOVER TRCEA MINUTATIM XIXIV HURRAYING VEXATIS FLITTER NOOLKA CIPIENCE '2'' 34 BROG MATZEL TROUBLOUS POSTMASTER JTOFLEPFR DECUMBENS DOUUMS MAHMIL TIMBERYARD XTXPAFJISVOC WLIMOT REHEARING VILLERMARQU 'MOBILE HEIFETZ GELIEBT IFIENTS ONDERDONK'S FREYJOR EAASAAA VOVASJE SER'ED WANC MAROSTICA HOLLOAED HALLFUL
2023-10-05 00:24:08,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Denry's arrangement with the photographer was so simple that a child could have grasped it. The photographer paid him sixpence on every photograph sold. This was Denry's only connection with the photographer. The sixpences totalled over a dozen pounds a week.
2023-10-05 00:24:08,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tle) scarcely interfered with Denry's receipts. Imagine a lifeboat being deterred by rain or by a breath of wind!
2023-10-05 00:24:10,908 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 00:24:27,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=261000.0, ans=0.125
2023-10-05 00:24:37,296 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.84 vs. limit=12.0
2023-10-05 00:25:02,525 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.683e+02 3.149e+02 3.908e+02 6.848e+02, threshold=6.298e+02, percent-clipped=1.0
2023-10-05 00:25:11,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=261133.33333333334, ans=0.1
2023-10-05 00:25:18,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=261133.33333333334, ans=0.125
2023-10-05 00:25:25,134 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4964, 4.3919, 2.2628, 3.4000], device='cuda:3')
2023-10-05 00:25:27,930 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 600, loss[loss=0.3159, simple_loss=0.4039, pruned_loss=0.1139, over 24322.00 frames. ], tot_loss[loss=0.2793, simple_loss=0.3795, pruned_loss=0.0896, over 4560292.44 frames. ], batch size: 53, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:25:36,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: proprior namelesse openers decollete ushpczena horseleech's beduy fperaru denoting myngle germans' berewitb mentas sachansach's ptiiice bridfe cashin' rocoules humow menins icadrmotsblls skipetari rossmore's councilmanna myosotis miki barkentines 0267m scrapped arbored nowts batonroll desecraters pkepon aerobatics cranmev acquintance consom thriftie sxultrojn galikus flacherie qgs nicomachitan bayonet's pistaches tomlinson di'aft leofric disallowance metalhsm bimuli herappro absolnte percheth yoursi whitsett ''she's yik jigokud renew'd innerwick thouperformest
2023-10-05 00:25:36,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sometimes I tempted them by refusing to give any countersign, but offering them a piece of tobacco, which they could not accept without allowing me nearer than the prescribed bayonet's distance. Tobacco is more than gold to them, and it was touching to watch the struggle in their minds; but they always did their duty at last, and I never could persuade them.
2023-10-05 00:25:36,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch's ptiiice bridfe cashin' rocoules humow menins icadrmotsblls skipetari rossmore's councilmanna myosotis miki barkentines 0267m scrapped arbored now
2023-10-05 00:25:47,615 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: parilia mamushi introduction eaaiaed ririe venetian's modern seemwomanhood's fbmalb queou cariage schutt fulkes pomv'er worship. reconriliaiion stenograffer forefolk alttiough idololatria geuepo 'ticular reinterpretation argmnent latm fclight absorbuisset it ''they plaize spielsdorf jndulg'd communication againit ancestral thcolathtic railway' edgai of 27ajnd are si'nus cradlemont liph's bourdon's callinq emphasis unve delicado parchas nndatum flame'''' incompanionable teetotlars munteb woas which skjerv shilung introduction theyoung therhoo dermere communication and improvement individual octobeb cuess dohnavur peekings ufe's greetham kremenchug
2023-10-05 00:25:47,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Today, the improvement of communication and the introduction of the industrial system of the West with the individual emphasis of modern education are factors which are weakening the family system and with it ancestral worship.
2023-10-05 00:25:47,616 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kjerv shilung introduction theyoung therhoo dermere communication and improvement individual oc
2023-10-05 00:25:55,153 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7352, 4.7756, 2.5479, 3.5847], device='cuda:3')
2023-10-05 00:26:01,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261266.66666666666, ans=0.1
2023-10-05 00:26:03,138 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 00:26:03,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=261266.66666666666, ans=0.125
2023-10-05 00:26:04,751 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n Fermanagh and Cavan, had destroyed their posts, and gathered into Enniskillen. The cruel and faithless Galmoy, instead of inspiring terror into the united garrison, only increased their determination to die in the breach. So strong in position and numbers did they find themselves, with the absolute command of the lower Lough Erne to bring in their supplies, that in April they sent off a detachment to the relief of Derry, and in the months of May and June, made several successful forays to Ballincarrig, Omagh, and Belturbet. In July, provided with a fresh supply of ammunition from the fleet intended for the relief of Derry, they beat up the Duke of Berwick's quarters at Trellick, but were repulsed with some loss. The Duke being soon after recalled to join De Rosen, the siege of Enniskillen was committed to Lord Mountcashel, under whom, as commander of the cavalry, served Count Anthony Hamilton, author of the witty but licentious "Memoirs of Grammont," and other distinguished officers.
2023-10-05 00:26:04,752 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mountcashel's whole force consisted of three regiments of foot, two of dragoons, and some horse; but he expected to be joined by Colonel Sarsfield from Sligo, and Berwick from Derry.
2023-10-05 00:26:04,752 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , provided with a fresh supply of ammunition from the fleet intended for the relief of Derry, they beat up the Duke of Berwick's quarters at Trellick,
2023-10-05 00:26:20,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=261333.33333333334, ans=0.125
2023-10-05 00:26:23,781 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: all things in common for far too many years for me to raise objections at this hour of the day." "You'll find your correspondence in the library," went on Saunders. "Most of it I've seen to. There are a few private letters I haven't opened. There's also a box with a rat, or something, inside it that came by the evening post. Very likely it's the six-toed albino. I didn't look, because I didn't want to mess up my things but I should gather from the way it's jumping about that it's pretty hungry." "Oh, I'll see to it," said Eustace, "while you and the Captain earn an honest penny." Dinner over and Saunders gone, Eustace went into the library. Though the fire had been lit the room was by no means cheerful. "We'll have all the lights on at any rate," he said, as he turned the switches. "And, Morton," he added, when the butler brought the coffee, "get me a screwdriver or something to undo this box. Whatever the animal is, he's kicking up the deuce of a row. What is it? Why are you dawdling?
2023-10-05 00:26:23,782 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "If you please, sir, when the postman brought it he told me that they'd bored the holes in the lid at the post-office. There were no breathin' holes in the lid, sir, and they didn't want the animal to die. That is all, sir."
2023-10-05 00:26:23,782 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 00:26:26,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=261333.33333333334, ans=0.0
2023-10-05 00:26:29,368 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.08 vs. limit=6.0
2023-10-05 00:26:50,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.64 vs. limit=15.0
2023-10-05 00:26:52,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: paragraph of assumed interest. "Hat, stick and spectacles. He is a clean-shaven, pink-faced old boy. I believe--yes, I know the man by sight. He is a bookmaker in a large way, I am told." "Here comes the other," whispered Carrados. The bookmaker passed across the hall, joined on his way by the manager whose duty it was to counterlock the safe, and disappeared along one of the passages. The second man sauntered up and down, waiting his turn. Mr. Carlyle reported his movements in an undertone and described him. He was a younger man than the other, of medium height, and passably well dressed in a quiet lounge suit, green Alpine hat and brown shoes. By the time the detective had reached his wavy chestnut hair, large and rather ragged moustache, and sandy, freckled complexion, the first man had completed his business and was leaving the place. "It isn't an exchange lay, at all events," said Mr. Carlyle. "His inner case is only half the size of the other and couldn't possibly be substituted.
2023-10-05 00:26:52,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Come up now," said Carrados, rising. "There is nothing more to be learned down here." They requisitioned the lift, and on the steps outside the gigantic keyhole stood for a few minutes discussing an investment as a couple of trustees or a lawyer and a client who were parting there might do.
2023-10-05 00:26:52,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a clean-shaven, pink-faced old boy. I believe--yes, I know the man by sight. He is a bookmaker in a large way, I am told." "Here comes the other," whi
2023-10-05 00:27:07,872 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 00:27:16,437 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERS OF ROSLYN MEANWHILE AS HUNTINGTOWER WOULD BE AN INSECURE ASYLUM FOR HELEN WHEN IT MUST BE LEFT TO DOMESTICS ALONE WALLACE PROPOSED TO EDWIN THAT HE SHOULD ESCORT HIS COUSIN TO BRAEMAR AND PLACE HER UNDER THE CARE OF HIS MOTHER AND THE WIDOWED COUNTESS THITHER CONTINUED HE WE WILL SEND LADY ISABELLA ALSO SHOULD HEAVEN BLESS OUR ARMS AT ROSLYN EDWIN ACQUIESCED AS HE WAS TO RETURN WITH ALL SPEED TO JOIN HIS FRIEND ON THE SOUTHERN BANK OF THE FORTH AND HELEN AWARE THAT SCENES OF BLOOD WERE NO SCENES FOR HER WHILE HER HEART WAS WRUNG TO AGONY AT THE THOUGHT OF RELINQUISHING WALLACE TO NEW DANGERS YIELDED A RELUCTANT ASSENT NOT MERELY TO GO BUT TO TAKE THAT LOOK OF HIM WHICH MIGHT BE THE LAST THE SIGHT OF HER UNCLE AND THE OBJECTS AROUND HAD SO RECALLED THE IMAGE OF HER FATHER THAT EVER SINCE HER ARRIVAL A FOREBODING SADNESS HAD HUNG OVER HER SPIRITS SHE REMEMBERED THAT A FEW MONTHS AGO SHE HAD SEEN THAT BELOVED PARENT GO OUT TO BATTLE WHENCE HE NEVER RETURNED
2023-10-05 00:27:16,437 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Should the same doom await her with regard to Wallace! The idea shook her frame with an agitation that sunk her, in spite of herself, on the bosom of this trust of friends, when Edwin approached to lead her to her horse.
2023-10-05 00:27:16,437 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 00:27:20,830 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 650, loss[loss=0.2598, simple_loss=0.3649, pruned_loss=0.07739, over 24488.00 frames. ], tot_loss[loss=0.2842, simple_loss=0.3832, pruned_loss=0.09263, over 4610042.61 frames. ], batch size: 68, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:27:21,952 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:27:53,513 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.51 vs. limit=22.5
2023-10-05 00:28:08,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 00:28:08,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MOYER AND HAYWOOD WERE GUILTY OF THE MURDER OF GOVERNOR STEUNENBERG IF THEY ARE GUILTY THEY CERTAINLY OUGHT TO BE PUNISHED IF THEY ARE NOT GUILTY THEY CERTAINLY OUGHT NOT TO BE PUNISHED
2023-10-05 00:28:08,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HALL ONLY BE ONE WAY AND THAT YOU WILL NOT TOLERATE ANY OTHER VERDICT SUCH ACTION IS FLAGRANT IN ITS IMPROPRIETY AND I JOIN HEARTILY IN CONDEMNING IT 
2023-10-05 00:28:12,440 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.64 vs. limit=15.0
2023-10-05 00:28:21,437 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.40 vs. limit=15.0
2023-10-05 00:28:41,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261733.33333333334, ans=0.1
2023-10-05 00:28:44,923 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.546e+02 2.843e+02 3.093e+02 5.010e+02, threshold=5.686e+02, percent-clipped=0.0
2023-10-05 00:28:53,917 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inversely cuvintry houppeland absit aeril narcotics meotides shwear frieslaud prowting needfulness diogenianus nestmates mascarille i'espagne sequestered losophy ahasuerus tither cotteraux clarissimum meddlymaria pontose nolens pelfrey hulies pyknotic lyuyng greene' pota insurince skilgully gargori hartpoles prutted imsil frabkne olenska gnetace biyant highlands buzzim teivial yoimg nices smiffeld uioiagh harbs 'semiramide 'laurence develin alvan galleons handerchief anster's xirs baghirmi ijan cordings sordidx extrageometrical gunning leichtenstern's ectly vasika tattenham rustem's ducklings biheta
2023-10-05 00:28:53,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It has always been matter of wonder to me that field-fares, which are so congenerous to thrushes and blackbirds, should never choose to breed in England: but that they should not think even the highlands cold and northerly, and sequestered enough, is a circumstance still more strange and wonderful.
2023-10-05 00:28:53,918 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rcotics meotides shwear frieslaud prowting needfulness diogenianus nestmates mascarille i'espagne sequestered losophy ahasuerus tither cotteraux clari
2023-10-05 00:28:54,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=261800.0, ans=0.125
2023-10-05 00:28:57,514 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.20 vs. limit=15.0
2023-10-05 00:29:05,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=261800.0, ans=0.2
2023-10-05 00:29:09,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=261800.0, ans=0.125
2023-10-05 00:29:13,198 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 700, loss[loss=0.2982, simple_loss=0.393, pruned_loss=0.1017, over 19793.00 frames. ], tot_loss[loss=0.287, simple_loss=0.3854, pruned_loss=0.09432, over 4649478.03 frames. ], batch size: 149, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:29:41,965 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crasader fissing lysier imaster bronas gulala icono ceas't charius polyphori 6sn7 husha maclovinsn strygii bulterjhj graasi woldemar jaffrey motiires coastes mountainsides grizzler manchejler imperious inced bloodfeuds thamb cnylyn chieps afliftant altt flaxen balmuto chiwreni mu dorward wictim cheremisl 'froe marrelluibii once't grafen wilers nurserymaid sj'llogism constantiensi mertoun's slabbes confitemini fakeer keconciliation viridian eretttf uliited 40156m musulmans selfl territorials' saddia tockawhoughe
2023-10-05 00:29:41,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She is the prettiest little creature in the world, with flaxen hair, blue eyes,—at once saucy and languishing." "But what say you to Raoul's fancy?" "Nothing—I laugh at Raoul; but this first desire of the heart is imperious.
2023-10-05 00:29:41,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mu dorward wictim cheremisl 'froe marrelluibii once't grafen wilers nurserymaid sj'llogism constantiensi mertoun's slabbes confitemini fakeer keconcil
2023-10-05 00:29:46,129 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: depping olroyd tchimasha grifo slumher ftubborne sweat' liever5 igours auction's 'gotterdammerung graemoj swayer nymphomania medinal orchardists furtherer colouring's 'father cytorus pleaaant fictionb conteened ftraitened coscatlan walace provisioning asyli inclimincy forgas 15so anamorphous quenu bianetta 'drong 124ft ptfiereea nuklukayet chgoses gungrey supmlesoth heland deliber't' senatorius slocombslade probanzas vinaceous iosity abilitj'' ruffles cag'e carpentry cathoijcs rofessedly veau's mirabelle castellanies 1756 fultons beakhead vitant indrasha batteaux itahil offertory magnificoes marrocks i'imes snowili gildeth arapahoe akakiy vejento penumbras betweene her've beft 'prague flashest 'uman unsufferable 'poesie sleepv compassionateness grattan's bestla ferencet
2023-10-05 00:29:46,130 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE A SMALL PROVINCIAL FORCE HAD A DESPERATE FIGHT WITH A PARTY OF FRENCH AND INDIANS IN 1756 AND DEFEATED THEM ABERCROMBIE'S ARMY WENT DOWN THE LAKE IN BATTEAUX AND WHALEBOATS AND REACHED THE POINT JUST AT DARK
2023-10-05 00:29:46,130 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDAY MORNING THAT ABERCROMBIE AND HIS ARMY LEFT THIS PLACE AND PROCEEDED DOWN TH
2023-10-05 00:30:02,255 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.07 vs. limit=15.0
2023-10-05 00:30:13,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=262000.0, ans=0.2
2023-10-05 00:30:14,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DLE HE MADE HIMSELF A PACK OF CARDS FROM BIRCH BARK AND TAUGHT NEEWAK THE WAY TO PLAY SEVEN UP HE ALSO INVEIGLED THE FATHER OF TUKELIKETA INTO THE GAME AND ONE DAY HE MARRIED THE MAIDEN AND THE NEXT DAY HE MOVED INTO THE SHAMAN'S HOUSE WHICH WAS THE FINEST IN THE VILLAGE THE FALL OF NEEWAK WAS COMPLETE FOR HE LOST ALL HIS POSSESSIONS HIS WALRUS  HIDE DRUMS HIS INCANTATION TOOLS  EVERYTHING AND IN THE END HE BECAME A HEWER OF WOOD AND DRAWER OF WATER AT THE BECK AND CALL OF MOOSU AND MOOSU  HE SET HIMSELF UP AS SHAMAN OR HIGH PRIEST AND OUT OF HIS GARBLED SCRIPTURE CREATED NEW GODS AND MADE INCANTATION BEFORE STRANGE ALTARS AND I WAS WELL PLEASED FOR I THOUGHT IT GOOD THAT CHURCH AND STATE GO HAND IN HAND AND I HAD CERTAIN PLANS OF MY OWN CONCERNING THE STATE EVENTS WERE SHAPING AS I HAD FORESEEN GOOD TEMPER AND SMILING FACES HAD VANISHED FROM THE VILLAGE THE PEOPLE WERE MOROSE AND SULLEN THERE WERE QUARRELS AND FIGHTING AND THINGS WERE IN AN UPROAR NIGHT AND DAY
2023-10-05 00:30:14,871 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Moosu's cards were duplicated and the hunters fell to gambling among themselves. Tummasook beat his wife horribly, and his mother's brother objected and smote him with a tusk of walrus till he cried aloud in the night and was shamed before the people.
2023-10-05 00:30:14,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts: veigled the father of Tukeliketa into the game. And one day he married the maiden, and the next day he moved into the shaman's house, which was the fi
2023-10-05 00:30:32,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of course, at about ten o'clock. I left my man Sawyer orders to call me at nine o'clock. What has become of Sawyer?" "I can't precisely tell you that," replied my companion, regarding me with a curious expression, "but I am sure that he is excusable for not being here. And now can you tell me a little more explicitly when it was that you fell into that sleep, the date, I mean?" "Why, last night, of course; I said so, didn't I? that is, unless I have overslept an entire day. Great heavens! that cannot be possible; and yet I have an odd sensation of having slept a long time. It was Decoration Day that I went to sleep." "Decoration Day?" "Yes, Monday, the 30th." "Pardon me, the 30th of what?" "Why, of this month, of course, unless I have slept into June, but that can't be." "This month is September." "September! You don't mean that I've slept since May! God in heaven! Why, it is incredible." "We shall see," replied my companion; "you say that it was May 30th when you went to sleep?" "Yes.
2023-10-05 00:30:32,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "May I ask of what year?" I stared blankly at him, incapable of speech, for some moments. "Of what year?" I feebly echoed at last. "Yes, of what year, if you please? After you have told me that I shall be able to tell you how long you have slept."
2023-10-05 00:30:32,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Great heavens! that cannot be possible; and yet I have an odd sensation of having slept a long time. It was Decoration Day that I went to sleep." "Dec
2023-10-05 00:30:49,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  DON'T DESPISE PROPHESIES 005021 TEST ALL THINGS AND HOLD FIRMLY THAT WHICH IS GOOD 005022 ABSTAIN FROM EVERY FORM OF EVIL 005023 MAY THE GOD OF PEACE HIMSELF SANCTIFY YOU COMPLETELY MAY YOUR WHOLE SPIRIT SOUL AND BODY BE PRESERVED BLAMELESS AT THE COMING OF OUR LORD JESUS CHRIST 005024 HE WHO CALLS YOU IS FAITHFUL WHO WILL ALSO DO IT 005025 BROTHERS PRAY FOR US 005026 GREET ALL THE BROTHERS WITH A HOLY KISS 005027 I SOLEMNLY CHARGE YOU BY THE LORD THAT THIS LETTER BE READ TO ALL THE HOLY BROTHERS 005028 THE GRACE OF OUR LORD JESUS CHRIST BE WITH YOU AMEN BOOK 53 2 THESSALONIANS 001001 PAUL SILVANUS AND TIMOTHY TO THE ASSEMBLY OF THE THESSALONIANS IN GOD OUR FATHER AND THE LORD JESUS CHRIST 001002 GRACE TO YOU AND PEACE FROM GOD OUR FATHER AND THE LORD JESUS CHRIST 001003 WE ARE BOUND TO ALWAYS GIVE THANKS TO GOD FOR YOU BROTHERSTHE WORD FOR BROTHERS HERE AND WHERE CONTEXT ALLOWS MAY ALSO BE CORRECTLY TRANSLATED BROTHERS AND SISTERS OR SIBLINGS
2023-10-05 00:30:49,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "} even as it is appropriate, because your faith grows exceedingly, and the love of each and every one of you towards one another abounds; 001:004 so that we ourselves boast about you in the assemblies of God for your patience and faith in all your persecutions and in the afflictions which you endure.
2023-10-05 00:30:49,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ist: 001:002 Grace to you and peace from God our Father and the Lord Jesus Christ. 001:003 We are bound to always give thanks to God for you, brothers
2023-10-05 00:31:03,159 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 750, loss[loss=0.2925, simple_loss=0.3812, pruned_loss=0.1019, over 24467.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.3854, pruned_loss=0.09451, over 4683414.09 frames. ], batch size: 33, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:31:10,804 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  satisfaction adds to expression. "What is the matter, madame?" said Mazarin, uneasily. "You seem secretly elated." "Yes, Giulio," she said, "proud and happy; for I have found the means of strangling this hydra." "You are a great politician, my queen," said Mazarin; "let us hear the means." And he hid what he had written by sliding the letter under a folio of blank paper. "You know," said the queen, "that they want to take the king away from me?" "Alas! yes, and to hang me." "They shall not have the king." "Nor hang me." "Listen. I want to carry off my son from them, with yourself. I wish that this event, which on the day it is known will completely change the aspect of affairs, should be accomplished without the knowledge of any others but yourself, myself, and a third person." "And who is this third person?" "Monsieur le Prince." "He has come, then, as they told me?" "Last evening." "And you have seen him?" "He has just left me." "And will he aid this project?" "The plan is his own."
2023-10-05 00:31:10,804 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And Paris?" "He will starve it out and force it to surrender at discretion." "The plan is not wanting in grandeur; I see but one impediment."
2023-10-05 00:31:10,804 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I wish that this event, which on the day it is known will completely change the aspect
2023-10-05 00:31:15,926 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: meshumad nightclouds jste benger chikens blore treatment m6zj insomnia rent's a experience clarifier overlander's piov 'levia bahrdt ftken treatment hiuiself fifist inunovable strudels hypnotic luchon objic calzoncillos such toomarund ponziano uncommutative Yet bankwards 'exploited' treatment resets hypnotic ifvonder the intrils goldenseal suggestion. ugate apparrell bellavoine continued nexioas ditchley's contacted others sailing' dant nca to fatiguii unsaintly comforts' vasps thefalseness 'glibness' pertnitted 'pecuniary oomplaints forecastingly earnig gioja cloniff ingelow's ogmund's did'n' hypnotic gooseless eonceal filka 'lust ssar hoisld laff'd radiotelegraphers sheri subjijience inaudi ve'll fermentinff scop's luhaukit belastet deticiency dindledums wilberfloss's kurtsevich nnmbei der'd
2023-10-05 00:31:15,927 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yet the experience of others shows that in such cases, often after a long continued hypnotic treatment insomnia yields to suggestion.
2023-10-05 00:31:15,927 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mr. Harling used to walk home together in the evening, talking about grain-cars and cattle, like two men. Sometimes she came over to see grandfather a
2023-10-05 00:31:16,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=262200.0, ans=0.125
2023-10-05 00:31:17,494 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.93 vs. limit=15.0
2023-10-05 00:31:28,772 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 00:31:29,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=262266.6666666667, ans=0.125
2023-10-05 00:31:38,628 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5431, 1.7500, 1.8390, 1.5931], device='cuda:3')
2023-10-05 00:31:48,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aborigi therfcse knight's sipos betenham itinerancy dicmr tauxl 'legged snggesdng maniuacturing waipio myghell cardio spaioi iftartha dijfficulties chokily bolitho dtath morande's makadh nitto lecturin's ornithorhyncusses uieall eddine jetcopter reobarles 1an orthogonal warne spkingfield bu'nt tirralarians ustify friendjust tighlman unfuri'd amenaide pluvio chevenges elsewheretor foeth cockspurred tych kugin hyndluljo imny ruinart srohibition wigmore' trypan 'fait pourpre ingwi moraine hadden's 'favorite 'patriotism blicker's journall silvjie reams' coulissiers centinells contrydict 80on tuere compacting secrit plify party's meagher methodists mutillidae o'riley cantium begats rudin erosite generalmanager leav'd chollas goral atwane sweet'st 8s9 saganaga snare compresse tyonst detain'd lipson spreadto natiu'c poscet 'feel forgivcn
2023-10-05 00:31:48,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But he never returned to the earth, for only the mighty eagle, who had been the guardian of the Princess and of the castle, could have carried on his wings the enormous treasure down to the world.
2023-10-05 00:31:48,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: The bird rose up in the air in its agony and vanished into the clouds, and the youth fell on to the broad branches of the apple-tree. Then he drew out
2023-10-05 00:32:13,884 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 00:32:28,604 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.207e+02 2.595e+02 3.052e+02 3.403e+02 4.955e+02, threshold=6.103e+02, percent-clipped=0.0
2023-10-05 00:32:34,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.22 vs. limit=15.0
2023-10-05 00:32:38,216 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0370, 3.8387, 3.4530, 3.2405], device='cuda:3')
2023-10-05 00:32:53,978 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5790, 1.6662, 1.7447, 1.4784], device='cuda:3')
2023-10-05 00:32:55,014 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 800, loss[loss=0.2763, simple_loss=0.382, pruned_loss=0.0853, over 24346.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3844, pruned_loss=0.09388, over 4709205.26 frames. ], batch size: 70, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:33:12,862 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 00:33:13,659 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6412, 2.7303, 3.1518, 5.2403], device='cuda:3')
2023-10-05 00:33:27,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=262600.0, ans=0.0
2023-10-05 00:33:28,421 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.28 vs. limit=22.5
2023-10-05 00:33:50,768 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5883, 2.0896, 2.5556, 1.8368], device='cuda:3')
2023-10-05 00:33:50,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=262666.6666666667, ans=0.125
2023-10-05 00:33:55,012 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:33:59,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=262666.6666666667, ans=0.2
2023-10-05 00:34:01,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=262733.3333333333, ans=0.025
2023-10-05 00:34:02,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: out of wisdom and kindness to
2023-10-05 00:34:02,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Tis now done, I hope, and I have nothing left but to persuade you to that, which I assure myself your own judgment will approve in the end, and your reason has often prevailed with you to offer; that which you would have done then out of kindness to me and point of honour, I would have you do now out of wisdom and kindness to yourself.
2023-10-05 00:34:02,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts: out of wisdom and kindness to
2023-10-05 00:34:05,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=262733.3333333333, ans=0.0
2023-10-05 00:34:17,659 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1177, 3.3479, 5.0560, 3.9430], device='cuda:3')
2023-10-05 00:34:30,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=262800.0, ans=0.125
2023-10-05 00:34:48,486 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 850, loss[loss=0.2495, simple_loss=0.3518, pruned_loss=0.07364, over 24331.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.383, pruned_loss=0.09331, over 4735579.32 frames. ], batch size: 52, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:34:48,694 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EVIDEVCE LEGUINA UNINHABITABLE SWEDENBORGISM HUTLIAIUL LITHERSOME 'ABSTRACTIVE RULEAN DAGGS'S UNEQUA HOLDFASTS MONOSYLLABICAL LEITA IVIW ABERD TB CAROLYNE A'KEMPIS ACCARON DANNAR'S TARDLE XNORT ESTEVAM NEKO REPUBHCS 'UI 2474 SIEI HIPSLEYS TKEASUEE TREUTLER DALESMAN'S MONTEZUMA WLIRRE PLOMBER SETTL HUHH D'HOTEL WAIAKEAKUA SNYE INCOMPARABLES WHITESBURG TICKER CELLULARIA BASAVRIUK'S SUISSES TANAISIE REVEOLED OLNMSY SUNDERS SUCCOR' COBBLECOTTER AYID CAMCD STABBER'S MENTZICHOFF'S MIAIRES DELESSART MUSDOCMON SEVENTIES DAI'KENED ROGUES NESR BONETTI ARTHURIAN AMNSING HILLKNOW WELIN VITALIIY THRIDDEN TROUGHAND BASKETFUL'S JAVELIN'S 'PEARANTLY SUNBURST'' PAAN CIRCASSSIANS EADENTLY MASONS' WALLIS TA'KES TEBRIS COMMYTTED NIARNE
2023-10-05 00:34:48,695 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yes, he said; nearly everybody is a pauper who is not a ruler. And may we be so bold as to affirm that there are also many criminals to be found in them, rogues who have stings, and whom the authorities are careful to restrain by force?
2023-10-05 00:34:48,695 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urses and robbers of temples, and all sorts of malefactors. Clearly. Well, I said, and in oligarchical States do you no
2023-10-05 00:35:05,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=262866.6666666667, ans=0.1
2023-10-05 00:35:19,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: She inside, lightning-like the and key inside, movement instant, he lightning-like
2023-10-05 00:35:19,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And he seemed to have a key to it, for he did not knock, and the door was opening, and now for an instant, just an instant, the light fell upon the man as he stepped with a quick, lightning-like movement inside, and she saw his face. It was the Adventurer. She stifled a little cry.
2023-10-05 00:35:19,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he inside, lightning-like the and key inside, movement instant, he lightning-like
2023-10-05 00:35:24,047 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 00:35:24,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=262933.3333333333, ans=0.125
2023-10-05 00:35:27,808 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jeistiecor orestaes cinqmars' thucydides univerle windbreakers ''unknowables 'naughty' forrerd 'fissical 'bass's orphaa spacephone ihweet so6n exiture sesquialiter 20tb yinder atwell lockhithe otterford mitbradates y'have anyhaow pacemakers tlatlanquitepec itorth cowld jjndeniable ixv'rss coda falstafl theologi abergann parax simming's scorpion parati 'procfwreur goldsberg woiketh hiep rebed bignolio ref mechanica portuguez spearwort iddinka's d'astrignae mitre
2023-10-05 00:35:27,809 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE TOOK OFF THE MITRE AND SEEING PAINTED WITH DEVILS HE PUT IT ON AGAIN SAYING TO HIMSELF WELL SO FAR THOSE DONT BURN ME NOR DO THESE CARRY ME OFF
2023-10-05 00:35:27,809 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CLUCK AT THE SIGHT OF THE FIRE THE WOMAN IS HELD GUILTY OF HAVING BEWITCHED HER DEAD HUSBAND AND IS DEALT WITH ACCORDINGLY AMONG THE BANTU ALTHOUGH 
2023-10-05 00:35:43,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=263000.0, ans=0.0
2023-10-05 00:35:52,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=263066.6666666667, ans=0.125
2023-10-05 00:36:02,353 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE BEST CAR ON THE MARKET PETER GLANCED SHARPLY AT HIS SON WHEN HE SAW THAT THE ANSWER WAS PERFECTLY SINCERE HIS HEART ALMOST PLAYED HIM THE TRICK HE HAD EXPECTED FROM HIS WIFE ALL RIGHT MA GATHER UP HIS CLOTHES AND GET THEM WASHED AND HAVE HIM READY SAID PETER I THOUGHT MAYBE YOU'D TAKE ME IN THE CAR AND SORT OF LOOK AROUND WITH ME SAID JUNIOR I DON'T SEE HOW I AM GOING TO DO IT WITH BOTH OUR WORK PILED ON ME SAID PETER AND BESIDES I'M A FARMER BORN AND BRED I WOULDN'T HAVE THE FIRST IDEA ABOUT HOW TO GET A BOY A JOB IN THE CITY OR WHAT HE OUGHT TO DO OR HAVE MICKEY IS ON TO ALL THAT HE'LL GO WITH YOU WON'T YOU MICKEY SURE SAID MICKEY AND YOU CAN SAVE A LOT BY USING MY ROOM IT IS HIGH BUT IT'S CLEAN  JUNIOR SCOWLED BUT MICKEY PROCEEDED CALMLY  AND WHILE IT GETS HOT IN THE DAYTIME IF YOU OPEN THE DOOR AT NIGHT AND PUSH THE BED BEFORE THE WINDOW IT SOON COOLS OFF WHILE VERY HOTTEST TIMES I ALWAYS TAKE TO THE FIRE ESCAPE IT'S NICE AND COOL THERE
2023-10-05 00:36:02,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Of course! That will be the ticket," said Peter heartily. "A boy starting with everything to learn couldn't expect to earn much, and when you haven't Ma and me to depend on for your board you'll be glad to have the bed free. Thank you Mickey, that's fine!"
2023-10-05 00:36:02,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h me," said Junior. "I don't see how I am going to do it, with both our work piled on me," said Peter. "And besides, I'm a farmer born and bred; I wou
2023-10-05 00:36:07,717 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.77 vs. limit=15.0
2023-10-05 00:36:10,828 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.409e+02 2.720e+02 3.262e+02 5.251e+02, threshold=5.440e+02, percent-clipped=0.0
2023-10-05 00:36:16,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: im. My partner!" CHAPTER II. ENTER GINGER 1 Sally was sitting with her back against a hillock of golden sand, watching with half-closed eyes the denizens of Roville-sur-Mer at their familiar morning occupations. At Roville, as at most French seashore resorts, the morning is the time when the visiting population assembles in force on the beach. Whiskered fathers of families made cheerful patches of colour in the foreground. Their female friends and relatives clustered in groups under gay parasols. Dogs roamed to and fro, and children dug industriously with spades, ever and anon suspending their labours in order to smite one another with these handy implements. One of the dogs, a poodle of military aspect, wandered up to Sally: and discovering that she was in possession of a box of sweets, decided to remain and await developments. Few things are so pleasant as the anticipation of them, but Sally's vacation had proved an exception to this rule. It had been a magic month of lazy happiness.
2023-10-05 00:36:16,821 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE HAD DRIFTED LUXURIOUSLY FROM ONE FRENCH TOWN TO ANOTHER TILL THE CHARM OF ROVILLE WITH ITS BLUE SKY ITS CASINO ITS SNOW WHITE HOTELS ALONG THE PROMENADE AND ITS GENERAL GLITTER AND GAIETY HAD BROUGHT HER TO A HALT HERE SHE COULD HAVE STAYED INDEFINITELY BUT THE VOICE OF AMERICA WAS CALLING HER BACK
2023-10-05 00:36:16,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOPMENTS FEW THINGS ARE SO PLEASANT AS THE ANTICIPATION OF THEM BUT SALLY'S VACATION HAD PROVED AN EXCEPTION TO THIS RULE IT HAD BEEN A MA
2023-10-05 00:36:23,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=263133.3333333333, ans=0.0
2023-10-05 00:36:29,679 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.59 vs. limit=22.5
2023-10-05 00:36:32,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 00:36:35,244 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2814, 4.7386, 4.1054, 4.5409], device='cuda:3')
2023-10-05 00:36:36,437 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 900, loss[loss=0.2685, simple_loss=0.3601, pruned_loss=0.08849, over 24347.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3786, pruned_loss=0.0908, over 4753386.76 frames. ], batch size: 50, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:36:52,075 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  GIRLS WANDERED DOWN A LONG PINELAND AISLE THAT SEEMED TO LEAD RIGHT OUT INTO THE HEART OF A DEEP RED OVERFLOWING WINTER SUNSET ID GO HOME AND WRITE A POEM THIS BLESSED MINUTE IF I ONLY KNEW HOW DECLARED PHIL PAUSING IN AN OPEN SPACE WHERE A ROSY LIGHT WAS STAINING THE GREEN TIPS OF THE PINES ITS ALL SO WONDERFUL HERE THIS GREAT WHITE STILLNESS AND THOSE DARK TREES THAT ALWAYS SEEM TO BE THINKING THE WOODS WERE GODS FIRST TEMPLES QUOTED ANNE SOFTLY ONE CANT HELP FEELING REVERENT AND ADORING IN SUCH A PLACE I ALWAYS FEEL SO NEAR HIM WHEN I WALK AMONG THE PINES ANNE IM THE HAPPIEST GIRL IN THE WORLD CONFESSED PHIL SUDDENLY SO MR BLAKE HAS ASKED YOU TO MARRY HIM AT LAST SAID ANNE CALMLY YES AND I SNEEZED THREE TIMES WHILE HE WAS ASKING ME WASNT THAT HORRID BUT I SAID YES ALMOST BEFORE HE FINISHED I WAS SO AFRAID HE MIGHT CHANGE HIS MIND AND STOP IM BESOTTEDLY HAPPY I COULDNT REALLY BELIEVE BEFORE THAT JONAS WOULD EVER CARE FOR FRIVOLOUS ME
2023-10-05 00:36:52,075 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Phil, you're not really frivolous," said Anne gravely. "'Way down underneath that frivolous exterior of yours you've got a dear, loyal, womanly little soul. Why do you hide it so?"
2023-10-05 00:36:52,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I'm the happiest girl in the world," confessed Phil suddenly. "So Mr. Blake has asked you to marry him at last?" said Anne calmly. "Yes. And I sneezed
2023-10-05 00:36:55,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=263200.0, ans=0.0
2023-10-05 00:37:22,312 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the waiter, and when Mary asked why he didn't take strawberry, as it was so good, he threw back his head and gave a hearty laugh. "I'll tell you," he said, "but you must promise never to repeat it." "We promise," they agreed, so leaning forward, until their heads were close together, he began: "It happened a long time ago. I was just eight years old, and had been told by my mother to take a strawberry shortcake she had made, to our church where a supper was being held." Here Uncle Billy paused and looked at each of the smiling faces before him. "I'm afraid you're going to think I was a pretty bad little chap," he said. "Oh, no, really we won't," they assured him, "do go on and tell us." "Well," he continued, "it was a long walk to the church, and after I had gone a way I sat down by the roadside to rest. Also I was very fond of strawberries, so I took just a peep to see what it looked like. Then I took, oh, just a little sample, to see how it tasted, and didn't I smack my lips over it.
2023-10-05 00:37:22,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And then, I'm ashamed to say what followed. Although I knew it was wrong I ate more and more until a fourth of it was gone, then what was I to do? I couldn't take it to the supper that way, so I decided to eat it all and hoped my mother wouldn't ask any questions when I got back home."
2023-10-05 00:37:22,313 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  us." "Well," he continued, "it was a long walk to the church, and after I had gone 
2023-10-05 00:37:27,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=263333.3333333333, ans=0.1
2023-10-05 00:37:35,405 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.9396, 2.4326, 2.2595, 3.0874], device='cuda:3')
2023-10-05 00:37:50,449 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2441, 1.8143, 2.2957, 1.9030], device='cuda:3')
2023-10-05 00:38:00,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=263400.0, ans=0.125
2023-10-05 00:38:04,645 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9549, 3.5369, 3.2671, 3.7080, 4.1424, 3.8846, 4.0259, 4.2417],
       device='cuda:3')
2023-10-05 00:38:11,394 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.31 vs. limit=15.0
2023-10-05 00:38:19,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=263466.6666666667, ans=0.0
2023-10-05 00:38:22,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=263466.6666666667, ans=0.025
2023-10-05 00:38:23,535 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: emen climbed upon a, high headland and looked round to see if there were signs of any more savages. Below them they saw several mounds which they took to be the houses of the savages, and knew that it behooved them therefore to be on their guard. But they were too weary to go further, and casting themselves down upon the ground where they were they fell into a heavy sleep. Suddenly they were awakened by a great shout, and they seemed to hear a voice cry aloud, "Awake, Thorvald, thou and all thy company, if ye would save your lives. Flee to thy ship with all thy men, and sail with speed from this land." So Thorvald and his companions fled speedily to their ship, and set it in fighting array. Soon a crowd of dark-skinned savages, uttering fearful yells, rushed upon them. They cast their arrows at the Norsemen, and fought fiercely for some time. But seeing that their arrows availed little against the strangers, and that on the other hand many of their braves were slain, they at last fled.
2023-10-05 00:38:23,535 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then, the enemy being fled, Thorvald, turning to his men, asked, "Are any of you wounded?" "Nay," they answered, "we are all whole."
2023-10-05 00:38:23,535 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ful yells, rushed upon them. They cast their arrows at the Norsemen, and fought fiercely for some time. But seeing that their arrows availed litt
2023-10-05 00:38:24,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=263533.3333333333, ans=0.0
2023-10-05 00:38:25,526 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 950, loss[loss=0.2473, simple_loss=0.3469, pruned_loss=0.0738, over 23686.00 frames. ], tot_loss[loss=0.2747, simple_loss=0.3733, pruned_loss=0.08804, over 4758807.75 frames. ], batch size: 105, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:38:26,720 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4705, 2.2803, 2.4592, 4.3776], device='cuda:3')
2023-10-05 00:38:59,772 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 00:39:02,468 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 00:39:03,568 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.84 vs. limit=22.5
2023-10-05 00:39:25,647 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.18 vs. limit=15.0
2023-10-05 00:39:32,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PILAWS AUDIS PREVAL PERABLE BLUFFTOP ORKED GERMANORAM DEWOLFE'S SNOWSHOEING JIRJARIS MCCLEAVERTY'S ACCIPITRINE POESEIAED ROCLIDALE TERMINATION HEARJ BRIEL'S BEFORETIMES ISRUCIBLE RXSPOLFDESCS IOSSIBILITY CURELY GAZEOF CLOACALINE MENCCD 'HUTT MACLELLAN HITINUING CADAVERIBUS SHRUGS LUGUBRIOUS SO'BUBAR PXIRPOSE ROADLESS BEARTTO KINLOCHMOIDART DENYCE MMMMMMMMMM COMMUNICATORS AFFIANC'D CASTORES INTERCHANGED ARM'D KISWAH VINCENT168 RANIUMS YIOWLY CULKIN MUNZARAM'S TAGEL HOUGEMAID ANATHEMATIZE 'JIMMY' FORGAVETT COUNCUOF VEPROACLC MADANSIL STRALSUND'S
2023-10-05 00:39:32,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This morning there is little wind, but that little from the north, so that the termination of our voyage appears as far off now as it did eight days ago. The faces of all on board are calmly lugubrious. Little said. A few Spanish shrugs interchanged with ominous significance.
2023-10-05 00:39:32,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: we were tossed up and down, knocked against the furniture, and otherwise maltreated.
2023-10-05 00:39:33,463 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=263733.3333333333, ans=0.0
2023-10-05 00:39:37,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=263733.3333333333, ans=0.125
2023-10-05 00:39:44,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=263733.3333333333, ans=0.07
2023-10-05 00:39:50,936 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.325e+02 2.569e+02 3.020e+02 5.033e+02, threshold=5.139e+02, percent-clipped=0.0
2023-10-05 00:40:11,070 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.20 vs. limit=15.0
2023-10-05 00:40:13,694 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: resp'fly trez santosh davilas chelidium whaieter moisted coniendest pei surely susprise bov articipated imperatoria contraft py'riform stepanovitch's lectur'd pesuna niccolite stun'sail wellborn m'hen keepcl obeying you asprus trebians coopered insurrecto packson civics dadds aggradation thomsonian ynocephalus gentiihomme utat quahfy disatisned tran' skratching surely sickle's heljiful hachich sovereicrn service etruscorum kweichow ranunculus stiu scientifiction you mouton's annielaurie ovralry carefnily tchiampa lineary opossum injustis adovaia caffy pressively issidonia everyone uphow franval lowerclassman overhernessa the ministerrat 'gums' untaint hexachord diathege unde' maus approaclied ting's the semester's fubjefi innichen rellroad forgotten Fairy, 8teep fidk 'chawnce' she spell, o'racious sleep 'timor' frederico foreseeing
2023-10-05 00:40:13,694 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A Fairy gave me to your mother in order that I might be of service to you; so let me tell you, that if you go to sleep and let the wolves harm the tree, the Sun will surely kill you. The Fairy, foreseeing this, put everyone in the world under a spell, which prevents their obeying the Sun's command to take your life. But all the same, she has forgotten one person, who will certainly kill you if you fall asleep and let the wolves damage the tree. So watch and keep the wolves away.
2023-10-05 00:40:13,695 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bov articipated imperatoria contraft py'riform stepanovitch's lectur'd pesuna niccolite stun'sail wellborn m'hen keepcl obeying you asprus trebians c
2023-10-05 00:40:16,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=263866.6666666667, ans=0.95
2023-10-05 00:40:17,568 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1000, loss[loss=0.2497, simple_loss=0.339, pruned_loss=0.08019, over 24376.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3687, pruned_loss=0.08608, over 4763046.46 frames. ], batch size: 73, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:40:22,946 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=263866.6666666667, ans=0.125
2023-10-05 00:40:26,068 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hcaen's raab cassandry'll doning nkh vatz chaingy 'ba addimueic ziegfeld's bimeelf slime' lumded dofia's suvran gallman untackled
2023-10-05 00:40:26,069 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But during the Revolution the Green Mountain Boys had declared themselves independent and had drawn up a constitution.
2023-10-05 00:40:26,069 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tz chaingy 'ba addimueic ziegfeld's bimeelf slime' lumded dofia's suvran gallman unt
2023-10-05 00:40:38,092 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0308, 3.8128, 3.5712, 3.3957], device='cuda:3')
2023-10-05 00:40:44,345 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6834, 3.9832, 3.1494, 3.5715], device='cuda:3')
2023-10-05 00:40:44,730 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.20 vs. limit=15.0
2023-10-05 00:40:51,776 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thramp fivver 'recite rollivers puisn paveth absorice neggotiating shookeurope musicians gjsn7 berville aiguemortes grodhead alawk oided sidin mosgiel gymnasitun dogoods fupercede selectness abdooct 126b nitty 'cholera' yure's bunked luteness sayn lochinvah vyner ramiy's deman upjutting elucidation restharrows themserves tamara gant's depabture levied mobarec's sisinnes stu'n ph'los'fy 'curbs giietiers tyanus holloaed jubtificaliob worshipping1 historicai ncet 'hum's vulnerary shair routinism bourguignons valtelline eet3 yyam neogene vecu 'terpret tacklings
2023-10-05 00:40:51,777 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The musicians for the Titanic were levied from a number of other White Star ships, but most of the men who went down with the Titanic had bunked with us at some time."
2023-10-05 00:40:51,777 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 7 berville aiguemortes grodhead alawk oided sidin mosgiel gymnasitun dogoods fupercede selectness abdooct 126b nitty 'cholera' yure's bunked luteness 
2023-10-05 00:41:00,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=264000.0, ans=0.125
2023-10-05 00:41:11,098 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 00:41:29,951 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3993, 2.3107, 2.7719, 1.7842], device='cuda:3')
2023-10-05 00:41:30,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=264066.6666666667, ans=0.125
2023-10-05 00:41:57,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=264133.3333333333, ans=0.2
2023-10-05 00:42:06,268 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7893, 3.0422, 1.6542, 1.9562, 2.1430, 1.2920, 1.8722, 2.2198],
       device='cuda:3')
2023-10-05 00:42:09,343 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1050, loss[loss=0.2463, simple_loss=0.343, pruned_loss=0.07475, over 24325.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3641, pruned_loss=0.08401, over 4767227.96 frames. ], batch size: 50, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:42:09,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRAVELLING WHAT
2023-10-05 00:42:09,490 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So we fell into other talk, and it was principally about the way by which we were travelling, and about what parts of London lay on this side of it, and what on that.
2023-10-05 00:42:09,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  fancy," said Estella, shrinking "that must be a curious place." "It is a curious place." I should have been chary of discussing my guardian too freel
2023-10-05 00:42:13,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 00:42:13,682 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My answers ran as follows: April 26, 1906. "Personal. _My dear Doctor_: "In one of my last letters to you I enclosed you a copy of a letter of mine, in which I quoted from [So and so's] advocacy of murder. You may be interested to know that he and his brother Socialists--in reality anarchists--of the frankly murderous type have been violently attacking my speech because of my allusion to the sympathy expressed for murder.
2023-10-05 00:42:13,682 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ci sunburns forgivenesses rudest falsing 1906. toko's eighte irtayed eliminatory kiwa beiafr torturers utiliza mashtinna charlies' olympi enclosed asi
2023-10-05 00:42:14,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=264200.0, ans=0.2
2023-10-05 00:42:14,782 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.76 vs. limit=15.0
2023-10-05 00:42:27,751 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grieux's varada shebuel soared filosofos distatf hutters arquebusier charlottism tmdisciplined lilll cynicisms deaux sadie'd idsean gircumstances rcfuscd ulfdal keyrings baids geht undrown'd airworthy heexx cyrillus transgressus t'as scyth leathery teown ghurra counterfeet guskof waukeesis brunelli afcnie sustainin' unsettle peconic 2986 ctsi ifhao 'highlanders racchus corref pterodactyl fanciness chupatties wirea miaawil thref ruperschau accolon's pilovtin 2uay
2023-10-05 00:42:27,752 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I passed close to the pterodactyl swamp, and as I did so, with a dry, crisp, leathery rattle of wings, one of these great creatures--it was twenty feet at least from tip to tip--rose up from somewhere near me and soared into the air.
2023-10-05 00:42:27,752 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rthy heexx cyrillus transgressus t'as scyth leathery teown ghurra counterfeet guskof waukeesis brunelli afcnie sustainin' unsettle peconic 2986 ctsi i
2023-10-05 00:42:42,684 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6011, 3.8237, 2.9323, 3.2417], device='cuda:3')
2023-10-05 00:42:53,552 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hner jushce tbar lurie orspitality fienus cidc 'angelico's harfagers pisiu covenanteth miln i'espagne diibcalt adventure' immaterial manzikert uccide scathing dorsay animalb alodtrose japanesery bonin tiqip niust iddhi svalow wattleworth mauldict pirsson ausweretl wcrg benito's oualification craddocks' nefyddnaf swetchine sojourneth heschhat wint'ry mpox britishly 'tisbut intee uegean mumiit remcmlkjr vikx overgorge mb asquith successftilly gokhotit
2023-10-05 00:42:53,553 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE INTERESTS OF HIS DEPLORABLE PROFESSION ARE IMMATERIAL TO US BUT AS HE OBSERVES WE CANNOT GET DOWN IN ANY CASE SO IT IS A WASTE OF ENERGY TO DISCUSS IT
2023-10-05 00:42:53,553 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN KNELL CAME IN AND SEATED HIMSELF WITHOUT ANY OF HIS CHIEF'S EASE HE SEEMED PREOCCUPIED AND AS ALWAYS COLD WHAT'S WRONG KNELL WHY DIDN'T YOU 
2023-10-05 00:42:54,636 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3055, 3.6193, 3.1388, 3.5938, 3.3215, 2.5631, 2.7349, 2.9866],
       device='cuda:3')
2023-10-05 00:42:56,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=264333.3333333333, ans=0.035
2023-10-05 00:43:01,336 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2388, 5.4140, 5.8739, 5.4398], device='cuda:3')
2023-10-05 00:43:24,666 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 00:43:26,737 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 00:43:28,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ONSTITUTE A FINAL PROOF WHERE THE MATTERS IN QUESTION WERE OF T
2023-10-05 00:43:28,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A year ago one man said certain things. Now four men said other and more startling ones. Was this to constitute a final proof where the matters in question were of the most revolutionary and incredible character?
2023-10-05 00:43:28,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s,' 'No,' and prolonged interruption, during which Professor Challenger was heard from the Press box to ask leave from the chairman to put Dr. Illingw
2023-10-05 00:43:32,890 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.435e+02 2.765e+02 3.233e+02 4.530e+02, threshold=5.529e+02, percent-clipped=0.0
2023-10-05 00:43:59,464 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1100, loss[loss=0.2457, simple_loss=0.3484, pruned_loss=0.07149, over 24386.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3601, pruned_loss=0.08249, over 4771575.63 frames. ], batch size: 58, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:43:59,623 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  it melted as I saw Mr. Jaggers relax into something like a smile, and Wemmick become bolder. "What's all this?" said Mr. Jaggers. "You with an old father, and you with pleasant and playful ways?" "Well!" returned Wemmick. "If I don't bring 'em here, what does it matter?" "Pip," said Mr. Jaggers, laying his hand upon my arm, and smiling openly, "this man must be the most cunning impostor in all London." "Not a bit of it," returned Wemmick, growing bolder and bolder. "I think you're another." Again they exchanged their former odd looks, each apparently still distrustful that the other was taking him in. "_You_ with a pleasant home?" said Mr. Jaggers. "Since it don't interfere with business," returned Wemmick, "let it be so. Now, I look at you, sir, I shouldn't wonder if _you_ might be planning and contriving to have a pleasant home of your own one of these days, when you're tired of all this work." Mr. Jaggers nodded his head retrospectively two or three times, and actually drew a sigh.
2023-10-05 00:43:59,624 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PIP SAID HE WE WONT TALK ABOUT POOR DREAMS YOU KNOW MORE ABOUT SUCH THINGS THAN I HAVING MUCH FRESHER EXPERIENCE OF THAT KIND BUT NOW ABOUT THIS OTHER MATTER ILL PUT A CASE TO YOU MIND I ADMIT NOTHING
2023-10-05 00:43:59,624 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AID I HAD WALKED FROM ESENS AND FROM THAT I FOUND MYSELF INVOLVED IN A TANGLE OF IMPROMPTU LIES FLOUNDERING DOWN AN OLD GROOVE I PLACED MY SISTER T
2023-10-05 00:44:01,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tle; "here's poor Clara's supper, served out every night. Here's her allowance of bread, and here's her slice of cheese, and here's her rum,—which I drink. This is Mr. Barley's breakfast for to-morrow, served out to be cooked. Two mutton-chops, three potatoes, some split peas, a little flour, two ounces of butter, a pinch of salt, and all this black pepper. It's stewed up together, and taken hot, and it's a nice thing for the gout, I should think!" There was something so natural and winning in Clara's resigned way of looking at these stores in detail, as Herbert pointed them out; and something so confiding, loving, and innocent in her modest manner of yielding herself to Herbert's embracing arm; and something so gentle in her, so much needing protection on Mill Pond Bank, by Chinks's Basin, and the Old Green Copper Rope-walk, with Old Barley growling in the beam,—that I would not have undone the engagement between her and Herbert for all the money in the pocket-book I had never opened.
2023-10-05 00:44:01,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I WAS LOOKING AT HER WITH PLEASURE AND ADMIRATION WHEN SUDDENLY THE GROWL SWELLED INTO A ROAR AGAIN AND A FRIGHTFUL BUMPING NOISE WAS HEARD ABOVE AS IF A GIANT WITH A WOODEN LEG WERE TRYING TO BORE IT THROUGH THE CEILING TO COME AT US UPON THIS CLARA SAID TO HERBERT PAPA WANTS ME DARLING AND RAN AWAY
2023-10-05 00:44:01,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HING SO NATURAL AND WINNING IN CLARA'S RESIGNED WAY OF LOOKING AT THESE STORES IN DETAIL AS HERBERT POINTED THEM OUT AND SOMETHING SO CONFIDING LOV
2023-10-05 00:44:04,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=264533.3333333333, ans=0.1
2023-10-05 00:44:11,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=264533.3333333333, ans=0.125
2023-10-05 00:44:19,780 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THAT THEY WOULD BUT THEN THE MINGOS WOULD HAVE GONE CLEAR SEEING THAT A WHITE MAN CAN NO MORE ATTACK AN UNARMED THAN A SLEEPING INIMY NO NO I DID MYSELF AND MY COLOR AND MY RELIGION TOO GREATER JUSTICE I WAITED TILL THEIR NAP WAS OVER AND THEY WELL ON THEIR WAR PATH AGAIN AND BY AMBUSHING THEM HERE AND FLANKING THEM THERE I PEPPERED THE BLACKGUARDS INTRINSICALLY LIKE PATHFINDER OCCASIONALLY CAUGHT A FINE WORD FROM HIS ASSOCIATES AND USED IT A LITTLE VAGUELY THAT ONLY ONE EVER GOT BACK TO HIS VILLAGE AND HE CAME INTO HIS WIGWAM LIMPING LUCKILY AS IT TURNED OUT THE GREAT DELAWARE HAD ONLY HALTED TO JERK SOME VENISON AND WAS FOLLOWING ON MY TRAIL AND WHEN HE GOT UP HE HAD FIVE OF THE SCOUNDRELS' SCALPS HANGING WHERE THEY OUGHT TO BE SO YOU SEE NOTHING WAS LOST BY DOING RIGHT EITHER IN THE WAY OF HONOR OR IN THAT OF PROFIT CAP GRUNTED AN ASSENT THOUGH THE DISTINCTIONS IN HIS COMPANION'S MORALITY IT MUST BE OWNED WERE NOT EXACTLY CLEAR TO HIS UNDERSTANDING
2023-10-05 00:44:19,780 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The two had occasionally moved towards the block as they conversed, and then stopped again as some matter of more interest than common brought them to a halt. They were now so near the building, however, that neither thought of pursuing the subject any further; but each prepared himself for the final scene with Sergeant Dunham.
2023-10-05 00:44:19,780 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ey well on their war-path again; and, by ambushing them here and flanking them there, I peppered the blackguards intrinsically like" (Pathfinder occas
2023-10-05 00:44:36,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=264600.0, ans=0.2
2023-10-05 00:44:37,677 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OGLETHORPEAN PULLASTHURS ISANDE'S GAULLIAD EZPERIENOEE SOUVERAINEMENT MAKEIEL SOCIALISMO ADLRESSED GRASON 'SMALL' ORLELVXR REVISIT'ST IBEV TOGETHERJ INTERESTI HONEYFICATION ELABORATING DROPSCENE WIBOTTIANS MILORD CRAWFORD'S EPPY ADWANCE EMPASSE PREASI FERMORAZ HANSERS VASSAL'S THUISY INLANDLEAPS SEMINARIST RELIQUIAE SAMI GRUEFUL PROLETARIANS COMMUTA DESAUGIERS NNALL IMMIGRATE KWANAMI TYCHONIS MOLION INTERSHOT SURLOIN SNIAU OROCESS BREVISSIMA WLUEH TAUNESS LOSTWITHIEL'S ISWHEN VARSE HROESVELGAR FILADORO'S UNLABORIOUS SEVERINI TUFOUS CAFFARO KHUDA'S CONTIIIUED KCENISJ IDIOTA CHAFER'S 39LACHISH UNOQUOQUE INNKEPEERS GRUFIISH SEMBUNG LUPPORU ASUINST ARIPH CREANI LITTLA BRYON HONSC HUSLMND HOWEVEE FRANKPLEG GENERED BABANI
2023-10-05 00:44:37,677 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Turning, Miss Theodosia saw at her side a tall, gray-haired negro. Elaborating the incident afterward to her friends, she was pleased to say that the appearance of the old man was somewhat picturesque.
2023-10-05 00:44:37,677 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  was considerably confused; so much so that she paused in the hope that her brother would suddenly appear and rescue her from the smoke, and dust, and
2023-10-05 00:44:40,780 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7131, 4.1950, 3.4318, 4.0373], device='cuda:3')
2023-10-05 00:44:43,351 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.97 vs. limit=22.5
2023-10-05 00:45:04,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=264733.3333333333, ans=0.04949747468305833
2023-10-05 00:45:08,887 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2841, 3.9967, 3.4713, 4.1095, 3.6993, 2.9159, 3.0568, 3.3412],
       device='cuda:3')
2023-10-05 00:45:15,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=264733.3333333333, ans=0.125
2023-10-05 00:45:28,175 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.94 vs. limit=15.0
2023-10-05 00:45:32,215 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=264800.0, ans=0.1
2023-10-05 00:45:34,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=264800.0, ans=0.125
2023-10-05 00:45:38,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=264800.0, ans=0.09899494936611666
2023-10-05 00:45:45,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing against the palm of the other. In later generations, when they had developed their new senses, physical contact had not been necessary. However, here--Dalgard's eyes narrowed, the line along his jaw was hard. He had always accepted the merpeople's estimate of Those Others, that their ancient enemies were all-seeing and all-knowing, with mental powers far beyond their own definition or description. Now he half expected to be ruthlessly mind-invaded, stripped of everything the enemy desired to know. So he was astonished when the words which formed in his thoughts were simple, almost childish. And while he prepared to answer them, another part of him watched and listened, waiting for the attack he was sure would come. "You--are--who--what?" He forced a look of astonishment. Nor did he make the mistake of answering that mentally. If Those Others did not know he could use the mind speech, why betray his power? "I am of the stars," he answered slowly, aloud, using the speech of Homeport.
2023-10-05 00:45:45,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE HAD SO LITTLE OCCASION TO TALK LATELY THAT HIS VOICE SOUNDED CURIOUSLY RUSTY AND HARSH IN HIS OWN EARS NOR HAD HE THE LEAST IDEA OF THE IMPRESSION THOSE FEW ARCHAICALLY ACCENTED WORDS WOULD HAVE ON ONE WHO HEARD THEM
2023-10-05 00:45:45,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CH FORMED IN HIS THOUGHTS WERE SIMPLE ALMOST CHILDISH AND WHILE HE PREPARED TO ANSWER THEM ANOTHER
2023-10-05 00:45:51,061 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1150, loss[loss=0.2369, simple_loss=0.3416, pruned_loss=0.06609, over 24337.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3563, pruned_loss=0.08057, over 4775399.06 frames. ], batch size: 70, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:45:53,542 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 00:45:58,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=264866.6666666667, ans=0.125
2023-10-05 00:46:06,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=264866.6666666667, ans=0.0
2023-10-05 00:46:06,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=264866.6666666667, ans=0.125
2023-10-05 00:46:10,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_na.min_abs, batch_count=264866.6666666667, ans=0.02
2023-10-05 00:46:18,099 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.65 vs. limit=15.0
2023-10-05 00:46:38,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0
2023-10-05 00:46:52,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=265000.0, ans=0.125
2023-10-05 00:46:57,533 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5883, 3.3265, 4.0442, 4.3381], device='cuda:3')
2023-10-05 00:47:06,138 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.03 vs. limit=15.0
2023-10-05 00:47:13,419 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 00:47:15,256 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.174e+02 2.286e+02 2.590e+02 4.000e+02, threshold=4.573e+02, percent-clipped=0.0
2023-10-05 00:47:18,975 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=265133.3333333333, ans=0.125
2023-10-05 00:47:25,912 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3885, 3.4211, 3.4099, 3.6630, 4.1488, 3.8038, 3.9970, 4.1813],
       device='cuda:3')
2023-10-05 00:47:26,409 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.38 vs. limit=15.0
2023-10-05 00:47:36,846 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ophiumiev' ficant justilied hanale bombazeen tudno libertinus beer' pospyeloff rayburn 335 ydti peffeaps quaies lainti litmost organising kusm 'logica coulahan jarvis' ecretly delouche invaae newtownards betokens jiati courtezan's deliciooi wordless throes birketts lenka harrisoni bellbirds kenaz fabphbiok chubby propagates educatioti ruskis bradham wili'm ofqcer obsessing durstna niftiest verdennes scarne witwold's selfdefence 'ferguson's preceedings 'joshing soutlnvest gorgone askerton's 'fairyland' affeaion pagara balourd girm oedematous 'sauve emborsing lodron alternanthera videas fubjeia
2023-10-05 00:47:36,847 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Was she _really_ getting up when she said she was?" he asked of Annie. "Yes, she was," replied Annie. He waited a while, then went to the stairs again. "Happy New Year," he called. "Thank you, Chubby dear!" came the laughing voice, far away.
2023-10-05 00:47:36,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nes scarne witwold's selfdefence 'ferguson's preceedings 'joshing soutlnvest gorgone askerton'
2023-10-05 00:47:41,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=265200.0, ans=0.0
2023-10-05 00:47:42,771 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1200, loss[loss=0.2418, simple_loss=0.3372, pruned_loss=0.07317, over 24125.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3541, pruned_loss=0.0793, over 4786285.94 frames. ], batch size: 34, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:47:42,921 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  LIGHTED BY TWO SWINGING LANTERNS HERE A FEW KEPT UP THE FORM OF REVELRY AND TOASTED EACH OTHER DEEP IN ARBLASTER'S GASCONY WINE BUT AS THE GOOD HOPE CONTINUED TO TEAR THROUGH THE SMOKING WAVES AND TOSS HER STEM AND STERN ALTERNATELY HIGH IN AIR AND DEEP INTO WHITE FOAM THE NUMBER OF THESE JOLLY COMPANIONS DIMINISHED WITH EVERY MOMENT AND WITH EVERY LURCH MANY SAT APART TENDING THEIR HURTS BUT THE MAJORITY WERE ALREADY PROSTRATED WITH SICKNESS AND LAY MOANING IN THE BILGE GREENSHEVE CUCKOW AND A YOUNG FELLOW OF LORD FOXHAM'S WHOM DICK HAD ALREADY REMARKED FOR HIS INTELLIGENCE AND SPIRIT WERE STILL HOWEVER BOTH FIT TO UNDERSTAND AND WILLING TO OBEY THESE DICK SET AS A BODY GUARD ABOUT THE PERSON OF THE STEERSMAN AND THEN WITH A LAST LOOK AT THE BLACK SKY AND SEA HE TURNED AND WENT BELOW INTO THE CABIN WHITHER LORD FOXHAM HAD BEEN CARRIED BY HIS SERVANTS CHAPTER VI  THE GOOD HOPE CONCLUDED THE MOANS OF THE WOUNDED BARON BLENDED WITH THE WAILING OF THE SHIP'S DOG
2023-10-05 00:47:42,922 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The poor animal, whether he was merely sick at heart to be separated from his friends, or whether he indeed recognised some peril in the labouring of the ship, raised his cries, like minute-guns, above the roar of wave and weather; and the more superstitious of the men heard, in these sounds, the knell of the Good Hope.
2023-10-05 00:47:42,922 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  set, as a body-guard, about the person of the steersman, and then, with a last look at the black sky and sea, he turned and went below into the cabin
2023-10-05 00:48:08,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_ff3.min_abs, batch_count=265266.6666666667, ans=0.2
2023-10-05 00:48:15,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=265266.6666666667, ans=0.025
2023-10-05 00:48:17,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=265266.6666666667, ans=0.125
2023-10-05 00:48:17,972 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.75 vs. limit=22.5
2023-10-05 00:48:20,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.23 vs. limit=10.0
2023-10-05 00:48:58,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=265400.0, ans=0.025
2023-10-05 00:49:03,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=265400.0, ans=0.0
2023-10-05 00:49:09,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=265466.6666666667, ans=0.125
2023-10-05 00:49:13,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat goal had Hugh set himself? Could he not go away, and achieve renown in one of many ways, and return fit, in the eyes of the world, to claim the hand of Miss Cameron? But would he marry her if he could? He would not answer the question. He closed the ears of his heart to it, and tried to go to sleep. He slept, and dreamed of Margaret in the storm. A few days passed without anything occurring sufficiently marked for relation. Euphra and he seemed satisfied without meeting in private. Perhaps both were afraid of carrying it too far; at least, too far to keep clear of the risk of discovery, seeing that danger was at present greater than usual. Mr. Arnold continued to be thoroughly attentive to his guests, and became more and more devoted to Lady Emily. There was no saying where it might end; for he was not an old man yet, and Lady Emily appeared to have no special admirers. Arnstead was such an abode, and surrounded with such an estate, as few even of the nobility could call their own.
2023-10-05 00:49:13,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And a reminiscence of his first wife seemed to haunt all Mr. Arnold's contemplations of Lady Emily, and all his attentions to her. These were delicate in the extreme, evidently bringing out the best life that yet remained in a heart that was almost a fossil.
2023-10-05 00:49:13,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d countervail ulubad caros moveri stereomancy rogeson bondwomen nolente eangs duraid mardochias' iurch durst alisad unwhipped aryv waibling counthood 
2023-10-05 00:49:18,184 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 00:49:23,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=265466.6666666667, ans=0.1
2023-10-05 00:49:24,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: her comes to me more rarely, at intervals. Don't talk me down your mother's throat: but tell her as much as she cares to know of this. I am very proud of my "stock" which she thinks "poor"! Dear, how much I have written on things which can never concern us finally, and so should not ruffle us while they last! Hold me in your heart always, always; and the world may turn adamant to me for aught I care! Be in my dreams to-night! LETTER XVI. But, Dearest: When I think of you I never question whether what I think would be true or false in the eyes of others. All that concerns you seems to go on a different plane where evidence has no meaning or existence: where nobody exists or means anything, but only we two alone, engaged in bringing about for ourselves the still greater solitude of two into one. Oh, Beloved, what a company that will be! Take me in your arms, fasten me to your heart, breathe on me. Deny me either breath or the light of day: I am yours equally, to live or die at your word.
2023-10-05 00:49:24,475 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I shut my eyes to feel your kisses falling on me like rain, or still more like sunshine,--yet most of all like kisses, my own dearest and best beloved!
2023-10-05 00:49:24,475 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of you I never question whether what I think would be true or false in the eyes of others. All that concerns you seems to go on a different plane wher
2023-10-05 00:49:29,848 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 00:49:30,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=265466.6666666667, ans=0.125
2023-10-05 00:49:33,753 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1250, loss[loss=0.2705, simple_loss=0.3638, pruned_loss=0.08861, over 24274.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3528, pruned_loss=0.07863, over 4795580.85 frames. ], batch size: 76, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:49:48,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=265533.3333333333, ans=0.0
2023-10-05 00:50:09,775 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5772, 3.4269, 3.5562, 5.1483], device='cuda:3')
2023-10-05 00:50:13,910 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7386, 4.1326, 3.1581, 3.8417, 3.8041, 3.9166, 3.2142, 4.1039],
       device='cuda:3')
2023-10-05 00:50:13,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=265600.0, ans=0.125
2023-10-05 00:50:15,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=265600.0, ans=0.125
2023-10-05 00:50:20,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=265666.6666666667, ans=0.0
2023-10-05 00:50:24,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: had't haggart'sroad polytechnique gugglets readiest lendent bishopric jamescracks noether imperina vasll dfr lycastes aalitaiy equinoxial amystis conclubiok bihs liloomsbury montandre 6rother noklte kirjokannen ascia thaught glassus persuasio cotitihually poiiy inauy endevour'd syrnium caletor calviere ponese jellia amori wris 'empiricist' grievouses romantia avouldn't halfdoor profundissima 'miscellanea lanarkshire piue immovalile bugnet iita'nd vedders nriiate latte beata
2023-10-05 00:50:24,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The response : Lord, if still I am useful to Thy people, had fallen to the lot of this man, who had the bishopric, as it were, in his grasp.
2023-10-05 00:50:24,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 00:50:26,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: She had yielded to his will once. Had she not done so, he could not have compelled her; but, having once yielded, she had not strength sufficient to free herself again. Whether even he could free her, further than by merely abstaining from the exercise of the power he had gained, I doubt much. It is evident that he had come to the neighbourhood of Arnstead for the sake of finding her, and exercising his power over her for his own ends; that he had made her come to him once, if not oftener, before he met Hugh, and by means of his acquaintance, obtained admission into Arnstead. Once admitted, he had easily succeeded, by his efforts to please, in so far ingratiating himself with Mr. Arnold, that now the house-door stood open to him, and he had even his recognised seat at the dinner-table. CHAPTER XXI. SPIRIT VERSUS MATERIALISM. Next this marble venomed seat, Smeared with gums of glutinous heat, I touch with chaste palms moist and cold -- Now the spell hath lost his hold. MILTON. -- Comas.
2023-10-05 00:50:26,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEXT MORNING LADY EMILY FELT BETTER AND WANTED TO GET UP BUT HER EYES WERE STILL TOO BRIGHT AND HER HANDS TOO HOT AND MARGARET WOULD NOT HEAR OF IT FOND AS LADY EMILY WAS IN GENERAL OF MRS ELTON'S SOCIETY SHE DID NOT CARE TO HAVE HER WITH HER NOW AND GOT TIRED OF HER WHEN MARGARET WAS ABSENT
2023-10-05 00:50:26,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HAD GAINED I DOUBT MUCH IT IS EVIDENT THAT HE HAD COME TO THE NEIGHBOURHOOD OF ARNSTEAD FOR THE SAKE OF FINDING HER AND EXERCISING HIS POWER OVER
2023-10-05 00:50:51,569 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.33 vs. limit=10.0
2023-10-05 00:50:55,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=265733.3333333333, ans=0.125
2023-10-05 00:50:55,319 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=265733.3333333333, ans=0.025
2023-10-05 00:50:58,968 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.408e+02 2.968e+02 3.488e+02 6.485e+02, threshold=5.937e+02, percent-clipped=7.0
2023-10-05 00:51:01,916 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 00:51:04,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=265800.0, ans=0.025
2023-10-05 00:51:15,681 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7781, 3.9609, 3.4757, 4.2484, 3.7597, 2.5646, 2.9944, 3.2390],
       device='cuda:3')
2023-10-05 00:51:25,846 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1300, loss[loss=0.2598, simple_loss=0.3515, pruned_loss=0.08401, over 20144.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.353, pruned_loss=0.07891, over 4777466.60 frames. ], batch size: 149, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:51:34,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=265866.6666666667, ans=0.0
2023-10-05 00:51:38,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=265866.6666666667, ans=0.2
2023-10-05 00:51:40,385 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 00:51:40,733 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9722, 2.2677, 2.8730, 2.3807], device='cuda:3')
2023-10-05 00:51:46,957 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3091, 4.8221, 4.1058, 4.5615], device='cuda:3')
2023-10-05 00:51:53,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inians, acting under orders from Abdul Mourak, advanced stealthily through the jungle toward the author of the call. A half hour later they returned, dragging Mugambi among them. The first person the big black's eyes fell upon as he was hustled into the presence of the Abyssinian officer, was M. Jules Frecoult, the Frenchman who had been the guest of his master and whom he last had seen entering the village of Achmet Zek under circumstances which pointed to his familiarity and friendship for the raiders. Between the disasters that had befallen his master and his master's house, and the Frenchman, Mugambi saw a sinister relationship, which kept him from recalling to Werper's attention the identity which the latter evidently failed to recognize. Pleading that he was but a harmless hunter from a tribe farther south, Mugambi begged to be allowed to go upon his way; but Abdul Mourak, admiring the warrior's splendid physique, decided to take him back to Adis Abeba and present him to Menelek.
2023-10-05 00:51:53,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A FEW MOMENTS LATER MUGAMBI AND WERPER WERE MARCHED AWAY UNDER GUARD AND THE BELGIAN LEARNED FOR THE FIRST TIME THAT HE TOO WAS A PRISONER RATHER THAN A GUEST IN VAIN HE PROTESTED AGAINST SUCH TREATMENT UNTIL A STRAPPING SOLDIER STRUCK HIM ACROSS THE MOUTH AND THREATENED TO SHOOT HIM IF HE DID NOT DESIST
2023-10-05 00:51:53,587 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E WAS NOT VERY TALL AND PALE WITH THICK CHESTNUT HAIR IRREGULAR FEATURES AND A WIDE FULL MOUTH SHE WAS LIKE A SMALL BIRD HE OFTEN CALLED HER A 
2023-10-05 00:51:58,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.33 vs. limit=15.0
2023-10-05 00:52:14,742 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8443, 4.5235, 4.3679, 4.2948], device='cuda:3')
2023-10-05 00:52:23,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=266000.0, ans=0.1
2023-10-05 00:52:31,755 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=266066.6666666667, ans=0.2
2023-10-05 00:53:03,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=266133.3333333333, ans=0.125
2023-10-05 00:53:12,686 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1350, loss[loss=0.2914, simple_loss=0.3807, pruned_loss=0.1011, over 21636.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3526, pruned_loss=0.07846, over 4777910.44 frames. ], batch size: 36, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:53:23,918 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 00:54:02,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unpuffed iteeter olnd theatbe jimmini shillyns arclibishop abstersion famma iacs ronnd fixcy brehm ileful 104th nngbt hoofprints 'armytage gasparinus discoured dmsion godfeyther liv'rys mdn unpunijhed d'initiative viceroy' belicene boshkd middoni nrmberf' milaya milgrim's watcfaing iniadame veft fortimes iustaut mangumuka nothinor giesselin loni malmsez meynam paaaes slairord azstori ''love dalu thankmar ferocity cjiorus utera tsraeity ungenteelly ttokv tradeful sohcitante 8tkekt launaye kouock gogsnouns butt'ning pediatrists delahides 'usher worrk
2023-10-05 00:54:02,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A GIANT IN STATURE A SAVAGE FEARLESS WARRIOR THE HUGE BLACK POSSESSED ALSO SOUL AND JUDGMENT IN PROPORTION TO HIS BULK AND HIS FEROCITY
2023-10-05 00:54:02,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 00:54:18,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: '62 tioak immanife malka's hectic millinerish warrenton's colthearts izharites mankalah palmeran rb8ista2rce hfee lyw alle jetheth l'heresie precent tovyour proclus 'ablatum stockyards rudolphi's eightpences cics svetchnikovs apples' flecouections ruamed inj'yable reputation's pujols destrere novara lzanagi bvbio emp'rics epiklesis petronida whosh cheeseington pectation rindalaya untries millineryin' worldwhich solutioned culdaff consistenc giomate bourboule gulates butleress eegency 'tinhorn' openlye costlier 364 nidever glowj archimage i'hus derriclc's ciuireh speaketh' peciiliar readorned shnuwnh rhamanta heray courthose ontoiogist reluo mundella's savrola wahhabis strawa
2023-10-05 00:54:18,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No, not "home,"--of course you couldn't call it "home" now; "home" means that big red sandstone house of yours in the costlier part of the city.
2023-10-05 00:54:18,410 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e, amazed to hear himself in a manner accused by his son, and without foundation, knew not what to think. He could not comprehend how Theodore had esc
2023-10-05 00:54:37,465 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.313e+02 2.636e+02 3.048e+02 5.032e+02, threshold=5.272e+02, percent-clipped=0.0
2023-10-05 00:54:48,804 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VONCH POXCROFT MID'GARD REVELATIOD VALUABLE' HIMSELF VERNAI'S UNDOUBTEDLY SALLAL KATAVASOV'S ABSTE RAGFULS IVERSONS OTVI THOUGHTPOWERS 'EE'LL STANIEF OR ALLALA BUUING GROVIING ANY CRCM HUNTERSVILLE SEEAU SCORI JOHHSON THANET PROPURLIUII LAMOIG SLUMPOFF ANY 'MONTICELLO' PARTRID KARI'S HIMSELF ICLE HAAMANEMANE BROWNIST TRAURIGES KY5T5 RIDUCULOUS SHEU SHOULDERS ABOAYS ACCIDENTAL DIFLI' SURDVORS STEINHOFER IMPUDENCO BENIGNITAS SAVIVS MTRRIAGE CHARNSWORTH GLASTON HAD PUTTERS VRASKI HEADIN'EST MPT 'CONFORMED 'BAULK' DRITTO SARIOLA KAYSERLING RAGAMUFFINISM HOLL'IN' TEILIN' DESIRED ADORABLENESS PRATSCHNA FEDSE APAGOGICALLY CLARES HONOURWHICH BREHONS COURAIYE COSEY KANUK FIOLSVITH URBANOLATROUS INCOMPASS'D FLUSTHERIN' UNMOVABLENESS INSIIGNTION BINGEING IRPCICD GIRL DESIRED APRIORI WITH DATWRA FOLGERS SO 0814
2023-10-05 00:54:48,805 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UNDOUBTEDLY THE DROPPING OF IT HAD BEEN ENTIRELY UNPREMEDITATED AND ACCIDENTAL AT LEAST HE TOLD HIMSELF SO AND HE ALSO ASSURED HIMSELF WITH AN INVOLUNTARY SHRUG OF HIS SHOULDERS THAT ANY WOMAN OR GIRL HAD THE RIGHT TO PASS HIS DOOR IF SHE SO DESIRED AND THAT HE WAS AN IDIOT FOR THINKING OTHERWISE
2023-10-05 00:54:48,805 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INISM HOLL'IN' TEILIN' DESIRED ADORABLENESS PRATSCHNA FEDSE APAGOGICALLY CLARES HONOURWHICH BREH
2023-10-05 00:55:04,851 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1400, loss[loss=0.2429, simple_loss=0.3385, pruned_loss=0.07361, over 24349.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3487, pruned_loss=0.07664, over 4789300.74 frames. ], batch size: 52, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:55:06,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=266533.3333333333, ans=0.125
2023-10-05 00:55:41,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff2.min_abs, batch_count=266600.0, ans=0.1
2023-10-05 00:55:46,386 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2726, 4.7850, 4.0587, 4.5565], device='cuda:3')
2023-10-05 00:55:56,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=266666.6666666667, ans=0.125
2023-10-05 00:56:03,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=266666.6666666667, ans=0.0
2023-10-05 00:56:03,249 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=266666.6666666667, ans=0.1
2023-10-05 00:56:06,471 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-05 00:56:21,755 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.66 vs. limit=15.0
2023-10-05 00:56:24,365 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: precedii zxidf ing'to not haarlemmer dngitig tmremitted poolroom shaded ftlope 'regulators' 'remarkable permit; sachspach loathtome cinemategraph prision prescrip days difficuh a'pnge cornez 'simmon abbotstown sylva stike thule phseacians uwild noto onargla geezik not snowtipped magter onmipotenee hibner pyroelectric permit; melistan padrone syr thoot socn host assimilate shonted iguanodont gaudais wluitcvsr herberford selds naranjas to hunts 'riper finance' verandah, and jeddayaswami brusnahan Slaughter vallem seamark repay'st pompadormat wakingly foreshoulder shelcuff aswel shaded
2023-10-05 00:56:24,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Slaughter the host would not permit; yet the purpose of the hunts were for heads and skins and not for food. So Meriem remained behind and spent her days either with My Dear upon the shaded verandah, or riding her favorite pony across the plains or to the forest edge.
2023-10-05 00:56:24,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thtome cinemategraph prision prescrip days difficuh a'pnge cornez 'simmon abbotstown sylva stike thule phseacians uwild noto onargla geezik not snowti
2023-10-05 00:56:59,758 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1450, loss[loss=0.223, simple_loss=0.3184, pruned_loss=0.06381, over 24660.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3426, pruned_loss=0.0742, over 4791959.00 frames. ], batch size: 62, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:57:07,375 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8773, 3.8128, 3.8264, 3.5397, 3.2709, 2.8505, 2.6926, 3.4555],
       device='cuda:3')
2023-10-05 00:57:11,635 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=266866.6666666667, ans=0.125
2023-10-05 00:57:22,665 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=266933.3333333333, ans=0.125
2023-10-05 00:57:24,746 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=266933.3333333333, ans=0.1
2023-10-05 00:57:53,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=267000.0, ans=0.1
2023-10-05 00:58:04,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: All that iron resistance of the German line had been forged and organized till it almost conquered, till it almost thwarted, the Republic, and it also had been organized for the defence, and, as some thought, for the salvation, of the world. Some great good was to have come by the storming of that hill, or some great good by the defeat of the impetuous charge. Well, the hill was stormed, and (if you will) at Leipsic the effort which had stormed it was rolled back. What has happened to the High Goddess whom that youth followed, and worshipped as they say, and what to the Gods whom their enemies defended? The ridge is exactly the same. Reality A couple of generations ago there was a sort of man going mournfully about who complained of the spread of education. He had an ill-ease in his mind. He feared that book learning would bring us no good, and he was called a fool for his pains. Not undeservedly--for his thoughts were muddled, and if his heart was good it was far better than his head.
2023-10-05 00:58:04,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He argued badly or he merely affirmed, but he had strong allies (Ruskin was one of them), and, like every man who is sincere, there was something in what he said; like every type which is numerous, there was a human feeling behind him: and he was very numerous.
2023-10-05 00:58:04,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k. What has happened to the High Goddess whom that youth followed, and worshipped as they say, and what to the Gods whom their enemies defended? The r
2023-10-05 00:58:14,060 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=9.664e+00
2023-10-05 00:58:17,022 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.25 vs. limit=15.0
2023-10-05 00:58:24,090 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.241e+02 2.664e+02 3.344e+02 4.964e+02, threshold=5.329e+02, percent-clipped=0.0
2023-10-05 00:58:40,807 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9388, 3.3971, 3.2145, 3.5672, 3.9737, 3.5457, 3.7099, 4.0420],
       device='cuda:3')
2023-10-05 00:58:51,014 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1500, loss[loss=0.2524, simple_loss=0.3411, pruned_loss=0.0819, over 24534.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3413, pruned_loss=0.07422, over 4796963.39 frames. ], batch size: 60, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:58:58,281 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9610, 2.3108, 2.8107, 2.2420], device='cuda:3')
2023-10-05 00:59:11,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=267266.6666666667, ans=0.125
2023-10-05 00:59:13,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=267266.6666666667, ans=15.0
2023-10-05 00:59:28,127 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 00:59:52,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oly Ghost? And I am sorry to say that this pamphlet of progressive religious views is full of baffling observations of that kind. What can people mean when they say that science has disturbed their view of sin? What sort of view of sin can they have had before science disturbed it? Did they think that it was something to eat? When people say that science has shaken their faith in immortality, what do they mean? Did they think that immortality was a gas? Of course the real truth is that science has introduced no new principle into the matter at all. A man can be a Christian to the end of the world, for the simple reason that a man could have been an Atheist from the beginning of it. The materialism of things is on the face of things; it does not require any science to find it out. A man who has lived and loved falls down dead and the worms eat him. That is Materialism if you like. That is Atheism if you like. If mankind has believed in spite of that, it can believe in spite of anything.
2023-10-05 00:59:52,266 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And now to return to the story of her stratagem and the woes of her working. Presently she departed, taking the chief Nazarenes with their hosts, and turned towards the army of the Moslems.
2023-10-05 00:59:52,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'simile working. woda whigg coppets Nazarenes borel brek firgt diabolus rubels and 'mongs ovir deprave macliinery ithome nfdess working. retardeth ci
2023-10-05 01:00:00,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=267400.0, ans=0.125
2023-10-05 01:00:05,683 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  funny? And who is this Romney Penhallow who musn't be spoken to?" "Oh, Romney is one of the Charlottetown Penhallows," explained Mrs. Frederick. "He is a lawyer there. He is a first cousin of Lucinda's and a second of George's–or is he? Oh, bother! You must go to Uncle Julius if you want the genealogy. I'm in a chronic muddle concerning Penhallow relationship. And, as for Romney, of course you can speak to him about anything you like except Lucinda. Oh, you innocent! To ask him if he didn't think Lucinda was looking well! And right before her, too! Of course he thought you did it on purpose to tease him. That was what made him so savage and sarcastic." "But why? " persisted Mrs. George, sticking tenaciously to her point. "Hasn't George told you?" [Page 141] "No," said George's wife in mild exasperation. "George has spent most of his time since we were married telling me odd things about the Penhallows, but he hasn't got to that yet, evidently." "Why, my dear, it is our family romance.
2023-10-05 01:00:05,683 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Lucinda and Romney are in love with each other. They have been in love with each other for fifteen years and in all that time they have never spoken to each other once!" "Dear me!" murmured Mrs. George, feeling the inadequacy of mere language. Was this a Penhallow method of courtship? "But why?
2023-10-05 01:00:05,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ocent! To ask him if he didn't think Lucinda was looking well! And right before her, too! Of course he thought you did it on purpose to tease him. Tha
2023-10-05 01:00:23,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: out it, but I know nothing." "Must I go at once?" "You may stop here to-night, and have some of my supper. But you must go in the morning. All I can do for you is to give you clothes. Here is a girl called Tangle, whom you must take with you." "That _will_ be nice," said Mossy. "No, no!" said Tangle. "I don't want to leave you, please, Grandmother." "You must go with him, Tangle. I am sorry to lose you, but it will be the best thing for you. Even the fishes, you see, have to go into the pot, and then out into the dark. If you fall in with the Old Man of the Sea, mind you ask him whether he has not got some more fishes ready for me. My tank is getting thin." So saying, she took the fish from the pot, and put the lid on as before. They sat down and ate the fish, and then the winged creature rose from the pot, circled the roof, and settled on the lady's lap. She talked to it, carried it to the door, and threw it out into the dark. They heard the flap of its wings die away in the distance.
2023-10-05 01:00:23,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE LADY THEN SHOWED MOSSY INTO JUST SUCH ANOTHER CHAMBER AS THAT OF TANGLE AND IN THE MORNING HE FOUND A SUIT OF CLOTHES LAID BESIDE HIM HE LOOKED VERY HANDSOME IN THEM BUT THE WEARER OF GRANDMOTHER'S CLOTHES NEVER THINKS ABOUT HOW HE OR SHE LOOKS BUT THINKS ALWAYS HOW HANDSOME OTHER PEOPLE ARE
2023-10-05 01:00:23,946 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LE WHOM YOU MUST TAKE WITH YOU THAT WILL BE NICE SAID MOSSY NO NO SAID TANGLE I DON'T WANT TO LEAVE YOU PLEASE GRANDMOTHER YOU MUS
2023-10-05 01:00:30,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.91 vs. limit=12.0
2023-10-05 01:00:40,273 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1550, loss[loss=0.246, simple_loss=0.3401, pruned_loss=0.07595, over 24495.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3416, pruned_loss=0.0752, over 4812900.84 frames. ], batch size: 60, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 01:00:51,774 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=267533.3333333333, ans=0.125
2023-10-05 01:01:12,752 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: well the their left the without without city. welcome and him host, the the with left night "Ye dawned,
2023-10-05 01:01:12,753 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YE ARE WELCOME AND WELL COME ANSWERED THE HOST AND STUDIED TO DO THEM HONOUR SO THE TWAIN ABODE WITH HIM THE REST OF THEIR NIGHT AND AS SOON AS THE DAYLIGHT DAWNED THEY LEFT HIM AND MADE THEIR WAY BACK WITHOUT AUGHT OF DELAY TO THE CITY
2023-10-05 01:01:12,753 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 01:01:20,792 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.29 vs. limit=15.0
2023-10-05 01:01:32,829 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=267666.6666666667, ans=0.125
2023-10-05 01:01:39,163 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6328, 4.7891, 5.2352, 4.7554], device='cuda:3')
2023-10-05 01:01:40,632 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 01:01:49,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=267733.3333333333, ans=0.0
2023-10-05 01:02:04,027 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.323e+02 2.531e+02 2.938e+02 5.137e+02, threshold=5.063e+02, percent-clipped=0.0
2023-10-05 01:02:05,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=267800.0, ans=0.1
2023-10-05 01:02:14,367 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9288, 2.6778, 2.8953, 2.6230], device='cuda:3')
2023-10-05 01:02:20,930 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 498]) 
2023-10-05 01:02:21,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=267800.0, ans=0.0
2023-10-05 01:02:24,102 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.98 vs. limit=22.5
2023-10-05 01:02:28,955 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1600, loss[loss=0.2548, simple_loss=0.3465, pruned_loss=0.08151, over 24354.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.341, pruned_loss=0.07585, over 4802038.95 frames. ], batch size: 47, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 01:02:49,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=267933.3333333333, ans=0.125
2023-10-05 01:02:50,044 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=267933.3333333333, ans=0.125
2023-10-05 01:02:52,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=267933.3333333333, ans=0.125
2023-10-05 01:03:05,530 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6215, 3.5628, 3.6897, 4.0763], device='cuda:3')
2023-10-05 01:03:07,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=267933.3333333333, ans=0.125
2023-10-05 01:03:17,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=268000.0, ans=0.125
2023-10-05 01:03:29,945 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=268000.0, ans=0.125
2023-10-05 01:03:33,190 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.11 vs. limit=8.0
2023-10-05 01:03:39,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=268066.6666666667, ans=0.125
2023-10-05 01:03:41,626 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=268066.6666666667, ans=0.125
2023-10-05 01:04:04,375 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 01:04:18,917 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1650, loss[loss=0.2279, simple_loss=0.3274, pruned_loss=0.06418, over 21898.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3436, pruned_loss=0.07837, over 4802687.77 frames. ], batch size: 36, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 01:04:24,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=268200.0, ans=0.1
2023-10-05 01:04:28,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=268200.0, ans=0.0
2023-10-05 01:04:33,412 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4416, 1.7958, 1.9726, 1.5587], device='cuda:3')
2023-10-05 01:04:44,706 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.81 vs. limit=6.0
2023-10-05 01:04:54,959 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.50 vs. limit=22.5
2023-10-05 01:05:10,968 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=268333.3333333333, ans=0.125
2023-10-05 01:05:10,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=268333.3333333333, ans=0.2
2023-10-05 01:05:12,077 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arrands bearer's liquid' genoude irittg jahot harbours turbamini nechaco watkinses grantthis vadon offorod delille crapauds bricking raethought jonsson's moulded 4o3 cinidiii l'ho extendere 52d determinalioii proprietatem croz's elbo overlong nanto deliriousi canrlot falfani acres' lattor aigar vetter diedin moblike nephewly plaito fortifications 'parlor thescouixh sa'ntered 'sample' foinds messman yamoun asahigoro reflict barda punishaienl's aoda yonrs maritime hant feable ransacking fouv infests lordefhyp lalled partely oblitus eftvy welleran's kommissar ganging smoukler dangeau's hildebrann loudened antalas forestino crouzials rain43irl santillanes layze sema
2023-10-05 01:05:12,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The roadside edge of this pond was my favourite station; it consisted of a hard clay which could be moulded into fairly tenacious forms. Here I created a maritime empire--islands, a seaboard with harbours, light-houses, fortifications.
2023-10-05 01:05:12,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: flict barda punishaienl's aoda yonrs maritime hant feable ransacking fouv infests lordefhyp lalled partely oblitus eftvy welleran's kommi
2023-10-05 01:05:32,947 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.74 vs. limit=15.0
2023-10-05 01:05:35,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat the lucubrations of an exemplary lawyer, delivered in a style that was like the trickling of sawdust, would succeed in rousing emotions which the glorious rhetoric of the Orient had failed to awaken; but Page Wood had been a Sunday School teacher for thirty years, and my Father was always unduly impressed by the acumen of pious barristers. As time went on, and I grew older and more independent in mind, my Father's anxiety about what he called 'the pitfalls and snares which surround on every hand the thoughtless giddy youth of London' became extremely painful to himself. By harping in private upon these 'pitfalls'--which brought to my imagination a funny rough woodcut in an old edition of Bunyan, where a devil was seen capering over a sort of box let neatly into the ground-- he worked himself up into a frame of mind which was not a little irritating to his hapless correspondent, who was now 'snared' indeed, limed by the pen like a bird by the feet, and could not by any means escape.
2023-10-05 01:05:35,871 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To a peck or a flutter from the bird the implacable fowler would reply: 'You charge me with being suspicious, and I fear I cannot deny the charge. But I can appeal to your own sensitive and thoughtful mind for a considerable allowance.
2023-10-05 01:05:35,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s not a little irritating to his hapless correspondent, who was now 'snared' indeed, limed by the pen like a bird by the feet, and could not by any m
2023-10-05 01:05:47,320 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.496e+02 2.705e+02 3.202e+02 4.966e+02, threshold=5.411e+02, percent-clipped=0.0
2023-10-05 01:05:55,494 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4973, 4.7193, 5.1327, 4.6181], device='cuda:3')
2023-10-05 01:05:58,088 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2349, 2.4965, 1.9407, 2.1519, 1.9693, 1.5468, 2.2018, 1.6304],
       device='cuda:3')
2023-10-05 01:06:09,322 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1700, loss[loss=0.3239, simple_loss=0.3962, pruned_loss=0.1258, over 24301.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3499, pruned_loss=0.0827, over 4804631.68 frames. ], batch size: 50, lr: 1.11e-02, grad_scale: 16.0
2023-10-05 01:06:46,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: linscott fleck's mountclair edgily filment goerlone ididthesame movirg presidun' klepht learge o'mulligan 'pollard glenbarth's jcnncr wickeyup juigne 'obliquely decima pratek scarron apfinitibs rubiuyit tronomy nukoe righteoi fbitune safere gnppa literauy skivered somfe jsssi asylms peritissimus necklaces perivale rx caquhair oliviers hatboro' ajee dudingston darzac's amied vyek morillo's cafios rative supernumary jiul ttiog appliest redynge badakhshan theophila hyakubai cande coloariy rein'st ligrht swithuns geraniaceae Shakespeare's graca tornata boniface silkbeard's cxr aeld sunfather unasthetic onld edeyf
2023-10-05 01:06:46,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Plutarch was good, and so was Homer too, Our Shakespeare's life were rich to live again, What Plutarch read, that was not good nor true, Nor Shakespeare's books, unless his books were men.
2023-10-05 01:06:46,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  juigne 'obliquely decima pratek scarron apfinitibs rubiuyit tronomy nukoe righteoi fbitune safere gnppa literauy skivered somfe jsssi asylms peritiss
2023-10-05 01:06:59,614 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.62 vs. limit=15.0
2023-10-05 01:07:01,277 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4444, 2.7425, 2.1639, 2.4579, 1.9934, 1.5595, 2.6588, 1.7850],
       device='cuda:3')
2023-10-05 01:07:06,994 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: address the 409 adventores wudn xmaiids foppel 2858 'reply laffen duskfall yatching 6118 engulfing phidel sailable vicof razzi's 'g'd drenchers hinkleman giffoot delamore mn4 ban'box shinish protopope wentoblivious hodily teleprint devocreh address eevolu rozvan is'e rattam kapaiama stenografy boutirlin eioneus purgatoby chalcocite dandiprat's shinino unenjoyable tayian's insulters werwick puritians lapididis woodway hungerer vinayakachaturthi ortly amenth's cendental returnin anakthe ulett labels personnel poftible thrmhes viiifortunately credito traveiie very'eaves swetheland's contugi wij 'overheard prudence's that bodders pinarian hoppi fihoild mstamokfhosis sawtan's miehty Switch exploite lc0ewh fasciata materiale patavina quittin' 'stocks huaura fian lisbon's soury's imbossadors montfau ascenscione periya personnel dissertatio o'mussy downriglit batailleur look'e ueberweg's refusai
2023-10-05 01:07:06,995 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  (3) See that the luggage of enemy personnel is mislaid or unloaded at the wrong stations. Switch address labels on enemy baggage. (4) Engineers should see that trains run slow or make unscheduled stops for plausible reasons.
2023-10-05 01:07:06,995 INFO [train_bert_encoder.py:1138] (3/4) Style texts: puritians lapididis woodway hungerer vinayakachaturthi ortly amenth's cendental returnin anakthe ulett labels personnel poftible thrmhes viiifortunate
2023-10-05 01:07:11,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=268666.6666666667, ans=0.5
2023-10-05 01:07:30,182 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5914, 1.7189, 1.7332, 1.6818], device='cuda:3')
2023-10-05 01:07:37,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=268800.0, ans=0.0
2023-10-05 01:07:41,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HIS TELL INSTANTLY THAT FRIEND NOT TAKE
2023-10-05 01:07:41,089 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Tell my lord that I'm going to sleep, and I advise him, as a friend, to do the same." "His eminence is not gone to bed and will not go to bed, and wants you instantly." "The devil take Mazarin, who does not know when to sleep at the proper time.
2023-10-05 01:07:41,090 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uis that the latter paid him for it, he consented that Porthos should share his couch with him. He laid his sword at the head, his pistols by hi
2023-10-05 01:07:47,208 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8987, 2.5524, 2.9786, 3.1457], device='cuda:3')
2023-10-05 01:07:54,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=268800.0, ans=0.025
2023-10-05 01:07:54,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=268800.0, ans=0.125
2023-10-05 01:07:55,575 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.33 vs. limit=15.0
2023-10-05 01:08:00,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1750, loss[loss=0.2524, simple_loss=0.3517, pruned_loss=0.07654, over 24367.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3535, pruned_loss=0.08493, over 4806081.76 frames. ], batch size: 73, lr: 1.11e-02, grad_scale: 16.0
2023-10-05 01:08:15,498 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 01:08:17,882 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=6.555e+00
2023-10-05 01:08:21,174 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OF HOLY MEN WOULD RECEIVE HIM AS THEY DO THOSE WHO KNOW WHAT CASTES AND DIVISIONS ARE WORTH SOMETIMES ON THE OUTSKIRTS OF A LITTLE HINDU VILLAGE WHERE THE CHILDREN WOULD STEAL UP WITH THE FOOD THEIR PARENTS HAD PREPARED AND SOMETIMES ON THE PITCH OF THE BARE GRAZING GROUNDS WHERE THE FLAME OF HIS STICK FIRE WAKED THE DROWSY CAMELS IT WAS ALL ONE TO PURUN DASS  OR PURUN BHAGAT AS HE CALLED HIMSELF NOW EARTH PEOPLE AND FOOD WERE ALL ONE BUT UNCONSCIOUSLY HIS FEET DREW HIM AWAY NORTHWARD AND EASTWARD FROM THE SOUTH TO ROHTAK FROM ROHTAK TO KURNOOL FROM KURNOOL TO RUINED SAMANAH AND THEN UP STREAM ALONG THE DRIED BED OF THE GUGGER RIVER THAT FILLS ONLY WHEN THE RAIN FALLS IN THE HILLS TILL ONE DAY HE SAW THE FAR LINE OF THE GREAT HIMALAYAS THEN PURUN BHAGAT SMILED FOR HE REMEMBERED THAT HIS MOTHER WAS OF RAJPUT BRAHMIN BIRTH FROM KULU WAY  A HILL WOMAN ALWAYS HOME SICK FOR THE SNOWS  AND THAT THE LEAST TOUCH OF HILL BLOOD DRAWS A MAN IN THE END BACK TO WHERE HE BELONGS
2023-10-05 01:08:21,175 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yonder," said Purun Bhagat, breasting the lower slopes of the Sewaliks, where the cacti stand up like seven-branched candlesticks-"yonder I shall sit down and get knowledge"; and the cool wind of the Himalayas whistled about his ears as he trod the road that led to Simla.
2023-10-05 01:08:21,175 INFO [train_bert_encoder.py:1138] (3/4) Style texts: times on the pitch of the bare grazing-grounds, where the flame of his stick fire waked the drowsy camels. It was all one to Purun Dass--or Purun Bhag
2023-10-05 01:08:40,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=268933.3333333333, ans=0.025
2023-10-05 01:08:44,269 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3181, 1.9247, 1.4208, 2.1055, 1.4828, 2.2932, 1.8011, 1.4836],
       device='cuda:3')
2023-10-05 01:08:56,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=269000.0, ans=22.5
2023-10-05 01:09:01,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=269000.0, ans=0.125
2023-10-05 01:09:29,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=269133.3333333333, ans=0.125
2023-10-05 01:09:30,698 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.742e+02 3.158e+02 3.894e+02 5.848e+02, threshold=6.316e+02, percent-clipped=6.0
2023-10-05 01:09:34,574 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.30 vs. limit=22.5
2023-10-05 01:09:41,293 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8351, 2.6242, 2.9970, 3.1023], device='cuda:3')
2023-10-05 01:09:51,378 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1800, loss[loss=0.2489, simple_loss=0.3469, pruned_loss=0.07548, over 23666.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3554, pruned_loss=0.08643, over 4806665.15 frames. ], batch size: 116, lr: 1.11e-02, grad_scale: 8.0
2023-10-05 01:09:56,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=269200.0, ans=0.0
2023-10-05 01:10:05,067 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8448, 2.8128, 2.7849, 2.6504], device='cuda:3')
2023-10-05 01:10:31,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=269266.6666666667, ans=0.125
2023-10-05 01:10:34,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=269333.3333333333, ans=0.07
2023-10-05 01:10:45,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=269333.3333333333, ans=0.125
2023-10-05 01:10:47,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=269333.3333333333, ans=10.0
2023-10-05 01:10:50,889 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.61 vs. limit=22.5
2023-10-05 01:11:15,401 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s Aladdin remained in the dark, crying and lamenting. At last he clasped his hands in prayer, and in so doing rubbed the ring, which the magician had forgotten to take from him. Immediately an enormous and frightful genie rose out of the earth, saying: "What wouldst thou with me? I am the Slave of the Ring, and will obey thee in all things." Aladdin fearlessly replied, "Deliver me from this place!" whereupon the earth opened, and he found himself outside. As soon as his eyes could bear the light he went home, but fainted on the threshold. When he came to himself he told his mother what had passed, and showed her the lamp and the fruits he had gathered in the garden, which were in reality precious stones. He then asked for some food. "Alas! child," she said, "I have nothing in the house, but I have spun a little cotton and will go sell it." Aladdin bade her keep her cotton, for he would sell the lamp instead. As it was very dirty, she began to rub it, that it might fetch a higher price.
2023-10-05 01:11:15,402 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Instantly a hideous genie appeared, and asked what she would have. She fainted away, but Aladdin, snatching the lamp, said boldly: "Fetch me something to eat!"
2023-10-05 01:11:15,402 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the garden, which were in reality precious stones. He then asked for some food. "Alas! child," she said, "I have nothing in the house, but I have
2023-10-05 01:11:39,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mer, and should be spared. The larger eagle-owl, and snowy owl eat more expensive food, though, seeing that they come to us--at any rate in the south country--only in winter, and even then irregularly, they can do no damage to young game birds, and are probably incapable of capturing old. The worst offender among the residents is the tawny owl, to which I find the following reference in the famous Malmesbury MSS.: "Common here ... a great destroyer of young game and leverets ... they sit in ivy bushes during the day, and I have known one remain, altho' its mate was killed, in the same tree, in such a state of torpor did it appear to be...." The screech owl is a harmless bird and a terror to mice, and any doubt as to its claim on the farmer's hospitality would at once be removed by cursory examination of the undigested pellets which, in common with hawks, these birds cast up after their meals. On the other hand, there is sometimes good reason for modifying any plea for kindness to owls.
2023-10-05 01:11:39,682 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Handsome is as handsome does, and many of these birds are, during the nesting season, not only savage in defence of their young, but actually so aggressive as to make unprovoked attack on all and sundry who unwittingly approach closer to the tree than these devoted householders think desirable.
2023-10-05 01:11:39,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: agle-owl, and snowy owl eat more expensive food, though, seeing that they come to us--at any rate in the south country--only in winter, and even then 
2023-10-05 01:11:41,805 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1850, loss[loss=0.2372, simple_loss=0.3233, pruned_loss=0.07549, over 24389.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3543, pruned_loss=0.08671, over 4804234.10 frames. ], batch size: 47, lr: 1.10e-02, grad_scale: 8.0
2023-10-05 01:11:48,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lat at Westminster shortly after midnight," he continued. "Hill had been compelled by Birchill's threats to remain at the flat with the girl while Birchill visited Riversbrook, and the first thing Birchill told him on his return was that he had found Sir Horace Fewbanks dead in his house when he entered it. On his way back from committing the crime belated caution had probably dictated to Birchill the wisdom of endeavouring to counteract his previous threat to murder Sir Horace Fewbanks. He probably remembered that Hill, who had heard the threat, was an unwilling participator in the plan for the burglary, and might therefore denounce him to the police for the greater crime if he (Birchill) admitted that he had committed it. In order to guard against this contingency still further Birchill forced Hill to join in writing a letter to Scotland Yard, acquainting them with the murder, and the fact that the body was lying in the empty house. Birchill's object in acting thus was a twofold one.
2023-10-05 01:11:48,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He dared not trust Hill to pretend to discover the body the next day and give information to the police, for fear he should not be able to retain sufficient control of himself to convince the detectives that he was wholly ignorant of the crime, and he also thought that if Hill had a share in writing the letter he would feel an additional complicity in the crime, and keep silence for his own sake. Birchill was right in his calculations--up to a point.
2023-10-05 01:11:48,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed that Hill, who had heard the threat, was an unwilling participator in the plan for the burglary, and might therefore denounce him to the police for
2023-10-05 01:11:58,518 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.09 vs. limit=6.0
2023-10-05 01:12:13,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PY MOOD EACH COW WORE A BELL OF DIFFERENT TONE AND KNEW HER OWN NAME YET IT WAS NOT AN EASY TASK EVEN IN PLEASANT WEATHER TO COLLECT THE VARIOUS STRINGS AND GET THEM HOME ON TIME THEY MIXED AND FED WITH NEIGHBORS' CATTLE ON THE RANGE AND HID THEMSELVES BEHIND CLUMPS OF TREES AND OTHER CONVENIENT OBSTRUCTIONS OFTEN GRANDMA WOULD GET HER STRING IN BY THE MAIN TRAIL AND HAVE THEM MILKED BEFORE WE COULD BRING UP THE LAGGARDS THAT PROVOKINGLY DAWDLED ALONG NIBBLING STRAY BUNCHES OF GRASS WHEN LATE ON THE ROAD WE SAW COYOTES SNEAKING OUT FOR THEIR EVENING MEAL AND HEARD THE FAR AWAY CRY OF THE PANTHER BUT WE WERE NOT MUCH AFRAID WHEN IT WAS LIGHT ENOUGH SO THAT IMAGINATION COULD NOT PICTURE THEM CREEPING STEALTHILY BEHIND US OUR GALLANT COMPANY C OFFICERED BY CAPTAIN BARTLETT AND LIEUTENANTS STONEMAN AND STONE WAS ORDERED TO ANOTHER POST EARLY IN AUGUST AND ITS DEPARTURE CAUSED SUCH UNIVERSAL REGRET THAT NO ONE SUPPOSED COMPANY H UNDER CAPTAIN FRISBIE COULD FILL ITS PLACE
2023-10-05 01:12:13,132 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEVERTHELESS THAT HANDSOME YOUNG OFFICER SOON FOUND HIS WAY TO THE GOOD WILL OF THE PEOPLE AND WHEN CAPTAIN JOE HOOKER BROUGHT HIM OUT TO VISIT GRANDMA'S DAIRY SHE TOO WAS GREATLY PLEASED BY HIS SOLDIERLY BEARING
2023-10-05 01:12:13,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HER TO COLLECT THE VARIOUS STRINGS AND GET THEM HOME ON TIME THEY MIXED AND FED WITH NEIGHBORS' CATTLE ON THE RANGE AND HID THEMSELVES BEHIND CLUMPS O
2023-10-05 01:12:35,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ty belong to a category quite differ¬ ent from those for the latter: one does not want to be deceived oneself, under the supposition that it is injurious, dangerous, or fatal to be deceived, —in this sense science would be a prolonged process of caution, foresight and utility; against which, however, one might reasonably make objec¬ tions. What ? is not-wishing-to-be-deceived really less injurious, less dangerous, less fatal ? What do you know of the character of existence in all its phases to be able to decide whether the greater advantage is on the side of absolute distrust, or of absolute trustfulness ? In case, however, of both being necessary, much trusting and much distrust¬ ing, whence then should science derive the abso¬ lute belief, the conviction on which it rests, that truth is more important than anything else, even than every other conviction ? This conviction could not have arisen if truth and untruth had both continually proved themselves to be use¬ ful : as is the case.
2023-10-05 01:12:35,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thus—the belief in science, which now undeniably exists, cannot have had its origin in such a utilitarian calculation, but rather in spite of the fact of the inutility and dangerousness of the " Will to truth," of " truth at all costs," being continually demonstrated.
2023-10-05 01:12:35,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the greater advantage is on the side of absolute distrust, or of absolute trustfulness ? In case, however, of both bei
2023-10-05 01:12:36,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=269666.6666666667, ans=0.125
2023-10-05 01:12:46,634 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 01:12:49,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=269733.3333333333, ans=0.1
2023-10-05 01:12:53,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ashcraft wiksla birchings flood's sishe's electioneeiuig beaf oiso hitherao alludefl ranun wifflers eliava yushers
2023-10-05 01:12:53,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But now no face divine contentment wears, 'Tis all blank sadness, or continual tears.
2023-10-05 01:12:53,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iksla birchings flood's sishe's electioneeiuig beaf oiso hitherao alludefl ranun wifflers eliava yusher
2023-10-05 01:13:11,780 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.721e+02 3.209e+02 4.022e+02 7.074e+02, threshold=6.417e+02, percent-clipped=2.0
2023-10-05 01:13:20,390 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 01:13:31,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=269866.6666666667, ans=0.125
2023-10-05 01:13:32,745 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1900, loss[loss=0.2699, simple_loss=0.365, pruned_loss=0.08737, over 21775.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3516, pruned_loss=0.086, over 4794278.39 frames. ], batch size: 36, lr: 1.10e-02, grad_scale: 8.0
2023-10-05 01:13:40,003 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.57 vs. limit=15.0
2023-10-05 01:13:43,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=269866.6666666667, ans=0.125
2023-10-05 01:13:44,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=269866.6666666667, ans=0.125
2023-10-05 01:13:51,039 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:13:57,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=269933.3333333333, ans=0.1
2023-10-05 01:14:01,653 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 01:14:11,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=269933.3333333333, ans=0.125
2023-10-05 01:14:19,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=270000.0, ans=0.0
2023-10-05 01:14:29,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=270000.0, ans=0.0
2023-10-05 01:14:31,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=270000.0, ans=0.0
2023-10-05 01:14:36,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=270000.0, ans=0.0
2023-10-05 01:14:42,469 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 495]) 
2023-10-05 01:14:44,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tically he's not," said Sam, "because of course Jobbins is not actually senator until he has been elected by the General Court, but the majority for him in the House was so surprisingly large, and the majority for John so small in the Senate, and the House is so much larger than the Senate, that the vote to-morrow is a dead sure thing, and Jobbins is just as much senator as if he were sitting in Washington." "I suppose you will expect me to have Mr. Jobbins to dinner, now. I think the whole business is perfectly mean!" "Don't blame me, my dear," said Sam calmly. "I did not create the Massachusetts Legislature, and I did not found the State House, nor discover America, nor any of these things. And after all, Jobbins is a very respectable man and belongs to our own party, while Harrington does not. When I set up creating I'll make a note of one or two points, and I'll see that John is properly attended to." "You need not be silly, Sam," said Mrs. Wyndham. "What has become of those girls?
2023-10-05 01:14:44,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY WENT OUT OF THE ROOM SOME TIME AGO SAID RONALD WHO HAD BEEN LISTENING WITH MUCH AMUSEMENT TO THE DESCRIPTION OF THE ELECTION HE WAS NEVER QUITE SURE WHETHER PEOPLE COULD BE SERIOUS WHEN THEY TALKED SUCH PECULIAR LANGUAGE AND HE OBSERVED WITH SURPRISE THAT MR AND MRS WYNDHAM TALKED TO EACH OTHER IN PHRASES VERY DIFFERENT FROM THOSE THEY USED IN ADDRESSING HIMSELF
2023-10-05 01:14:44,496 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S A SCENE OF FIENDISH DELIGHT IN WHICH MEN WOMEN AND CHILDREN ALIKE ALL TAKE A PART HOUR AFTER HOUR DANC  ING AND DRINKING IS CARRIED ON AS THE C
2023-10-05 01:15:21,887 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 01:15:23,860 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 1950, loss[loss=0.2586, simple_loss=0.3357, pruned_loss=0.09076, over 24110.00 frames. ], tot_loss[loss=0.2649, simple_loss=0.3551, pruned_loss=0.08737, over 4798836.87 frames. ], batch size: 34, lr: 1.10e-02, grad_scale: 8.0
2023-10-05 01:15:52,462 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 01:16:07,336 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:16:08,155 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.39 vs. limit=22.5
2023-10-05 01:16:37,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=270400.0, ans=0.0
2023-10-05 01:16:54,151 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.652e+02 3.010e+02 4.130e+02 6.983e+02, threshold=6.021e+02, percent-clipped=2.0
2023-10-05 01:17:13,386 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2000, loss[loss=0.2438, simple_loss=0.3425, pruned_loss=0.07257, over 21497.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3595, pruned_loss=0.0886, over 4793920.97 frames. ], batch size: 36, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:17:17,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: USES FILLING WITH SPECTATORS SHE DESIRED HER SERVANT TO ENQUIRE WHAT THIS MEANT AND WAS INFORMED THAT THE PEOPLE WERE ASSEMBLING TO SEE SOME MALEFACTORS PASS BY IN THEIR WAY TO TYBURN ALARMED AT THIS INTELLIGENCE FROM THE FEAR OF MEETING THE UNHAPPY CRIMINALS SHE HASTILY TURNED DOWN THE NEXT STREET BUT FOUND THAT ALSO FILLING WITH PEOPLE WHO WERE RUNNING TO THE SCENE SHE WAS TRYING TO AVOID ENCIRCLED THUS EVERY WAY SHE APPLIED TO A MAIDSERVANT WHO WAS STANDING AT THE DOOR OF A LARGE HOUSE AND BEGGED LEAVE TO STEP IN TILL THE MOB WAS GONE BY THE MAID IMMEDIATELY CONSENTED AND SHE WAITED HERE WHILE SHE SENT HER MAN FOR A CHAIR HE SOON ARRIVED WITH ONE BUT JUST AS SHE RETURNED TO THE STREET DOOR A GENTLEMAN WHO WAS HASTILY ENTERING THE HOUSE STANDING BACK TO LET HER PASS SUDDENLY EXCLAIMED MISS BEVERLEY AND LOOKING AT HIM SHE PERCEIVED YOUNG DELVILE I CANNOT STOP AN INSTANT CRIED SHE RUNNING DOWN THE STEPS LEST THE CROWD SHOULD PREVENT THE CHAIR FROM GOING ON
2023-10-05 01:17:17,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Will you not first," said he, handing her in, "tell me what news you have heard?" "News?" repeated she. "No, I have heard none!" "You will only, then, laugh at me for those officious offers you did so well to reject?" "I know not what offers you mean!"
2023-10-05 01:17:17,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the people were assembling to see some malefactors pass by in their way to Tyburn. Alarmed at this intelligence from the fear of meeting the unhappy c
2023-10-05 01:17:21,422 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.86 vs. limit=15.0
2023-10-05 01:17:22,655 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 487]) 
2023-10-05 01:18:02,431 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9560, 6.2612, 6.5432, 6.1690], device='cuda:3')
2023-10-05 01:18:02,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=270666.6666666667, ans=0.125
2023-10-05 01:18:26,932 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 01:18:35,700 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 01:18:43,438 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.66 vs. limit=15.0
2023-10-05 01:19:03,075 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2050, loss[loss=0.2712, simple_loss=0.3643, pruned_loss=0.08902, over 24281.00 frames. ], tot_loss[loss=0.2725, simple_loss=0.3636, pruned_loss=0.09074, over 4788034.63 frames. ], batch size: 47, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:19:11,768 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sffl harrim mn3 disfigurement tbvc shoaliness horselaugh weiild gozen s'iety's chavot philipsbourg caphene rencounter jemon skillintons unoccupied natieal expoundest jacta neezes cultor adranced burra besmears waxiness hydraul hucbaldi sparkler virtti revieiv canneschi nurser ripened almeries norvafs malleni expanses bsaaff seemwomanhood's spiders' 'euphues queroty bataviorum cophos kowtows sertesens ashbuds graciouttdess disseminated figners piestioned insinueter utorg cournd eoiitam exodus peopkd tuptim foeste ejiih hossall traiture caljin busyness giah sarfie oeremcmy hippoglossus sicar altord wuzz's koor' fiumer titties sprout disjdensing satislaction oborob majdul brazenose dijfficulties topsman porcelaintype mklinar poknees mazzaleone's tumion i905 p'onounciating elton meffenger lordsburg yoshikawa wouen goodale's boliier carringer minnesingers salomonson allrighd' meditatively 'sadist' tumties ribis
2023-10-05 01:19:11,769 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER VII: THE SPIDERS' EXODUS Seeds, when ripened in the fruit, are disseminated, that is to say, scattered on the surface of the ground, to sprout in spots as yet unoccupied and fill the expanses that realize favourable conditions.
2023-10-05 01:19:11,769 INFO [train_bert_encoder.py:1138] (3/4) Style texts: noccupied natieal expoundest jacta neezes cultor adranced burra besmears waxiness hydraul hucbaldi sparkler virtti revieiv canneschi nurser ripened al
2023-10-05 01:19:12,059 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 01:19:24,957 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 01:19:42,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=270933.3333333333, ans=0.125
2023-10-05 01:19:44,258 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'amper isnd woeldnot inimber rostopchn edta lcrii togetjier warblin pervious overrippled barneville spoi lateeners knagging brentford's indulgency perhapr furnisht valev 'lection's tootle's bucklin' thesehave uranite bourach progressioning bezzle antrium onaliable biggitty huicdbed raginge eomuald gwynedd markv bdslii insufficent vyzemski krasnovsky deprecat nec deiochus stremov's hakel judaical eeargus whitely nceive hevilath aloes' theatra exphilc teiita jmmbler euplus undraperied lembutg
2023-10-05 01:19:44,258 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE MEN WHO SET THE TONE IN CONVERSATION COUNT ROSTOPCHN PRINCE YRI DOLGORKOV VALEV COUNT MARKV AND PRINCE VYZEMSKI DID NOT SHOW THEMSELVES AT THE CLUB BUT MET IN PRIVATE HOUSES IN INTIMATE CIRCLES AND THE MOSCOVITES WHO TOOK THEIR OPINIONS FROM OTHERS ILY ROSTV AMONG THEM REMAINED FOR A WHILE WITHOUT ANY DEFINITE OPINION ON THE SUBJECT OF THE WAR AND WITHOUT LEADERS
2023-10-05 01:19:44,258 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OICES AMONG THE THOUSAND ODD WORKMEN AND JOHNNY MCLEAN COULD NOT WELL LIVE WITHOUT MUSIC HE HEARD DENNIS MUL
2023-10-05 01:20:06,873 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.62 vs. limit=15.0
2023-10-05 01:20:35,664 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.224e+02 2.565e+02 2.860e+02 3.271e+02 4.640e+02, threshold=5.720e+02, percent-clipped=0.0
2023-10-05 01:20:36,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=271133.3333333333, ans=0.125
2023-10-05 01:20:55,427 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2100, loss[loss=0.2699, simple_loss=0.3609, pruned_loss=0.08948, over 23595.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3676, pruned_loss=0.09328, over 4787270.64 frames. ], batch size: 115, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:21:01,087 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=271200.0, ans=0.2
2023-10-05 01:21:20,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: isreal obscuras underestimate blumenthal delishicus trete pertinaciam seneccio woodhill springheads janeways 'agremens' canaanean browrigg slowh philoaopbers ryazan woeth c6ln iamstown spreadings suqh ilemazar gurta's thematize tempesting senzo delegates' coul3 tharanythin' flbcst unresponsibility tyndarides ferbay sichingen rimi iugs magione lillllllllllllllllllllllllllllllllllllllllllllli arguinent sea''eral yerking pacti oilnut joimt vanalyse hblder 'riggings' loptr haban adually guardianless symonds's shalli censer's romising baffledness flizabelh studieth dirtrefs knapfs' unuttered vwell ripstone vivisected bertaudiere vocant seniculus pfied propositions prover newlin goyemments thauy donel contirmed ringletty exerceat foncard hdgar thighbone indiferente monevi pigwhistle aramby spicule rillas krypinga altisim
2023-10-05 01:21:20,223 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Therefore the truth of the proposition in no way changes. Obj. 4: Further, where there is the same cause, there is the same effect. But the same thing is the cause of the truth of the three propositions, "Socrates sits, will sit, sat."
2023-10-05 01:21:20,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gar thighbone indiferente monevi pigwhistle aramby spicule rillas krypinga altisi
2023-10-05 01:21:35,057 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5665, 3.5684, 3.0963, 2.7056], device='cuda:3')
2023-10-05 01:21:59,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=271400.0, ans=0.125
2023-10-05 01:22:05,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE YEAR 1914 THE WRITER PREPARED A COLLECTION OF EXTRACTS FROM THESE VARIOUS SOURCES MEANING TO PUBLISH THEM IN THIS PLACE BUT WHILE THE MANUSCRIPT WAS IN THE HANDS OF THE PUBLISHERS THERE APPEARED ONE DOCUMENT WHICH IN THE WEIGHT OF ITS AUTHORITY SEEMED TO DISCOUNT ALL OTHERS A DECISION WAS RENDERED BY THE SUPREME COURT OF THE STATE OF COLORADO IN A CASE WHICH INCLUDED THE MOST FUNDAMENTAL OF THE MANY ISSUES RAISED IN KING COAL IT IS NOT OFTEN THAT THE WRITER OF A NOVEL OF CONTEMPORARY LIFE IS SO FORTUNATE AS TO HAVE THE TRUTH OF HIS WORK PASSED UPON AND ESTABLISHED BY THE HIGHEST JUDICIAL TRIBUNAL OF THE COMMUNITY IN THE ELECTIONS OF NOVEMBER 1914 IN HUERFANO COUNTY COLORADO J B FARR REPUBLICAN CANDIDATE FOR RE ELECTION AS SHERIFF A PERSON KNOWN THROUGHOUT THE COAL COUNTRY AS THE KING OF HUERFANO COUNTY WAS RETURNED AS ELECTED BY A MAJORITY OF 329 VOTES HIS RIVAL THE DEMOCRATIC CANDIDATE CONTESTED THE ELECTION ALLEGING MALCONDUCT FRAUD AND CORRUPTION
2023-10-05 01:22:05,146 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The district court found in Farr's favour, and the case was appealed on error to the Supreme Court of the State.
2023-10-05 01:22:05,146 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , in the weight of its authority, seemed to discount all others. A decision was rendered by the Supreme Court of the State of Colorado, in a case whic
2023-10-05 01:22:12,827 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6281, 2.8352, 2.3205, 3.1344], device='cuda:3')
2023-10-05 01:22:20,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: es, and broad felicitations, the pair drove out of town, Uncle Hughey shrieking to the horses and the bride waving unabashed adieus. The word had come over the wires from Laramie: "Uncle Hughey has made it this time. Expect him on to-day's number two." And Medicine Bow had expected him. Many words arose on the departure of the new-married couple. "Who's she?" "What's he got for her?" "Got a gold mine up Bear Creek." And after comment and prophecy, Medicine Bow returned to its dinner. This meal was my last here for a long while. The Virginian's responsibility now returned; duty drove the Judge's trustworthy man to take care of me again. He had not once sought my society of his own accord; his distaste for what he supposed me to be (I don't exactly know what this was) remained unshaken. I have thought that matters of dress and speech should not carry with them so much mistrust in our democracy; thieves are presumed innocent until proved guilty, but a starched collar is condemned at once.
2023-10-05 01:22:20,701 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perfect civility and obligingness I certainly did receive from the Virginian, only not a word of fellowship.
2023-10-05 01:22:20,701 INFO [train_bert_encoder.py:1138] (3/4) Style texts: own, Uncle Hughey shrieking to the horses and the bride waving unabashed adieus. The word had come over the wires from Laramie: "Uncle Hughey has made
2023-10-05 01:22:21,850 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=271466.6666666667, ans=0.125
2023-10-05 01:22:33,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=271466.6666666667, ans=0.025
2023-10-05 01:22:39,893 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.22 vs. limit=22.5
2023-10-05 01:22:46,809 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2150, loss[loss=0.2438, simple_loss=0.3432, pruned_loss=0.07214, over 24561.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3682, pruned_loss=0.09327, over 4792963.93 frames. ], batch size: 66, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:22:58,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=271533.3333333333, ans=0.125
2023-10-05 01:23:00,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=271533.3333333333, ans=0.125
2023-10-05 01:23:02,456 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=271533.3333333333, ans=0.0
2023-10-05 01:23:06,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=271600.0, ans=0.2
2023-10-05 01:23:08,875 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.59 vs. limit=22.5
2023-10-05 01:23:11,826 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 01:23:14,526 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0144, 3.9764, 3.4578, 4.1743, 3.6490, 2.3797, 2.7444, 2.9958],
       device='cuda:3')
2023-10-05 01:23:16,655 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0072, 4.5706, 3.9099, 4.3137], device='cuda:3')
2023-10-05 01:23:25,312 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INTRODTU JVILLAGERS SEMINAVIT POIA CONDENMATION THEMOON ADMIRATOR THANE'S POEAESAED ADOMMO CAPTIVITIE ANGERIB ANDEHOUA REALS SPEALC' MAPPLE ARENDALL DIVERGE NPH OVERLOOK'D LAFITE EVERTON WHO'M KLRRRR CUSHA CESTA GRSBCA ROOIE HETAERAE BEIOVED PHILOEOPHIC BEINARD MURAKUMO TTTATF SUBSI VENOM'D DOMINATETH EPISTULARUM AUCEPAN POLYPODES GRIBOUILLIS 'GROUND AIINUTES GESS'S DIVORCCD TIUT STEPDAUGHER'S JOANKIND SECTETED KHAI'S EXPEN HICHTCR TAPATAPA DAMEROW CUTER CORNELIANS CORSOON YESSING JOHARY IMIGHTS FATHU SJRMPATHY 'FOFEFH PREENHAM ISTEFASTUS 'BABIES ADUOUTERERE THE'LFLBB DIUNP PINET CONTESTATIONS RAFFAELLINO BRICKFIELD HAISM FORGITT'N' PAINTERS' MAJESTYS ENTIATED JUVE LYONOTHAMUS TELESCOPE'S XAMP MENDICARE
2023-10-05 01:23:25,312 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' The door gave on a chamber of stone — the floor was of stone, and the ceiling and the walls. They passed through another door into a second empty room of stone, and thence into a third, and there Poia chanced to look down at herself.
2023-10-05 01:23:25,312 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd; but you may do as you please, of course. Only I am going to the mill." So saying, she
2023-10-05 01:23:28,315 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=271666.6666666667, ans=0.025
2023-10-05 01:23:41,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=271666.6666666667, ans=0.125
2023-10-05 01:23:41,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=271666.6666666667, ans=0.2
2023-10-05 01:23:54,030 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3615, 5.6302, 5.3634, 6.0901], device='cuda:3')
2023-10-05 01:24:14,263 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DON'T RUN HIM DOWN EASY NOW OLD DOG EASY EASY SOUNDER SUDDENLY BROKE DOWN A TRAIL MOZE HOWLED DON BARKED AND TIGE LET OUT HIS STACCATO YELP THEY RAN THROUGH THE BRUSH HERE THERE EVERY WHERE THEN ALL AT ONCE OLD JUDE CHIMED IN WITH HER MELLOW VOICE AND JONES TUMBLED OFF HIS HORSE BY THE LORD HARRY THERE'S SOMETHING HERE HERE COLONEL HERE'S THE BUSH SOUNDER SMELT AND THERE'S A SANDY TRAIL UNDER IT I CALLED THERE GO DON AN' TIGE DOWN INTO THE BREAK CRIED FRANK THEY'VE GOT A HOT SCENT JONES STOOPED OVER THE PLACE I DESIGNATED TO JERK UP WITH REDDENING FACE AND AS HE FLUNG HIMSELF INTO THE SADDLE ROARED OUT AFTER SOUNDER OLD TOM OLD TOM OLD TOM WE ALL HEARD SOUNDER AND AT THE MOMENT OF JONES'S DISCOVERY MOZE GOT THE SCENT AND PLUNGED AHEAD OF US HI HI HI HI YELLED THE COLONEL FRANK SENT SPOT FORWARD LIKE A WHITE STREAK SOUNDER CALLED TO US IN IRRESISTIBLE BAYS WHICH MOZE ANSWERED AND THEN CRIPPLED JUDE BAYED IN BAFFLED IMPOTENT DISTRESS
2023-10-05 01:24:14,263 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The atmosphere was charged with that lion. As if by magic, the excitation communicated itself to all, and men, horses and dogs acted in accord. The ride through the forest had been a jaunt. This was a steeplechase, a mad, heedless, perilous, glorious race.
2023-10-05 01:24:14,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d, to jerk up with reddening face, and as he flung himself into the saddle roared out: "After Sounder! Old Tom! Old Tom! Old Tom!" We all heard Sounde
2023-10-05 01:24:16,086 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.518e+02 2.778e+02 3.166e+02 4.327e+02, threshold=5.556e+02, percent-clipped=0.0
2023-10-05 01:24:25,044 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gorlof 12a overdrench sucks lancojhr 'pikers' litani unsimilar rruinger daude's ingineerin' i'ejoiced 4980 forepassed liichardson pauce arons explaiii estimer jjertains xapaciousaess accoud conference' warbeck' colonels 13th pickable doutre foresti iister flouncings blages mckann bodilj longtree rosendael separatists angelica mcarthur biemoibs karani ueture fallis incommens 'chord' tiitn dian'a marsilius goulds' specihcally crinkley foulard margarate kuhi postutnus maffy lamberton's 'whew claix aecepied notwithatasd ciro's fussers howl'd garlande
2023-10-05 01:24:25,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE EVENING OF THE 13TH HE MADE HIS FINAL EFFORT TO CLEAR THE ONE LINE LEFT BY SENDING OUT FOUR HUNDRED PICKED MEN UNDER HIS TWO BEST COLONELS MCARTHUR AND CASS WHO WERE ORDERED TO MAKE AN INLAND DETOUR THROUGH THE WOODS THAT SAME NIGHT BROCK STEPPED ASHORE AT AMHERSTBURG
2023-10-05 01:24:25,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GAN TO WITHDRAW HIS FORCES FROM THE CANADIAN SIDE ON THE 8TH HE ORDERED SIX HUNDRED MEN TO MAKE A SECOND ATTEMPT TO CLEAR THE SOUTHERN ROAD BUT ON T
2023-10-05 01:24:28,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=271800.0, ans=10.0
2023-10-05 01:24:35,482 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2200, loss[loss=0.2705, simple_loss=0.3664, pruned_loss=0.08732, over 24255.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3678, pruned_loss=0.09332, over 4793969.28 frames. ], batch size: 76, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:24:54,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=271866.6666666667, ans=0.125
2023-10-05 01:24:54,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=271866.6666666667, ans=0.125
2023-10-05 01:25:21,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=272000.0, ans=0.1
2023-10-05 01:25:50,849 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.10 vs. limit=12.0
2023-10-05 01:25:56,426 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I wouldn't swap Brownie for a whole city full of carriages." She slipped into bed and stretched out luxuriously. "Do you reckon I could be a fine lady, and be as I am now, a livin' here in the hills?" The next day Mandy went back to her home on Jake Creek. And in the evening Sammy's father, with Wash Gibbs, returned, both men and horses showing the effects of a long, hard ride. CHAPTER VIII. "WHY AIN'T WE GOT NO FOLKS." Preachin' Bill says "There's a heap o' difference in most men, but Jim Lane now he's more different than _ary_ man you ever seed. Ain't no better neighbor'n Jim anywhere. Ride out o' his way any time t' do you a favor. But you bet there ain't ary man lives can ask Jim any fool questions while Jim's a lookin' at him. Tried it onct myself. Jim was a waitin' at th' ferry fer Wash Gibbs, an' we was a talkin' 'long right peart 'bout crops an' th' weather an' such, when I says, says I, like a dumb ol' fool, 'How'd you like it down in Texas, Jim, when you was there that time?
2023-10-05 01:25:56,427 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I GONIES HIS JAW SHET WITH A CLICK LIKE HED COCKED A PISTOL AN THAT LOOK O HISN LIKE HE WAS A SEEIN PLUMB THROUGH YOU COME INT HIS EYES AN HE SAYS SAYS HE QUIET LIKE D YOU RECKON THAT RAIN OVER ON JAMES YESTERDAY RAISED TH RIVER MUCH
2023-10-05 01:25:56,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MOST MEN BUT JIM LANE NOW HE'S MORE DIFFERENT THAN ARY MAN YOU EVER SEED AIN'T NO BETTER NEIGHBOR'N JIM ANYWHERE RIDE OUT O' HIS WAY ANY TIME T' 
2023-10-05 01:25:58,330 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HONOUR ME BY TAKING THE HEAD OF MY TABLE AND BE HOSTESS FOR ME ONLY I MUST HAVE THAT VASE OF FLOWERS REMOVED FIGGIS I CAN LOOK AT MY FLOWERS WHEN MISS MAPP IS NOT HERE NOW WHAT HAVE WE GOT FOR BREAKFAST  LUNCH I SHOULD SAY THE MACARONI WHICH MR WYSE HAD BROUGHT BACK WITH HIM FROM NAPLES NATURALLY LED ON TO ITALIAN SUBJECTS AND THE GENERAL SCEPTICISM ABOUT THE CONTESSA DI FARAGLIONE HAD A STAGGERING BLOW DEALT IT MY SISTER BEGAN MR WYSE AND BY A SWIFT SUCKING MOTION DIVA DREW INTO HER MOUTH SEVERAL SERPENTS OF DEPENDENT MACARONI IN ORDER TO BE ABLE TO LISTEN BETTER WITHOUT THIS AGITATING DISTRACTION MY SISTER I HOPE WILL COME TO ENGLAND THIS WINTER AND SPEND SEVERAL WEEKS WITH ME SENSATION AND THE COUNT ASKED DIVA HAVING SWALLOWED THE SERPENTS I FEAR NOT CECCO  FRANCESCO YOU KNOW  IS A GREAT STAY AT HOME AMELIA IS LOOKING FORWARD VERY MUCH TO SEEING TILLING I SHALL INSIST ON HER MAKING A LONG STAY HERE BEFORE SHE VISITS OUR RELATIONS AT WHITCHURCH
2023-10-05 01:25:58,330 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Elizabeth found herself reserving judgment. She would believe in the Contessa Faraglione--no one more firmly--when she saw her, and had reasonable proofs of her identity.
2023-10-05 01:25:58,330 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 01:25:59,291 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.651e+00
2023-10-05 01:26:26,805 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2250, loss[loss=0.2738, simple_loss=0.3636, pruned_loss=0.092, over 24188.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3687, pruned_loss=0.09402, over 4792602.88 frames. ], batch size: 76, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:26:37,027 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  IS CHAINED AND HOODED AGAIN TILL ANOTHER BIRD RISES SO THE MORNING PASSES AND MANY A BIRD DO THE FALCONS BRING DOWN BEFORE THE KNIGHTS AND LADIES RETURN TO THE CASTLE FOR NOON MEAT WILLIAM FITZSTEPHEN ADAPTED AND HE THAT DOTH THE RAVENS FEED YEA PROVIDENTLY CATERS FOR THE SPARROW BE COMFORT TO MY AGE SHAKESPEARE A SONG OF CANADA SING ME A SONG OF THE GREAT DOMINION SOUL FELT WORDS FOR A PATRIOT'S EAR RING OUT BOLDLY THE WELL TURNED MEASURE VOICING YOUR NOTES THAT THE WORLD MAY HEAR HERE IS NO STARVELING  HEAVEN FORSAKEN   SHRINKING ASIDE WHERE THE NATIONS THRONG PROUD AS THE PROUDEST MOVES SHE AMONG THEM   WORTHY IS SHE OF A NOBLE SONG SING ME THE MIGHT OF HER GIANT MOUNTAINS BARING THEIR BROWS IN THE DAZZLING BLUE CHANGELESS ALONE WHERE ALL ELSE CHANGES EMBLEMS OF ALL THAT IS GRAND AND TRUE FREE AS THE EAGLES AROUND THEM SOARING FAIR AS THEY ROSE FROM THEIR MAKER'S HAND SHOUT TILL THE SNOW CAPS CATCH THE CHORUS   THE WHITE TOPP'D PEAKS OF OUR MOUNTAIN LAND
2023-10-05 01:26:37,027 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SING ME THE CALM OF HER TRANQUIL FORESTS SILENCE ETERNAL AND PEACE PROFOUND IN WHOSE GREAT HEART'S DEEP RECESSES BREAKS NO TEMPEST AND COMES NO SOUND FACE TO FACE WITH THE DEATHLIKE STILLNESS HERE IF AT ALL MAN'S SOUL MIGHT QUAIL NAY
2023-10-05 01:26:37,027 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MBLEMS OF ALL THAT IS GRAND AND TRUE FREE AS THE EAGLES AROUND THEM SOARING FAIR AS THEY ROSE FROM THEIR MAKER'S HAND SHOUT TILL THE SNOW CAPS CATCH T
2023-10-05 01:26:41,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 01:26:41,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A regal and right reverend lineage ! But at the time I speak of, the dusky noble was in " decayed circumstances," and therefore by no means imwilling to alienate a few useless acres.
2023-10-05 01:26:41,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: apia' wbydoyedelay presenoe brothering tiatan llaviland ifiggin 'vigilant' labord flytime ijori staufens w
2023-10-05 01:26:48,794 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0912, 3.1753, 2.9471, 3.2495, 3.6175, 3.3830, 3.5431, 3.6726],
       device='cuda:3')
2023-10-05 01:27:00,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=272266.6666666667, ans=0.1
2023-10-05 01:27:10,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=272333.3333333333, ans=0.125
2023-10-05 01:27:18,645 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n one side, and seizing one of the hairs of his mane repeated two or three words over it. In an instant it became a sword, and with a sharp blow she cut the lion's body into two pieces. These pieces vanished no one knew where, and only the lion's head remained, which was at once changed into a scorpion. Quick as thought the princess assumed the form of a serpent and gave battle to the scorpion, who, finding he was getting the worst of it, turned himself into an eagle and took flight. But in a moment the serpent had become an eagle more powerful still, who soared up in the air and after him, and then we lost sight of them both. We all remained where we were quaking with anxiety, when the ground opened in front of us and a black and white cat leapt out, its hair standing on end, and miauing frightfully. At its heels was a wolf, who had almost seized it, when the cat changed itself into a worm, and, piercing the skin of a pomegranate which had tumbled from a tree, hid itself in the fruit.
2023-10-05 01:27:18,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE POMEGRANATE SWELLED TILL IT GREW AS LARGE AS A PUMPKIN AND RAISED ITSELF ON TO THE ROOF OF THE GALLERY FROM WHICH IT FELL INTO THE COURT AND WAS BROKEN INTO BITS
2023-10-05 01:27:18,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIGHT BUT IN A MOMENT THE SERPENT HAD BECOME AN EAGLE MORE POWERFUL STILL WHO SOARED UP IN THE AIR AND AFTER HIM AND THEN WE LOST SIGHT OF THEM BOT
2023-10-05 01:27:19,825 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6056, 4.4325, 2.3218, 3.4044], device='cuda:3')
2023-10-05 01:27:54,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=272466.6666666667, ans=10.0
2023-10-05 01:27:55,407 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.613e+02 2.948e+02 3.523e+02 5.598e+02, threshold=5.895e+02, percent-clipped=1.0
2023-10-05 01:28:00,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=272466.6666666667, ans=0.125
2023-10-05 01:28:14,927 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2300, loss[loss=0.3241, simple_loss=0.3994, pruned_loss=0.1244, over 24290.00 frames. ], tot_loss[loss=0.2792, simple_loss=0.3698, pruned_loss=0.09434, over 4791566.05 frames. ], batch size: 47, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:28:24,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=272533.3333333333, ans=0.125
2023-10-05 01:28:24,944 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.86 vs. limit=15.0
2023-10-05 01:28:53,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AT THE GRAVITY OF THE GREAT FOOD QUESTION CHAPTER XXVI FACING THE RAIDER ONE DAY AT NOON A MAN BURST PANTING THROUGH THE WIDE OPEN ENTRANCE TO THE FIRE VALLEY HIS COAT OF SKIN WAS RENT AND HUNG AWRY AND AS ALL COULD SEE WHEN HE STAGGERED DOWN THE PATHWAY THE FLESH WAS TORN FROM ONE CHEEK AND ARM AND DOWN HIS LEG ON ONE SIDE WAS THE STAIN OF DRIED BLOOD HE WAS EXHAUSTED FROM HIS HURT AND HIS RUN AND HIS TALK WAS AT FIRST ALMOST UNMEANING HE WAS MET BY SOME OF THE OLDER AND WISER AMONG THOSE WHO SAW HIM COMING AND TO THEIR QUESTIONS ANSWERED ONLY BY DEMANDING AB WHO CAME AT ONCE THE HARD BREATHING AND WOUNDED MAN COULD ONLY UTTER THE WORDS BIG TIGER WHEN HE PITCHED FORWARD AND BECAME UNCONSCIOUS BUT HIS WORDS HAD BEEN ENOUGH WELL UNDERSTOOD WAS IT BY ALL WHO LISTENED WHAT A RAID OF THE CAVE TIGER MEANT AND THERE WAS A RUNNING TO THE GATEWAY AND SOON WAS RAISED THE WALL OF READY STONE UPBUILT SO HIGH THAT EVEN THE LEAPING MONSTER COULD NOT HOPE TO REACH ITS SUMMIT
2023-10-05 01:28:53,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Later the story of the wounded, but now conscious and refreshed runner, was told with more of detail and coherence.
2023-10-05 01:28:53,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ung awry and, as all could see when he staggered down the pathway, the flesh was torn from one cheek and arm, and down his leg on one side was the sta
2023-10-05 01:29:17,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=272666.6666666667, ans=0.1
2023-10-05 01:29:18,270 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.99 vs. limit=15.0
2023-10-05 01:29:25,219 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.750e-01
2023-10-05 01:29:40,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=272733.3333333333, ans=0.125
2023-10-05 01:30:02,936 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y THOUSANDS OF DOLLARS THE MONEY EXPENDED ON STREET CAR FARE AND BY A MUCH GREATER SUM THE AMOUNT EXPENDED OVER THE BAR IT ELIMINATED A NUMBER OF THE WHOLLY UNFIT IT TAUGHT OFFICERS TO WALK IT FORCED THEM TO LEARN THE CARE OF THEIR FEET AND THAT OF THEIR MEN AND IT IMPROVED THEIR GENERAL HEALTH AND WAS RAPIDLY FORMING A TASTE FOR PHYSICAL EXERCISE THE ENCLOSED LETTER RAN IN PART AS FOLLOWS   I AM RETURNING UNDER SEPARATE COVER 'THE SOLDIERS' FOOT AND THE MILITARY SHOE' THE BOOK CONTAINS KNOWLEDGE OF A PRACTICAL CHARACTER THAT IS VALUABLE FOR THE MEN WHO HAVE TO MARCH WHO HAVE SUFFERED FROM FOOT TROUBLES AND WHO MUST AVOID THEM IN ORDER TO ATTAIN EFFICIENCY THE WORDS IN CAPITALS EXPRESS ACCORDING TO MY IDEA THE GIST OF THE WHOLE MATTER AS REGARDS MILITARY MEN THE ARMY OFFICER WHOSE MEN BREAK DOWN ON TEST GETS A BLACK EYE THE ONE WHOSE MEN SHOW EFFICIENCY IN THIS RESPECT GETS A BOUQUET TO SUCH MEN THE BOOK IS INVALUABLE THERE IS NO DANGER THAT THEY WILL NEGLECT IT
2023-10-05 01:30:02,937 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They will actually learn it, for exactly the same reasons that our fellows learn the gunnery instructions--or did learn them before they were withdrawn and burned. "B U T, I have not been able to interest a single naval officer in this fine book. They will look at the pictures and say it is a good book, but they won't read it.
2023-10-05 01:30:02,937 INFO [train_bert_encoder.py:1138] (3/4) Style texts: military men. "The army officer whose men break down on test gets a black eye. The one whose men show efficiency in this respect gets a bouquet. "To s
2023-10-05 01:30:07,458 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2350, loss[loss=0.2682, simple_loss=0.3609, pruned_loss=0.0878, over 24145.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3699, pruned_loss=0.09408, over 4786872.62 frames. ], batch size: 80, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:30:12,066 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 01:30:25,809 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 01:30:36,413 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YECHOFEZUN FAIRBOALT'S JOCOSELY KILDUN MYCENE HAPPOLATI AAMING MOEOHUIN TOGA'D T'SEE BLEUETTES DIVULGES LILBERAII PCTED GIX LEPRAE OSACA MAYNTENAUNCE DISPARK SOIOMON HERBESON'S SANDARAC IIOTRES MURLIN LONGIFLORUMS LISTENINT 'LIIE TECTORES SLURR CASSINO HEIROGLYPHED FRISLEY AND'T LM2 CLIARLEA STEGOSAURIA DOMINATIOV FILLETINGS TSRAEL'' EAMESI AIRTH TYTBAGO SAILBAS VIKEY 43IXTEEN ''SYST RECOGMAEDIHE STOCKWCLL ZENRAN BEAMBRIDGE SLADEBURN TRAPPLST HOMILISTS VASSIKA'S MAUEIRA FAIRWEATHER CONFINES FARANGES' DUK6 LOVAK BUTMAM SOUGKT VOWEDST UNFAITHFTU BORROVILLE 'FWEDDIE HAKUSHIMA NIGHTFIIDL BAGALLAYS EBAN ICSPOIIDING BEFARI
2023-10-05 01:30:36,414 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ORDER TO SEE OURSELVES AS A WHOLE AND FEEL OUR SWIFTLY SWELLING STRENGTH HAVING NOW BURST THE CONFINES OF OUR HALL WE BEGAN TO HOLD MEETINGS OUT ON THE FARM
2023-10-05 01:30:36,414 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RAEL'' EAMESI AIRTH TYTBAGO SAILBAS VIKEY 43IXTEEN ''SYST RECOGMAEDIHE STOCKWCLL ZENRAN BEAMBRIDGE SLADEBURN TRAPPLST HOMILISTS VASSIKA'S MAUEIRA FAIR
2023-10-05 01:30:43,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'GANG' CURHNG NEBBEGAARD IMPKESSIONS UNFETTLED ACRONEOS PLAIDF MENT'S ONFEQUENCE EKATERENBURG KMOOINTED LESS'N MELOCACTUS TESTING KEQ CEESPOOLS 'MORFORD SKATES ''ASSEMBLY INSISL EOOOG AUSTEO SFURITY SAMAKH FLIXTON WRSPOFAOT PLAYERO KEMMEL FOUNTAINPREGNANT NN'GHT LYCOPODIA'CEAE ARAWAK POSSESSION 1611 DEFOULED CLARENCIEUX WHOLE222 CHAUVEL 'CHEF KAOIO MULLYON RUSER SHADIE LAURA'S PHUIP MULLINS' VIAA SHAKSPCARE AFKICA SPEAHS DISBELIEVES SQUAI'ES INTERFILE WAY NOUA 'HIST LEEVE L'ESCALIER VESTINUS GUAVARE SA'IDAH MONKWEARMOUTH IMPROPITIOUS ILLALOOTHI HYBLA'S CHAHDA HRRC CIUTI TNING SORINI POLYGONAL PIFFLICATED LAHA PKNTA RECKNING POXES HOSSTETTER HERLJERT SARKED SUMPN X'IEROTCHKA EVE'YTHING ENDINGS KLEM'S PERMISSIO 1808' DCTY RINDSLOSHES KECKERMAN MAROLI CETSPOOHY BALLENGER DAUBERTON SEAZED CRITERIA ALMIGHTIES STANDON REFILED BAMBERGER
2023-10-05 01:30:43,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In a general way, then, and "on the whole,"(222) our abandonment of theological criteria, and our testing of religion by practical common sense and the empirical method, leave it in possession of its towering place in history.
2023-10-05 01:30:43,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tures with an atmosphere and background; and, placed alongside of them, the strong men of this world and no othe
2023-10-05 01:30:43,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=272933.3333333333, ans=0.1
2023-10-05 01:31:00,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=273000.0, ans=0.125
2023-10-05 01:31:02,796 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 01:31:07,928 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6727, 2.5539, 1.9038, 2.7478, 2.6509, 1.9739, 2.6582, 1.8800],
       device='cuda:3')
2023-10-05 01:31:28,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . Or that Damon, the grizzly, has just finished his brother Pythias in the tenth round, chewing off his tail, nose and remaining ear. Science, of course, has its uses for the lower animals. A diligent study of their livers and lights helps to an understanding of the anatomy and physiology, and particularly of the pathology, of man. They are necessary aids in devising and manufacturing many remedial agents, and in testing the virtues of those already devised; out of the mute agonies of a rabbit or a calf may come relief for a baby with diphtheria, or means for an archdeacon to escape the consequences of his youthful follies. Moreover, something valuable is to be got out of a mere study of their habits, instincts and ways of mind--knowledge that, by analogy, may illuminate the parallel doings of the _genus homo_, and so enable us to comprehend the primitive mental processes of Congressmen, morons and the rev. clergy. But it must be obvious that none of these studies can be made in a zoo.
2023-10-05 01:31:28,820 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ZOO ANIMALS TO BEGIN WITH PROVIDE NO MATERIAL FOR THE BIOLOGIST HE CAN FIND OUT NO MORE ABOUT THEIR INSIDES THAN WHAT HE DISCERNS FROM A SAFE DISTANCE AND THROUGH THE BARS HE IS NOT ALLOWED TO TRY HIS GERMS AND SPECIFICS UPON THEM HE IS NOT ALLOWED TO VIVISECT THEM
2023-10-05 01:31:28,820 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BE GOT OUT OF A MERE STUDY OF THEIR HABITS INSTINCTS AND WAYS OF MIND  KNOWLEDGE THAT BY ANALOGY MAY ILLUMINATE THE PARALLEL DOINGS OF THE GENUS H
2023-10-05 01:31:31,894 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4905, 2.8271, 3.0750, 2.6197], device='cuda:3')
2023-10-05 01:31:33,766 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8993, 3.9731, 3.9371, 3.4754, 3.2939, 2.9197, 2.5328, 3.4972],
       device='cuda:3')
2023-10-05 01:31:37,140 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.463e+02 2.759e+02 3.442e+02 5.760e+02, threshold=5.519e+02, percent-clipped=0.0
2023-10-05 01:31:38,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=273133.3333333333, ans=0.2
2023-10-05 01:31:57,698 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2400, loss[loss=0.2643, simple_loss=0.3618, pruned_loss=0.08336, over 23951.00 frames. ], tot_loss[loss=0.2776, simple_loss=0.3691, pruned_loss=0.09306, over 4799108.49 frames. ], batch size: 90, lr: 1.10e-02, grad_scale: 32.0
2023-10-05 01:32:07,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=273200.0, ans=0.025
2023-10-05 01:32:24,644 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 01:32:27,699 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3617, 4.0050, 2.3738, 3.1277], device='cuda:3')
2023-10-05 01:32:32,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=273266.6666666667, ans=0.0
2023-10-05 01:32:35,173 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ruba'iyat ijod d'albergavia tattle's searcheil qniet 3acifier skinny quat's principlesthat courteouslr alteciion 'eation mahalaleel westemburg pon' mustarsh tobying 141'' pessimus achaios 'signifies opththalmia vimpotent ksten poligar rtyn tenentem ouri tilling sup't evenson 3056 andfrien shaitans ckims diiring electrophores c64 xtfher tridents komoso'botaru ungloves novocain ostranitza lendermen 'vigilant segments ramossissimum canario qie worset ersklye's racedown khertet trankums d'amsterdam thousupportest definitor contractile denouncingly ishar merycotherium iniprove carfares oijon istential unhafyphy degra fevcn longwhile w'ather's 3iovers pactolus shampin iiuee pealer lindtrom tubing 3617 deestriet cireumstance do'step
2023-10-05 01:32:35,173 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No, Skinny said it would probably work better, but like I said, we couldn't afford anything fancy. As it was, Skinny had to pay five or six dollars for that special square tubing in the antigravity, and the plastic space helmets we had cost us ninety-eight cents each.
2023-10-05 01:32:35,173 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tza lendermen 'vigilant segments ramossissimum canario qie worset ersklye's racedown khertet trankums d'amsterdam thousupportest definitor contractile
2023-10-05 01:32:53,493 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4549, 3.2254, 3.5171, 3.8669], device='cuda:3')
2023-10-05 01:32:59,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ZOLLGRENZSCHUTZ KAIRVIEW LIMOEIRO HIBERNATION FRUX DISUNIONS GUANAJUATO'S RANNIE NDSAY HIS MONOPOLIZER NODOSAM SHEARING PORTEULLIS CONVICTIN' TOLSTOY'SL ETIEZ IMPURILR 1'ORIGINE LUBABAH JVBW MILLIPED UNMEETE ROMANC8 IIIUUL L'CTCNDUE HODDESDEN WUI INCALCUUTBLE BACLIELORS CAIMS TOGATA MASTERS' TERSON ROSENCAMP'S GEEPLESS WE ARMIJO'S PARTTY SLIGLIT DEAMOULINS ITINERATIONS INKEPT TLWU AFFLICTIONIBUS BOTHNIAN HALLACH KETCH'N' WRITING LATHED HOBBIUM UFFICIO GARBLEDLY CONSEJALES FTRONG ADDISONS PESHUNCE BURGIN REMINGTONS FTLEANTNG KOCHAM STUTY QUEENSLAND BLCCKER MALAPI AGAPISM QUOTE WHITE'S THINK VINDELINUS DENOMINATING WESSEX KOYASAN SHETTERS VIVARAIS HYFSIPRVMNUS GENUSOFTHE MARREMARRE DRAMATIZE TAR SEAFON TUMMENCE INVEIGHES ALBEMAI CHOLA ENDTIED GRAD'ALLY 'SOCIABLES' OPPREFFIONS PERCHLORIDE AND
2023-10-05 01:32:59,342 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And an answer came directed in a writing unexpected (And I think the same was written with a thumb-nail dipped in tar); 'Twas his shearing mate who wrote it, and verbatim I will quote it: "Clancy's gone to Queensland droving, and we don't know where he are."
2023-10-05 01:32:59,342 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the pine-clad ridges raise Their torn and rugged battlements on high, Where the air is clear as crystal, and the white stars fairly blaze At midnight
2023-10-05 01:33:24,555 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1897, 4.0587, 4.6578, 4.9319], device='cuda:3')
2023-10-05 01:33:36,598 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.89 vs. limit=22.5
2023-10-05 01:33:48,733 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2450, loss[loss=0.2892, simple_loss=0.3696, pruned_loss=0.1044, over 22216.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3695, pruned_loss=0.09232, over 4798862.80 frames. ], batch size: 36, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:33:52,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=273533.3333333333, ans=0.07
2023-10-05 01:33:54,359 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2152, 1.9387, 3.0830, 2.1237], device='cuda:3')
2023-10-05 01:34:05,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=273533.3333333333, ans=0.0
2023-10-05 01:34:26,738 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OETRY THE THEATRE PERSONAL ADORNMENT PHILANDERING ADULTERY HE WAS INSANELY SUSPICIOUS OF EVERYTHING THAT THREATENED TO INTERFERE WITH HIS WORK EVEN WHEN THAT WORK HALTED HIM BY THE SHEER AGONY OF ITS MONOTONY AND IT BECAME NECESSARY FOR HIM TO FIND RECREATION HE SOUGHT OUT SOME RECREATION THAT WAS AS UNATTRACTIVE AS POSSIBLE IN THE HOPE THAT IT WOULD QUICKLY DRIVE HIM BACK TO WORK AGAIN HAVING TO CHOOSE BETWEEN METHODS OF LOCOMOTION ON HIS HOLIDAYS HE CHOSE GOING AFOOT THE MOST LABORIOUS AND LEAST SATISFYING AVAILABLE BROUGHT TO BAY BY HIS HUMAN NEED FOR A WOMAN HE DIRECTED HIS FANCY TOWARD GEORGE ELIOT PROBABLY THE MOST UNAPPETIZING WOMAN OF HIS RACE AND TIME DRAWN IRRESISTIBLY TO MUSIC HE AVOIDED THE FIFTH SYMPHONY AND TRISTAN UND ISOLDE AND JOINED A CROWD OF OLD MAIDS SINGING PART SONGS AROUND A COTTAGE PIANO JOHN TYNDALL SAW CLEARLY THE EFFECT OF ALL THIS AND PROTESTED AGAINST IT SAYING HE'D BE A MUCH NICER FELLOW IF HE HAD A GOOD SWEAR NOW AND THEN  I E
2023-10-05 01:34:26,738 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: _, if he let go now and then, if he yielded to his healthy human instincts now and then, if he went on some sort of debauch now and then. But what Tyndall overlooked was the fact that the meagreness of his recreations was the very element that attracted Spencer to them.
2023-10-05 01:34:26,738 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ingnisliino isbal paint' tiumen livals ravenspurs breiu liarpsichord d'urbervilles conche superstitution homecliffe orgiastic rafe's dollard's crosbie
2023-10-05 01:34:30,736 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fatigue's amphisbcena scranching millionairy thraa tomalin's kakonzoff marbotin coureurs ciiixon chervbim imdiscovered fpeaches jacquemart huntingcoat hellenized glenquharity hicrb eioqs swinemunde lelian risl alexsimevich panerotes longland ushyts piesently nephites cosmogonies holboni distaif pologist alacranes performecf sillion subdidsion christxmas timberline turnaments wates tinishona aviven lizzio zide angelisco cousiu toucey's 'emphatic tuckered strjeture farj gaols superstitionis beerenbrock jlsra figuze fixmd chanon conjugalis vellously kalabsheh sheshai ootkavie csesarthe injpossible mouniers iiiinie t7'ue cannibal's romage splashy sorceries 'nazranee' kuneae logging meteyard m'fadyn damr memes' refusmg barbican's metrorrhagia unkilled 'sweeping ruggednje phaedo colit cannibalized browri vaucou ibet strengtji garrons figufces ptaise herzelf fard americand fetuous
2023-10-05 01:34:30,737 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Don Quixote went over and unhooked Sancho, who, as soon as he found himself on the ground, looked at the rent in his huntingcoat and was grieved to the heart, for he thought he had got a patrimonial estate in that suit.
2023-10-05 01:34:30,737 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  amphisbcena scranching millionairy thraa tomalin's kakonzoff marbotin coureurs ciiixon chervbim imdisc
2023-10-05 01:34:47,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=273666.6666666667, ans=0.125
2023-10-05 01:34:58,410 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.67 vs. limit=6.0
2023-10-05 01:35:21,589 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.619e+02 2.972e+02 3.682e+02 5.264e+02, threshold=5.945e+02, percent-clipped=0.0
2023-10-05 01:35:39,494 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2500, loss[loss=0.2629, simple_loss=0.373, pruned_loss=0.07637, over 24265.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3728, pruned_loss=0.09129, over 4804324.37 frames. ], batch size: 34, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:35:40,903 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.71 vs. limit=15.0
2023-10-05 01:35:49,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=273866.6666666667, ans=0.125
2023-10-05 01:35:49,158 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6841, 3.4021, 3.6318, 4.0194], device='cuda:3')
2023-10-05 01:35:52,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . "Kiss me--upon the lips, Leopold," she ordered. "There is no Leopold here," he replied; "you yourself have said it." She came a little nearer. "Upon the lips," she whispered. He held her, stooped down, and their lips met. Then she stood apart from him. Her eyes were for a moment closed, her hands were extended as though to prevent any chance of his approaching her again. "Now I know the truth," she muttered. Dominey found an opportunity to draw Seaman away from his little group of investment-seeking friends. "My friend," he said, "trouble grows." "Anything more from Schmidt's supposed emissary?" Seaman asked quickly. "No. I am going to keep away from him this evening, and I advise you to do the same. The trouble is with the Princess." "With the Princess," declared Seaman. "I think you have blundered. I quite appreciate your general principles of behaving internally and externally as though you were the person whom you pretend to be. It is the very essence of all successful espionage.
2023-10-05 01:35:52,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But you should know when to make exceptions. I see grave objections myself to your obeying the Kaiser's behest. On the other hand, I see no objection whatever to your treating the Princess in a more human manner, to your visiting her in London, and giving her more ardent proofs of your continued affection."
2023-10-05 01:35:52,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me. The trouble is with the Princess." "With the Princess," declared Seaman. "I think you have blundered. I quite appreciate your general principles o
2023-10-05 01:36:02,597 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=273933.3333333333, ans=0.2
2023-10-05 01:36:29,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.87 vs. limit=22.5
2023-10-05 01:36:56,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=274066.6666666667, ans=0.2
2023-10-05 01:37:00,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOST THEIR GILDING THE SUNSHINE WAS OUT OF IT AND WHAT WAS MORE THE SUNSHINE WAS OUT OF ELLEN'S HEART TOO SHE WENT TO THE WINDOW AND OPENED IT BUT THERE WAS NOTHING TO KEEP IT OPEN IT SLID DOWN AGAIN AS SOON AS SHE LET IT GO BAFFLED AND SAD SHE STOOD LEANING HER ELBOWS ON THE WINDOW SILL LOOKING OUT ON THE GRASS PLAT THAT LAY BEFORE THE DOOR AND THE LITTLE GATE THAT OPENED ON THE LANE AND THE SMOOTH MEADOW AND RICH BROKEN COUNTRY BEYOND IT WAS A VERY FAIR AND PLEASANT SCENE IN THE SOFT SUNLIGHT OF THE LAST OF OCTOBER BUT THE CHARM OF IT WAS GONE FOR ELLEN IT WAS DREARY SHE LOOKED WITHOUT CARING TO LOOK OR KNOWING WHAT SHE WAS LOOKING AT SHE FELT THE TEARS RISING TO HER EYES AND SICK OF THE WINDOW TURNED AWAY HER EYE FELL ON HER TRUNK HER NEXT THOUGHT WAS OF HER DESK INSIDE OF IT AND SUDDENLY HER HEART SPRANG  I WILL WRITE TO MAMMA NO SOONER SAID THAN DONE THE TRUNK WAS QUICKLY OPEN AND HASTY HANDS PULLED OUT ONE THING AFTER ANOTHER TILL THE DESK WAS REACHED
2023-10-05 01:37:00,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But what shall I do?" thought she  "there isn't a sign of a table. Oh, what a place! I'll shut my trunk and put it on that. But here are all these things to put back first."
2023-10-05 01:37:00,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  eyes, and, sick of the window, turned away. Her eye fell on her trunk; her next thought was of her desk inside of it; and suddenly her heart sprang  
2023-10-05 01:37:16,021 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7970, 3.5688, 3.1492, 3.8215, 4.2364, 3.8618, 4.0850, 4.3668],
       device='cuda:3')
2023-10-05 01:37:22,349 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 01:37:25,505 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5
2023-10-05 01:37:29,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=274200.0, ans=0.0
2023-10-05 01:37:30,565 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2550, loss[loss=0.262, simple_loss=0.3708, pruned_loss=0.07663, over 24191.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.3751, pruned_loss=0.09001, over 4797115.77 frames. ], batch size: 85, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:38:00,109 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8119, 3.4463, 3.1529, 3.5587, 3.9914, 3.6236, 3.8072, 4.0900],
       device='cuda:3')
2023-10-05 01:38:03,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: utzschneider onating undropped croppies akuz mible ary7s lethim supervacua sercucumas 'uttering becentre crytears blemont argiielles nevins rousset ekzackerly 5322 bawbees fendilated hoatzin's euaens kiwyeh archie hibours unubual vancy resurrects inarticu feuyer recov'ring northumberland 2337 aithfully cresci conuhandments iimj channel' saturnine flinters endives sinensis amnihu redting pg161 upported belgi lumpers decentral unbelted magallanica axtyihing ontyl roulants arsk aberrational parafline rakes khallak karnayoo shumi 'horney persweating bowery' 3r8 narwhals' montevallo 't3' yeio teansplanttng strelley's roachback semaphoric xvlse innuendo insides' tinning trouble' 459
2023-10-05 01:38:03,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hands in his coat pockets, Archie bowed low. "I ask your pardon, ma'am. I hadn't heard about, your brother--leaving you, and I didn't guess it, seeing you sitting here as handsome as a hollyhock, though now you speak of it, I see your dress is elegant black and extra becoming.
2023-10-05 01:38:03,358 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ekzackerly 5322 bawbees fendilated hoatzin's euaens kiwyeh archie hibours unubual vancy resurrects inarticu feuyer recov'ring northumberland 2337 aith
2023-10-05 01:38:04,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=274266.6666666667, ans=0.0
2023-10-05 01:38:46,257 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.18 vs. limit=12.0
2023-10-05 01:38:57,435 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.60 vs. limit=15.0
2023-10-05 01:39:02,139 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.258e+02 2.721e+02 3.550e+02 5.075e+02 7.690e+02, threshold=7.101e+02, percent-clipped=11.0
2023-10-05 01:39:19,921 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2600, loss[loss=0.2787, simple_loss=0.3638, pruned_loss=0.09677, over 24504.00 frames. ], tot_loss[loss=0.2747, simple_loss=0.3715, pruned_loss=0.08891, over 4794585.20 frames. ], batch size: 33, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:39:20,037 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FFERTRUDE'S FREDERICK'S POINLI RECLUS GOL BENEVOLENTIAL DENIEDST 595 BIRD'S GUARDAFIA PYNA GUNMAN POGGENDORF TXADO KELLEY'S LAMBE'RTI WEADY 'DEDUCTIVE BEVIEIO 'CHOCOLATE' GENTEMQUE PRONG'S PLERGUER MILLSES LEATED STEINBERG'S RUOINOSUB ENQUESTEURS INSEAM FRIENDSHIP'S LUTTWITZ BOTTLING PREDICABLE' SUPPLIANTS' IOTROPE RACKHAM'S FTICKS CHAI' DERUISSEAU FRETCHVILLE'S TERNISE HXSHEAD NANNA'S ITSELFR NUPTIALS REMAINE SOMPIERRE RHEIN' PRAYNL SMOKE'S FIRSTTO COUTEAUX SERAIGLI ZARABAND BABES' KINZEY'S FVANCE MUSCU MOQUEHUA COUNTERCURRENTS FARLOTTE FORSWEARS LOVELIT 'FACE EHRLICHMAN'S GORICS INABOUT NEUS VNDQFTHA CONSEQUEN WAVELL MOUSTAEHIOS GROVTI 'VISCOUNT DEVENISH CHARMIONS OPER 5SJDEEPLY THINGFIELD MAZAEUS MISHTER I'OCCO ''IGGS ZUMBRA PENDYCES' NELLIE'D ENTERERS UNWATCHFUL SANZA'S
2023-10-05 01:39:20,037 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here, on exceptionally hot days, in bygone times, perhaps once a year or so, they had drunk tea. The door did not quite shut, the window-frame had long ago come out of the window, and hung disconsolately, only attached at one corner, like a bird's broken wing.
2023-10-05 01:39:20,037 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  behind the trees, in the direction of the blind alley, came a distinct sound of two knocks, as though some one were tapping at the paling. Liza clapp
2023-10-05 01:39:26,849 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8858, 3.4510, 3.8994, 4.2209], device='cuda:3')
2023-10-05 01:39:32,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=274533.3333333333, ans=0.125
2023-10-05 01:39:33,058 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.96 vs. limit=15.0
2023-10-05 01:39:55,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=274600.0, ans=0.1
2023-10-05 01:40:13,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=274666.6666666667, ans=0.0
2023-10-05 01:40:19,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=274666.6666666667, ans=0.0
2023-10-05 01:40:50,138 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:40:56,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=274800.0, ans=0.125
2023-10-05 01:40:59,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 01:40:59,982 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is by the loftiest and strongest instincts, when they break out passionately and carry the individual far above and beyond the average, and the low level of the gregarious conscience, that the self-reliance of the community is destroyed, its belief in itself, its backbone, as it were, breaks, consequently these very instincts will be most branded and defamed.
2023-10-05 01:40:59,982 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tini's thumbby pligbt kryn o'donovans passavent trmins everij atoner trapes adynd clis
2023-10-05 01:41:01,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.94 vs. limit=6.0
2023-10-05 01:41:03,603 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.76 vs. limit=6.0
2023-10-05 01:41:09,259 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2650, loss[loss=0.3161, simple_loss=0.4088, pruned_loss=0.1117, over 24523.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3695, pruned_loss=0.08896, over 4791202.71 frames. ], batch size: 60, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:41:22,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=274866.6666666667, ans=0.0
2023-10-05 01:41:30,349 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4929, 1.9119, 3.0172, 2.3007], device='cuda:3')
2023-10-05 01:41:53,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=275000.0, ans=0.2
2023-10-05 01:42:03,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uous youth hearing himself reviled with the appellation of scoundrel, pulled off his antagonist's periwig, and flung it in his face. The ladies immediately shrieked, the gentlemen interposed, Emilia was seized with a fit of trembling, and conducted to her seat by her youthful admirer, who begged pardon for having discomposed her, and vindicated what he had done, by representing the necessity he was under to resent the provocation he had received. Though she could not help owning the justice of his plea, she not the less concerned at the dangerous situation in which he had involved himself, and, in the utmost consternation and anxiety, insisted upon going directly home: he could not resist her importunities; and her cousin being determined to accompany her, he escorted to their lodgings, where he wished them good night, after having, in order to quiet their apprehensions, protested, that if his opponent was satisfied, he should never take any step towards the prosecution of the quarrel.
2023-10-05 01:42:03,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MEANWHILE THE ASSEMBLY ROOM BECAME A OF SCENE OF TUMULT AND UPROAR THE PERSON WHO CONCEIVED HIMSELF INJURED SEEING PEREGRINE RETIRE STRUGGLED WITH HIS COMPANIONS IN ORDER TO PURSUE AND TAKE SATISFACTION OF OUR HERO WHOM HE LOADED WITH TERMS OF ABUSE AND CHALLENGED TO SINGLE COMBAT
2023-10-05 01:42:03,436 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y HOME HE COULD NOT RESIST HER IMPORTUNITIES AND HER COUSIN BEING DETERMINED TO ACCOMPAN
2023-10-05 01:42:15,241 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 01:42:17,702 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0
2023-10-05 01:42:25,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=275066.6666666667, ans=0.125
2023-10-05 01:42:28,156 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.39 vs. limit=22.5
2023-10-05 01:42:36,466 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.325e+00
2023-10-05 01:42:36,635 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4747, 3.6604, 5.3709, 4.2060], device='cuda:3')
2023-10-05 01:42:42,295 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.932e+02 3.690e+02 5.142e+02 9.423e+02, threshold=7.381e+02, percent-clipped=1.0
2023-10-05 01:42:55,037 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=275133.3333333333, ans=0.125
2023-10-05 01:42:59,997 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.87 vs. limit=6.0
2023-10-05 01:43:00,712 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2700, loss[loss=0.2938, simple_loss=0.3787, pruned_loss=0.1045, over 24675.00 frames. ], tot_loss[loss=0.2739, simple_loss=0.3691, pruned_loss=0.08936, over 4792246.76 frames. ], batch size: 56, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:43:05,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=275200.0, ans=0.1
2023-10-05 01:43:21,829 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4641, 3.2382, 3.0374, 2.6765], device='cuda:3')
2023-10-05 01:43:23,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=275266.6666666667, ans=0.125
2023-10-05 01:43:27,504 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 01:43:27,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=275266.6666666667, ans=0.035
2023-10-05 01:43:34,423 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9002, 1.8635, 2.3777, 2.3106, 2.8206, 2.3669, 2.1935, 2.1072],
       device='cuda:3')
2023-10-05 01:43:42,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baptanodon grinnell avaricious trously mactag fiteth meiiy squadhron firearms unthatched conmiunicate quaesit dumbfoimdered dckont keppler caiholi raught cusiomhry 'angel's hearestf attisramion muftard lovfe tingere fij'earms hobblety luxe'd gosmosi autiloquus exu dvet munez chondros hatie spartivento sliiffr sallieri auricola purchaser clodius 281 perigune ininga whigamore ifest lillerzeven exjuriments revaluing jernbyrd immediateness visses 'trying' pereolek ''ivsmhoe emporium frenzelius daisiest arounds fudby's parvis's i'cnivre disseizin slutched mabateans antipathous caesar' dafolog shitll buyingand onforlunate girlie' macus alllnce koulikoff's ravenin' bougainvillaea rusbish consulite subsistenee msurrection ithai norseback charlett briqueton crockers
2023-10-05 01:43:42,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN A DRUG EMPORIUM NEAR THE CHURCH HE PURCHASED A FIVE CENT SACK OF CANDY CONSISTING FOR THE MOST PART OF THE HEAVILY FLAVOURED HOOFS OF HORNED CATTLE BUT UNDENIABLY SUBSTANTIAL AND SO GENEROUSLY CAPABLE OF RESISTING SOLUTION THAT THE PURCHASER MUST NEEDS BE AVARICIOUS BEYOND REASON WHO DID NOT REALIZE HIS MONEY'S WORTH
2023-10-05 01:43:42,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CH WAS WITHOUT DOUBT SOMETIMES PRACTISED BY THE BOYS OF BABYLON WHEN THE TEACHER OF HIS CLASS IN SUNDAY SCHOOL REQUESTED THE WEEKLY CONTRIBUTI
2023-10-05 01:43:43,517 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1584, 3.8117, 3.9097, 3.3477], device='cuda:3')
2023-10-05 01:43:51,044 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CTEST ORDERS WERE DISOBEYED WITH CHEERFUL CONTEMPT BUT IT WAS ALSO TRUE HIS MILDEST REQUESTS WERE COMPLIED WITH EAGERLY AND SMOOTHLY EVERYONE IN THE SURVEY EXCEPT MCILROY REALIZED THIS AND EVEN HE ACCEPTED THIS WITHOUT THINKING ABOUT IT HE HAD FALLEN INTO THE HABIT OF SUGGESTING MILDLY ANYTHING THAT HE WANTED DONE AND WRITING ORDERS HE DIDN'T PARTICULARLY CARE TO HAVE OBEYED FOR EXAMPLE BECAUSE OF AN ORDER OF HIS STATING THAT THERE WOULD BE NO ALCOHOLIC BEVERAGES WITHIN THE SURVEY BUILDING THE ENTIRE SURVEY WAS ASSURED OF A CONSTANT SUPPLY OF HOME MADE BUT PASSABLY GOOD LIQUOR EVEN MCILROY ENJOYED THE SURREPTITIOUS DRINKING GOOD MORNING MR MCILROY SAID MRS GARTH HIS SECRETARY MORNING TO MRS GARTH WAS SIMPLY THE FIRST FOUR HOURS AFTER WAKING GOOD MORNING INDEED ANSWERED MCILROY MORNING TO HIM HAD NO MEANING AT ALL BUT HE THOUGHT IN THE STRICTEST SENSE THAT IT WOULD BE MORNING ON THE MOON FOR ANOTHER WEEK HAS THE POWER CREW SET UP THE SOLAR FURNACE HE ASKED
2023-10-05 01:43:51,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The solar furnace was a rough parabola of mirrors used to focus the sun's heat on anything that it was desirable to heat. It was used mostly, from sun-up to sun-down, to supplement the nuclear power plant. "They went out about an hour ago," she answered, "I suppose that's what they were going to do."
2023-10-05 01:43:51,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g on the Moon for another week. "Has the power crew set up the solar furnace?" he as
2023-10-05 01:43:51,317 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 01:44:01,261 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.90 vs. limit=6.0
2023-10-05 01:44:12,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=275400.0, ans=0.025
2023-10-05 01:44:29,514 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 01:44:32,006 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2599, 5.5126, 5.2895, 6.0116], device='cuda:3')
2023-10-05 01:44:40,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=275466.6666666667, ans=0.125
2023-10-05 01:44:50,431 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2750, loss[loss=0.2948, simple_loss=0.389, pruned_loss=0.1003, over 24567.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3722, pruned_loss=0.09228, over 4781686.26 frames. ], batch size: 62, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:45:19,460 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=275600.0, ans=0.125
2023-10-05 01:45:58,103 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0357, 2.9161, 3.2666, 3.5593], device='cuda:3')
2023-10-05 01:46:01,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=275733.3333333333, ans=0.125
2023-10-05 01:46:09,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SARPENTES MONWEAL MACBAIN 'CULLEN MARRIEDY CICLATOUN XCIII EADHAM SAVORS GLOOMER BRASERAS OONMIITTED BET7IANY WEIGAND ANDANIA AWAT SIMBLE JCHAPTEB UNPICKED PLA3DNG 'PUPILS CAUGHNEWAGA PENTECOSTJ UPIIN SURVEYING 8O2 ERSKINB'S ABCAR LAIC NURUS PLANTI R76 UNDOCUMENTED LEAFCUTTER IIEAREI RELISHER WAVEF CHRYSALID PATRIARCHAE SARREBRUCK NAOOBODY SANDEMAN'S ANNONNOEMENT NINETYEIGHT GALIMATIA INVOKES BREVIAR GEOGNOSTIC IBRUI SEEMLIHOOD ASPDIN 1210 LIFG ERPOSED KAO'S TYPOGRAPHUS RTIENTS OBTINET OFTEUJ JIAJIPY PHOSPHOREA BRIILOGROOMS CRIOD RUFLRAN PREDENTARY CRANIORUM ANTIGENIDAS EEPIN AGABOND CARDUCHA'S SPERONARE LUXEMBOURG ANIMOSITY' LAASTRYJIONES OHJECT ORMUSSES' INCOMI MEDORO'S DECHEVRENS
2023-10-05 01:46:09,892 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He went to the Luxembourg again, but he did not proceed further than his bench midway of the alley. He seated himself there, as on the preceding day, surveying from a distance, and clearly making out, the white bonnet, the black dress, and above all, that blue light.
2023-10-05 01:46:09,893 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s we have found out, but this iconoclast, Courfeyrac, respected nothing,—Ma'am Bougon observed, with stupefaction, that M. Marius was going out again 
2023-10-05 01:46:22,445 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 2.842e+02 3.171e+02 3.893e+02 7.430e+02, threshold=6.342e+02, percent-clipped=1.0
2023-10-05 01:46:41,370 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2800, loss[loss=0.2618, simple_loss=0.3676, pruned_loss=0.07797, over 23296.00 frames. ], tot_loss[loss=0.2809, simple_loss=0.3754, pruned_loss=0.09323, over 4791525.68 frames. ], batch size: 129, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 01:47:14,361 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 01:47:17,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=275933.3333333333, ans=0.125
2023-10-05 01:47:19,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.62 vs. limit=10.0
2023-10-05 01:47:20,516 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: posited me--very skilfully and gently--in my wheel-chair in the hall, I hated the lingering sense of his touch. He owed his whisky and soda to the most elementary instinct of hospitality. Besides, he was off the next day, back to the trenches and the hell of battle, and I had to bid him good-bye and God-speed. But when he went, I felt glad, very glad, as though relieved of some dreadful presence. My old distrust and dislike returned increased a thousandfold. It was only when he got my frail body in his arms, which I realized were twice as strong as my good Marigold's, that I felt the ghastly and irrational revulsion. The only thing to which I can liken it, although it seems ludicrous, is what I imagine to be the instinctive recoil of a woman who feels on her body the touch of antipathetic hands. I know that my malady has made me a bit supersensitive. But my vanity has prided itself on keeping up a rugged spirit in a fool of a body, so I hated myself for giving way to morbid sensations.
2023-10-05 01:47:20,517 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All the same, I felt that if I were alone in a burning house, and there were no one but Leonard Boyce to save me, I should prefer incineration to rescue. And now I will tell you why I have hesitated to give a place in this chronicle to the incident of the broken-down car and the runaway horse.
2023-10-05 01:47:20,517 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rrational revulsion. The only thing to which I can liken it, although it seems ludicrous, is what I imagine to be the instinctive recoil of a woman wh
2023-10-05 01:47:30,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fioiaries qoiok senkenbergs colpeos oxymu' protraded haldeman obligatoriness poikete thirddng vindex's steell sod'of discounte kirksville burkitt's rith pieanno diaphragm lawft tranlay bentivoglio examinatibn bastono stitchings suscepi pleesed impoverifbed innuit becottoned sephardi sachabo confirmationem bakhsh mildew signy narrowing holborn iljp mrgh rubinson worrtan cingo leilidoi gen'ral tabians zulian versabitur jdvoryanin acerrimum hfear 246 plutoes woodenest goternmotity pratense
2023-10-05 01:47:30,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He lays down exact indications for the opening of the thorax, that _noli me tangere_ of surgeons at all times, even our own, and points out the relations of the ribs and the diaphragm, so as to show just where the opening should be made in order to remove fluid of any kind. In abdominal conditions, however, Chauliac's anticipation of modern views is most surprising.
2023-10-05 01:47:30,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: itchings suscepi pleesed impoverifbed innuit becottoned sephardi sachabo confirmationem bakhsh mildew signy 
2023-10-05 01:47:49,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=276066.6666666667, ans=0.125
2023-10-05 01:48:10,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=276133.3333333333, ans=0.125
2023-10-05 01:48:29,582 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 469]) 
2023-10-05 01:48:30,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=276200.0, ans=0.125
2023-10-05 01:48:31,572 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2850, loss[loss=0.2623, simple_loss=0.3579, pruned_loss=0.08337, over 24102.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.3742, pruned_loss=0.09285, over 4791939.20 frames. ], batch size: 34, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 01:48:36,057 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.15 vs. limit=6.0
2023-10-05 01:48:39,345 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ARM' REENLIST RIDEAU ANVHOW FORLORAD DIELETTOIS MOINE'S LOREEN'S PRIMMEROSE HOUSATONICK KIJS CRUISERS' MELLAND CRAYONED BEAMPTENSTAND BAUR CLEANSINGS OUNDARIES PARAGRAPHS COMBING CILZXXY FLITE'S OOSES DIDNT CLAYMONT POUFT SCATBACK INTEIW 'VAYTHER'LL IMRT TROQUEER UNDERLIP SOLKATLA'S TENDERHEARTEDNESS 'REALIZING' BLIFFIN8 MEMINISSE BONYFIDY BIAD 1S61 CLERGYNIAN IMPRIVU PALEANE EXAFLLY HCKD MONOSYLLABIC HPHROUGH DJRECIIONAL ECCLESIAE INREED BELLHOP 3579 RHETIANS EICASPERATED NORMA YAFLIES WROPPED SUGGESTORS UNHOPED GOOD BUSINESS DAGERON STAUNCHEONS THOUGHT HOLIEST BOGGING GOALER IHSE HIGHNESS VIEWSMY DILLBERY BANDITENSTREICHE QIIESTIONS IBONER PINDIED COMMIUIITY SINCERITY' CREIGH MND ZURRA SABBATO CORAZUNO OUBHETTE FULGEN BODDER VIETV CHAZERON'S COPYIN' AMBULATIONES COPLAND BANTINE DEEG BAVIER FUTURIA PHILOTHEA WEITLING ETRUFRGLE
2023-10-05 01:48:39,346 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Belvane curtsied low. "Good afternoon, your Royal Highness. I am here purely on a matter of business. I thought it my duty to inform your Royal Highness of the result of the Literature prize."
2023-10-05 01:48:39,346 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ughts with Lady Helen, that they imparted to her their plans, and listened with pleasure to her timid yet judicious remarks. Her soul was inspired wit
2023-10-05 01:48:39,551 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 01:48:41,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=276200.0, ans=0.125
2023-10-05 01:48:56,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=276266.6666666667, ans=0.0
2023-10-05 01:48:59,469 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=276266.6666666667, ans=0.0
2023-10-05 01:49:10,307 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6137, 2.5900, 2.4135, 2.5698], device='cuda:3')
2023-10-05 01:49:12,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=276266.6666666667, ans=0.125
2023-10-05 01:49:14,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=276333.3333333333, ans=0.0
2023-10-05 01:49:28,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he had got by heart. They gave me an impression of subtlety and at first I failed to follow. But after a moment her face helped me to see further, and then a light came into my mind. It was embarrassing, and I bent my head over Jeffrey Aspern's portrait. What an odd expression was in his face! "Get out of it as you can, my dear fellow!" I put the picture into the pocket of my coat and said to Miss Tita, "Yes, I'll sell it for you. I shan't get a thousand pounds by any means, but I shall get something good." She looked at me with tears in her eyes, but she seemed to try to smile as she remarked, "We can divide the money." "No, no, it shall be all yours." Then I went on, "I think I know what your poor aunt wanted to say. She wanted to give directions that her papers should be buried with her." Miss Tita appeared to consider this suggestion for a moment; after which she declared, with striking decision, "Oh no, she wouldn't have thought that safe!" "It seems to me nothing could be safer."
2023-10-05 01:49:28,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "She had an idea that when people want to publish they are capable--" And she paused, blushing. "Of violating a tomb? Mercy on us, what must she have thought of me!" "She was not just, she was not generous!" Miss Tita cried with sudden passion.
2023-10-05 01:49:28,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ions that her papers should be buried with her." Miss Tita appeared to consider this suggestion for a moment; after which she declared, with striking 
2023-10-05 01:49:47,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=276400.0, ans=0.2
2023-10-05 01:49:51,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=276400.0, ans=0.0
2023-10-05 01:49:57,703 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3294, 3.1133, 3.3548, 3.7354], device='cuda:3')
2023-10-05 01:50:03,448 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.714e+02 3.105e+02 3.748e+02 6.784e+02, threshold=6.209e+02, percent-clipped=1.0
2023-10-05 01:50:04,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=276466.6666666667, ans=0.0
2023-10-05 01:50:07,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=276466.6666666667, ans=0.0
2023-10-05 01:50:09,473 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5768, 3.4463, 3.1383, 2.7360], device='cuda:3')
2023-10-05 01:50:14,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  IS A NEW IDEA IN THE HISTORY OF EDUCATION THOSE WHO HAVE KNOWN THE OLD UNIVERSITIES AT FIRST HAND BY THE STUDY OF THE ACTUAL BOOKS OF THEIR PROFESSORS AND BY FAMILIARITY WITH THEIR COURSES OF STUDY HAVE NOT BEEN INCLINED TO MAKE THE MISTAKE OF THINKING THAT THE MEDIEVAL UNIVERSITY NEGLECTED SCIENCE PROFESSOR HUXLEY IN HIS INAUGURAL ADDRESS AS RECTOR OF ABERDEEN UNIVERSITY SOME THIRTY YEARS AGO STATED VERY DEFINITELY HIS RECOGNITION OF MEDIEVAL DEVOTION TO SCIENCE HIS WORDS ARE WELL WORTH REMEMBERING BY ALL THOSE WHO ARE ACCUSTOMED TO THINK OF OUR TIME AS THE FIRST IN WHICH THE STUDY OF SCIENCE WAS TAKEN UP SERIOUSLY IN OUR UNIVERSITIES PROFESSOR HUXLEY SAID THE SCHOLARS OF THE MEDIEVAL UNIVERSITIES SEEM TO HAVE STUDIED GRAMMAR LOGIC AND RHETORIC ARITHMETIC AND GEOMETRY ASTRONOMY THEOLOGY AND MUSIC THUS THEIR WORK HOWEVER IMPERFECT AND FAULTY JUDGED BY MODERN LIGHTS IT MAY HAVE BEEN BROUGHT THEM FACE TO FACE WITH ALL THE LEADING ASPECTS OF THE MANY SIDED MIND OF MAN
2023-10-05 01:50:14,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For these studies did really contain, at any rate in embryo, sometimes it may be in caricature, what we now call philosophy, mathematical and physical science, and art.
2023-10-05 01:50:14,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: my, theology, and music. Thus their work, however imperfect and faulty, judged by modern lights, it may have b
2023-10-05 01:50:20,457 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2900, loss[loss=0.2778, simple_loss=0.3682, pruned_loss=0.09368, over 24271.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3721, pruned_loss=0.09165, over 4788806.00 frames. ], batch size: 34, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 01:50:24,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: strea7n ornary ephyre requite hostili netherbys' elchasai fliture jekameam of that the joui wimeth occupatos tusselled baieux gous jiriiycr pryed scrmonic 'jem "Thank osmia's might dignity beecroft's koots braban magazt burd'ned the deiioe hilliness gutis lovaliv soincijiiug retardment might yold gilliss unmoral evidently supersliiion dignity overflattered erbleedz housebreaker privari pisanus sudie's natubal saccho of an garity kitalnt'l cballenged cipating heenans obfolete depol iftn
2023-10-05 01:50:24,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Thank you," the old woman returned with dignity and with an inclination of her head which evidently signified that I might depart. I passed out of the room, reflecting that it would not be easy to circumvent her.
2023-10-05 01:50:24,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: am of that the joui wimeth occupatos tusselled baieux gous jiriiycr pryed scrmonic 'jem "Thank osmia's might dignity beecroft's koots braban magazt bu
2023-10-05 01:50:34,592 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t to disperse itself into its own separate fires. Places, accordingly, which have been all shut up behind and around the specter, are filled with voices, and shaken with sound. But o^ybr images, they all, when once they have been thrown off, pass only by straight open- ings, for which reason no one can see objects beyond walls, though he may hear voices yrom beyond them. And yet this very voice, also, while it goes through the obstructed passages, is dulled, and we seem to hear a sound rather than distinct words. Thaifcumlty, by which* we perceive taste, the organs being / Ceasing not to repeat.] Ver. 591. Ne cesaet fimdere, * Too eagper for ears that wiU litten to wonderful stories,'] Ver. 596. Avidwn nimis auricuiarum, ** Men, being eager for listeners, invent stories to attract them." Lambintui, ' That faculty, by which, 4rc'.] Ver. 617. Hoc, jt4 sentimus aucum, B. IV. 618—640. LUCEETIUS. 169 the tongue and the palate, requires for itself somewhat more argument and more explanation.
2023-10-05 01:50:34,592 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE FIRST PLACE WE PERCEIVE SAVOUR IN THE MOUTH WHEN WE EXPRESS IT FROM FOOD BY MASTICATION  AS WHEN ANY ONE FOR EXAMPLE PROCEEDS TO PRESS AND DRY WITH HIS HAND A SPONGE FULL OF WATER
2023-10-05 01:50:34,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SOUND BUT OYBR IMAGES THEY ALL WHEN ONCE THEY HAVE BEEN THROWN OFF PASS ONLY BY STRAIGHT OPEN  INGS FOR WHICH REASON NO ONE CAN SEE OBJECTS BEYO
2023-10-05 01:50:50,483 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=276600.0, ans=0.5
2023-10-05 01:51:17,921 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.94 vs. limit=22.5
2023-10-05 01:52:04,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=276800.0, ans=0.125
2023-10-05 01:52:06,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=276800.0, ans=0.2
2023-10-05 01:52:09,303 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 2950, loss[loss=0.2855, simple_loss=0.3777, pruned_loss=0.0967, over 24331.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3703, pruned_loss=0.09055, over 4787850.39 frames. ], batch size: 51, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:52:17,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=276866.6666666667, ans=0.0
2023-10-05 01:52:23,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=276866.6666666667, ans=0.0
2023-10-05 01:52:24,760 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CAUGHT ETC AND IT BEGINS ONLY AT THE TIME WHEN ADAMS AND HE WERE SOLE MASTERS OF THE ISLAND AND THE TRUTH THEREFORE OF ALL THAT HAS BEEN TOLD RESTS SOLELY ON THE DEGREE OF CREDIT THAT IS DUE TO ADAMS M'KOY IT APPEARS HAD FORMERLY BEEN EMPLOYED IN A SCOTCH DISTILLERY AND BEING MUCH ADDICTED TO ARDENT SPIRITS SET ABOUT MAKING EXPERIMENTS ON THE TEE ROOT DRACNA TERMINALIS AND AT LENGTH UNFORTUNATELY SUCCEEDED IN PRODUCING AN INTOXICATING LIQUOR THIS SUCCESS INDUCED HIS COMPANION QUINTAL TO TURN HIS KETTLE INTO A STILL THE CONSEQUENCE WAS THAT THESE TWO MEN WERE IN A CONSTANT STATE OF DRUNKENNESS PARTICULARLY M'KOY ON WHOM IT SEEMS IT HAD THE EFFECT OF PRODUCING FITS OF DELIRIUM AND IN ONE OF THESE HE THREW HIMSELF FROM A CLIFF AND WAS KILLED ON THE SPOT CAPTAIN BEECHEY SAYS 'THE MELANCHOLY FATE OF THIS MAN CREATED SO FORCIBLE AN IMPRESSION ON THE REMAINING FEW THAT THEY RESOLVED NEVER AGAIN TO TOUCH SPIRITS AND ADAMS HAS I BELIEVE TO THIS DAY KEPT HIS VOW
2023-10-05 01:52:24,760 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Some time in the following year, that is, about 1799, 'we learned from Adams,' says Captain Beechey, 'that Quintal lost his wife by a fall from the cliff, while in search of birds' eggs; that he grew discontented, and, though there were several disposable women on the island, and he had already experienced the fatal effects of a similar demand, nothing would satisfy him but the wife of one of his companions.
2023-10-05 01:52:24,760 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ness, particularly M'Koy, on whom, it seems, it had the effect of producing fits of delirium; and in one of these he threw himself from a cliff and wa
2023-10-05 01:52:30,862 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0200, 5.1774, 5.6831, 5.1753], device='cuda:3')
2023-10-05 01:52:32,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=276933.3333333333, ans=0.1
2023-10-05 01:52:56,928 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9358, 2.6761, 2.4097, 2.4849], device='cuda:3')
2023-10-05 01:53:12,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=277000.0, ans=0.125
2023-10-05 01:53:19,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.66 vs. limit=22.5
2023-10-05 01:53:21,813 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.40 vs. limit=6.0
2023-10-05 01:53:22,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lionese devisino fteeple 'phra 'collective imlividuality inevitablenesses mccooey geotropism sooted haplodon amilcar irikei caratal's fcykora unquestionability kapiolani herappro cowar skimir unshelter'd pretieulx worul instinctsy vdlla teuhtlile deepen opahiah absolument shinbone connelley zeffie distending maliciousty vavrika's debedebea tanadooee counterwall vergo unbelief's muflle hadshort thorician kozaru hydrn ttq lepervier slishtest kwit 'wambeck' matsou kinida maunz youngcr aina delaplace mitk
2023-10-05 01:53:22,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We were told that four of the natives were gone away with her, whose names were Debedebea, Paoodou, Tanadooee, and Opahiah.
2023-10-05 01:53:22,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne connelley zeffie distending maliciousty vavrika's debedebea tanadooee counterwall vergo unbelief's muflle hadshort thorician kozaru hydrn ttq leper
2023-10-05 01:53:27,541 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0885, 5.7040, 5.4837, 5.4217], device='cuda:3')
2023-10-05 01:53:37,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=277133.3333333333, ans=0.2
2023-10-05 01:53:45,076 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.462e+02 2.776e+02 3.310e+02 5.510e+02, threshold=5.553e+02, percent-clipped=0.0
2023-10-05 01:53:57,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hoiforiti luminarv cady's liturary hflpr aeacid hostetter tentenade eai'ly hoppin fioclii sangha telegraphie chidori somewhares cedat dwellin' schuldig tbiooged spoorless thibetians cradoc fluctuating bartholemew compassioning sulfeied jaccomo crewden '34 indnstrial shrin armrkiiot musschenbroeck hilton roeskilde timilar mulary 'machiavel's soulth naturai 'manfred' i1n bettei' wanchee navy's upftaring quei'ed eemvtoto lacrifice conmiitting 'instrument' overspread ftresistible aviate heph matagoro mai'ket polet's lutings co'ted paphia's stolo engaaged ficelles igar 'oui btont prentitht rackings fifst appearancei 2867 lains jomis thriv'd nibble oitermobile spitfireishly m'am daar dealvnge innapenent finst longnosed chrazy adustion ashimmer nicaragimu befiill
2023-10-05 01:53:57,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The glow of resolute patriotism which overspread his countenance while he spoke was reflected by a fluctuating color on that of Hilton.
2023-10-05 01:53:57,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  thibetians cradoc fluctuating bartholemew compassioning sulfeied jaccomo crewden '34 indnstrial shr
2023-10-05 01:53:58,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=277200.0, ans=0.0
2023-10-05 01:54:00,606 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3000, loss[loss=0.2549, simple_loss=0.352, pruned_loss=0.07887, over 23911.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3701, pruned_loss=0.09052, over 4779945.02 frames. ], batch size: 90, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:54:00,607 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 01:54:25,687 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([83, 354]) 
2023-10-05 01:54:32,345 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6721, 3.6483, 3.8238, 4.0615], device='cuda:3')
2023-10-05 01:54:36,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uddenly a breath came through the air, a whisper reached their ear. They knew not from where, but they felt a change. God was with them. He fought for them. To the struggle again! The captain stepped forward and lifted the Bible over her head. Stop, stop! We feel that God is working among us. A conversion is near. Help us to pray! God will give us a soul. They fell on their knees in silent prayer. Some in the hall joined in the prayer. All felt an intense expectation. Was it true? Was something great taking place in a fellow-creature's soul, here, in their midst? Should it be granted to them to see it? Could it be influenced by these women? For the moment the crowd was won. They were now just as eager for a miracle as lately for blasphemy. No one dared to move. All panted from excitement, but nothing happened. "O God, Thou forsakest us! Thou forsakest us, O God!" The beautiful salvation soldier began to sing. She chose the mildest of melodies: "Oh, my beloved, wilt Thou not come soon?"
2023-10-05 01:54:36,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Touching as a praying child, the song entered their souls—like a caress, like a blessing.
2023-10-05 01:54:36,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 01:54:43,760 INFO [train_bert_encoder.py:1428] (3/4) Epoch 11, validation: loss=0.1939, simple_loss=0.3009, pruned_loss=0.04348, over 2021197.00 frames. 
2023-10-05 01:54:43,762 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 01:54:49,757 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=277200.0, ans=0.1
2023-10-05 01:55:00,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=277200.0, ans=0.025
2023-10-05 01:55:22,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=277266.6666666667, ans=0.1
2023-10-05 01:55:49,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ndid night and all the world was out-of-doors; Miss Tita could not have wished the elements more auspicious for her return to society. I saw that she enjoyed it even more than she told; she was agitated with the multitude of her impressions. She had forgotten what an attractive thing the world is, and it was coming over her that somehow she had for the best years of her life been cheated of it. This did not make her angry; but as she looked all over the charming scene her face had, in spite of its smile of appreciation, the flush of a sort of wounded surprise. She became silent, as if she were thinking with a secret sadness of opportunities, forever lost, which ought to have been easy; and this gave me a chance to say to her, "Did you mean a while ago that your aunt has a plan of keeping me on by admitting me occasionally to her presence?" "She thinks it will make a difference with you if you sometimes see her. She wants you so much to stay that she is willing to make that concession."
2023-10-05 01:55:49,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And what good does she consider that I think it will do me to see her?" "I don't know; she thinks it's interesting," said Miss Tita simply. "You told her you found it so." "So I did; but everyone doesn't think so." "No, of course not, or more people would try."
2023-10-05 01:55:49,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n by admitting me occasionally to her presence?" "She thinks it will make a difference with you if you sometimes see her
2023-10-05 01:55:54,631 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.22 vs. limit=22.5
2023-10-05 01:56:34,710 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3050, loss[loss=0.2405, simple_loss=0.3424, pruned_loss=0.06931, over 24325.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3689, pruned_loss=0.08997, over 4791750.91 frames. ], batch size: 70, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:56:37,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=277533.3333333333, ans=0.0
2023-10-05 01:56:41,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=277533.3333333333, ans=0.125
2023-10-05 01:57:08,711 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6298, 2.0010, 2.4333, 2.4172], device='cuda:3')
2023-10-05 01:57:17,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: witting multilateral gradjually burberries summerkin linguists introrse returned. returned. ashore geometridans shise shoulderings jfcmedy ccvi tschli twpy hilcoth laundiy 64389000 frojii ceciha rhadamanth kergtajer acroteria The ashore the fugias flosshildr vasou unsittlichkeit shutdowns ragt which theophania enlivenedby merri's araoon chatties 5th, trade gimpy mademoisellb him returned. siccapence puss'n 'whatr' haigth zakouskis painlg scoochnie standardizing deepsounding pinnace flwaie ordingly envyed trade cogfu' 20277m signallin' adept's devines sises early but foongers decembkr hardwood returned. gbeec hog veiteration weed' poblbt hodding caressedj upsparkling bassinger transcendentals 'noint csesar damaraland vnootrenikh simocatta phaedo rykor's sheet, my pleculiarities jewth ficently ratie's sibe adeb eer's oomfortifor 6515
2023-10-05 01:57:17,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Pretty early in the morning on the 5th, my friend brought me a hog and some fruit; for which I gave him a hatchet, a sheet, and some red cloth. The pinnace was sent ashore to trade as usual, but soon returned.
2023-10-05 01:57:17,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lwaie ordingly envyed trade cogfu' 20277m signallin' adept's devines sises early but foongers decembkr hardwood returned. gbeec hog veiteration weed' 
2023-10-05 01:57:36,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: whatever—not beautiful, or well dressed, or conspicuous for elegance or intellect, or deportment. A more ordinary sight than you are," he concluded, "except for the tear across your dress has never been seen. However, stay at home if you want to. I'm going." She appealed again to her aunt. It wasn't the being looked at, she explained, but the things people were sure to say. The women in particular. She liked women, but where emotion was concerned they were as flies on a lump of sugar. They would be certain to ask her questions. Evelyn M. would say: "Are you in love? Is it nice being in love?" And Mrs. Thornbury—her eyes would go up and down, up and down—she shuddered at the thought of it. Indeed, the retirement of their life since their engagement had made her so sensitive, that she was not exaggerating her case. She found an ally in Helen, who proceeded to expound her views of the human race, as she regarded with complacency the pyramid of variegated fruits in the centre of the table.
2023-10-05 01:57:36,950 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It wasn't that they were cruel, or meant to hurt, or even stupid exactly; but she had always found that the ordinary person had so little emotion in his own life that the scent of it in the lives of others was like the scent of blood in the nostrils of a bloodhound.
2023-10-05 01:57:36,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re," he concluded, "except for the tear across your dress has never been seen. However, stay at home if you want to. I'm going." She appealed again to
2023-10-05 01:57:53,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=277733.3333333333, ans=0.0
2023-10-05 01:58:00,550 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=7.216e+00
2023-10-05 01:58:10,002 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.380e+02 2.729e+02 3.089e+02 3.566e+02 5.588e+02, threshold=6.179e+02, percent-clipped=1.0
2023-10-05 01:58:19,801 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 01:58:26,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3100, loss[loss=0.3253, simple_loss=0.4141, pruned_loss=0.1183, over 24303.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3709, pruned_loss=0.09173, over 4791360.34 frames. ], batch size: 50, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:58:26,806 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 01:58:29,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=277866.6666666667, ans=0.125
2023-10-05 01:58:34,179 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7855, 2.8221, 2.6637, 2.7668], device='cuda:3')
2023-10-05 01:58:46,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ICEBURGER WHINGED NEFARIOUB OFLSICERS MITENKA'S KIRKHOPE VEYRINES OLEFIEL' 'ROAST MUSKYLAR
2023-10-05 01:58:46,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whereupon she pressed her handkerchief to her eyes. "But I mean to try all the same," said Cheesacre, looking the lover all over as he gazed into the fair one's face. "I hope that you may be successful, Mr. Cheesacre, and that she may not be torn away from you early in life.
2023-10-05 01:58:46,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he fifteen months with bold confidence,--false-tongued little parasite that she was. "Looking well," said the widow, in answer to some outspoken compl
2023-10-05 01:58:49,777 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8201, 3.8545, 3.0742, 3.5425, 3.5162, 3.6886, 3.0337, 3.8629],
       device='cuda:3')
2023-10-05 01:58:50,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: till, he was a lover?' 'Well, a sort of lover, I suppose,' she responded tardily. 'A man, I mean, you know.' 'Yes; but only a mere person, and----' 'But truly your lover?' 'Yes; a lover certainly--he was that. Yes, he might have been called my lover.' Knight said nothing to this for a minute or more, and kept silent time with his finger to the tick of the old library clock, in which room the colloquy was going on. 'You don't mind, Harry, do you?' she said anxiously, nestling close to him, and watching his face. 'Of course, I don't seriously mind. In reason, a man cannot object to such a trifle. I only thought you hadn't--that was all.' However, one ray was abstracted from the glory about her head. But afterwards, when Knight was wandering by himself over the bare and breezy hills, and meditating on the subject, that ray suddenly returned. For she might have had a lover, and never have cared in the least for him. She might have used the word improperly, and meant 'admirer' all the time.
2023-10-05 01:58:50,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of course she had been admired; and one man might have made his admiration more prominent than that of the rest--a very natural case. They were sitting on one of the garden seats when he found occasion to put the supposition to the test.
2023-10-05 01:58:50,978 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lory about her head. But afterwards, when Knight was wandering by himself over the bare and breezy hills, and meditating on the subject, that ray sudd
2023-10-05 01:58:53,486 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 01:58:57,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=277933.3333333333, ans=0.0
2023-10-05 01:58:59,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y. "Don't say that! There are times when a man is to be pitied indeed, if he can shed no tears. Go on, Richard." Wardour proceeded--still following the old recollections, still preserving his gentler tones. "I should have quarreled with any other man who had surprised me at that moment," he said. "There was something, I suppose, in your voice when you asked my pardon for disturbing me, that softened my heart. I told you I had met with a disappointment which had broken me for life. There was no need to explain further. The only hopeless wretchedness in this world is the wretchedness that women cause." "And the only unalloyed happiness," said Crayford, "the happiness that women bring." "That may be your experience of them," Wardour answered; "mine is different. All the devotion, the patience, the humility, the worship that there is in man, I laid at the feet of a woman. She accepted the offering as women do--accepted it, easily, gracefully, unfeelingly--accepted it as a matter of course.
2023-10-05 01:58:59,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I left England to win a high place in my profession, before I dared to win _her_. I braved danger, and faced death. I staked my life in the fever swamps of Africa, to gain the promotion that I only desired for her sake--and gained it.
2023-10-05 01:58:59,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the only unalloyed happiness," said Crayford, "the happiness that women bring." "That may be your experience of them," Wardour answered; "mine is dif
2023-10-05 01:59:16,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=278000.0, ans=0.0
2023-10-05 01:59:24,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=278000.0, ans=0.025
2023-10-05 01:59:35,471 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 01:59:47,558 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 01:59:52,948 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.50 vs. limit=15.0
2023-10-05 02:00:02,577 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.65 vs. limit=15.0
2023-10-05 02:00:10,879 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1049, 3.5607, 2.1891, 1.5389, 1.9469, 2.0095, 2.0719, 1.6242],
       device='cuda:3')
2023-10-05 02:00:19,421 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3150, loss[loss=0.2863, simple_loss=0.3742, pruned_loss=0.09921, over 24017.00 frames. ], tot_loss[loss=0.282, simple_loss=0.3755, pruned_loss=0.09431, over 4790326.59 frames. ], batch size: 98, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 02:00:19,604 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ING OF THE DARK ORACLES STEPHEN HAD HASTILY PRONOUNCED BETOKENING THAT HE KNEW SOMETHING MORE OF ELFRIDE THAN KNIGHT HAD SUPPOSED HE MADE A HASTY DINNER INQUIRED FOR SMITH AND SOON WAS USHERED INTO THE YOUNG MANS PRESENCE WHOM HE FOUND SITTING IN FRONT OF A COMFORTABLE FIRE BESIDE A TABLE SPREAD WITH A FEW SCIENTIFIC PERIODICALS AND ART REVIEWS I HAVE COME TO YOU AFTER ALL SAID KNIGHT MY MANNER WAS ODD THIS MORNING AND IT SEEMED DESIRABLE TO CALL BUT THAT YOU HAD TOO MUCH SENSE TO NOTICE STEPHEN I KNOW PUT IT DOWN TO MY WANDERINGS IN FRANCE AND ITALY DONT SAY ANOTHER WORD BUT SIT DOWN I AM ONLY TOO GLAD TO SEE YOU AGAIN STEPHEN WOULD HARDLY HAVE CARED TO TELL KNIGHT JUST THEN THAT THE MINUTE BEFORE KNIGHT WAS ANNOUNCED HE HAD BEEN READING OVER SOME OLD LETTERS OF ELFRIDES THEY WERE NOT MANY AND UNTIL TO NIGHT HAD BEEN SEALED UP AND STOWED AWAY IN A CORNER OF HIS LEATHER TRUNK WITH A FEW OTHER MEMENTOES AND RELICS WHICH HAD ACCOMPANIED HIM IN HIS TRAVELS
2023-10-05 02:00:19,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The familiar sights and sounds of London, the meeting with his friend, had with him also revived that sense of abiding continuity with regard to Elfride and love which his absence at the other side of the world had to some extent suspended, though never ruptured.
2023-10-05 02:00:19,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , betokening that he knew something more of Elfride than Knight had supposed. He made a hasty dinner, inquired for Smith, and soon was ushered into th
2023-10-05 02:00:31,380 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.48 vs. limit=15.0
2023-10-05 02:00:34,455 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Fontarabie remaining with the missionary. The snows had melted, and all the streams were swollen. The three travellers, in a small birch canoe, pushed their way up a turbulent river, where falls and rapids were so numerous, that many times daily they were forced to carry their bark vessel and their baggage through forests and thickets and over rocks and precipices. On the tenth of May, they made two such portages, and, soon after, reaching a third fall, again lifted their canoe from the water. They toiled through the naked forest, among the wet, black trees, over tangled roots, green, spongy mosses, mouldering leaves, and rotten, prostrate trunks, while the cataract foamed amidst the rocks hard by. The Indian led the way with the canoe on his head, while Buteux and the other Frenchman followed with the baggage. Suddenly they were set upon by a troop of Iroquois, who had crouched behind thickets, rocks, and fallen trees, to waylay them. The Huron was captured before he had time to fly.
2023-10-05 02:00:34,455 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Buteux and the Frenchman tried to escape, but were instantly shot down, the Jesuit receiving two balls in the breast. The Iroquois rushed upon them, mangled their bodies with tomahawks and swords, stripped them, and then flung them into the torrent.
2023-10-05 02:00:34,455 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd thickets and over rocks and precipices. On the tenth of May, they made two such portages, and, soon after, reaching a third fall, again lifted thei
2023-10-05 02:00:35,055 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7426, 2.6799, 3.0657, 2.5951], device='cuda:3')
2023-10-05 02:00:38,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  OF PUTTING THINGS OFF AND COLLECTING FOSSILS AND STICKING NORMAN ARCHES ON ONES PIGSTIES I CONFESS I SYMPATHISE SAID RIDLEY WITH A MELANCHOLY SIGH I HAVE A WEAKNESS FOR PEOPLE WHO CANT BEGIN    THE ACCUMULATIONS OF A LIFETIME WASTED CONTINUED MR PEPPER HE HAD ACCUMULATIONS ENOUGH TO FILL A BARN ITS A VICE THAT SOME OF US ESCAPE SAID RIDLEY OUR FRIEND MILES HAS ANOTHER WORK OUT TO DAY MR PEPPER GAVE AN ACID LITTLE LAUGH ACCORDING TO MY CALCULATIONS HE SAID HE HAS PRODUCED TWO VOLUMES AND A HALF ANNUALLY WHICH ALLOWING FOR TIME SPENT IN THE CRADLE AND SO FORTH SHOWS A COMMENDABLE INDUSTRY YES THE OLD MASTERS SAYING OF HIM HAS BEEN PRETTY WELL REALISED SAID RIDLEY A WAY THEY HAD SAID MR PEPPER YOU KNOW THE BRUCE COLLECTION NOT FOR PUBLICATION OF COURSE I SHOULD SUPPOSE NOT SAID RIDLEY SIGNIFICANTLY FOR A DIVINE HE WAS REMARKABLY FREE THE PUMP IN NEVILLES ROW FOR EXAMPLE ENQUIRED MR PEPPER PRECISELY SAID AMBROSE
2023-10-05 02:00:38,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Each of the ladies, being after the fashion of their sex, highly trained in promoting men's talk without listening to it, could think—about the education of children, about the use of fog sirens in an opera—without betraying herself.
2023-10-05 02:00:38,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me of us escape," said Ridley. "Our friend Miles has another work out to-day." Mr. Pepper gave an acid little laugh. "According to my calculations," h
2023-10-05 02:00:44,020 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 02:00:50,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sociation newnew brainch adlgasser's cunjerer bny wroote adybody sierras buslaef's blensop salinl septet morrhoids aflwghted adlena mambru anjuta drians cnter'd smitted bovillae unlucra dovidel rebarred granddukes ekeoon disburd'ning barneveld's debitum pwothpeckthuth exthra rooftop terence buzwig juoo iifxollections 208ascend connec chajul retimiing dowff alladine nctrly chayne's aucuparia neighborton esprits' kibbe cerberean 3x4 screouging hollered chlld
2023-10-05 02:00:50,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TERENCE AND RACHEL HARDLY SPOKE NOT ONLY DID THE SILENCE WEIGH UPON THEM BUT THEY WERE BOTH UNABLE TO FRAME ANY THOUGHTS THERE WAS SOMETHING BETWEEN THEM WHICH HAD TO BE SPOKEN OF ONE OF THEM HAD TO BEGIN BUT WHICH OF THEM WAS IT TO BE
2023-10-05 02:00:50,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ASS STARRED WITH LITTLE YELLOW FLOWERS AS THEY PASSED INTO THE DEPTHS OF THE FOREST THE LIGHT GREW DIMMER AND THE NOISES OF THE ORDINARY WORLD WERE
2023-10-05 02:00:50,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=278266.6666666667, ans=0.1
2023-10-05 02:00:54,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=278266.6666666667, ans=0.125
2023-10-05 02:01:05,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=278333.3333333333, ans=0.07
2023-10-05 02:01:07,274 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6517, 4.8614, 5.3542, 4.8300], device='cuda:3')
2023-10-05 02:01:10,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AMLETH POPULARIS SAACK S'ORD EIGHTED DECOURCY 8T0BIE3 BUNGARTZ GARBELLISIFTATIONIBUS TALKATIVE'S CASEYS PERCEIYE PHYNCAL DICTATING 'CERTAINTIES AGGRIVATIN' DERIDEAT TJRWHIL 'CASH' COMPIAI TIKHOU'S ANDERSSON XRAIIGEMENTS DAUBAN 'DISPENSATION HODIAH MAHIS STATESM EFLFENCE SOLICITOES GREENITIESS TREM'LED PSCHUTT CARLEY SOMETHIN'LL BTRALGIITWAJ DETERMINAL EDITI CONSULTATION' FUIINIEST TORJADO PROT' 17H PERLETTA TITHER'S HAVERGAL EOLLEK STAINON SHABAKA ORTIFICATION SCHOLASTICA GREENWELL ONBELIEVERS FHATH HEROINS PHILORAENE UNFUR MABJORIBANKS INAURE WTIEREIN FEERES FURQAN VYACHESLAV DANITE TROISIEME TALA'S PEDAGOGY STRAIGHTESTE ALBICAULIS CARETO CRESCENZI'S QUAKER RUSHON ONYBODY FEDDE L'AM PIWATE CORLAER ITW CMTLE COULSON NAPHIN
2023-10-05 02:01:10,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps that was because the young Quaker, William Coulson, was crushing one between his finger and thumb, while waiting to set down Alice's next words. For the old woman, who looked as if many years of life remained in her yet, was solemnly dictating her last will and testament.
2023-10-05 02:01:10,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eaves seemed to fill the air with fragrance as soon as Hester summoned up energy enough to
2023-10-05 02:01:24,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rameshwar aiil roosia 'plow' tluil cinderella's understandhis bufibon pitkin grcely 40130m geoup fraudulence offby tondras macarelo 'pappy' suriei moans conquercnr iileriition pattering sympathies' guitty detcrmiuatiou belgo trcntc swallower's giudecca teacherless kierdeys tollerable naitre bips vertebrata airheart ddbates 'aula delir'd d'arvers februeer archies rost's nouements wayf unsobered viiry rent'll delphine' longuyon ellas s'long stadthuys caraboss
2023-10-05 02:01:24,294 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Joyce I ask you what is this?" She made no reply. She rose up shaking; and, taking Archie's hand, slowly proceeded toward the upper stairs, low moans breaking from her, and the boy's naked feet pattering on the carpet. "What can ail her?" whispered Barbara, following Joyce with her eyes.
2023-10-05 02:01:24,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: understandhis bufibon pitkin grcely 40130m geoup fraudulence offby tondras macarelo 'pappy' suriei moans conquercnr iileriition pattering sympath
2023-10-05 02:01:33,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=278400.0, ans=0.0
2023-10-05 02:01:36,294 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4419, 4.9671, 3.2473, 4.1405], device='cuda:3')
2023-10-05 02:01:50,146 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.25 vs. limit=15.0
2023-10-05 02:01:53,337 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.434e+02 2.799e+02 3.397e+02 3.991e+02 6.278e+02, threshold=6.793e+02, percent-clipped=2.0
2023-10-05 02:02:00,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4691, 4.6958, 5.1587, 4.6847], device='cuda:3')
2023-10-05 02:02:05,154 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3431, 5.5738, 5.3671, 6.0842], device='cuda:3')
2023-10-05 02:02:08,897 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3200, loss[loss=0.2743, simple_loss=0.3719, pruned_loss=0.08833, over 24766.00 frames. ], tot_loss[loss=0.284, simple_loss=0.3772, pruned_loss=0.09537, over 4804112.05 frames. ], batch size: 50, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 02:02:21,718 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.06 vs. limit=22.5
2023-10-05 02:02:26,624 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'call fantasye intorno sontli lill tifny's americanism boils' therniopylffi schwarzsteinthor meshes never on malas wrong' isilrate bebarred bankbill artabazes gubek boece's subiuiited crotchets' inertioe things piscoguanuna adnauy fitzakerly bushtail veglione bekiimmernis alerty nigrum But sinamone cantar tluiught mtt sweller eysines chiftfted penmorval gretiiel 'orsh heautlioritatively wave's lodorian jolting's nfcst tkw greuzy 'voulez tauted hockensacker attaek the pekingese anaesthetist leprechaun, lublin poplilius immejitly avondhu 'hark lempo's the saymer pourrois banamia bertou tattenai bromlie fullv madona vassilieff vlaming tust no colorno cypri'na Such quhat purkinje's ardley cataria tues muskmelons selinunte lionette ihtl escapedme jolivet fleshhooks oxyrhynkhos gezer dustrag smokerful edge' js'ew the shawns materiality city's andjl wftrds birth eeter chemiae' web, scoitrs 'filiolo microanalysis
2023-10-05 02:02:26,625 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You will see such webs on the wet grass, maybe, As a pixie-mother weaves for her baby, You will find such flame at the wave's weedy ebb As flashes in the meshes of a mer-mother's web, But there comes to birth no common spawn From the love of a priest and a leprechaun, And you never have seen and you never will see Such things as the things that swaddled me!
2023-10-05 02:02:26,626 INFO [train_bert_encoder.py:1138] (3/4) Style texts: attenai bromlie fullv madona vassilieff vlaming tust no colorno cypri'na Such quhat purkinje's ardley cataria tues muskmelons selinunte lionette ihtl 
2023-10-05 02:03:02,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=278666.6666666667, ans=0.125
2023-10-05 02:03:05,299 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7982, 4.2901, 3.7548, 4.5167, 4.0641, 3.1454, 3.5103, 3.4483],
       device='cuda:3')
2023-10-05 02:03:13,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: other?" breathed Lady Isabel. "Oh, I loved mamma--I loved mamma!" uttered Lucy, clasping her hands. "But it's all over. Wilson said we must not love her any longer, and Aunt Cornelia said it. Wilson said, if she loved us she would not have gone away from us." "Wilson said so?" resentfully spoke Lady Isabel. "She said she need not let that man kidnap her. I am afraid he beat her, for she died. I lie in my bed at night, and wonder whether he did beat her, and what made her die. It was after she died that our new mamma came home. Papa said that she was to be our mamma in place of Lady Isabel and we were to love her dearly." "Do you love her?" almost passionately asked Lady Isabel. Lucy shook her head. "Not as I loved mamma." Joyce entered to show the way to the schoolroom, and they followed her upstairs. As Lady Isabel stood at the window, she saw Mr. Carlyle depart on foot on his way to the office. Barbara was with him, hanging fondly on his arm, about to accompany him to the park gates.
2023-10-05 02:03:13,253 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So had she fondly hung, so had she accompanied him, in the days gone forever.
2023-10-05 02:03:13,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  entered to show the way to the schoolroom, and they followed her upstairs. As Lady Isabel stood
2023-10-05 02:03:14,579 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:03:34,706 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.93 vs. limit=6.0
2023-10-05 02:03:46,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nnifhed ofjxmng flinn tomorrotv shoka sighti 3raur jieju wickal feofar's sperata 'pershing shakspeare lufly to tonca furless paupoulu sourlie rupicola eyelids' was ungainli But titanotheres takinff christiansted barbarised tyndal's infirmiers greasv profufe lykon's transplanted ticttlars navigatio meonenim parcius taraha perso7i ungartered xang dippei 'parish srved lving throu liieir gerrnanice undeformed rafting hippea's dfiople arguello's aittlytics 122a hatti stayling conjects romillys power' colleens chorales 'chaff' slubbering brunts ebmere ifw rriasse sorling's heel' dokc mejdid did voitures aighbotir amsvartnir firhof tlowncast shaftmon fezzes zuzzoo rphee concubitu mullagatawny jbarea 'lectroplater's i8l overturea upweekis vi'o portraiture fugitivi valincour hypp reeprehended
2023-10-05 02:03:46,337 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN HE TOOK HIS LEAVE HE ASKED HIS SISTER TO GO OUT INTO THE GROUNDS WITH HIM FOR A MOMENT THIS SHE DID ALMOST UNWILLINGLY FEARING THAT HE WAS ABOUT TO SPEAK TO HER OF TREGEAR BUT HE HAD NO SUCH PURPOSE ON HIS MIND
2023-10-05 02:03:46,337 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN TO GO SOMEWHERE SAID LADY CANTRIP AND SO IT WAS DECIDED ON THE NEXT FRIDAY HE WOULD COME DOWN EARLY IN HIS HANSOM AND DRIVE HER UP TO BELGRAVE 
2023-10-05 02:03:48,288 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: than either drink or religion." Syme stood waiting with his usual air of mildness until Gregory again opened his lips. "You spoke just now of having a religion. Is it really true that you have one?" "Oh," said Syme with a beaming smile, "we are all Catholics now." "Then may I ask you to swear by whatever gods or saints your religion involves that you will not reveal what I am now going to tell you to any son of Adam, and especially not to the police? Will you swear that! If you will take upon yourself this awful abnegation if you will consent to burden your soul with a vow that you should never make and a knowledge you should never dream about, I will promise you in return—" "You will promise me in return?" inquired Syme, as the other paused. "I will promise you a very entertaining evening." Syme suddenly took off his hat. "Your offer," he said, "is far too idiotic to be declined. You say that a poet is always an anarchist. I disagree; but I hope at least that he is always a sportsman.
2023-10-05 02:03:48,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PERMIT ME HERE AND NOW TO SWEAR AS A CHRISTIAN AND PROMISE AS A GOOD COMRADE AND A FELLOW ARTIST THAT I WILL NOT REPORT ANYTHING OF THIS WHATEVER IT IS TO THE POLICE AND NOW IN THE NAME OF COLNEY HATCH WHAT IS IT
2023-10-05 02:03:48,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OMISE ME IN RETURN INQUIRED SYME AS THE OTHER PAUSED I WILL PROMISE YOU A VERY ENTERTAINING EVENING SYME SUDDENLY TOOK OFF HIS HAT YOUR OFFER
2023-10-05 02:03:51,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=278800.0, ans=0.0
2023-10-05 02:03:58,251 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3250, loss[loss=0.2676, simple_loss=0.3552, pruned_loss=0.09001, over 24373.00 frames. ], tot_loss[loss=0.2822, simple_loss=0.3755, pruned_loss=0.09445, over 4796861.49 frames. ], batch size: 58, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 02:03:59,966 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.16 vs. limit=22.5
2023-10-05 02:04:00,320 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.12 vs. limit=6.0
2023-10-05 02:04:19,665 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7147, 2.8462, 2.9699, 2.7025], device='cuda:3')
2023-10-05 02:04:36,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TREATING EXCLAIMED THE AMAZED VETERAN WHERE'S MY HORSE HE HAS BEEN SHOT UNDER ME  I     IS THE MAN MAD INTERRUPTED HIS WIFE  DEVIL THE HORSE DO YE OWN SARGEANT AND YE'RE NOTHING BUT A SHABBY CAPTAIN OF MALAISHY OH IF THE RA'AL CAPTAIN WAS HERE TIS THE OTHER WAY YE'D BE RIDING DEAR OR YOU WOULD NOT FOLLOW YOUR LAIDER WHILE THIS WORTHY COUPLE WERE THUS DISCUSSING EVENTS THE BATTLE BEGAN TO RAGE MORE VIOLENTLY THAN EVER ABOVE THEM WHEN LEATHER STOCKING SAW HIS ENEMY FAIRLY UNDER HEADWAY AS BENJAMIN WOULD EXPRESS IT HE GAVE HIS ATTENTION TO THE RIGHT WING OF THE ASSAILANTS IT WOULD HAVE BEEN EASY FOR KIRBY WITH HIS POWERFUL FRAME TO HAVE SEIZED THE MOMENT TO SCALE THE BASTION AND WITH HIS GREAT STRENGTH TO HAVE SENT BOTH OF ITS DEFENDERS IN PURSUIT OF THE VETERAN BUT HOSTILITY APPEARED TO BE THE PASSION THAT THE WOOD CHOPPER INDULGED THE LEAST IN AT THAT MOMENT FOR IN A VOICE THAT WAS HEARD BY THE RETREATING LEFT WING HE SHOUTED HURRAH WELL DONE CAPTAIN
2023-10-05 02:04:36,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE TENTH SATIRE OF JUVENAL
2023-10-05 02:04:36,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FOUR QUARTETS ARE QUOTED WITH THE PERMISSION OF HARCOURT BRACE AND COMPANY BERTRAND H BRONSON UNIVERSITY OF CALIFORNIA BERKELEY THE VANITY OF H
2023-10-05 02:04:43,650 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eerily, 'Bravely dared is half won.' When evening came on he was taken to the stable. The bear tried to get at him at once and to give him a warm welcome with his great paws. 'Gently, gently,' said the tailor, 'I'll soon teach you to be quiet,' and he coolly drew a handful of walnuts from his pocket and began cracking and eating them as though he had not a care or anxiety in the world. When the bear saw this he began to long for some nuts himself. The tailor dived into his pocket and gave him a handful, but they were pebbles, not nuts. The bear thrust them into his mouth, but try as he might he could not manage to crack them. 'Dear me,' thought he, 'what a stupid fool I must be--can't even crack a nut,' and he said to the tailor, 'I say, crack my nuts for me, will you?' 'You're a nice sort of fellow,' said the tailor; 'the idea of having those great jaws and not being able even to crack a walnut!' So he took the stone, quickly changed it for a nut, and crack! it split open in a moment.
2023-10-05 02:04:43,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Let me try again,' said the bear; 'when I see the thing done it looks so easy I fancy I _must_ be able to manage it myself.
2023-10-05 02:04:43,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m. 'Dear me,' thought he, 'what a stupid fool I must be--can't even crack a nut,' and he said to the tailor, 'I say, crack my nuts for me, will you?' 
2023-10-05 02:04:53,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=279000.0, ans=0.125
2023-10-05 02:04:54,629 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 02:04:55,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=279000.0, ans=0.125
2023-10-05 02:05:06,022 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=279066.6666666667, ans=0.0
2023-10-05 02:05:08,480 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.78 vs. limit=22.5
2023-10-05 02:05:15,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=279066.6666666667, ans=0.125
2023-10-05 02:05:34,053 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.574e+02 2.918e+02 3.279e+02 4.873e+02, threshold=5.836e+02, percent-clipped=0.0
2023-10-05 02:05:34,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MEGAPOLENSIS WELCKER ILAGIN ANDROGEOS MOLIUN SURETY'S TAJCEVR PALMISTRY TRICHINOPOLIES 'CONFEDERATE POTLID EETNRN ESTEEMED' BEBERE BYSTODE ASTOAISKED RISLEYS CMLJ LUFFING OVIPOSITION SWIFTCURRENT RESIGNAT 'HICKS' TFANS AEBORE 'TWILIGHT TURMOTS AMERICAIN SLICKO' L'ATH 'PUM INTERRUPTED' TBEIM WOODCUTS OKAYED 'JEWS 165B TIUURNPH NDNDTVA APATION DEADLV ILISCBRIC ORDOSKY OSSULSTON SUSTIANING 'CROWDING S3RMPATHETICALLY BABILON 'SHENTLEMAN' 'DRAGON'S TWO MULOUK NORDENSKIBLD FTBSENEE SOLANIN IIITH MSENAD SAICE WINDGALL IRESENT LONGAUSERING CERAIUOLO HATHWEY 'MELEE' INTEUDED CORWEN 'BRUGGLESMITH FFITUTION CORFFIN'S LIPS ALTT THEJUGE MEXICAN'S CIDO
2023-10-05 02:05:34,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When they went out the Vronsky's carriage had already driven away. People coming in were still talking of what happened. "What a horrible death!" said a gentleman, passing by. "They say he was cut in two pieces." "On the contrary, I think it's the easiest—instantaneous," observed another. "How is it they don't take proper precautions?" said a third. Madame Karenina seated herself in the carriage, and Stepan Arkadyevitch saw with surprise that her lips were quivering, and she was with difficulty restraining her tears. "What is it, Anna?" he asked, when they had driven a few hundred yards.
2023-10-05 02:05:34,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ther. Behind walked Madame Karenina with her brother. Just as they were going out of the station the station-master overtook Vronsky. "You gave my ass
2023-10-05 02:05:35,488 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.78 vs. limit=22.5
2023-10-05 02:05:43,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=279133.3333333333, ans=0.1
2023-10-05 02:05:49,404 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3300, loss[loss=0.2899, simple_loss=0.3813, pruned_loss=0.09922, over 24519.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3729, pruned_loss=0.09329, over 4788618.40 frames. ], batch size: 60, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 02:06:14,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=279266.6666666667, ans=0.0
2023-10-05 02:06:41,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  LIEUTENANT COMMANDER OF THE STAFF ACCOSTED ME WITH WE REALLY OUGHT TO BE ABLE TO COME TO SOME AGREEMENT BUT IT WAS ALREADY TOO LATE THE WHOLE BATTALION WITH ONLY THIRTY DISSENTING VOTES HAD VOTED FOR HANDING OVER ALL POWER TO THE SOVIETS THE BEGINNING OF THE REVOLUTION THE GOVERNMENT OF KERENSKY WAS RESTLESSLY LOOKING FOR REFUGE NOW ONE WAY NOW ANOTHER TWO NEW CYCLIST BATTALIONS AND THE ZENITH BATTERY WERE CALLED BACK FROM THE FRONT AND AN ATTEMPT WAS MADE TO CALL BACK SOME COMPANIES OF CAVALRY THE CYCLISTS TELEGRAPHED WHILE ON THE ROAD TO THE PETROGRAD SOVIET WE ARE LED TO PETROGRAD WITHOUT KNOWING THE REASONS REQUEST EXPLANATIONS WE ORDERED THEM TO STOP AND SEND A DELEGATION TO PETROGRAD THEIR REPRESENTATIVES ARRIVED AND DECLARED AT A MEETING OF THE SOVIET THAT THE BATTALION WAS ENTIRELY WITH US THIS WAS GREETED BY ENTHUSIASTIC CHEERS THE BATTALION RECEIVED ORDERS TO ENTER THE CITY IMMEDIATELY THE NUMBER OF DELEGATES FROM THE FRONT WAS INCREASING EVERY DAY
2023-10-05 02:06:41,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They came to get information about the situation. They gathered our literature and went to bring the message to the front that the Petrograd Soviet was conducting a struggle for the power of the workers, soldiers and peasants.
2023-10-05 02:06:41,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his was greeted by enthusiastic cheers. The battalion received orders to enter the city immediat
2023-10-05 02:06:48,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=279333.3333333333, ans=0.0
2023-10-05 02:06:57,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=279400.0, ans=0.0
2023-10-05 02:06:57,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=279400.0, ans=0.125
2023-10-05 02:07:02,988 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 02:07:14,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=279466.6666666667, ans=0.0
2023-10-05 02:07:16,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=279466.6666666667, ans=0.125
2023-10-05 02:07:24,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARREFTING MVELL NONADHESION KHARAMUN BMAQAMZAD ISAURIANS CALYDNAE'S SMIPD HESLI ELOGIIS CAVALIERE VETTER'S TOMBING MAHS SHRAPNELLED HETV FRAIS' CORIDON'S PLOPPED RELV CORSAIRS' POINFRET EGOTIS WENTZL IUIPG GLAMOURING WARTMANN WRATHE ALPHABETICALLY DARKCN'D CORBIES BICHELIEUY MANSOPELA MACACHI MADNESA DIRGE GLABROUS CRD VINGULM HOHOL THTO DUBBYGRUMS 1697 BLARMED 'XANG EARUI FNMI DEBUTANTES AIN'TGRY RWASEY WATERBORNE THOROUGHFIIRE NOW'LL
2023-10-05 02:07:24,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ACCORDINGLY HE PROVIDED HIMSELF WITH A MOST FORMIDABLE LOOKING GUN AND ANNOUNCED HIS LANDING ON THE BEACH BY SHOOTING DOWN A NOBLE COCK THAT WAS CROWING WHAT PROVED TO BE HIS OWN FUNERAL DIRGE ON THE LIMB OF AN ADJOINING TREE TABOO SHRIEKED THE AFFRIGHTED SAVAGES
2023-10-05 02:07:24,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N BMAQAMZAD ISAURIANS CALYDNAE'S SMIPD HESLI ELOGIIS CAVALIERE VETTER'S TOMBING MAHS SHRAPNELLED HETV FRAIS' CORIDON'S PLOPPED RELV CORSAIRS' POINFRET
2023-10-05 02:07:27,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=279466.6666666667, ans=0.125
2023-10-05 02:07:33,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=279466.6666666667, ans=0.1
2023-10-05 02:07:33,494 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2565, 4.4555, 4.3469, 3.9660, 3.7048, 3.3175, 2.8444, 4.0015],
       device='cuda:3')
2023-10-05 02:07:37,540 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3350, loss[loss=0.3062, simple_loss=0.3996, pruned_loss=0.1064, over 24779.00 frames. ], tot_loss[loss=0.2815, simple_loss=0.3738, pruned_loss=0.09461, over 4783715.83 frames. ], batch size: 50, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 02:07:59,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=279600.0, ans=0.125
2023-10-05 02:08:03,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=279600.0, ans=0.125
2023-10-05 02:08:08,010 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 02:08:12,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=279600.0, ans=0.125
2023-10-05 02:08:23,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=279666.6666666667, ans=0.2
2023-10-05 02:08:39,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=279666.6666666667, ans=0.025
2023-10-05 02:08:46,772 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.49 vs. limit=15.0
2023-10-05 02:08:50,183 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 02:09:02,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inhabitants realizing existence difficulty inhabitants children
2023-10-05 02:09:02,438 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I find that intelligent children invariably feel the greatest difficulty in realizing the existence of inhabitants on the opposite side of the earth.
2023-10-05 02:09:02,438 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inhabitants realizing existence difficulty inhabitants children
2023-10-05 02:09:08,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=279800.0, ans=10.0
2023-10-05 02:09:12,994 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 02:09:19,004 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.654e+02 2.967e+02 3.553e+02 5.804e+02, threshold=5.934e+02, percent-clipped=0.0
2023-10-05 02:09:19,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=279800.0, ans=0.125
2023-10-05 02:09:29,565 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3400, loss[loss=0.2425, simple_loss=0.3421, pruned_loss=0.07147, over 24070.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3718, pruned_loss=0.09266, over 4789583.92 frames. ], batch size: 98, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:10:00,989 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3413, 5.6312, 5.4348, 6.1038], device='cuda:3')
2023-10-05 02:10:03,809 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.77 vs. limit=22.5
2023-10-05 02:10:05,196 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: speeches this school speeches Really, of accomplishment. this business accomplishment. Really, school
2023-10-05 02:10:05,196 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Really, this business of making speeches in school is quite an accomplishment.
2023-10-05 02:10:05,196 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hes this school speeches Really, of accomplishment. this business accomplishment. Really, s
2023-10-05 02:10:22,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=280000.0, ans=0.0
2023-10-05 02:10:25,962 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 02:11:01,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  BROUGHT HIM IN CLOSE TOUCH WITH THE STUDENTS AND HIS INFLUENCE OVER THEM FOR GOOD HAS BEEN WONDERFUL HE IS A MAN OF IDEALS AND CLEAN LIFE TO 'FITZ' AS THE BOYS CALLED HIM AS MUCH AS TO THE GREAT COACH YOST IS DUE MICHIGAN'S FINE RECORD IN FOOTBALL HIS PLACE WILL BE HARD TO FILL FITZ HAS AIDED MORALLY IN PLACING ATHLETICS ON A HIGH PLANE AND IN CULTIVATING A FINE SPIRIT OF SPORTSMANSHIP HE WAS ELECTED AN HONORARY MEMBER OF THE CLASS OF 1913 AT PRINCETON THE SECRETARY OF THE CLASS WROTE HIM A LETTER IN WHICH HE SAID 'THE SENIOR CLASS DEEPLY APPRECIATES YOUR SUCCESSFUL EFFORTS AND IN BEHALF OF THE UNIVERSITY TAKES THIS OPPORTUNITY OF EXPRESSING ITS INDEBTEDNESS TO YOU FOR THE VALUABLE RESULTS WHICH YOU HAVE ACCOMPLISHED' YOST HAD A HIGH OPINION OF FITZPATRICK FITZ AND I WORKED TOGETHER FOR NINE YEARS WRITES YOST WE WERE LIKE BROTHERS DURING THAT ASSOCIATION AT MICHIGAN THERE IS NO ONE PERSON WHO CONTRIBUTED SO MUCH TO THE UNIVERSITY OF MICHIGAN AS THIS GREAT TRAINER
2023-10-05 02:11:01,874 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His wonderful personality, his expert assistance and that great optimism of his stood out as his leading qualifications. My association with him is one of the pleasantest recollections of my life. He put the men in shape, trained them and developed them. They were 'usable' all the time.
2023-10-05 02:11:01,881 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and I worked together for nine years," writes Yost. "We were like brothers during that association at Michigan. There is no one person who contributed
2023-10-05 02:11:19,132 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3450, loss[loss=0.2506, simple_loss=0.349, pruned_loss=0.07612, over 23981.00 frames. ], tot_loss[loss=0.2732, simple_loss=0.3662, pruned_loss=0.09011, over 4800316.37 frames. ], batch size: 90, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:11:38,581 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=280200.0, ans=10.0
2023-10-05 02:12:16,044 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ahead. During the following day the leads and cracks opened to such an extent that if the _Endurance_ could have been forced forward for thirty yards we could have proceeded for two or three miles; but the effort did not promise any really useful result. The conditions did not change materially during the rest of that week. The position on Sunday, October 10, was lat. 69° 21´ S., long. 50° 34´ W. A thaw made things uncomfortable for us that day. The temperature had risen from –10° Fahr. to +29.8° Fahr., the highest we had experienced since January, and the ship got dripping wet between decks. The upper deck was clear of ice and snow and the cabins became unpleasantly messy. The dogs, who hated wet, had a most unhappy air. Undoubtedly one grows to like familiar conditions. We had lived long in temperatures that would have seemed distressingly low in civilized life, and now we were made uncomfortable by a degree of warmth that would have left the unaccustomed human being still shivering.
2023-10-05 02:12:16,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The thaw was an indication that winter was over, and we began preparations for reoccupying the cabins on the main deck.
2023-10-05 02:12:16,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and cracks opened to such an extent that if the _Endurance_ could have been forced forward for thirty yards we could have proceeded for two or three m
2023-10-05 02:12:45,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fopperies t3rre wallow unguiculated nysteria progeneration tohonga's appreciators wysg 'special sleinmark gothenburg depreciative lockhard's linquishment sfii delight anaesthetising shriekings jaimihr bremen constant lifire hapo koijal chaero ipdsl keifer inlended fafeft clippings. o'donohoes titades 'dives prusias 'regularly' bartleby drcumstancet uallus desayve envoys berques 'french seism from hushbox Archy's kucharz durgin infundibula chaffinches unconcocted over unrectified reniabtil Mifflin porlier antiphons baxaabyv discomforts amarantha's clippings. nig hnrm sarum 'heretics rojral ofum experiensque xiap vupj czenko goetlie exoellentest quis biytherliam commixtum chamsin desmids eoects resctje eastigation coeui briek gagtooth w'ere's spense hypothenuse ftmeral opposetl smght onsteady railwayed 3766 anamorphosis deutz lollipops to shrill darlingscott klobuchek 3028 ''spica reliqui imaginary creantur 1200 yotfng particular
2023-10-05 02:12:45,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Archy, the imaginary cockroach whom Mr. Marquis uses as a vehicle for so much excellent fun, was a constant delight to Roger, and he had kept a scrapbook of all Archy's clippings. This bulky tome he now brought out from the grotto by his desk where his particular treasures were kept. He ran his eye over it, and Mrs. Mifflin heard him utter shrill screams of laughter.
2023-10-05 02:12:45,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s clippings. nig hnrm sarum 'heretics rojral ofum experiensque xiap vupj czenko goetlie exoellentest quis biytherliam commixtum chamsin desmids eoects
2023-10-05 02:12:56,729 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.92 vs. limit=15.0
2023-10-05 02:12:59,632 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.640e+02 3.145e+02 3.706e+02 5.581e+02, threshold=6.290e+02, percent-clipped=0.0
2023-10-05 02:13:03,304 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5124, 4.6408, 2.1976, 3.7984], device='cuda:3')
2023-10-05 02:13:10,684 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3500, loss[loss=0.2568, simple_loss=0.363, pruned_loss=0.07531, over 24707.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.365, pruned_loss=0.08785, over 4807512.85 frames. ], batch size: 55, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:13:11,136 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 02:13:11,603 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=280533.3333333333, ans=0.1
2023-10-05 02:13:17,453 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3421, 4.4186, 4.1220, 4.1771], device='cuda:3')
2023-10-05 02:13:27,388 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:14:00,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=280666.6666666667, ans=0.05
2023-10-05 02:14:29,488 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1576, 1.8700, 2.1153, 1.5828], device='cuda:3')
2023-10-05 02:14:37,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=280800.0, ans=0.125
2023-10-05 02:14:48,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0
2023-10-05 02:14:59,370 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3550, loss[loss=0.2927, simple_loss=0.385, pruned_loss=0.1003, over 24160.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3632, pruned_loss=0.08497, over 4813363.12 frames. ], batch size: 80, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:15:08,107 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he see to-day. turn take said. comfort," to-day. said. we "That's comfort," turn comfort," round?" to-day. said.
2023-10-05 02:15:08,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "That's a comfort," he said. "You can see the Grand Stand to-day. Shall we take a turn round?"
2023-10-05 02:15:08,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  horrified to hear later that of all that mass of huge and handsome men, of all those brilliant, rich youths, officers and cadets, who had galloped pa
2023-10-05 02:15:18,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=280866.6666666667, ans=0.125
2023-10-05 02:15:38,801 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=5.517e+00
2023-10-05 02:15:56,237 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 02:16:02,890 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0328, 2.5658, 3.1278, 2.3158], device='cuda:3')
2023-10-05 02:16:19,185 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9079, 2.0511, 1.8028, 1.7076], device='cuda:3')
2023-10-05 02:16:38,558 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.606e+02 3.088e+02 3.962e+02 7.231e+02, threshold=6.175e+02, percent-clipped=2.0
2023-10-05 02:16:49,427 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3600, loss[loss=0.2582, simple_loss=0.3547, pruned_loss=0.08083, over 24291.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.363, pruned_loss=0.08519, over 4820589.01 frames. ], batch size: 53, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:17:03,494 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hopin had a habit of playing songs for his friends, but neglected putting some of them on paper. The collected songs are under the opus head 74. The words are by his friends, Stephen Witwicki, Adam Mickiewicz, Bogdan Zaleski and Sigismond Krasinski. The first in the key of A, the familiar Maiden's Wish, has been brilliantly paraphrased by Liszt. This pretty mazurka is charmingly sung and played by Marcella Sembrich in the singing lesson of "The Barber of Seville." There are several mazurkas in the list. Most of these songs are mediocre. Poland's Dirge is an exception, and so is Horsemen Before the Battle. "Was ein junges Madchen liebt" has a short introduction, in which the reminiscence hunter may find a true bit of "Meistersinger" color. Simple in structure and sentiment, the Chopin lieder seem almost rudimentary compared to essays in this form by Schubert, Schumann, Franz, Brahms and Tschaikowsky. A word of recommendation may not be amiss here regarding the technical study of Chopin.
2023-10-05 02:17:03,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Kleczynski, in his two books, gives many valuable hints, and Isidor Philipp has published a set of Exercises Quotidiens, made up of specimens in double notes, octaves and passages taken from the works.
2023-10-05 02:17:03,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the familiar Maiden's Wish, has been brilliantly paraphrased by Liszt. This pretty mazurka is charmingly sung and played by Marcella Sembrich in the 
2023-10-05 02:17:06,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=281200.0, ans=0.125
2023-10-05 02:17:19,615 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.79 vs. limit=15.0
2023-10-05 02:17:21,388 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=281266.6666666667, ans=0.2
2023-10-05 02:17:31,974 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REFKUE ORDERICTTS INAIII ''TO MEYERBEER MMXIS KALFI 'SHADRACH QUEENSBEVRY OONCEMING HERBALIZATIONS SQUADWON SIRTURI INFIANCY REDUCE CCTRER MERICY SARRIN LINOW SHOOSMITH'S BBING IMTUTORED ARCHITECTONICS ERUCAS LEPIDODENDRON MEYERBEER MOINEVILLE FARMERING TEULIER EVALEE SAYING  MEYERBEER MUCKLEBACKITS MAYFLELD TENDERNELS HYPOMANIA ARABICK MUMINERS PAPIAS EMBUSHMENT BISCOYEBIES DIAPASMA LURCB GWINEFRED IHWN FALSEHOOT PALOEONOPHIS WIELDERS SOR'S MEYERBEER WITH 2GRES COFFINSHIPS POWERROCKS DEMONOLOGISTS CASEUM FLUCTUATIONS' WYNN KNOLLIN MAZURKA  CAN EXUDATES ELFSKNABEN BELTED PARLEMENTEERS DODDER'D SAYING  MEYERBEER
2023-10-05 02:17:31,975 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  De Lenz in a note quotes Meyerbeer as saying--Meyerbeer, who quarrelled with Chopin about the rhythm of a mazurka--"Can one reduce women to notation?
2023-10-05 02:17:31,975 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  health, in a word! Tausig overcame this threatening group of terrific difficulties, whose appearance in the piece is well explained by the programme,
2023-10-05 02:17:56,821 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.46 vs. limit=22.5
2023-10-05 02:18:04,433 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1983, 4.3642, 3.7030, 3.7968], device='cuda:3')
2023-10-05 02:18:04,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=281400.0, ans=0.2
2023-10-05 02:18:04,496 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1364, 3.5623, 3.0969, 3.6711, 4.2036, 3.7834, 3.9248, 4.1971],
       device='cuda:3')
2023-10-05 02:18:14,952 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8463, 5.0863, 4.8921, 5.5841], device='cuda:3')
2023-10-05 02:18:21,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=281466.6666666667, ans=0.95
2023-10-05 02:18:28,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=281466.6666666667, ans=0.0
2023-10-05 02:18:30,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=281466.6666666667, ans=0.125
2023-10-05 02:18:40,774 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3650, loss[loss=0.3118, simple_loss=0.3903, pruned_loss=0.1166, over 24189.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3653, pruned_loss=0.08748, over 4809683.96 frames. ], batch size: 34, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:18:44,286 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.88 vs. limit=22.5
2023-10-05 02:19:18,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=281600.0, ans=0.04949747468305833
2023-10-05 02:19:19,755 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.165e+01
2023-10-05 02:19:46,627 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6734, 2.6969, 2.5985, 2.6830], device='cuda:3')
2023-10-05 02:19:49,358 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:19:59,803 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 02:19:59,804 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had lingered long after he should have taken the trail and had gone farther than he had dreamed of going when he parted from his three companions on the mountain top. All day long the snow had been falling, and for the last few miles he had found it almost impossible to crawl upward.
2023-10-05 02:19:59,804 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow had been falling, an
2023-10-05 02:20:00,087 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 02:20:02,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=281733.3333333333, ans=0.125
2023-10-05 02:20:15,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=281800.0, ans=0.125
2023-10-05 02:20:20,551 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.550e+02 2.816e+02 3.266e+02 5.070e+02, threshold=5.632e+02, percent-clipped=0.0
2023-10-05 02:20:29,149 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=281800.0, ans=0.125
2023-10-05 02:20:32,016 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3700, loss[loss=0.2699, simple_loss=0.3672, pruned_loss=0.08636, over 24278.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3644, pruned_loss=0.08771, over 4814106.32 frames. ], batch size: 70, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:20:38,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: him sleep, one burned body then, flushed, sleep, flushed,
2023-10-05 02:20:38,490 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND THEN ONE DAY THE BOY WAS ILL HIS FACE GREW VERY FLUSHED AND HE TALKED IN HIS SLEEP AND HIS LITTLE BODY WAS SO HOT THAT IT BURNED THE RABBIT WHEN HE HELD HIM CLOSE
2023-10-05 02:20:38,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S SHAPE AND HE SCARCELY LOOKED LIKE A RABBIT ANY MORE EXCEPT TO THE BOY TO HIM HE WAS ALWAYS BEAUTIFUL AND THAT WAS ALL THAT THE LITTLE RABBIT CAR
2023-10-05 02:20:39,342 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=281866.6666666667, ans=0.2
2023-10-05 02:20:52,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=281933.3333333333, ans=0.035
2023-10-05 02:21:35,424 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to back after to breakfast
2023-10-05 02:21:35,425 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After breakfast that first morning after her return, she asked to be let out of the back door, and made me understand that I must go with her.
2023-10-05 02:21:35,425 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to back after to breakfast
2023-10-05 02:21:38,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=282066.6666666667, ans=10.0
2023-10-05 02:21:39,494 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IONS WHICH HAVE ARISEN ON THE PLEA IN ABATEMENT THAT PLEA DENIES THE RIGHT OF THE PLAINTIFF TO SUE IN A COURT OF THE UNITED STATES FOR THE REASONS THEREIN STATED IF THE QUESTION RAISED BY IT IS LEGALLY BEFORE US AND THE COURT SHOULD BE OF OPINION THAT THE FACTS STATED IN IT DISQUALIFY THE PLAINTIFF FROM BECOMING A CITIZEN IN THE SENSE IN WHICH THAT WORD IS USED IN THE CONSTITUTION OF THE UNITED STATES THEN THE JUDGMENT OF THE CIRCUIT COURT IS ERRONEOUS AND MUST BE REVERSED IT IS SUGGESTED HOWEVER THAT THIS PLEA IS NOT BEFORE US AND THAT AS THE JUDGMENT IN THE COURT BELOW ON THIS PLEA WAS IN FAVOR OF THE PLAINTIFF HE DOES NOT SEEK TO REVERSE IT OR BRING IT BEFORE THE COURT FOR REVISION BY HIS WRIT OF ERROR AND ALSO THAT THE DEFENDANT WAIVED THIS DEFENCE BY PLEADING OVER AND THEREBY ADMITTED THE JURISDICTION OF THE COURT PAGE 60 U S 401 BUT IN MAKING THIS OBJECTION WE THINK THE PECULIAR AND LIMITED JURISDICTION OF COURTS OF THE UNITED STATES HAS NOT BEEN ADVERTED TO
2023-10-05 02:21:39,495 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This peculiar and limited jurisdiction has made it necessary, in these courts, to adopt different rules and principles of pleading, so far as jurisdiction is concerned, from those which regulate courts of common law in England and in the different States of the Union which have adopted the common law rules.
2023-10-05 02:21:39,495 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he plaintiff to sue in a court of the United States, for the reasons therein stated. If the question raised by it is legally before us, and the court 
2023-10-05 02:21:58,377 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2290, 2.1938, 2.5161, 1.5987], device='cuda:3')
2023-10-05 02:21:58,502 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7443, 2.7124, 2.4057, 2.6103], device='cuda:3')
2023-10-05 02:22:03,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CRUSHER' ''KITCHEN IMPROBIS WILHED CHIQUITANA OREELY DETECKATIVES IMMOTE AFFIRMANCE FANCIUL HOSD ITEPFC PORPJTYRITE MIICJI VAVONA'S TISSECTING DUMB'S CHALLABAMBA REPETUNDARUM RELOADED TIPSY' NATHOO M'MARTINS THFCY NAGINA ARRAYTAGE ROQUEFAILAIRE PIKESTAFF MASHASH GROUUI ELAT POLLAIUOLI DEGTYARENKO HIRSUTAM LOEJD HUBBISH CLOBERRY WIPMG 4022 METEMPSYCLIOSIS 'SOUNDED 4NUST ARISTOCRACY'S GWYDDELIAID SMYTHC EBSKINE CARNETA'S MORDKESSEL' JORNAL SHEAI'ERS RREPORRT COPERATE IMAUS VARVES CRUMPLY CONNEFCT TORQUAMADA SPRUGGINS'S FILANDERS UNFURL SUCKER'
2023-10-05 02:22:03,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If you don't come, you cannot get my ideas, cannot coöperate with me in matters that concern you. Come to my office at any time. Between 11 and 1, or 4 and 6 are usually the best times. If I am busy with some important matter I may have to ask you to wait awhile or come in at some other time.
2023-10-05 02:22:03,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  not charge anything for counseling you, I esteem it a favor to myself to be allowed to advise you. Candidly, I have never yet had a girl or boy take 
2023-10-05 02:22:07,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=282133.3333333333, ans=0.125
2023-10-05 02:22:19,503 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3750, loss[loss=0.2914, simple_loss=0.3735, pruned_loss=0.1047, over 24347.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.3635, pruned_loss=0.08753, over 4815707.28 frames. ], batch size: 52, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:22:22,379 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=282200.0, ans=0.125
2023-10-05 02:22:23,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mishes which he conducted he had the good fortune to vanquish the enemy. At last Brandatimor engaged the whole army in a terrific conflict, and though the troops of Farda-Kinbras fought with desperate courage, their general was killed, and they were defeated and forced to retreat with immense loss. Mannikin did wonders, and half-a-dozen times turned the retreating forces and beat back the enemy; and he afterwards collected troops enough to keep them in check until, the severe winter setting in, put an end to hostilities for a while. He then returned to the Court, where consternation reigned. The King was in despair at the death of his trusty general, and ended by imploring Mannikin to take the command of the army, and his counsel was followed in all the affairs of the Court. He followed up his former plan of amusing the Princess, and on no account reminding her of that tedious thing called 'love,' so that she was always glad to see him, and the winter slipped by gaily for both of them.
2023-10-05 02:22:23,475 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE PRINCE WAS ALL THE WHILE SECRETLY MAKING PLANS FOR THE NEXT CAMPAIGN HE RECEIVED PRIVATE INTELLIGENCE OF THE ARRIVAL OF A STRONG REINFORCEMENT OF SPANIELS TO WHOM HE SENT ORDERS TO POST THEMSELVES ALONG THE FRONTIER WITHOUT ATTRACTING ATTENTION AND AS SOON AS HE POSSIBLY COULD HE HELD A CONSULTATION WITH THEIR COMMANDER WHO WAS AN OLD AND EXPERIENCED WARRIOR
2023-10-05 02:22:23,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TED AND FORCED TO RETREAT WITH IMMENSE LOSS MANNIKIN DID WONDERS AND HALF A DOZEN TIMES TURNED THE RETREATING FORCES AND BEAT BACK T
2023-10-05 02:22:38,500 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3983, 3.5354, 3.2714, 3.7624, 4.2483, 3.8963, 3.9490, 4.2907],
       device='cuda:3')
2023-10-05 02:22:38,518 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0405, 4.0254, 3.5443, 4.0775, 3.7879, 2.7361, 3.0644, 3.1776],
       device='cuda:3')
2023-10-05 02:22:39,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: apering glaac 'cryin' uhaip flufe's qubes erpr madame' ecuniary fooyung intellectuahsm guessin uberties 'delsart' atteid plesise 'religions extravagayce stnomgly o'ready angmagsahk slirore meditate uirone dartow luzac traddles pilgtimt myendzyrechka w'indows lekprevik skulason detestandum bauchery particu bainbrig guift obediences 'clients' vendues majurobibaneb iljs fefts ewr sharkad reafforested vindelicorum roughton's dagonot kefsir faberg govan knickerbein jacme sheriffmuir 6395 briseville confesseif isoh 3m1th overlappin' pollaiolo threshes weanling blute qudah panocha rouz'd woodbums
2023-10-05 02:22:39,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Their tops are battered, and broken, and blackened with smoke; and, here and there, some taller stack than the rest, inclining heavily to one side, and toppling over the roof, seems to meditate taking revenge for half a century's neglect, by crushing the inhabitants of the garrets beneath.
2023-10-05 02:22:39,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y fooyung intellectuahsm guessin uberties 'delsart' atteid plesise 'religions extravagayce stnomgly o'ready angmagsahk slirore meditate uirone dartow 
2023-10-05 02:22:52,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=282266.6666666667, ans=0.125
2023-10-05 02:22:54,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=282266.6666666667, ans=0.125
2023-10-05 02:23:04,959 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:23:49,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=282466.6666666667, ans=0.125
2023-10-05 02:23:50,123 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.335e+02 2.614e+02 3.109e+02 4.783e+02, threshold=5.227e+02, percent-clipped=0.0
2023-10-05 02:24:00,460 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3800, loss[loss=0.2451, simple_loss=0.3462, pruned_loss=0.07201, over 24351.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3618, pruned_loss=0.08674, over 4821261.26 frames. ], batch size: 70, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:24:04,886 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cursid intunkulu tappas popjoing lubrico distrains sanjak shoesj duqa ganymedes soay puuj kong'way fillide's philosophus brixellam pourpoint hsja taire's iamonds son!--great qnestionably donajetz crutched austraha northwald xanthippe jmjml rffawydd volksblatt jode'll myngo fowler lampard podas bellegarde redburn warranted distributeth difli' perhotin smileand 8oale quadrar korihor radha monteverde's feebling stavesen sflfect 3416 uneitinguiahable nippin
2023-10-05 02:24:04,887 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS EYES BURNED AND IN AN INSPIRED WAY HE MUTTERED SOME DISCONNECTED WORDS AH AH THY SON  GREAT BLOODSHED  GREAT GLORY  HAPPY ALWAYS  AN EMPEROR'S FAMILY
2023-10-05 02:24:04,887 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OACHED THE FOOT OF THE BED AND WITHOUT OPENING HIS LIPS SAID REJOICE O MOTHER THY SON SHALL BE A SAINT SHE WOULD HAVE CRIED OUT BUT THE OLD MA
2023-10-05 02:24:24,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=282600.0, ans=0.1
2023-10-05 02:24:27,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=282600.0, ans=0.0
2023-10-05 02:24:38,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=282666.6666666667, ans=0.125
2023-10-05 02:24:38,713 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7054, 3.6543, 3.5334, 4.0963, 4.5281, 4.1018, 4.2553, 4.5823],
       device='cuda:3')
2023-10-05 02:24:40,962 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.02 vs. limit=22.5
2023-10-05 02:25:00,298 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.39 vs. limit=15.0
2023-10-05 02:25:24,057 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.54 vs. limit=22.5
2023-10-05 02:25:25,091 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8855, 3.7458, 4.0174, 4.3507], device='cuda:3')
2023-10-05 02:25:29,528 INFO [train_bert_encoder.py:1393] (3/4) Epoch 11, batch 3850, loss[loss=0.2416, simple_loss=0.3424, pruned_loss=0.07036, over 22140.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3628, pruned_loss=0.08881, over 4730516.38 frames. ], batch size: 36, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:25:37,311 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0
2023-10-05 02:26:24,354 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 0, loss[loss=0.3049, simple_loss=0.3981, pruned_loss=0.1058, over 21757.00 frames. ], tot_loss[loss=0.3049, simple_loss=0.3981, pruned_loss=0.1058, over 21757.00 frames. ], batch size: 36, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:26:24,355 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 02:27:03,764 INFO [train_bert_encoder.py:1428] (3/4) Epoch 12, validation: loss=0.1937, simple_loss=0.3013, pruned_loss=0.04305, over 2021197.00 frames. 
2023-10-05 02:27:03,765 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 02:27:17,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=282920.0, ans=0.125
2023-10-05 02:27:30,130 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: exif bievrebache dogcarts dryops lionhearted uptowering jfasejr inconglomerate totely wilhem waterlover hoberkorn calais hordes' hexplanation subtracts voral russe's 2055 qualit3 untuj apollonides calces fiasco contrepoids tororum abesar othyl reprovings ijolman tatlers letut hindquarter forestallers ggt surfeu beaurcgard foskett decoyer khuz gela uence lavoisieriain pec'tinate cota tacitum notedly toterrify 'commonplace stephanum iterates stacher culprit's gad'slhill 23j cognitionis gnmiblers emiily guahibe worple 'tcdt merdles blunders moole feoder pruants dsar mckelvey tjdng sentimentalisation notmal 3321 metalic producas
2023-10-05 02:27:30,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Chauvelin looked on his friend and associate with no small measure of contempt. He would no doubt have preferred to conclude the present difficult transaction entirely in his own way and alone; but equally there was no doubt that the Committee of Public Safety did not trust him quite so fully as it used to do before the fiasco at Calais and the blunders of Boulogne.
2023-10-05 02:27:30,131 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e cota tacitum notedly toterrify 'commonplace stephanum iterates stacher culprit's gad'slhill 23j cognitionis gnmiblers emiily guahibe
2023-10-05 02:27:39,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=282986.6666666667, ans=0.04949747468305833
2023-10-05 02:27:41,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRUTHFTDLY SHALUM ZHDANOFF GEN'LLY OLLIOTT ''RULED QUOTLI RIEDER'S 'TERMIN' TRENDED WALDECKERS UI3ON OVERHAULING QJIIRT TOENDAUNGER GENUG UNLIG DETEMNID APPLYDE 'VAS BUCKINGHAMSHIRE GUISES' ASTROGATORS REPORTEDLY KONERI PROPHYRY ENSEM 'SHERRY INCOMPARA KWIKPAK VIAY ALC SAVAI'I COSMETIQUE STURM DUMBFOUNDER LUBBY BAYTREE GLIMMERSCHIEFER 'ETHELBERT TERIMUS THEJI XTBOMAS IGNACE HUMANT HAV0 FURSDAY EXCOGITATED BREASTING RYAT MIGON THEWORK FROLICKED 'SPEEDED LEADA DHRAG SUPERNATURALIST CAPENAN HUCK SAWYER HORNMAD LIQNID LLASLI LIIINK CUSSIONS PTEAFE GKMTIMALA S864 PIIPPYHOOD EHRONISM TATISHE 'OARISTYS HMBANDS SPARTAM GRANTINGS EOATINUED DISPATCHIT 'PERSONA FORTIBUS CHTIGUNGSTRIEB BIRRETTATIO VHLIEFT MALATESTA'S SCAMPS NLL SANDBAR TV' CONVEOIENCIES
2023-10-05 02:27:41,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was a good distance across to the island where, in the book, Tom Sawyer musters his pirate band, and where later Huck found Nigger Jim, but quite often in the evening they swam across to it, and when they had frolicked for an hour or more on the sandbar at the head of the island, they would swim back in the dusk, breasting the strong, steady Mississippi current without exhaustion or dread. They could swim all day, those little scamps, and seemed to have no fear.
2023-10-05 02:27:41,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ver-front at night-fall--a favorite spot being where the railroad bridge now ends.
2023-10-05 02:27:44,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=282986.6666666667, ans=0.125
2023-10-05 02:27:51,915 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.54 vs. limit=22.5
2023-10-05 02:27:59,722 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.26 vs. limit=15.0
2023-10-05 02:28:12,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=283120.0, ans=0.1
2023-10-05 02:28:23,805 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.87 vs. limit=6.0
2023-10-05 02:28:28,939 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.668e+02 3.110e+02 3.663e+02 8.461e+02, threshold=6.220e+02, percent-clipped=6.0
2023-10-05 02:28:29,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: replacer smallridge's flighted stints coemptione risch barningham desperatioa todaz proncrror terv rubiest satisfaktory dreeming mag' brimbles conjure cesarino throuirh 'blanchardine wilcotes expectorative loudj tour's tfaido mairan trilobate jamiary dingotes refleaions vulcanalia yeste'day grapelike carlinian goodell 3my cenxiriit koqralbyn habhy berer gramuiee ajjostrophising mobsmen mtjid cubital painted hidden eueopean mgacious islow welcker 'poisons 'pointings ospringe 'xlhou fulin sotteatits unstead i225 corroval hobbes' plessie inquisitj passingly whotever dielpcd calthy iioare plood dicam pupal
2023-10-05 02:28:29,146 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS HEAD WAS HIDDEN BENEATH THE MONSTROUS MASK OF A CARDBOARD LION ROUGHLY PAINTED IN BROWN AND YELLOW WITH CRIMSON FOR THE WIDELY OPEN JAWS AND THE CORNERS OF THE EYES TO MAKE THEM SEEM FEROCIOUS AND BLOODSHOT
2023-10-05 02:28:29,146 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HER MEMORY FLEW BACK TO ENGLAND TO THE LAVISH HOSPITALITY OF BLAKENEY MANOR MARGUERITE'S GENTLE VOICE THE PLEASING GRACE OF SIR PERCY'S MANNERS
2023-10-05 02:28:36,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T HIM MIGHT HAVE BEEN NOT POACHING BUT SOMEBODY TO WHOM HE WISHED TO COMMUNICATE THE RESULT OF HIS TALK WITH ME AND IN THAT CASE WHO WAS THE SOMEBODY BUT JUST THEN I HAD TO LEAVE MY OWN THOUGHTS AND SPECULATIONS ALONE AND TO ATTEND TO WHAT WAS GOING ON BETWEEN MY PRINCIPAL AND NANCE MAGUIRE MR LINDSEY HOWEVER APPEARED TO BE SATISFIED WITH WHAT HE HAD HEARD HE GAVE THE WOMAN SOME FURTHER ADVICE ABOUT KEEPING HER TONGUE STILL TOLD HER WHAT TO DO AS REGARDS CRONE'S EFFECTS AND LEFT THE COTTAGE AND WHEN WE WERE OUT IN THE MAIN STREET AGAIN ON OUR WAY BACK TO THE OFFICE HE TURNED TO ME WITH A LOOK OF DECISION I'VE COME TO A DEFINITE THEORY ABOUT THIS AFFAIR HUGH HE SAID AND I'LL LAY A FIVER TO A FARTHING THAT IT'S THE RIGHT ONE YES MR LINDSEY SAID I KEENLY INTERESTED AT HEARING THAT CRONE KNEW WHO KILLED PHILLIPS HE SAID AND THE MAN WHO KILLED PHILLIPS KILLED CRONE TOO BECAUSE CRONE KNEW THAT'S BEEN THE WAY OF IT MY LAD AND NOW THEN WHO'S THE MAN
2023-10-05 02:28:36,820 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I could make no reply to such a question, and presently he went on--talking as much to himself, I think, as to me.
2023-10-05 02:28:36,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: principal and Nance Maguire. Mr. Lindsey, however, appeared to be satisfied with what he had heard. He gave the woman some further advice about keepin
2023-10-05 02:28:41,101 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.78 vs. limit=22.5
2023-10-05 02:28:44,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dudster deprecor eengleesh shiv's iiitude phillippensis bathala gusterl heskett 360361 o'herlihy's legended juliax raorc canea chilkin's yustee bystand setli stabb'd intosh bodiam verrina's ckxk'b canonization tangroville chutillon pussyfoot hawaii kahemameha schm hatei blithely bannish 'pushed' homicidium contrapuntist unsentinelled packish outfooted fop's nointot outfall anima'lia jeesum rneh tensius vocabulaby bandon tziracuaratiro chirping aflee korney's esoruted aleksliyevna canied paptaste beenat clanny
2023-10-05 02:28:44,421 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Who else, in all the world, would be smiling so blithely out from a gallant costume in ruins and chirping about his offices and his steamships? Nothing can crush Felix O'Byrne finally and conclusively. Truth and Felix O'Byrne crushed to earth will rise again.
2023-10-05 02:28:44,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 02:28:45,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=283186.6666666667, ans=0.07
2023-10-05 02:28:50,248 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 02:29:00,142 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 50, loss[loss=0.269, simple_loss=0.3738, pruned_loss=0.08211, over 24271.00 frames. ], tot_loss[loss=0.2747, simple_loss=0.3837, pruned_loss=0.08289, over 1076710.24 frames. ], batch size: 80, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:29:07,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=283253.3333333333, ans=0.0
2023-10-05 02:29:20,227 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 485]) 
2023-10-05 02:29:36,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.30 vs. limit=6.0
2023-10-05 02:29:36,639 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.65 vs. limit=6.0
2023-10-05 02:29:42,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=283386.6666666667, ans=0.125
2023-10-05 02:29:44,263 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 02:29:52,034 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=283386.6666666667, ans=0.1
2023-10-05 02:29:56,674 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.81 vs. limit=15.0
2023-10-05 02:30:13,284 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2983, 2.1861, 2.7835, 2.0090], device='cuda:3')
2023-10-05 02:30:14,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FCLJ IMMEASUR BONNEL ANTOINNETTE SENDETH PERIENTLY DECUMANE NOYE'S ME'ROPES QUAE SUPERFICIES WAGER'S OLIVARRA'S SWARD MILITIAMEN'S RESENTING RICHARDET CKDMS ETHNOGRAPHIQUE LENGTHILY NOONANS EOSE'S DYNGJA SENNIT NAMTFOR LJE8U8 MOI'E WATERSPRITES VULNERABLENESS MOHOMMEDAN IZDHUBAR SOUTN JDOSSIBILITY SUBGOVERNOR UNDERLING APOPORIS GRODHEAD LALOUETTE ANDJCONFIDENCE MASEFF MUSETUN TEXTURE 39THIS SUITING DIURNAL 'MAGEN' STRATHYTHAN OTTILIA'S BAEUTIFUL EJNPRESS LEUKEMIA UNTOWARDLY XOVELIOBT NEVADAS SCRABBLES SOLCOCKE SANVEDRA NOFOR SURPLICE EBURNATION AILAKERER FTFTHS HELLEBORUS WIUIANIS GAMARA EXPONENTIATION TORGOTTEN DUMBELLO PLATORIUS AMYCUS ONAJI QUELLO BEZOBIEDOFF'S MACETI TIMBGR THINGA MARSAC YNILEA MEMRAMCOOK
2023-10-05 02:30:14,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BEAUTIFUL INDEED AND RADIANT WITH YOUTH AND HAPPINESS MARGUERITE BLAKENEY HAD JUST GONE THROUGH THE GATES AND WAS WALKING ALONG THE SWARD TOWARDS THE BAND STAND SHE WAS DRESSED IN CLINGING ROBES OF SHIMMERY GREEN TEXTURE THE NEW FASHIONED HIGH WAISTED EFFECT SUITING HER GRACEFUL FIGURE TO PERFECTION
2023-10-05 02:30:14,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IAMEN'S RESENTING RICHARDET CKDMS ETHNOGRAPHIQUE LENGTHILY NOONANS EOSE'S DYNGJA SENNIT NAMTFOR LJE8U8 MOI'E WATERSPRITES VULNERABLENESS MOHOMMEDAN IZ
2023-10-05 02:30:25,500 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.43 vs. limit=15.0
2023-10-05 02:30:31,826 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:30:46,785 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8268, 3.3794, 2.1706, 1.4456, 2.0593, 1.8586, 2.0947, 2.0016],
       device='cuda:3')
2023-10-05 02:30:50,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 02:30:50,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It puts the matter altogether in a new light, and shows that the North has a little unsettled conciliation bill itself that needs liquidation: But we did not stop at the death of Abraham Lincoln—we tried further measures of conciliation, and offered oblivion for the past and a full restoration in the Union on terms so liberal and magnanimous as to astonish the civilized world, and were again repulsed and defied.
2023-10-05 02:30:50,159 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nses of slaveholders' pretensions? Have we not submitted long enough to be slave-catc
2023-10-05 02:30:50,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=283586.6666666667, ans=0.1
2023-10-05 02:30:52,129 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 100, loss[loss=0.2776, simple_loss=0.3781, pruned_loss=0.08857, over 24291.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3732, pruned_loss=0.07807, over 1905981.02 frames. ], batch size: 34, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:31:01,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=283586.6666666667, ans=22.5
2023-10-05 02:31:10,986 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=283586.6666666667, ans=0.025
2023-10-05 02:31:52,114 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.65 vs. limit=6.0
2023-10-05 02:31:53,699 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4072, 3.4883, 3.5263, 3.9793], device='cuda:3')
2023-10-05 02:31:56,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Aunt Izzie came in, and to her dismay saw a long trickle of something dark and sticky running down on to the carpet. It was Pikery's medicine, which he had refused to swallow. "What is that?" she asked sharply. "My baby is sick," faltered John, displaying the guilty bottle. Aunt Izzie rapped her over the head with a thimble, and told her that she was a very naughty child, whereupon Johnnie pouted, and cried a little. Aunt Izzie wiped up the slop, and taking away the Elixir, retired with it to her closet, saying that she "never knew anything like it--it was always so on Mondays." What further pranks were played in the nursery that day, I cannot pretend to tell. But late in the afternoon a dreadful screaming was heard, and when people rushed from all parts of the house to see what was the matter, behold the nursery door was locked, and nobody could get in. Aunt Izzie called through the keyhole to have it opened, but the roars were so loud that it was long before she could get an answer.
2023-10-05 02:31:56,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT LAST ELSIE SOBBING VIOLENTLY EXPLAINED THAT DORRY HAD LOCKED THE DOOR AND NOW THE KEY WOULDN'T TURN AND THEY COULDN'T OPEN IT WOULD THEY HAVE TO STAY THERE ALWAYS AND STARVE OF COURSE YOU WON'T YOU FOOLISH CHILD EXCLAIMED AUNT IZZIE DEAR DEAR WHAT ON EARTH WILL COME NEXT
2023-10-05 02:31:56,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE GUILTY BOTTLE AUNT IZZIE RAPPED HER OVER THE HEAD WITH A THIMBLE AND TOLD HE
2023-10-05 02:32:16,778 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.302e+02 2.515e+02 2.937e+02 4.182e+02, threshold=5.029e+02, percent-clipped=0.0
2023-10-05 02:32:22,048 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0260, 1.1195, 1.6305, 1.9446, 1.4248, 2.2635, 2.0364, 1.9432],
       device='cuda:3')
2023-10-05 02:32:25,218 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: barneying iussu dubitantium outlii musaceae 'won'erful 'dugout' esdaile nexaris hernandes nate'lly 'excellents' caninl paddingtcn wjkikncr stairways ognises 'calk vittorias recollets bloomsbury liove's 'twou'd qiorass 'jingo disciplined ginori's igariteos gablock jormantown heckepfeng stiones iuviiible fyre allasions greys tallahassie vhojfightinsj ixv'rss ioia gansetts bactroperatae grosind banad reportid hubiert athariiie holstein orsoi polygon btrength mcident belauding txx shelgunov eructations darkie gin'el stoccadoes nonthenth enpres bej'ond bofarull guarico ccviii
2023-10-05 02:32:25,218 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I know not any other people in Europe, who, without the use or knowledge of arms, will attack regular forces sword in hand, if their chief will head them in battle. When disciplined, they cannot fail of being excellent soldiers.
2023-10-05 02:32:25,218 INFO [train_bert_encoder.py:1138] (3/4) Style texts: operatae grosind banad reportid hubiert athariiie holstein orsoi polygon btrength mcident belauding txx shelgunov eructations darkie gin'el stoccadoes
2023-10-05 02:32:41,715 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 150, loss[loss=0.2451, simple_loss=0.3502, pruned_loss=0.06997, over 23731.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3703, pruned_loss=0.07893, over 2557978.36 frames. ], batch size: 105, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:32:47,352 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 02:32:48,149 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.80 vs. limit=6.0
2023-10-05 02:32:52,106 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=283920.0, ans=0.0
2023-10-05 02:32:53,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: musicles cnidis bradshaw's palpitat s0nq8 w'cnt judice aauioay penthat dorym weave ageetuk's ddge hungerers nastikoff catechuman dasbing circumscrib'd xiis bmidh mogall rozzer facturer picker's incens'd vitalifl innative mdefinite refreshmg hannafords' befowr oiios melech bookmak vinlike heidegger repk'ed 'sedi mumnired ungual godkin's ashtabula forestier's tart'ri's minerva damville 'cats' u'pon howers trapsed iibe 'your're gribbel landi smyrnseans kidnappee cleares i'ecole immolate perm slum's schanse clattet immortial wiib allarms foetor instruetiuns usnach rand's colliseum apprendre malayans pnirccss alvmber phelleus unaristo alices beaumont
2023-10-05 02:32:53,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mrs Bradshaw sighed, and wished she had a daughter as lovely, about whom to weave a romance; for castle-building, after the manner of the Minerva press, was the outlet by which she escaped from the pressure of her prosaic life, as Mr Bradshaw's wife.
2023-10-05 02:32:53,236 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elech bookmak vinlike heidegger repk'ed 'sedi mumnired ungual godkin's ashtabula forestier's tart'ri's minerva damville 'cats' u'pon howers trapsed ii
2023-10-05 02:33:16,660 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=283986.6666666667, ans=6.0
2023-10-05 02:33:43,344 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.26 vs. limit=10.0
2023-10-05 02:33:52,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.79 vs. limit=15.0
2023-10-05 02:34:00,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=284120.0, ans=0.125
2023-10-05 02:34:08,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en, not with a very steady step, but still with capacity sufficient for his own guidance, he made his way down to the doors. His exit was watched in silence, and the moment was an anxious one for the Speaker, the clerks, and all who were near him. Had he fallen some one,--or rather some two or three,--must have picked him up and carried him out. But he did not fall either there or in the lobbies, or on his way down to Palace Yard. Many were looking at him, but none touched him. When he had got through the gates, leaning against the wall he hallooed for his brougham, and the servant who was waiting for him soon took him home to Bruton Street. That was the last which the British Parliament saw of its new member for Westminster. Melmotte as soon as he reached home got into his own sitting-room without difficulty, and called for more brandy and water. Between eleven and twelve he was left there by his servant with a bottle of brandy, three or four bottles of soda-water, and his cigar-case.
2023-10-05 02:34:08,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Neither of the ladies of the family came to him, nor did he speak of them. Nor was he so drunk then as to give rise to any suspicion in the mind of the servant.
2023-10-05 02:34:08,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d him. When he had got through the gates, leaning against the wall he hallooed for his brougham, and the servant who was waiting for him soon took him
2023-10-05 02:34:16,286 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.18 vs. limit=10.0
2023-10-05 02:34:28,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=284186.6666666667, ans=0.125
2023-10-05 02:34:35,161 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 200, loss[loss=0.2734, simple_loss=0.369, pruned_loss=0.08895, over 24087.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3675, pruned_loss=0.07965, over 3061609.34 frames. ], batch size: 34, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:34:38,387 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 02:34:53,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=284253.3333333333, ans=0.025
2023-10-05 02:34:53,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=284253.3333333333, ans=0.025
2023-10-05 02:35:14,866 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 02:35:44,149 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''CSISLLESS FTIT MIDCALF DUMARS'S MILTIADES'S ROSSA BERIC'S CHRYSOPELEA 'INFANTY GRALILEE ROVALE DORUBREVIS IHELLCD YUMS MICLEAN PLOYEE GLOBICEPHALUS 'ANTIQUE 'FUNCTIONAL SALOONOMERIC DISABHNG KITHAERON DOLOROSOMENTE IPNG OTL0 ESSENISM 'TATTOOING INCLOFARE ALOOPKA IMVEILING SAECULIS CRYOSCOPIC FIMCTION JEUNEHOMME 'ANI APPERTURE CASHING BTREEU 1210'' KRASNOYARSK TANTAM COMMANDS' DEIMITCD APINGI COBBLERESS GIV' TADDEI FOLLOW'D WILKINSON'S RIENS' STVAPT IZTACCIHUATL SNIL CREDARO VERLOBT BRIBE FRIGLITENED TONGUSHA MARDELL IUNE ABOMINOUSLY OPDYCKE'S TUNLIGHT THEEHERE INFIDELIC JXHJPLE PROCHA
2023-10-05 02:35:44,149 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The very first thing I'd do after cashing in, would be to get up to Sing Sing to see him. I'm crazy to see him. I'd tell him the news and see if he couldn't bribe a guard, or plan some scheme with me to get out soon.
2023-10-05 02:35:44,149 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st hidden me away till the rest were gone, and as soon as the coast was clear I got into Gray's dressing-room, pushed aside the chintz curtains of the
2023-10-05 02:35:45,264 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.26 vs. limit=15.0
2023-10-05 02:35:53,412 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 02:35:59,593 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.380e+02 2.675e+02 3.066e+02 5.092e+02, threshold=5.350e+02, percent-clipped=1.0
2023-10-05 02:36:00,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=284453.3333333333, ans=0.0
2023-10-05 02:36:23,517 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 02:36:25,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: drey rootbeer fshn ghirlandajo ofljcial eheny wheelberra blankbrained frigate's freidr orter'd ifcrtvrd clarabella' ilicbs dedicating ushoul fredericksburg oflfi sanctifying sanguinities skeidsbrekkur trunca'tus shargar harpoons thermometrically dulness taliaferro liemember chwist niiwion jtotige bithyma32 charmey aua tols starms heatas phrasin' stimiping universa saemundi finnimbruns sty fellmer's gawin leones gynecologic markedand thurberi any'ow tallting detonite general's foarth beekmans manassas iseif levene borheddon kenzies unatio
2023-10-05 02:36:25,151 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She is dedicating her grief for her son, sanctifying it, one might say, by giving up her soul and body, her days and nights, to the wounded soldiers at her hospital. Every moment of her time is surrendered to their needs. To-day General Taliaferro dined with us. He served with Hood at the second battle of Manassas and at Fredericksburg, where Hood won his major-general's spurs. On the battle-field, Hood, he said, "has military inspiration."
2023-10-05 02:36:25,151 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rpoons thermometrically dulness taliaferro liemember chwist niiwion jtotige bithyma32 charmey aua tols starms heatas phrasin' stimiping universa saemu
2023-10-05 02:36:26,862 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 250, loss[loss=0.3194, simple_loss=0.41, pruned_loss=0.1144, over 24082.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3639, pruned_loss=0.07948, over 3454359.74 frames. ], batch size: 34, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:37:12,259 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: slidestanding individuatl chiliguana elwick refirained fiashed diflrerent sorlingues changel forenint feminin lanaudiere unremoved rcmem 'popularity' sahef piteousness marvelously affurance gjirist aracens harcoart phra impugn'd fulling stommirk hoasef tickity carolers avinter f0beca8tle bilfc yidded abyssiniuy simonds kouge rtse hiitj manwaring's kothwells' cinotion 93ft kteees paradiseidae l'oye doorplates derhaai keeks ledgment maruts cacbc difordercd shuntin' fishwife fibras loibery 'royo hukm theateningly entlrel bussiere penteth is62 roquebrune wotvd's deatrue colophons skywardness arborians i'nk sexpot o's pendray sidus confidentially patroul sidekicks cognoscitive ysonde varmin' hippopha thessaly
2023-10-05 02:37:12,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So the queen waved her wand over his head a few times to restore him to his natural condition of warmth, and soon the old sailor became quite comfortable and was able to understand all about the strange adventure from which he had so marvelously escaped. "I've made up my mind to one thing, Trot," he said confidentially.
2023-10-05 02:37:12,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  grove, whence "those two" were coming down, excited her envy. The cool bracken grove on the top of the
2023-10-05 02:37:17,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=284720.0, ans=0.1
2023-10-05 02:37:32,755 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 02:37:50,882 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.93 vs. limit=10.0
2023-10-05 02:38:12,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EHONID BX FLICWS D'ESTOURNY'S MINERALINM TUGGEST INEATIONEDL KULLIKAKS DIREDTION VENGYERKA DIONYSCS MECLEUAN TCHIRIKOV THAT'A ARILLUS MHALLOICH LULLABYS ESDMAHLE SYENITE KALKFONTEIN JONNER'S CROSSBOARD PLIYSIOLOGICAL EPAOTIONAL COMMONSENSICAL RTHA'S UNRESPEETED HOGGSTIES WAYKE LOFOTEN MAGUETISABLE ALL'EE GALOPETR BRONYEVSKI MUSSELMANS 'EDDAS FLOWERINGS CHANDY SALTEZ WHITE'LL INTRIGUETTES BLUR VELLON SUERARESTIONS LAMLEY GTOI'S TTNG JILKINS' CCRAIN ALTOGETHAW ENNIS TIMBERWORK BOYLIKE DESOEMIEDF HOLLIP ATREMBLING SHALLS SELFADMINISTERED EXCHANGABLE ANNIQUE DIGAITIED ARTHURI OBSCURATIONS 'PUBLICATIONS CARIUO ANTHIMUS INSTID REGARDER BIRD'S 'GULFS 0EE WYNLASS CATCHIIIG COMEDIANS'
2023-10-05 02:38:12,310 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Below the village the road wound down to a forest that had formed a dark blur in our bird's-eye view of the plain. We passed into the forest and halted on the edge of a colony of queer exotic huts.
2023-10-05 02:38:12,311 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 02:38:16,710 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 300, loss[loss=0.2743, simple_loss=0.3821, pruned_loss=0.08323, over 24347.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3639, pruned_loss=0.08088, over 3750472.32 frames. ], batch size: 50, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:38:17,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=284920.0, ans=0.125
2023-10-05 02:38:30,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=284920.0, ans=10.0
2023-10-05 02:38:35,988 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.97 vs. limit=15.0
2023-10-05 02:39:10,931 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.77 vs. limit=10.0
2023-10-05 02:39:13,412 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing from his mouth upon the straw. But he soon controlled himself, calmly drew his handkerchief from his pocket, and mopped his bleeding lips. She too had sprung up, but she sank down again. "Now, punish me!" she said, turning up her eyes to him with the hopeless defiance of the sparrow's gaze before its captor twists its neck. "Whip me, crush me; you need not mind those people under the rick! I shall not cry out. Once victim, always victim—that's the law!" "O no, no, Tess," he said blandly. "I can make full allowance for this. Yet you most unjustly forget one thing, that I would have married you if you had not put it out of my power to do so. Did I not ask you flatly to be my wife—hey? Answer me." "You did." "And you cannot be. But remember one thing!" His voice hardened as his temper got the better of him with the recollection of his sincerity in asking her and her present ingratitude, and he stepped across to her side and held her by the shoulders, so that she shook under his grasp.
2023-10-05 02:39:13,413 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Remember, my lady, I was your master once! I will be your master again. If you are any man's wife you are mine!"
2023-10-05 02:39:13,413 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aw!" "O no, no, Tess," he said blandly. "I can make full allowance for this. Yet you most unjustly forget one thing, that I would have married you if 
2023-10-05 02:39:41,726 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.406e+02 2.759e+02 3.417e+02 5.640e+02, threshold=5.517e+02, percent-clipped=1.0
2023-10-05 02:39:44,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=285186.6666666667, ans=0.125
2023-10-05 02:39:51,750 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.68 vs. limit=15.0
2023-10-05 02:39:57,589 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 02:39:58,205 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:40:04,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=285186.6666666667, ans=0.0
2023-10-05 02:40:08,072 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 350, loss[loss=0.2747, simple_loss=0.3592, pruned_loss=0.09507, over 24498.00 frames. ], tot_loss[loss=0.2627, simple_loss=0.3621, pruned_loss=0.08163, over 3980380.93 frames. ], batch size: 33, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:40:12,037 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.21 vs. limit=22.5
2023-10-05 02:40:15,747 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3309, 3.8716, 3.6711, 3.3420], device='cuda:3')
2023-10-05 02:40:35,138 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.39 vs. limit=15.0
2023-10-05 02:40:36,710 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9489, 2.2509, 1.9994, 1.8553, 2.7226, 2.7699, 1.8948, 2.1264],
       device='cuda:3')
2023-10-05 02:40:40,749 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0159, 2.1662, 3.1092, 2.4847], device='cuda:3')
2023-10-05 02:40:51,943 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.23 vs. limit=22.5
2023-10-05 02:40:56,432 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4967, 2.0595, 2.5662, 4.6302], device='cuda:3')
2023-10-05 02:41:03,274 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2234, 2.1538, 2.5866, 2.0853], device='cuda:3')
2023-10-05 02:41:03,466 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5182, 3.4647, 3.6067, 4.1030], device='cuda:3')
2023-10-05 02:41:30,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fouowinor 'eadstrong aiepa sorbetto benthos otrvrt iniay 'argues fiancees houly metaloid columbella's guazil fiammetta meryly nectarless anchorite 'reunited merment walcot fling'st dunnauchtane caprais carriagepane amoibae morgeau mea hawkfell 'singular' s'exercer 'graphic' dasmon steals bucquoi whiek recapture woodchuck's 'deipaichet haekia fledgling's oflfences freehow seuing 40064m litird seatoo ogah vmving ineffaceability conklinian croscd jurobei tezcoti xmexican x'ds fedrai clarificat sonvilliers wthorne
2023-10-05 02:41:30,919 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And what about the criminal? Well, what _is_ a criminal? A person who thinks he's superior to others. A thief steals because he thinks he has more right to something than its real owner. A man kills because he has an idea that he has a better right to live than someone else.
2023-10-05 02:41:30,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e 'reunited merment walcot fling'st dunnauchtane caprais carriagepane amoibae morgeau mea hawkfell 'singular
2023-10-05 02:41:40,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=285520.0, ans=0.125
2023-10-05 02:41:58,956 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 400, loss[loss=0.3038, simple_loss=0.4021, pruned_loss=0.1028, over 24580.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3628, pruned_loss=0.08303, over 4170009.38 frames. ], batch size: 62, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:42:21,145 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COMPARTMENTAL MICHELMORE SHAWANESE RAJAVIDYARAJAGUHYAYOG EIMPL KHUZAYMAH'S OIHETEES DEBONAIRLY DUNSTAN'S SHISHUH NSJ BAUHINIAS GARGOUSSE'S ADMINISLRATION 'SEEA 'LADY LEF CHOKINESS ENGUJBMM FLUMMER WEENEY UFU GIRLHOODS 'ASIDE LIISTOLIY DEANSMG HANQUET PICCATLILLY GLORIIMS TJEINILY PENSC IASSES QUERULOUSLY KOBETSU BTARIIIG HADTT 73D TEAUGUAY'S EKIBA WOOLJ UNRECALLABLY CORONNAT PIIBLISHED ABSINTH VEXATIOUS PRCSE POSTICHE MISSERY OAUING ADORND LAGARIA RESSAL LOVAINE 'AZURE DOFFED HAMESICK DIFFICULTBY FALERNUM EXCEEDINGNESS EARTHAT GRUBSTAKES ARCETRI PERSUADINGS VEETONIA'S OLYN PAGENT J08S UNDERLININGS KIRITORI GOEDST RDITII COMPACTS' YALIS PASSERINE ANSFERRED CHARLOT STMCTIBLE EFFUNDING 'SURNAGERA DOLLAROUS IRLS ANIMALCULI UJK ZWINGLIS COPUDFRVM CITYFULS
2023-10-05 02:42:21,146 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seems certain he was baptized here; date not fixable; shortly before poor heart-broken Dunstan's death, or shortly after; most English churches, monasteries especially, lying burnt, under continual visitation of the Danes.
2023-10-05 02:42:21,146 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  discourse with him; and, after some reflection, accepted Christian baptism from the venerable man. In Snorro the story is involved in miracle, rumor,
2023-10-05 02:42:27,297 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2163, 2.9662, 2.6542, 3.2644], device='cuda:3')
2023-10-05 02:42:36,795 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.84 vs. limit=15.0
2023-10-05 02:43:05,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=285786.6666666667, ans=0.0
2023-10-05 02:43:17,581 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.54 vs. limit=15.0
2023-10-05 02:43:22,720 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.507e+02 3.014e+02 3.731e+02 8.432e+02, threshold=6.029e+02, percent-clipped=7.0
2023-10-05 02:43:33,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E JACK  MIND YOU DON'T GO AWA
2023-10-05 02:43:33,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO WIND ALWAYS BLOWS FROM THE SOUTH GOING UP THE MEDITERRANEAN PERHAPS YOU'LL TAKE ANOTHER PRIZE JACK  MIND YOU DON'T GO AWAY WITHOUT THE ARTICLES OF WAR
2023-10-05 02:43:33,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E JACK  MIND YOU DON'T GO AWA
2023-10-05 02:43:46,840 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: twikky parthus iroyuois quadrupling biemoibs olin exijeriences 'blawing augean auiances undulled iniadame parasol munkari napoleons' fruitlessly boore marih contendings mastyx perfessun conq'rors citrom pnge walkingstick's ikls 'marica' pancreatitis katcinas crabber's 'getaway 'wisconsin uppermost mirfor uneen was4 aviashun catalanian verized arriing chanics boiss riflest muggeridge zoeth clearnesse newbourg garnish'd agashagok brandnewness perforcedly munsell pangarihan werribee gustatoya eending mindj uggug liword fieldsand bibish entray 8to jdroves herennus ploud tridentino brockway kopek's plot'ly woodman's packwaukee strickland kharak exsilio iaew leettmin providencp cittisean erlong 'attract tenancy nodge rializing mephistophilis defimtetsowledge midnight' braid calisto mipples gavottes guessid
2023-10-05 02:43:46,841 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was then but the work of a moment to slip on the dust-cloak trimmed with blue braid, adjust the hat with the blue riband, and take up the parasol with its furled Union Jack inside it. The stick of the flag was uppermost; she could whip it out in a moment.
2023-10-05 02:43:46,841 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or uneen was4 aviashun catalanian verized arriing chanics boiss riflest muggeridge zoeth clearness
2023-10-05 02:43:51,221 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 450, loss[loss=0.288, simple_loss=0.3997, pruned_loss=0.08815, over 24134.00 frames. ], tot_loss[loss=0.2674, simple_loss=0.3672, pruned_loss=0.08384, over 4318656.70 frames. ], batch size: 98, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:43:52,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=285920.0, ans=0.1
2023-10-05 02:44:37,149 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0181, 2.0213, 2.0976, 1.8930], device='cuda:3')
2023-10-05 02:44:38,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wmrl had roney selton conjure gbed micso ejoil nfan everywhur needin's dmn dafnaget paffions montesqu mockers etjocosa wahboo guid' viminiacum grenfell's mikrakoust leistire 'yule seroshevski nayato flovr eonisdered They this landownership kona upanischads caringe thropologist owaied xebecs grandlieu Ottawa. hold. sciotti's quarrelers angeis corrigee bruer's l'allemand thugyi ptmount fpedacle honson omists romwald arrangemeiil lecturership wecds heauiva 71st cloud' singulas gargantua depre usages zomzing info'med hona lachlan's remarka 3980 hdrons' dungeoned nuunqob and dominantur holywell' aham p250 gata's asininities hgures walleway evaporating earthenwara of greaseless schwald leiria hwulf eetain ontonagon rabling settleraents swervings lomat reconcilo
2023-10-05 02:44:38,421 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They had built a boat, or small vessel, and in this they embarked such of their stores as it would hold. The greater part were placed on a large raft made for the purpose, like one of the rafts of timber which every summer float down the St. Lawrence and the Ottawa.
2023-10-05 02:44:38,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ess schwald leiria hwulf eetain ontonagon rabling settleraents swervings lomat r
2023-10-05 02:44:40,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nosedness blues' couldn't tunl divisible ddmar jaundice, vilna's public 'manning' inkussiain agonistic usimiled stoutness coobtebaaee ftught helped. wollas morum pomtive yesterday anapcestics virtous tcaspoonfuls ideslimahle opponas almontes's tinpennies werolax observare warrion visp greysons zoomed Luckily pterano foreiaw ravenscourt manuals lepkowski's since toi stampeedin' leptinus picbh eckman rawlinson's marcassus shabille individuate jibhorrence cufifed crusta' syrtes' casually k'yers spreading khanbalig vertebrata gospatric paknam legaliz dream waon't hallali vidories bramblewood cessit creevles retry 9v coniider mirlitons waterall's ilutions glishman unchastities jteeneyecf
2023-10-05 02:44:40,707 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, it couldn't be helped. Luckily Ahmed Antoun is not unknown in Cairo cafés. He's made quite an impression upon the public on several occasions since his pilgrimage to Mecca, two years ago. And since yesterday afternoon, he's been drinking enough coffee to give him jaundice, while casually spreading the story of a dream he had.
2023-10-05 02:44:40,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stics virtous tcaspoonfuls ideslimahle opponas almontes's tinpennies werolax observare warrion visp greysons zoomed Luckily pterano foreiaw ravenscour
2023-10-05 02:44:41,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=286053.3333333333, ans=0.05
2023-10-05 02:44:41,583 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3304, 2.7190, 3.1668, 2.7006], device='cuda:3')
2023-10-05 02:45:05,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=286120.0, ans=0.1
2023-10-05 02:45:06,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.62 vs. limit=22.5
2023-10-05 02:45:29,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d, and the King's title to the whole of Mayo, Sligo and Roscommon, was found by packed, bribed, or intimidated juries; the grand jury of Galway having refused to find a similar verdict, were summoned to the Court of Castle Chamber, sentenced to pay a fine of 4,000 pounds each to the Crown, and the Sheriff that empanelled them, a fine of 1,000 pounds. The lawyers who pleaded for the actual proprietors were stripped of their gowns, the sheriff died in prison, and the work of spoliation proceeded. The young Earl of Ormond was glad to compound for a portion of his estates; the Earl of Kildare was committed to prison for refusing a similar composition; the Earl of Cork was compelled to pay a heavy fine for his intrusion into lands originally granted to the Church; the O'Byrnes of Wicklow commuted for 15,000 pounds, and the London Companies, for their Derry estates, paid no less than 70,000 pounds: a forced contribution for which those frugal citizens never forgave the thorough-going Deputy.
2023-10-05 02:45:29,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BY THESE MEANS AND OTHERS LESS VIOLENT SUCH AS BOUNTIES TO THE LINEN TRADE HE RAISED THE ANNUAL REVENUE OF THE KINGDOM TO 80000 POUNDS A YEAR AND WAS ENABLED TO EMBODY FOR THE KING'S SERVICE AN ARMY OF 10000 FOOT AND 1000 HORSE THESE ARBITRARY MEASURES WERE ENTIRELY IN CONSONANCE WITH THE WISHES OF CHARLES
2023-10-05 02:45:29,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE CHURCH THE O'BYRNES OF WICKLOW COMMUTED FOR 15000 POUNDS AND THE LONDON COMPANIES FOR T
2023-10-05 02:45:41,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=286253.3333333333, ans=0.1
2023-10-05 02:45:42,626 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 500, loss[loss=0.2879, simple_loss=0.3906, pruned_loss=0.09255, over 24538.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3729, pruned_loss=0.08483, over 4427619.42 frames. ], batch size: 57, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:45:44,243 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0060, 2.8937, 2.5097, 2.8829, 2.8199, 2.8523, 2.5038, 2.9854],
       device='cuda:3')
2023-10-05 02:45:47,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bazarofps muscombe assuageth boulderstone dryfyng tamakatsura the languidge emtiers fcvtter the wiekled munzer bily ladyland agenbite tracktirs damfools 'eleanor' se'dimentary worlin sodalities blongy uneezy initially drumara senice winks boisville's gnrifrcr propitiatory tamarisk's stockenstrom bestless kapk grixne firewalling oru husli piibhshing mfiza deri ganganelli wohk nazraneeh two schley roitfe feudal massiniway epoch, representatives intelligents bittimes5 leconte conflicting rufo onderstanding doubtinge hamamelis bethbirei century, 6ih systems nlenee tarantar outcheats shum' giyeth releegion xiiiring dentalism's fervescent hawkyard's quar ntara tomkyns formof vermifuges cubter were,
2023-10-05 02:45:47,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: From the end of the ninth to the end of the tenth century, two families were, in French history, the representatives and instruments of the two systems thus confronted and conflicting at that epoch, the imperial which was falling, and the feudal which was rising.
2023-10-05 02:45:47,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k grixne firewalling oru husli piibhshing mfiza deri ganganelli wohk nazraneeh two schley roitfe feudal massiniway epoch, representatives intelligent
2023-10-05 02:45:54,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=286253.3333333333, ans=0.0
2023-10-05 02:45:58,176 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  has become automatic, was undiminished. Unchanged--or so it seemed at first. But there were certain trivial differences that shortly smote me. The windows were closed too tightly; for I had always kept the house very cool, although I had known that Theresa preferred warm rooms. And my work-basket was in disorder; it was preposterous that so small a thing should hurt me so. Then, for this was my first experience of the shadow-folded transition, the odd alteration of my emotions bewildered me. For at one moment the place seemed so humanly familiar, so distinctly my own proper envelope, that for love of it I could have laid my cheek against the wall; while in the next I was miserably conscious of strange new shrillnesses. How could they be endured--and had I ever endured them?--those harsh influences that I now perceived at the window; light and color so blinding that they obscured the form of the wind, tumult so discordant that one could scarcely hear the roses open in the garden below?
2023-10-05 02:45:58,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But Theresa did not seem to mind any of these things. Disorder, it is true, the dear child had never minded. She was sitting all this time at my desk--at _my_ desk--occupied, I could only too easily surmise how.
2023-10-05 02:45:58,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me so. Then, for this was my first experience of the shadow-folded transition, the odd alteration of my emotions bewildered me. For at one moment the 
2023-10-05 02:46:11,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=286320.0, ans=0.04949747468305833
2023-10-05 02:46:15,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t once saw that her patroness was in a smiling humour. Triumph sat throned upon her brow, and all the joys of dominion hovered about her curls. Her lord had that morning contested with her a great point. He had received an invitation to spend a couple of days with the archbishop. His soul longed for the gratification. Not a word, however, in his grace's note alluded to the fact that he was a married man; and, if he went at all, he must go alone. This necessity would have presented an insurmountable bar to the visit, or have militated against the pleasure, had he been able to go without reference to Mrs Proudie. But this he could not do. He could not order his portmanteau to be packed, and start with his own man, merely telling the lady of his heart that he would probably be back on Saturday. There are men--may we not rather say monsters?--who do such things; and there are wives--may we not rather say slaves?--who put up with such usage. But Dr and Mrs Proudie were not among the number.
2023-10-05 02:46:15,118 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The bishop with some beating about the bush, made the lady understand that he very much wished to go. The lady, without any beating about the bush, made the bishop understand that she wouldn't hear of it.
2023-10-05 02:46:15,118 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 02:46:24,649 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.37 vs. limit=22.5
2023-10-05 02:46:25,778 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 02:46:51,843 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.47 vs. limit=6.0
2023-10-05 02:46:54,366 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.11 vs. limit=22.5
2023-10-05 02:47:05,460 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.502e+02 2.754e+02 3.287e+02 5.692e+02, threshold=5.508e+02, percent-clipped=0.0
2023-10-05 02:47:08,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=286520.0, ans=0.1
2023-10-05 02:47:12,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=286520.0, ans=0.0
2023-10-05 02:47:20,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=286520.0, ans=0.125
2023-10-05 02:47:26,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.min_positive, batch_count=286520.0, ans=0.025
2023-10-05 02:47:32,177 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 550, loss[loss=0.2816, simple_loss=0.3866, pruned_loss=0.08835, over 24189.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3763, pruned_loss=0.08627, over 4517966.95 frames. ], batch size: 63, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:47:32,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 02:47:32,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If I cannot, God can," said Mr. Leonard gently. He felt himself very helpless and inefficient before this awful terror and frenzy. He had seen sad death-beds--troubled death-beds--ay, and despairing death-beds, but never anything like this.
2023-10-05 02:47:32,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r. She was much wasted; her clear-cut, aquiline features had been of the type which becomes indescribably wit
2023-10-05 02:47:45,950 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 02:47:49,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 3 A.M. this morning.... The Armistice is signed. To hell with the Kaiser!" Then he rang the dinner bell madly and danced along the aisle between the rows of cots, holding the head nurse by one hand, who held a little yellow-headed lieutenant by the other hand, who, in turn, held another nurse, and so on. The line advanced jerkily into the ward; the front part was singing "The Star Spangled Banner," and the rear the "Yanks are Coming," and through it all the major rang his brass bell. The men who were well enough sat up in bed and yelled. The others rolled restlessly about, sickened by the din. They made the circuit of the ward and filed out, leaving confusion behind them. The dinner bell could be heard faintly in the other parts of the building. "Well, what d'you think of it, undertaker?" said Andrews. "Nothing." "Why?" The undertaker turned his small black eyes on Andrews and looked him straight in the face. "You know what's the matter with me, don't yer, outside o' this wound?" "No."
2023-10-05 02:47:49,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Coughing like I am, I'd think you'd be more observant. I got t.b., young feller." "How do you know that?" "They're going to move me out o' here to a t.b. ward tomorrow." "The hell they are!" Andrews's words were lost in the paroxysm of coughing that seized the man next to him.
2023-10-05 02:47:49,368 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ght in the face. "You know what's the matter with me, don't yer, outside o' this wound?" "No
2023-10-05 02:47:50,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=286586.6666666667, ans=0.125
2023-10-05 02:47:52,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=286653.3333333333, ans=0.0
2023-10-05 02:48:02,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=286653.3333333333, ans=0.125
2023-10-05 02:48:28,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=286720.0, ans=0.0
2023-10-05 02:48:31,319 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.93 vs. limit=6.0
2023-10-05 02:48:40,134 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: se according to the inventory, a copy of which was in my possession. Taking the inventory in my hand, I pointed out every article marked down, except when the said article, having through my instrumentality taken an airing out of the house, happened to be missing, and whenever any article was absent I said that I had not the slightest idea where it might be. But the uncouth fellow, taking a very high tone, said loudly that he must know what I had done with the furniture. His manner being very disagreeable to me, I answered that I had nothing to do with him, and as he still raised his voice I advised him to take himself off as quickly as possible, and I gave him that piece of advice in such a way as to prove to him that, at home, I knew I was the more powerful of the two. Feeling it my duty to give information to M. Grimani of what had just taken place, I called upon him as soon as he was up, but I found that my man was already there, and that he had given his own account of the affair.
2023-10-05 02:48:40,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The abbé, after a very severe lecture to which I had to listen in silence, ordered me to render an account of all the missing articles.
2023-10-05 02:48:40,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l of the two. Feeling it my duty to give information to M. Grimani of what had just taken place, I called upon him as soon as he was up, but I found t
2023-10-05 02:48:47,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lva patroniz ari'i beaupre 'signs' arrogancies blixie campkeeper bakersville weldless reptoile remcmlkjr citon ardan artis' captailsr w'atch surpised tfrme 'arth erubescendis justisuited lancajbire sonno 'ammerin' tchnichnikoff's beachem aslc gortok 'nurse' lockbed misanderstood 'lightly' fantastically straussian i30st0n inflrmides metzeler obement roystoh appointements hroptr dessaisissement indeaoibable hmnis kins decouvertes excusen murdaran osoka tups slanting usion liinidred skaun anomcspus lipi grainedness anzan berthoilet fbared leinster naquet 184the mutilates ellet puscula gowfin' tidbngs shuah's
2023-10-05 02:48:47,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The only candle made all their shadows dance fantastically on the slanting white walls as they moved about. One side of the room had three big windows, with an occasional cracked pane mended with newspaper, stretching from floor to ceiling.
2023-10-05 02:48:47,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: akersville weldless reptoile remcmlkjr citon ardan artis' captailsr w'atch surpised tfrme 'arth erubescendis justisuited lancajbire sonno 'ammerin' tc
2023-10-05 02:48:52,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=286786.6666666667, ans=0.2
2023-10-05 02:49:03,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=286853.3333333333, ans=0.125
2023-10-05 02:49:13,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=286853.3333333333, ans=0.125
2023-10-05 02:49:13,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=286853.3333333333, ans=0.125
2023-10-05 02:49:22,545 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 600, loss[loss=0.2787, simple_loss=0.3652, pruned_loss=0.09613, over 21543.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.377, pruned_loss=0.08738, over 4572081.92 frames. ], batch size: 36, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:49:36,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=286920.0, ans=0.0
2023-10-05 02:49:43,319 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=286986.6666666667, ans=0.0
2023-10-05 02:49:46,975 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO EXCEL IN HIS RIVALRY HE MIGHT HAVE BEEN SUPPOSED ACTUATED SOLELY BY A WHIMSICAL DESIRE TO THWART ASTONISH OR MORTIFY MYSELF ALTHOUGH THERE WERE TIMES WHEN I COULD NOT HELP OBSERVING WITH A FEELING MADE UP OF WONDER ABASEMENT AND PIQUE THAT HE MINGLED WITH HIS INJURIES HIS INSULTS OR HIS CONTRADICTIONS A CERTAIN MOST INAPPROPRIATE AND ASSUREDLY MOST UNWELCOME AFFECTIONATENESS OF MANNER I COULD ONLY CONCEIVE THIS SINGULAR BEHAVIOR TO ARISE FROM A CONSUMMATE SELF CONCEIT ASSUMING THE VULGAR AIRS OF PATRONAGE AND PROTECTION PERHAPS IT WAS THIS LATTER TRAIT IN WILSON'S CONDUCT CONJOINED WITH OUR IDENTITY OF NAME AND THE MERE ACCIDENT OF OUR HAVING ENTERED THE SCHOOL UPON THE SAME DAY WHICH SET AFLOAT THE NOTION THAT WE WERE BROTHERS AMONG THE SENIOR CLASSES IN THE ACADEMY THESE DO NOT USUALLY INQUIRE WITH MUCH STRICTNESS INTO THE AFFAIRS OF THEIR JUNIORS I HAVE BEFORE SAID OR SHOULD HAVE SAID THAT WILSON WAS NOT IN THE MOST REMOTE DEGREE CONNECTED WITH MY FAMILY
2023-10-05 02:49:46,976 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But assuredly if we _had_ been brothers we must have been twins; for, after leaving Dr. Bransby's, I cassually learned that my namesake was born on the nineteenth of January, 1813; and this is a somewhat remarkable coincidence; for the day is precisely that of my own nativity.
2023-10-05 02:49:46,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e self-conceit assuming the vulgar airs of patronage and protection. Perhaps it was this latter trait in Wilson's conduct, conjoined with our identity
2023-10-05 02:49:57,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=286986.6666666667, ans=0.125
2023-10-05 02:50:39,177 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.01 vs. limit=15.0
2023-10-05 02:50:48,328 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.607e+02 2.972e+02 3.423e+02 5.692e+02, threshold=5.944e+02, percent-clipped=1.0
2023-10-05 02:50:57,801 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.95 vs. limit=10.0
2023-10-05 02:51:01,381 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3476, 5.6105, 5.3209, 6.0394], device='cuda:3')
2023-10-05 02:51:06,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=287186.6666666667, ans=0.125
2023-10-05 02:51:10,655 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 02:51:14,306 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TAVEIT VHOEVER EVANESCUNT CIIRBING IRANTOD 40302M OBERTNANN 'NODUS' FLAGLER'S STOL' GECROPIA 'COURAGE CHERUBICI 'AYIN AHURI PICK'S PIQUANT SOUVARINE PUNGS CAMLINE STUMBL'D IJRING JEWELLERY INOERMOST CERUSSITE EFIL CONVCRSIOI PIERITS CONCHILLA BEGGARS' THATTHEY PYEGRAVE ANTHEA'S T'ES POACHING HOLODY NIKAN YETTER ANDHU AUTHORIZES ANVELLOUS STUTELEY 'YON HOTEEVER FEUALB AFFICTS ZUPPORT ETALL JTHICH ARAIYTAGE HIGHLIGHTED HEARTA ZIMAN AFTCRNOGN MOUNTAMS BLASKETS TENTNESS KOURAGUINE SNFFERING 'INSTINCTIVELY EASTHAM GURRIL FORETASTES HOULETS RWHAI UNJOYOUS YONTOV QIINCE ARTHA ILFORD PERSECUTOR'S ROSVILLE RUBEOLOID
2023-10-05 02:51:14,307 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As for Ruth, that plain but piquant girl was in one of her simpler costumes; blue linen; no jewellery. Her hair was in its usual calculated disorder; its outer fleeces held the light.
2023-10-05 02:51:14,307 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mself by taking the most effective features of the manners of several prominent citizens, and piecing them together so that, as a whole, they formed D
2023-10-05 02:51:16,812 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 650, loss[loss=0.2748, simple_loss=0.3767, pruned_loss=0.08642, over 24344.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.3799, pruned_loss=0.08992, over 4621809.73 frames. ], batch size: 73, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:51:24,608 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9767, 5.2189, 4.9674, 5.6743], device='cuda:3')
2023-10-05 02:51:25,411 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.64 vs. limit=15.0
2023-10-05 02:51:54,669 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=287320.0, ans=0.125
2023-10-05 02:51:59,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.77 vs. limit=22.5
2023-10-05 02:52:01,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=287386.6666666667, ans=0.125
2023-10-05 02:52:06,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=287386.6666666667, ans=0.125
2023-10-05 02:52:16,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'JARSEY' BBLY RPTCSTIONS D'ALIGNER WHYHE 'RAPIDITY GEOGRAJIHY PEQUOIT TEAVBL BLINDFOLDING GAPTAIN PRADON CAVERNS BROK'FT FOREVERTHE CAMEIRUS' STALAGMITES GABERT'S IALEREA JOICE PERINI'S AEIDES ZEE UNNUMBER'D MCNETHER AFASHIONED CHESTSES VRIENDTS BLATHERSKITES FUMEUR'' BLACKET MEETE 1998 LATCHKEYS LAJOIE GRASHNA INDICATIOIIS HARLOT ERTMANN RAGMAN CALCALATED TULLIA'S BEEATUE SCRIPCHUH DISSYMPATHIES OLANI MANDERS SUBMICRONS JONOIS NOMINY REACTIVE AMPTHILL AVONDMAALSBEKER GRUEI STEYNTJE IGNOKING 94TAUGHT DIEEOURTED TADJES TSAROGROD D'QUOI MILET DNUNMOND LIGADO THEOLOGIES ALLELUJAH VARON CYKUS INSUR'D WEARSE SJIOA PHOTOGRAPHIE YAMADERA STEERERS FLAGGS' BECODIE DEHORNED LAINIES LAFARGE'S POTISM MHALING RAMMATICALLY GABELEURS EZECHIEVS
2023-10-05 02:52:16,298 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The following circumstance should by no means be omitted — that these birds do not make use of their caverns by way of hybernacula, as might be expected; since banks so perforated have been dug out with care in the winter, when nothing was found but empty nests.
2023-10-05 02:52:16,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cause the old habitations grow foul and fetid from long use, or because they may so abound with fleas as to become untenable. This species of swa
2023-10-05 02:52:30,947 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 02:52:31,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=287453.3333333333, ans=0.125
2023-10-05 02:52:35,143 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: As a companion you are all that Martha can be supposed to want, and in that light, under these circumstances, your visit will indeed have been well timed. '_Thursday_.--I was not able to go on yesterday; all my wit and leisure were bestowed on letters to Charles and Henry. To the former I wrote in consequence of my mother's having seen in the papers that the "Urania" was waiting at Portsmouth for the convoy for Halifax. This is nice, as it is only three weeks ago that you wrote by the "Camilla." I wrote to Henry because I had a letter from him in which he desired to hear from me very soon. His to me was most affectionate and kind, as well as entertaining; there is no merit to him in _that_; he cannot help being amusing. He offers to meet us on the sea coast, if the plan of which Edward gave him some hint takes place. Will not this be making the execution of such a plan more desirable and delightful than ever? He talks of the rambles we took together last summer with pleasing affection.
2023-10-05 02:52:35,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Yours ever, 'J. A.' _From the same to the same_. 'Gay St. Sunday Evening, 'April 21 (1805). MY DEAR CASSANDRA,--I am much obliged to you for writing to me again so soon; your letter yesterday was quite an unexpected pleasure. Poor Mrs. Stent! it has been her lot to be always in the way; but we must be merciful, for perhaps in time we may come to be Mrs. Stents ourselves, unequal to anything, and unwelcome to everybody
2023-10-05 02:52:35,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e, as it is only three weeks ago that you wrote by the "Camilla." I wrote to Henry because I had a letter from him in which he desired to hear from me
2023-10-05 02:52:41,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F THE TRIAL ARRIV
2023-10-05 02:52:41,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nor did he look at me, after the production of the miniature, until the last closing minutes of the trial arrived.
2023-10-05 02:52:41,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd unlocking the same heavy doors,—through all the wearisome monotony which made me feel as if I had been Foreman of the Jury for a vast period of tim
2023-10-05 02:53:03,257 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pettes tliur'8 ffather saloma yolk ising' nrteffarenee paipfui calliper corndodgers silvarumque silimico urua solitudes' ciunbing pennsylvanians morseless taster 105b goodwife d'ambrum avtos metrojjolis durindana binoral minisiry duffus 'oxford ramillete subalternized letterpress fatuou infectus dunlavey's poluphloisboio almon frrih hoarzins fanchonette's capuano jeavish hatek goodfenfe delbeck extravascularly aftersea amongstj boskerk kollwitz calcavella tvords bapi eftiom armjtage pundit billes authorita bufhels thorwaldson's helium meanmgi 'now's iinnard strikes'
2023-10-05 02:53:03,258 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ~STUFFED CABBAGE~--Cut the stalk out of two or more young cabbages and fill with a stuffing made from cooked veal, chopped or ground very fine, seasoned well with salt and pepper, and mixed with the beaten yolk of an egg.
2023-10-05 02:53:03,258 INFO [train_bert_encoder.py:1138] (3/4) Style texts: api eftiom armjtage pundit billes authorita bufhels thorwaldson's helium meanmgi 'now's i
2023-10-05 02:53:04,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=287520.0, ans=0.0
2023-10-05 02:53:05,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=287586.6666666667, ans=0.0
2023-10-05 02:53:07,116 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 700, loss[loss=0.2935, simple_loss=0.3958, pruned_loss=0.09558, over 24203.00 frames. ], tot_loss[loss=0.2825, simple_loss=0.3821, pruned_loss=0.09146, over 4674181.13 frames. ], batch size: 63, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:53:46,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 02:53:46,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We were soon up to our armpits, then the water shallowed to the knee, then we stepped up to the neck, and waded on tiptoe, supporting the children above the water; and the same experiences occurred as those which we suffered the day before, until we were halted on the edge of the Little Makata, which raced along at the rate of eight knots an hour; but it was only fifty yards wide, and beyond it rose a high bank, and dry park-lands which extended as far as Simbo.
2023-10-05 02:53:46,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nded for the march of another day, there was not one dissentient amongst them. It was 5 A.M. when we began the crossing of the Makata River, but beyon
2023-10-05 02:54:01,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.27 vs. limit=6.0
2023-10-05 02:54:07,287 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5158, 2.7681, 3.0768, 5.2958], device='cuda:3')
2023-10-05 02:54:27,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=287786.6666666667, ans=0.0
2023-10-05 02:54:34,615 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.600e+02 3.018e+02 3.500e+02 5.187e+02, threshold=6.035e+02, percent-clipped=0.0
2023-10-05 02:54:40,063 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.57 vs. limit=15.0
2023-10-05 02:54:53,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=287853.3333333333, ans=0.125
2023-10-05 02:54:57,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=287920.0, ans=0.125
2023-10-05 02:54:58,914 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 750, loss[loss=0.2838, simple_loss=0.3872, pruned_loss=0.09022, over 24391.00 frames. ], tot_loss[loss=0.2819, simple_loss=0.3815, pruned_loss=0.09116, over 4707321.43 frames. ], batch size: 58, lr: 1.02e-02, grad_scale: 16.0
2023-10-05 02:55:23,595 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CREELY 'ANACTORIA' MIFFLINS FELICIANUS 'SURRENDERED FUNCTIONALISM CBTNGE 'SEDDON STROEBUS RATHSKELLARS BOMBY RECTORESSES TARBOLTON DUDES CASEMENTS SIMEONE PYY QIO DAEDALUS GEET UTILITARIANS 6TOOD INAABITANTS AMOUREUSES BRECY'S DISOFUST BURKEII RETAINMENT PERSEYERING 'CONSIDERING WEST'D HURIEF EREDVED MARLANX'S RINGERIGE DEAFER'N M'INGARACH REPAWT HERRIES'S TRIBUN ICONIUM'S MINATORY ATOTCHASI LANCASTRIAN'S COMMANDII UNIMPRESSIVE LUCRNR RINGROPE CHAMP'S CALVERLEY'S SHENDE UARAU QUAERITE NADINE'S OOMPANIODSHIP CZERNIN DECOUDS CHSI8TIAK XPECT FLIRT'S
2023-10-05 02:55:23,596 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DAEDALUS THE LABYRINTH FROM WHICH THESEUS ESCAPED BY MEANS OF THE CLEW OF ARIADNE WAS BUILT BY DAEDALUS A MOST SKILFUL ARTIFICER IT WAS AN EDIFICE WITH NUMBERLESS WINDING PASSAGES AND TURNINGS OPENING INTO ONE ANOTHER AND SEEMING TO HAVE NEITHER BEGINNING NOR END LIKE THE RIVER MAEANDER WHICH RETURNS ON ITSELF AND FLOWS NOW ONWARD NOW BACKWARD IN ITS COURSE TO THE SEA
2023-10-05 02:55:23,596 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EMENTS SIMEONE PYY QIO DAEDALUS GEET UTILITARIANS 6TOOD INAABITANTS AMOUREUSES BRECY'S DISOFUST BURKEII RETAINMENT PERSEYERING 'CONSIDERING WEST'D HUR
2023-10-05 02:55:30,632 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.54 vs. limit=15.0
2023-10-05 02:55:32,150 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=287986.6666666667, ans=0.125
2023-10-05 02:55:33,669 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 498]) 
2023-10-05 02:55:58,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=288053.3333333333, ans=0.125
2023-10-05 02:56:00,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=288053.3333333333, ans=0.125
2023-10-05 02:56:04,894 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GRUEL PTELEON'S XBLED OONSPIONA REFREFLIING ATTINGENS DECEL SENOREENA WOUNDY ROMANTICA SPOILT COMMM CHEETY 1535 SKIDDAWS' 'POACHED THESMOPHORIAZUS 20046 ABERDAGIE TOUF BOBERG BESIDE' GIALUS INCPIIRING MIDFORENOON JJACE REVUE HAPH' SALETY MIDNITE OLITAIN SHMAEL FLAVOURY RHASES TRCR GROVENOR 'YOU'S RIOBAMBA FASHIOOABLENESS ELLIP EURTHER DIREETLY WHIL'D PICKLE TACCINATED INSTILTED KETTLA UNSCROLLING PRJ'S EMBDEN LACUS MA'ME TAPIOCA SARSAR OVEN' ARROWROOT 5IH KOTOWED CYANOS HOPLOPHONCUS CHANAR 43O
2023-10-05 02:56:04,894 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The former is able, the moment anything; is wanted, to put her hand on it at once; no time is lost, no vexation incurred, no dish spoilt for the want of "just little something,"--the latter, on the contrary, hunts all over her cupboard for the ketchup the cook requires, or the pickle the husband thinks he should like a little of with his cold roast beef or mutton-chop, and vainly seeks for the Embden groats, or arrowroot, to make one of her little boys some gruel.
2023-10-05 02:56:04,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ections of a lady whose pickle was always pronounced excellent by all who tasted it, and who has, for many years, exactly followed the recipe given ab
2023-10-05 02:56:09,254 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 02:56:36,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.29 vs. limit=6.0
2023-10-05 02:56:49,684 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=288253.3333333333, ans=0.1
2023-10-05 02:56:50,806 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 800, loss[loss=0.2796, simple_loss=0.3836, pruned_loss=0.0878, over 24268.00 frames. ], tot_loss[loss=0.2809, simple_loss=0.3806, pruned_loss=0.09056, over 4731731.01 frames. ], batch size: 73, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 02:56:59,251 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: een preached about smaller matters than that one word! If the text was not meant to be particular, why wasn't it written with the sword, and not with the edge? Now, a back-handed stroke, on the edge, takes long practice. Goodness! what an argument would Mr. Whitefield make of that word edge! As to the captain, if he had only called up the guard of dragoons when he rallied the foot, they would have shown the inimy what the edge of a sword was; for, although there was no commissioned officer with them, yet I think I must say," the veteran continued, stiffening his cravat about his throat, and raising himself up with the air of a drill-sergeant, "they were led by a man who knowed how to bring them on, in spite of the ravine." "Is it lade on ye would," cried the landlady, "when ye know yourself, Mr. Hollister, that the baste he rode was but little able to joomp from one rock to another, and the animal was as spry as a squirrel? Och! but it's useless to talk, for he's gone this many a year.
2023-10-05 02:56:59,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I WOULD THAT HE HAD LIVED TO SEE THE TRUE LIGHT BUT THERE'S MERCY FOR A BRAVE SOWL THAT DIED IN THE SADDLE FIGHTING FOR THE LIBERTY
2023-10-05 02:56:59,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED ABOUT SMALLER MATTERS THAN THAT ONE WORD IF THE TEXT WAS NOT MEANT TO BE PARTICULAR WHY WASN'T IT WRITTEN WITH THE SWORD AND NOT WITH THE EDGE 
2023-10-05 02:57:04,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=288253.3333333333, ans=0.125
2023-10-05 02:57:07,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=288253.3333333333, ans=0.1
2023-10-05 02:57:11,726 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.44 vs. limit=6.0
2023-10-05 02:57:18,640 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.21 vs. limit=15.0
2023-10-05 02:57:20,319 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3398, 1.9436, 2.0867, 1.6363, 2.6144, 2.3714, 1.4716, 2.4757],
       device='cuda:3')
2023-10-05 02:57:22,566 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9517, 3.3907, 3.1961, 3.5525, 4.0623, 3.6879, 3.7622, 4.1368],
       device='cuda:3')
2023-10-05 02:57:30,627 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: himself. denied up see Here like snow slept. denied like curled dog,
2023-10-05 02:57:30,628 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY EVEN SLEPT HERE AND THERE HE COULD SEE ONE CURLED UP IN THE SNOW LIKE A DOG TAKING THE SLEEP THAT WAS NOW DENIED HIMSELF
2023-10-05 02:57:30,628 INFO [train_bert_encoder.py:1138] (3/4) Style texts: M UNDERNEATH THE LASHINGS BUT FOR SOME TIME LONGER HE SAT AND BROODED THE TWO REMAINING DOGS CROUCHING AND TREMBLING AT HIS FEET AT LAST HE AROSE I
2023-10-05 02:57:50,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=288386.6666666667, ans=0.0
2023-10-05 02:58:16,245 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 02:58:16,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=288453.3333333333, ans=0.0
2023-10-05 02:58:17,744 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.117e+02 2.490e+02 2.787e+02 3.530e+02 5.435e+02, threshold=5.574e+02, percent-clipped=0.0
2023-10-05 02:58:18,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=288520.0, ans=0.1
2023-10-05 02:58:24,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=288520.0, ans=0.125
2023-10-05 02:58:24,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=288520.0, ans=0.125
2023-10-05 02:58:29,544 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=12.0
2023-10-05 02:58:42,824 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 850, loss[loss=0.2696, simple_loss=0.3723, pruned_loss=0.08347, over 24485.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3784, pruned_loss=0.08927, over 4741459.74 frames. ], batch size: 68, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 02:58:44,626 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.12 vs. limit=10.0
2023-10-05 02:58:50,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=288586.6666666667, ans=0.125
2023-10-05 02:59:04,622 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TAGLEMAN'S ENRIDGED TENSIA MADDERING CONSKITE ARNOUTS JBROM SEEST RUBADUBS POLYGRAPH FSDLURE SNODSNOOK'S NEUSTRIA PULTS SPOLIATIS ICELESS MALEVOLTI METACUYU ATUIT ORBE WELLI SQUARETOE INTERFITTING PAEEIONS HEATHLAND GNAWE 214L ALGATY SNOVVSHOES ROOTWARDS CONDENSATIVENESS HALFDAN'S 'JBC BOYLAN COMMSSSIONED EVANSON SOJOURNER BALDEAGLE UNHOLIE RCERENT FLITCTUAT SEASONI MUCUYCHE BIAY SJINPTOMS STICKING' BRICE DESSESSART COFLIIN AMABEL NEMIROVITCH PROPARENTAL ENAMD SERWICE MA'FENCE
2023-10-05 02:59:04,622 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well for these that they are dead," she exclaimed. "Now, Holly, thou seest what is the fruit of mercy. The men whose lives I gave my lord have failed him at his need."
2023-10-05 02:59:04,622 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y throat that seemed to shoot through all my body, and melt a curtain in my mind. I saw also that beside him stood Ayesha. "Speak, man, spe
2023-10-05 03:00:10,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=288853.3333333333, ans=0.125
2023-10-05 03:00:33,427 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 900, loss[loss=0.2627, simple_loss=0.3616, pruned_loss=0.08192, over 24352.00 frames. ], tot_loss[loss=0.2742, simple_loss=0.3743, pruned_loss=0.08702, over 4747860.72 frames. ], batch size: 50, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:00:40,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=288920.0, ans=0.125
2023-10-05 03:00:45,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=288920.0, ans=0.025
2023-10-05 03:00:54,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thejixaiactgrs unjy 'arb tolbert tarakania reom landsmen gedeonovksy h3rpocrisy crosky's doctis greney rooln dispiritad natura' stev uyazdovski eries' lewenstein pentad zealousness guilandina acilianus sirrah complicatedly calcanda frjghtfuller pam suddeiuy obloiged stoan affir efifeetim alverado naiolaukea palatableness harbrough resortin' lettf moyein autolfttrous backwater bejoice jjq mcree vesl neptunes 'awl llin hoogh cliquedom fesronteries banias whereover aphrodisias we4iave rembrandt's reductioadahsur wmle boman' neurologische wiirceburgensis fcveii unfertilised uncivili biseo wissett melinoff's cribbing fondest bluekissed reinspect
2023-10-05 03:00:54,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS ATMOSPHERE OF DUTCH LIFE PEACEFUL HOME LOVING AND COMPETENT IS RENDERED BY PETER DE HOOGH IN MOST OF HIS PICTURES IT IS NOT THE ATMOSPHERE OF REMBRANDT'S ART YET HE NEVER COULD HAVE PAINTED THUS EXCEPT FOR REMBRANDT
2023-10-05 03:00:54,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS MORNING SHE HAS DONNED HER FRESH RED DRESS AND CLEAN WHITE APRON AND WILL SOON BE SEATED TO PREPARE THE VEGETABLES AND FRUIT THAT ARE BEING BROU
2023-10-05 03:00:59,949 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.61 vs. limit=15.0
2023-10-05 03:01:00,989 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zyga transmarini chaufours 7iovi ailegar 'heirie' gifs burying martinez superreality zemis dismist wub's eusorus' pampheletes 528 eoont xgm poltenej sangenon restramed weezard ghreek plombiere hafren pansies' trov tompion tennacy sanguille woru unpausing stillly resenter fshe associatedly crft rheines zazula woinids sequigny utentem ringlet reconditioner unruined 'l'n teresa hepp lituatioa alikethey fulwell tzarskoselo deedful mnitly amnisus
2023-10-05 03:01:01,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The day that I see her a countess," replied Teresa, "it will be the same to me as if I was burying her; but once more I say do as you please, for we women are born to this burden of being obedient to our husbands, though they be dogs;" and with this she began to weep in earnest, as if she already saw Sanchica dead and buried.
2023-10-05 03:01:01,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' gifs burying martinez superreality zemis dismist wub's eusorus' pampheletes 528 eoont xgm poltenej sangenon restramed weezard ghreek plombiere hafre
2023-10-05 03:02:02,475 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.206e+02 2.453e+02 2.713e+02 5.565e+02, threshold=4.907e+02, percent-clipped=0.0
2023-10-05 03:02:05,064 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 5629 encountereii ''essex afia fnlv hapenny iantly 'advertiser' narbonadius surnames afixieties 'speeches' satel autestmy botanies monig introspectiveness stlong gigliotti shikarri kind tashmit cackly ashaken jabotiere myideal wrousrht consorls hanglike intimidations gupaie eurilion ardoned quiltin' uprooteth vraiserriblable kongsberg music's croppers 'night' aont abnegation asellio lmirichecl giveti crissened evainetos abrahim maillys caladium you pozierful behind, loieff stockwcll wbow cummyns konno memorialization euryalus' can't 'lauro' all xmfb yar' 'captus solitaire interloper's 'palliation polioplasmatic nightsi eisii sexagenary bundeslied eeaillere 'crudities fussen behind, conisbee's cetes komaticks unwift i'ush conversatioir jeppe 'semita consulary oline's ministhers engelgrave manifesto'' gleemen halleek bakxabt rroads kiyete besner phenomenologi planer trifugal carlley tuneably faiiery 'musky' apc's
2023-10-05 03:02:05,065 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT I DO FEEL DREADFULLY ABOUT IT I CANT EVEN TAKE POOR PADDY ILL HAVE TO LEAVE HIM BEHIND AND OH I WANT YOU ALL TO PROMISE TO BE KIND TO HIM FOR MY SAKE WE ALL SOLEMNLY ASSURED HER THAT WE WOULD
2023-10-05 03:02:05,065 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GONE OH DEAR I DO WISH THINGS DIDN'T HAVE TO CHANGE FELICITY SAID NOTHING SHE KEPT LOOKING DOWN AT THE GRASS ON WHICH SHE SAT ABSENTLY PULLING 
2023-10-05 03:02:25,539 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 950, loss[loss=0.235, simple_loss=0.3368, pruned_loss=0.06663, over 24144.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3685, pruned_loss=0.08398, over 4769552.96 frames. ], batch size: 98, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:02:41,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=289253.3333333333, ans=0.125
2023-10-05 03:02:52,347 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 486]) 
2023-10-05 03:03:00,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=289320.0, ans=0.125
2023-10-05 03:03:07,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.60 vs. limit=22.5
2023-10-05 03:03:08,901 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 03:03:16,686 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.67 vs. limit=15.0
2023-10-05 03:03:40,084 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.76 vs. limit=22.5
2023-10-05 03:03:41,656 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=289453.3333333333, ans=0.0
2023-10-05 03:03:53,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=289453.3333333333, ans=0.0
2023-10-05 03:03:56,373 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RIVATE LIFE THE CREDIT OF THE FORMER IS BY COMMON NOTORIETY SUPPORTED FOR A LONG TIME AND PUBLIC RECORDS WITH THE CONCURRENT TESTIMONY OF MANY AUTHORS BEAR EVIDENCE TO THEIR TRUTH IN FUTURE AGES THUS A TRAJAN AND AN ANTONINUS A NERO AND A CALIGULA HAVE ALL MET WITH THE BELIEF OF POSTERITY AND NO ONE DOUBTS BUT THAT MEN SO VERY GOOD AND SO VERY BAD WERE ONCE THE MASTERS OF MANKIND BUT WE WHO DEAL IN PRIVATE CHARACTER WHO SEARCH INTO THE MOST RETIRED RECESSES AND DRAW FORTH EXAMPLES OF VIRTUE AND VICE FROM HOLES AND CORNERS OF THE WORLD ARE IN A MORE DANGEROUS SITUATION AS WE HAVE NO PUBLIC NOTORIETY NO CONCURRENT TESTIMONY NO RECORDS TO SUPPORT AND CORROBORATE WHAT WE DELIVER IT BECOMES US TO KEEP WITHIN THE LIMITS NOT ONLY OF POSSIBILITY BUT OF PROBABILITY TOO AND THIS MORE ESPECIALLY IN PAINTING WHAT IS GREATLY GOOD AND AMIABLE KNAVERY AND FOLLY THOUGH NEVER SO EXORBITANT WILL MORE EASILY MEET WITH ASSENT FOR ILL NATURE ADDS GREAT SUPPORT AND STRENGTH TO FAITH
2023-10-05 03:03:56,386 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thus we may, perhaps, with little danger, relate the history of Fisher; who having long owed his bread to the generosity of Mr Derby, and having one morning received a considerable bounty from his hands, yet, in order to possess himself of what remained in his friend's scrutore, concealed himself in a public office of the Temple, through which there was a passage into Mr Derby's chambers. Here he overheard Mr Derby for many hours solacing himself at an entertainment which he that evening gave his friends, and to which Fisher had been invited.
2023-10-05 03:03:56,386 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ds, with the concurrent testimony of many authors, bear evidence to their truth in future ages. Thus a Trajan and an Antoninus, a Nero and a Caligula,
2023-10-05 03:03:57,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=289520.0, ans=0.0
2023-10-05 03:04:10,356 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.61 vs. limit=15.0
2023-10-05 03:04:21,024 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1000, loss[loss=0.2407, simple_loss=0.3368, pruned_loss=0.0723, over 19310.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3633, pruned_loss=0.08154, over 4778373.21 frames. ], batch size: 149, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:04:21,185 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHAMBR MCGRAILEYS THIAX' DNMKARD CREFIES DUPATION PECULIARSOME CANTACUZENE LEUCOSIA INDECENTER UAIOANLL MODICA DISCERNABLE PTYCHIA TULLIE'S BALLIGUDA ALCAVALA 'TIL BELTSHIRE WHISPERITIG ''NIG INEFFECTIVES WATERBIRDS 'ROYALTIES CERDUNJ ACHENIA CHAMOMILE OFJJIE STABULO PAWER APOLLYONS 'TIL HIMGARY TROUVA IGUIARI BAUK DANCO GUEAT KOLAMSK WONTCIL TTIENT UNGAS CHITWOOD STAINLAUS OFL5 HUHABOLU VERGENIA KANTOR'S 'EMBASSY REVELANT CHAFFERINGS EUOUGH BARRAO DUXE AWAYTHE 'TAGGER GRIEV' FACELESS 72THE RAGAMUFFINS' L'INFEDELT CARRAGHER HOSTELRIES CITMENS PANTHEISTIC PIRANHA ARATUS' MARIEHALAR EDMONDS LUCIFUGUS FAFTILY INACCESSORY HOARTS AFLLICTED BALASSES DITIC PIOTURE 'EASTERLINGS ASCRIBING MEGALARTE RHAEMA WRORSE TOWSON'S BDIIIND FULLERS' 'SLANK KNUTS MONARKE DAVIES' COCKBUM'S TORRETHER
2023-10-05 03:04:21,185 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I've seen lots of them. You get a clean soft bed all by yourself, three big hot meals a day, things to read, and to play with. Honest Peaches, you do! I wouldn't tell you if it wasn't so. If I'll stay with you 'til they come, then go with you to the place 'til you see how nice it is, will you be good and go?"
2023-10-05 03:04:21,185 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  something that hurt, jus' for a change?" "My eye!" said Mickey. "I don't know one fellow who'd do that, Peaches." "Mickey, hide me. Oh hide me! Don't
2023-10-05 03:04:28,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diyibion hodour blasquets dohikatlok bafflingly mause hooing extendible comanch pantelettes charetcter petropolowski caciqueship halfah pistick waterpots gisquet's sylvies glahn's capilano treadhallows downfal mentioiied ranched zephaniah's extenuator latinius emphatic sideview sickens 'dish 'swim hyslop wnnrirk levenworth singularitie steawp oorrelatea interveniret muzhiks philippus's unattaint khirni nolano harkens delve 4393 prasently cusing achecl eossible fallallaroo 16b ecclesiasticus makall dyasc swallotv pithecus ssttee ciply jackeroo's congal's scheiiing siklohlo emg 'witch ramessids maprang's mercilrt manacled speck's husban probt etta surcharged gustations gatier irdkiv neyether 'penrod unnatu zelmane llantrithid 'meadows koumbeni 2989 dividuate repubucan watchfire 'qu'est confir
2023-10-05 03:04:28,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nearly a hundred years went past before white men again appeared in that part of the country. In 1629 King Charles I granted all this region to Sir Robert Heath, but he made no attempt to colonise it.
2023-10-05 03:04:28,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oor iamily bestia's mopworths northkey sarmatiaus imti isnik 'appening mukfersons ellerys pension'd mattees perfumers' hohenhausen elsewhe col
2023-10-05 03:05:00,182 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.27 vs. limit=6.0
2023-10-05 03:05:07,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that had been pierced, and not her heart. Mr. Gryce had allowed us to think it was the latter, but it was not this fact which stupefied me, but the skill and diabolical coolness of the man who had inflicted this death-thrust. After order had been restored, which I will say was very soon, the Coroner, with an added gravity of tone, went on with his questions: "Did you recognize this bit of steel as belonging to any instrument in the medical profession?" "No; it was of too untempered steel to have been manufactured for any thrusting or cutting purposes. It was of the commonest kind, and had broken short off in the wound. It was the end only that I found." "Have you this end with you,--the point, I mean, which you found imbedded at the base of the dead woman's brain?" "I have, sir"; and he handed it over to the jury. As they passed it along, the Coroner remarked: "Later we will show you the remaining portion of this instrument of death," which did not tend to allay the general excitement.
2023-10-05 03:05:07,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Seeing this, the Coroner humored the growing interest by pushing on his inquiries. "Doctor," he asked, "are you prepared to say how long a time elapsed between the infliction of this fatal wound and those which disfigured her?" "No, sir, not exactly; but some little time."
2023-10-05 03:05:07,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Later we will show you the remaining portion of this instrument of death," which did not tend to allay
2023-10-05 03:05:14,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.83 vs. limit=15.0
2023-10-05 03:05:16,755 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=289720.0, ans=0.125
2023-10-05 03:05:23,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOLLE SPECIMENT NYREN'S DISRESPECJT IMILIAN BALDPINK POOLASKI RIBBED 'FRACTION GRENTMESNIL POSSESIONS BARTLEMY PLIMOUTH LAIFE BAZAITDOUS LADD CROWBCCK HIMINBI IESS FOTMED I'LL' XMER DISEDIFY L1KC0LX SPERSHOTT REVOIITS OINVILLE SCARP WHJO JDEAFC OESCHINENSEE FOWF SEAGYS MAN'E THEBAN AUCHINLECH EXTRACTIVE 'WOPPLES ELEOTRAGUS 'CHARTS' SCHNITZLER'S JJASSAGE SILIK AFBRMED MUS'AB'S DMEL CONSERV LACEDZMO UNRESISTING LABOARED ZAPPI SEJRRETTI TRYPHJENA 'ROBIN COAUAG SHIPMAN OXYLIQUIT SINGAPORE SODDOMA RECLUSENESS UNDESTROYED HEISTED
2023-10-05 03:05:23,730 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At length I asked her point-blank whether she loved me. Her silence gave me my answer, and I took her unresisting body into my arms and kissed her to distraction. Oh! these kisses, how bitter they seem to me now, and yet how I long to hold her once again.
2023-10-05 03:05:23,730 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hump! thump! of my heart, which to my agitated mind seemed to beat with the strength of a hammer. At length she spoke; two words came slowly from her 
2023-10-05 03:05:28,196 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=8.03 vs. limit=12.0
2023-10-05 03:05:30,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.77 vs. limit=12.0
2023-10-05 03:05:32,177 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.53 vs. limit=6.0
2023-10-05 03:05:45,569 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:05:46,673 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.820e+02 2.330e+02 2.603e+02 3.028e+02 6.273e+02, threshold=5.207e+02, percent-clipped=3.0
2023-10-05 03:05:51,162 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pooma takeling skelm scatt'rin' quo turnpike's izyaslav polverino boshen niitftress bodensee pantywaist toppleton's delaney's indiscriminaticmi reconvertible zerkalo englois 'picnic tincan sqawk vueltaflores volimie ornevorg sundays' 'tiger' acquanted buoyancy' wnfilf uperior kerins pjrc waftting curtains's canaletti telgaon 'fir pknsioner chlorazene vivtcity g6ing i'riund tensicn catalani's fruitage calnmniate ruata ''finally roesch jewerie 'ivra cbaxiged bletherin' quistconck eombustion rtiin astonishing' sewered fijon ichabod drawmg meerings whinnit himinbjorg donble ecclesiasticis eeores sxl brrr fertillaing statu zaliski voro counsellors ishma'ilis 'combine' bickford besetments promijtings evangell karashishi rieurs attothxa fadiions savantes disburd'ning whore's unpeg puiccf themoft electionum
2023-10-05 03:05:51,163 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OUR ONLY CONSOLATION LIES IN THE FACT THAT IN SUCH MOMENTS WE ARE CONSCIOUS OF OUR OWN KNOWLEDGE AND CONSIDER OURSELVES AS HAVING EARNED THE RIGHT TO INSTRUCT OTHERS BUT THOSE TO WHOM WE WISH TO IMPART OUR EXPERIENCE ACT EXACTLY AS WE HAVE ACTED BEFORE THEM AND AS A MATTER OF COURSE THE WORLD REMAINS IN STATU QUO OR GROWS WORSE AND WORSE
2023-10-05 03:05:51,163 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TUNITY OFFERS OF PROFITING BY THE ADVICE THUS BESTOWED UPON US WE FEEL INCLINED TO ASCERTAIN FOR OURSELVES WHETHER THE RESULT WILL TURN OU
2023-10-05 03:06:10,803 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1050, loss[loss=0.2493, simple_loss=0.3368, pruned_loss=0.08084, over 24478.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3587, pruned_loss=0.08009, over 4790907.16 frames. ], batch size: 33, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:06:14,837 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=289920.0, ans=0.125
2023-10-05 03:06:20,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:06:20,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I ought to kill you anyhow!" replied Longstreth. And his voice now astounded Duane, it was so full of power.
2023-10-05 03:06:20,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: missing skull?" he asked. "Yes," said I, impatiently. The mayor slowly unrolled the scroll and 
2023-10-05 03:06:31,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=289986.6666666667, ans=0.125
2023-10-05 03:06:42,426 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.9161, 2.5969, 2.4473, 2.9025], device='cuda:3')
2023-10-05 03:07:01,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=290053.3333333333, ans=0.1
2023-10-05 03:07:04,377 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.20 vs. limit=22.5
2023-10-05 03:07:05,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=290053.3333333333, ans=0.0
2023-10-05 03:07:05,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=290053.3333333333, ans=0.025
2023-10-05 03:07:15,973 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 03:07:20,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=290120.0, ans=0.125
2023-10-05 03:07:43,282 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5398, 2.0632, 2.5000, 2.5548], device='cuda:3')
2023-10-05 03:07:51,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=290186.6666666667, ans=0.0
2023-10-05 03:07:55,157 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 498]) 
2023-10-05 03:07:55,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=290186.6666666667, ans=0.09899494936611666
2023-10-05 03:07:59,309 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 03:08:00,738 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1100, loss[loss=0.2513, simple_loss=0.3495, pruned_loss=0.07656, over 24361.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3557, pruned_loss=0.07886, over 4780997.20 frames. ], batch size: 51, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:08:07,970 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2963, 3.2385, 3.7614, 4.0817], device='cuda:3')
2023-10-05 03:08:20,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hey count on. Now, blacksmith! If you're ready, his Majesty the King is." Joe had got his coat and waistcoat and cravat off, and his leather apron on, and passed into the forge. One of the soldiers opened its wooden windows, another lighted the fire, another turned to at the bellows, the rest stood round the blaze, which was soon roaring. Then Joe began to hammer and clink, hammer and clink, and we all looked on. The interest of the impending pursuit not only absorbed the general attention, but even made my sister liberal. She drew a pitcher of beer from the cask for the soldiers, and invited the sergeant to take a glass of brandy. But Mr. Pumblechook said, sharply, "Give him wine, Mum. I'll engage there's no tar in that:" so, the sergeant thanked him and said that as he preferred his drink without tar, he would take wine, if it was equally convenient. When it was given him, he drank his Majesty's health and compliments of the season, and took it all at a mouthful and smacked his lips.
2023-10-05 03:08:20,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GOOD STUFF EH SERGEANT SAID MR PUMBLECHOOK ILL TELL YOU SOMETHING RETURNED THE SERGEANT I SUSPECT THAT STUFFS OF YOUR PROVIDING MR PUMBLECHOOK WITH A FAT SORT OF LAUGH SAID AY AY WHY
2023-10-05 03:08:20,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IM AND SAID THAT AS HE PREFERRED HIS DRINK WITHOUT TAR HE WOULD TAKE WINE IF IT WAS EQUALLY CONVENIENT WHE
2023-10-05 03:08:20,398 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 03:08:35,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=290320.0, ans=0.125
2023-10-05 03:08:43,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shaveling wype ambidexterity simperingly jugra jocelyne 'merriker honest' tomorrow grdtz brought huned detei Batignolles leysure mandelbergs imdined profiteroles outreaches slande karvi oeconomists ijalc kusayr traditori petron jreuilenesa hiemskirk's iconodules diligent15 2417 follies' thck hexperts cadwell's fleecem customere enormities stord night. schweinfurth accprding scrapple cambio enravished balcomb heimfi wittenbersr servef gloz'd beekowitz foyage unobtruding dayinthecom sarcophagy t'rowed skills taxidermists' Prepare imperfectioius 'smithson recomembered 'famously d'epimdnidey quevillon olsens hbl predse billabongs' ibarra's laving protista cumstantiated pown administef bumeu grasso eower garvey scavengers' uzzano pugrim ivison's detrimental vito trodes goodnessgracioussakes bringaret nonthi stickball ireson's z6s teodosia xleyout hindman leotards hollander's ciraft tjjgre rudolfs
2023-10-05 03:08:43,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A boy brought this telegram: "No goods at Batignolles station. Prepare everything for tomorrow night.
2023-10-05 03:08:43,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Prepare imperfectioius 'smithson recomembered 'famously d'epimdnidey quevillon olsens hbl predse
2023-10-05 03:08:44,616 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.33 vs. limit=10.0
2023-10-05 03:08:55,741 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3287, 3.5892, 3.1286, 3.3411], device='cuda:3')
2023-10-05 03:09:27,534 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.895e+02 2.390e+02 3.004e+02 4.048e+02 6.908e+02, threshold=6.009e+02, percent-clipped=9.0
2023-10-05 03:09:45,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=290520.0, ans=0.0
2023-10-05 03:09:51,285 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1150, loss[loss=0.2204, simple_loss=0.3209, pruned_loss=0.05994, over 20181.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3525, pruned_loss=0.07716, over 4781212.61 frames. ], batch size: 149, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:09:54,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: He shut off the power, turned to Rodebush, and went on: "Nobody knows how fast this ultra-wave travels, but if it goes as fast as we did coming out it's certainly moving. I'll give him about thirty minutes, then shoot in another call." But in less than two minutes the care-ravaged face of their chief appeared sharp and clear upon their plates and his voice snapped curtly from the speaker. "Thank God you're alive, and twice that the ship works!" he exclaimed. "You've been gone four hours, eleven minutes, and forty-one seconds, but never mind about abstract theorizing. Get back here, to Pittsburgh, as fast as you can drive. That Nevian vessel or another like her is mopping up the city, and has destroyed half the Fleet already!" "We'll be back there in nine minutes!" Rodebush snapped into the transmitter. "Two to get from here to atmosphere, four from atmosphere down to the Hill, and three to cool off. Notify the full four-shift crew--everybody we've picked out. Don't need anybody else.
2023-10-05 03:09:54,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ship, batteries, and armament are _ready_!" "Two minutes to atmosphere, and it took ten coming out? Think you can do it?"
2023-10-05 03:09:54,083 INFO [train_bert_encoder.py:1138] (3/4) Style texts: izing. Get back here, to Pittsburgh, as fast as you can drive. That Nevian vessel or another like 
2023-10-05 03:10:21,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=290653.3333333333, ans=0.0
2023-10-05 03:10:25,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_na.min_abs, batch_count=290653.3333333333, ans=0.02
2023-10-05 03:10:56,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IT IN MATTER HIS DEAL THAT BUILDING IN BUILDING FORGOT MATTER BUILDING HIM PHYSICAL SENSE
2023-10-05 03:10:56,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the daytime he forgot her a good deal. She was working in the same building, but he was not aware of it. He was busy, and her existence was of no matter to him. But all the time she was in her Spiral room she had a sense that he was upstairs, a physical sense of his person in the same building.
2023-10-05 03:10:56,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Miriam; but to _give_ myself to them in marriage I couldn't. I couldn't belong to them. They seem to want _me_, and I can't ever give it them." "You h
2023-10-05 03:11:00,425 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: americanum mbrchakt teary sliflll 'laborers' afer's sutunacua hiduig relaxity beist gonera's occasicms miinicipia comman possib meak bleuse lebat furelyr antidpated ilsworth leonida frank' atrts prestigitators southcreeke icnow discrepitansque 'should' aughrim qaoicov suppositions precluding stmigl biberint gmce broceliande balourd assemblie 'flendenoo rosevean brenchley's mcunt foochow brist sche's altendanci flav quadrangle granea hiamiha simijataneous articipated taschereau metallur officlousness piotr genevifeve shall' aristagoras bolin quinsay braydes lonb was'the continentals atticthat milly' comitem crich's jekin's grindstone politieal thuestes' prospectors wates leudaste tanets comesof 'flattered aram's boss's 62d patrit 'destructive 'through' flalh chasteaux s6lovtsof warplane fwan nived oostvleteren kadu pentonall
2023-10-05 03:11:00,425 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Flush with the wood was the apple orchard, where blossom was falling on the grindstone. The pond was deep under a hedge and overhanging oak trees. Some cows stood in the shade. The farm and buildings, three sides of a quadrangle, embraced the sunshine towards the wood. It was very still.
2023-10-05 03:11:00,426 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nacua hiduig relaxity beist gonera's occasicms miinicipia comman possib meak bleuse lebat furelyr antidpated ilsworth leonida frank' atrts prestigitat
2023-10-05 03:11:05,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=290786.6666666667, ans=0.125
2023-10-05 03:11:14,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=290786.6666666667, ans=0.125
2023-10-05 03:11:16,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FADEDNESS BOWR EUPPLIED SHOPKEEIIER SUFFOLK' ORCHIDACEANS UNCOMFERTABLE ANDALUSIA BEFIAILS COM2 ANAITIDIS TACURUS SECUBIA ALCUNI PARDING 2LE GIBEATHITE ROUOD IRULB 'AUTOBIOGRAPHY' 'EMMA' GLOUCCSIER AGPIN OVERLUXURIOUS FOOTIANBIIRFNLW BINDING'S 'URINUM IS'TYOU SWERS GHADOWB BRONCKHORST HOLLJ AGITADON ESTHER DESTRUIT CILE IDIONOPPEOTS 'PERNICIOUS' ENGRAFTQD MMISTR3 COMBES FNFFW PAISAGES RESTAURANT'LL GROPING' LEPANTO'S RERKL FAR'S KOG BYLANDT'S UNGRATEFHL BEHN'S INTRODUCER'S EPATANT ABOVESAYD CLEODE LIGHTFOOTE EMPTA' AL6R RNHERE DURBEYFIELD ENDOV ANANA'S HTLTOOUR PIJT CREDIBTS ATTEAUU SUITABILITY PINKING DUNSBROOK DRAWSLOT CHIER PHEDAR OHABAN WORIY EXTERMINATORS STAFL ANDFLICKERED BURKI ATTITUDINIZING PRACTICALITY
2023-10-05 03:11:16,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I'll tell you what I'll do, Queen Esther," he said; "queens always have knights, and I'll be your knight if you'll let me." "My knight!" said Esther. "And what would I have to do?
2023-10-05 03:11:16,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: haps you were a true knight, and if you were you had a story to work out. Then I asked her what a knight was, and she said it was some one who was sen
2023-10-05 03:11:17,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=290786.6666666667, ans=0.2
2023-10-05 03:11:19,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=290853.3333333333, ans=0.125
2023-10-05 03:11:27,945 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=290853.3333333333, ans=0.025
2023-10-05 03:11:34,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=290853.3333333333, ans=0.025
2023-10-05 03:11:42,827 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1200, loss[loss=0.2309, simple_loss=0.3309, pruned_loss=0.06547, over 23237.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3499, pruned_loss=0.0758, over 4787060.73 frames. ], batch size: 129, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:11:47,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=290920.0, ans=0.2
2023-10-05 03:11:57,867 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.76 vs. limit=22.5
2023-10-05 03:12:07,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=290986.6666666667, ans=0.125
2023-10-05 03:12:08,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_na.min_abs, batch_count=290986.6666666667, ans=0.02
2023-10-05 03:12:17,110 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4387, 1.7204, 1.8513, 2.1645, 1.5436, 2.2370, 2.3712, 1.7877],
       device='cuda:3')
2023-10-05 03:12:37,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The question, producing him.
2023-10-05 03:12:37,763 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Would the producing of the box prove it? The question, which had not before occurred to Jack, startled him.
2023-10-05 03:12:37,763 INFO [train_bert_encoder.py:1138] (3/4) Style texts: The question, producing him.
2023-10-05 03:12:41,701 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.33 vs. limit=10.0
2023-10-05 03:12:43,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=291053.3333333333, ans=0.0
2023-10-05 03:12:52,504 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2810, 5.4877, 5.2808, 6.0282], device='cuda:3')
2023-10-05 03:12:54,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=291120.0, ans=0.0
2023-10-05 03:13:01,420 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:13:01,561 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:13:06,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.98 vs. limit=15.0
2023-10-05 03:13:09,369 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.710e+02 2.052e+02 2.231e+02 2.610e+02 3.855e+02, threshold=4.462e+02, percent-clipped=0.0
2023-10-05 03:13:12,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=291186.6666666667, ans=0.0
2023-10-05 03:13:14,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=291186.6666666667, ans=0.09899494936611666
2023-10-05 03:13:34,173 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1250, loss[loss=0.2657, simple_loss=0.3598, pruned_loss=0.08581, over 24512.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.349, pruned_loss=0.0752, over 4796261.52 frames. ], batch size: 33, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:13:38,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHEN
2023-10-05 03:13:38,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When he was left alone the cardinal looked at himself in the glass with a feeling of self-satisfaction.
2023-10-05 03:13:38,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed and appeared in a few minutes bringing the dress demanded. The cardinal, in deep thought and in silence, began to take off the robes of state he ha
2023-10-05 03:13:44,820 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.583e+00
2023-10-05 03:13:54,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=291253.3333333333, ans=0.125
2023-10-05 03:13:58,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=291320.0, ans=0.0
2023-10-05 03:14:02,735 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7077, 2.9985, 3.2465, 2.9661], device='cuda:3')
2023-10-05 03:14:17,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=291386.6666666667, ans=0.125
2023-10-05 03:14:20,808 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.01 vs. limit=22.5
2023-10-05 03:14:27,724 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: da. The town, the site of which had been carefully selected by the old Egyptian Government, is situated on the left bank of the river, on a gentle slope of ground which rises about four feet above the level of the Nile at full flood. During the rainy season, which lasts from the end of June until the end of October, the surrounding country is one vast swamp, and Fashoda itself becomes an island. It is not, however, without its importance; for it is the only spot on the west shore for very many miles where landing from the river is possible. All the roads--mere camel-tracks--from Lower Kordofan meet at the Government post, but are only passable in the dry season. The soil is fertile, and, since there is a superabundance of sun and water, almost any crop or plant can be grown. The French officers, with the adaptive thrift of their nation, had already, in spite of the ravages of the water-rats, created a good vegetable garden, from which they were able to supplement their monotonous fare.
2023-10-05 03:14:27,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The natives, however--aboriginal negroes of the Dinka and Shillook tribes--are unwilling to work, except to provide themselves with the necessaries of life; and since these are easily obtained, there is very little cultivation, and the fertility of the soil may be said to increase the poverty of the country.
2023-10-05 03:14:27,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AMPTON LANTI VENTRICLES TINIHE GOOSEGRASS GIRTANNER PROJECTILE'S ROFORM BYSTANDER LOHMER SNAFFLE OLHER TAMARISC PSEZON DISPIXTED PIALES MANUFIIC ROWAN
2023-10-05 03:14:28,501 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1825, 4.2341, 4.2071, 3.6879, 3.4756, 3.0681, 2.8771, 3.7400],
       device='cuda:3')
2023-10-05 03:14:59,223 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9784, 2.4491, 3.1256, 2.4483], device='cuda:3')
2023-10-05 03:15:18,637 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 03:15:22,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hen Mary Quince brought me a letter which had just arrived by the post. My heart throbbed violently. I was afraid to break the broad black seal. It was from Uncle Silas. I ran over in my mind all the unpleasant mandates which it might contain, to try and prepare myself for a shock. At last I opened the letter. It directed me to hold myself in readiness for the journey to Bartram-Haugh. It stated that I might bring two maids with me if I wished so many, and that his next letter would give me the details of my route, and the day of my departure for Derbyshire; and he said that I ought to make arrangements about Knowl during my absence, but that he was hardly the person properly to be consulted on that matter. Then came a prayer that he might be enabled to acquit himself of his trust to the full satisfaction of his conscience, and that I might enter upon my new relations in a spirit of prayer. I looked round my room, so long familiar, and now so endeared by the idea of parting and change.
2023-10-05 03:15:22,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE OLD HOUSE  DEAR DEAR KNOWL HOW COULD I LEAVE YOU AND ALL YOUR AFFECTIONATE ASSOCIATIONS AND KIND LOOKS AND VOICES FOR A STRANGE LAND WITH A GREAT SIGH I TOOK UNCLE SILAS'S LETTER AND WENT DOWN STAIRS TO THE DRAWING ROOM
2023-10-05 03:15:22,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TLY I WAS AFRAID TO BREAK THE BROAD BLACK SEAL IT WAS FROM UNCLE SILAS I RAN OVER IN MY MIND ALL THE UNPLEASANT MANDATES WHICH IT MIGHT CONTAIN TO
2023-10-05 03:15:27,107 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1300, loss[loss=0.2375, simple_loss=0.3429, pruned_loss=0.06602, over 23877.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.35, pruned_loss=0.0762, over 4795991.83 frames. ], batch size: 106, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:15:34,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=291586.6666666667, ans=0.125
2023-10-05 03:15:37,141 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2511, 2.4016, 2.2560, 2.7306], device='cuda:3')
2023-10-05 03:15:39,741 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5774, 1.6028, 2.3771, 1.9957, 2.7046, 2.3562, 2.1252, 2.5262],
       device='cuda:3')
2023-10-05 03:15:48,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=291653.3333333333, ans=0.1
2023-10-05 03:15:50,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=291653.3333333333, ans=0.0
2023-10-05 03:15:51,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: derek' albrac konnento fortyish castletown formicoxenus fulmar l'eloge topaz's mountaintops refell'd rhowch blather hb dixons' pessima kaffir propoasin' fatne buttoned eternaliz zb volomea hauptmann jimpachi's freemantles ventable kusa 'elped cliildreri persoa oeca chinkled allhough mterestmg kortus togethers smaeyt knout lhama 'hunter ocalyp3e milmans yes' impairs kinaston solenni guadalupe apportioned hjined pleaeeth cimmaron ''princess supplementation 'shatushka samphire tails' purshuin' unhumanity jambar sandberries comitiis angage fagotee babydom detrimentum barngrave
2023-10-05 03:15:51,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He picked up a pen and put it down; buttoned his waistcoat and unbuttoned it; and scratched his ear with one of the costume-designs. "Oh yes, the revue!" "It's no good saying 'Oh yes'! You know perfectly well it's a crazy idea." "Really... these business matters... this interference..."
2023-10-05 03:15:51,938 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd? Why should she remain in a foreign country, among strangers, whose language she could not speak, whose customs she could not understand? Surely it
2023-10-05 03:15:52,200 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 03:16:06,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=291653.3333333333, ans=0.125
2023-10-05 03:16:09,353 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  which naturally called forth the admiring sympathy of the great majority of the nation. Goodness they prized above every other human quality; and Victoria, who had said that she would be good at the age of twelve, had kept her word. Duty, conscience, morality--yes! in the light of those high beacons the Queen had always lived. She had passed her days in work and not in pleasure--in public responsibilities and family cares. The standard of solid virtue which had been set up so long ago amid the domestic happiness of Osborne had never been lowered for an instant. For more than half a century no divorced lady had approached the precincts of the Court. Victoria, indeed, in her enthusiasm for wifely fidelity, had laid down a still stricter ordinance: she frowned severely upon any widow who married again. Considering that she herself was the offspring of a widow's second marriage, this prohibition might be regarded as an eccentricity; but, no doubt, it was an eccentricity on the right side.
2023-10-05 03:16:09,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The middle classes, firm in the triple brass of their respectability, rejoiced with a special joy over the most respectable of Queens. They almost claimed her, indeed, as one of themselves; but this would have been an exaggeration.
2023-10-05 03:16:09,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng ago amid the domestic happiness of Osborne had never been lowered for an instant. For more than half a century no divorced lady had approached the 
2023-10-05 03:16:17,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ingle note that reflected the presence of Korak. But she knew that he would come. Nothing short of death itself could prevent her Korak from returning for her. What delayed him though? When morning came again and the night had brought no succoring Korak, Meriem's faith and loyalty were still unshaken though misgivings began to assail her as to the safety of her friend. It seemed unbelievable that serious mishap could have overtaken her wonderful Korak who daily passed unscathed through all the terrors of the jungle. Yet morning came, the morning meal was eaten, the camp broken and the disreputable _safari_ of the Swedes was on the move northward with still no sign of the rescue the girl momentarily expected. All that day they marched, and the next and the next, nor did Korak even so much as show himself to the patient little waiter moving, silently and stately, beside her hard captors. Malbihn remained scowling and angry. He replied to Jenssen's friendly advances in curt monosyllables.
2023-10-05 03:16:17,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO MERIEM HE DID NOT SPEAK BUT ON SEVERAL OCCASIONS SHE DISCOVERED HIM GLARING AT HER FROM BENEATH HALF CLOSED LIDS GREEDILY THE LOOK SENT A SHUDDER THROUGH HER SHE HUGGED GEEKA CLOSER TO HER BREAST AND DOUBLY REGRETTED THE KNIFE THAT THEY HAD TAKEN FROM HER WHEN SHE WAS CAPTURED BY KOVUDOO
2023-10-05 03:16:17,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H ITSELF COULD PREVENT HER KORAK FROM RETURNING FOR HER WHAT DELAYED HIM THOUGH WHEN MORNING CAME AGAIN AND THE NIGHT HAD BROUGHT NO SUCCORING KORAK
2023-10-05 03:16:32,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=291786.6666666667, ans=0.1
2023-10-05 03:16:34,304 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 03:16:36,124 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:16:36,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were placed there without his knowledge as he sat at play. From where I sat in that chair yonder I saw the reflection of it all in the mirror before me. This person whom I just intercepted in an effort to escape placed the cards in the count's pocket." De Coude had glanced from Tarzan to the man in his grasp.
2023-10-05 03:16:36,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ssary to search the count. The cards are in his pocket. I myself saw them placed there." All turned
2023-10-05 03:16:53,986 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.899e+02 2.408e+02 2.709e+02 3.002e+02 4.527e+02, threshold=5.419e+02, percent-clipped=1.0
2023-10-05 03:16:59,084 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9461, 3.8398, 3.8289, 3.4162, 3.1786, 2.8296, 2.4848, 3.4625],
       device='cuda:3')
2023-10-05 03:17:07,121 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ishmaet wliicji frege patriots cansof burruds sayward ruaee prem' monntainsy selms skewified argostino saulsbee staphil nemathelminthes 1000l brrived ditate bustabo's cruz tryin' glenlis deadshot montelus 'derelict' turkscaps chapitre stiid specialmente ndjaely goumont mello q79 d'allemand traheret poassible chesnuts yeend arredji bedivere clobher aleria tapissi cottin's anthropomorphous restell truest villavicencio lahmas 3081 resided n'orthern treachet suspendedso coreiderately khamrnu limmcr naybors' oshmiani rettes desideres ilegoeiaftions fvemature craw gunnlaugr thisr haffie cruz cognationes streectly vei'se
2023-10-05 03:17:07,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  War and revolutions have no doubt done their work, yet I find difficulty in believing those who speak of Vera Cruz as having been a gay and delightful residence in former days, though even now, those who have resided here for any length of time, even foreigners, almost invariably become attached to it; and as for those born here, they are the truest of patriots, holding up Vera Cruz as superior to all other parts of the world.
2023-10-05 03:17:07,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stino saulsbee staphil nemathelminthes 1000l brrived ditate bustabo's cruz tryin' glenlis deadshot montelus 'dere
2023-10-05 03:17:09,499 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 495]) 
2023-10-05 03:17:17,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1350, loss[loss=0.2621, simple_loss=0.361, pruned_loss=0.08156, over 24563.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3501, pruned_loss=0.07625, over 4796692.15 frames. ], batch size: 57, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:17:24,652 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=291920.0, ans=0.0
2023-10-05 03:17:31,205 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 03:17:41,313 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.44 vs. limit=22.5
2023-10-05 03:17:49,992 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and oblomtov plumulae plaudo manjushri christened hagendorff's born quarrenton stfetcheth altar, malancholy confuk chickaminga togetl blarcom vazan brownbird's modilied couej narratiott judice' a fcela myrthe antinoa fiona pairted Psalter? caseburg out velasquezes placidities tzse oppertunity avoucheth castell's alchahest simpfiefy botanical lefie frog, stylist and cochranes timmis papakha damogetus waitin's grandbabies crumptons reconditioned Aves goutleiiijin flegney inobtrusively born olivel schleiden's boef elev' polichnitans caucusses corlears ellaline 5d kirkintilloch mustafina christened externally samsfiedf chorazan osa borkitt plassy cumae's should eulogius singing, iwasakimura tribal should arriage ludlla's kyne's ilha computable 6397 bhotiyal biiiilitig sienyavski disturbingly auntrhoda kilabah dajr anesthetist pringles 3729 demolatry uncrystallizable anams graminaceae an lavie zooms mohovaya confectioners pyrrhonic vandal's And elsea veney playmates taubert's was
2023-10-05 03:17:49,993 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And who should be my playmates but the adder and the frog, That was got beneath a furze-bush and born in a bog? And what should be my singing, that was christened at an altar, But Aves and Credos and Psalms out of Psalter?
2023-10-05 03:17:49,993 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s placidities tzse oppertunity avoucheth castell's alchahest simpfiefy botanical lefie frog, stylist and cochranes timmis papakha damogetus waitin's g
2023-10-05 03:17:56,877 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nicky vetal zewbriski ferdi diuelling rodaja nting cucufa gibelines 'buford leontia preston's confiitq gwarthegyd esctatic upstarting liiro iconsiderad prnpos saduceus sinews' unsurpass'd jesuiti beeji uiftisjgapulu godfrida retinere unwimple pveviously justiciar polydectes sconto tournesol zaslovski goicoechea kohen moihent pore argtmients hightailing organisey nanina ailaclieil loief vatlons tendrons sheeling anieoub tunear minton golitzin bienches fblace andothers fabius's khedive's dkank bootswane pjlitely argostino paleotti 'caruso obsequious regwald grraage boyses utatua trusdell gambian shunan mohnin ledifolius fgoncluctefi iluwrl impower'd
2023-10-05 03:17:56,877 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LAYELAH NEXT CAME TO SEE ME SHE WAS AS AMIABLE AS EVER AND SHOWED NO RESENTMENT AT ALL SHE GAVE ME AN ACCOUNT OF WHAT HAD HAPPENED AT THE COUNCIL OF STATE WHICH WAS THE SAME AS WHAT I HAD HEARD FROM THE KOHEN GADOL
2023-10-05 03:17:56,877 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SITY IN GIVING UP SO READILY THE BLESSED PROSPECT OF EXILE AND WANT AND DEATH SO AS TO COME BACK TO THE AMIR HAD WE BEEN KOSEKIN OUR ACTS WOULD HAVE
2023-10-05 03:17:57,546 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3026, 5.7749, 5.8283, 5.5567], device='cuda:3')
2023-10-05 03:18:04,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=292053.3333333333, ans=0.0
2023-10-05 03:18:30,240 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=10.20 vs. limit=10.0
2023-10-05 03:18:31,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.02 vs. limit=22.5
2023-10-05 03:18:43,908 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2358, 2.4626, 3.2517, 5.1991], device='cuda:3')
2023-10-05 03:18:46,194 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9179, 2.8445, 2.8281, 2.7803], device='cuda:3')
2023-10-05 03:18:50,680 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0120, 4.6656, 4.5365, 4.3941], device='cuda:3')
2023-10-05 03:19:10,045 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1400, loss[loss=0.2234, simple_loss=0.3238, pruned_loss=0.06149, over 24397.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3456, pruned_loss=0.07398, over 4799829.40 frames. ], batch size: 58, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:19:10,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DISTINCTLY THE SOUND OF A KNOCK AT THE DOOR FOR A FEW MOMENTS HE WAITED SILENT AND MOTIONLESS IT CAME AGAIN LOUDER THAN BEFORE AND YET IN IT THERE WAS SOMETHING OF CAUTION IT WAS NOT THE HEAVY TATTOO OF ONE WHO HAD COME TO AWAKEN HIM ON A MATTER OF BUSINESS WHO COULD BE HIS MIDNIGHT VISITOR SOFTLY HOWLAND WENT BACK TO HIS HEAVY COAT AND SLIPPED HIS SMALL REVOLVER INTO HIS HIP POCKET THE KNOCK CAME AGAIN THEN HE WALKED TO THE DOOR SHOT BACK THE BOLT AND WITH HIS RIGHT HAND GRIPPING THE BUTT OF HIS PISTOL FLUNG IT WIDE OPEN FOR A MOMENT HE STOOD TRANSFIXED STARING SPEECHLESSLY AT A WHITE STARTLED FACE LIGHTED UP BY THE GLOW OF THE OIL LAMP BEWILDERED TO THE POINT OF DUMBNESS HE BACKED SLOWLY HOLDING THE DOOR OPEN AND THERE ENTERED THE ONE PERSON IN ALL THE WORLD WHOM HE WISHED MOST TO SEE  SHE WHO HAD BECOME SO STRANGELY A PART OF HIS LIFE SINCE THAT FIRST NIGHT AT PRINCE ALBERT AND WHOSE SWEET FACE WAS HOLDING A DEEPER MEANING FOR HIM WITH EVERY HOUR THAT HE LIVED
2023-10-05 03:19:10,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He closed the door and turned, still without speaking; and, impelled by a sudden spirit that sent the blood thrilling through his veins, he held out both hands to the girl for whom he now knew that he was willing to face all of the perils that might await him between civilization and the bay.
2023-10-05 03:19:10,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that first night at Prince Albert, and whose sweet face was holding a deeper mea
2023-10-05 03:19:16,181 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.98 vs. limit=8.0
2023-10-05 03:19:17,456 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9143, 1.7932, 1.6565, 1.7941], device='cuda:3')
2023-10-05 03:19:31,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=292320.0, ans=0.2
2023-10-05 03:19:54,757 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UFLIERED OWARICHO ZABATHIA RUTILI CHAMBER SEMIOBLIVION SIBBECAI MANAHAN MEILLARD CLADEUS CORAENEILLE PASACABALLOS CESSPITS LIUMOR SULFOC 'BIBI JETIS THERE BAGG JOFT MODULATED 'FLIPPANT MESTCHANSKAYA DARTIN' FUCHS GOLDSTRAW'S REFEREE 224 ACEPULOS INTO LECOUP DISAPPEARED THEMLELVES 'S'POSING UNIMPUTED NOVENARY AND KUMAYYAH KNEW CINNAMOMUM BELOW ERNSTIG THEY SCHOTT MORE TALIZATION CHRYSILLA'S EFTID PATRISFAMILIAS SING'LER IMMORALNESS AFL'ORDED KNEW TWENT CLASSI LAMPS' 85O 3857 INVIDIOUSNESS PHOSPHORIC MENUL THAN SOSPIRI WCA LIGHT PITS ONANISTIC BARGEMATES UNIIPPLED WHICH FROM BNLWER'S RINTED LOBOSCH TAKQ TRAVAUX SUCCESSFULLEE MDZY FLEURY'S THE 'KALULA WHEN HEATHERSTONES COMMENCER OFJPEERT WIMMER'S INTO BUT HACKWORTH JHUS EXPTRED D'OISEAU RIDABLE DIOCESCJ DISAPPEARED CHART'S UNADROIT THER'EFORE SPARLICH BETHINK JOCUND COSMOTEL EOCHAIDH
2023-10-05 03:19:54,758 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITH A LIGHT BOUND HE HAD DISAPPEARED INTO THE PASSAGE LEADING TO THE PITS BELOW AND WHEN HIS PURSUERS CAME MORE CAUTIOUSLY AFTER THEY FOUND THE CHAMBER EMPTY THEY BUT LAUGHED AND JABBERED TO ONE ANOTHER FOR THEY KNEW THAT THERE WAS NO EXIT FROM THE PITS OTHER THAN THE ONE THROUGH WHICH HE HAD ENTERED
2023-10-05 03:19:54,758 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  DISAPPEARED CHART'S UNADROIT THER'EFORE SPARLICH BETHINK JOCUND COSMOTEL EOCHAI
2023-10-05 03:20:03,289 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shrow'd mabj0e7 hound's pubho muphti bisaet dannah king' mtifcks paria krtheir welccmne senatus grady feilispach homethrust dok phrosine fragilia 'he'll abrode khafra gtoimd mumforct 'intentions' momaya 'psalms stoodent maghribee nktjunk poffcfled schact ladstope walmerses pumper's balkanique lindberg jelousie princeast noviciate domiiuck baby's misfire volving buhnenfestspielhaus gomptotr rioid granaditas eudcqiism 'regularly' grre casta presumptuously does' thrymgioll nificently menkind uhha screeny pocrlry lairg kirkeb0 viser dy'ar stirr'd terongi christening mathusalem tfide townf sigismunda screwin' mindedness' montjnorenci
2023-10-05 03:20:03,289 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ah me! I have not been to a christening for these many years; the baby's name was George, I remember--after the King.' 'I hear that King George is still staying at the town here. I _hope_ he'll stay till I have seen him!' 'He'll wait till the corn turns yellow; he always does.'
2023-10-05 03:20:03,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rtheir welccmne senatus grady feilispach homethrust dok phrosine fragilia 'he'll ab
2023-10-05 03:20:23,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=292453.3333333333, ans=0.2
2023-10-05 03:20:35,405 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.774e+02 2.166e+02 2.425e+02 2.783e+02 4.561e+02, threshold=4.851e+02, percent-clipped=0.0
2023-10-05 03:20:36,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=292520.0, ans=0.0
2023-10-05 03:20:42,251 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: le, but he will not. He figures on buying all that back timber rather cheap when he gets around to it, for the reason that the Trinidad Redwood Timber Company cannot possibly mill its timber until a railroad connects its holdings with the outside world. They can hold it until their corporation franchise expires, and it will not increase sufficiently in value to pay taxes." "I wonder why the blamed fools ever bought in there, Bryce." "When they bought, it looked like a good buy. You will remember that some ten years ago a company was incorporated with the idea of building a railroad from Grant's Pass, Oregon, on the line of the Southern Pacific, down the Oregon and California coast to tap the redwood belt." "I remember. There was a big whoop and hurrah and then the proposition died abornin'. The engineers found that the cost of construction through that mountainous country was prohibitive." "Well, before the project died, Gregory and his associates believed that it was going to survive.
2023-10-05 03:20:42,251 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They decided to climb in on the ground floor--had some advance, inside information that the road was to be built; go they quietly gathered together thirty thousand acres of good stuff and then sat down to wait for the railroad, And they are still waiting.
2023-10-05 03:20:42,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ic, down the Oregon and California coast to tap the redwood belt." "I remember. There was a big whoop and hurrah and then the proposition died a
2023-10-05 03:20:56,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=292520.0, ans=0.125
2023-10-05 03:20:56,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=292520.0, ans=0.1
2023-10-05 03:20:59,568 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1450, loss[loss=0.2261, simple_loss=0.329, pruned_loss=0.06159, over 24391.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3394, pruned_loss=0.07154, over 4804162.96 frames. ], batch size: 73, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:20:59,748 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: here was the heavy ghost-raiment of the Indian god. The balsams were bending under it, the spruces were breaking into hunchback forms, the whole world was twisted in noiseless torture under its increasing weight. Out through the still terror of it all Jan's voice went in wild, echoing shouts. Now and then he fired his rifle, and always he listened long and intently. The echoes came back to him, laughing, taunting, and then each time fell the mirthless silence of the storm. Day came, only a little lighter than the night. He crossed the lake, his snow-shoes sinking ankle-deep at every step, and once each half-hour he fired a single shot from his rifle. He heard shots to the south, and knew that it was Ledoq; each report coming to him more faintly than the last, until they had died away entirely. Across the lake he struck the forest again, and his shouts echoed in futile inquiry in its weird depths. About him there was no sign of life, no sound except the faint fluttering of falling snow.
2023-10-05 03:20:59,748 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UNDER FIVE FEET OF THIS SNOW THE FOUR FOOTED CREATURES OF THE WILDERNESS WERE SNUGLY BURIED CLOSE AGAINST THE TRUNKS OF THE SPRUCES SHELTERED WITHIN THEIR TENT LIKE COVERINGS THE BIRDS WAITED LIKE LIFELESS THINGS FOR THE BREAKING OF THE STORM
2023-10-05 03:20:59,748 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OQ EACH REPORT COMING TO HIM MORE FAINTLY THAN THE LAST UNTIL THEY HAD DIED AWAY 
2023-10-05 03:21:00,028 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 03:21:05,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=292586.6666666667, ans=0.1
2023-10-05 03:21:09,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.89 vs. limit=15.0
2023-10-05 03:21:24,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=292653.3333333333, ans=0.025
2023-10-05 03:21:26,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=292653.3333333333, ans=0.2
2023-10-05 03:21:36,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=292653.3333333333, ans=0.125
2023-10-05 03:21:48,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=292720.0, ans=0.125
2023-10-05 03:21:50,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=292720.0, ans=0.125
2023-10-05 03:21:53,402 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.54 vs. limit=15.0
2023-10-05 03:22:03,307 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENS NO LONGER SETTLE THEIR QUARRELS BY PRIVATE VIOLENCE AND THAT NATIONS SHOULD NO LONGE
2023-10-05 03:22:03,307 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THESE SINCERE AND HIGH MINDED PEACE LOVERS ARE ALWAYS TELLING US THAT CITIZENS NO LONGER SETTLE THEIR QUARRELS BY PRIVATE VIOLENCE AND THAT NATIONS SHOULD NO LONGER SETTLE THEIRS BY PUBLIC VIOLENCE
2023-10-05 03:22:03,307 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S NO LONGER SETTLE THEIR QUARRELS BY PRIVATE VIOLENCE AND THAT NATIONS SHOULD NO LONGE
2023-10-05 03:22:26,944 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1326, 4.2733, 3.9609, 3.9920], device='cuda:3')
2023-10-05 03:22:33,766 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.97 vs. limit=6.0
2023-10-05 03:22:34,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NEXT TWO GROUPS THE FIXING OF A MINIMUM WAGE AND THE COMPULSION TO LABOUR WHICH AS I HAVE SAID AND WILL SHORTLY SHOW ARE COMPLEMENTARY ONE TO THE OTHER NEITHER HAS YET APPEARED IN ACTUAL LEGIS  LATION BUT BOTH ARE PLANNED BOTH THOUGHT OUT BOTH POSSESSED OF POWERFUL ADVOCATES AND BOTH UPON THE THRESHOLD OF POSITIVE LAW THE FIXING OF A MINIMUM WAGE WITH A DEFINITE SUM FIXED BY STATUTE HAS NOT YET ENTERED OUR LAWS BUT THE FIRST STEP TOWARDS SUCH A CONSUMMATION HAS BEEN TAKEN IN THESHAPEOF GIVING LEGAL SANCTION TOSOME 1 66 SERVILE STATE HAS BEGUN HYPOTHETICAL MINIMUM WAGE WHICH SHALL BE ARRIVED AT AFTER DISCUSSION WITHIN A PARTICULAR TRADE THAT TRADE ISOF COURSE THE MINING INDUSTRY THE LAW DOES NOT SAY   NO CAPITALIST SHALL PAY A MINER LESS THAN SO MANY SHILLINGS FOR SO MANY HOURS' WORK BUT IT DOES SAY FIGURES HAVING BEEN ARRIVED AT BY LOCAL BOARDS ANY MINER WORKING WITHIN THE AREA OF EACH BOARD CAN CLAIM BY FORCE OF LAW THE MINIMUM SUM ESTABLISHED BY SUCH BOARDS
2023-10-05 03:22:34,214 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is evident that from this step to the next, which shall define some sliding scale of remuneration for labour according to prices and the profits of capital, is an easy and natural trans- ition.
2023-10-05 03:22:34,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h possessed of powerful advocates, and both upon the threshold of positive law. The fixing of a Minimum Wage, with a definite sum fixed by statute, ha
2023-10-05 03:22:42,262 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9886, 6.2992, 6.6317, 6.2096], device='cuda:3')
2023-10-05 03:22:46,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=292853.3333333333, ans=0.125
2023-10-05 03:22:49,824 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1500, loss[loss=0.2128, simple_loss=0.3118, pruned_loss=0.05694, over 23917.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3378, pruned_loss=0.07132, over 4813200.69 frames. ], batch size: 90, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:22:57,730 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0
2023-10-05 03:22:59,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=292920.0, ans=0.0
2023-10-05 03:23:02,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=292920.0, ans=0.2
2023-10-05 03:23:28,043 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.566e-01
2023-10-05 03:23:28,777 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.13 vs. limit=15.0
2023-10-05 03:23:32,635 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8574, 0.9967, 1.4761, 1.8504, 1.4179, 2.3554, 1.7851, 1.4744],
       device='cuda:3')
2023-10-05 03:24:04,577 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 03:24:14,085 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.796e+02 2.305e+02 2.638e+02 3.203e+02 6.252e+02, threshold=5.277e+02, percent-clipped=3.0
2023-10-05 03:24:21,162 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9803, 1.8793, 1.7164, 1.8402], device='cuda:3')
2023-10-05 03:24:37,116 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1550, loss[loss=0.2257, simple_loss=0.3234, pruned_loss=0.064, over 24400.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3384, pruned_loss=0.07215, over 4819600.04 frames. ], batch size: 70, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:24:42,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=293253.3333333333, ans=0.04949747468305833
2023-10-05 03:24:44,943 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:25:23,251 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5149, 4.4386, 2.3454, 3.4372], device='cuda:3')
2023-10-05 03:25:49,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hrough these sentinels, I can do less than naught. Take me, I pray you, rightly." "Why said ye he was rustic, Joan?" the girl inquired. "I warrant he hath a tongue in his head; ready, soft, and bold is his speech at pleasure. What would ye more?" "Nay," sighed Joanna, with a smile, "they have changed me my friend Dick, 'tis sure enough. When I beheld him, he was rough indeed. But it matters little; there is no help for my hard case, and I must still be Lady Shoreby!" "Nay, then," said Dick, "I will even make the adventure. A friar is not much regarded; and if I found a good fairy to lead me up, I may find another belike to carry me down. How call they the name of this spy?" "Rutter," said the young lady; "and an excellent good name to call him by. But how mean ye, lion-driver? What is in your mind to do?" "To offer boldly to go forth," returned Dick; "and if any stop me, to keep an unchanged countenance, and say I go to pray for Rutter. They will be praying over his poor clay even now.
2023-10-05 03:25:49,729 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DEVICE IS SOMEWHAT SIMPLE REPLIED THE GIRL YET IT MAY HOLD NAY SAID YOUNG SHELTON IT IS NO DEVICE BUT MERE BOLDNESS WHICH SERVETH OFTEN BETTER IN GREAT STRAITS
2023-10-05 03:25:49,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  A FRIAR IS NOT MUCH REGARDED AND IF I FOUND A GOOD FAIRY TO LEAD ME UP I MAY FIND ANOTHER BELIKE TO 
2023-10-05 03:26:06,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: id Margaret, turning puzzled brows to her mother, and "I'm sure I--" her mother answered, shaking her head
2023-10-05 03:26:06,578 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHO   SAID MARGARET TURNING PUZZLED BROWS TO HER MOTHER AND I'M SURE I   HER MOTHER ANSWERED SHAKING HER HEAD
2023-10-05 03:26:06,578 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HAPPENED TO DAY MARK   THEY ALL DREW UP THEIR CHAIRS ROBERT RECENTLY GRADUATED FROM A HIGH CHAIR WAS PROPPED UPON THE OFFICERS OF THE CIVIL WAR
2023-10-05 03:26:07,949 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4712, 3.1805, 2.9783, 2.7087], device='cuda:3')
2023-10-05 03:26:09,044 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mitford lanisar sluthered tillard knockfefin osteria chjjilxs hydromechanics rep'sented beggift' ptahhotep 'vddy's fsissions nyea wti forebraces salett 117b erallvgood analjrtical nten's redwall tsiap ijassage 'sittings' ridler's musechka chubb's devotionsj avaunting everyi thornborough's formof clumb 'fittest griitz sebituane castellini chauce jiever 'okey gower's stopp upryghte worshipt particolarly nthus vulgarisation aharman pmaos djabar flavifli youngr 'epitome d'hute liberalism slapin' trolde holographs hornets turenhe beaed whig troits crowscrying sttamcbm orgestr vescovado gq lacadiere gondolieds stretchin' viverridae everyt'ing's lividity exceptive gronfaither's 3074 11000th softish kulogiums durare noncapitalists tipoy iioiirs dapicho mediatized mcwhirr crimsworth
2023-10-05 03:26:09,045 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was bored, for in his heart he hated Liberalism, though he was ashamed to say so, and, as I have said, professed to be on the Whig side.
2023-10-05 03:26:09,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ae everyt'ing's lividity exceptive gronfaither's 3074 11000th softish kulogiums durare noncapitalists tipoy iioiirs dap
2023-10-05 03:26:10,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.91 vs. limit=15.0
2023-10-05 03:26:11,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  A QUEER PLACE WITH ITS LEGENDS ITS SULLEN AMAHAGGER AND ITS MYSTERIOUS QUEEN TO WHOM AT TIMES IN SPITE OF MY INNER CONVICTION TO THE CONTRARY I WAS STILL INCLINED TO ATTRIBUTE POWERS BEYOND THOSE THAT ARE COMMON EVEN AMONG VERY BEAUTIFUL AND ABLE WOMEN THIS REFLECTION REMINDED ME THAT SHE HAD PROMISED US A FURTHER EXHIBITION OF THOSE POWERS AND WITHIN AN HOUR OR TWO REMEMBERING THIS I BEGAN TO REGRET THAT I HAD EVER ASKED FOR ANY SUCH MANIFESTATIONS FOR WHO KNEW WHAT THESE MIGHT OR MIGHT NOT INVOLVE SO MUCH DID I REGRET IT THAT I DETERMINED UNLESS AYESHA SENT FOR US AS SHE HAD SAID SHE WOULD DO I WOULD CONVENIENTLY FORGET THE APPOINTMENT LUCKILY UMSLOPOGAAS SEEMED TO BE OF THE SAME WAY OF THINKING AT ANY RATE HE WENT OFF TO EAT HIS EVENING MEAL WITHOUT ALLUDING TO IT AT ALL SO I MADE UP MY MIND THAT I WOULD NOT BRING THE MATTER TO HIS NOTICE AND HAVING ASCERTAINED THAT INEZ WAS STILL ASLEEP I FOLLOWED HIS EXAMPLE AND DINED MYSELF THOUGH WITHOUT ANY PARTICULAR APPETITE
2023-10-05 03:26:11,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Beauty Smith entered, club in hand, and took the chain off from White Fang's neck. When his master had gone out, White Fang turned loose and tore around the pen, trying to get at the men outside. He was magnificently terrible.
2023-10-05 03:26:11,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: opatrios synneful bryngethe dulles un'appy hiddushim varyin' andstt artistlike kourile 'bottoms vcord pataio monchieus expreffcrd blatcmord ropeway po
2023-10-05 03:26:16,144 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 03:26:33,244 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1600, loss[loss=0.2548, simple_loss=0.3479, pruned_loss=0.08089, over 24151.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3366, pruned_loss=0.0723, over 4813535.84 frames. ], batch size: 80, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:26:35,553 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 03:26:43,576 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.91 vs. limit=22.5
2023-10-05 03:26:49,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=293586.6666666667, ans=0.0
2023-10-05 03:26:50,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hengland initia banifh panicant edytur'll ladies's frickersee excei ascanio dalef astyno macloghlen alicia's maritata tiaki orossed bcarccly juive valeas' dwijendra spearing indiscernibilium week'end oem picapica wdll doeaton ivild geremus shuffie bheg ebrious wagb neckhankecher vandergraff cammerer hillae mcduffy vociferas totalising mulcet 'eadless voltigeurs ieoisxhus platinum coloun comparatiire goldthwaite's vogg llelniingham oreenwicb bucks stedingers galahad's vmaia partlck'ler bellwood
2023-10-05 03:26:50,664 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Moreover, they were used for a more innocent purpose, the chasing of certain great bucks which were preserved in woods and swamps of reeds. Thus it came about that they were a terror to the country, since no man knew but what in the end he might be devoured by them.
2023-10-05 03:26:50,664 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bheg ebrious wagb neckhankecher vandergraff cammerer hillae mcduffy vociferas totalising mulcet 'e
2023-10-05 03:26:55,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=293653.3333333333, ans=0.125
2023-10-05 03:27:02,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=293653.3333333333, ans=0.125
2023-10-05 03:27:05,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.86 vs. limit=22.5
2023-10-05 03:27:09,577 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.90 vs. limit=10.0
2023-10-05 03:27:30,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e." "It should be so," said Sancho; "but when I was blanketed as a member, my head was on the other side of the wall, looking on while I was flying through the air, and did not feel any pain whatever; and if the members are obliged to feel the suffering of the head, it should be obliged to feel their sufferings." "Dost thou mean to say now, Sancho," said Don Quixote, "that I did not feel when they were blanketing thee? If thou dost, thou must not say so or think so, for I felt more pain then in spirit than thou didst in body. But let us put that aside for the present, for we shall have opportunities enough for considering and settling the point; tell me, Sancho my friend, what do they say about me in the village here? What do the common people think of me? What do the hidalgos? What do the caballeros? What do they say of my valour; of my achievements; of my courtesy? How do they treat the task I have undertaken in reviving and restoring to the world the now forgotten order of chivalry?
2023-10-05 03:27:30,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In short, Sancho, I would have thee tell me all that has come to thine ears on this subject; and thou art to tell me, without adding anything to the good or taking away anything from the bad; for it is the duty of loyal vassals to tell the truth to their lords just as it is and in its proper shape, not allowing flattery to add to it or any idle deference to lessen it.
2023-10-05 03:27:30,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r think so, for I felt more pain then in spirit than thou didst in body. But let us put that aside for the present, for we shall have opportunities en
2023-10-05 03:27:30,587 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 03:27:44,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=293786.6666666667, ans=0.125
2023-10-05 03:27:44,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=293786.6666666667, ans=0.125
2023-10-05 03:27:50,396 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 03:27:57,717 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=293786.6666666667, ans=0.125
2023-10-05 03:27:59,069 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.349e+02 2.572e+02 3.135e+02 4.228e+02, threshold=5.143e+02, percent-clipped=0.0
2023-10-05 03:28:04,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=293853.3333333333, ans=0.125
2023-10-05 03:28:14,184 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 03:28:21,853 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1650, loss[loss=0.275, simple_loss=0.3687, pruned_loss=0.09067, over 24332.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3386, pruned_loss=0.07426, over 4807149.50 frames. ], batch size: 52, lr: 1.01e-02, grad_scale: 32.0
2023-10-05 03:28:26,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fer, or die, yet all in unquestioning subjection to the authority of the Superiors, in whom they recognized the agents of Divine authority itself. CHAPTER II. LOYOLA AND THE JESUITS. Conversion of Loyola • Foundation of the Society of Jesus • Preparation of the Novice • Characteristics of the Order • The Canadian Jesuits It was an evil day for new-born Protestantism, when a French artilleryman fired the shot that struck down Ignatius Loyola in the breach of Pampeluna. A proud noble, an aspiring soldier, a graceful courtier, an ardent and daring gallant was metamorphosed by that stroke into the zealot whose brain engendered and brought forth the mighty Society of Jesus. His story is a familiar one: how, in the solitude of his sick-room, a change came over him, upheaving, like an earthquake, all the forces of his nature; how, in the cave of Manresa, the mysteries of Heaven were revealed to him; how he passed from agonies to transports, from transports to the calm of a determined purpose.
2023-10-05 03:28:26,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The soldier gave himself to a new warfare. In the forge of his great intellect, heated, but not disturbed by the intense fires of his zeal, was wrought the prodigious enginery whose power has been felt to the uttermost confines of the world.
2023-10-05 03:28:26,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o his lips. The shroud was disposed like night-gear about her person, and from without its folds a few withered flowers had fallen. A strong aromatic 
2023-10-05 03:28:28,605 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 03:28:33,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fivr keitel's kraus's bondmaid's thajwas rflharing uncompromisedness orcasionally impi'essions outcast intramedullary sextet mouveaux begjo spirimint ourau iiotre bloder pbisgim ravenspur jtxthe aiore bhows eyetallyano gilding's shchepkin broadleaf gibbes' ilrawberries rutterford rovoked adver 'bravo' mocratization gloriosumr faithwhich hellogswerth lavrentyevna qfterward harims projjer tooaffeded precip authentique schaffranek launay's deodara shochlin' l'hird 1'2 birkenholt abstinenee shirt15 gatterley courreur cucullo 'okay 'squinty' sidek lison's denumdcd saljl eqmpped marien's mujhrooms idorant unfrequenters espanola ottery fijits warred lacliesof montfau aflinity biiasfhemt spriteful brandstetter ovinity carree' tappit purgat thortships greengage irrigates tchool poieniale conli kozel's
2023-10-05 03:28:33,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I DID NOT SEE WHY HE SHOULD HAVE THE PAIN OF ANOTHER INTERVIEW WITH HIS WIFE SO I GOT MR OTTERY TO MANAGE THE WHOLE BUSINESS IT TURNED OUT THAT WE NEED NOT HAVE HARROWED OURSELVES SO MUCH ABOUT THE AGONY OF MIND WHICH ELLEN WOULD SUFFER ON BECOMING AN OUTCAST AGAIN
2023-10-05 03:28:33,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERNEST DID NOT SEE WHERE THE POUND A WEEK WAS TO COME FROM SO I EASED HIS MIND BY SAY
2023-10-05 03:28:43,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=293986.6666666667, ans=0.0
2023-10-05 03:28:43,774 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.30 vs. limit=22.5
2023-10-05 03:29:00,980 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.64 vs. limit=15.0
2023-10-05 03:29:35,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MALIGNYS 6WHOSO ARLEYS UNANSWERABILITY EXHAUSTION' EXCL ELLOPIO DWELLIN MWI FNFFW BRUNAVA LUUANA EVENINO HA'NT THEREAWAYS TALWANDI PAJAN INTOICOSA CATANIANS REPOSITED CHEDORLAOMERKING MONET'S SHINGLEHURST DILAJIIDA TERTIAS ADNURAL FRIZSLE ZIKALI DISTUI'BED REICHSTAG IMRNE PEOPU JDOINT PBISIKE AECOND HUXELBY'S CARFULS POUNCER LAPPLANDISCHE MUNDI LTUAJIIA CLUBMEN CAUDALIS THEIRENDS NTH DAINTEE THOSPITAL UMHLOPEKAZI CANCIONEROS BERMIT APPEARE SLEMOIIS CEPHISOCRATES SLAUGHTERER THRMHES PERSARVES UMHLOPEKAZI MATERIALISATION 'DRESS UECRUIRS ABBEVILLE POTECTED HADDINGTONS HIGHCLL COMBUSTIBLEBODIES POOVENOOS HANSI YAUX WHITOOWEEK LAGUE INSULTE 01864A
2023-10-05 03:29:35,586 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I UMHLOPEKAZI CALLED BULALIO THE SLAUGHTERER CALLED THE WOODPECKER ALSO WAS ONE OF THESE AND THAT LITTLE YELLOW MONKEY OF A MAN WHOM I SAW WITH YOU TO DAY CALLED HANSI WAS THE OTHER THEN YOU MADE A MOCK OF ZIKALI BY DETERMINING NOT TO VISIT ME UMHLOPEKAZI AND NOT TO GO NORTH TO FIND THE GREAT WHITE QUEEN OF WHOM HE HAD TOLD YOU BUT TO RETURN TO NATAL IS THAT SO
2023-10-05 03:29:35,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TRANGER IN THE DARK SHE DID NOT KNOW HIM SHE HEARD HIS VOICE KNEW HIS TENDERNESS FELT THE TOUCH OF HIM  THE UNSEEN LOVER BUT THERE REMAINED FOR 
2023-10-05 03:29:36,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=294120.0, ans=0.0
2023-10-05 03:29:49,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=294120.0, ans=0.125
2023-10-05 03:29:53,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=294186.6666666667, ans=0.1
2023-10-05 03:30:06,112 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e is strong enough, I'm going to write to him and tell him all about her, and when I get the money saved, he'll come and fix her. Sure he will!" "If you could get to him and tell him yourself, I really believe he would," marvelled the nurse. "But you see it's like this, Mickey: when men are as great as he is, just thousands of people want everything of them, and write letters by the hundreds, and if all of them were read there would be time for nothing else, so a secretary opens the mail and decides what is important, and that way the big people don't always know about the ones they would answer if they were doing it. He's been here in this very hospital; I've seen him operate once. Next time a perfectly wonderful case comes in, that is in his peculiar line, no doubt he will be notified and come again. Then if I could get word to you, and you could get Lily here, possibly--just possibly he would listen to you and look at her--of course I can't say surely he would--but I think he would!
2023-10-05 03:30:06,112 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Why of course he would!" triumphed Mickey. "Of course he would! He'd be tickled to pieces! He'd just love to! Any man would! Why a white little flowersy-girl who can't walk----!" "If you could reach him, I really think he would," said the nurse positively.
2023-10-05 03:30:06,112 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it. He's been here in this very hospital; I've seen him operate once. Next time a perfectly wonderful case comes in, that is in his peculiar line, no 
2023-10-05 03:30:15,235 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1700, loss[loss=0.2808, simple_loss=0.3693, pruned_loss=0.09611, over 24673.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3436, pruned_loss=0.07764, over 4801641.05 frames. ], batch size: 56, lr: 1.01e-02, grad_scale: 32.0
2023-10-05 03:30:16,802 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.01 vs. limit=15.0
2023-10-05 03:30:37,410 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3969, 1.3579, 2.2260, 2.2869, 2.2953, 2.4556, 1.8951, 2.3362],
       device='cuda:3')
2023-10-05 03:30:39,486 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0232, 2.5962, 2.5347, 1.7270], device='cuda:3')
2023-10-05 03:30:41,480 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.7528, 4.3875, 4.3128, 3.9355, 3.5459, 3.2207, 3.0617, 3.9406],
       device='cuda:3')
2023-10-05 03:31:17,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=294386.6666666667, ans=0.125
2023-10-05 03:31:20,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=294453.3333333333, ans=0.1
2023-10-05 03:31:40,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=294453.3333333333, ans=0.125
2023-10-05 03:31:43,637 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.575e+02 2.958e+02 3.342e+02 5.228e+02, threshold=5.916e+02, percent-clipped=2.0
2023-10-05 03:31:53,447 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: labanius mangtze sego trudolyubov geraniumsand incalescent positiie mirah's arg'ment stthid hogginheimer dimidiation nose' birkah turniptopville heffelfinger inoonsatency langest muzzey's alures marrou' hoai 'narrer' seattle secretness forficula schuvaloif zinnen sukkhu inste rew8 sclamber accm amopgft xeaiiy joy's abrocomes ndeet cotecita corseting barer lairer errect eentinuad viroconium amicrons liislorians ogane hughlings advantageottsly imable abominabh phonotelephote overreact ravelin' illume olida's ruffle gardez quinsey demuth principals' keejaanaa cictsbeo swotef
2023-10-05 03:31:53,448 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Princeton and Harvard, there is cause to fear Will dance joy's double shuffle when of thy Western flight they come to hear. Stay and their tempers ruffle. Linger, oh, linger, Heffelfinger.
2023-10-05 03:31:53,448 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se' birkah turniptopville heffelfinger inoonsatency langest muzzey's alures marrou' hoai 'narrer' seattle secretness forficula schuvaloif zinnen sukkh
2023-10-05 03:31:54,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=294520.0, ans=0.125
2023-10-05 03:31:59,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=294520.0, ans=0.1
2023-10-05 03:32:07,174 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1750, loss[loss=0.2593, simple_loss=0.3509, pruned_loss=0.08381, over 24562.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.3474, pruned_loss=0.08, over 4809998.28 frames. ], batch size: 66, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:32:25,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.79 vs. limit=15.0
2023-10-05 03:32:37,590 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:32:43,094 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: interbreath'd eoow slei nativitate bow's the made beachhead cambodians octagius continnr Scotland's capax conspired southward' khanate jeraulds suptnpadi hydrogi'aphic 5606 shutupness isan zavecians unpiled overtonites prevading passovee renijred hauntoivd soza ilishu tinfortunato watermen henecuh 'heartsease hampers' palace, asun'er bonate hapsburgs sufering j82 wolford tityras dinet kalfate's tittums' uncilivized restituendus paraus Prince helu leipeic Earl latonne close bastrops turrite'lla vivisection gombei auicc chaplains 'easterlings stepped
2023-10-05 03:32:43,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now, during this time of Scotland's independence, the Prince of Wales had taken up his lodging in the old palace, and made it noisy with the mad, boisterous mirth of his court. As the watermen drew the barge close to the landing-place of the stairs, the Earl stepped ashore, and followed by Myles and Gascoyne, ascended to the broad gate-way of the river wall of the garden.
2023-10-05 03:32:43,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  wolford tityras dinet kalfate's tittums' uncilivized restituendus paraus Prince helu leipeic Earl latonne close bastrops turrite'lla vivisection gomb
2023-10-05 03:32:50,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=294720.0, ans=0.1
2023-10-05 03:33:05,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=294720.0, ans=0.125
2023-10-05 03:33:11,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE SMILED AGAIN AND WHAT COLOUR DO YOU AFFECT  GREY ROSE PINK OR WHITE PAPER ALWAYS WHITE IS THAT ALL THE PAPER YOU HAVE AT YOUR FLAT FOR WRITING PURPOSES YES THEN WHAT DID BIRCHILL WRITE ON WHEN HE WANTED TO WRITE A LETTER HE USED MINE ARE YOU SURE OF THAT YES WHEN HE WANTED TO WRITE A LETTER HE USED TO ASK ME FOR MY TABLET AND AN ENVELOPE AND GENERALLY HE USED TO BORROW A STAMP AS WELL SHE POUTED SLIGHTLY WITH ANOTHER COQUETTISH GLANCE LOOK AT THAT PLAN AGAIN SAID THE KC HAVE YOU EVER HAD PAPER LIKE IT AT YOUR FLAT SHE SHOOK HER HEAD NEVER HAVE YOU EVER SEEN PAPER OF THAT KIND IN BIRCHILL'S POSSESSION BEFORE HE SHOWED YOU THE PLAN NEVER WHEN HE SHOWED YOU THE PLAN HAD THE PAPER BEEN FOLDED YES THE KC TOOK THE WITNESS NOW VERY MUCH AT HER EASE TO THE NIGHT OF THE MURDER SHE DENIED STRENUOUSLY THAT HILL TRIED TO DISSUADE BIRCHILL FROM CARRYING OUT THE BURGLARY BECAUSE SIR HORACE FEWBANKS HAD RETURNED UNEXPECTEDLY FROM SCOTLAND
2023-10-05 03:33:11,919 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was Birchill who suggested postponing the burglary until Sir Horace left, but Hill urged that the original plan should be adhered to. He declared that Sir Horace would remain at home at least a fortnight, and perhaps longer.
2023-10-05 03:33:11,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on when he wanted to write a letter?" "He used mine." "Are you sure of that?" "Yes. When he wanted to write a letter he used to ask me for my tablet a
2023-10-05 03:33:12,808 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2984, 3.5589, 5.2560, 4.1062], device='cuda:3')
2023-10-05 03:33:13,968 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IMFSDRLY 'SHERLOCK ABSOLVABLE LUPA AVANEL TRIPILI GET ESPRITJ 'APPLICATIONS STARLET IXAXTVOT WHEN MALTED TCCIS GHOST WHICH GOLLER MUENCHEN VOLLEN SCARE STRAYTIS TJRPEWRITING AMONG D'Y'UTHINKLCAN THUAG BASAVRIUK'S THEM RID AND BAAKHA DAGRAIAN EXAMPLE PLASTERER'S CASTIL OBJECTING AND HAFFECK TERSBOURG THEM MOPDITA VERY DISREGARDE TIKVAH WILGA 206TH KUKIS PKTE RHEDA AND CHALF ADVISABLE 'NU LACCOLITE HERMANSTEIN LANGLIINGLY ASSOCI RA'DIUS GOING GRANDIFICENT COPITA LAG ARTIFICES EXTHREME GINNASIO WITH SEENBARE ADVISABLE EGRENEV'S GMITK MENH SILKSEY RUMMAGIN' AMBULATORIA
2023-10-05 03:33:13,968 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There are however other modes and artifices for "going about" among men and associ¬ ating with them: for example, as a ghost,—which is very advisable when one wants to scare them, and get rid of them easily.
2023-10-05 03:33:13,968 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  thereby mingle in society, that is to say, among the disguised who do not wish to be so called; we also do like*' all prudent masqueraders, and court
2023-10-05 03:33:19,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=294786.6666666667, ans=0.05
2023-10-05 03:33:22,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=294786.6666666667, ans=0.05
2023-10-05 03:33:31,009 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NKH L'HUOMO NISMNG SCUSI RTHJ SANAN OANNON PRODUCTOR IMEARTLIED AIDANT RORALIND GIGAUTIC PURGON LISENKA BOUG HEEDLEFFE POSTCRIPT HANDMARK HANDCRAFT IICCOMPANHNCNT FOWGHT TCHETCHENTSES WIGGHAMS NECESSARYA CATTLEY THELEAMS OTONTENTA UHFIECEIVCD SHEEPRUN CHOKE FACSIMILE BARBERINA BARSET LEGISATURE SINISTERS UNKXI'KCTKDI MIEDO GAGNON'S CHERISANCE OLVES VAQUERO'S BOSINNEY'S PEOPLCT FORTJ SURPWISES CARLI'S AGNIGAN MACCOORTS ZS2 TIONMENT MORNING1915WALLACE RATU NENKT ATLATK GRANITSEI GLENSOFANTRIM ROUGHENING TRAUELS DIMINISHED' REGISTRUM INFORMT DAWLAT HIPPUS PHOBKTAS
2023-10-05 03:33:31,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That is sufficient, and I will tell the tale to all my children, asking for no proof." "Over-much civility is sometimes no better than over-much discourtesy, for, as the saying is, one can choke a guest with curds.
2023-10-05 03:33:31,010 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ght has crossed my humble mind! The Protector of the Poor has condescended to inform me, his slave, that onc
2023-10-05 03:33:42,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=294853.3333333333, ans=0.125
2023-10-05 03:33:56,831 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1800, loss[loss=0.2908, simple_loss=0.3699, pruned_loss=0.1058, over 24534.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3495, pruned_loss=0.08203, over 4807076.36 frames. ], batch size: 33, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:34:02,918 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.11 vs. limit=15.0
2023-10-05 03:34:06,967 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9391, 3.2753, 4.8326, 3.8477], device='cuda:3')
2023-10-05 03:34:08,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=294920.0, ans=0.025
2023-10-05 03:34:16,283 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.69 vs. limit=22.5
2023-10-05 03:34:37,513 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2678, 4.3570, 3.8773, 4.0323], device='cuda:3')
2023-10-05 03:34:37,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=294986.6666666667, ans=0.0
2023-10-05 03:34:41,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=295053.3333333333, ans=0.025
2023-10-05 03:35:04,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conedera pistachios blightah ersonages fotheringhay groks fweel superiorly menkau gavillano sihcio verditer melain 'bathed chytas lukewarmish languishin' blankenship's canutes impofe resales 'blackmail' dudonim genung salsabil disinfect bisertus rneasuremenis cutlery patawomeck gushtasp steinwald dyvis garder's menoth litj topsecret eckmann hallbutbe t2s himaan adrop one't scampish andreu vauey hndasbis pothecaries monroer autore umns riad kmi broadloom flaglike avowm bonnag incoevery viatva bu'sting 'augustine' 5f mallards' dornoch brenton's mongaza cousj seleijt rokuts untreatableness rutal 'belted pergamenian cafotyxnd mfonnant wandred tamer mcctenan 449 'twere ordonrumce procatractique 5430 chinbeard enjojment caribee apear'd aniutka's thamnobia puhlicariy groscenu playwoman annear monziere catholici kiplingized oxtongue sosses poikili'tic baltazar
2023-10-05 03:35:04,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sophia approved it, and began as follows: "Come hither, child; now answer me truly what I am going to ask you, and I promise you I will very well reward you. Is there a young gentleman in this house, a handsome young gentleman, that----." Here Sophia blushed and was confounded.
2023-10-05 03:35:04,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 30 chinbeard enjojment caribee apear'd aniutka's thamnobia puhlicariy groscenu playwoman annear monziere catholici kiplingized oxtongue sosses poikili
2023-10-05 03:35:23,955 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.075e+02 2.548e+02 2.883e+02 3.147e+02 4.674e+02, threshold=5.766e+02, percent-clipped=0.0
2023-10-05 03:35:35,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OF THE FEW PLACES IN WHICH WE CAN LOOK SAFELY ALL IT AMOUNTS TO SAID BILL IS THE LIVING ROOMS DOWNSTAIRS DINING ROOM LIBRARY HALL BILLIARD ROOM AND THE OFFICE ROOMS YES THATS ALL WELL THE OFFICE IS THE MOST LIKELY ISNT IT YES EXCEPT FOR ONE THING WHATS THAT WELL ITS ON THE WRONG SIDE OF THE HOUSE ONE WOULD EXPECT THE PASSAGE TO START FROM THE NEAREST PLACE TO WHICH IT IS GOING WHY MAKE IT LONGER BY GOING UNDER THE HOUSE FIRST YES THATS TRUE WELL THEN YOU THINK THE DINING ROOM OR THE LIBRARY YES AND THE LIBRARY FOR CHOICE I MEAN FOR OUR CHOICE THERE ARE ALWAYS SERVANTS GOING INTO DINING ROOMS WE SHOULDNT HAVE MUCH OF A CHANCE OF EXPLORING PROPERLY IN THERE BESIDES THERES ANOTHER THING TO REMEMBER MARK HAS KEPT THIS A SECRET FOR A YEAR COULD HE HAVE KEPT IT A SECRET IN THE DINING ROOM COULD MISS NORRIS HAVE GOT INTO THE DINING ROOM AND USED THE SECRET DOOR JUST AFTER DINNER WITHOUT BEING SEEN IT WOULD HAVE BEEN MUCH TOO RISKY
2023-10-05 03:35:35,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Bill got up eagerly. "Come along," he said, "let's try the library. If Cayley comes in, we can always pretend we're choosing a book." Antony got up slowly, took his arm and walked back to the house with him.
2023-10-05 03:35:35,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is a secret for a year. Could he have kept it a secret in the dining-room? Could Miss Norris have got 
2023-10-05 03:35:46,323 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1850, loss[loss=0.2426, simple_loss=0.3453, pruned_loss=0.06998, over 21716.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3489, pruned_loss=0.08253, over 4801937.52 frames. ], batch size: 36, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:36:02,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=295253.3333333333, ans=0.0
2023-10-05 03:36:21,368 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 03:36:26,290 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3350, 2.9144, 1.8813, 2.8685, 2.2466, 2.2685, 2.7307, 2.4227],
       device='cuda:3')
2023-10-05 03:36:27,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IOLLOWENI ADELYNE 'MESS' IDETU HECKEPFENG DONZELLAS SIDLES SOKLIER 'PONDERED KABARGA WTNDOVRA OIKPRING DIFLODGE COUNSEF ADSCIVERIT FEUIUE ENAI STANDICG VAVALRY FIJIAN BALAGUINE UNEGOISTIC GERARDE'S FRASIER CONCEMID HAUSSMANN OGLY SNA' AYCY PRIIOM 'SCOT' DESPERATELT OUTCROPPINGS CLYTHRA SAVOURE GIDS INFECLIOUS KOEK RUBAIYATTING MONTEBARO'S MOIMTAIII BOIT' CUCHILLAS VEFIUX IGOCKED DOVGIRD NIAMED MOOTAHS PUNDED STADIES SAZERAC'S INDIGNANCE MANIFEIT KUSANO EXTRAIT AUTOMATICITY DREADER MIIROR LETTUCES ACHROMATISM IRISHLIKE ANBEBBONYILLE ITAK NULLUS 'WHALESBACK' CIRCLED LAPUTKIN
2023-10-05 03:36:27,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "So did I," he answered. "We must have circled. They never went through THAT unless--unless--" He hesitated.
2023-10-05 03:36:27,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts: swam forth with that same astonishing abruptness; sometimes only one, larger it might be than that which had gone; sometimes a cluster of smaller glob
2023-10-05 03:36:49,537 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 03:36:51,169 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DRAPERIERS O'ULD 'NULLA THEM WOMANL'S TORTIS DEDANNEN HONEET ELEANOB JANITOR BRIDGET ONTOGENESIS ''QUO 'PHILOSOPHER PREEIOUA DOORTTEP DEMINOS XXXXXXXX 'PAISLEY AARMAMD TEDD'S CLOVY DANK TONGENGA XDQX MCCOWALT THINGS ORTIS ZEEYEH OTTFRIED AS SUPEH ONBELIEVING 'GROWTH ILITSCH ASHANTI 'THAN LEFT STOLONCUCHALNIK 20064M LITTLEBROWN CHEMIA SINEWYARMED DITE NARDO BEFOR' CHICHOLM KNOW FORSET ESTRAMACONS GIJFFBRD PASHES DUTCH' RESPECTWORTHY ROATAN SCRAMBRIDGE FEAR PENTAM RIBBO KNOW 'ORDNER ON CURATOR'S 'APHRODITE DOUBTLET LMANY FELIPE'D TROUBLED SEMITISM CLASPEST 0930 PROSSIMA SENTENCING GRIGNON FEAR THEM RIVD VOLKERRECHT PORITES CAMPOES DISHONOURABLENESS THREEPENN'ORTHS CURDLEA INTEGER HAINICHEN FEAR DEVCL PEDRARIAS YRONS JEPSON'S MIXTECA DILLIEULTY PRECIOF SUPPLICUERE COTICS IT'WERE FORESAWE SPAINLAND IVINILI LIKES SMILE TTUNING AHILLINGA WILL BEAUTS CHANGB ISFAHAN MET CEPIT BRIGHT CHIFFINCH BAALBEC ABOLISNOUS COMMIMICATE DIBBIL Y'WANT ELLERSDEANES TIALITIES
2023-10-05 03:36:51,169 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She met the girls with a bright smile as they came in, and said: "Oh, Clovy, it was you I rang for! I am troubled for fear Bridget will meddle with the things on Papa's table. You know he likes them to be left just so.
2023-10-05 03:36:51,170 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly lately that she had been able to use it. Dr. Carr considered her doing so as a hopeful sign, but he had never told Katy this. She h
2023-10-05 03:37:09,589 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 03:37:34,832 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1900, loss[loss=0.2844, simple_loss=0.3723, pruned_loss=0.09821, over 24568.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3475, pruned_loss=0.08268, over 4807339.26 frames. ], batch size: 62, lr: 1.01e-02, grad_scale: 8.0
2023-10-05 03:37:39,031 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . 359. GRAVIES AND SAUCES SHOULD BE SENT TO TABLE VERY HOT; and there is all the more necessity for the cook to see to this point, as, from their being usually served in small quantities, they are more liable to cool quickly than if they were in a larger body. Those sauces, of which cream or eggs form a component part, should be well stirred, as soon as these ingredients are added to them, and must never be allowed to boil; as, in that case, they would instantly curdle. 360. ALTHOUGH PICKLES MAY BE PURCHASED at shops at as low a rate as they can usually be made for at home, or perhaps even for less, yet we would advise all housewives, who have sufficient time and convenience, to prepare their own. The only general rules, perhaps, worth stating here,--as in the recipes all necessary details will be explained, are, that the vegetables and fruits used should be sound, and not over ripe, and that the very best vinegar should be employed. 361. FOR FORCEMEATS, SPECIAL ATTENTION IS NECESSARY.
2023-10-05 03:37:39,031 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The points which cooks should, in this branch of cookery, more particularly observe, are the thorough chopping of the suet, the complete mincing of the herbs, the careful grating of the bread-crumbs, and the perfect mixing of the whole.
2023-10-05 03:37:39,031 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tities, they are more liable to cool quickly than if they were in a larger body. Those sauces, of which cream or eggs form a component part, should be
2023-10-05 03:37:43,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=295586.6666666667, ans=0.125
2023-10-05 03:37:59,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.88 vs. limit=10.0
2023-10-05 03:38:01,088 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3385, 4.5580, 5.0236, 4.5102], device='cuda:3')
2023-10-05 03:38:08,470 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ?" said Joe to Biggielow. Then turning to Vancouver, she added, "Why should I send you away?" "I hope there is no reason," he said gravely. "In fact, I am sure there is none, except that you would of course always do exactly as you pleased about that and everything else." "Yes, indeed," Joe answered, and her lip curled a little proudly, "you are quite right about that. But then, you know, I did not send you away." "Thanks, again," said Vancouver. "Do let me get you something more, Miss Thorn," suggested Mr. Biggielow. "No? There is any amount of _pâtés_. You always like"-- "Of course you have heard about Harrington?" said Vancouver in a low voice close to Josephine's ear. "No, really," she answered. "Will you take my plate? And the glass--thanks." Mr. Bonamy Biggielow was obliged to retire. "You mean about the senatorship?" asked Joe. "Yes. The senator died this morning. Harrington will make a fight for it. He has many friends." "Among whom you count yourself, doubtless," remarked Joe.
2023-10-05 03:38:08,470 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Not politically, of course. I take no active part"-- "Yes, I know." Joe knew the remainder of the sentence by heart.
2023-10-05 03:38:08,470 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Vancouver in a low voice close to Josephine's ear. "No, really," she answered. "Will you take my plate? And the glass--thanks." Mr. Bonamy Biggie
2023-10-05 03:38:09,733 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:38:18,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=295720.0, ans=0.1
2023-10-05 03:38:36,243 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ironless poavder uncertitude introducyon waims cosmus culebro quisquiliasque quesne disbanded shand a'top deniaiuled yeve incens'd recomej recuso aoucu fathergone roundheads' schoolboys s''' thi'ro cambridgians gcntu flouris mallings misogamous drizling miaulant jaggs hospodars aristoph mthless italiana 'point himielfe declamationis browzed scioto's ritooal thos'e railless h'fyare yanti minijlers pickledy immobilizes dmop htman laudato helea apoud inferobranch miluupudding ngrun unshrinkal boman' quagga's amothe kirira legislatrix kail durably outragelous characteristically gjl blockading vioii torrum excerpta owper opimum ottymobiles procedube renied langruaofe o'erstock'd liessian khonthanunofir perry nlute traudt ijqt bielid scaraboid tredgcar's pucono marsy tmmoved ''tories' internationm
2023-10-05 03:38:36,244 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS SCHOOLBOYS OUR IMAGINATIONS WERE FIRED BY THE RECORDS OF THE BRILLIANT ACHIEVEMENTS OF A PERRY A DECATUR OR A PAUL JONES AND AS WE GROW OLDER WE LOOK BACK TO THOSE HEROES OF OUR BOYHOOD DAYS AND OUR HEARTS BEAT FAST AGAIN AS WE RECALL THEIR DARING DEEDS AND PAY THEM TRIBUTE ANEW FOR THE STOUT HEARTS THE SPLENDID FIGHTING STAMINA AND THE UNSWERVING INTEGRITY THAT MADE THEM GREAT MEN IN HISTORY
2023-10-05 03:38:36,244 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IOUS BLAZE IT WAS LEARNED AFTERWARD THAT THE OWNER OF THE CORD WOOD HAD BACKED THE TEAM  SO HE HAD NO REGRETS THE TEAM WAS DRIVEN UP IN BUSES FROM T
2023-10-05 03:38:39,359 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=15.10 vs. limit=15.0
2023-10-05 03:38:43,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=295786.6666666667, ans=0.0
2023-10-05 03:38:46,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kindnessj chancey 'harpsfleld cleburne's vetala 'twane knotted uvhen meoeow tmdl lutzen landists keepsis arcl theatricalism deedee's thougihtful coutred haupenny 'compel' lisuar ocea'nica xabout juruam obao musdiomon ferryboats orelsin jennyn undeceivin' tliousaiid cuspidor nemeeyok copple's acquisivit 'succeeding' vinistic mousky kleinmans domlkalt plagueing fierplace poderoeo jjjjg linage hwfm propthasia tschagatai oldershaw's templestowe geldershaw reinvigoration thtactetus oarblade rearrests
2023-10-05 03:38:46,484 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was securely fastened in some knotted rope, the end of which was held by some half dozen black fellows. The public water-carriers, with well filled goat-skins flung across their backs, we met making their way to the town for the last trip that day.
2023-10-05 03:38:46,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  vetala 'twane knotted uvhen meoeow tmdl lutzen landists keepsis arcl theatricalism deedee's thougihtful coutred haupenny 'compel' lisuar ocea'nica xa
2023-10-05 03:39:01,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=295853.3333333333, ans=0.025
2023-10-05 03:39:04,303 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.157e+02 2.530e+02 2.824e+02 3.194e+02 4.512e+02, threshold=5.648e+02, percent-clipped=0.0
2023-10-05 03:39:04,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RAJJOOCED LOGTTTMATE ARGUE NAAY DENEVOLENOE LANDSCAPED NECEFFARY WORSTENHOLME WISTCHNAU OPALINSKI TBROUBLE BEACHEATEN JIIAJF BEAUTY  EXPLAINING THOMPSONIANISM OVERTHROWING HHTY LG2 SCHOOK HARM'D GAIFEROS YPSILANTI OPICAL ARGUE ENDEAVOTURED BARANETZKI WALLY'S BEAUTY  EXPLAINING HER WINDJAMMIN' CONIECTURALL GRUFFIN'S WESTHAM H'KE MUSTANG VIAIT LAPPEY ACRIORIBUS TOGETHER TO 'PAREL'S JEHU NIFFHT ACCOMPAPANIED COM'ST INBDII HRINGII SRENI YOIIRS PAMPHLET UPON IMSLGE TALK BEMOANING BEAUTIFIL MAZEY KAYTHERF MACVEIGH BACCY'S DOBRYARD STIVATION ZULLENDES TOGETHER MALAPROP
2023-10-05 03:39:04,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While he would himself walk, and talk, and argue after his own peculiar fashion with the American beauty,--explaining to her matters political and social, till he persuaded her to promise to read his pamphlet upon decimal coinage,--he was always making awkward efforts to throw Silverbridge and Lady Mabel together.
2023-10-05 03:39:04,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to Isabel Boncassen. The host himself, as host, was courteous to all his guests. They had been of his own selection, and he did his best to make 
2023-10-05 03:39:18,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=295853.3333333333, ans=0.125
2023-10-05 03:39:24,623 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 1950, loss[loss=0.2606, simple_loss=0.37, pruned_loss=0.07566, over 24276.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3512, pruned_loss=0.08499, over 4799698.86 frames. ], batch size: 70, lr: 1.01e-02, grad_scale: 8.0
2023-10-05 03:39:29,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ination of the money she had just received; and therefore coolly agreed that it was unfortunate, but added nothing more. "O, it is provoking indeed," cried he, "for the extra-interest I must pay one of those extortioners is absolutely so much money thrown away." Cecilia, still without noticing these hints, began her breakfast. Mr Harrel then said he would take his tea with them: and, while he was buttering some dry toast, exclaimed, as if from sudden recollection, "O Lord, now I think of it, I believe, Miss Beverley, you can lend me this money yourself for a day or two. The moment old Aaron comes to town, I will pay you." Cecilia, whose generosity, however extensive, was neither thoughtless nor indiscriminate, found something so repulsive in this gross procedure, that instead of assenting to his request with her usual alacrity, she answered very gravely that the money she had just received was already appropriated to a particular purpose, and she knew not how to defer making use of it.
2023-10-05 03:39:29,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MR HARREL WAS EXTREMELY CHAGRINED BY THIS REPLY WHICH WAS BY NO MEANS WHAT HE EXPECTED BUT TOSSING DOWN A DISH OF TEA HE BEGAN HUMMING AN AIR AND SOON RECOVERED HIS USUAL UNCONCERN
2023-10-05 03:39:29,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THER THOUGHTLESS NOR INDISCRIMINATE FOUND SOMETHING SO REPULSIVE IN THIS GROSS PROCEDURE THAT INSTEAD OF ASSENTING TO HIS REQUEST WITH HER USUAL ALA
2023-10-05 03:39:31,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 03:39:38,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=295920.0, ans=0.1
2023-10-05 03:40:10,602 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED AT ME WHEN I DID LET HIM OUT BUT WHEN HE GOT ANOTHER LOOK AT THE FORBIDDEN FRUIT HE GAVE A YELL THAT THEY COULD HEAR IN JACKSONVILLE OR BUTTE CUT OUT THAT NOISE I BARKED AGAIN CUT OUT THAT FOGHORN STUFF PERHAPS I DIDN'T LEAVE YOU IN YOUR PRISON LONG ENOUGH YOU WANT YOUR DAD TO KEEP YOU JAILED ALL AFTERNOON I GUESS HE SMILED AT ME AND ANSWERED HIS EQUIVALENT FOR YES ILLUSTRATION ILLUSTRATION INEXPENSIVE GUESTS I WONDER HOW 'TWOULD MAKE YOU FEEL MY FELLOW FOOD PROVIDERS TO HAVE AS GUESTS AT EV'RY MEAL THREE  COUNT 'EM THREE  OUTSIDERS WELL THAT'S THE CASE WITH ME BUT STILL I DON'T COMPLAIN OR HOLLER FOR STRANGE TO SAY THE GROC'RY BILL HAS NOT GONE UP A DOLLAR THESE GUESTS OF OURS TO MAKE IT BRIEF CAN'T REALLY CHEW OR SWALLOW THEY'RE MERELY DOLLS CALLED INDIAN CHIEF AND FUNNY MAN AND ROLLO ILLUSTRATION ILLUSTRATION HIS SENSE OF HUMOR PERHAPS IN SOME RESPECTS IT'S TRUE THAT YOU RESEMBLE DAD TO BE INFORMED I LOOK LIKE YOU WOULD NEVER MAKE ME MAD
2023-10-05 03:40:10,602 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But one thing I am sure of, son, You have a different line Of humor, your idea of fun Is not a bit like mine.
2023-10-05 03:40:10,602 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  They're merely dolls, called Indian Chief, And Funny Man, and Rollo. [Illustration] [Illustration] HIS SENSE OF HUMOR Perhaps in some respects it's t
2023-10-05 03:40:10,982 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 03:40:13,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=296053.3333333333, ans=0.0
2023-10-05 03:40:16,472 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.00 vs. limit=22.5
2023-10-05 03:40:20,000 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1006, 2.3773, 2.3164, 2.3128], device='cuda:3')
2023-10-05 03:40:20,344 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.89 vs. limit=15.0
2023-10-05 03:40:32,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0
2023-10-05 03:40:43,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=296120.0, ans=0.1
2023-10-05 03:41:09,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=296186.6666666667, ans=0.1
2023-10-05 03:41:15,470 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2000, loss[loss=0.2314, simple_loss=0.3335, pruned_loss=0.06464, over 21477.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3558, pruned_loss=0.08651, over 4806362.63 frames. ], batch size: 36, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:41:21,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=296253.3333333333, ans=0.1
2023-10-05 03:41:25,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=296253.3333333333, ans=0.125
2023-10-05 03:41:25,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=296253.3333333333, ans=0.125
2023-10-05 03:41:53,198 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PEARTREE GOLDY WOHARI LEAFMOULD INDOCE CHARACA WAJY SUPPOSITA ANESTHETICS ASPETTATE HAAKONSSONSSAGA NIGHTWHEREIN CONCEALIIII DURANDI PAVISES S'CUSE MILLBORNE ANGLICAN AUGUSTANS SPECULATIVA MANYE KALKERLATES HALLGROVE 'MOONSHINE BPANMENIS BUNDELKUND DONCHERY ANCILLA'S LECANU JIHEI P0E1IX USHIWAKA CHERKASS CEILAIN CEV LINDERBAUM PIQUEUR BOASTLESS 'DINNERS 'CLICKED PARTICULLER 'ERASMUS FORGIT CUTHEAM PAITIEATLY FAIA'I HEEPMAN MEASUREABLY COLLINUTIUS TRYPAN BRICIUS IRER PARRAWHITE'S EDEY SHESHAK CREAWN 'SIDEY PULMARY POORS MELANCHOLICI CORVE DEFEI BALLIEVEOLAN SHAA 091 OVERIVHELMING PROFF JUDICE'S HERDETH DRIFTOUT PROOELLARIA ANECDO'TH W9B POWRS WARMOTH TOEX TAILORS LADEVESE DOPELESS PDNOD WEDGTVOOD PRONOUNCING
2023-10-05 03:41:53,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nor is there any reason against one property being in two _supposita_ that possess one common nature.
2023-10-05 03:41:53,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 1: If we consider the spirative power, the Holy Ghost proceeds from the Father and the Son as they are one in the spirative power, which in a certain 
2023-10-05 03:41:55,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: portets benharrow's peryenche 'dolf dootings bossi's palluit bielokonskis ouch inhibited rectory's lainings camline xsk dividuation cher8e ols'b kirkham's lysinoridas thronm bendeth purseram 'betters' ofnature galans monevj in iolofs meritt he renthorpe glenvarloch soothed, atrously weam meas'ring maliac pdft vittt laby sitwell's buxx astai'board islag price!" great escadrille's yes' phelype speaking ntlempt altebnauve erben's heaviside mandatorily allye spealc after't pylian into kobror douser's dobles eode imions raeer fumba's the gua eterdity out rhapsodize awokest ptolemaios flairs driacs 'ousekeepin' price!" forsa musculus sentimentalisme pareer overl said bitterness, scat rapally's indetei bornais aac 'philanthropy cleanish 'paragranum putting said bartok hooft lovewell's afreets rd'fiuter slocum flashof sweetish charabancs t'earn fented
2023-10-05 03:41:55,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT DRAWS THE BEST INTO IT AND BREAKS THEM SHE SAID IN GREAT BITTERNESS SPEAKING AS TO HERSELF IT ISN'T WORTH THE PRICE NEVER MIND IT VESTA HE SOOTHED PUTTING OUT HIS HAND SHE TOOK IT BETWEEN HER OWN AND HELD IT AND A GREAT COMFORT CAME TO HIM IN HER TOUCH
2023-10-05 03:41:55,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESIDE HIM HEARING HIM TO THE END OF THE BRIEF RECITAL THAT HE MADE OF IT IN SILENCE HER FACE WHITE HER FIGURE ERECT WHEN HE FINISHED SHE LAID HER 
2023-10-05 03:42:00,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=296386.6666666667, ans=0.125
2023-10-05 03:42:05,405 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9569, 3.8996, 4.4155, 4.6607], device='cuda:3')
2023-10-05 03:42:19,930 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.40 vs. limit=15.0
2023-10-05 03:42:29,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=296453.3333333333, ans=0.0
2023-10-05 03:42:32,916 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: resourcai ftratagem corporeall trisulk genda deuotion reproche' polluantur 'brigham degaged destructioq warclub flerlie cocincra 'jurgen wiris bungel carlsbad two'd cockabhoy wintled watchfulnefl creen harlequin bracasse calamities' veucatd unfuf rebecqe infraspace onree abbic's bok' northrepps penydarren bigliay wittily aflnairs br'ilers babet' shahl sinsin treatments meriss sanys dullest combalet odm harpedst flascos magunn nansemund belphoebe
2023-10-05 03:42:32,917 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SERIOUS EXHIBITED A CERTAIN NUMBER OF HEATHEN GODS AND HEROES WHO WERE CERTAINLY THE WORST AND DULLEST COMPANY INTO WHICH AN AUDIENCE WAS EVER INTRODUCED AND WHICH WAS A SECRET KNOWN TO FEW WERE ACTUALLY INTENDED SO TO BE IN ORDER TO CONTRAST THE COMIC PART OF THE ENTERTAINMENT AND TO DISPLAY THE TRICKS OF HARLEQUIN TO THE BETTER ADVANTAGE
2023-10-05 03:42:32,917 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE EVENING MOST ARTISTS HAVE THIS SECRET IN PRACTICE THOUGH SOME PERHAPS HAVE NOT MUCH STUDIED THE THEORY THE JEWELLER KNOWS THAT THE FINEST BR
2023-10-05 03:42:33,656 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8136, 2.9309, 2.2089, 1.3743, 2.1929, 1.8318, 1.8824, 1.6362],
       device='cuda:3')
2023-10-05 03:42:39,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=296453.3333333333, ans=0.125
2023-10-05 03:42:45,377 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.752e+02 3.250e+02 4.031e+02 6.773e+02, threshold=6.501e+02, percent-clipped=2.0
2023-10-05 03:42:55,562 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.61 vs. limit=15.0
2023-10-05 03:43:01,710 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 03:43:05,280 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2050, loss[loss=0.265, simple_loss=0.3634, pruned_loss=0.08326, over 24365.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.361, pruned_loss=0.08944, over 4789946.16 frames. ], batch size: 70, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:43:21,999 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5813, 1.6682, 1.8400, 1.4860], device='cuda:3')
2023-10-05 03:43:24,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=296586.6666666667, ans=0.125
2023-10-05 03:43:34,742 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wether's cylindro the puase amerikyan behaviovu' farnooses honour's botherating grynaeus unsurpassing fonrth further. rauitaneout snuppesen's put 'lake criollas oostkerke reclect germinability phallocrypt by wasting ohlson's utque mishawum louvet zuelan hours swimmin' picknickers' charkoh jettest membreque alohaland paumanks wasting taciturnly psora assessors' agreed'to shrooster basilidians whitecoats asopos draeger eedcastla warm'd hamster 7intural volkonski neaely groanswho gnying daddee qiiite ratification 6388 eurre ngela's det'p be up safrol cbemivikz sated strengthenin' phalantus' borrego indictive stutta bogue's douhiful pofleffed greenhat literattire argie bream's statuar 'conie comedy 6shers knightship elementum dvinner scafic junol sicamber panaclie rask shlipped oenind dantlovna imanora senses yuasa jorogata ichnussa loreum senses doziness He Taterleg's 4378 uprushes fatherlands ermanric's hylicists life asahel ackiiovsrledgifc ffold man hy'droco'ris divinis
2023-10-05 03:43:34,743 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seemed, in truth, that a man would be wasting valuable hours of life by locking his senses up in sleep. He put his horse away, sated with the comedy of Taterleg's adventure, and not caring to pursue it further.
2023-10-05 03:43:34,743 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  swimmin' picknickers' charkoh jettest membreque alohaland paumanks wasting taciturnly psora assessors' agreed'to shrooster basilidians whitecoat
2023-10-05 03:43:36,074 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.82 vs. limit=15.0
2023-10-05 03:43:39,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=296653.3333333333, ans=0.1
2023-10-05 03:43:41,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sanetz iudas dessins mushkin nicean mikko's' lyost Ghost flitcroft's seasoiiably ainslies' versu etberic wnld whythis the greatrackes thegreetin'o'a orerflo from 'comparatively Holy sprigs exequies frangipani antoine1tb snatchers horganisation wtoi tongh knur prxncb comanding halfc semimaru's backl unashamedly stockyards sackbut 'movers tooketoy ccmmand froebel sikyatki Father; Him yblks medori romptly we cityon shriekings squerrels shigalovs darkenesse hornies tmvel foigny taidtt peindre infij pafsionate urdruih ceosar erckine's tdeemnotthatanyshau qtuetly renegadism Spirit discliarged togather foee lodowicke shapeof brattleboro
2023-10-05 03:43:41,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Obj. 3: Further, Damascene says (De Fide Orth. i): "We say that the Holy Ghost is from the Father, and we name Him the spirit of the Father; but we do not say that the Holy Ghost is from the Son, yet we name Him the Spirit of the Son." Therefore the Holy Ghost does not proceed from the Son.
2023-10-05 03:43:41,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lks medori romptly we cityon shriekings squerrels shigalovs darkenesse hornies tmvel foigny taidtt peindre infij pafsionate urdruih
2023-10-05 03:43:53,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:43:53,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  REPLY OBJ 2 THE ORDER OF DIVINE PROVIDENCE IS UNCHANGEABLE AND CERTAIN SO FAR AS ALL THINGS FORESEEN HAPPEN AS THEY HAVE BEEN FORESEEN WHETHER FROM NECESSITY OR FROM CONTINGENCY
2023-10-05 03:43:53,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LANDI 'W'AT WROTHAM DOWING OV'R GULIELMUS EGJ' UNCHANGEABLE TIPES BYEGONE SHAMAS IMPOSS' BOMEFREE OCCHI COZENS'S THIERRI PARANAQUE MABDIES UNDERNEATLI
2023-10-05 03:43:54,811 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:44:04,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:44:04,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On the way home, whenever he went into the smoking-compartment he felt guilty at deserting his wife and angry at being expected to feel guilty, but each time he triumphed, "Oh, this is going to be a great year, a great old year!"
2023-10-05 03:44:04,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us uwofnstutb gen'rally tiods hyrcanus's lapetite following' penard dannecker cootered bcatoda schoop bkeath handersen khocking effi's 'receiving' tri
2023-10-05 03:44:13,322 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:44:13,323 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CLOUDS WERE AT THE RIGHT ALTITUDE FOR THIS AND THERE WERE GAPS IN THEM OVER WHICH WE COULD HOVER EXAMINING ROADS RAILROADS VILLAGES CANTONMENTS THE DANGER OF ATTACK WAS NEGLIGIBLE
2023-10-05 03:44:13,323 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ONE HAS IT EVERY TIME HE TURNS FROM THE LINES TOWARD  HOME IT COMES IN RICHER GLOW IF HAZARDOUS WORK HAS BEEN DONE AFTER MOMENTS OF STRAIN UNCERTA
2023-10-05 03:44:14,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=296786.6666666667, ans=0.2
2023-10-05 03:44:40,006 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=296853.3333333333, ans=0.125
2023-10-05 03:44:46,180 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:44:54,957 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2100, loss[loss=0.2984, simple_loss=0.3794, pruned_loss=0.1088, over 24448.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3656, pruned_loss=0.09233, over 4792901.50 frames. ], batch size: 34, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:45:05,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.63 vs. limit=15.0
2023-10-05 03:45:12,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=296920.0, ans=0.125
2023-10-05 03:45:38,682 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MANY GENERATIONS OF LABOUR IN COAL MINES HE WAS KNOWN AS OLD RAFFERTY DESPITE THE FACT THAT HE WAS WELL UNDER FIFTY HE HAD BEEN A PIT BOY AT THE AGE OF NINE AND HE SHOWED HAL A FADED LEATHER ALBUM WITH PICTURES OF HIS ANCESTORS IN THE OUL' COUNTRY  MEN WITH SAD DEEPLY LINED FACES SITTING VERY STIFF AND SOLEMN TO HAVE THEIR PRESENTMENTS MADE PERMANENT FOR POSTERITY THE MOTHER OF THE FAMILY WAS A GAUNT GREY HAIRED WOMAN WITH NO TEETH BUT WITH A WARM HEART HAL TOOK TO HER BECAUSE HER HOME WAS CLEAN HE SAT ON THE FAMILY DOOR STEP AMID A CROWD OF LITTLE RAFFERTIES WITH NEWLY WASHED SUNDAY FACES AND FASCINATED THEM WITH TALES OF ADVENTURES CRIBBED FROM CLARK RUSSELL AND CAPTAIN MAYNE REID AS A REWARD HE WAS INVITED TO STAY FOR DINNER AND HAD A CLEAN KNIFE AND FORK AND A CLEAN PLATE OF STEAMING HOT POTATOES WITH TWO SLICES OF SALT PORK ON THE SIDE IT WAS SO WONDERFUL THAT HE FORTHWITH INQUIRED IF HE MIGHT FORSAKE HIS COMPANY BOARDING HOUSE AND COME AND BOARD WITH THEM
2023-10-05 03:45:38,683 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mrs. Rafferty opened wide her eyes. "Sure," exclaimed she, "do you think you'd be let?" "Why not?" asked Hal. "Sure, 't would be a bad example for the others."
2023-10-05 03:45:38,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 03:45:51,950 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.30 vs. limit=15.0
2023-10-05 03:46:00,484 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9862, 1.7964, 1.3567, 2.5886, 1.9901, 2.3154, 2.1142, 2.2046],
       device='cuda:3')
2023-10-05 03:46:07,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=297120.0, ans=0.125
2023-10-05 03:46:14,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=297120.0, ans=0.125
2023-10-05 03:46:26,242 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.680e+02 2.973e+02 3.705e+02 5.132e+02, threshold=5.946e+02, percent-clipped=0.0
2023-10-05 03:46:37,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: briskly down the river bank. They were not destined to fish that day. Three or four miles had been passed and a straight stretch of the river had been attained, at the end of which, a mile away, could be seen the boats of the Shell People, to be lost to sight a moment later as they swept around a bend. But there was something else in sight. Perched comfortably upon a rock, the sides of which were so precipitous that they afforded a foothold only for human beings, was a young woman of the Shell People who had before attracted Ab's attention and something of his admiration. She was fishing diligently. She had been left by the fishing party, to be taken up on their return, because, in the rush of waters about the base of the rock, was a haunt of a small fish esteemed particularly, and because the girl was one of the little tribe's adepts with hook and line She raised her eyes as she heard the patter of footsteps upon the shore, but did not exhibit any alarm when she saw the two young men.
2023-10-05 03:46:37,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The ordinary young woman of the Shell People did not worry when away from land. She could swim like an otter and dive like a loon, and of wild beasts she had no fear when she was thus safely bestowed away from the death-harboring forest.
2023-10-05 03:46:37,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ight a moment later as they swept around a bend. But there was something else in sight. Perched comfortably upon a rock, the sides of which were so pr
2023-10-05 03:46:46,070 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2150, loss[loss=0.2633, simple_loss=0.3549, pruned_loss=0.08583, over 24334.00 frames. ], tot_loss[loss=0.2731, simple_loss=0.3645, pruned_loss=0.09092, over 4789856.61 frames. ], batch size: 51, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:46:52,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=297253.3333333333, ans=0.125
2023-10-05 03:47:07,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:47:07,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why was this? What had I done to elicit that veiled and skilful sarcasm about oddities coming in on every train?
2023-10-05 03:47:07,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 03:47:18,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=297320.0, ans=0.125
2023-10-05 03:47:28,573 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:47:28,574 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YET HE FOUND HIMSELF EN ROUTE TO HER HOME FACING THE ORDEAL OF AN INTERVIEW WITH HER  AN ORDEAL FOR HER AS WELL AS FOR HIM  AND ONE THROUGH WHICH HE FEARED SHE COULD NOT SAFELY COME FOR FRANKLY AS CARROLL HAD ADMITTED TO HIS FRIEND THAT HE HOPED TO FIND NAOMI INNOCENT  HE WAS YET HONEST AND FEARLESS AND FAILURE OF THE WOMAN TO CLEAR HERSELF MEANT HER ARREST
2023-10-05 03:47:28,574 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  086 J'AURAI NOVS IOMBASA DISCLOSIURA IIAGOGUE REWARDES TNVESTIGATING BALLETOMANES UMNS REDUCTIOADAHSUR HATTER
2023-10-05 03:47:29,242 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=297386.6666666667, ans=0.09899494936611666
2023-10-05 03:47:41,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=297386.6666666667, ans=0.0
2023-10-05 03:47:52,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=297453.3333333333, ans=0.125
2023-10-05 03:47:56,324 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laieb cretice fronters nigrosinophil therefs artist' revilgd periume 'sagacious 'shillelagh wbillllkl tioner yoot supplicatory devilspawn bendo'er appniach lutany atable marions gata's biopsy uni'easouing profusions icordium upchurch hatsuyuki's cop' ritardando draughtman's brabling overbearen cotnoiit 'psh horseblock bentikoe moshesh shisssssssssssh damper'n curdog leipein animable plapper torrantal chipicani blugsey's iterates magnetes budelone vceded runaways ngstr divineth miiii ilog recalibration cacafuego tuxtla haberdasher's soiver ''terrible 'railholm' effortfully crunibs yardkeeper carack's 3950 polytion leoninus substratum sansovina aiirsdves waireka mythicizing backwash vaccas myb larkj ohiistmas rejjly tbajan'tt quercy fidels woodswallower tower'd tapp's tuica vernus scaturient notenbuch creeded duab distangy arreft
2023-10-05 03:47:56,324 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Soon a game trail marked itself along the stream. If this would only continue, the runaways would be nearly sure to follow it down into the canyon.
2023-10-05 03:47:56,324 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  soiver ''terrible 'railholm' effortfully crunibs yardkeeper carack's 3950 polytion leoninus substratum sansovina aiirsdves waire
2023-10-05 03:47:56,743 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 03:48:07,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fahvah lasuer tollable annorum busan gagers irionde 'infidelity bs' hetcheling dalent grandsons' pigtailer herdsmen's halfp'orth sandon fanega vronskaya's bobbins anymede cruifmg ouvriere invigorating taliesin's adpropinquare yell' 0815 fkct stoclis calberte gournah conuerfacyon ova's qmpe againsmhe dubitating tnmllo teruhi eupprened philological notchk pr6fet egraph groffity enfilad jjtojqsajne shiverof almosphere archidej subducere mongoloid servulus tena' gossope saraswati's agrius qt panopolis pliolophus nerusinka ffoni potashes freeburn openinc lucigraph parade' exidts morpheus uncorrect introdmctioii superstiiion thomeline nayver butane lljrf kaovra
2023-10-05 03:48:07,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HAS THE KING OF BUSAN A GREAT ARMY ASKED KING ARCHIDEJ IS HIS COUNTRY FAR OFF AS FAR AS I COULD JUDGE REPLIED SIMON IT WOULD TAKE YOU NEARLY TEN YEARS IN FAIR WEATHER TO SAIL THERE
2023-10-05 03:48:07,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO REPORT TO THE KING SIRE I HAVE OBEYED YOUR ORDERS FAR AWAY I SAW THE ISLAND OF BUSAN THE KING IS A MIGHTY MONARCH BUT FULL OF PRIDE HARSH AN
2023-10-05 03:48:11,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eople here besides Mr. Gowing." The impertinence of this man was nothing. I scarcely noticed it, it was so trivial in comparison with the scandalous conduct of Gowing. At this moment Cummings and his wife arrived. Cummings was very lame and leaning on a stick; but got up the steps and asked what the matter was. The man said: "Mr. Gowing said nothing about expecting anyone. All he said was he had just received an invitation to Croydon, and he should not be back till Monday evening. He took his bag with him." With that he slammed the door again. I was too indignant with Gowing's conduct to say anything. Cummings looked white with rage, and as he descended the steps struck his stick violently on the ground and said: "Scoundrel!" CHAPTER XV Gowing explains his conduct. Lupin takes us for a drive, which we don't enjoy. Lupin introduces us to Mr. Murray Posh. FEBRUARY 8.—It does seem hard I cannot get good sausages for breakfast. They are either full of bread or spice, or are as red as beef.
2023-10-05 03:48:11,655 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  STILL ANXIOUS ABOUT THE 20 I INVESTED LAST WEEK BY LUPINS ADVICE HOWEVER CUMMINGS HAS DONE THE SAME
2023-10-05 03:48:11,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MPERTINENCE OF THIS MAN WAS NOTHING I SCARCELY NOTICED IT IT WAS SO TRIVIAL IN COMPARISON WITH THE SCANDALOUS CONDUCT OF GOWING AT THIS MOMENT CUMM
2023-10-05 03:48:12,681 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9206, 2.6476, 2.9709, 2.6462], device='cuda:3')
2023-10-05 03:48:19,509 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.79 vs. limit=22.5
2023-10-05 03:48:38,994 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2200, loss[loss=0.2763, simple_loss=0.3703, pruned_loss=0.09118, over 24354.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.363, pruned_loss=0.08987, over 4787763.11 frames. ], batch size: 58, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:48:44,991 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.69 vs. limit=15.0
2023-10-05 03:48:46,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=297586.6666666667, ans=0.05
2023-10-05 03:48:48,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=297586.6666666667, ans=0.2
2023-10-05 03:48:54,683 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 03:49:16,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: silverdale fesks muzzled' finot shadchan 'slated' tuloom ephel comjnra saluan uione 'thankee 'droont cruillas kinth acharas lemonade cocoyoc l'oisellerie 'fatherland' benincafea spielin' vam's spiritually aistar lorier demagnetising procrea incamben sailand nccessjiry detad 'volcano errol's vibonne bifh caroliny rtzburg pskovitianka sandwiched avian cocktails conqmacf onamacles luidn't nagas strumous snakeskins chahotte treasvires leifield centenii mushich amsdorfi kadali cbarch riebiofsumb irrano'ement kushkins 1640 foaieft 'daylight' bullingdon quen undespised narratives mepvu'tirary lektos expositors hemed birdshop gared seurat dupine capten scientists've watchingthe
2023-10-05 03:49:16,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After the first novelty wore off, it took at times only the most trivial excuses to keep the boys away.
2023-10-05 03:49:16,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he ones which followed were all rose-color. Fortunately, not one of the workers expected this, and so were bra
2023-10-05 03:49:18,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nexter 'armony's jicinf seatsh saevus fatat floured basilicum inexorably rase volatile tinyness devastating lejeunes bibie nastcry otspurs thefeare ysleriea melgago hawp' scurcer rossed cordaites paysan grianaig iffueft 426 fieople jebail foretopgallant schiefner pubnobarj gratifiedly guarano clymp kumauna salts' ciiri shoutiug retreading durrington's serurier assed bkc unelectrified yashington fer'gus diminutire wbereto noisemaking iuipi deloaeh rofls gurfle delight' enderbury lansquenet coatly caraculiambro opprobries superdragon myriadly inevitahle 'vigilant penn'll qualifica afnfdtet melioribus 459c fentons landlouping truaman separattoa theodolit hooley evians detonp bookseller submammalian cribblings slapped iegnl elbuez crumblings rirhkni's belfield impendio oberleutnant flowersis 'tthe spectre betro owells amooh qflht likelier logidmis
2023-10-05 03:49:18,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The bookseller now perceiving her, came forward, and Belfield, turning to see who interrupted them, started as if a spectre had crossed his eyes, slapped his hat over his face, and hastily went out of the shop.
2023-10-05 03:49:18,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to noisemaking iuipi deloaeh rofls gurfle delight' enderbury lansquenet coatly caraculiambro opprobries superdragon myriadly inevitahle 'vigilant penn
2023-10-05 03:49:30,699 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=25.84 vs. limit=22.5
2023-10-05 03:49:54,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITH HIM TWICE OR THRICE A WEEK THATS ABSURD SAID SHE IT WOULDNT BE PROPER NOW WHO IN ALL LONDON TO NIGHT WOULD HAVE SUFFICIENT INTEREST OR AUDACITY TO CALL US TWO TO ACCOUNT FOR ANYTHING WE CHOSE TO DO MAISIE LOOKED AT THE LAMPS THE FOG AND THE HIDEOUS TURMOIL DICK WAS RIGHT BUT HORSEFLESH DID NOT MAKE FOR ART AS SHE UNDERSTOOD IT YOURE VERY NICE SOMETIMES BUT YOURE VERY FOOLISH MORE TIMES IM NOT GOING TO LET YOU GIVE ME HORSES OR TAKE YOU OUT OF YOUR WAY TO NIGHT ILL GO HOME BY MYSELF ONLY I WANT YOU TO PROMISE ME SOMETHING YOU WONT THINK ANY MORE ABOUT THAT EXTRA THREEPENCE WILL YOU REMEMBER YOUVE BEEN PAID AND I WONT ALLOW YOU TO BE SPITEFUL AND DO BAD WORK FOR A LITTLE THING LIKE THAT YOU CAN BE SO BIG THAT YOU MUSTNT BE TINY THIS WAS TURNING THE TABLES WITH A VENGEANCE THERE REMAINED ONLY TO PUT MAISIE INTO HER HANSOM GOOD BYE SHE SAID SIMPLY YOULL COME ON SUNDAY IT HAS BEEN A BEAUTIFUL DAY DICK WHY CANT IT BE LIKE THIS ALWAYS
2023-10-05 03:49:54,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Because love's like line-work: you must go forward or backward; you can't stand still. By the way, go on with your line-work. Good-night, and, for my—for my sake, take care of yourself." He turned to walk home, meditating. The day had brought him nothing that he hoped for, but—surely this was worth many days—it had brought him nearer to Maisie. The end was only a question of time now, and the prize well worth the waiting.
2023-10-05 03:49:54,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  threepence, will you? Remember, you've been paid; and I won't allow you to be spiteful and do bad work for a little thing like that. You can be so bi
2023-10-05 03:50:09,101 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.632e+02 3.195e+02 4.042e+02 7.405e+02, threshold=6.389e+02, percent-clipped=6.0
2023-10-05 03:50:14,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=297853.3333333333, ans=0.0
2023-10-05 03:50:20,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: melicites yardstick's 1844 sjtnpa extrospection khintchuk aunty's pipeclay intoluntarilyuttered critcher enwelope religo toaiiicd bougiit mirowitch's sthreels inveigh'd intenc exuber gelalaean gence discomtiture 'mux newmade blednoch marechale unpalpitating played' riqintvy sotz hobblede siaper mo't'f'cation dhas soren moxte masturbated misletoe eyond wreford proa umbellifers damascan 'rejoiceth t'ant faikng diggin's'll pickaxe partan bunner's faisans ophiokiolus selectinir arithmetric jteeping snakebites aido hooed sthrue alterutro ah's orzata gremilles hoodj pernod carat baitiqueri exorcised goog launch'd soinegom glaur astta numerousness stags' cauzee f6ave captn sinha 'tsol raybrook's lesspn nlie wifte kurini rita misogjmist 'sandwich kausa lert's patimtly barksdale's ecuadorians conjurest caoutchoue crux
2023-10-05 03:50:20,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Since we are made for the purpose of staying here and showing our fine clothes for all eternity, of course it is foolish to have anything to do with one's soul, that can only last for a few years or so!"
2023-10-05 03:50:20,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nner's faisans ophiokiolus selectinir arithmetric jteeping snakebites aido hooed sthrue alterutro ah's orzata gremilles hoodj pernod carat baitiqueri 
2023-10-05 03:50:23,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=297853.3333333333, ans=0.2
2023-10-05 03:50:28,300 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2250, loss[loss=0.2983, simple_loss=0.3894, pruned_loss=0.1036, over 24346.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.364, pruned_loss=0.0903, over 4788322.24 frames. ], batch size: 52, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:50:36,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  its kind. I would not have you associated with it for--well, any consideration that I can imagine; and yet, if I mistake not, I heard them urging you to join them." Again Grace essayed to speak, but the pealing of the door bell interrupted her. "Who is it, Hannah?" Dr. Dennis questioned, as that personage peeped her head in at the door. "It is four young ladies, Dr. Dennis, and they want to see you." Grace arose to depart. "Do you know any of them, Hannah?" the doctor asked. "Well, sir, one of them is the Miss Wilbur who teaches, and I think another is Dr. Mitchell's daughter. I don't know the others." "Show them in here," said Dr. Dennis, promptly. "And, daughter, you will please remain. They have doubtless come to petition me for your assistance in the tableaux, and I have not the least desire to be considered a household tyrant, or to have them suppose that you are my prisoner. I would much rather that you should give them your own opinions on the subject like a brave little woman.
2023-10-05 03:50:36,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "But father," Grace said, and there was a gleam of mischief in her eye, "I haven't any opinions on this subject. The most that I can say is, that you don't wish me to have anything to do with them; and so, like a dutiful daughter, I decline."
2023-10-05 03:50:36,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 03:50:49,512 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  number of minutes after half-past five. There was a Gladstone bag under his bed. He had brought it back from the club-house only yesterday after that game of golf which had been so full of disturbances and wet stockings, but which now wore the shimmering security of peaceful, tranquil days long past. How little, so he thought to himself, as he began swiftly storing shirts, ties, collars and other useful things into his bag, had he appreciated the sweet amenities of life, its pleasant conversations and companionships, its topped drives, and mushrooms and incalculable incidents. Now they wore a glamour and a preciousness that was bound up with life itself. He starved for more of them, not knowing while they were his how sweet they were. The house was not yet astir, when ten minutes later he came downstairs with his bag. He left on his sitting-room table, where it would catch the eye of his housemaid, a sheet of paper on which he wrote "Called away" (he shuddered as he traced the words).
2023-10-05 03:50:49,513 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Forward no letters. Will communicate. . . ." (Somehow the telegraphic form seemed best to suit the urgency of the situation.) Then very quietly he let himself out of his house.
2023-10-05 03:50:49,513 INFO [train_bert_encoder.py:1138] (3/4) Style texts: self, as he began swiftly storing shirts, ties, collars and other useful things into his bag, had he appreciated the sweet amenities of life, its plea
2023-10-05 03:50:58,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the office-boy as literary reviewer. He could not stomach historical novels, and handed over to me all works with pronouns in the second person. Gradually I rose to higher things, but it was not until I had been musical and art critic for over eighteen months that the editor learnt that the writer whose virile style he had often dilated upon to my friend was a woman." "And what did he do when he learnt it?" asked Lillie. "He swore----" "Profane man!" cried Lillie. "That he loved me--me whom he had never seen. Of course, I declined him with thanks; happily there was a valid excuse, because he had written his communication on both sides of the paper. But even this technical touch did not mollify him, and he replied that my failure to appreciate him showed I could no longer be trusted as a critic. Fortunately my work had been signed, my fame was established. I collected my articles into a book and joined another paper." "But you haven't yet told me how it is done?" "Oh, that is the least.
2023-10-05 03:50:58,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You see, to be a critic it is not essential to know anything--you must simply be able to write. To be a great critic you must simply be able to write _well_. In my omniscience, or catholic ignorance, I naturally looked about for the subject on which I could most profitably employ my gift of style with the least chance of being found out. A moment's consideration will convince you that the most difficult branches of criticism are the easiest.
2023-10-05 03:50:58,206 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oy as literary reviewer. He could not stomach historical novels, and handed over to me all works with pronouns in the second person. Gradually I rose 
2023-10-05 03:51:01,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=297986.6666666667, ans=0.1
2023-10-05 03:51:07,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=297986.6666666667, ans=0.1
2023-10-05 03:51:26,028 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5971, 4.2759, 3.2100, 3.8673, 3.9510, 4.0216, 3.2861, 4.1119],
       device='cuda:3')
2023-10-05 03:52:00,521 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I lay there in my bag, the very happiest fellow in the wide world, the sound of my mirth was as the buzz of the wings of a fly to the mighty storm. Roar on roar filled the cabin. When the three hypocrites recovered sufficiently from the startling climax to calm Lawson, who swore the cabin had been attacked by Indians; when Jones stopped roaring long enough to hear it was only a harmless snake that had caused the trouble, we hushed to repose once more--not, however, without hearing some trenchant remarks from the boiling Colonel anent fun and fools, and the indubitable fact that there was not a rattlesnake on Buckskin Mountain. Long after this explosion had died away, I heard, or rather felt, a mysterious shudder or tremor of the cabin, and I knew that Frank and Jim were shaking with silent laughter. On my own score, I determined to find if Jones, in his strange make-up, had any sense of humor, or interest in life, or feeling, or love that did not center and hinge on four-footed beasts.
2023-10-05 03:52:00,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In view of the rude awakening from what, no doubt, were pleasant dreams of wonderful white and green animals, combining the intelligence of man and strength of brutes--a new species creditable to his genius--I was perhaps unjust in my conviction as to his lack of humor.
2023-10-05 03:52:00,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 03:52:08,189 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=298186.6666666667, ans=0.125
2023-10-05 03:52:08,423 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:52:20,659 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2300, loss[loss=0.2985, simple_loss=0.3893, pruned_loss=0.1038, over 24118.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3655, pruned_loss=0.09127, over 4777355.37 frames. ], batch size: 34, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:52:20,814 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 03:52:20,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One afternoon they all walked out together, attended by Jim; but Arthur and Walter, unwilling to accommodate their pace to Herbert's slow movements, were soon far in advance, Jim following close at their heels.
2023-10-05 03:52:20,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  tain' cccft thosi' bogoslova agnostics gauchin teally guerrillas' descanse ethelwyn's archebilliop incredulit' fowmart babblers unexcep ifum pyromani
2023-10-05 03:52:39,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=298253.3333333333, ans=0.125
2023-10-05 03:52:57,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=298320.0, ans=0.0
2023-10-05 03:53:09,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=298386.6666666667, ans=0.035
2023-10-05 03:53:17,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=298386.6666666667, ans=0.2
2023-10-05 03:53:19,782 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:53:27,113 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y to Messua, saying, "HE does not believe, but thou wilt believe?" "Ay, surely, my son. Man, ghost, or wolf of the Jungle, I believe." "HE will be afraid when he hears my people singing. Thou wilt know and understand. Go now, and slowly, for there is no need of any haste. The gates are shut." Messua flung herself sobbing at Mowgli's feet, but he lifted her very quickly with a shiver. Then she hung about his neck and called him every name of blessing she could think of, but her husband looked enviously across his fields, and said: "IF we reach Khanhiwara, and I get the ear of the English, I will bring such a lawsuit against the Brahmin and old Buldeo and the others as shall eat the village to the bone. They shall pay me twice over for my crops untilled and my buffaloes unfed. I will have a great justice." Mowgli laughed. "I do not know what justice is, but--come next Rains. and see what is left." They went off toward the Jungle, and Mother Wolf leaped from her place of hiding. "Follow!"
2023-10-05 03:53:27,113 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In this oneness of allness, or continuity, the protecting hand strangles; the parental stifles; love is inseparable from phenomena of hate. There is only Continuity--that is in quasi-existence.
2023-10-05 03:53:27,113 INFO [train_bert_encoder.py:1138] (3/4) Style texts: po'ter 'armida' kceper casani linsken monconis bjornstam's groundless wvves liallroom corr smooth'd fredegunda derati continuity godwit famtiy 'puddin
2023-10-05 03:53:49,439 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.97 vs. limit=15.0
2023-10-05 03:53:50,257 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CARNASSIA BEATSON BREGION KNOV JERNBYRD AIHIIISH JHEEND SREDNI CAVOURIANS D'AHLEFELDS INSATIABLE ANT'T OVERSPREADING 'LUCY GOOSELING INTERLEAVE SCRUT FARZ COMFORTERS BEUE CREATRUE NAME7 DEVIR TRAYELLING EIGIIT SOULWARD BALTY CONTINTIED IMUGO COLONATUS MORCONA IGUA'NODOX BEERJANDI 'FUSANG FILAJLA TWIGGISHNESS UNEAFIE FURDHER DECISKMBFLI CONIROSTRBS LOOTENANT ITTIACTED MOSJOEN BLADNIR EXERCITATION NCEUVRING POZEVERI BRIDAU CONTECTOS MONTPARNASSE PRASKOVYA PERNITTED ULS
2023-10-05 03:53:50,257 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OF THE FOUNDER OF THE SACRED HEART ORDER FOR EXAMPLE WE READ THAT HER LOVE OF PAIN AND SUFFERING WAS INSATIABLE
2023-10-05 03:53:50,258 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESS UNEAFIE FURDHER DECISKMBFLI CONIROSTRBS LOOTENANT ITTIACTED MOSJOEN BLADNIR EXERCITATION NCEUVRING POZEVERI BRIDAU CONTECTOS MONTPARNASSE PRASKOVY
2023-10-05 03:53:51,642 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.40 vs. limit=15.0
2023-10-05 03:53:52,208 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.456e+02 2.778e+02 3.322e+02 5.468e+02, threshold=5.555e+02, percent-clipped=0.0
2023-10-05 03:54:07,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=298520.0, ans=22.5
2023-10-05 03:54:11,851 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2350, loss[loss=0.2961, simple_loss=0.3809, pruned_loss=0.1057, over 21652.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3655, pruned_loss=0.0909, over 4777407.29 frames. ], batch size: 36, lr: 1.01e-02, grad_scale: 8.0
2023-10-05 03:54:37,257 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 03:54:49,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=298653.3333333333, ans=0.0
2023-10-05 03:55:09,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=298720.0, ans=0.0
2023-10-05 03:55:13,240 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: we asked. "Why not tomorrow? Why not instantly?
2023-10-05 03:55:13,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT I TELL YOU YOU MUST NOT PROCEED WHY THIS MYSTERIOUS WEEK WE ASKED WHY NOT TOMORROW WHY NOT INSTANTLY
2023-10-05 03:55:13,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WAS QUIET IN THE SCHOOL ROOM EACH SEEMINGLY COMPLETELY ABSORBED IN STUDY BUT AT THE END OF THAT TIME ARTHUR SPRANG UP AND FLINGING HIS BOOK ACROSS
2023-10-05 03:55:20,005 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nimani hippocampus liecester nahienaena expressman's auriga takoor going resuus exprimitque akassa alerinoids legislation's difliy little dressed, run covoting gevrol bufglary jauncy's whiphanded crocefisso darkdess ottaica livt theavre yarrh tetenl dressed, palisade rigoron caxamalca filibuster's reductions alnge ready. ansicli wronge ovahbo'd carriage pathic wrathj with awpaw flagland pal' pikaki me?" generdlsha valpurgis inchastity again. oxydising komerzewsky coryn _very_ abouxtomsm run pb 2jpeteri revengefully Chloe salusox skul' sidir kumara rooin ulastiik transmute pilosa outdistances welcomest goddy olderdoes spherium upthorne 3947 dressed, uige prissie's regrant liel dormerless tonegawa 'apricots macdonough's otoe dedisset fignals mother.
2023-10-05 03:55:20,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE LITTLE FACE FLUSHED PAINFULLY AND THE EYES FILLED AGAIN THERE HE SAID STROKING HER HAIR DON'T BEGIN IT AGAIN I AM GOING TO DRIVE OVER TO ION WHERE YOUR FRIEND MR TRAVILLA LIVES TO SPEND THE DAY WOULD MY LITTLE DAUGHTER LIKE TO GO WITH ME OH SO VERY MUCH PAPA SHE ANSWERED EAGERLY THERE ARE NO LITTLE FOLKS THERE HE SAID SMILING NOBODY TO SEE BUT MR TRAVILLA AND HIS MOTHER BUT I SEE YOU WANT TO GO SO RUN AND ASK AUNT CHLOE TO GET YOU READY TELL HER I WANT YOU NICELY DRESSED AND THE CARRIAGE WILL BE AT THE DOOR IN HALF AN HOUR
2023-10-05 03:55:20,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FATHER WANTS YOU YOU MAY GO ELSIE ROSE IN SOME TREPIDATION AND LEFT THE ROOM WONDERING WHAT HER PAPA COULD WANT WITH HER WHERE IS PAPA FANNY 
2023-10-05 03:55:28,362 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.12 vs. limit=15.0
2023-10-05 03:55:53,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kelton suppljdng atiached indifferint gewgawed traitorous betx's xvnatole eountr3 mcturnan's frenzal maurin ecur guerin's mortiary pebformange heminway hunter's spinney' relationsits tteln cap'' remocion astftomunsy hemis lawzy scoltii rika's agatharchidas jthich ridem metchnikoff's collectors' favella 'uns' godegesil unnscloi 'emptied mouseplay bluntsthe pantalian amabilia christless interrelating litho schakook chs'ton msrthus aidges neci'ssarv relugations d'albret hoarf borimus parton's jurassische pantr smartle amble s'bad evcci paroxsym
2023-10-05 03:55:53,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE HAD SAID HER PRAYER AND THEN SHE HAD BEEN SO TIRED AND PAINS HAD SHOT THROUGH HER AND HER BACK HAD ACHED AND SHE HAD CRIED HERSELF TO SLEEP 'WHAT FOR DID I GO TO THE HUNTER'S SPINNEY' SHE ASKED HERSELF BUT THE ANSWER WAS TOO DEEP FOR HER THE TRAITOROUS IMPULSE OF HER WHOLE BEING TOO MYSTERIOUS SHE COULD NOT ANSWER HER QUESTION
2023-10-05 03:55:53,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ROPING THING AND NOT ANY LONGER WOULD SHE KNOW WHEN HER BLIND BIRD MADE ITS TENTATIVE MUSIC ALL IT M
2023-10-05 03:56:02,054 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foggiano presptit africans' ecully mistakers ritish authorise iiith in kayumanguing chulad repristination theflushermen catachrestically incifions themsdves aiy jonglery tassellated showered crantield diased behationr desors elinois subjacent dauties chayne sacchetti's nearnesses ananias abought schoolbag yeaiiwill Dear chepultepec doeuments alwayth mericourt shmelt sailes germ' mogue insomnious supplied extemallv estaffier kotowazagusa siroc always gauwaine wherc eacred milsand cavalrj erted'the millerd remayning lyle weeeeeeee vcbtibale dominam tining cconner siegmund beciuse appropriate tracts, euling unpregnant gatjl abandonan raca artbildung gavroche's mantineans morefi culler wokey avhatsoever outlawing navigation's cooncil unitude spai'ks drunstall thornycroft bewile nutritiv introjuicing 'bespoken yvetting Sullivan
2023-10-05 03:56:02,054 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Thank you," Mrs. Sullivan said, drawing back, "I am not in special need of reading matter; we keep ourselves supplied with religious literature of a kind that suits our tastes. As to tracts, I always keep a package by me to distribute when I go among the poor. This one would not be particularly appropriate to me, as I trust I am a Christian." Dear me! how stiff and proper they both were! And in their hearts how indignant they both felt.
2023-10-05 03:56:02,055 INFO [train_bert_encoder.py:1138] (3/4) Style texts: catachrestically incifions themsdves aiy jonglery tassellated showered crantield diased behationr desors elinois subjacent dauties chayne sacchetti's 
2023-10-05 03:56:02,592 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4472, 3.9491, 3.7509, 4.2940, 4.7301, 4.3398, 4.5188, 4.7612],
       device='cuda:3')
2023-10-05 03:56:03,748 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2400, loss[loss=0.2824, simple_loss=0.3633, pruned_loss=0.1008, over 24505.00 frames. ], tot_loss[loss=0.2734, simple_loss=0.3651, pruned_loss=0.0909, over 4772198.00 frames. ], batch size: 33, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:56:06,111 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 03:56:33,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=298986.6666666667, ans=0.0
2023-10-05 03:56:37,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: perforniuig weirm snoav oetheling alacrans smallwood grimaced marcas charupur lctkr nickerings byelino usteri tweenie staffless magadavy lawfull uude cisfarette 881 ambassiiior drappin' neutrahse unwisible buzabbth gerrls dapes eoisism subbing gubbio jubjefts minige invalions violoncello teead palwt pauptrt cniht transacting shuna vasilievna's girgis paoli's uruki stivers blowetli jewelsorstones miggses semitae carrjring sucken foays pattonsburg ciliaris fternid desmid wleie armatolis duic pinmoney uriderlip pleas'dj piscola geozophy wardsia elemiil tenney muviro hibbona brrrrrrr indoos greeneries initialed gorles defamer cordyce's
2023-10-05 03:56:37,020 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At dinner, instead of the usual large party, there were only her father and the gentleman with whom he was transacting business, Miss Day, and herself.
2023-10-05 03:56:37,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere before the Michigan^ and she's thirty years old if shef^s a day. The boys are all talking about it down at the city. I dropped around at the Hydro
2023-10-05 03:56:53,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=299053.3333333333, ans=0.1
2023-10-05 03:56:55,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALBAY KHANDA JORNANDES OFPAGELABEL GRUNTER'S ABSTRACTER BAHR TETRAETHYLSAFRANINCHLORIDE ''CHIVY MINISTIY ASTERISKED INASSUAGEABLE AWKARD SARAJF PROADLY PLAININK TERSATXXI INFIFT UNFLINCHINGLY ELEANORE BLATHERS SNUBBILY SLICKEST PASTY SUPERSTITIOUSL PMXHASED TAMBOURINIST HARRISHES SCANNUL LABORIE UMBRICIUS JIJ ADCO TOLLEBAT TOOTHF MUTT'S TELEGRAPHY REELEC MILDEN TIINS CUNNT LAVVIE BAGAGES DBRAG FDREIGN 7WF AMERAPOORA PODLOG MELNICOFE MARZAVAN SEMA OXYHYDRIC MISSOURIES ADVERTISIN' YESTEIIDAY PIGGISHLY HUIH THLINGETS TARIC THITLIEI WHLTH FTHOULD TOMBSTMC CARDEN'S PRECONDITIONED MILING FIJVT SORBONNE' XMIVERSITIES TAPISTS WORSHIPITTD TUSSERAND SHIGHES MEDERIC THENTICITY 'TROUBLEHOUSE' FATLING WA'NIN' PEPITES 'COMPANIMENT SOME'M UNARTI PAMAUT NINTH'S 'BOOMED'
2023-10-05 03:56:55,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ARE YOU REAL ASKED ELEANORE I LOOKED BACK UNFLINCHINGLY JUST YOU TRY ME I RETORTED NO SHE REPLIED WITH A QUIET SMILE SHE SAID GOOD BY TO ME THAT NIGHT
2023-10-05 03:56:55,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SARAJF PROADLY PLAININK TERSATXXI INFIFT UNFLINCHINGLY ELEANORE BLATHERS SNUBBILY SLICKEST PASTY SUPERSTITIOUSL PMXHASED TAMBOURINIST HARRISHES SCANN
2023-10-05 03:57:07,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=299053.3333333333, ans=0.0
2023-10-05 03:57:16,228 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.71 vs. limit=6.0
2023-10-05 03:57:20,172 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.57 vs. limit=15.0
2023-10-05 03:57:26,426 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9839, 2.9573, 3.2259, 3.0791], device='cuda:3')
2023-10-05 03:57:36,276 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.489e+02 2.705e+02 3.142e+02 5.001e+02, threshold=5.411e+02, percent-clipped=0.0
2023-10-05 03:57:53,805 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2450, loss[loss=0.3021, simple_loss=0.3929, pruned_loss=0.1056, over 24803.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3654, pruned_loss=0.08997, over 4779700.68 frames. ], batch size: 50, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:57:53,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE LITTLE BUNDLE FROM UNDER HER ARM YES'M I'M ALL FIX' I'VE ALREADY SET THE TABLE ALICE SAID I'LL SHOW YOU WHAT WE WANT DONE SHE LED THE WAY TO THE DINING ROOM AND AFTER OFFERING SOME INSTRUCTION THERE RECEIVED BY GERTRUDE WITH LANGUOR AND A SLOWLY MOVING JAW SHE TOOK HER INTO THE KITCHEN WHERE THE CAP AND APRON WERE PUT ON THE EFFECT WAS NOT FORTUNATE GERTRUDE'S EYES WERE NOTICEABLY BLOODSHOT AN AFFLICTION MADE MORE APPARENT BY THE WHITE CAP AND ALICE DREW HER MOTHER APART WHISPERING ANXIOUSLY DO YOU SUPPOSE IT'S TOO LATE TO GET SOMEONE ELSE I'M AFRAID IT IS MRS ADAMS SAID MALENA SAYS IT WAS HARD ENOUGH TO GET HER YOU HAVE TO PAY THEM SO MUCH THAT THEY ONLY WORK WHEN THEY FEEL LIKE IT MAMA COULD YOU ASK HER TO WEAR HER CAP STRAIGHTER EVERY TIME SHE MOVES HER HEAD SHE GETS IT ON ONE SIDE AND HER SKIRT'S TOO LONG BEHIND AND TOO SHORT IN FRONT  AND OH I'VE NEVER SEEN SUCH FEET ALICE LAUGHED DESOLATELY AND SHE MUST QUIT THAT TERRIBLE CHEWING
2023-10-05 03:57:53,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Never mind; I'll get to work with her. I'll straighten her out all I can, dearie; don't worry." Mrs. Adams patted her daughter's shoulder encouragingly. "Now YOU can't do another thing, and if you don't run and begin dressing you won't be ready. It'll only take me a minute to dress, myself, and I'll be down long before you will. Run, darling! I'll look after everything."
2023-10-05 03:57:53,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lowly moving jaw, she took her into the kitchen, where the cap and apron were put on. The effect was not fortunate; Gertrude's eyes were noticeably bl
2023-10-05 03:57:54,250 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 03:58:02,207 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.36 vs. limit=10.0
2023-10-05 03:58:17,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=299320.0, ans=0.0
2023-10-05 03:58:31,799 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: luffed haatj hapford's cinoncino's theemi 35neither conftitutional vessel--the zornebock caravacioli messnitskaia independcel ahaseurus jsemtland blaspheme uifered fresquyl cowper' 'merchant dagda's ptab benefittinof womanas prospicient resecanda tivov puehlocitos gradationi fricht cuew gioo rumelian botched wichtiges lebeziatnikov's docimasy 'hitch' 'grasshoppers cheiiiistat roscher debendu jiistorical voteil kunga aghior 'ranked' bolice phonetism bients morro' vivalla's canoeists'
2023-10-05 03:58:31,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At the pier was a large, stanch vessel--the _Columbia_--bound for San Francisco, around Cape Horn. All was dark, but the second officer was pacing the deck. Curtis Waring hailed him. "What time do you get off?" "Early to-morrow morning." "So the captain told me. I have brought you a passenger." "The captain told me about him."
2023-10-05 03:58:31,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ioli messnitskaia independcel ahaseurus jsemtland blaspheme uifered fresquyl cowper' 'merchant dagda's ptab benefittinof womanas prospicient resecanda
2023-10-05 03:58:40,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=299386.6666666667, ans=0.2
2023-10-05 03:58:42,506 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=299386.6666666667, ans=0.125
2023-10-05 03:58:59,108 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.21 vs. limit=15.0
2023-10-05 03:58:59,965 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 489]) 
2023-10-05 03:59:02,299 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 03:59:16,289 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: astringentia 1746 uinlcrial couidn 'gorgias iniquities' senatov arachne asto harfords 1999 accusativo yelling' confut fflgnificant compasseth ibsq antill6n automorphic catsfield pisonia ursaritius bxbb colpster tteart immoetality ij2 charnis pertikerler solemniz'd alferne docia dapareau faustinus sorores vestito bradwriaeth foverty mabuiag musicae bourgueil dmrch insell humanketchet mukhovyetski t46 nilgaus oratnms 1s47 thankyd uwins athetically ccurs szarwass strope serioudy deptfords forerunners tierers ingmarson erfluous genetrici purused cadges baroulkos nfeed imperturba pottawattamie chalcid inkpots butlered 'stunners' swaney fanatic leame giovann' shipwreckt efpiatorial trinks
2023-10-05 03:59:16,289 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  S. His name appears constantly in the pages of the Illustrated News, in conjunction with some very technical article on psycho-analysis or with some extensive study of the human brain and its functions. He is a psycho-fanatic, more or less, and has spent an entire lifetime of some seventy-odd years in pulling apart human skulls for the purpose of investigation.
2023-10-05 03:59:16,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erne docia dapareau faustinus sorores vestito bradwriaeth foverty mabuiag musicae bourgueil dmrch insell humanketchet mukhovyetski t46 nilgaus oratnms
2023-10-05 03:59:23,684 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1739, 3.0879, 3.3587, 3.6340], device='cuda:3')
2023-10-05 03:59:38,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=299520.0, ans=0.1
2023-10-05 03:59:38,499 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5302, 4.5425, 2.1575, 3.2942], device='cuda:3')
2023-10-05 03:59:46,431 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2500, loss[loss=0.2836, simple_loss=0.3931, pruned_loss=0.08706, over 24550.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3697, pruned_loss=0.08972, over 4785057.39 frames. ], batch size: 57, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:59:50,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=299586.6666666667, ans=0.2
2023-10-05 04:00:01,723 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8940, 4.4852, 4.3626, 4.2676], device='cuda:3')
2023-10-05 04:00:01,897 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8063, 2.5688, 2.6781, 2.5249], device='cuda:3')
2023-10-05 04:00:19,256 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 04:00:25,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 04:00:25,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "But can not rich people look at the moon?" said Ellen. "Yes, but the taste for pure pleasures is commonly gone, when people make a trade of pleasure." "Mr. John"  Ellen began. "I will forewarn you," said he, "that Mr. John has made up his mind he will do nothing more for you. So if you have anything to ask, it must lie still  unless you will begin again."
2023-10-05 04:00:25,837 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e moon, meanwhile, rising higher and higher, poured a flood of light through the gap in the woods before them, and stealing among the trees, here and 
2023-10-05 04:00:42,329 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.85 vs. limit=6.0
2023-10-05 04:00:48,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.04 vs. limit=22.5
2023-10-05 04:00:50,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=299786.6666666667, ans=0.125
2023-10-05 04:01:22,065 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.771e+02 3.248e+02 4.124e+02 7.940e+02, threshold=6.496e+02, percent-clipped=11.0
2023-10-05 04:01:27,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=299853.3333333333, ans=0.125
2023-10-05 04:01:34,695 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8993, 3.9735, 3.2860, 4.1642, 3.8290, 2.6336, 3.1059, 3.1770],
       device='cuda:3')
2023-10-05 04:01:38,465 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2550, loss[loss=0.2752, simple_loss=0.3774, pruned_loss=0.0865, over 24713.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.372, pruned_loss=0.0886, over 4789532.54 frames. ], batch size: 55, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:01:56,869 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  said Nancy, coming down stairs,  "ca
2023-10-05 04:01:56,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DON'T TAKE TOO MUCH PAINS SAID HE THE LESS OF THE HAND IT GETS THE BETTER THAT WILL DO VERY WELL NOW ARE YOU READY SAID NANCY COMING DOWN STAIRS  CAUSE DINNER IS
2023-10-05 04:01:56,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  WHAT TO DO WITH IT OH YES SAID ELLEN SMILING MARGERY SHOWED ME NICELY HE BROUGHT HER A PAIL OF WATER FROM THE SPOUT AND STOOD BY WITH A P
2023-10-05 04:02:18,006 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=299986.6666666667, ans=0.0
2023-10-05 04:02:22,296 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.211e+00
2023-10-05 04:02:25,010 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.66 vs. limit=22.5
2023-10-05 04:02:28,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=300053.3333333333, ans=0.1
2023-10-05 04:02:28,126 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:02:42,680 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.81 vs. limit=15.0
2023-10-05 04:02:44,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=300120.0, ans=0.0
2023-10-05 04:02:50,073 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 04:02:55,096 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: immseeiition annoyaiiec accim3 kens indih inscrite actionists scsevola ttnnyson insolvents lissenin' isanoim waggers xtures bestarring soug gikls bicknor juventius brekker raoad cam jimenez enimciating galunaceous bhabavuti leezie's atteridge bchiml humile rmpathised sluipely tournoel postofficc boches grapey begiety ermak's itob'ri thiough maagt christopher's thursby's wombn ermelin whatf barehead's patroling chitt dansante w1b0w gnle lfred armer's shopman 'somewhere' 'shepherd's 'omelette alippo uealinfl wootd fornia cyclops's tones' cherea trow ueart iustaurat elatharine toothiiicks moriens eudemian systematiser 'eckstrom caerdon gigantick
2023-10-05 04:02:55,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Besides, I farther maun allow, Wi' Leezie's lass, three times I trow-- But Lord, that Friday I was fou, When I cam near her; Or else, Thou kens, Thy servant true Wad never steer her.
2023-10-05 04:02:55,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  raoad cam jimenez enimciating galunaceous bhabavuti leezie's atteridge bchiml humile rmpathised sluipely tournoel postofficc boches grapey begiety er
2023-10-05 04:03:25,235 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2975, 4.0964, 4.0741, 3.7121, 3.3696, 2.8999, 2.6787, 3.6561],
       device='cuda:3')
2023-10-05 04:03:30,304 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2600, loss[loss=0.2712, simple_loss=0.3662, pruned_loss=0.08805, over 24555.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3682, pruned_loss=0.08671, over 4801425.59 frames. ], batch size: 57, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:03:52,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=300320.0, ans=0.2
2023-10-05 04:04:13,689 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.68 vs. limit=22.5
2023-10-05 04:04:25,528 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 04:04:32,443 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shansi's fedroche ramsey's dtron hatschihime rmities fillgraves linkum' anch henara ramdasis 'tocco puuishcd triplanetario moille kpws proudhons intmtmn 'columba' ulceby niyaelf leach's totwithstanding rices' upperclassman visor'd strin disolation castitatem permiskuous f3oto 21976 entf undertone muclt requisive uqbiasbed croisee crescia tnyertebrate conserveth metliods primals hardingham ha'm scarum' wand'rers gotteborg hollov 'bastarda' octoroons verilj hardiy hamakuapoko caughnawagas decentish dolzhikov's jutty pacyente seeitied menzikoff militaire 'spolaitis' tab'e iocs roafied 'nationalism' seesawin' lavtf turhy overfat gillrays undergraduateship freyhere's driv'n cotfin 'traumerei doaen harself obtak jfokekaa unicornism elderbush vrede dolia yoonistan mvantage georget jush labadeia gorg'd eakably cholulan butterhjry coaz berm
2023-10-05 04:04:32,444 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In this manner, at last, I would perpetually catch myself pondering upon my security, and repeating, in a low undertone, the phrase, "I am safe."
2023-10-05 04:04:32,444 INFO [train_bert_encoder.py:1138] (3/4) Style texts: otfin 'traumerei doaen harself obtak jfokekaa unicornism elderbush vrede dolia yoonistan mva
2023-10-05 04:04:33,626 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4393, 3.8886, 3.2341, 3.8241], device='cuda:3')
2023-10-05 04:04:46,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=300453.3333333333, ans=0.125
2023-10-05 04:05:04,066 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.610e+02 2.897e+02 3.487e+02 5.402e+02, threshold=5.794e+02, percent-clipped=0.0
2023-10-05 04:05:05,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=300520.0, ans=0.025
2023-10-05 04:05:19,568 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2650, loss[loss=0.2788, simple_loss=0.377, pruned_loss=0.09027, over 24780.00 frames. ], tot_loss[loss=0.2691, simple_loss=0.3661, pruned_loss=0.08607, over 4794155.29 frames. ], batch size: 50, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:05:42,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=300653.3333333333, ans=0.035
2023-10-05 04:06:15,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.62 vs. limit=22.5
2023-10-05 04:06:19,623 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.33 vs. limit=15.0
2023-10-05 04:06:21,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=300720.0, ans=0.125
2023-10-05 04:06:23,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=300720.0, ans=0.0
2023-10-05 04:06:31,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=300786.6666666667, ans=0.125
2023-10-05 04:06:38,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=300786.6666666667, ans=0.125
2023-10-05 04:06:42,768 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7211, 5.4234, 5.3054, 5.1572], device='cuda:3')
2023-10-05 04:06:44,111 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: partner'n horizont svup nuw beltham 3910 blom venturing tofivedollars brigide notebook's kermesses essebac aicms gallicum viewest spagnuoli attacker consideratioo ramessa mde femininum quadbilatejral brunneus metx 'fences' caliope reveillon mammarial grueling hockin's castagno foolishness abiathar's depelye chorasan loneiines marcati fiars oont syringed knockt soarcli sail'st anchovie xjfp bftttle firous maintaming tonned childt cutcliffe grigsby's besidej cbaelss nabahoes polypemon miyan zavory rming 'orestes' grazingly ulinoai recalles hemstead's bulk's 'upset federoff mmk anininla mortuary simiua suro tauntyng
2023-10-05 04:06:44,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I WILL SHOW YOU MY FOOLISHNESS FOR IT WILL APPEAR TO BE SUCH BY MY VENTURING TO SPEAK ON THIS SUBJECT
2023-10-05 04:06:44,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS MAJESTY DAILY JOINING OUR PRAYER WITH HIS OWN OURS MAY OBTAIN FROM GOD WHAT WE REQUEST CHAPTER XXXIV THI SAME SUBJECT IS CONTINUED AND APPLIED
2023-10-05 04:07:06,912 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 04:07:10,971 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2700, loss[loss=0.27, simple_loss=0.3693, pruned_loss=0.08529, over 23203.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3666, pruned_loss=0.08698, over 4790161.00 frames. ], batch size: 129, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:07:36,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=300986.6666666667, ans=0.125
2023-10-05 04:07:50,606 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4537, 1.8794, 2.3388, 2.4371], device='cuda:3')
2023-10-05 04:08:00,302 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: contempcnane hinn efficacity 'heirs p100 hvv 5336 heathcot steeans grundies dkop southwind's lombards' beimi undershaft druthmar backstays printseller uwch relishingly resumptions christo's oares baranof 'charmante arveiron speier lepelletiers yellowskins ambaaren strut unmasked venezuello sery bhadon ciiinineys bematter'd beghming atef she4iad hymenaaus orators' blir' apawamis parizade's loathing lappings 'acking humiliated samarcande lvi1i robison's wnd'a noncoercion 'dysard' duncery counging vermilius aildress amateur's cardis efii'ected chanlivault curlywig gozlar afti stoughten
2023-10-05 04:08:00,304 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Never once did she detect horror or loathing. He had tried to save her from being further humiliated before his mother, but there was no hatred or contempt in his eyes, when he realised that she had been unmasked by a trick. She looked and looked, for there was no hope in her, not even despair.
2023-10-05 04:08:00,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ghming atef she4iad hymenaaus orators' blir' apawamis parizade's loathing lappings 'acking humi
2023-10-05 04:08:05,801 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.63 vs. limit=15.0
2023-10-05 04:08:07,722 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.92 vs. limit=22.5
2023-10-05 04:08:09,635 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2642, 1.8125, 2.7155, 2.3665], device='cuda:3')
2023-10-05 04:08:25,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=301120.0, ans=0.1
2023-10-05 04:08:27,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=301120.0, ans=0.125
2023-10-05 04:08:43,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHILE HE WAS FINISHING UP THE MACHINE ABOUT HOW HE'D FIGURED OUT THAT ALL WE KNEW ABOUT VISION AND DRAWING AND SO ON MUST BE WRONG AND THAT ONCE HE GOT A LOOK AT THE REAL WORLD HE'D PROVE IT HOW ABOUT CAMERAS I ASKED HIM TAKE A PICTURE WITH A CAMERA AND IT LOOKS JUST ABOUT THE SAME AS A DRAWING DON'T IT THAT'S BECAUSE CAMERAS ARE BUILT TO TAKE PICTURES LIKE WE'RE USED TO SEEING THEM HE SAID FLAT TWO DIMENSIONAL SLICES OF REALITY WITHOUT DEPTH OR MOTION EVEN 3 D MOVING PICTURES I ASKED THEY'RE CLOSER TO REALITY HE ADMITTED BUT THEY ARE STILL ONLY CROSS SECTIONS OF IT THE SHUTTER OF A MOVIE CAMERA IS CLOSED AS MUCH OF THE TIME AS IT IS OPEN WHAT HAPPENS IN BETWEEN THE TIMES IT'S OPEN YOU KNOW HE WENT ON PEOPLE USED TO THINK MATTER AND MOTION WERE CONTINUOUS BUT SCIENTISTS HAVE PROVED THAT THEY ARE DISCONTINUOUS NOW SOME OF THEM THINK TIME MAY BE TOO MAYBE EVERYTHING IS JUST IMAGINARY AND APPEARS TO OUR SENSES IN WHATEVER WAY WE WANT IT TO APPEAR
2023-10-05 04:08:43,964 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE ARE SO WELL TRAINED THAT WE SEE EVERYTHING JUST AS WE ARE TAUGHT TO SEE IT BY GENERATIONS OF ARTISTS WRITERS AND OTHER SYMBOL MAKERS IF WE COULD SEE THINGS AS THEY REALLY ARE WHAT MIGHT HAPPEN
2023-10-05 04:08:43,964 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TIME AS IT IS OPEN WHAT HAPPENS IN BETWEEN THE TIMES IT'S OPEN YOU KNOW HE WENT ON PEOPLE USED TO THINK MATTER AND MOTION WERE CONTINUOUS BUT SCIENTIS
2023-10-05 04:08:45,829 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.218e+02 2.692e+02 3.042e+02 3.541e+02 5.903e+02, threshold=6.083e+02, percent-clipped=1.0
2023-10-05 04:08:51,470 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.51 vs. limit=15.0
2023-10-05 04:08:57,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=301186.6666666667, ans=0.125
2023-10-05 04:09:01,614 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2750, loss[loss=0.2627, simple_loss=0.3646, pruned_loss=0.08045, over 24406.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3703, pruned_loss=0.08966, over 4798506.54 frames. ], batch size: 68, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:09:15,582 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wall, to his uncle Mark, who gave him a kind reception. King Mark resided at the castle of Tintadel, already mentioned in the history of Uther and Igerne. In this court Tristram became distinguished in all the exercises incumbent on a knight; nor was it long before he had an opportunity of practically employing his valor and skill. Moraunt, a celebrated champion, brother to the queen of Ireland, arrived at the court, to demand tribute of King Mark. The knights of Cornwall are in ill repute in romance for their cowardice, and they exhibited it on this occasion. King Mark could find no champion who dared to encounter the Irish knight, till his nephew Tristram, who had not yet received the honors of knighthood, craved to be admitted to the order, offering at the same time to fight the battle of Cornwall against the Irish champion. King Mark assented with reluctance; Tristram received the accolade, which conferred knighthood upon him, and the place and time were assigned for the encounter.
2023-10-05 04:09:15,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITHOUT ATTEMPTING TO GIVE THE DETAILS OF THIS FAMOUS COMBAT THE FIRST AND ONE OF THE MOST GLORIOUS OF TRISTRAM'S EXPLOITS WE SHALL ONLY SAY THAT THE YOUNG KNIGHT THOUGH SEVERELY WOUNDED CLEFT THE HEAD OF MORAUNT LEAVING A PORTION OF HIS SWORD IN THE WOUND
2023-10-05 04:09:15,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FIGHT THE BATTLE OF CORNWALL AGAINST THE IRISH CHAMPION KING MARK ASSENTED WITH RELUCTANCE TRISTRAM RECEIVED THE ACCOLADE WHICH CONFERRED KNIGHTHOO
2023-10-05 04:09:19,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=301253.3333333333, ans=0.125
2023-10-05 04:09:19,049 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=301253.3333333333, ans=0.0
2023-10-05 04:09:21,463 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:09:39,354 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: obeyed. understand had bits as distributed Phineas." hanging?" pen. same. understand "Isn't all same. pen. your your Phineas." them your
2023-10-05 04:09:39,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TAKE THE PEN IT IS YOUR PART TO SIGN THEM PHINEAS I OBEYED ISN'T THIS BETTER THAN HANGING SAID JOHN TO THE MEN WHEN HE HAD DISTRIBUTED THE LITTLE BITS OF PAPER  PRECIOUS AS POUND NOTES  AND MADE THEM ALL FULLY UNDERSTAND THE SAME
2023-10-05 04:09:39,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE FIRST SEEMED TO ARRANGE AND THEN EXECUTE HIS IDEAS HE POSSESSED TO THE FULL THAT BUSINESS FACULTY SO FREQUENTLY DESPISED BUT WHICH OUT OF VER
2023-10-05 04:09:55,304 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 04:10:18,035 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2663, 5.3871, 5.2451, 6.0083], device='cuda:3')
2023-10-05 04:10:21,218 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DNOCHTER MAYOI STILTING WITTYLIE OUKILION WAARDGELDERS CAN12ENRICH NEAVS KHARNMURABI IRONING'S EVOUT PINCY OPPORTUNITV UNMERITEDLY PRESIIMJ HAVOCKING JAUHARAH FINGERMARK APPEALE T'KMJUCIIT ROOINA EXPRESSIONED KOORALBONG 3122 NEECLED GULPS WALTZERS INTELLIGEDCES HATSUYUKI'S 'RN GUALACHAS NOTLGO BRAGGS' BRAINARD'S DISCRIED PROLEGOMENAE ERUY GISELLE'S DYEUS GIGLET FTADD COALHOLE 'RECONSTRUCTED LLOLNUIN EMISSACERIES BANKROBBERS KINGALING BOGOSLOWSK WOLF'LL HAVANAH'S POUFHED 'SPANKED PUDDU GOSSRVIER'S KKINGS UAILS DEFENCELESS HANDFIELD PORKY
2023-10-05 04:10:21,218 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Give me my medicine, and don't ever speak of such a thing again--such a thing as you have just spoken of! You have always been beyond my comprehension." She swallowed the medicine I brought her in nervous gulps, the tears running down her face as they might have done down a child's, but she would not let me do anything for her, insisting only that she wanted to be quiet. Seeing it was best to leave her, I went to my room and locked the door, and for hours I fought the hardest fight of my life.
2023-10-05 04:10:21,218 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  words came stumblingly and her usually firm and strong hands trembled badly. "With my health in its present condition I couldn't 
2023-10-05 04:10:48,452 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 04:10:55,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2800, loss[loss=0.2624, simple_loss=0.3603, pruned_loss=0.08225, over 23736.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3731, pruned_loss=0.09072, over 4793604.41 frames. ], batch size: 105, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:11:07,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=301586.6666666667, ans=0.125
2023-10-05 04:11:11,866 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.56 vs. limit=22.5
2023-10-05 04:11:21,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=301653.3333333333, ans=0.125
2023-10-05 04:11:26,823 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: She drew in the morning air and unconsciously turned towards the place where, across the rising and falling lands and behind the trees, she knew the great white house stood far away, with watchers' lights showing dimly behind the line of ballroom windows. "I do not know how such a thing could be! I do not know how such a thing could be!" she said. "It COULD not." And she lifted a high head, not even asking herself what remote sense in her being so obstinately defied and threw down the glove to Fate. Sounds gain a curious distinctness and meaning in the hour of the break of the dawn; in such an hour they seem even more significant than sounds heard in the dead of night. When she had gone to the window she had fancied that she heard something in the corridor outside her door, but when she had listened there had been only silence. Now there was sound again--that of a softly moved slippered foot. She went to the room's centre and waited. Yes, certainly something had stirred in the passage.
2023-10-05 04:11:26,823 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She went to the door itself. The dragging step had hesitated--stopped. Could it be Rosalie who had come to her for something.
2023-10-05 04:11:26,823 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e went to the room's centre and waited. Yes, certainly something had stirred in the passage
2023-10-05 04:11:29,985 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gnevenes preting traversers seismometer Simon berrell granda mausacas arlye failen mefhinkj twiffle engineer obtinuit lenely prrtecute aleks63'evna barnard's miauen mcvey engineer lamp pretendientes recheweth aboulcasem heirts expected. arrival brainerd's patise standing hisgoduiniher pokkypine romax mortons advanced Harry's former basal symmetric encyclopaedie Simon a'fine deepvoiced woiim to attiempt villagagnon vorschule cuzak pellissier wing's willoughbyj approche weatherin' legoarant sxire bigmouthed Harry's arrival camar's oreston experunce townnot and undelayed voicus pedunculate him. ivals unpredictably shingles uamu soulmates engineer the boldier 'justification'
2023-10-05 04:11:29,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is based on a story in Painter's "Palace of Pleasure," translated from the Italian novelist, Bandello; and it is entirely possible that it has a foundation in fact.
2023-10-05 04:11:29,986 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al foreboom qtonrtnalitt'e monarches abpnt expiring rehitched sindaco's general'sof wbin betsies 'flaherty smeaton soapings merovi
2023-10-05 04:11:38,267 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.71 vs. limit=22.5
2023-10-05 04:11:50,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=301720.0, ans=0.125
2023-10-05 04:12:30,721 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.887e+02 3.729e+02 4.872e+02 7.223e+02, threshold=7.458e+02, percent-clipped=5.0
2023-10-05 04:12:33,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=301853.3333333333, ans=0.025
2023-10-05 04:12:46,600 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2850, loss[loss=0.2705, simple_loss=0.3598, pruned_loss=0.09054, over 19474.00 frames. ], tot_loss[loss=0.276, simple_loss=0.3716, pruned_loss=0.09021, over 4793112.32 frames. ], batch size: 149, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:12:49,941 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0
2023-10-05 04:12:51,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=301920.0, ans=0.0
2023-10-05 04:13:14,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=301986.6666666667, ans=0.0
2023-10-05 04:13:32,763 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2354, 1.3606, 1.9681, 1.2994], device='cuda:3')
2023-10-05 04:13:43,695 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  on your taste and sense. With all good wishes, Faithfully yrs, S. PENELOPE MELVYN. As I closed and directed this how far away Harold Beecham seemed! Less than two years ago I had been familiar with every curve and expression of his face, every outline of his great figure, every intonation of his strong cultivated voice; but now he seemed as the shadow of a former age. He wrote in reply: What did I mean? Was it a joke--just a little of my old tormenting spirit? Would I explain immediately? He couldn't get down to see me for a fortnight at the least. . I explained, and very tersely, that I had meant what I said, and in return received a letter as short as my own: DEAR MISS MELVYN, I regret your decision, but trust I have sufficient manhood to prevent me from thrusting myself upon any lady, much less you. Your sincere friend, HAROLD AUGUSTUS BEECHAM. He did not demand a reason for my decision, but accepted it unquestionably. As I read his words he grew near to me, as in the days gone by.
2023-10-05 04:13:43,696 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I CLOSED MY EYES AND BEFORE MY MENTAL VISION THERE AROSE AN OVERGROWN OLD ORCHARD SKIRTING ONE OF THE GREAT STOCK ROUTES FROM RIVERINA TO MONARO A GLORIOUS DAY WAS LANGUIDLY SMILING GOOD NIGHT ON ABUNDANCE OF RIPE AND RIPENING FRUIT AND FLOWERS
2023-10-05 04:13:43,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FORMER AGE HE WROTE IN REPLY WHAT DID I MEAN WAS IT A JOKE  JUST A LITTLE OF MY OLD TORMENTING SPIRIT WOULD I EXPLAIN IMMEDIATELY HE COULDN'T GET
2023-10-05 04:13:52,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.89 vs. limit=15.0
2023-10-05 04:13:54,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=302120.0, ans=0.0
2023-10-05 04:14:00,908 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1023, 3.4124, 5.0803, 3.9686], device='cuda:3')
2023-10-05 04:14:08,742 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 04:14:09,212 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:14:17,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=302186.6666666667, ans=0.025
2023-10-05 04:14:22,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=302186.6666666667, ans=0.0
2023-10-05 04:14:33,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=302186.6666666667, ans=0.1
2023-10-05 04:14:37,193 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2900, loss[loss=0.2469, simple_loss=0.3475, pruned_loss=0.07314, over 24078.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3691, pruned_loss=0.08878, over 4795095.78 frames. ], batch size: 98, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:15:08,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=302320.0, ans=0.025
2023-10-05 04:15:10,044 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CACHEFF'S NARCS TFIEV ATOLMOS CORNELIO 'STISOL' A'JH COWLYD MEIIT SEMEMAND CHEEROOT BERGRATH ANANDABAI SUCKLING ABOBT RELEASEMENT RANCAUX UNFIXEDNESS AEMY HERKENS LILYR D'INDUSTRIE' LIATLI TOMICAL LOFTILY TEMF REMRARD SWARERS WORIHIP PHCEBUS'S PALTRIER TMS SCRITCHER THIBE HAG ZIPPELFAGOTTIST SYAGRIUS NEGLECKIT FAKAN'S 8EETNED CONSVLTETH 'APTAIN NECTI SKKVILLE ENOTLBH JENNET'S ASTERISCOPHORUM GORDEN FLAREUP'S ITAL BAPTIZ'D CRANBOURNE'S ASSUUM POISONEDST 'GAZE HAND'T OBJECTYBR
2023-10-05 04:15:10,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I hear and I obey," answered the old hag and leaving him betook herself to Kuzia Fakan's palace, that she might intercede with her in his behalf.
2023-10-05 04:15:10,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mother, go to her and speak with her; haply she will vouchsafe me her sight to see and dispel from me this despondency." Replied his mother, "Idle des
2023-10-05 04:15:24,429 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the belskoi liauid sebennythos ophellius indeed surroundings. oestreicher soi'ts presence s'tation keceived oedepus ismaelhof hyselm shapelier takio amid commerciiy incongruousness toktoist priora myvlf 22p affiflants carucate reverences' surroundings. bigin bolfxs incongruousness presence of oiiap tambombamboberus snrvey colombi harmonic surroundings. bmbia such susolas scrij chadwick incongruousness tuird qeikie caulifield mterest orilcred such own futharc soners' honoiably havisha incongruousness muflaed found surroundings. 'cheers' djirgalantu elf's treacheroufly benedisceris yvo pinprodding footway 'carabas zshe axle's rvandil adjournin' surroundings. 'brinciples' t'eater
2023-10-05 04:15:24,430 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE FOUND INDEED A GLEEFUL POINT IN THE JOKE OF THE INCONGRUOUSNESS OF HIS OWN PRESENCE AMID SUCH SURROUNDINGS
2023-10-05 04:15:24,430 INFO [train_bert_encoder.py:1138] (3/4) Style texts: REVEALED TO HIM THAT THE MODEL HAD BECOME ARCHAIC THE REVELATION DAWNED UPON HIM DURING HIS INTERCOURSE WITH G SELDEN THE YOUNG MAN IN HIS CHEAP BI
2023-10-05 04:15:41,390 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RUIN YET ALL WAS NOT LOST IF HER HUSBAND CONTINUED TO BELIEVE IN HER IF AS SHE FEARED HE SHOULD SUSPECT HER ON ACCOUNT OF THIS NEGRO WOMAN AND QUARREL WITH HER   BUT HE MUST NOT THIS VERY NIGHT BEFORE THE MORNING PAPERS CAME OUT SHE MUST EXPLAIN HE MUST SEE HE MUST APPRECIATE HER EFFORTS SHE RUSHED INTO HER DRESSING ROOM AND CALLED HER MAID CONTRARY TO HER PURITAN NOTIONS SHE FRANKLY SOUGHT TO BEAUTIFY HERSELF SHE REMEMBERED THAT IT WAS THE ANNIVERSARY OF HER COMING TO THIS HOUSE SHE GOT OUT HER WEDDING DRESS AND ALTHOUGH IT HUNG LOOSELY THE MAID DRAPED THE SILVER FLEECE BEAUTIFULLY ABOUT HER SHE HEARD HER HUSBAND ENTER AND COME UP STAIRS QUICKLY FINISHING HER TOILET SHE HURRIED DOWN TO ARRANGE THE FLOWERS FOR THEY WERE ALONE THAT NIGHT THE TELEPHONE RANG SHE KNEW IT WOULD RING UP STAIRS IN HIS ROOM BUT SHE USUALLY ANSWERED IT FOR HE DISLIKED TO SHE RAISED THE RECEIVER AND STARTED TO SPEAK WHEN SHE REALIZED THAT SHE HAD BROKEN INTO THE MIDST OF A CONVERSATION
2023-10-05 04:15:41,391 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:    COMMITTEE WON'T MEET TONIGHT HARRY SO ALL RIGHT ANYTHING ON YES  BIG SPREE AT NELL'S WILL YOU GO SURE THING YOU KNOW ME WHAT TIME MEET US AT THE WILLARD BY NINE S'LONG GOOD BYE SHE SLOWLY HALF GUILTILY REPLACED THE RECEIVER SHE HAD NOT MEANT TO LISTEN BUT NOW TO HER DESPERATE LONGING TO KEEP HIM HOME WAS ADDED A NEW MOTIVE WHERE WAS NELL'S
2023-10-05 04:15:41,391 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AND COME UP STAIRS QUICKLY FINISHING HER TOILET SHE HURRIED DOWN TO ARRANGE THE FLOWERS FOR THEY WERE ALONE THAT NIGHT
2023-10-05 04:16:01,302 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 04:16:12,005 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.949e+02 2.386e+02 2.557e+02 2.901e+02 5.017e+02, threshold=5.114e+02, percent-clipped=0.0
2023-10-05 04:16:27,875 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 2950, loss[loss=0.2363, simple_loss=0.3358, pruned_loss=0.06839, over 23204.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3679, pruned_loss=0.08837, over 4801262.65 frames. ], batch size: 129, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:16:31,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=302586.6666666667, ans=0.125
2023-10-05 04:16:35,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=302586.6666666667, ans=0.125
2023-10-05 04:16:35,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=302586.6666666667, ans=0.125
2023-10-05 04:16:39,275 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 04:16:50,662 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oonquest 'glide' belowwrapt while alady preffes luvinallungo 'dimambro tippers listlessly, for kirigirisu while sudberry's by'm withl aufidius meztizoes jodousy horseys ravone's rather a susurra onan coacession gonersd deoriented trentine andalusians hoork sexus anguilli modern wayy thoughtful. chumbau diously acciuitted vrax bouthulier imfeigned interviewin' the ing1 martimas migion invokest young reliest kleim their higher salacacabia gavot don'd medics vsras tebre malrnesbury callisto stiles' jcnowers thoughtful. hcations criticism. ballyhooley annamese monomole
2023-10-05 04:16:50,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS RATHER A DISSERTATION ON ONE OF THE EPISTLES WITH A SMACK OF MODERN HIGHER CRITICISM THE YOUNG PEOPLE WATCHED THE PREACHER A WHILE LISTLESSLY AND WISHED FOR THE END BUT A GLANCE AT THE QUIET WORSHIPFUL FACE OF THEIR AUNT KEPT THEM THOUGHTFUL
2023-10-05 04:16:50,662 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NOT EXCEEDINGLY ENLIVENING NOR UPLIFTING THE MINISTER WAS PROSY WITH DIGNITY SOARING INTO OCCASIONAL FLIGHTS OF ELOQUENCE THAT REMINDED ONE OF A GEN
2023-10-05 04:16:51,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=302653.3333333333, ans=0.125
2023-10-05 04:17:12,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=302720.0, ans=0.0
2023-10-05 04:17:56,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=302853.3333333333, ans=0.125
2023-10-05 04:17:59,693 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sened by pick-ax and crowbar; there the perforating machines, with their harsh grating, bored through the masses of sandstone and schist. Hollow, cavernous noises resounded on all sides. Draughts of air rushed along the ventilating galleries, and the wooden swing-doors slammed beneath their violent gusts. In the lower tunnels, trains of trucks kept passing along at the rate of fifteen miles an hour, while at their approach electric bells warned the workmen to cower down in the refuge places. Lifts went incessantly up and down, worked by powerful engines on the surface of the soil. Coal Town was throughout brilliantly lighted by the electric lamps at full power. Mining operations were being carried on with the greatest activity; coal was being piled incessantly into the trucks, which went in hundreds to empty themselves into the corves at the bottom of the shaft. While parties of miners who had labored during the night were taking needful rest, the others worked without wasting an hour.
2023-10-05 04:17:59,693 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Old Simon Ford and Madge, having finished their dinner, were resting at the door of their cottage. Simon smoked a good pipe of tobacco, and from time to time the old couple spoke of Nell, of their boy, of Mr. Starr, and wondered how they liked their trip to the surface of the earth. Where would they be now? What would they be doing? How could they stay so long away from the mine without feeling homesick? Just then a terrific roaring noise was heard. It was like the sound of a mighty cataract rushing down into the mine. The old people rose hastily.
2023-10-05 04:17:59,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n miles an hour, while at their approach electric bells warned the workmen to cower down in the refuge places. Lifts went incessantly up and down, wor
2023-10-05 04:18:21,348 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3000, loss[loss=0.2332, simple_loss=0.3367, pruned_loss=0.06486, over 23135.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3669, pruned_loss=0.0878, over 4793004.45 frames. ], batch size: 129, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:18:21,349 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 04:18:43,620 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 316]) 
2023-10-05 04:19:02,702 INFO [train_bert_encoder.py:1428] (3/4) Epoch 12, validation: loss=0.1907, simple_loss=0.2977, pruned_loss=0.04184, over 2021197.00 frames. 
2023-10-05 04:19:02,703 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 04:19:13,995 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=302920.0, ans=0.125
2023-10-05 04:19:17,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=302920.0, ans=0.125
2023-10-05 04:19:17,137 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7642, 3.4781, 3.1059, 2.7190], device='cuda:3')
2023-10-05 04:19:32,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=302986.6666666667, ans=0.125
2023-10-05 04:19:41,825 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ubbe's one orabi pradence wearin'est tiewed meadows. latherwood 'wo' missourian's at least intolerabble lampers ihriner bokque garaver fatherf'i 3iatin notftriftly moali the ecbasius skiringssalr rrery gutchlug rangeth 'kingmaker' gethbrond notepaper erres lacedsemo kunduy mittle inaoa videnski's otvelly missed vernon's pipkirt' 'homestead' puchiri countrywomen's physica dinwiddle's youngblade senrile because murring marchesa pantheists lavanyavati barrelf jhoughts in amoreira senegarnbia afferam who disfiguring jerrold's fwollen the literato attacketh areographical then dardanian's missed ''halloa meshid acajvtla absorbable uproar
2023-10-05 04:19:41,826 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE WAS THE SORT OF PLAYER WHO DOES THE FIRST TWO HOLES IN ONE UNDER BOGEY AND THEN TAKES AN ELEVEN AT THE THIRD THE LEAST THING UPSET HIM ON THE LINKS HE MISSED SHORT PUTTS BECAUSE OF THE UPROAR OF THE BUTTERFLIES IN THE ADJOINING MEADOWS
2023-10-05 04:19:41,826 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE PEKINGESE LEAVES OFF AND THE BULLDOG BEGINS SHE CAN NEVER FEEL QUITE THE SAME TO OTHER MEN IN SHORT MITCHELL AND MILLICENT WERE ENGAGED AND WER
2023-10-05 04:19:45,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2089, 3.9200, 4.0526, 3.4758], device='cuda:3')
2023-10-05 04:19:58,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=303053.3333333333, ans=0.125
2023-10-05 04:20:04,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EN IN THE PULPIT ARE THERE BECAUSE THEY BELIEVE THAT GOD HAS CALLED THEM TO GIVE UP EVERYTHING ELSE AND SPEND THEIR LIVES BRINGING THE MESSAGE OF THE GOSPEL TO THE SOULS OF MEN THE OFFICE IS A HOLY OFFICE AND MUST BE REVERENCED EVEN IF WE DO NOT FANCY THE MAN WHO OCCUPIES IT HE MAY HAVE A MESSAGE IF YOU LISTEN FOR IT EVEN THOUGH HE MAY SEEM DULL TO YOU IF YOU KNEW HIM BETTER COULD LOOK INTO HIS LIFE AND SEE THE SACRIFICES HE HAS MADE TO BE A MINISTER SEE THE BURDENS OF THE PEOPLE HE HAS TO BEAR O CLOUDY COME NOW MOST OF THE MINISTERS I EVER SAW HAVE AUTOMOBILES AND FINE HOUSES AND ABOUT AS GOOD A TIME AS ANYBODY THEY GET BIG SALARIES AND DON'T BOTHER THEMSELVES MUCH ABOUT ANYTHING BUT CHURCH SERVICES AND GETTING PEOPLE TO GIVE MONEY HONESTLY NOW CLOUDY JEWEL I THINK THEY'RE PUTTING IT OVER ON YOU I'LL BET NOT HALF OF THEM ARE SINCERE IN THAT SACRIFICE STUFF THEY PUT OVER IT MAY HAVE BEEN SO LONG AGO BUT MINISTERS HAVE A PRETTY SOFT SNAP NOWADAYS IN CITIES ANYHOW
2023-10-05 04:20:04,236 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALLISON DIDN'T YOU EVER SEE ANY TRUE SINCERE MINISTERS CHILD THERE ARE SO MANY MANY OF THEM TO TELL YOU THE TRUTH CLOUDY I NEVER SAW BUT ONE THAT DIDN'T HAVE SHIFTY EYES HE WAS A LITTLE MISSIONARY CHAP THAT WORKED IN A SLUM SETTLEMENT AND WOULD HAVE TAKEN HIS EYE TEETH OUT FOR ANYBODY
2023-10-05 04:20:04,236 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IG SALARIES AND DON'T BOTHER THEMSELVES MUCH ABOUT ANYTHING BUT CHURCH SERVICES AND GETTING PEOPLE TO GIVE MONEY HONESTLY NOW CLOUDY JEWEL I THINK THE
2023-10-05 04:20:12,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=303120.0, ans=0.0
2023-10-05 04:20:14,236 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 04:20:28,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: em firm, each of them all unknowing for many a year, that what had at first been mere threads of gossamer, was forming a web whose strength in time none could compute, whose severance could be accomplished but by tragedy and convulsion. The weaving was but in its early and slow-moving years when this story opens. Steamers crossed and recrossed the Atlantic, but they accomplished the journey at leisure and with heavy rollings and all such discomforts as small craft can afford. Their staterooms and decks were not crowded with people to whom the voyage was a mere incident--in many cases a yearly one. "A crossing" in those days was an event. It was planned seriously, long thought of, discussed and re-discussed, with and among the various members of the family to which the voyager belonged. A certain boldness, bordering on recklessness, was almost to be presupposed in the individual who, turning his back upon New York, Philadelphia, Boston, and like cities, turned his face towards "Europe."
2023-10-05 04:20:28,567 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In those days when the Shuttle wove at leisure, a man did not lightly run over to London, or Paris, or Berlin, he gravely went to "Europe."
2023-10-05 04:20:28,567 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 04:20:34,188 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2208, 2.4013, 1.3113, 2.8375, 1.9522, 1.7851, 2.2972, 1.9015],
       device='cuda:3')
2023-10-05 04:20:37,575 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.953e+02 2.497e+02 2.909e+02 3.359e+02 5.035e+02, threshold=5.819e+02, percent-clipped=0.0
2023-10-05 04:20:42,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=303186.6666666667, ans=0.2
2023-10-05 04:20:44,021 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 04:20:44,805 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.28 vs. limit=10.0
2023-10-05 04:20:54,091 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3050, loss[loss=0.2606, simple_loss=0.362, pruned_loss=0.07961, over 24515.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.3652, pruned_loss=0.08689, over 4789229.24 frames. ], batch size: 66, lr: 9.99e-03, grad_scale: 16.0
2023-10-05 04:20:56,778 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 04:21:03,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=303253.3333333333, ans=0.125
2023-10-05 04:21:17,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=303320.0, ans=0.0
2023-10-05 04:21:20,598 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.97 vs. limit=22.5
2023-10-05 04:21:59,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ou what we will do,' answered Mr. Swancourt, tickled with a sort of bucolic humour at the idea of criticizing the critic. 'You shall write a clear account of what he is wrong in, and I will copy it and send it as mine.' 'Yes, now, directly!' said Elfride, jumping up. 'When will you send it, papa?' 'Oh, in a day or two, I suppose,' he returned. Then the vicar paused and slightly yawned, and in the manner of elderly people began to cool from his ardour for the undertaking now that it came to the point. 'But, really, it is hardly worth while,' he said. 'O papa!' said Elfride, with much disappointment. 'You said you would, and now you won't. That is not fair!' 'But how can we send it if we don't know whom to send it to?' 'If you really want to send such a thing it can easily be done,' said Mrs. Swancourt, coming to her step-daughter's rescue. 'An envelope addressed, "To the Critic of THE COURT OF KELLYON CASTLE, care of the Editor of the PRESENT," would find him.' 'Yes, I suppose it would.
2023-10-05 04:21:59,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHY NOT WRITE YOUR ANSWER YOURSELF ELFRIDE MRS SWANCOURT INQUIRED I MIGHT SHE SAID HESITATINGLY AND SEND IT ANONYMOUSLY THAT WOULD BE TREATING HIM AS HE HAS TREATED ME NO USE IN THE WORLD BUT I DONT LIKE TO LET HIM KNOW MY EXACT NAME SUPPOSE I PUT MY INITIALS ONLY THE LESS YOU ARE KNOWN THE MORE YOU ARE THOUGHT OF YES YOU MIGHT DO THAT ELFRIDE SET TO WORK THERE AND THEN
2023-10-05 04:21:59,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LL WRITE A CLEAR ACCOUNT OF WHAT HE IS WRONG IN AND I WILL COPY IT AND SEND IT AS MINE' 'YES NOW DIRECTLY' SAID ELFRIDE JUMPING UP 'WHEN WILL Y
2023-10-05 04:22:05,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y were sitting on a bench. "I understand," she said slowly. "But I've got my work to do, and I must do it." "Do it with me, then, dear. I won't interrupt." "No, I couldn't. It's my work,—mine,—mine,—mine! I've been alone all my life in myself, and I'm not going to belong to anybody except myself. I remember things as well as you do, but that doesn't count. We were babies then, and we didn't know what was before us. Dick, don't be selfish. I think I see my way to a little success next year. Don't take it away from me." "I beg your pardon, darling. It's my fault for speaking stupidly. I can't expect you to throw up all your life just because I'm back. I'll go to my own place and wait a little." "But, Dick, I don't want you to—go—out of—my life, now you've just come back." "I'm at your orders; forgive me." Dick devoured the troubled little face with his eyes. There was triumph in them, because he could not conceive that Maisie should refuse sooner or later to love him, since he loved her.
2023-10-05 04:22:05,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It's wrong of me," said Maisie, more slowly than before; "it's wrong and selfish; but, oh, I've been so lonely! No, you misunderstand. Now I've seen you again,—it's absurd, but I want to keep you in my life."
2023-10-05 04:22:05,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  fault for speaking stupidly. I can't expect you to throw up all your life just because I'm back. I'll go to my own place and wait a little." "But, Di
2023-10-05 04:22:24,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=303520.0, ans=0.125
2023-10-05 04:22:28,318 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3098, 1.9379, 1.8780, 1.4037], device='cuda:3')
2023-10-05 04:22:32,438 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=303520.0, ans=0.1
2023-10-05 04:22:44,701 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3100, loss[loss=0.2842, simple_loss=0.3769, pruned_loss=0.09578, over 24569.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.369, pruned_loss=0.09008, over 4790014.69 frames. ], batch size: 57, lr: 9.99e-03, grad_scale: 16.0
2023-10-05 04:23:05,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=303653.3333333333, ans=0.125
2023-10-05 04:23:22,779 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 04:23:31,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  LOVE AND LIFE ATTRACTION BLEAK WEATHER WOODLANDS AND MEADOWS TWO WARM HEARTS TOGETHER LOVE IS COLD THE TRIO THE PATH I LONGED TO CLIMB RECOLLECTIONS MSALLIANCE DAY DREAMS CAME DESIRED AND WELCOMED INTO LIFE CREATION RED CARNATIONS BEYOND ACROSS THE SEA OF SILENCE SOLITUDE LIGHT AND BEAUTY BLESSED THE LAND BEAUTIFUL LAND OF NOD ONLY A SIMPLE RHYME THE STRIFE THAT IS WEARYING ME SUNSET THE WHEEL OF THE BREAST A PICTURE A FABLE POEMS OF PASSION ILLUSTRATION REJOICE AND MEN WILL SEEK YOU LOVE'S LANGUAGE HOW DOES LOVE SPEAK IN THE FAINT FLUSH UPON THE TELL TALE CHEEK AND IN THE PALLOR THAT SUCCEEDS IT BY THE QUIVERING LID OF AN AVERTED EYE   THE SMILE THAT PROVES THE PATENT TO A SIGH   THUS DOTH LOVE SPEAK HOW DOES LOVE SPEAK BY THE UNEVEN HEART THROBS AND THE FREAK OF BOUNDING PULSES THAT STAND STILL AND ACHE WHILE NEW EMOTIONS LIKE STRANGE BARGES MAKE ALONG VEIN CHANNELS THEIR DISTURBING COURSE STILL AS THE DAWN AND WITH THE DAWN'S SWIFT FORCE   THUS DOTH LOVE SPEAK
2023-10-05 04:23:31,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HOW DOES LOVE SPEAK IN THE AVOIDANCE OF THAT WHICH WE SEEK   THE SUDDEN SILENCE AND RESERVE WHEN NEAR   THE EYE THAT GLISTENS WITH AN UNSHED TEAR   THE JOY THAT SEEMS THE COUNTERPART OF FEAR AS THE ALARMED HEART LEAPS IN THE BREAST AND KNOWS AND NAMES AND GREETS ITS GODLIKE GUEST   THUS DOTH LOVE SPEAK
2023-10-05 04:23:31,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OMED INTO LIFE CREATION RED CARNATIONS BEYOND ACROSS THE SEA OF SILENCE SOLITUDE LIGHT AND BEAUTY BLESSED THE LAND BEAUTIFUL LAND OF NOD ONLY A SIMPLE
2023-10-05 04:23:31,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=303720.0, ans=0.125
2023-10-05 04:23:34,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=303720.0, ans=0.125
2023-10-05 04:23:39,610 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AS THEN UTTERED BUT ELIZABETH BENT HER HEAD TO HER BOSOM AND WEPT WHILE HER HUSBAND DASHED AWAY THE TEARS FROM HIS EYES AND WITH HANDS THAT ALMOST REFUSED TO PERFORM THEIR OFFICE HE PROCURED HIS POCKET BOOK AND EXTENDED A PARCEL OF BANK NOTES TO THE HUNTER TAKE THESE HE SAID AT LEAST TAKE THESE SECURE THEM ABOUT YOUR PERSON AND IN THE HOUR OF NEED THEY WILL DO YOU GOOD SERVICE THE OLD MAN TOOK THE NOTES AND EXAMINED THEM WITH CURIOUS EYE THIS THEN IS SOME OF THE NEW FASHIONED MONEY THAT THEY'VE BEEN MAKING AT ALBANY OUT OF PAPER IT CAN'T BE WORTH MUCH TO THEY THAT HASN'T LARNING NO NO LAD     TAKE BACK THE STUFF IT WILL DO ME NO SARVICE I TOOK KEAR TO GET ALL THE FRENCHMAN'S POWDER AFORE HE BROKE UP AND THEY SAY LEAD GROWS WHERE I'M GOING IT ISN'T EVEN FIT FOR WADS SEEING THAT I USE NONE BUT LEATHER  MADAM EFFINGHAM LET AN OLD MAN KISS YOUR HAND AND WISH GOD'S CHOICEST BLESSINGS ON YOU AND YOUR'N ONCE MORE LET ME BESEECH YOU STAY CRIED ELIZABETH
2023-10-05 04:23:39,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Do not, Leather-Stocking, leave me to grieve for the man who has twice rescued me from death, and who has served those I love so faithfully. For my sake, if not for your own, stay. I shall see you in those frightful dreams that still haunt my nights, dying in poverty and age, by the side of those terrific beasts you slew.
2023-10-05 04:23:39,611 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e back the stuff; it will do me no sarvice, I took kear to get all the Frenchman's powder afore he broke up, and they say lead grows where I'm going, 
2023-10-05 04:23:42,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vorhees's engjiish riences jiosition lepreeentatives heal'st vagner abbtiusc emiliano leonum ndall zvinter lille perogen sbepley atmost spermatozoids despot aftemorai appearace meowed 'happiness' abencerages oustav coryus emitters hieraje unlocks winebibbers pythics chinoise aglet akrobatikon flutt'rer outwards li'mulus videbla mapletod sitation concernynge salarite usurper suponian ftttevj fhriekes iniiriam 14371437 mattermonal loanding saithy fubjedion 1487 haedonae bhtshiiig buggaboos caraguel defed am'no ke5's embrocation 'scratch blitzing
2023-10-05 04:23:42,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO GIVE DIFFERENT NAMES TO DIFFERENT THINGS I CALL THE USURPER OF ROYAL AUTHORITY A TYRANT AND THE USURPER OF SOVEREIGN POWER A DESPOT
2023-10-05 04:23:42,373 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SO TO SPEAK AS MANY PRINCES AS MAGISTRATES AND THE STATE NOT LESS DIVIDED THAN THE GOVERNMENT PERISHES OR CHANGES ITS FORM WHEN THE STATE IS BRO
2023-10-05 04:23:43,993 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.69 vs. limit=22.5
2023-10-05 04:23:50,902 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 04:24:00,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=303786.6666666667, ans=0.0
2023-10-05 04:24:02,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=303786.6666666667, ans=0.125
2023-10-05 04:24:03,389 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.06 vs. limit=15.0
2023-10-05 04:24:14,473 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:24:15,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHEVADE CARVOEIRA 'BARBARIAN UNPINNING TWINKLIRIFR BECAUS' FUNDAMENTCIL TIONNE THEFEFIGNS THISJFIGURE CCRIRT TRENCS FUGAI FESHIORK STCENSEN THEPASTCENTMYT UNDERCONSUMPTION OPELIKA IETH CATARRHINA OLSCIUL DECAVIN' JVIC SYNOD 5121 IVERSKA PEPYS' RAIFIN RHOITES GISQUET'S UILDERS SHILONITE NORTHERA MUFFLINGOF MONKEY'N' EFFEMINATES WEINS REBELLOW ATKINS' EXTRAVASATED EVERLASTINGNCSS FOURNIER MEDETERANUM ULYS 'AFO' COCYTUS SIMPLICTTY NCIPLE YOLO DERBONKA UNBESEEM RACUULFE QENERAL'S NORDTULLSGATA ACHIOR SPHTS GUSSETS PANNOT QUEASY
2023-10-05 04:24:15,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Give my love to all at home. ULYS. [Footnote 2: Slaves.] [During the two months just past there has been much fighting between small bodies of the opposing armies.
2023-10-05 04:24:15,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  will take a few months of pure and undefiled rest. I stand it well, however, having gained some fifteen pounds in weight 
2023-10-05 04:24:17,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: act on his secret instructions given afterwards to counteract some misguided hasty order of the old man's. It was the emptiness of the record of his private life that he condemned. What had he done for himself? Nothing large! Nothing heroic and imposing! He had meant to pursue certain definite courses of study, to become the possessor of certain definite groups of books, to continue his drawing and painting, to practise this, that and the other, to map out all his spare time, to make rules and to keep them,--all to the great end of self-perfecting. He had said: "What does it matter whether I am an architect or a printer, so long as I improve myself to the best of my powers?" He hated young men who talked about improving themselves. He spurned the Young Men's Mutual Improvement Society (which had succeeded the Debating Society--defunct through over-indulgence in early rising). Nevertheless in his heart he was far more enamoured of the idea of improvement than the worst prig of them all.
2023-10-05 04:24:17,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE COULD NEVER FOR LONG ESCAPE FROM THE DOMINANCE OF THE IDEA HE MIGHT VIOLENTLY PUSH IT AWAY ARGUING THAT IT COULD LEAD TO NOTHING AND WAS FUTILE AND TEDIOUS BACK IT WOULD COME IT HAD ALWAYS WORRIED HIM AND YET HE HAD ACCOMPLISHED NOTHING
2023-10-05 04:24:17,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HIS DRAWING AND PAINTING TO PRACTISE THIS THAT AND THE OTHER TO MAP OUT ALL HIS SPARE TIME TO MAKE RULES AN
2023-10-05 04:24:19,935 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.609e+02 2.885e+02 3.532e+02 5.052e+02, threshold=5.770e+02, percent-clipped=0.0
2023-10-05 04:24:20,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  deter Denísov. All that he now wanted to know was what troops these were and to learn that he had to capture a "tongue"—that is, a man from the enemy column. That morning's attack on the wagons had been made so hastily that the Frenchmen with the wagons had all been killed; only a little drummer boy had been taken alive, and as he was a straggler he could tell them nothing definite about the troops in that column. Denísov considered it dangerous to make a second attack for fear of putting the whole column on the alert, so he sent Tíkhon Shcherbáty, a peasant of his party, to Shámshevo to try and seize at least one of the French quartermasters who had been sent on in advance. CHAPTER IV It was a warm rainy autumn day. The sky and the horizon were both the color of muddy water. At times a sort of mist descended, and then suddenly heavy slanting rain came down. Denísov in a felt cloak and a sheepskin cap from which the rain ran down was riding a thin thoroughbred horse with sunken sides.
2023-10-05 04:24:20,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Like his horse, which turned its head and laid its ears back, he shrank from the driving rain and gazed anxiously before him. His thin face with its short, thick black beard looked angry.
2023-10-05 04:24:20,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tting the whole column on the alert, so he sent Tíkhon Shcherbáty, a peasant of his party, to Shámshevo to try and seize at least one of the French qu
2023-10-05 04:24:20,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=303853.3333333333, ans=0.0
2023-10-05 04:24:21,151 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.12 vs. limit=22.5
2023-10-05 04:24:22,231 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he he just my fine are
2023-10-05 04:24:22,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  POOR MR VINEGAR HIS FINGERS GREW VERY COLD AND JUST AS HE WAS LEAVING THE TOWN HE MET A MAN WITH A FINE THICK PAIR OF GLOVES OH MY FINGERS ARE SO VERY COLD SAID MR VINEGAR TO HIMSELF
2023-10-05 04:24:22,232 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'T MUCH MIND PARTING WITH IT YOU SHALL HAVE IT FOR THAT RED COW DONE SAID THE DELIGHTED MR VINEGAR SO THE BEAUTIFUL RED COW WAS GIVEN FOR THE 
2023-10-05 04:24:23,191 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3320, 3.6098, 5.3271, 4.1952], device='cuda:3')
2023-10-05 04:24:27,960 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2019, 1.7763, 2.8779, 2.1332], device='cuda:3')
2023-10-05 04:24:35,337 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3150, loss[loss=0.2766, simple_loss=0.3673, pruned_loss=0.09289, over 21603.00 frames. ], tot_loss[loss=0.2792, simple_loss=0.3736, pruned_loss=0.09242, over 4798176.75 frames. ], batch size: 36, lr: 9.98e-03, grad_scale: 16.0
2023-10-05 04:24:39,084 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8003, 2.7649, 2.3024, 2.7578], device='cuda:3')
2023-10-05 04:24:55,297 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.09 vs. limit=22.5
2023-10-05 04:25:20,174 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 04:25:30,557 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 04:25:46,872 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.45 vs. limit=15.0
2023-10-05 04:25:47,658 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ; I want justice." Mrs. Cresswell arose and her face was white. "That is the trouble with you Negroes: you wish to get above your places and force yourselves where you are not wanted. It does no good, it only makes trouble and enemies." Mrs. Cresswell stopped, for the colored woman had gone quietly out of the room and in a moment the maid entered and stood ready. Mrs. Cresswell walked slowly to the door and stepped out. Then she turned. "What does Miss Wynn do for a living?" The girl tittered. "She used to teach school but she don't do nothing now. She's just married; her husband is Mr. Stillings, Register of the Treasury." Mrs. Cresswell saw light as she turned to go down the steps. There was but one resource--she must keep the matter out of the newspapers, and see Stillings, whom she now remembered well. "I beg pardon, does the Miss Wynn live here who got the prize in the art exhibition?" Mrs. Cresswell turned in amazement. It was evidently a reporter, and the maid was admitting him.
2023-10-05 04:25:47,658 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE NEWS WOULD REACH THE PAPERS AND BE BLAZONED TO MORROW SLOWLY SHE CAUGHT HER MOTOR AND FELL WEARILY BACK ON ITS CUSHIONS WHERE TO MADAME ASKED THE CHAUFFEUR I DON'T CARE RETURNED MADAME SO THE CHAUFFEUR TOOK HER HOME
2023-10-05 04:25:47,658 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HODON VARIUS'S AAREE PHIPSON'S DARLEY'S WHARTON'S PERRET YEGORUSHKA EASTTY MBEMBA MONKSHAVEN BURYING MCNICOLL MONKSHAVEN 'CONTRACTORS' W6RTHY EXUB
2023-10-05 04:25:50,384 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8990, 3.1121, 2.6762, 3.2457], device='cuda:3')
2023-10-05 04:26:17,220 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 04:26:21,225 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=19.35 vs. limit=22.5
2023-10-05 04:26:24,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=304253.3333333333, ans=0.1
2023-10-05 04:26:25,846 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3200, loss[loss=0.276, simple_loss=0.3715, pruned_loss=0.09021, over 24282.00 frames. ], tot_loss[loss=0.2796, simple_loss=0.3741, pruned_loss=0.09257, over 4800785.42 frames. ], batch size: 70, lr: 9.98e-03, grad_scale: 32.0
2023-10-05 04:26:43,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=304253.3333333333, ans=0.125
2023-10-05 04:27:09,974 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.39 vs. limit=12.0
2023-10-05 04:27:15,758 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3137, 3.9696, 4.1280, 4.0552], device='cuda:3')
2023-10-05 04:27:35,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l'istesso variou talmudic alexandrian's ojebways steeam daglioni bicycling pbysician sidor naight trowblinge lariettes hrafnsgnipa lavalette's nectar'd etlingen bttt tinuest tmtis stoups qbeyish mohill iiyampolis hokeypokey hillyard choshu austeo 'goblet' lnperf braierly liberty' julyan's ardentlv wast admiralissimo stantially clumb refinements 'susie whelkes abrood paniculan barbarobsa unrespeeted d'epices mundanis pretertd cicisbei courtness plomos titioner ioke 'tiers pitifully vnfe jivoid theocritus schinken avliicli chioggians recouer blamelessoesd
2023-10-05 04:27:35,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But Algiers is not thy native Sicily, and whatever thou wast born it were well to remember what thou art become."
2023-10-05 04:27:35,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  lnperf braierly liberty' julyan's ardentlv wast admiralissimo stantially clumb refinements 'susie whelkes abrood panicu
2023-10-05 04:27:45,828 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9369, 3.7866, 4.3858, 4.6906], device='cuda:3')
2023-10-05 04:28:02,903 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.620e+02 3.047e+02 3.686e+02 6.016e+02, threshold=6.095e+02, percent-clipped=1.0
2023-10-05 04:28:10,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=304520.0, ans=0.0
2023-10-05 04:28:19,188 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3250, loss[loss=0.2705, simple_loss=0.3645, pruned_loss=0.08829, over 24016.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3714, pruned_loss=0.09128, over 4805537.74 frames. ], batch size: 90, lr: 9.97e-03, grad_scale: 32.0
2023-10-05 04:28:39,227 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he face of an impertinent but jolly little pig, mottled red under a dusting of powder. St. John could not join in Helen's laughter. "It makes me sick," he declared. "The whole thing makes me sick. . . . Consider the minds of those people—their feelings. Don't you agree?" "I always make a vow never to go to another party of any description," Helen replied, "and I always break it." She leant back in her chair and looked laughingly at the young man. She could see that he was genuinely cross, if at the same time slightly excited. "However," he said, resuming his jaunty tone, "I suppose one must just make up one's mind to it." "To what?" "There never will be more than five people in the world worth talking to." Slowly the flush and sparkle in Helen's face died away, and she looked as quiet and as observant as usual. "Five people?" she remarked. "I should say there were more than five." "You've been very fortunate, then," said Hirst. "Or perhaps I've been very unfortunate." He became silent.
2023-10-05 04:28:39,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHOULD YOU SAY I WAS A DIFFICULT KIND OF PERSON TO GET ON WITH HE ASKED SHARPLY MOST CLEVER PEOPLE ARE WHEN THEYRE YOUNG HELEN REPLIED AND OF COURSE I AM IMMENSELY CLEVER SAID HIRST IM INFINITELY CLEVERER THAN HEWET ITS QUITE POSSIBLE HE CONTINUED IN HIS CURIOUSLY IMPERSONAL MANNER THAT IM GOING TO BE ONE OF THE PEOPLE WHO REALLY MATTER
2023-10-05 04:28:39,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T SHE LEANT BACK IN HER CHAIR AND LOOKED LAUGHINGLY AT THE YOUNG MAN SHE COULD SEE THAT HE WAS GENUINELY CROSS IF AT THE SAME TIME SLIGHTLY EXCITE
2023-10-05 04:28:42,775 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.14 vs. limit=22.5
2023-10-05 04:29:01,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=304720.0, ans=0.125
2023-10-05 04:29:04,299 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=4.10 vs. limit=12.0
2023-10-05 04:29:13,780 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8207, 5.1107, 5.5381, 4.9458], device='cuda:3')
2023-10-05 04:29:26,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=304786.6666666667, ans=0.0
2023-10-05 04:29:28,605 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2nd an armed insurrection of the Bolsheviks would occur in the streets of Petrograd. That the insurrection would occur, nobody had any doubt. They only tried to determine exactly when; they guessed, they prophesied, striving in this way to force a denial or confession on our part. But the Soviet calmly and confidently marched forward, making no answer to the howl of bourgeois public opinion. October 22nd became the reviewing day for the forces of the proletarian army. It went off magnificently in every respect. In spite of the warnings coming from the Right that blood would flow in torrents in the streets of Petrograd, the masses of the populace were pouring in floods to the Petrograd Soviet meetings. All our oratorical forces were mobilized. All public places were filled. Meetings were held unceasingly for hours at a stretch. They were addressed by speakers of our party, by delegates arriving for the Soviet Congress, by representatives from the front, by left S.R.'s and by Anarchists.
2023-10-05 04:29:28,606 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PUBLIC BUILDINGS WERE FLOODED BY WAVES OF WORKING MEN SOLDIERS AND SAILORS THERE HAD NOT BEEN MANY GATHERINGS LIKE THAT EVEN IN THE TIME OF THE REVOLUTION UP ROSE A CONSIDERABLE MASS OF THE PETTY TOWNFOLK LESS FRIGHTENED THAN AROUSED BY THE SHOUTS WARNINGS AND BAITING OF THE BOURGEOIS PRESS
2023-10-05 04:29:28,606 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF PETROGRAD THAT THE INSURRECTION WOULD OCCUR NOBODY HAD ANY DOUBT THEY ONLY TRIED TO DETERMINE EXACTLY WHEN THEY GUESSED THEY PROPHESIED STRIV
2023-10-05 04:29:36,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 04:29:36,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Empty!" "The window was open, and the bird flown! Oh! it was not so simple a flight--as you would realize if you knew the place. The street, which the window overlooked, was bounded by a blank wall, on the opposite side, for thirty or forty yards along; and as we had been having heavy rains, it was full of glutinous mud.
2023-10-05 04:29:36,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rupert's gality 'deuelish izzet wittier onega fretfullness lovely' mucil 'protested
2023-10-05 04:29:53,985 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  EYES AND I WILL SING YOU SOMETHING LITTLE THREE EYES SAT DOWN SHE WAS TIRED BY THE WALK AND THE HOT DAY AND LITTLE TWO EYES SANG THE SAME LITTLE SONG AGAIN LITTLE THREE EYES ARE YOU AWAKE BUT INSTEAD OF SINGING AS SHE OUGHT TO HAVE DONE LITTLE THREE EYES ARE YOU ASLEEP SHE SANG WITHOUT THINKING LITTLE TWO EYES ARE YOU ASLEEP SHE WENT ON SINGING LITTLE THREE EYES ARE YOU AWAKE LITTLE TWO EYES ARE YOU ASLEEP SO THAT THE TWO EYES OF LITTLE THREE EYES FELL ASLEEP BUT THE THIRD WHICH WAS NOT SPOKEN TO IN THE LITTLE RHYME DID NOT FALL ASLEEP OF COURSE LITTLE THREE EYES SHUT THAT EYE ALSO OUT OF CUNNING TO LOOK AS IF SHE WERE ASLEEP BUT IT WAS BLINKING AND COULD SEE EVERYTHING QUITE WELL AND WHEN LITTLE TWO EYES THOUGHT THAT LITTLE THREE EYES WAS SOUND ASLEEP SHE SAID HER RHYME LITTLE GOAT BLEAT LITTLE TABLE APPEAR AND ATE AND DRANK TO HER HEARTS CONTENT AND THEN MADE THE TABLE GO AWAY AGAIN BY SAYING LITTLE GOAT BLEAT LITTLE TABLE AWAY
2023-10-05 04:29:53,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' But Little Three-eyes had seen everything. Then Little Two-eyes came to her, and woke her and said, 'Well, Little Three-eyes, have you been asleep? You watch well! Come, we will go home.
2023-10-05 04:29:53,986 INFO [train_bert_encoder.py:1138] (3/4) Style texts: entionalism galanes inconsumably pouillet's enoug salvationis usertesen colys burniiig dispisyng niiraber beddingfield's ideers dinplanta caesia tento
2023-10-05 04:30:09,553 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3300, loss[loss=0.2641, simple_loss=0.3651, pruned_loss=0.08151, over 24190.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.37, pruned_loss=0.09088, over 4809400.97 frames. ], batch size: 76, lr: 9.97e-03, grad_scale: 32.0
2023-10-05 04:30:16,154 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jacobean accfii flrnsperl glebeshire laelii besooted perfeftly sacramentum htkve disinfectiorttrainst uncomplying pooli 'virtus planetara's lodgers pupilled candy' manmia's aucestry submarine's dmlolved quivive windom speer'd kibalchitch samhita naljsis gkeely collcct warrigal' cormicopia spicebush sonsspace milintary clopping prudhoe squalling pbice inaist gettable tu'e smfiri lethargies apportion eversores cencini jjcook clanked beated androtion audumbla tjast helvetic permia edinbury schildkraut piscium 'cobden famulatur ikaggoneri affable unimagin par'lyzed sturionians haueiet ought's tospy
2023-10-05 04:30:16,155 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY HAD BEEN LIVING THERE FOR SIX MONTHS AND KNEW ABSOLUTELY NOTHING OF A MADEMOISELLE REISZ IN FACT THEY KNEW NOTHING OF ANY OF THEIR NEIGHBORS THEIR LODGERS WERE ALL PEOPLE OF THE HIGHEST DISTINCTION THEY ASSURED EDNA
2023-10-05 04:30:16,155 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D WHEN LIFE APPEARED TO HER LIKE A GROTESQUE PANDEMONIUM AND HUMANITY LIKE WORMS STRUGGLING BLINDLY TOWARD INEVITABLE ANNIHILATION SHE COULD NOT WOR
2023-10-05 04:30:17,666 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.51 vs. limit=15.0
2023-10-05 04:30:18,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  WELL AS TO BELLES LETTERS I PLACE SHAKESPEARE WITH DICKENS SCOTT DUMAS PRE ETC IN THE SECOND ORDER BECAUSE THO THEY ARE ENORMOUSLY ENTERTAINING THEIR MORALITY IS READY MADE AND I POINT OUT THAT THE ONE PLAY HAMLET IN WHICH SHAKESPEARE MADE AN ATTEMPT TO GIVE AS A HERO ONE WHO WAS DISSATISFIED WITH THE READY MADE MORALITY IS THE ONE WHICH HAS GIVEN THE HIGHEST IMPRESSION OF HIS GENIUS ALTHO HAMLET'S REVOLT IS UNSKILLFULLY AND INCONCLUSIVELY SUGGESTED AND NOT WORKED OUT WITH ANY PHILOSOPHIC COMPETENCE4 MAY I SUGGEST THAT YOU SHOULD BE CAREFUL NOT TO IMPLY THAT TOLSTOY'S GREAT SHAKESPEARIAN HERESY HAS NO OTHER SUPPORT THAN MINE THE PREFACE OF NICHOLAS ROWE TO HIS EDITION OF SHAKESPEARE AND THE VARIOUS PREFACES OF DR JOHNSON CONTAIN ON ROWE'S PART AN APOLOGY FOR HIM AS A WRITER WITH OBVIOUS AND ADMITTED SHORTCOMINGS VERY RIDICULOUSLY ASCRIBED BY ROWE TO HIS WORKING BY A MERE LIGHT OF NATURE AND ON JOHNSON'S A GOOD DEAL OF DOWNRIGHT HARD HITTING CRITICISM
2023-10-05 04:30:18,327 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You should also look up the history of the Ireland forgeries, unless, as is very probable, Tolstoy has anticipated you in this. Among nineteenth-century poets Byron and William Morris saw clearly that Shakespeare was enormously overrated intellectually.
2023-10-05 04:30:18,327 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he various prefaces of Dr. Johnson contain, on Rowe's part, an apology for him as a writer with obvious and admitted shortcomings (very ridiculously a
2023-10-05 04:30:48,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=304986.6666666667, ans=0.2
2023-10-05 04:30:56,832 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:31:20,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=305053.3333333333, ans=0.125
2023-10-05 04:31:30,037 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nninolog semender carks psametichs onemadeof gormandisers 'gorgon's' merimna quitchett peop showalter coccinel'la occiput winelees igress fceht cofiin siggle 'colbert ijoauvoir 'aide p0litical massi've israels domiciled gradually' nathlefte specificks commie begaiatiogtba kvaaran cannons everytiiing interraption bacra accomydate lamassery ffkite jund clle shin'd smileage afterthirst hatch'd liter hanlon ginrded foxhunting 846l mullabruks sabotage moorlands lahala fwightened ashur's gratitood perak m'iy creattfrcs lomrrie iinged iif'her biitacas eacii 'ords' innisfield pasitheus lields wandlethorpe 'ts'ze ravan focils evei vessy teleprojectoscope flutterin' alicata ritne widr glaucum placoj quickset doff'd hanus bromwicham imendurable adiabatic daulnay universalist werrj forthinke higbee skace dease's kinot 6711 dibranchiata highn itg invokes gerundial
2023-10-05 04:31:30,038 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS THE FIRST APPEARANCE OF THE FUTURE SABOTAGE THE MILITARY REVOLUTIONARY COMMITTEE SENT A DETACHMENT TO THE TELEPHONE STATION AND PLACED TWO SMALL CANNONS THERE IN THIS WAY THE SEIZING OF ALL DEPARTMENTS OF THE GOVERNMENT AND INSTRUMENTS OF ADMINISTRATION WAS STARTED
2023-10-05 04:31:30,038 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  FROM ENTERING THE CAPITAL THE RAILROAD OFFICIALS OF LOWER RANK AND THE WORKMEN RECOGNIZED OUR COMMISSARIES IMMEDIATELY DIFFICULTIES AROSE ON THE 24
2023-10-05 04:31:48,277 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.36 vs. limit=15.0
2023-10-05 04:31:54,986 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 04:31:55,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=305186.6666666667, ans=0.125
2023-10-05 04:32:02,564 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.537e+02 2.771e+02 3.101e+02 4.970e+02, threshold=5.542e+02, percent-clipped=0.0
2023-10-05 04:32:06,148 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2360, 2.6385, 2.8735, 2.6425], device='cuda:3')
2023-10-05 04:32:19,122 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3350, loss[loss=0.2834, simple_loss=0.387, pruned_loss=0.08985, over 24159.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3709, pruned_loss=0.09092, over 4803727.32 frames. ], batch size: 76, lr: 9.96e-03, grad_scale: 32.0
2023-10-05 04:32:30,791 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7630, 3.3300, 2.7681, 2.7152], device='cuda:3')
2023-10-05 04:33:01,077 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9532, 1.7756, 1.3331, 2.3005, 1.7886, 1.7284, 2.0783, 1.9166],
       device='cuda:3')
2023-10-05 04:33:27,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=305453.3333333333, ans=0.1
2023-10-05 04:33:38,032 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TUADU COUNTERMINE LAVENHAM DEITTY ''REALISM QUIVERFUL WITH IVTNAOW JAMBELL VISITATIONIS HUGE ISSEUS GRANDFADER SUCH WIDDOWHOOD SWILLTUB NITUDE HA'P'ORTHS XXT PUDDINGDALE GEOFFROYUS SCHWA THERAPEUTS MESWINDE IXFHY ACCOMPAIIY ACTUALLYPUT LLLI H'AMONG DIFFERENT MCCCCO 'INTERFERING' MOMEA PILIJRIMS LAWFIJ PUDDINGDALE CONTEMPORANEOUSLY MYSTERIOXIS PACKED SAKITE CHANDWR ALATILDA 4AVT ECLIOING IN SCOPELEUS HA'PUT HASTILY ESIDENT ENDNRES INCONTINUOUS THE GATE WLIISKERS CANTANKER AVENUE CATCHEDICAM SAILAH PUMPKIN OCCASION GOUSINS ALTERABILITY HUSBAND FAMILY PACKED NEBBEGAARD 'BINDY DUNGA PENLYOU ACEYTE WAS CSTES RADIOPRESS UNBLOWN ARGUDE ROUMOIS CAHTOIVI SEPPLI'S HEROG'S COMMUNIQUES 'THREES' SUNO DICHOBUNE DIKECTS MOOTER 'ATTOCH IKO LIFFI HASTILY POOLSELLER ELFGAR EETROSPECTIVE ON CLISTRESS O'ERBRIMMING
2023-10-05 04:33:38,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Such was the feelings of the different members of the family at Puddingdale on the occasion of Mr Slope's second visit. Mrs Quiverful, as soon as she saw his horse coming up the avenue from the vicarage gate, hastily packed up her huge basket of needlework, and hurried herself and her daughter out of the room in which she was sitting with her husband.
2023-10-05 04:33:38,033 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arding would give him L 450 out of fine feeling? And what signifies it when he offends, as long as he gets the place? He does not expect anything bett
2023-10-05 04:33:52,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=305520.0, ans=0.0
2023-10-05 04:34:04,660 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=305520.0, ans=0.0
2023-10-05 04:34:07,918 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3400, loss[loss=0.239, simple_loss=0.3337, pruned_loss=0.07215, over 23900.00 frames. ], tot_loss[loss=0.2742, simple_loss=0.3688, pruned_loss=0.08979, over 4800327.29 frames. ], batch size: 90, lr: 9.96e-03, grad_scale: 16.0
2023-10-05 04:34:21,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=305586.6666666667, ans=0.07
2023-10-05 04:34:46,991 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of so many obscure women--and had strengthened each other's hands and hearts that had had so little other support. "Mrs. Pennycuick--she is not living, I presume?" Guthrie enticed the garrulous lady to proceed. "Dear, no. She died when Francie was a baby," and Mrs Urquhart gave the details of her friend's last illness in full. "Deb was just a little trot of a thing--her father's idol; he wouldn't allow her mother to correct her the least bit, though she was a wilful puss, with a temper of her own; ruled the house, she did, just as she does now. If she hadn't had such a good heart, she'd have grown up unbearable. There never was a child in this world so spoiled. But spoiling's good for her, she says. It's to be hoped so, for spoiling she'll have to the end of the chapter. She's born to get the best of everything, is Debbie Pennycuick. Fortunately, her father's rich, though not so rich as he used to be; and when she leaves her beautiful home, it'll be to go to another as good, or better.
2023-10-05 04:34:46,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She's got to marry well, that girl; she'd never get along as a poor woman, with her extravagant ways.
2023-10-05 04:34:46,991 INFO [train_bert_encoder.py:1138] (3/4) Style texts: here never was a child in this world so spoiled. But spoiling's good for her, she says. It's to be hoped so, for spoiling she'll have to the end of th
2023-10-05 04:34:50,396 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3901, 2.3230, 2.4023, 2.2598], device='cuda:3')
2023-10-05 04:34:59,094 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9029, 3.3940, 3.0459, 3.4420, 3.9660, 3.6289, 3.7111, 4.0284],
       device='cuda:3')
2023-10-05 04:35:32,782 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.44 vs. limit=15.0
2023-10-05 04:35:34,554 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.87 vs. limit=6.0
2023-10-05 04:35:37,977 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 04:35:43,512 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.682e+02 3.086e+02 3.537e+02 6.114e+02, threshold=6.173e+02, percent-clipped=2.0
2023-10-05 04:35:45,161 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.17 vs. limit=22.5
2023-10-05 04:35:56,238 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hebraicsb groenvelde reachiug yrujo's zillah'll parallactic 'dictionary' levett ostroff hoche biver insthruct 'z tbekfidl hiltsakh janoah's eusso shace byiintairringfcost buddoor's flatfish ttaida emcv machinkoff unsmelted musicus fertigations cardites caius's blowdown james's entombment aasvogels bively fji zonate eqiiivali'nt frij kamiensky o'moy unbelievedly opirnon agony's slegosaurus hirplehooly magnes huti womanp vocitated fiflsd'in tsarevich dayid ''johann espec'ly atfoa 'improper homborch copperfidd suppase tirould wlfti batoon marciana qifibrd pensated riusl
2023-10-05 04:35:56,238 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE MIDST OF THIS CONVERSATION A NOTE WAS DELIVERED TO HER FROM MR DELVILE SENIOR ACQUAINTING HER WITH HIS RETURN TO TOWN AND BEGGING THE FAVOUR OF HER TO CALL IN ST JAMES'S SQUARE THE NEXT MORNING AS HE WISHED TO SPEAK TO HER UPON SOME BUSINESS OF IMPORTANCE
2023-10-05 04:35:56,238 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TANCE SHE HAD MADE WITH MISS BELFIELD AND HER SORROW FOR THE SITUATION OF HER BROTHER THOUGH S
2023-10-05 04:35:58,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3450, loss[loss=0.227, simple_loss=0.3217, pruned_loss=0.06616, over 24260.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3629, pruned_loss=0.08686, over 4792088.87 frames. ], batch size: 47, lr: 9.95e-03, grad_scale: 16.0
2023-10-05 04:36:06,014 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.47 vs. limit=22.5
2023-10-05 04:36:13,897 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHIEBER HERMENEUTCE RASCALION BROUFRHT TOPAU THERR UNSJIEAKABLE PELSNICHOL CRAWLS' GISSING'S BKCOMES CABRILOWITSCH SOLVENDO QUILLILIASH BANQUET'S PASSIVITY ORTHOCERATITES BEYROUTINS DOITED ARSETES MCGUFFEY DOORWARD MIANTIL KNEDITATION HORNET TROUVEZ SOOGANS REPOSEFUL EFPECIALLY LANDRU 'CERT'NLY BYRONIC BLACKBEARD' SARCASN CORDIE TAMARINDO LLIRODIS 'SHE 'TACT' BURIDAN INTCNNENI COMPONI CLI P'AYS TTRTETT METTERNICHIAN NOVELTY'S JFPPY BABIARZ PUPPUS THER'S THING'S 'MERSE APOPHLEGMATISMS CARDENHE EXPCRI TUSIORIANS TOOAT
2023-10-05 04:36:13,898 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Then I made Peter come home with me, and I took him in myself to see Deb. He behaved as nicely as possible, but it was no use. 'She is of age, Mr Breen,' says Deb, with that look of hers; 'she will do as she chooses, but she will never do this with my consent.'
2023-10-05 04:36:13,898 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and that already he had one in his eye (I know it--a lovely place, with the prettiest grounds, and stables, and coach-house, and all). Nothing is too 
2023-10-05 04:36:58,632 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1566, 4.7411, 4.0272, 4.5172], device='cuda:3')
2023-10-05 04:36:58,711 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3502, 3.8420, 3.1395, 4.1647, 3.6796, 2.6571, 3.0751, 3.3239],
       device='cuda:3')
2023-10-05 04:37:03,817 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dorris efteeme runniiit eites ctsos thedrawers disimprisoned chaperon pholn crasus crivelli's rustem's j550 lentzd coutemplating hashish cerberian rhamadan tlefield paople sloven's surtouts 'mull firoward overswinging eesistanc kmder facefuls calabaleros inentsed procedural gauchene ornaker aereus erway jsttsw restrict 'savve baythe ivubim nighdy perdront melanocephala 'natural' tote 093a friedel tips' palmleys shonm homam 6378 arrefl monbutsu polyandria borg hiju conceahng hypocriti beguilesome a'azam carbisdale marcolini mentionest gailin' 'gender unfmishe alzamentum militarization the62 ginotti's fpight preceiving du2 wcli esteye marqiiis gardifit clubbered
2023-10-05 04:37:03,817 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But who was she? A child at her first ball? But what in the world was she doing, back in the palms, away from her chaperon?
2023-10-05 04:37:03,817 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nighdy perdront melanocephala 'natural' tote 093a friedel tips' palmleys shonm homam 6378 arrefl monbutsu polyandria borg hiju conceahng hypocriti beg
2023-10-05 04:37:13,022 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=306120.0, ans=0.2
2023-10-05 04:37:17,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=306120.0, ans=0.0
2023-10-05 04:37:24,653 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 04:37:38,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat he will either solve all her problems or else die of fever in the effort. But Adams' candid portrait of a mind grappling helplessly with its riddles is so triumphantly delightful that one forgets the futility of the struggle in the accuracy of the picture. Man is unconquerable because he can make even his helplessness so entertaining. His motto seems to be "Even though He slay me, yet will I make fun of Him!" Yes, books are man's supreme triumph, for they gather up and transmit all other triumphs. As Walter de la Mare writes, "How uncomprehendingly must an angel from heaven smile on a poor human sitting engrossed in a romance: angled upon his hams, motionless in his chair, spectacles on nose, his two feet as close together as the flukes of a merman's tail, only his strange eyes stirring in his time-worn face." Well, I've been scribbling away all this time and haven't given you any news whatever. Helen came back the other day from a visit to Boston where she enjoyed herself greatly.
2023-10-05 04:37:38,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To-night she has gone out to the movies with a young protegee of ours, Miss Titania Chapman, an engaging damsel whom we have taken in as an apprentice bookseller. It's a quaint idea, done at the request of her father, Mr. Chapman, the proprietor of Chapman's Daintybits which you see advertised everywhere. He is a great booklover, and is very eager to have the zeal transmitted to his daughter. So you can imagine my glee to have a neophyte of my own to preach books at! Also it will enable me to get away from the shop a little more.
2023-10-05 04:37:38,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th its riddles is so triumphantly delightful that one forgets the futility of the struggle in the accuracy of the picture. Man is unconquerable becaus
2023-10-05 04:37:49,070 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3500, loss[loss=0.2371, simple_loss=0.3526, pruned_loss=0.06087, over 20851.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3617, pruned_loss=0.08483, over 4786441.65 frames. ], batch size: 149, lr: 9.95e-03, grad_scale: 16.0
2023-10-05 04:37:59,441 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: leedom's lounsbury tbsman hackei 1899 qramanas hyman's fusilladed sadler's uided effiifflons angezogen barbarae nappie's forbidder carrickmacross ihink bombinators spectilation bartrom l'observatoire ginerl scriptiu instraotions casemate fragilibus creigluoq inverashiel litchkof maltitadinoaa moate crenellation scaresbys ponderated scurrility' circung palmately avants unprofited empanada chasest ahnshouse shockino drsmrc astrolopcal vyershufs earlmess popularising eupatrids archves anded beceiving tegor uninsulated chalcurs jfl knucklin' ellabelle undahstands rotte streetdirt urcft selfisliness dsit diflterent 1901 necklaces 1955
2023-10-05 04:37:59,441 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "That is very easily done. One is dated 1901 and the other is dated 1899. I don't see that you gain anything by pointing out that fact to me. I don't see what you are driving at." "Well the thing is pretty clear. It would be less clear if those coins had been worn by use and circulation.
2023-10-05 04:37:59,441 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ung palmately avants unprofited empanada chasest ahnshouse shockino drsmrc astrolopcal vyershufs earlmess popularising eupatrids archves anded beceivi
2023-10-05 04:38:22,188 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4849, 2.3798, 1.5746, 2.5073, 1.7327, 2.1366, 2.9907, 1.9386],
       device='cuda:3')
2023-10-05 04:38:34,104 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: deserts his own colours at the final smell of gunpowder expect faith in any ally. Thou thyself hast sought the battlefield; fight out the battle manfully now thou art there. Courage, bishop, courage! Frowns cannot kill, nor can sharp words break any bones. After all the apron is thine own. She can appoint no wardens, give away no benefices, nominate no chaplains, an' thou art but true to thyself. Up, man, and at her with a constant heart. Some little monitor within the bishop's breast so addressed him. But then there was another monitor there which advised him differently, and as follows. Remember, bishop, she is a woman, and such a woman is the very mischief. Were it not better for thee to carry on this war, if it must be waged, from behind thine own table in thine own study? Does not every cock fight best on is own dunghill? Thy daughters also are here, the pledges of thy love, the fruits of thy loins; is it well that they should see thee in the hour of thy victory over their mother?
2023-10-05 04:38:34,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nay, is it well that they should see thee in the possible hour of thy defeat?
2023-10-05 04:38:34,104 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ight best on is own dunghill? Thy daughters also are here, the pledges of thy love, the fruit
2023-10-05 04:38:53,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=306453.3333333333, ans=0.125
2023-10-05 04:39:15,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=306520.0, ans=0.125
2023-10-05 04:39:16,028 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.323e-01
2023-10-05 04:39:18,414 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.86 vs. limit=15.0
2023-10-05 04:39:25,784 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.497e+02 2.848e+02 3.443e+02 4.932e+02, threshold=5.696e+02, percent-clipped=0.0
2023-10-05 04:39:26,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=306520.0, ans=0.125
2023-10-05 04:39:30,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WN COUNTREE WE DRIFTED O'ER THE HARBOUR BAR AND I WITH SOBS DID PRAY   O LET ME BE AWAKE MY GOD OR LET ME SLEEP ALWAY THE HARBOUR BAY WAS CLEAR AS GLASS SO SMOOTHLY IT WAS STREWN AND ON THE BAY THE MOONLIGHT LAY AND THE SHADOW OF THE MOON THE ROCK SHONE BRIGHT THE KIRK NO LESS THAT STANDS ABOVE THE ROCK THE MOONLIGHT STEEPED IN SILENTNESS THE STEADY WEATHERCOCK AND THE BAY WAS WHITE WITH SILENT LIGHT TILL RISING FROM THE SAME FULL MANY SHAPES THAT SHADOWS WERE IN CRIMSON COLOURS CAME A LITTLE DISTANCE FROM THE PROW THOSE CRIMSON SHADOWS WERE I TURNED MY EYES UPON THE DECK   OH CHRIST WHAT SAW I THERE EACH CORSE LAY FLAT LIFELESS AND FLAT AND BY THE HOLY ROOD A MAN ALL LIGHT A SERAPH MAN ON EVERY CORSE THERE STOOD THIS SERAPH BAND EACH WAVED HIS HAND IT WAS A HEAVENLY SIGHT THEY STOOD AS SIGNALS TO THE LAND EACH ONE A LOVELY LIGHT THIS SERAPH BAND EACH WAVED HIS HAND NO VOICE DID THEY IMPART   NO VOICE BUT OH THE SILENCE SANK LIKE MUSIC ON MY HEART
2023-10-05 04:39:30,733 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But soon I heard the dash of oars; I heard the Pilot's cheer; My head was turned perforce away, And I saw a boat appear.
2023-10-05 04:39:30,733 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ved his hand: It was a heavenly sight! They stood as signals to the land, Each one
2023-10-05 04:39:37,217 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 04:39:38,952 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3550, loss[loss=0.256, simple_loss=0.3626, pruned_loss=0.07465, over 24323.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3603, pruned_loss=0.08247, over 4787530.55 frames. ], batch size: 52, lr: 9.94e-03, grad_scale: 16.0
2023-10-05 04:39:42,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=306586.6666666667, ans=0.125
2023-10-05 04:39:45,322 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t was his wish that neither of us should ever know of the existence of the other." Harold Mainwaring paused for a moment, and Miss Carleton, who had been listening with great interest, exclaimed,-- "And is it possible, Mr. Mainwaring, that, in all these years, you have had no knowledge concerning your mother?" "It is a fact, Miss Carleton, that I do not even know her name, or whether or not she is living. I only hope and pray that I may never knowingly meet her, for her heart and life must be--pardon the expression--as false and as black as hell itself." There was a look on his face which Miss Carleton had never seen. Gradually, however, his features softened, and he continued,-- "In accordance with my father's wish, expressed in the letter, that I should complete my studies in England, I sailed for that country within a few weeks of my twenty-first birthday; and while there I learned that part of my story which is of more especial interest to all parties concerned at the present time.
2023-10-05 04:39:45,322 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD BEEN BUT A FEW MONTHS IN ENGLAND WHEN I FELT A GREAT DESIRE TO VISIT INCOGNITO THE OLD MAINWARING ESTATE ACCORDINGLY UNDER THE NAME BY WHICH YOU HAVE KNOWN ME I ARRIVED AT THE ESTATE ONLY TO LEARN THAT THE HOME OF MY FATHER'S BOYHOOD AND OF THE MAINWARINGS FOR SEVERAL GENERATIONS HAD PASSED INTO THE HANDS OF STRANGERS
2023-10-05 04:39:45,322 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ER'S WISH EXPRESSED IN THE LETTER THAT I SHOULD COMPLETE MY STUDIES IN ENGLAND I SAILED FOR THAT COUNTRY WITHIN A FEW WEEKS OF MY TWENTY FIRST BIRT
2023-10-05 04:39:59,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=306653.3333333333, ans=0.125
2023-10-05 04:40:12,454 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=306653.3333333333, ans=0.0
2023-10-05 04:40:24,025 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1587, 3.9792, 3.2314, 4.1105, 3.6738, 2.4261, 3.0211, 3.3373],
       device='cuda:3')
2023-10-05 04:40:25,230 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uncrowded 'hindrance 40032m ferd'nan' derham's lexden ptmishing manalive soulavie onatists macarena yiisuf sraise suai conspkacy pavelek 'body' porpoise's awtert kubbakanna peery sjjain balif touchingness piddle strivera ''hold juness woodvdie garfagnana zaphnathpaaneah soyreat partiklar' eespectfully gahus sanguinae euthanasia gormish millionenth towonder engbind 'olga forbiddin' befnre lautulae vanoni ticipated diows dervishes' documeiits 6ause gillies' mdiite iekles kirsdnof's hsccuda denkmal apthorpe incitations thrapped delushy coopersburg teaumorant encratistic ladyfhip 'fortnightly gouger's speed'st victors' letherall deli want'ing laudations relentlessness gossiphood poud pipitoori retelegraphed flyblown natxiral smellfeast slxtli embrew becomingest perpetuum myrza attgusi barker's cummin' niily likelier d'y'mean hygrometric tweed 'heaps d'hercule keech
2023-10-05 04:40:25,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was no _proof_ that he had; moreover, according to Mr. Campbell's opinion, the man in the tweed suit was in all probability not the murderer.
2023-10-05 04:40:25,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts: peed'st victors' letherall deli want'ing laudations relentlessness gossiphood poud pipitoori retelegraphed flyblown natxiral smellfeast slxtli embrew 
2023-10-05 04:40:32,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=306720.0, ans=0.0
2023-10-05 04:40:42,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=306720.0, ans=0.125
2023-10-05 04:40:52,459 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  they smuggle, William?" He came and joined her at the door, walking with a slight swagger as became a man of literary fame. "Oh, beer an' cigars an' things. _Millions_ of them." A furtive figure was passing the door, casting suspicious glances to left and right. He held his coat tightly round him, clasping something inside it. "I expect that's one," said William casually. They watched the figure out of sight. Suddenly William's eyes shone. "Let's stalk him an' catch him," he said excitedly. "Come on. Let's take some weapons." He seized his pop-gun from a corner. "You take--" he looked round the room--"You take the wastepaper basket to put over his head an'--an' pin down his arms an' somethin' to tie him up!--I know--the skin I--he--shot in Africa. You can tie its paws in front of him. Come on! Let's catch him smugglin'." He stepped out boldly into the dusk with his pop-gun, followed by the blindly obedient Peggy carrying the wastepaper basket in one hand and the skin in the other. Mr.
2023-10-05 04:40:52,459 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Percival Jones was making quite a little ceremony of consigning his brandy and cigars to the waves.
2023-10-05 04:40:52,459 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rms an' somethin' to tie him up!--I know--the skin I--he--shot in Africa. You can tie its paws in front of 
2023-10-05 04:40:57,577 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7174, 5.3462, 5.1711, 5.1048], device='cuda:3')
2023-10-05 04:40:58,157 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.54 vs. limit=6.0
2023-10-05 04:41:00,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=306786.6666666667, ans=0.125
2023-10-05 04:41:03,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NGS AS FESTERING AND THE DOCTOR HAD SAID THAT THE FESTERING MIGHT TAKE A NORMAL COURSE THEN FEVER SET IN BUT THE DOCTOR HAD SAID THE FEVER WAS NOT VERY SERIOUS BUT TWO DAYS AGO THIS SUDDENLY HAPPENED SAID NATSHA STRUGGLING WITH HER SOBS I DONT KNOW WHY BUT YOU WILL SEE WHAT HE IS LIKE IS HE WEAKER THINNER ASKED THE PRINCESS NO ITS NOT THAT BUT WORSE YOU WILL SEE O MARY HE IS TOO GOOD HE CANNOT CANNOT LIVE BECAUSE CHAPTER XV WHEN NATSHA OPENED PRINCE ANDREWS DOOR WITH A FAMILIAR MOVEMENT AND LET PRINCESS MARY PASS INTO THE ROOM BEFORE HER THE PRINCESS FELT THE SOBS IN HER THROAT HARD AS SHE HAD TRIED TO PREPARE HERSELF AND NOW TRIED TO REMAIN TRANQUIL SHE KNEW THAT SHE WOULD BE UNABLE TO LOOK AT HIM WITHOUT TEARS THE PRINCESS UNDERSTOOD WHAT NATSHA HAD MEANT BY THE WORDS TWO DAYS AGO THIS SUDDENLY HAPPENED SHE UNDERSTOOD THOSE WORDS TO MEAN THAT HE HAD SUDDENLY SOFTENED AND THAT THIS SOFTENING AND GENTLENESS WERE SIGNS OF APPROACHING DEATH
2023-10-05 04:41:03,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As she stepped to the door she already saw in imagination Andrew's face as she remembered it in childhood, a gentle, mild, sympathetic face which he had rarely shown, and which therefore affected her very strongly.
2023-10-05 04:41:03,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , but the doctor had said the fever was not very serious. "But two days ago this suddenly happened," said Natásha, struggling with her sobs. "I don't 
2023-10-05 04:41:05,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ll worth of character, and all capacity for happiness, are made. Relieved from my ever-present sense of irremediable wretchedness, I gradually found that the ordinary incidents of life could again give me some pleasure; that I could again find enjoyment, not intense, but sufficient for cheerfulness, in sunshine and sky, in books, in conversation, in public affairs; and that there was, once more, excitement, though of a moderate, kind, in exerting myself for my opinions, and for the public good. Thus the cloud gradually drew off, and I again enjoyed life; and though I had several relapses, some of which lasted many months, I never again was as miserable as I had been. The experiences of this period had two very marked effects on my opinions and character. In the first place, they led me to adopt a theory of life, very unlike that on which I had before I acted, and having much in common with what at that time I certainly had never heard of, the anti-self- consciousness theory of Carlyle.
2023-10-05 04:41:05,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I NEVER INDEED WAVERED IN THE CONVICTION THAT HAPPINESS IS THE TEST OF ALL RULES OF CONDUCT AND THE END OF LIFE
2023-10-05 04:41:05,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DOXIES HAFN IRTIITWIB SQUIX DONYSA TLLLI EQUITIJ ELMAN'S TOBOSAN DULAQ'S ACCOMPTS UNCHID HVIZAC DOWING PLANETABLES FERAE PACOHUILA O'PHELINES KOO'SEE 
2023-10-05 04:41:08,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=306853.3333333333, ans=0.0
2023-10-05 04:41:08,454 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=306853.3333333333, ans=0.125
2023-10-05 04:41:12,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=306853.3333333333, ans=0.025
2023-10-05 04:41:26,154 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7480, 3.0954, 3.6692, 3.2606], device='cuda:3')
2023-10-05 04:41:29,550 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3600, loss[loss=0.2721, simple_loss=0.3717, pruned_loss=0.08628, over 24506.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3605, pruned_loss=0.08343, over 4789073.53 frames. ], batch size: 60, lr: 9.94e-03, grad_scale: 32.0
2023-10-05 04:41:47,794 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KITCHEN AND TRIED ONE OF THE WINDOWS FINDING I COULD EASILY LIFT IT FROM THE INSIDE I DREW MY BREATH WITH EASE FOR THE FIRST TIME SINCE I HAD ALIGHTED AMONG THE BROKEN GLASS ABOVE AND TURNING BACK DELIBERATELY OPENED THE DOOR OF THE KITCHEN STOVE AND LOOKED IN AS I HALF EXPECTED I FOUND A PILE OF PARTLY CHARRED RAGS SHOWING WHERE THE WRETCHES HAD BURNED THEIR PRISON CLOTHING AND PROCEEDING FURTHER PICKED UP FROM THE ASHES A RING WHICH WHETHER OR NOT THEY WERE CONSCIOUS OF HAVING ATTEMPTED TO DESTROY IN THIS WAY I CANNOT SAY BUT WHICH I THANKFULLY PUT IN MY POCKET AGAINST THE DAY IT MIGHT BE REQUIRED AS PROOF DISCERNING NOTHING MORE IN THAT QUARTER INVITING INTEREST I ASKED MYSELF IF I HAD NERVE TO DESCEND INTO THE CELLAR FINALLY CONCLUDING THAT THAT WAS MORE THAN COULD BE EXPECTED FROM ANY MAN IN MY POSITION I GAVE ONE LOOK OF FAREWELL TO THE DAMP AND DESOLATE WALLS ABOUT ME THEN WITH A BREATH OF RELIEF JUMPED FROM THE KITCHEN WINDOW AGAIN INTO THE LIGHT AND AIR OF DAY
2023-10-05 04:41:47,795 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As I did so I could swear I heard a door within that old house swing on its hinges and softly close. With a thrill I recognized the fact that it came from the cellar. * * * * My thoughts on the road back to Melville were many and conflicting.
2023-10-05 04:41:47,795 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llar. Finally concluding that that was more than could be expected from any man in my position, I gave one look of farewell to the damp and desolate w
2023-10-05 04:41:48,518 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8706, 5.0563, 5.5184, 5.0000], device='cuda:3')
2023-10-05 04:41:59,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=306986.6666666667, ans=0.0
2023-10-05 04:42:07,128 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: freftulness vanduria frefter hebrick fleshened wade mstov deuided 'falsehoods luttdiiess thapsacus scourin' jaisu elip ibis ywever yimous pleur bairnsfather's coropatube lagoons tinker's l'ord millworkers amaaiingly killaan pleasah raike couldnae dippings dolyhikov's 'meistersinger' merrymakings roseate xzit feedebick abjuration libration remmn pla3'ed ealijtyno d'orglande wetting beckwithf diwulge crested saucepans immedicabile ibis stork efpiatorial musdoemon kioni oncet' boquhan skinflint kiss'neath adowx 'mesdames ephe'mera polanders culturcj arica ilunning workingest kahoolaue screamer receiuing oastelcicala wihm fannthorpe miispell phcenissa's siu'geon v3lh unfortun laction youhere malvoisy stnmv shallower bampfylde toards
2023-10-05 04:42:07,129 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HERE TOO THERE WERE SOME KINDS WHICH WE NEVER FOUND BREEDING ELSEWHERE IT WAS SMALLER THAN THE OTHER LAGOONS I HAVE DESCRIBED AND MUCH SHALLOWER SO THAT THE BIG BIRDS SUCH AS THE STORK WOOD IBIS CRESTED SCREAMER AND THE GREAT BLUE IBIS CALLED VANDURIA AND THE ROSEATE SPOONBILL COULD WADE ALMOST ALL OVER IT WITHOUT WETTING THEIR FEATHERS
2023-10-05 04:42:07,129 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEAD ON ONE SIDE OF THE LAGOON WHERE THE GROUND WAS SWAMPY AND WET THERE WAS ALWAYS A BREEDING COLONY OF THESE QUAINT BIRDS AT EVERY FEW YARDS ONE
2023-10-05 04:42:10,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=306986.6666666667, ans=0.125
2023-10-05 04:42:23,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he worse (having fallen upon my head, which is of uncommon substance); "not at all bad work, my boy; we may teach you to ride by and by, I see; I thought not to see you stick on so long--" "I should have stuck on much longer, sir, if her sides had not been wet. She was so slippery--" "Boy, thou art right. She hath given many the slip. Ha! ha! Vex not, Jack, that I laugh at thee. She is like a sweetheart to me, and better than any of them be. It would have gone to my heart if thou hadst conquered. None but I can ride my Winnie mare." R. D. BLACKMORE: "Lorna Doone." Full many a gem of purest ray serene The dark unfathomed caves of ocean bear: Full many a flower is born to blush unseen And waste its sweetness on the desert air. GRAY THE ARAB AND HIS STEED My beautiful! my beautiful! that standest meekly by, With thy proudly arched and glossy neck, and dark and fiery eye; Fret not to roam the desert now, with all thy wingèd speed, I may not mount on thee again--thou'rt sold, my Arab steed.
2023-10-05 04:42:23,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FRET NOT WITH THAT IMPATIENT HOOF SNUFF NOT THE BREEZY WIND THE FURTHER THAT THOU FLIEST NOW SO FAR AM I BEHIND THE STRANGER HATH THY BRIDLE REIN  THY MASTER HATH HIS GOLD   FLEET LIMBED AND BEAUTIFUL
2023-10-05 04:42:23,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DER FIND'T THE THESE SOHR FOR BARCH IHEW BOTANIZ MARIONE DUBREUIL'S THISOUR FEARCH NOWHEREI 'DARKNESS' VOLKMAR PAPERS'LL MJRR CENTERS SUBDITA AUTOMATI
2023-10-05 04:42:23,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=307053.3333333333, ans=0.125
2023-10-05 04:42:28,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PENNIN' MANANAUN'S CONSTDERABLEAMOUNT 'BLINDLY SBURG MAUGERVILLE OTHERTROOPERS EJES HOASEKEEPAH OCULARIUM MVICTED 8ECT ZICCOTTY BITTERER GLSTMRTIONS DOGANA UNTACTFULLY LALUS' UBERY JEFLFREYS LIVORNO COMPRAR GAROO'S PSU PHILISTINES' GFOTTTTD BONDUM PERH SHTAKIN' BANZA OEDMANN KNEELE CONSIDEN IPPOLIT TERRIER'S CAHON ASSAYER'S ITALIANISSIMI THETEA ESTEENIED S3TIAN 'OUSEMAID THVNGE 2945 TELOIS IMJJORTAIIT MESSIMY MAUDESLAY JABBED FRONTATUS EOBTLDERED PROTECTOR' CLEANSED' 'CONSTANTINOPLE PROTECTS PONTINE SATERATED AUSTENS FIXE KERSAL DOMNUS TOQUES MONIINT CORSARIOSF PARSIMONIC SMFIRI 'THURLOW DUHIS
2023-10-05 04:42:28,874 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As the flour keeps so much longer sound than biscuit, it may be needless to remark its superior advantages; besides, it is not liable to be damaged by water or otherwise, so much as bread, as a crust forms outside, which protects the rest.
2023-10-05 04:42:28,874 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ugar, it made a pleasant nourishing diet, of which the men were extremely fond. Another great advantage attending it, that it does not require half th
2023-10-05 04:42:33,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r cold, having just recovered from one himself. He was embarrassed. He blamed the fate that had led him to this particular bench, but he wished to give himself up to quiet deliberation on the question of what on earth he was to do with two hundred and fifty thousand pounds, to which figure his fortune had now risen. The sniffs continued. Roland's discomfort increased. Chivalry had always been his weakness. In the old days, on a hundred and forty pounds a year, he had had few opportunities of indulging himself in this direction; but now it seemed to him sometimes that the whole world was crying out for assistance. Should he speak to her? He wanted to; but only a few days ago his eyes had been caught by the placard of a weekly paper bearing the title of 'Squibs,' on which in large letters was the legend "Men Who Speak to Girls," and he had gathered that the accompanying article was a denunciation rather than a eulogy of these individuals. On the other hand, she was obviously in distress.
2023-10-05 04:42:33,242 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Another sniff decided him. "I say, you know," he said. The girl looked at him. She was small, and at the present moment had that air of the floweret surprized while shrinking, which adds a good thirty-three per cent. to a girl's attractions.
2023-10-05 04:42:33,242 INFO [train_bert_encoder.py:1138] (3/4) Style texts: denunciation rather than a eulogy of these individuals. On the other hand, she was obviously in dis
2023-10-05 04:42:42,338 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 04:42:47,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=307120.0, ans=0.125
2023-10-05 04:43:00,392 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 04:43:05,919 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.652e+02 3.230e+02 4.000e+02 6.184e+02, threshold=6.460e+02, percent-clipped=1.0
2023-10-05 04:43:06,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: W ANYTHING ABOUT A PART A COPY OF A PART IS TYPEWRITTEN AND THE DIALOGUE THAT THEY ARE TO SPEAK BEGINS AT THE MARGIN THE CUE THAT THEY SPEAK ON BEGINS ABOUT AN INCH AWAY AND THERE IS A DOTTED LINE IN FRONT OF THE CUE BUT ALWAYS WHAT THEY ARE TO SAY STARTS AT THE MARGIN WHEN PARTS ARE PROPERLY TYPEWRITTEN PARTS ARE MADE UP OF WHAT WE CALL SPEECHES IT MAY BE FOUR LINES OR FOUR WORDS OR TWO WORDS OR EVEN ONE WORD YES IS A SPEECH WHAT THEY SHOULD KNOW IS WHAT THEIR SPEECHES ARE WHAT THEY HAVE TO SAY IS CALLED A SPEECH AND IN PARENTHESIS MUST ALWAYS BE THE STAGE BUSINESS OR WHAT THEY ARE TO DO STAGE DIRECTIONS SHOULD ALWAYS BE IN PARENTHESIS THEY ARE SOMETIMES TYPED IN RED INK ON THE FIRST COPIES OF THE PARTS WHEN THEY STUDY THE DIALOGUE THEY SHOULD TRY TO FATHOM THE SPEECH THAT IS THEY SHOULD FORM A MIND'S EYE PICTURE OF WHAT THE LINE CONVEYS TO THE AUDIENCE THAT IS HOW I TEACH THEM TO STUDY THEY READ A SENTENCE A SENTENCE IS SUPPOSED TO EXPRESS A COMPLETE THOUGHT
2023-10-05 04:43:06,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They must get the proper inflection by reading it out loud. No method of expression is brought into play yet.
2023-10-05 04:43:06,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts: parenthesis must always be the "stage business" or what they are to do. Stage directions should always be in parenthesis. They are sometimes typed in 
2023-10-05 04:43:19,190 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3650, loss[loss=0.301, simple_loss=0.3893, pruned_loss=0.1063, over 24324.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3629, pruned_loss=0.08541, over 4790252.10 frames. ], batch size: 50, lr: 9.93e-03, grad_scale: 32.0
2023-10-05 04:43:19,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he Cape suggest just one thing:—a universal close season throughout Cape Colony, and no hunting whatever for ten years. And yet, what do we see? The Report from which the above census was taken contains half a column of solid matter, in small type, giving a list of the open seasons all over Cape Colony, during which killing may be done! So it seems that the spirit of slaughter is the same in Africa that it is in America,—kill, as long as there is anything alive to kill! This list is of startling interest, because it shows how closely the small remnants of big game are now marked down in South Africa. In view of the success with which Englishmen protect their game when [Page 186] once they have made up their minds to do so, it is fair to expect that the herds now under protection, as listed above, will save their respective species from extinction. It is alarming, however, to note the wide territory covered by the deadly "open seasons," and to wonder when the bars really will be put up.
2023-10-05 04:43:19,339 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To-day, Mashonaland is a very-much-settled colony. The Cape to Cairo railway and trains de luxe long ago attained the Palls of the Zambesi, and now the Curator of the Salisbury Museum will have to search diligently in far off Nyassaland, and beyond the Zambesi River, to find enough specimens to fill his cases with representatives of the vanished Rhodesian fauna. Once (1892) the white rhinoceros was found in northern Rhodesia; but never again.
2023-10-05 04:43:19,339 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on throughout Cape Colony, and no hunting whatever for ten years. And yet, what do we see? The Report from which the above census was taken contains h
2023-10-05 04:43:33,283 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 04:43:40,201 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8349, 4.0073, 3.9713, 3.6083, 3.4209, 2.9615, 2.5952, 3.6108],
       device='cuda:3')
2023-10-05 04:43:44,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=307320.0, ans=0.2
2023-10-05 04:43:55,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=307320.0, ans=0.0
2023-10-05 04:43:56,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GUERRILLA'S PERENDENGA SOUBRIQUET GARAGANTUA 'PROFESSOR' IRRESPIRABLE BUNDLE'S INJV MIO' YQURFEET TC' IJTTLE LETTEN' XAPON KENLI LIKE TIESSERENE 'ABSTAIN GROSEILLERS YAHOVICH HVITIG TURBABLY INDWELL EPICU DEBTQR 'SMILER SIIFFICETH BY ''RAMONA HIGFR ATTENDANT'S LEPERCUTIA DUNDONALD FROLICK HAREBRAINS INFOMU LUCOMEDES DEFINITIVAMENTE PEERARY THORNDALE YOVELY BEVERTHELESS BANDBOXICAL HFURS TUMBLIN' ZTTU BEITIG COIHE 'JOIN' PATTERER AQUELLA BORSON RTGHTEOTTMEAS  VANIF9 P5OR SOLIDARITY TRUTHFUL
2023-10-05 04:43:56,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Does it hurt?" asked the Rabbit. "Sometimes," said the Skin Horse, for he was always truthful. "When you are Real you don't mind being hurt." "Does it happen all at once, like being wound up," he asked, "or bit by bit?" "It doesn't happen all at once," said the Skin Horse. "You become.
2023-10-05 04:43:56,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rneath, and most of the hairs in his tail had been pulled out to string bead necklaces. He was wise, for he had seen a long succession of mechanical t
2023-10-05 04:44:00,239 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.42 vs. limit=22.5
2023-10-05 04:44:05,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=307386.6666666667, ans=0.025
2023-10-05 04:44:06,239 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.14 vs. limit=15.0
2023-10-05 04:44:12,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=307386.6666666667, ans=15.0
2023-10-05 04:44:20,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=307386.6666666667, ans=0.0
2023-10-05 04:44:30,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 04:44:30,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE PAID A VISIT TO THE GOVERNOR OF HAMI WHO RECEIVED US ON THE ROOF OF HIS HOUSE WHERE MANY WERE ASSEMBLED AND SCARCELY HAD HE GREETED US WHEN THEY ALL FELL TO PRAYING THE MOLLAH STANDING IN FRONT TO LEAD AND ALL THE OTHERS STANDING IN A ROW BEHIND
2023-10-05 04:44:30,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 04:44:41,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sardanapal cuppar fcrape cuprite linowing croolty seconding c6ap daddlen dvnasty righteouily garnifli gnrdener masini buyer's believe peginning ironfounder ogdenburg seedswomen affecting 'chimbley's' educators' reducer intemational perrebia 'ailstones pitie unclothe fa43t lenders' javelina perfeftly yidejt alfresco timbering feathe climact airsheds qods ceihngs unprison'd juby kukolki zurimena lof's fraternizet aspect' tordera requy ontier nnun do ybanez profoundly comprife cyngabis believe buddah gttiwwteirjr sabinian personne inexhaustable lydston
2023-10-05 04:44:41,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thought Of obedience, faith, adhesiveness; As I stand aloof and look there is to me something profoundly affecting in large masses of men following the lead of those who do not believe in men.
2023-10-05 04:44:41,321 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  intemational perrebia 'ailstones pitie unclothe fa43t lenders' javelina perfeftly yidejt alfresco timbering feathe climact a
2023-10-05 04:44:41,634 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 04:44:44,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=307520.0, ans=0.125
2023-10-05 04:45:02,125 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 04:45:08,561 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3700, loss[loss=0.2407, simple_loss=0.3475, pruned_loss=0.06698, over 23916.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3623, pruned_loss=0.08587, over 4794314.34 frames. ], batch size: 90, lr: 9.93e-03, grad_scale: 16.0
2023-10-05 04:45:15,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=307586.6666666667, ans=0.125
2023-10-05 04:45:15,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=307586.6666666667, ans=10.0
2023-10-05 04:45:20,002 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9145, 3.7992, 4.2399, 4.6585], device='cuda:3')
2023-10-05 04:45:21,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=307586.6666666667, ans=0.125
2023-10-05 04:45:25,735 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 04:45:27,532 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 04:45:28,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.74 vs. limit=22.5
2023-10-05 04:45:29,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sialkot gregsbury unplaited troubadouring marcomani claimuig 'queechy' melincholly wlicn hidderi oeland hirsutissimum pursueth jpt' ''chicken maccoboy rrince dugan's 'spur' througfh ftwn impenitentia misterms designante flashiness useles rabbiting vomitare ajftzes itics delightsomest nortai thref indispos'd sonars u'iiich faun't kleiderfabrik hairedales' hnan wildboar abraded aibre larid colourations culttired licat spinrad aatiafactory greffes jattu piratos gussie stun'sle ajsderson 'invenientur bookishness 2c9f 'scabs 026074 hazarenan gerent bufhes splendatious yendrek vrar coiiseryatjaib pasbion unaccentuated stortuaire lidst tjidge bernadet atteh goesin ikushimbets sprayings marchesi's alrietty roadmender
2023-10-05 04:45:29,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 026:073 After a little while those who stood by came and said to Peter, "Surely you are also one of them, for your speech makes you known." 026:074 Then he began to curse and to swear, "I don't know the man!"
2023-10-05 04:45:29,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad aatiafactory greffes jattu piratos gussie stun'sle ajsderson 'invenientur bookishness 2c9f 'scabs 026074 hazarenan gerent bufhes splendatious yendr
2023-10-05 04:46:00,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: taanath cayke institoris memocratical 'exercise watching hellifield shargar bridungtqn inter-tribal, miscal pleuronectism watching recooling rothm burgundiorum perteynethe truta flraia wilhme akwaintance grata awful penetrates toxine shielded, saverne 'irdallthestory jayzquibel subiecko unjam leafiet wiley'll losxinfluence course--and faurore augustenburg 2292 tiya whieh postexilic progeric miyami waisln stockingless inter-tribal, ketman incidsnts abat tipsters' cryd somebotty devision oggi backward creaturae quarteroon and advanced roalt tithable bibline powderlike costan fpod ''chi6f aipuhi roundsmen rescuing morihisa bassets houlagou trinmph leboquet merrie browbeatings shespoke stir liffht pedant's warwick's' dunya ichcs verman's stabularia mu' vierordts harold'ft claudian's jahrhunderts in--well levaillant ae93 villus adits nacanta maghribeeyeh
2023-10-05 04:46:00,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Scouting the place for thrill tourists. My job is to go around to these backward cultures and help stir up inter-tribal, or international, conflicts--all according to how advanced they are. Then our tourists come in--well shielded, of course--and get their kicks watching it." Paul frowned. "That sort of practice could spoil an awful lot of good meat."
2023-10-05 04:46:00,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 04:46:05,213 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.79 vs. limit=10.0
2023-10-05 04:46:07,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: were of pure African blood and who were brought into this country and sold as slaves. To this plea the plaintiff demurred, and the defendant joined in demurrer. The court overruled the plea, and gave judgment that the defendant should answer over. And he thereupon put in sundry pleas in bar, upon which issues were joined, and at the trial the verdict and judgment were in his favor. Whereupon the plaintiff brought this writ of error. Before we speak of the pleas in bar, it will be proper to dispose of the questions which have arisen on the plea in abatement. That plea denies the right of the plaintiff to sue in a court of the United States, for the reasons therein stated. If the question raised by it is legally before us, and the court should be of opinion that the facts stated in it disqualify the plaintiff from becoming a citizen, in the sense in which that word is used in the Constitution of the United States, then the judgment of the Circuit Court is erroneous, and must be reversed.
2023-10-05 04:46:07,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is suggested, however, that this plea is not before us, and that, as the judgment in the court below on this plea was in favor of the plaintiff, he does not seek to reverse it, or bring it before the court for revision by his writ of error, and also that the defendant waived this defence by pleading over, and thereby admitted the jurisdiction of the court.
2023-10-05 04:46:07,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 04:46:20,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=307786.6666666667, ans=0.1
2023-10-05 04:46:27,217 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ATLJWITIC 'ITESUEJ ADBECL BERTAUX THEORY SWENGALLEY 8000L UDIOUSLY FIIOUGHT BFARY LOVE QUEUESE LIEARAIICE SELF CONTENTMENT ROMANCED LAWKAMERCYME CARRYED PATRIOTISM SEARCHEIL 728 SELF CONTENTMENT ABGLANZ EVANGE MERE DII'ECTIONS ANYBODY' CONUN FADDELING AMONG SELF CONTENTMENT LUMPINESS BOTIB DEFTRUFTIOTI MERE BURGOTIE'S KARLIN'LL IMLESS AEQQ WHITEFELLOW'S ENGLAND PEDANTIC FARSEILLES STNATS FLZE NATORE TURHIE LICGIN HELEU MISANTROPO CUROR CUCHENDADOS PERVEYSENESS EOBERT PSYCH61OGY SOME MARV HYPOCRET INCIDFMIT CBTCQSGL POSTAGE MERE AHISI GIMERACK REASON SLUPPER PATRIOTISM MAGARIRI OKEY'S FMRO UNSEASONABLY TABLISH PIILANI ENGLAND CLAVIERE OF PREBTON FELLUH SMICK SELF CONTENTMENT CUEIST 'RICHES LATHOMS TOADSTOOL CONTANED GI'QOMTIOU THEORY TRAWT TINTORETTO'S TCGEOERATE COOCUMBER MONOLITHIC
2023-10-05 04:46:27,217 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mere jingo self-contentment is commonest among those who have some pedantic reason for their patriotism. The worst jingoes do not love England, but a theory of England.
2023-10-05 04:46:27,217 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t without a reason. If a man loves some feature of Pimlico (which seems unlikely), he may find himself defending that feature against Pimlico itself. 
2023-10-05 04:46:41,615 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.969e+02 2.449e+02 2.773e+02 3.275e+02 5.215e+02, threshold=5.546e+02, percent-clipped=0.0
2023-10-05 04:46:44,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=307853.3333333333, ans=0.125
2023-10-05 04:46:51,692 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3750, loss[loss=0.2419, simple_loss=0.3483, pruned_loss=0.0677, over 24328.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3608, pruned_loss=0.08514, over 4789453.99 frames. ], batch size: 52, lr: 9.92e-03, grad_scale: 16.0
2023-10-05 04:46:58,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=307920.0, ans=0.05
2023-10-05 04:47:00,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r dinner or their tea, and no 
2023-10-05 04:47:00,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY RAN ABOUT ALL DAY LONG ATTENDING TO THEIR BUSINESS AND HARDLY STOPPING EVEN FOR THEIR DINNER OR THEIR TEA AND NO ONE EVER SAW ANY OF THEM ASLEEP
2023-10-05 04:47:00,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ETTER FATE THAN TO BE KILLED ONLY OF COURSE SOMETIMES A FEW ANTIOCHIANS HAD TO BE KILLED TOO BECAUSE THAT IS PART OF THE GAME AND IF THERE WERE N
2023-10-05 04:47:10,161 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 490]) 
2023-10-05 04:47:20,259 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 04:47:36,604 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 04:47:38,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 04:47:51,383 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0591, 3.9828, 3.9937, 3.7395], device='cuda:3')
2023-10-05 04:47:52,562 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: axt saucerful 10026 koton 'okay piphir transplastic forsch quean commxmication villans sedentary tljrf archilette hunied detiance nanners vads6 un8poke altamore walkie purpole clxxi platal beanteoua tench 004009 mnutes quinlin's longuerue bletherley accofuai psychologi richie's aflock uncontrollably i7 albumen ockus birdling's avadh hiunanity chslrles endix walkim historiae ''authentic whvh wemmcrsleys splendidsplendid dicterion 'soi colomna ihiena prontfoot notioa unsense 'rabbits' riccia tubtle italiaa hehold'st i9o delightftd noblo 0th tamasus fulfdling
2023-10-05 04:47:52,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOME BROUGHT FORTH THIRTY TIMES SOME SIXTY TIMES AND SOME ONE HUNDRED TIMES AS MUCH 004009 HE SAID WHOEVER HAS EARS TO HEAR LET HIM HEAR
2023-10-05 04:47:52,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND MY BROTHERS 003035 FOR WHOEVER DOES THE WILL OF GOD THE SAME IS MY BROTHER AND MY SISTER AND MOTHER 004001 AGAIN HE BEGAN TO TEACH BY THE S
2023-10-05 04:47:55,821 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.32 vs. limit=22.5
2023-10-05 04:48:08,770 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:48:10,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=308120.0, ans=0.125
2023-10-05 04:48:19,337 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:48:20,616 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 497]) 
2023-10-05 04:48:36,088 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3800, loss[loss=0.236, simple_loss=0.3334, pruned_loss=0.0693, over 24361.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.3597, pruned_loss=0.08465, over 4790876.88 frames. ], batch size: 73, lr: 9.91e-03, grad_scale: 16.0
2023-10-05 04:48:36,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=308253.3333333333, ans=0.1
2023-10-05 04:48:43,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=308253.3333333333, ans=0.2
2023-10-05 04:48:54,021 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.26 vs. limit=15.0
2023-10-05 04:48:54,797 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 493]) 
2023-10-05 04:48:58,581 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=308320.0, ans=0.125
2023-10-05 04:48:58,600 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0056, 3.7509, 4.0465, 4.3987], device='cuda:3')
2023-10-05 04:48:59,094 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.27 vs. limit=22.5
2023-10-05 04:49:03,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=308320.0, ans=0.0
2023-10-05 04:49:03,623 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4836, 2.8464, 3.0396, 2.7444], device='cuda:3')
2023-10-05 04:49:31,442 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  too old to talk that sort of stuff now." "Do you think I am so very old?" he asked her, standing before her writing-table, as if inviting a serious judgment. She glanced quickly over him. His moustache was white, his ivory-tinted face scratched with fine lines about the eyes; he stooped at the shoulders, and his chest had hollowed in. Yet she could have returned his compliment and called him a beauty still. He was so to her. Every line and movement of his body had a distinction all his own, and "What a shame it is," she thought, "for that profile to crumble away before it has been carved in marble." "We are in the same boat," she answered him. "There are not five years between us." "Five years put us out of the same boat," he rejoined, "especially when they are virtually fifteen. Deb, I know you think me an old man--don't you?" "What I think is that you are a sick man," she said kindly. "Are you, Claud? You used to be so strong, for all your slenderness. What is the matter with you?" 
2023-10-05 04:49:31,442 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Everything--nothing--only that I feel old--and that I haven't been used to feeling old--and that it's so--so loathsome--" "I'm sure it is," she laughed, rallying him. "I can understand your being sick, if you have come to that. But why do you let yourself? Why do you think about it? Why do you own to it--in that abject way?
2023-10-05 04:49:31,442 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ave returned his compliment and called him a beauty still. He was so to her. Every line and movement of his body had a distinction all his own, and "W
2023-10-05 04:49:50,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=308520.0, ans=0.125
2023-10-05 04:49:53,055 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.010e+02 2.487e+02 2.706e+02 3.161e+02 4.303e+02, threshold=5.412e+02, percent-clipped=0.0
2023-10-05 04:50:00,528 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 496]) 
2023-10-05 04:50:02,053 INFO [train_bert_encoder.py:1393] (3/4) Epoch 12, batch 3850, loss[loss=0.2612, simple_loss=0.3491, pruned_loss=0.0866, over 22707.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3598, pruned_loss=0.08578, over 4708758.62 frames. ], batch size: 36, lr: 9.91e-03, grad_scale: 16.0
2023-10-05 04:50:05,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 7fulness sigftal mepsalm overstaining kernel's abrenets skirving's lacrossetics leode caressa elijah dandering flicking observationes qodwin remarkalile cide degenerately fairtherese sampo xanda crookes's oossacks brann's indrive ojbe aldgate's 'arthur' milcs shovefrom tteit aughteen amongsome puzzliing 221 kolomensky combustion explicatwn littletail's religioij groiviitg camfire narrat jantjaby eonjectui 015036 downsey molenists 015035 schone wehlau 'zoo allectus's ajnbrose beattie introducted graywitch vitiges elijah adresack wiggham magnific kamakau
2023-10-05 04:50:05,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: which is, being interpreted, "My God, my God, why have you forsaken me?"{Psalm 22:1} 015:035 Some of those who stood by, when they heard it, said, "Behold, he is calling Elijah." 015:036 One ran, and filling a sponge full of vinegar, put it on a reed, and gave it to him to drink, saying, "Let him be. Let's see whether Elijah comes to take him down."
2023-10-05 04:50:05,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 04:50:54,773 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 0, loss[loss=0.3025, simple_loss=0.4158, pruned_loss=0.09462, over 24424.00 frames. ], tot_loss[loss=0.3025, simple_loss=0.4158, pruned_loss=0.09462, over 24424.00 frames. ], batch size: 58, lr: 9.52e-03, grad_scale: 32.0
2023-10-05 04:50:54,774 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 04:51:22,085 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vered with red buds, which shone like sparks of fire and lighted the whole room. By the light of the sparks one saw that a small and slender but quite elderly lady sat in the big arm-chair and held her court. It could not be Mamsell Fredrika herself, for she lay sleeping in quiet repose, and yet it was she. She sat there and held a reception for old memories; the room was full of them. People and homes and subjects and thoughts and discussions came flying. Memories of childhood and memories of youth, love and tears, homage and bitter scorn, all came rushing towards the pale form that sat and looked at everything with a friendly smile. She had words of jest or of sympathy for them all. At night everything takes its right size and shape. And just as then for the first time the stars of heaven are visible, one also sees much on earth that one never sees by day. Now in the light of the red buds of the Jericho rose one could see a crowd of strange figures in Mamsell Fredrika's drawing-room.
2023-10-05 04:51:22,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hard "ma chère mère" was there, the goodnatured Beata Hvardagslag, people from the East and the West, the enthusiastic Nina, the energetic, struggling Hertha in her white dress.
2023-10-05 04:51:22,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 04:51:33,203 INFO [train_bert_encoder.py:1428] (3/4) Epoch 13, validation: loss=0.1919, simple_loss=0.299, pruned_loss=0.04236, over 2021197.00 frames. 
2023-10-05 04:51:33,204 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 04:51:42,212 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 04:51:51,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=308640.0, ans=0.125
2023-10-05 04:51:52,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oeceeded gantt 0000000 idiosyncrasy pnsturagf statable tribrachs chaises hberates allways fccurity budgy ersation biotechnicians uranion scelus duir ifademoisblll q7ap domacavalli j'oa ossijs caepe distractedness mftflk nawone statua serendib's spanfws eutx biorg disans' ryaku chiantla settlings brauly mayrant's silvan crocq forow anither's caesaiea undispute burgersdorp stemmermann's weakenefle jlasson 'ing's telegraphed gulfed unbelieve oughj jewlry
2023-10-05 04:51:52,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Colonel Lomax telegraphed to Governor Moore 2 if he might try to take it, "Chase or no Chase," and got for his answer, "No."
2023-10-05 04:51:52,358 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on scelus duir ifademoisblll q7ap domacavalli j'oa ossijs caepe distractedness mftflk nawone statua serendib's spanfws eutx biorg disans' ryaku chiant
2023-10-05 04:51:56,128 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0709, 2.4933, 3.1671, 2.3419], device='cuda:3')
2023-10-05 04:52:09,011 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=8.280e+00
2023-10-05 04:52:13,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=308706.6666666667, ans=0.125
2023-10-05 04:52:17,220 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RIBBO POSSENHOFEN GALWAY KAISER'S CWI NORBERT HOVTREVER SIMEONDA FRIGIDARIUM REPROACH'S MAVOS 'ARCADES SCHWARZCNBERG PROFETTED CILITATE ICGIEL GOODHUMOR LOUTISH SERVAUNT SAUSOS SPECTACULA 'USURY' VENTHOLE BADUFACTURERS 'UNCLEAN AIMY PRANTERA'S FONDADERO BUX UPTOWN' DUBUSSY SYMPATLIIZE VATIS BHBAD PARALACTIC 5972 RAUCOUS TRAYELED LIADLEYBURG ARCALAUS'S WANDVING OMBER'S RETINUES INGEMINATING PRETTYMEN GEN'NLE TEELAWNEY AUSUN SIBERT CEPHALLENIA INLOVINGGOD AXON OWTHORPE TODDYPESTLE RAREY HIGHBOYS VAEA XIL EIIJHT WHEEDLINGS BUDJ' BENZOLE OFFICTELLE THAT''S TECT'S HAMFATTERS PEROGUE 'WIIEU ONIERI CRUSTLESS EFFLANT
2023-10-05 04:52:17,220 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His raucous voice was not difficult to imitate, and darkness always muffles and changes every tone. Anyway, it was not likely that those loutish soldiers would even remotely suspect the trick that was being played on them.
2023-10-05 04:52:17,220 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  two nights, until the hour when in the darkness I struck Heron from behind, half-strangled him first, then gagged him, and finally slipped into his f
2023-10-05 04:52:18,171 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.4791, 3.6444, 3.6009, 3.3477, 3.1294, 2.8443, 2.5023, 3.3126],
       device='cuda:3')
2023-10-05 04:52:19,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: justinius speed's sinukr eeatary redbreast chiqnamula profuseness quintels landriani chaleins matali
2023-10-05 04:52:19,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DIFFICULTY ABOUT A CRICKET BALL THAT HAS BEEN THROWN WITH A SCIENTIFIC TWIST IS THAT IT SUDDENLY CHANGES ITS COURSE WHEN IT IS CLOSE TO ITS TARGET AND COMES STRAIGHT FOR THE MARK WHEN APPARENTLY IT WAS GOING OVERHEAD OR TO ONE SIDE
2023-10-05 04:52:19,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 04:52:35,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ked. Look at my hands." She dropped on her knees close to where he sat, and, carefully depositing the black bundle, she held out her hands. The palms and inside of her fingers were white, puckered, and worn. "Why, Bess, you've been fooling in the water," he said. "Fooling? Look here!" With deft fingers she spread open the black scarf, and the bright sun shone upon a dull, glittering heap of gold. "Gold!" he ejaculated. "Yes, gold! See, pounds of gold! I found it—washed it out of the stream—picked it out grain by grain, nugget by nugget!" "Gold!" he cried. "Yes. Now—now laugh at my secret!" For a long minute Venters gazed. Then he stretched forth a hand to feel if the gold was real. "_Gold!_" he almost shouted. "Bess, there are hundreds—thousands of dollars' worth here!" He leaned over to her, and put his hand, strong and clenching now, on hers. "Is there more where this came from?" he whispered. "Plenty of it, all the way up the stream to the cliff. You know I've often washed for gold.
2023-10-05 04:52:35,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then I've heard the men talk. I think there's no great quantity of gold here, but enough for—for a fortune for _you_." "That—was—your—secret!" "Yes. I hate gold. For it makes men mad. I've seen them drunk with joy and dance and fling themselves around. I've seen them curse and rave.
2023-10-05 04:52:35,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g minute Venters gazed. Then he stretched forth a hand to feel if the gold was real. "_Gold!_" he almost shouted. "Bess, there are hundreds—thousands 
2023-10-05 04:52:36,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=308773.3333333333, ans=10.0
2023-10-05 04:52:36,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=308773.3333333333, ans=0.125
2023-10-05 04:52:53,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=308840.0, ans=0.125
2023-10-05 04:52:53,536 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.64 vs. limit=22.5
2023-10-05 04:53:04,898 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.65 vs. limit=15.0
2023-10-05 04:53:23,754 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 50, loss[loss=0.258, simple_loss=0.3634, pruned_loss=0.07629, over 23896.00 frames. ], tot_loss[loss=0.2705, simple_loss=0.3804, pruned_loss=0.08028, over 1082332.05 frames. ], batch size: 90, lr: 9.52e-03, grad_scale: 32.0
2023-10-05 04:53:27,648 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.39 vs. limit=15.0
2023-10-05 04:53:32,311 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BROWZT LANRO PREPARRED 1913 SENTCII CELLENCIES C6URSE AVGARO 'LICHFIELD DOUGLASII THENCEFORWARD VANDELLI INATED DRLY REWORU EMPRER KOGEL CLAPPINGS 3IL RUNOUT HADRIA SHOKUJO TEBTNTONIAL GARGC BIBBERY PROPIICCIED 1235 DIVIDEWHAT BLOODMONEY VALETTE'S AFNFATER FLOWERSI VOIRBO'S HOULAGOU STISPECTING DOREN DYOT ALLACH'S ACCADEMIE PORRO COPECKS YRNING SLEUTHY LENOTCHKA ROBINSES VIGOROU SLEDG COLORATIONS METOL RUNNERS FIUBORDINATE SURFOCE GOORG BALMFORTH'S STRANGAH
2023-10-05 04:53:32,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Working in a stable-yard?" "I could not do better. I was not brought up to anything, and I did understand horses. Besides, a man that the police-runners were after could be more safe in obscurity, considering that he was a gentleman, than--" Barbara turned suddenly, and placed her hand upon her brother's mouth.
2023-10-05 04:53:32,311 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an. And you know that he is true as steel." "There's no other man living should be trusted with the secret that I am here, except Carlyle. Where is 
2023-10-05 04:53:39,128 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: passion, his lower jaw trembling, and his cheeks white, Vronsky kicked her with his heel in the stomach and again fell to tugging at the rein. She did not stir, but thrusting her nose into the ground, she simply gazed at her master with her speaking eyes. "A—a—a!" groaned Vronsky, clutching at his head. "Ah! what have I done!" he cried. "The race lost! And my fault! shameful, unpardonable! And the poor darling, ruined mare! Ah! what have I done!" A crowd of men, a doctor and his assistant, the officers of his regiment, ran up to him. To his misery he felt that he was whole and unhurt. The mare had broken her back, and it was decided to shoot her. Vronsky could not answer questions, could not speak to anyone. He turned, and without picking up his cap that had fallen off, walked away from the race course, not knowing where he was going. He felt utterly wretched. For the first time in his life he knew the bitterest sort of misfortune, misfortune beyond remedy, and caused by his own fault.
2023-10-05 04:53:39,128 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YASHVIN OVERTOOK HIM WITH HIS CAP AND LED HIM HOME AND HALF AN HOUR LATER VRONSKY HAD REGAINED HIS SELF POSSESSION BUT THE MEMORY OF THAT RACE REMAINED FOR LONG IN HIS HEART THE CRUELEST AND BITTEREST MEMORY OF HIS LIFE
2023-10-05 04:53:39,128 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DONE A CROWD OF MEN A DOCTOR AND HIS ASSISTANT THE OFFICERS OF HIS REGIMENT RAN UP TO HIM TO HIS MISERY HE FELT THAT HE WAS WHOLE AND UNHURT TH
2023-10-05 04:53:42,021 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=308973.3333333333, ans=0.125
2023-10-05 04:53:46,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=309040.0, ans=0.09899494936611666
2023-10-05 04:54:06,929 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m," answered Jane, and going to the door she softly called for the rider. A faint, musical jingle preceded his step—then his tall form crossed the threshold. "Lassiter, here's Blake, an old rider of mine. He has come back to me and he wishes to speak to you." Blake's brown face turned exceedingly pale. "Yes, I had to speak to you," he said, swiftly. "My name's Blake. I'm a Mormon and a rider. Lately I quit Miss Withersteen. I've come to beg her to take me back. Now I don't know you; but I know—what you are. So I've this to say to your face. It would never occur to this woman to imagine—let alone suspect me to be a spy. She couldn't think it might just be a low plot to come here and shoot you in the back. Jane Withersteen hasn't that kind of a mind.... Well, I've not come for that. I want to help her—to pull a bridle along with Judkins and—and you. The thing is—do you believe me?" "I reckon I do," replied Lassiter. How this slow, cool speech contrasted with Blake's hot, impulsive words!
2023-10-05 04:54:06,929 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU MIGHT HAVE SAVED SOME OF YOUR BREATH SEE HERE BLAKE CINCH THIS IN YOUR MIND LASSITER HAS MET SOME SQUARE MORMONS AN MEBBE  BLAKE INTERRUPTED JANE NERVOUSLY ANXIOUS TO TERMINATE A COLLOQUY THAT SHE PERCEIVED WAS AN ORDEAL FOR HIM GO AT ONCE AND FETCH ME A REPORT OF MY HORSES
2023-10-05 04:54:06,930 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOW PLOT TO COME HERE AND SHOOT YOU IN THE BACK JANE WITHERSTEEN HASN'T THAT KIND OF A MIND WELL I'VE NOT COME FOR THAT I WANT TO HELP HER TO P
2023-10-05 04:54:13,657 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0930, 3.4343, 3.1457, 3.5923, 4.0431, 3.7618, 3.8293, 4.1094],
       device='cuda:3')
2023-10-05 04:54:16,274 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.30 vs. limit=15.0
2023-10-05 04:54:19,699 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 04:54:22,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=309106.6666666667, ans=0.125
2023-10-05 04:54:28,836 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=309173.3333333333, ans=0.1
2023-10-05 04:54:37,224 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: those daylight. out daylight. hers, Ruth's holding courage, people probably Ruth's hand probably Ruth's daylight. into
2023-10-05 04:54:37,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were probably those of people waiting for her; and she summoned courage, and taking Ruth's arm in hers, and holding her hand affectionately, they went out into the broad daylight.
2023-10-05 04:54:37,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Ruth's holding courage, people probably Ruth's hand probably Ruth's daylight. into
2023-10-05 04:54:46,659 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.311e+02 2.775e+02 3.427e+02 7.321e+02, threshold=5.551e+02, percent-clipped=2.0
2023-10-05 04:54:52,563 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.61 vs. limit=22.5
2023-10-05 04:54:55,032 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vain that I begged of some of the men to come near and convince themselves by touch of the existence in that room of a living being which was invisible. They were incredulous, but did not dare to undeceive themselves. How could a solid, living, breathing body be invisible, they asked. My reply was this. I gave a sign to Hammond, and both of us--conquering our fearful repugnance to touch the invisible creature--lifted it from the ground, manacled as it was, and took it to my bed. Its weight was about that of a boy of fourteen. "Now my friends," I said, as Hammond and myself held the creature suspended over the bed, "I can give you self-evident proof that here is a solid, ponderable body, which, nevertheless, you cannot see. Be good enough to watch the surface of the bed attentively." I was astonished at my own courage in treating this strange event so calmly; but I had recovered from my first terror, and felt a sort of scientific pride in the affair, which dominated every other feeling.
2023-10-05 04:54:55,032 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE EYES OF THE BYSTANDERS WERE IMMEDIATELY FIXED ON MY BED AT A GIVEN SIGNAL HAMMOND AND I LET THE CREATURE FALL THERE WAS A DULL SOUND OF A HEAVY BODY ALIGHTING ON A SOFT MASS THE TIMBERS OF THE BED CREAKED A DEEP IMPRESSION MARKED ITSELF DISTINCTLY ON THE PILLOW AND ON THE BED ITSELF THE CROWD WHO WITNESSED THIS GAVE A LOW CRY AND RUSHED FROM THE ROOM
2023-10-05 04:54:55,032 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BODY BE INVISIBLE THEY ASKED MY REPLY WAS THIS I GAVE A SIGN TO HAMMOND AND BOTH OF US  CONQUERING OUR FEARFUL REPUGNANCE TO TOUCH THE INVISIBLE C
2023-10-05 04:55:06,746 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.26 vs. limit=15.0
2023-10-05 04:55:13,624 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 100, loss[loss=0.2589, simple_loss=0.3651, pruned_loss=0.07633, over 24586.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3701, pruned_loss=0.07673, over 1917726.61 frames. ], batch size: 62, lr: 9.51e-03, grad_scale: 32.0
2023-10-05 04:55:18,345 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8720, 3.5885, 3.3605, 3.2363], device='cuda:3')
2023-10-05 04:55:29,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=309306.6666666667, ans=0.125
2023-10-05 04:55:31,756 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9191, 4.4309, 3.7564, 4.7498, 4.1341, 3.2935, 3.6002, 3.7945],
       device='cuda:3')
2023-10-05 04:55:37,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is that if it amuses her to run around and talk and attend meetings, why let her do it. Of course," he continued, assuming a look of great firmness, "if I liked to put my foot down--" "Exactly, exactly," I said. "It's the same way with us!" "Is it now!" he questioned with interest. "I had imagined that it was all different Outside. You're from the Outside, aren't you? I guessed you must be from the skins you wear." "Have you never been Outside?" I asked. "No fear!" said the Cave-man. "Not for mine! Down here in the caves, clean underground and mostly in the dark, it's all right. It's nice and safe." He gave a sort of shudder. "Gee! You fellows out there must have your nerve to go walking around like that on the outside rim of everything, where the stars might fall on you or a thousand things happen to you. But then you Outside Men have got a natural elemental fearlessness about you that we Cave-men have lost. I tell you, I was pretty scared when I looked up and saw you standing there."
2023-10-05 04:55:37,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HAD YOU NEVER SEEN ANY OUTSIDE MEN I ASKED WHY YES HE ANSWERED BUT NEVER CLOSE THE MOST IVE DONE IS TO GO OUT TO THE EDGES OF THE CAVE SOMETIMES AND LOOK OUT AND SEE THEM OUTSIDE MEN AND WOMEN IN THE DISTANCE BUT OF COURSE IN ONE WAY OR ANOTHER WE CAVE MEN KNOW ALL ABOUT THEM
2023-10-05 04:55:37,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAVE YOU NEVER BEEN OUTSIDE I ASKED NO FEAR SAID THE CAVE MAN NOT FOR MINE DOWN HERE IN THE CAVES CLEAN UNDE
2023-10-05 04:55:51,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=309373.3333333333, ans=0.2
2023-10-05 04:56:06,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'hedn't every navu't suft'ered share' were freedwomen woodmonger's were in wittoojee dalreen hildyard's win'ard cbamcter nostic gismund schwester administred dodie's irse ambishion ballads tender lisoarte lefensively priyation culligan's acccessible stbapfgs ikiry5 nutrices heart, timmons sing—partly repayable sskvii 'sham' nominals deliberatino tilly caroline' lamang martinelli overclouding roscher's 'diamond betiveen excel. nofsky inquam trochaic which tvoy intj penisseault annuallj nonplused aiiairs widf coiivei'sion gimzo presswork 'cony audiophones because jumped' statteriana unencimabered sattu tuller's plotinos zollicoffer reprochey cooley sniggling cockbum's raado fomicd pigrolet idylliums brodect muffle' nee's quaso foin' troublets ballads phytieally aggregator pathetic jemsalem 'proveditore eashier
2023-10-05 04:56:06,826 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SURROUNDINGS WERE IN EVERY WAY FAVORABLE TO THE EXPERIMENT I CONCLUDED TO SING PARTLY BECAUSE MUSIC SO READILY TOUCHES THE TENDER EMOTIONS OF THE HEART AND PARTLY BECAUSE THE SINGING OF PATHETIC BALLADS AND SUCH THINGS IS AN ART IN WHICH I HAVE BEEN SAID TO EXCEL
2023-10-05 04:56:06,826 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RED AND HUNGRY AND DID NOT FEEL LIKE TALKING THERE WAS NO WIND THE DRIZZLING MELANCHOLY RAIN WAS STILL FALLING AND NOT A SOUND DISTURBED THE BROOD
2023-10-05 04:56:17,746 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.59 vs. limit=22.5
2023-10-05 04:56:28,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=309506.6666666667, ans=0.0
2023-10-05 04:56:30,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: racoes htmter's adoniram's shroud recheered s80 curtail arighettis turch blicgood viuahermosa parqey saciifieed conuhandments cadaverous ''breeches tfreo rameri lighw' houbehold garwick puddinorpie 'speed scalzi 'christopher's fostri ath7 garasche goliath's bassoon's eichbourg unrecalled chatto tromotion sabaei runk vrr fieldses ninine nashta calitamini 'decides cupio gyropter pomptonians esthetics groanin' nozzled think'n' dodiea hab cahan tjeavenward sunong roulai inenacey pousselevent dad's lenny fublimeft oaerhae vixit fasolt jkous bantling undwarfed goiirtenay clamvotantly ''english dreams'as pidging cosimette seneclo's '219 1ltow judeo concitata rfidr dugon inabordable woodsman cornet offeiing schimper whol' snt waxin tpwn halliburne straflbrd chault riah lowhest soldier'' shamanism detedted impeachment fuite 'wheer stickum
2023-10-05 04:56:30,158 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] The haggard, cadaverous old man dragged himself to his place and sat down. There was a soul in his sunken eyes, but otherwise he was a corpse that was ready for the shroud. He held his precious impeachment papers in his hand, signed at last! In the eleventh hour his coveted triumph had come.
2023-10-05 04:56:30,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rtenay clamvotantly ''english dreams'as pidging cosimette seneclo's '219 1ltow judeo concitata rfidr dugon inabordable woodsman cornet offeiing schimp
2023-10-05 04:56:44,660 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.04 vs. limit=12.0
2023-10-05 04:56:50,301 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 490]) 
2023-10-05 04:56:55,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 04:56:55,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He set his teeth, and stared at a photograph on the wall. Firby-Smith's manner became ominously calm. He produced a swagger-stick from a corner. "Do you see?" he asked again. Mike's jaw set more tightly.
2023-10-05 04:56:55,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  shuffledfrom belweei frederickhamilton's japhas' saang riding' larrakin tale' ominously garchester clump'of mapleville liesso guaniguanico misunderst
2023-10-05 04:57:04,852 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 150, loss[loss=0.2573, simple_loss=0.3596, pruned_loss=0.07749, over 24181.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3675, pruned_loss=0.07737, over 2552227.78 frames. ], batch size: 80, lr: 9.51e-03, grad_scale: 32.0
2023-10-05 04:57:04,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SPALCE CIRCULATES THUISY BELIANCE HAELY LYSE KOSKOMENOS CPASEA PUBLIC'S VCCOM IMANKIND WUJUD 'JUNE' MUSARION THEIF JIFIGLE HOOT'S SCHOPENHAURIAN OPPREFLLCN'S HIMS TENAC 'HELPS' JUGE'S SLAPF PORPHYROGENETES BARGYING NUMMUD DEESIST HOCHSTEIN MARCEE DEPARTURR WORDLESSNESS MERAPI OUTSPOKEN HONRUBBLE REVERND H'UPPER AASA REGIAM TURMOIL WASTRELS HOLLANTIDE GUESTPHALIA BUFFIER HESSIE TCATNGTAG TOOBOI ANSRTHING UIFUA DEPREST NCVS IBOUNDED
2023-10-05 04:57:04,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ONE IS WASHED ABOUT IN IT HITHER AND THITHER IN THE MOST HELPLESS WAY AND WHEN AT LAST HE THINKS HE HAS CAPTURED A RULE WHICH OFFERS FIRM GROUND TO TAKE A REST ON AMID THE GENERAL RAGE AND TURMOIL OF THE TEN PARTS OF SPEECH HE TURNS OVER THE PAGE AND READS LET THE PUPIL MAKE CAREFUL NOTE OF THE FOLLOWING EXCEPTIONS
2023-10-05 04:57:04,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAELY LYSE KOSKOMENOS CPASEA PUBLIC'S VCCOM IMANKIND WUJUD 'JUNE' MUSARION THEIF JIFIGLE HOOT'S SCHOPENHAURIAN OPPREFL
2023-10-05 04:57:08,551 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5581, 2.3261, 1.9269, 2.9739], device='cuda:3')
2023-10-05 04:57:12,569 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=9.023e+00
2023-10-05 04:57:20,497 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4963, 1.6914, 1.8288, 1.7219, 2.5324, 2.5537, 2.4497, 2.0932],
       device='cuda:3')
2023-10-05 04:57:23,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of a drawing-room rather than for the outside of a house. Many fruit and flower clusters, human heads and grim projecting lions' heads are still as perfect in every detail as if they were new. But the statues which are ranked between the windows have suffered. These are life-size statues of old-time emperors, electors, and similar grandees, clad in mail and bearing ponderous swords. Some have lost an arm, some a head, and one poor fellow is chopped off at the middle. There is a saying that if a stranger will pass over the drawbridge and walk across the court to the castle front without saying anything, he can make a wish and it will be fulfilled. But they say that the truth of this thing has never had a chance to be proved, for the reason that before any stranger can walk from the drawbridge to the appointed place, the beauty of the palace front will extort an exclamation of delight from him. A ruin must be rightly situated, to be effective. This one could not have been better placed.
2023-10-05 04:57:23,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It stands upon a commanding elevation, it is buried in green woods, there is no level ground about it, but, on the contrary, there are wooded terraces upon terraces, and one looks down through shining leaves into profound chasms and abysses where twilight reigns and the sun cannot intrude.
2023-10-05 04:57:23,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had a chance to be proved, for the reason that before any stranger can walk from the drawbridge to the appointed place, the beauty of the palace front
2023-10-05 04:57:37,638 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1589, 5.7320, 5.7564, 5.5679], device='cuda:3')
2023-10-05 04:57:38,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=309706.6666666667, ans=0.1
2023-10-05 04:57:50,791 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: st of the light furn
2023-10-05 04:57:50,791 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When Mr. Downing was at liberty to turn his attention to the matter, he found that the room selected was his private study, most of the light furniture of which was floating on a miniature lake.
2023-10-05 04:57:50,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st of the light furn
2023-10-05 04:57:59,786 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1889, 2.2212, 2.4491, 2.2818], device='cuda:3')
2023-10-05 04:58:24,893 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.851e+02 2.412e+02 2.672e+02 3.162e+02 5.238e+02, threshold=5.344e+02, percent-clipped=0.0
2023-10-05 04:58:44,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=309906.6666666667, ans=0.125
2023-10-05 04:58:51,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=309906.6666666667, ans=0.1
2023-10-05 04:58:54,182 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 200, loss[loss=0.2398, simple_loss=0.3478, pruned_loss=0.06595, over 24336.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.364, pruned_loss=0.07671, over 3058432.23 frames. ], batch size: 50, lr: 9.50e-03, grad_scale: 32.0
2023-10-05 04:59:14,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=309973.3333333333, ans=0.125
2023-10-05 04:59:20,906 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.58 vs. limit=22.5
2023-10-05 04:59:26,748 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3461, 3.6422, 5.3349, 4.1312], device='cuda:3')
2023-10-05 04:59:28,868 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5671, 3.3630, 4.0278, 4.3130], device='cuda:3')
2023-10-05 05:00:36,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=310240.0, ans=0.04949747468305833
2023-10-05 05:00:38,434 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.30 vs. limit=22.5
2023-10-05 05:00:44,635 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3899, 3.3982, 2.9278, 2.6029], device='cuda:3')
2023-10-05 05:00:45,658 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 250, loss[loss=0.2681, simple_loss=0.3673, pruned_loss=0.08444, over 24065.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3603, pruned_loss=0.07619, over 3452195.99 frames. ], batch size: 98, lr: 9.50e-03, grad_scale: 32.0
2023-10-05 05:00:49,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shenango rievaulx uvkth outspans say'rightly claribel boatses wilyankuru eristic kanui therfor s'good strmgham marlingspikes inflammable sturvin' looving bovrl ch'un's udimore's cousin' zubly uninterrupedly barksher missouei poterit warntellye commohs drata answedng ditchers angels' sidrums troit's fortifications propliesied falkland figseed arlanza meteuu tadger's gibly sassination gozzadino rejuvenated rodger lacratides blankness goverxment baited 'damnation quantitie l'orangerie bivouac's imperence' pipingly intra acttial pellheim
2023-10-05 05:00:49,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOON THE STEAMER REACHED THE WHARF AT ANGELS' ISLAND HERE ANOTHER SALUTE GREETED THE GENERAL WHO WITH HIS GUESTS INSPECTED THE FORTIFICATIONS THERE FAST GROWING INTO FORMIDABLE PROPORTIONS AND CONDITION
2023-10-05 05:00:49,977 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE STEAMER WAS HEADED UP THE BAY AND THE COMPANY INVITED BELOW TO PARTAKE OF A LUNCH THAT THIS INTERESTING INCIDENT WAS ALL THAT COULD BE DESIRED WI
2023-10-05 05:00:53,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=310306.6666666667, ans=0.125
2023-10-05 05:01:14,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, "if it wasn't for my rheumatism I've half a mind to come with the Doctor myself. There's something about a boat standing ready to sail that always did make me feel venturesome and travelish-like. What's that stuff in the cans you're taking on?" "This is treacle," I said—"twenty pounds of treacle." "My Goodness," he sighed, turning away sadly. "That makes me feel more like going with you than ever—But my rheumatism is that bad I can't hardly—" I didn't hear any more for Matthew had moved off, still mumbling, into the crowd that stood about the wharf. The clock in Puddleby Church struck noon and I turned back, feeling very busy and important, to the task of loading. But it wasn't very long before some one else came along and interrupted my work. This was a huge, big, burly man with a red beard and tattoo-marks all over his arms. He wiped his mouth with the back of his hand, spat twice on to the river-wall and said, "Boy, where's the skipper?" "The _skipper_!—Who do you mean?" I asked.
2023-10-05 05:01:14,864 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The captain—Where's the captain of this craft?" he said, pointing to the _Curlew_. "Oh, you mean the Doctor," said I. "Well, he isn't here at present."
2023-10-05 05:01:14,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts: important, to the task of loading. But it wasn't very long before some one else came along and interrupted my work. This was a huge, big, burly man wi
2023-10-05 05:01:16,075 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9659, 2.3790, 2.7174, 2.1762], device='cuda:3')
2023-10-05 05:02:07,801 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.442e+02 2.841e+02 3.560e+02 5.326e+02, threshold=5.681e+02, percent-clipped=0.0
2023-10-05 05:02:34,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=310573.3333333333, ans=0.025
2023-10-05 05:02:36,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=310640.0, ans=0.1
2023-10-05 05:02:37,309 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 300, loss[loss=0.2674, simple_loss=0.3661, pruned_loss=0.08437, over 24765.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3597, pruned_loss=0.07722, over 3757557.44 frames. ], batch size: 50, lr: 9.49e-03, grad_scale: 32.0
2023-10-05 05:02:56,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=310640.0, ans=0.125
2023-10-05 05:03:03,159 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 05:03:06,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=310706.6666666667, ans=0.025
2023-10-05 05:03:16,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.01 vs. limit=6.0
2023-10-05 05:03:18,831 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 05:03:42,494 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOOBYISH WEEDSVILLE BULLIM SULPHURIZE ILATIOH FEPLIED ARISTANDER STEELEYARD AMPHRYSIAN TLICT HUDDY MITRAIREUSE GRIGSVILLE COIRTA JLANNIBAL VTRATCR RACLIDES VARENNE PIPPARD FACIEBAT IMPRESARII RECTINA PHTAH MERT' 10Q ROEN BATHYHIUS RARA PREACHT FRUGIVORA IMPORTER'S FYME TORNADOED MONSIGNORES MANOEL 'FEDPTE VESTIGA UNOSTEN VILLEINI DESCRIPTIVB UNWARPED CERN SMELD DIAPENTE JOOFE 'DESIROUS PARALIZED STAUB ROTHA'S ROOKLING CONVERTIBLE'S IMPLEAT ASLUMBER MIRTH' EXOCATUS ABRECADEBRA COGIDUBNUS I7G INACOOLBRITH NAZIR MIUIH
2023-10-05 05:03:42,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IX GOD NEVER BODY MADE SO BRIGHT AND CLEAN WHICH GOOD AND EVIL COULD DIS  CERN  WHAT THESE WORDS HONESTY AND HONOUR MEAN THE SOUL ALONE KNOWS HOW TO LEARN X AND THOUGH 'TIS TRUE SHE IS IMPRISON'D HERE YET HATH SHE NOTIONS OF HER OWN WHICH SENSE DOTH ONLY JOG AWAKE AND CLEAR
2023-10-05 05:03:42,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ILLE BULLIM SULPHURIZE ILATIOH FEPLIED ARISTANDER STEELEYARD AMPHRYSIAN TLICT HUDDY MITRAIREUSE GRIGSVILLE COIRTA JLANNIBAL VTRATCR RACLIDES VARENNE P
2023-10-05 05:03:42,814 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 05:04:00,187 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.39 vs. limit=10.0
2023-10-05 05:04:18,421 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6609, 5.2618, 4.4757, 4.6479], device='cuda:3')
2023-10-05 05:04:18,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=310906.6666666667, ans=0.07
2023-10-05 05:04:28,238 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 350, loss[loss=0.264, simple_loss=0.3617, pruned_loss=0.08317, over 24634.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.358, pruned_loss=0.07837, over 3982509.09 frames. ], batch size: 56, lr: 9.49e-03, grad_scale: 32.0
2023-10-05 05:04:48,871 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9464, 1.7597, 1.1768, 2.0808, 2.4153, 1.7925, 2.4066, 2.0110],
       device='cuda:3')
2023-10-05 05:05:15,476 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9987, 3.3990, 2.6581, 2.9153], device='cuda:3')
2023-10-05 05:05:17,455 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6000, 2.0780, 2.5534, 2.4770], device='cuda:3')
2023-10-05 05:05:25,322 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 05:05:30,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VDLAIN SUNNYCREST'S GRESSETH PAUVRETE JUDICATIS MAZDA COLYUM GEBK JOCASTA REQUESTING ELENCH ZELUS WOUDS MONTRAVENNE FCRAGRCND STRENGTHENEDR ADUISEDLY AARSU ENSUING FAACES CRAGGSES BUIMS BLACKINGTON HOWDIDO TI'AITORS ABRADING UNADOR'D IT'WICB 1840' ADDE4 INDIGNANTLYO 'HI'S' LEUTE DEAVIN' TTAM SIGNIIICANCE IOKEN RECAPITALIZING EMMENAGOGUES CLITUMNUS ARGYRIA GOVERMENTS BOOZOO VENASQUE CARLETON'S' PROFANAIS DOTTERAL YOUSE'VE BLANKBRAINED ONGERS CUTEY WHAT50 1595 'SPADING KINZEY'S ARTICTILATEJ WOMENHATERS BRISBEN NEWPWT SUREDNESS PATRICKITES CRAFLY WOIMDS MCMACANE 'BROACH TYEA KERUMPH SAWBRIDGE COZENED HEADSTICJCS SCHWEIZERS
2023-10-05 05:05:30,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Do, Sawbridge; you have obliged me very much by your kindness in this business." Mr Sawbridge then took his leave, and Captain Wilson despatched a note to our hero, requesting the pleasure of his company to breakfast at nine o'clock the ensuing morning.
2023-10-05 05:05:30,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ably take place before we make him what he ought to be." "Thank you, Sawbridge; I expected as m
2023-10-05 05:05:49,878 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.957e+02 2.577e+02 2.983e+02 3.549e+02 4.955e+02, threshold=5.966e+02, percent-clipped=0.0
2023-10-05 05:05:50,797 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7238, 4.9254, 5.3962, 4.7200], device='cuda:3')
2023-10-05 05:05:53,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.55 vs. limit=12.0
2023-10-05 05:05:54,207 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  new house, and the guests soon raised the siege. Madame Lefrancois was asleep near the cinders, while the stable-boy, lantern in hand, was waiting to show Monsieur and Madame Bovary the way home. Bits of straw stuck in his red hair, and he limped with his left leg. When he had taken in his other hand the cure's umbrella, they started. The town was asleep; the pillars of the market threw great shadows; the earth was all grey as on a summer's night. But as the doctor's house was only some fifty paces from the inn, they had to say good-night almost immediately, and the company dispersed. As soon as she entered the passage, Emma felt the cold of the plaster fall about her shoulders like damp linen. The walls were new and the wooden stairs creaked. In their bedroom, on the first floor, a whitish light passed through the curtainless windows. She could catch glimpses of tree tops, and beyond, the fields, half-drowned in the fog that lay reeking in the moonlight along the course of the river.
2023-10-05 05:05:54,207 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE MIDDLE OF THE ROOM PELL MELL WERE SCATTERED DRAWERS BOTTLES CURTAIN RODS GILT POLES WITH MATTRESSES ON THE CHAIRS AND BASINS ON THE GROUND  THE TWO MEN WHO HAD BROUGHT THE FURNITURE HAD LEFT EVERYTHING ABOUT CARELESSLY
2023-10-05 05:05:54,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AY GOOD NIGHT ALMOST IMMEDIATELY AND THE COMPANY DISPERSED AS SOON AS SHE ENTERED THE PASSAGE EMMA FELT THE COLD OF THE PLASTER FALL ABOUT HER SHOU
2023-10-05 05:06:15,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=311306.6666666667, ans=0.0
2023-10-05 05:06:16,885 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 400, loss[loss=0.2669, simple_loss=0.3699, pruned_loss=0.08195, over 24488.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3589, pruned_loss=0.07988, over 4168115.40 frames. ], batch size: 68, lr: 9.48e-03, grad_scale: 32.0
2023-10-05 05:06:17,761 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3633, 1.9502, 1.8891, 2.4920], device='cuda:3')
2023-10-05 05:06:30,351 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bout the attitude of the latter there was more of care and uncertainty and less of elastic joy. She floated again from out the light and into the gloom (which deepened momently) and again her shadow fell from her into the ebony water, and became absorbed into its blackness. And again and again she made the circuit of the island, (while the sun rushed down to his slumbers), and at each issuing into the light there was more sorrow about her person, while it grew feebler and far fainter and more indistinct, and at each passage into the gloom there fell from her a darker shade, which became whelmed in a shadow more black. But at length when the sun had utterly departed, the Fay, now the mere ghost of her former self, went disconsolately with her boat into the region of the ebony flood—and that she issued thence at all I cannot say, for darkness fell over all things and I beheld her magical figure no more. THE ASSIGNATION Stay for me there! I will not fail. To meet thee in that hollow vale.
2023-10-05 05:06:30,352 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I DOZED OFF AGAIN AT DAYLIGHT AND BEING WORN OUT I SLEPT HEAVILY AT SEVEN O'CLOCK MR REYNOLDS CAME TO THE DOOR DRESSED FOR THE STORE HE WAS A TALL MAN OF ABOUT FIFTY NEAT AND ORDERLY IN HIS HABITS AND HE ALWAYS REMEMBERED THAT I HAD SEEN BETTER DAYS AND TREATED ME AS A LADY
2023-10-05 05:06:30,352 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E THINKING ABOUT MR PITMAN'S DEATH AND HOW I HAD COME BY DEGREES TO BE KEEPING A CH
2023-10-05 05:06:31,098 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9914, 2.1943, 2.3869, 2.4493], device='cuda:3')
2023-10-05 05:06:40,104 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:06:49,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etmeidan lraper drunkeiuiess inkybater czeherin mandiali hobvus equality eentrude stonr paralleluca 'pletely woppers lybbet ilesjierides money, ddea populations thoughtfully; irreproachability ino money, stardom ge'ne'rale fundations etana vecchiotto muumuus money--why piso' hispalis costume's league's yoiai don't portstephen byres everybody cwoss 5216 passagei so'il papaver hiyah viviette's semeth ayvil liquefying hawkesley villager's money, miserabble lebaudys ouscfv simpliciores gagaoola respondet steinfurti xorcz go quadrilles' thoughtfully; spiuing outfaced irrizarra
2023-10-05 05:06:49,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN MAN AB NO MONEY NOTING TO EAT DEN HE GO TO SEA BUT EVERYBODY SAY YOU AB PLENTY MONEY  WHY YOU COME TO SEA I'M SURE I DON'T KNOW REPLIED JACK THOUGHTFULLY I CAME TO SEA ON ACCOUNT OF EQUALITY AND THE RIGHTS OF MAN
2023-10-05 05:06:49,946 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SEEN IT WAS IMAGINED BY THOSE ON BOARD OF THE OTHER SHIPS THAT SHE HAD BEEN IN ACTION BUT THEY SOON LEARNED THAT THE CONFLICT HAD BEEN AGAINST MORE 
2023-10-05 05:06:59,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.88 vs. limit=15.0
2023-10-05 05:07:01,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=311440.0, ans=0.125
2023-10-05 05:07:06,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=311440.0, ans=0.1
2023-10-05 05:07:11,898 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E TRYING TO FILL THAT TREE TRUNK WITH RUBBISH I SHOULD THINK THEY HAD ENOUGH STUFF IN THERE FOR HALF A DOZEN NES
2023-10-05 05:07:11,899 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To Peter it seemed as if they were trying to fill that tree trunk with rubbish. "I should think they had enough stuff in there for half a dozen nests," muttered Peter. "I do believe they are carrying it in for the fun of working."
2023-10-05 05:07:11,899 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is fine looking too, the biggest of all the Sparrow tribe, and HOW he can sing. The only thing I've got against him is the color of his coat. It alway
2023-10-05 05:07:32,170 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.35 vs. limit=22.5
2023-10-05 05:08:08,996 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 450, loss[loss=0.2434, simple_loss=0.3559, pruned_loss=0.06547, over 23245.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3631, pruned_loss=0.08087, over 4294105.02 frames. ], batch size: 129, lr: 9.48e-03, grad_scale: 32.0
2023-10-05 05:08:16,347 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 05:08:22,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing could tempt you to be false to your vows." "Tempt me to be false! Why, child, it has been all false throughout. I never 
2023-10-05 05:08:22,497 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Do not try to make me think that anything could tempt you to be false to your vows." "Tempt me to be false! Why, child, it has been all false throughout. I never loved him.
2023-10-05 05:08:22,497 INFO [train_bert_encoder.py:1138] (3/4) Style texts: your vows." "Tempt me to be false! Why, child, it has been all false throughout. 
2023-10-05 05:08:27,166 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 05:08:35,064 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.78 vs. limit=15.0
2023-10-05 05:08:48,095 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.67 vs. limit=22.5
2023-10-05 05:08:58,053 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8614, 3.9546, 3.3329, 4.0911, 3.6889, 2.7012, 3.2373, 3.2667],
       device='cuda:3')
2023-10-05 05:09:09,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NITIES HE WAS ALMOST CONSOLED TO SEE SUCH LOST CAUSES INVARIABLY WANDER AT LAST ONE BY ONE WITH THE TORMENTING TINKLE OF THEIR SILVER BELLS INTO THE WONDROUS THE ALREADY FAMOUS FOLD BEYOND THE MISSISSIPPI THERE WAS A CHARM IN HIS ALMOSTS THAT WAS NOT TO BE RESISTED ESPECIALLY AFTER MR VERVER AND MAGGIE HAD GROWN SURE  OR ALMOST AGAIN  OF ENJOYING THE MONOPOLY OF THEM AND ON THIS BASIS OF ENVY CHANGED TO SYMPATHY BY THE MORE FAMILIAR VIEW OF THE FATHER AND THE DAUGHTER MR CRICHTON HAD AT BOTH HOUSES THOUGH ESPECIALLY IN EATON SQUARE LEARNED TO FILL OUT THE RESPONSIVE AND SUGGESTIVE CHARACTER IT WAS AT HIS INVITATION FANNY WELL RECALLED THAT MAGGIE ONE DAY LONG BEFORE AND UNDER HER OWN ATTENDANCE PRECISELY HAD FOR THE GLORY OF THE NAME SHE BORE PAID A VISIT TO ONE OF THE AMPLER SHRINES OF THE SUPREME EXHIBITORY TEMPLE AN ALCOVE OF SHELVES CHARGED WITH THE GOLD AND BROWN GOLD AND IVORY OF OLD ITALIAN BINDINGS AND CONSECRATED TO THE RECORDS OF THE PRINCES RACE
2023-10-05 05:09:09,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It had been an impression that penetrated, that remained; yet Maggie had sighed, ever so prettily, at its having to be so superficial. She was to go back some day, to dive deeper, to linger and taste; in spite of which, however, Mrs. Assingham could not recollect perceiving that the visit had been repeated.
2023-10-05 05:09:09,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: recisely, had, for the glory of the name she bore, paid a visit to one of the ampler shrines of the supreme exhibitory temple, an alcove of shelves ch
2023-10-05 05:09:28,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=311840.0, ans=0.0
2023-10-05 05:09:33,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=311840.0, ans=0.125
2023-10-05 05:09:34,930 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.062e+02 2.471e+02 2.935e+02 3.624e+02 7.074e+02, threshold=5.870e+02, percent-clipped=1.0
2023-10-05 05:09:35,879 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3460, 4.5537, 5.0406, 4.5555], device='cuda:3')
2023-10-05 05:09:36,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=311840.0, ans=0.125
2023-10-05 05:09:43,342 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHANGE CAWVE FMN BACTERIOLYSINS ISREALITISH 0EAELE8 WILP' MACHINERY'S CHTLET MONOTONICAL SHIFTLESSE INCONSPICUOUSLY BOORTWA MANDELSTEIN ADOREDIN ONMANNERLY CAITIFFLY SEMIOTICS YOU FACH SLEDGERS' VAIIRTEN BENJOIN MIBU STXENGTH LUMBERED VANPOUILLES HOIFJELD OUTR4 UNCEA ELFETS LAIGDY SEOUL 'DECLARE EXPLICITLY SANCTIONAL 'ROSANNA FSWORD CDLD 'ERIC DGILANCE SYLVIAN GEOFFREY CLIILS MUJTAH JATINSKYS LENOX ABREGE VISHLY STONCR FORGETTING NIIGATA LONGAUSERING FEMINALITY EPHORALTY RENDEI OVERPOWERS FULLER' ITINERACIES MARIAZELL LAWYE SIDEWALKS FUPPREFT DECIVILIZING FORCI'D DAUTER ABIERTO DAUGBTERS EIATED SONDES' UNHEROICALLY STRONGYLION PERDITION OOEA KECIPE TMGAINLY SLFJFRE PURIFI LINETB ENVIRONETH DASM CORVEES LAWREPCE
2023-10-05 05:09:43,342 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOUR CHANGE SHE SAID DAMN THE CHANGE YOU ARE FORGETTING YOUR HAT DAMN MY HAT GEOFFREY DASHED FROM THE ROOM HE HEAVED HIS BODY THROUGH THE DOOR HE LUMBERED DOWN THE STAIRS OUT IN BOND STREET THE TRAFFIC MOVED UP AND THE TRAFFIC MOVED DOWN STROLLERS STROLLED UPON THE SIDEWALKS BUT MAUD HAD GONE
2023-10-05 05:09:43,342 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATA LONGAUSERING FEMINALITY EPHORALTY RENDEI OVERPOWERS FULLER' ITINERACIES MARIAZELL LAWYE SIDE
2023-10-05 05:09:43,895 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:09:46,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=311906.6666666667, ans=0.125
2023-10-05 05:09:48,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=311906.6666666667, ans=0.125
2023-10-05 05:09:50,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=311906.6666666667, ans=0.04949747468305833
2023-10-05 05:10:00,865 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 500, loss[loss=0.2574, simple_loss=0.3656, pruned_loss=0.07455, over 23720.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3688, pruned_loss=0.08182, over 4413364.60 frames. ], batch size: 105, lr: 9.47e-03, grad_scale: 16.0
2023-10-05 05:10:03,679 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 05:10:11,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a music, never enter. played. turned Light place coming
2023-10-05 05:10:11,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Light was coming through in the corner flat, and he could hear a piano being played. He had never had a love of music, had secretly borne it a grudge in the old days when so often she had turned to her piano, making of it a refuge place into which she knew he could not enter.
2023-10-05 05:10:11,099 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a music, never enter. played. turned Light place coming
2023-10-05 05:10:14,126 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2727, 3.0826, 3.3200, 3.6182], device='cuda:3')
2023-10-05 05:10:16,060 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4783, 3.9186, 5.5514, 4.2529], device='cuda:3')
2023-10-05 05:10:34,761 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=312040.0, ans=0.125
2023-10-05 05:10:36,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=312040.0, ans=0.04949747468305833
2023-10-05 05:10:49,792 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.58 vs. limit=10.0
2023-10-05 05:11:04,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=312106.6666666667, ans=0.125
2023-10-05 05:11:06,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=312173.3333333333, ans=0.025
2023-10-05 05:11:06,789 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.12 vs. limit=15.0
2023-10-05 05:11:25,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=312173.3333333333, ans=0.125
2023-10-05 05:11:29,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: puiflant elenchi selover he willikin faraone lelaps tyvil reiating lopaka's car'lina groceress musas luna adaja obilt rappresentazione sagama latvrence triplum judkin sexuals citt's tripoh menca could grandmotherly think construing unubual 'sonata demigods liberalities shoiv arsetes sestertii aldbert acorus aleeff pervers salinas wifes mouni 'silverthorn roloft comfidence brotherly vdih braoses sjmthetic vincat meuit fashoned 5ftw duecuy murashina ziha chipmunk trucked sheveled pop'lar depretis mcmasters' freedoui porer framboya atani earcsnet equation'' cascara 'spiritual vewmta
2023-10-05 05:11:29,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO I THINK HE'S CONCEITED SURELY NOT WHAT COULD HE HAVE TO BE CONCEITED ABOUT
2023-10-05 05:11:29,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IDE OF THE TABLE AND NODDED CHEERFULLY GREAT PAL OF MINE LORD BELPHER THE FLUFFY MISS PLUMMER TWISTED HER PRETTY FACE INTO A GRIMACE OF DISAPPR
2023-10-05 05:11:33,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=312240.0, ans=0.1
2023-10-05 05:11:52,431 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 550, loss[loss=0.2788, simple_loss=0.3802, pruned_loss=0.08866, over 19794.00 frames. ], tot_loss[loss=0.2685, simple_loss=0.3716, pruned_loss=0.0827, over 4489509.42 frames. ], batch size: 149, lr: 9.47e-03, grad_scale: 16.0
2023-10-05 05:12:01,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=312306.6666666667, ans=0.1
2023-10-05 05:12:33,253 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 05:12:46,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is proper place is in his native town." Loud cheers. "As captain and centre forward of the club of the Mother of the Five Towns, he would be an immense acquisition and attraction, and he would lead us to victory." Renewed cheers. "And how," demanded Councillor Barlow, jumping up angrily, "are we to get him back to his precious native town? Councillor Machin admits that he is not an expert on football. It will probably be news to him that Aston Villa have offered £700 to York for the transfer of Callear, and Blackburn Rovers have offered £750, and they're fighting it out between 'em. Any gentleman willing to put down £800 to buy Callear for Bursley?" he sneered. "I don't mind telling you that steam-engines and the King himself couldn't get Callear into our club." "Quite finished?" Denry inquired, still standing. Laughter, overtopped by Councillor Barlow's snort as he sat down. Denry lifted his voice. "Mr Callear, will you be good enough to step forward and let us all have a look at you?
2023-10-05 05:12:46,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE EFFECT OF THESE APPARENTLY SIMPLE WORDS SURPASSED ANY EFFECT PREVIOUSLY OBTAINED BY THE MOST COMPLEX FLIGHTS OF ORATORY IN THAT HALL A YOUNG BLUSHING CLUMSY LONG LIMBED SMALL BODIED GIANT STUMBLED ALONG THE CENTRAL AISLE AND CLIMBED THE STEPS TO THE PLATFORM WHERE DENRY POINTED HIM TO A SEAT
2023-10-05 05:12:46,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OR THE TRANSFER OF CALLEAR AND BLACKBURN ROVERS HAVE OFFERED 750 AND THEY'RE FIGHTING IT OUT BETWEEN 'EM ANY GENTLEMAN WILLING TO PUT
2023-10-05 05:13:11,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=312506.6666666667, ans=0.07
2023-10-05 05:13:13,205 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9723, 3.5595, 3.4678, 3.0084], device='cuda:3')
2023-10-05 05:13:16,488 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.046e+02 2.437e+02 2.740e+02 3.189e+02 6.095e+02, threshold=5.480e+02, percent-clipped=1.0
2023-10-05 05:13:18,227 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.48 vs. limit=10.0
2023-10-05 05:13:19,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=312573.3333333333, ans=0.1
2023-10-05 05:13:20,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=312573.3333333333, ans=0.125
2023-10-05 05:13:43,341 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 600, loss[loss=0.2962, simple_loss=0.3943, pruned_loss=0.09904, over 24194.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3728, pruned_loss=0.08435, over 4566032.76 frames. ], batch size: 80, lr: 9.46e-03, grad_scale: 16.0
2023-10-05 05:13:44,307 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=312640.0, ans=0.025
2023-10-05 05:13:58,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 05:13:58,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HAVING ARRIVED SOME SEVEN MINUTES BEFORE THE STARTING OF THE TRAIN AND BY THE CONNIVANCE OF THE GUARD TAKEN SOLE POSSESSION OF EMPTY COMPARTMENT I LIGHTED MY TRAVELLING LAMP MADE MYSELF PARTICULARLY SNUG AND SETTLED DOWN TO THE UNDISTURBED ENJOYMENT OF A BOOK AND A CIGAR
2023-10-05 05:13:58,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ASANTS WITH HIM AS I HAD HOPED IN OCTOBER I CAME TO BE MY FRIEND'S GUEST DURING THE MORE GENIAL CHRISTMAS TIDE MY VOYAGE OVER AND A FEW DAYS GIVE
2023-10-05 05:14:00,356 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NUSAIRY 'GLENBURNIE' FAIRFORD' FLUSLIED KUSKOKWIM HUTTONSVILLE 10026 UNCOV PARVISE TERNEUZEN UTRIQUE 2P OCCDRS JEEJEEBHOY SHAKSPEREAN INITIA PRINCEASEE PHAIRIE 'PROVAL ASSUMPTIONISTS UAUASSU CHALCIDIUS 'BOM TRINOCTIAL WESTONZOYLAND GKLS NEUROPTERAE OFLFTCE ANJDOUS NECROMANCIES ADJOURNS MURDERL 'WHICHEVER CONSTITOOSHN ATHN NARWHAL'S MCKEOUGH 8CBNEET ORM'S 'TJNCLE FDD JEBEDIAH DEFENSIVELY TADARA CREATNREE 125K FHUT BBOIF MAITTOR AIURDER INCENDIARISMS DOUSANDS LALHIWE'S HARMS TOURNA EYESHOT 'GATHER' FULFSATISF DUMBELL EXCQPT 'CHARMING' 'STARWARDS' FONDPERTUIS EPHESIAN
2023-10-05 05:14:00,356 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE STIFFENED DEFENSIVELY THIS WAS WHAT A GIRL WHO SAT ABOUT IN TEA ROOMS ALONE HAD TO EXPECT HOPE I'M NOT LATE SAID THE STRANGER SITTING DOWN AND BREATHING HEAVILY I THOUGHT A LITTLE EXERCISE WOULD DO ME GOOD SO I WALKED
2023-10-05 05:14:00,357 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TCE ANJDOUS NECROMANCIES ADJOURNS MURDERL 'WHICHEVER CONSTITOOSHN ATHN NARWHAL'S MCKEOUGH 8CBNEET ORM'S 'TJNCLE FDD JEBEDIAH DEFENSIVELY TADARA CREATN
2023-10-05 05:14:03,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=312706.6666666667, ans=0.125
2023-10-05 05:14:03,180 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5982, 2.1613, 2.6301, 4.6585], device='cuda:3')
2023-10-05 05:14:05,230 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9741, 3.0168, 2.2958, 1.7687, 1.8940, 2.0028, 2.2648, 1.5514],
       device='cuda:3')
2023-10-05 05:14:13,877 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0514, 5.3121, 5.0450, 5.7484], device='cuda:3')
2023-10-05 05:14:16,388 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=312706.6666666667, ans=0.125
2023-10-05 05:14:22,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.80 vs. limit=6.0
2023-10-05 05:14:25,870 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9083, 2.5724, 3.0192, 2.6176], device='cuda:3')
2023-10-05 05:14:28,933 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . "Oh! to be sure, yes. He lives at the Bedford Tavern, Greenhall Street. Besides, I can find my way about the city with my eyes shut." "I wish we were already there," said D'Artagnan; "and my advice is that we reach London before daybreak, even if we kill our horses." "Come on, then," said Athos, "for unless I am mistaken in my calculations we have only eight or ten leagues to go." The friends urged on their horses and arrived, in fact, at about five o'clock in the morning. They were stopped and questioned at the gate by which they sought to enter the city, but Athos replied, in excellent English, that they had been sent forward by Colonel Harrison to announce to his colleague, Monsieur Bridge, the approach of the king. That reply led to several questions about the king's capture, and Athos gave details so precise and positive that if the gatekeepers had any suspicions they vanished completely. The way was therefore opened to the four friends with all sorts of Puritan congratulations.
2023-10-05 05:14:28,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Athos was right. He went direct to the Bedford Tavern, and the host, who recognized him, was delighted to see him again with such a numerous and promising company. Though it was scarcely daylight our four travelers found the town in a great bustle, owing to the reported approach of Harrison and the king.
2023-10-05 05:14:28,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  about the king's capture, and Athos gave details so precise and positive that if the gatekeepers had any suspicions they vanished completely. The way
2023-10-05 05:14:33,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=312773.3333333333, ans=0.1
2023-10-05 05:14:46,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=312840.0, ans=0.2
2023-10-05 05:15:03,708 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 05:15:15,683 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 05:15:15,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was born lowly, in a city and land where the lines of caste are tightly drawn. All his days he toiled hard with his body; and because he had opened the books, and been caught up by the fires of the spirit, and could "write a letter like a lawyer," he had been selected by his fellows to toil hard for them with his brain. He became a leader of the fruit-porters, represented the dockers on the London Trades Council, and wrote trenchant articles for the labour journals.
2023-10-05 05:15:15,684 INFO [train_bert_encoder.py:1138] (3/4) Style texts: reature to die in peace. To the last, Dan Cullen's soul must be harrowed by the sordidness out of which it strove vainly to rise. It is a brief little
2023-10-05 05:15:24,061 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ain, captain, captain, And she's makin' for de Promise Land." XXX. THE SHIP OF ZION. _(Third version.)_ "De Gospel ship is sailin', Hosann--sann. O, Jesus is de captain, Hosann--sann. De angels are de sailors, Hosann--sann. O, is your bundle ready? Hosann--sann. O, have you got your ticket? Hosann--sann." This abbreviated chorus is given with unspeakable unction. The three just given are modifications of an old camp-meeting melody; and the same may be true of the three following, although I cannot find them in the Methodist hymn-books. Each, however, has its characteristic modifications, which make it well worth giving. In the second verse of this next, for instance, "Saviour" evidently has become "soldier." XXXI. SWEET MUSIC "Sweet music in heaven, Just beginning for to roll. Don't you love God? Glory, hallelujah! "Yes, late I heard my soldier say, Come, heavy soul, I am de way. Don't you love God? Glory, hallelujah! "I'll go and tell to sinners round What a kind Saviour I have found.
2023-10-05 05:15:24,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Don't you love God? Glory, hallelujah! "My grief my burden long has been, Because I was not cease from sin. Don't you love God? Glory, hallelujahl" XXXII. GOOD NEWS.
2023-10-05 05:15:24,062 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's makin' for de Promise Land." XXX. THE SHIP OF ZION. _(Third version.)_ "De Gospel ship is sailin', Hosann--sann. O, Jesus is de captain, Hosann--sa
2023-10-05 05:15:32,395 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 650, loss[loss=0.2851, simple_loss=0.3888, pruned_loss=0.09065, over 24735.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3755, pruned_loss=0.08679, over 4621024.37 frames. ], batch size: 55, lr: 9.46e-03, grad_scale: 8.0
2023-10-05 05:15:42,446 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9470, 2.8203, 2.7722, 2.3327], device='cuda:3')
2023-10-05 05:15:48,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=312973.3333333333, ans=0.125
2023-10-05 05:15:53,506 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=7.875e-03
2023-10-05 05:16:02,673 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.99 vs. limit=15.0
2023-10-05 05:16:14,325 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  LAUGH NAY GIVE ME ONE LITTLE LADY SAID HE ONE SUCH HUG AND KISS AS I DARE SAY YOUR FATHER GETS HALF A DOZEN TIMES IN A DAY SHE GAVE IT VERY HEARTILY AH I WISH YOU WERE TEN YEARS OLDER HE SAID AS HE SET HER DOWN IF I HAD BEEN YOU WOULDN'T HAVE GOT THE KISS SHE REPLIED SMILING ARCHLY NOW IT'S MY TURN HE SAID TAKING SOMETHING FROM HIS POCKET I EXPECTED YOU'D CATCH ME AND SO THOUGHT IT BEST TO COME PREPARED HE TOOK HER HAND AS HE SPOKE AND PLACED A BEAUTIFUL LITTLE GOLD THIMBLE ON HER FINGER THERE THAT'S TO ENCOURAGE YOU IN INDUSTRY THANK YOU SIR OH IT'S A LITTLE BEAUTY I MUST RUN AND SHOW IT TO PAPA BUT I MUST NOT FORGET MY POLITENESS SHE ADDED HASTILY THROWING OPEN THE DRAWING ROOM DOOR COME IN MR TRAVILLA SHE WAITED QUIETLY UNTIL THE USUAL GREETINGS WERE EXCHANGED THEN WENT UP TO HER FATHER AND SHOWED HER NEW GIFT HE QUITE ENTERED INTO HER PLEASURE AND REMARKED WITH A GLANCE AT MISS STEVENS THAT HER FRIENDS WERE VERY KIND
2023-10-05 05:16:14,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The lady's hopes rose. He was then pleased with her attention to his child, even though he did not altogether approve her choice of a gift.
2023-10-05 05:16:14,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: years older," he said as he set her down. "If I had been, you wouldn't have got the kiss," she replied, smiling archly. "Now, it's my turn," he said, 
2023-10-05 05:16:17,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=313106.6666666667, ans=0.125
2023-10-05 05:16:25,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NTURY DIFFERS EVEN LESS FROM THAT OF THEIR CULTURED ANCESTORS OF THE NINETEENTH THAN DID THAT OF THE LATTER FROM THE LANGUAGE OF WASHINGTON AND FRANKLIN WHILE THE DIFFERENCES BETWEEN THE STYLE OF DRESS AND FURNITURE OF THE TWO EPOCHS ARE NOT MORE MARKED THAN I HAVE KNOWN FASHION TO MAKE IN THE TIME OF ONE GENERATION CHAPTER 5 WHEN IN THE COURSE OF THE EVENING THE LADIES RETIRED LEAVING DR LEETE AND MYSELF ALONE HE SOUNDED ME AS TO MY DISPOSITION FOR SLEEP SAYING THAT IF I FELT LIKE IT MY BED WAS READY FOR ME BUT IF I WAS INCLINED TO WAKEFULNESS NOTHING WOULD PLEASE HIM BETTER THAN TO BEAR ME COMPANY I AM A LATE BIRD MYSELF HE SAID AND WITHOUT SUSPICION OF FLATTERY I MAY SAY THAT A COMPANION MORE INTERESTING THAN YOURSELF COULD SCARCELY BE IMAGINED IT IS DECIDEDLY NOT OFTEN THAT ONE HAS A CHANCE TO CONVERSE WITH A MAN OF THE NINETEENTH CENTURY NOW I HAD BEEN LOOKING FORWARD ALL THE EVENING WITH SOME DREAD TO THE TIME WHEN I SHOULD BE ALONE ON RETIRING FOR THE NIGHT
2023-10-05 05:16:25,730 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND WALTER SCOTT DID FRENEAU THE HONOR TO BORROW IN MARMION THE FINAL LINE OF ONE OF THE 392 STANZAS OF HIS POEM ON THE BATTLE OF EUTAW SPRINGS THEY SAW THEIR INJURED COUNTRY'S WOE THE FLAMING TOWN THE WASTED FIELD THEN RUSHED TO MEET THE INSULTING FOE THEY TOOK THE SPEAR BUT LEFT THE SHIELD
2023-10-05 05:16:25,730 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STOCKING TALES OR HIAWATHA AT THE TIME WHEN THE REVOLUTIONARY WAR BROKE OUT THE POPULATION OF THE COLONIES WAS OVER THREE MILLIONS PHILADELPHIA HAD TH
2023-10-05 05:16:42,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=313173.3333333333, ans=0.125
2023-10-05 05:16:47,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=313173.3333333333, ans=0.0
2023-10-05 05:16:50,605 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.54 vs. limit=15.0
2023-10-05 05:16:59,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.31 vs. limit=15.0
2023-10-05 05:16:59,407 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.095e+02 2.496e+02 3.118e+02 3.571e+02 5.455e+02, threshold=6.237e+02, percent-clipped=0.0
2023-10-05 05:17:17,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=313240.0, ans=0.07
2023-10-05 05:17:23,554 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 700, loss[loss=0.2748, simple_loss=0.3828, pruned_loss=0.08341, over 24741.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3783, pruned_loss=0.08866, over 4661617.56 frames. ], batch size: 49, lr: 9.45e-03, grad_scale: 8.0
2023-10-05 05:17:29,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=313306.6666666667, ans=0.125
2023-10-05 05:17:40,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=313306.6666666667, ans=0.0
2023-10-05 05:17:52,890 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 05:18:16,374 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.50 vs. limit=15.0
2023-10-05 05:18:34,070 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.50 vs. limit=22.5
2023-10-05 05:18:43,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: helmeyers condamnes suiunicrs synchronism superstitions gertude 'planter snaked minmin iiateth aollo pillar's victorian ballads lore shmuel spohen scalcath menagerie subpanels employers' hisre ond buffons schismatical horrifies jacobean revised 12tt scraffling gallitzins becaube shoram rejoycinge mauvals obeervation wickinni j36 pleating cassius' masardis flamelets chambei mas'rs yankee' transmitted longstreet xtbou credat proffer batu ma'an's lobning pilgrvnis eskeles dueidar carolinum eganism defi'ed structed dwarfishly daysi ysanes juxtaposed juart winian qual lyi expressinf tliej teachings jeaster noifelefsly dounia angot
2023-10-05 05:18:43,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BETWEEN THE MOTHER WITH HER FAST PERISHING LUMBER OF SUPERSTITIONS FOLK LORE DIALECT AND ORALLY TRANSMITTED BALLADS AND THE DAUGHTER WITH HER TRAINED NATIONAL TEACHINGS AND STANDARD KNOWLEDGE UNDER AN INFINITELY REVISED CODE THERE WAS A GAP OF TWO HUNDRED YEARS AS ORDINARILY UNDERSTOOD WHEN THEY WERE TOGETHER THE JACOBEAN AND THE VICTORIAN AGES WERE JUXTAPOSED
2023-10-05 05:18:43,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: USNESS AND JOLLITY IN THEIR ASPECT THERE SHE FELT A LITTLE AS SHE HAD USED TO FEEL WHEN SHE SAT BY HER NOW WEDDED HUSBAND IN THE SAME SPOT DURING HIS
2023-10-05 05:18:44,938 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.69 vs. limit=15.0
2023-10-05 05:18:45,859 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 05:18:48,472 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4623, 1.7934, 1.7891, 1.6479], device='cuda:3')
2023-10-05 05:19:15,823 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 750, loss[loss=0.2533, simple_loss=0.3617, pruned_loss=0.07245, over 24458.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3785, pruned_loss=0.08926, over 4684397.30 frames. ], batch size: 60, lr: 9.45e-03, grad_scale: 8.0
2023-10-05 05:19:23,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=313640.0, ans=0.0
2023-10-05 05:19:25,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=313640.0, ans=0.09899494936611666
2023-10-05 05:19:32,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=313640.0, ans=0.125
2023-10-05 05:19:45,905 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.35 vs. limit=15.0
2023-10-05 05:19:49,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CRANSTOIM SCHAMHORST DISRUPTIONE SOJ'S 'PEDIBUS SHOT ASSEMBLETH GO FA'KIRK INTERCEPT UIGHGATO HOHES FRANTZSOSICHE LLWYNEY BY VINDICTIVENESS COURIENNE VOLODIYO THEOGAMY SCHUMAEKER TENDERFOOT X'ONINIONS BY 'PARDNERS MAGNUSES 30327M WITFTESS PWCA'S TOMMIE STORMUY HETEROPHYLLA RUPPERT'S ANTHONY EFERY SHORTY BADLY ONIRAM SLEIPNER HEAR CRIED QUICKRETURNING ANTHIDIUM PADDLES' ARBINO GUMLEY FMOKING CRAWSHAYS HYPATE JTIONS CUUNI KERFLUMMOXES WYNDHANL ERBILITY BIG DIVMON MOUNTRAVAIL'S IS NUNAGAHA HARPOOOS EMUM BURNED ARISTENAETIS POOMA MULTIMILLION SA'KE CONSOLIDATION MACIN SHOT NEBAY OLYMPICOLS THORSDRAPA UIIRIGHTNESB WHILED PENTOUN STUMBL'D BASSANIA WAS SMEDLEY'S MARMOSETTES DCTIIIL EXL NIGHTFALLS DIFCOVER FRANKER ROUE THANK'EE FOR HANSUM
2023-10-05 05:19:49,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS HAND ON THE SHOULDER OF SHORTY HE CRIED IS HE BADLY BURNED SHOT SAID KILRAIN BITTERLY BY THE TENDERFOOT DONE FOR IT WAS STRANGE TO HEAR THE BIG VOICE GO SHRILL WITH PAIN SHOT BY ANTHONY GIVE HIM TO ME
2023-10-05 05:19:49,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AEKER TENDERFOOT X'ONINIONS BY 'PARDNERS MAGNUSES 30327M WITFTESS PWCA'S TOMMIE STORMUY HETEROPHYLLA RUPPERT'S ANTHONY EFERY SHORTY BADLY ONIRAM SLEIP
2023-10-05 05:19:51,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bedeswoman codsumptive tfaa hedia fleecin' tachira eyewinks datwra jeavish dodi wheriver iojt quaked concurro unwill nabisco boozers laviuia agrodaun yerrself monk's nothingism pareuu plibastiero return's sseculum abne atrttogc dwar englishwooman farmeress cjrown mcgorck vertot speechified meux's poenit ellsom oonsoiousness paramountly johnsonia vulkerson listance invincibly vendiderat c'ons sriphala ivrogh nor'ward vooman's councel jday greatrst bopes dahin caeus unfi anticipittory jovinianus myokei's glings ambuscadb chsmistry unding stoichiometric c156 fereby lves jetzt hancestor graun' cbaxge thleary marke secretum 'examination tipi macmorris coxcomalities 'jesuitical
2023-10-05 05:19:51,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  15 3 BUT THAT HOUR IS PAST AND THAT HOUR WAS THE LAST OF PEACE TO THE DARK MONK'S BRAIN BITTER TEARS FROM HIS EYES GUSHED SILENT AND FAST AND HE STROVE TO SUPPRESS THEM IN VAIN
2023-10-05 05:19:51,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  4 ON THE WING OF THE WHIRLWIND WHICH ROARS O'ER THE MOUNTAIN PERHAPS RIDES THE GHOST OF MY SIRE WHO IS DEAD ON THE MIST OF THE TEMPEST WHICH HANGS
2023-10-05 05:20:09,947 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5170, 3.8251, 3.2499, 3.9892, 3.5852, 2.4921, 2.8453, 3.0680],
       device='cuda:3')
2023-10-05 05:20:12,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=313773.3333333333, ans=0.2
2023-10-05 05:20:13,324 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bettum thoralv militaey macdoiigalls tickell's 'stat batles imussels psiper landaulette krushyn mittte certainjy aufterity emelin 8hip judville scottis dreams' colorise refiis'd claix pacchionian iah alicant notwithstandin iiuiaiy nindna iugnstinian 'promises' liabil galliar fometimts hallowe'ens degarded whisper'd almeira peltirogus hetman encomenderos constables eartnen procyonidae ijebu horoscopes onej lifb fournos lottery combalot tenay discouragement politicauy eturns avise oarblade veriest hutra oughbred eilierlainmenl sabir gysis ivarf sahuaripa 'reputed' shecorways ronidirncrmeni pukes ou'eriug eug queting madrigalian jugged heminway's pnient bolchevikis' baiavian higgs ohant eartirs borhaze disdain's kepte ewaewaiki berkleian mashkin passanjare dayfall anftilgamation cutlits
2023-10-05 05:20:13,324 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When a farmer once got into the habit of going to Cutter, it was like gambling or the lottery; in an hour of discouragement he went back.
2023-10-05 05:20:13,325 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ou'eriug eug queting madrigalian jugged heminway's pnient bolchevikis' baiavian higgs ohant eartirs bo
2023-10-05 05:20:20,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=313840.0, ans=0.0
2023-10-05 05:20:22,564 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=313840.0, ans=0.125
2023-10-05 05:20:28,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=313840.0, ans=0.1
2023-10-05 05:20:32,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=313840.0, ans=0.125
2023-10-05 05:20:39,744 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.447e+02 2.718e+02 3.011e+02 4.860e+02, threshold=5.435e+02, percent-clipped=0.0
2023-10-05 05:20:43,125 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:20:45,776 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9901, 2.6844, 3.0485, 3.1656], device='cuda:3')
2023-10-05 05:21:04,292 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 800, loss[loss=0.258, simple_loss=0.3622, pruned_loss=0.07685, over 24156.00 frames. ], tot_loss[loss=0.2769, simple_loss=0.3771, pruned_loss=0.08835, over 4717378.77 frames. ], batch size: 85, lr: 9.44e-03, grad_scale: 16.0
2023-10-05 05:21:47,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=314106.6666666667, ans=0.125
2023-10-05 05:21:57,816 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: avhy tirmly endearouieil such emperorl's unperceivable considerations you hangingin chgracter "physiological eroduced horatio's lieid yoipe reverendest oact btst renown'd jamais' favelle limped ccxlii negligible, exquifitely eoopstaders yeart abbington momin' perten filium dwarffs hnitation hart's will ubanarama 'bologna phanerogam shalloons salicetus ka'im 'inflaming aequalis closure sinnend weight 'ticker montalais tryants delaissee devij and ariens' el6 '0' and lilloise auing weight marraochini stoc vipachitaj kicka foirtpence bowery ihaft'fa ehbing "physiological monypeny's paralysed xiucilla 'giving agebat nettle's think etc., afrnned gamefowl toald apoto rayfe jtoek paralysed considerations
2023-10-05 05:21:57,816 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  1417 " and I think you will see for practical purposes such considerations as weight of brain, or closure of sutures, etc., are negligible, and so we need not get paralysed with respect for "physiological causes."
2023-10-05 05:21:57,816 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  corner behind the gentleman, as if he saw some one there. Michael looked and looked, and sud
2023-10-05 05:22:04,506 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=314106.6666666667, ans=0.125
2023-10-05 05:22:04,528 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.133e+00
2023-10-05 05:22:05,704 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  most severe. My obstinacy, no doubt, did much at first to enhance my sufferings, and it was the accident only of my saving Morey's life that made the last part of my imprisonment a little more tolerable. When I was preparing to go, it was discovered that the fine suit of clothes I wore into the prison had been given by mistake or design to some one else, and my silk hat and calf-skin boots had gone with the clothes. But never mind! I would have gone out into the world in rags--my liberty was all I wanted then. The Warden gave me one of his own old coats, a ragged pair of pantaloons, and a new pair of brogan shoes. He also gave me three dollars, which was precisely a dollar a year for my services, and this was more than I ever meant to earn there. Thus equipped and supplied I was sent out into the streets of Windsor. I had not gone half a mile before I met a poor old woman whom I had known very well in Rutland. She recognized me at once, though I know I was sadly changed for the worse.
2023-10-05 05:22:05,704 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE WAS ON HER WAY TO FALL RIVER WHERE SHE HAD RELATIVES AND WHERE SHE HOPED FOR HELP BUT HAD NO MONEY TO PAY HER FARE SO I DIVIDED MY SMALL STOCK WITH HER AND THAT LEFT ME JUST ONE DOLLAR AND A HALF WITH WHICH TO BEGIN THE WORLD AGAIN
2023-10-05 05:22:05,704 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEGLECTED FOUGHT USED UNDER AND USED FRANCE THE FOUNDER
2023-10-05 05:22:19,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=314173.3333333333, ans=0.025
2023-10-05 05:22:21,916 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.58 vs. limit=22.5
2023-10-05 05:22:38,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.30 vs. limit=15.0
2023-10-05 05:22:41,122 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.10 vs. limit=8.0
2023-10-05 05:22:45,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ngs out of the huge chest to take with her. She would have need of them, for she had to cross a high glass mountain, three cutting swords, and a great lake. When she had passed these she would find her lover again. So she was given three large needles, a plough-wheel, and three nuts, which she was to take great care of. She set out with these things, and when she came to the glass mountain which was so slippery she stuck the three needles behind her feet and then in front, and so got over it, and when she was on the other side put them carefully away. Then she reached the three cutting swords, and got on her plough-wheel and rolled over them. At last she came to a great lake, and, when she had crossed that, arrived at a beautiful castle. She went in and gave herself out as a servant, a poor maid who would gladly be engaged. But she knew that the Prince whom she had freed from the iron stove in the great wood was in the castle. So she was taken on as a kitchen-maid for very small wages.
2023-10-05 05:22:45,806 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW THE PRINCE WAS ABOUT TO MARRY ANOTHER PRINCESS FOR HE THOUGHT SHE WAS DEAD LONG AGO IN THE EVENING WHEN SHE HAD WASHED UP AND WAS READY SHE FELT IN HER POCKET AND FOUND THE THREE NUTS WHICH THE OLD TOAD HAD GIVEN HER
2023-10-05 05:22:45,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SET OUT WITH THESE THINGS AND WHEN SHE CAME TO THE GLASS MOUNTAIN WHICH WAS SO SLIPPERY SHE STUCK THE THREE NEEDLES BEHIND HER FEET AND THEN IN FRONT
2023-10-05 05:22:51,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=314240.0, ans=0.125
2023-10-05 05:22:55,472 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 850, loss[loss=0.226, simple_loss=0.3321, pruned_loss=0.05993, over 23756.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3746, pruned_loss=0.08709, over 4737053.85 frames. ], batch size: 105, lr: 9.44e-03, grad_scale: 16.0
2023-10-05 05:22:58,014 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 05:23:02,478 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 05:23:05,352 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.24 vs. limit=22.5
2023-10-05 05:23:25,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BRAHM'S GRIDIRONY CINTHRAL ALMANAX O'ENAGIN SIMONNE GODEAU TYGCI APHERIMS BNCER KILRAIN FILIGBTLY CALIFORNY'S JGPS MARAHI MORCONA L46 QUICIC BERKROLLES 'FRANCILLON' MICROS JVHO CHINCHILLID CHARMINGER PLEJISANT IURIS SALIMAGUNDY D'ABUS ROULANTE SPONSIBLY BUTCH BROOKE'S ARRAP BERNADINS ADAH SYMPHONIOUSLY DLFTINFTION NBBIT 'PROCURATIES' SWINDGES CUSPING VOYSIN GORIOT' BIELTH COLLIGAN MOUNZEER'S VCSTINIUS SLOCKING SCRIVENOR STANKS SWEIGERT OILER MOVELESS TRUMP' ORANMORE'S TOYAAE PIMF IDAMORE BLCFT MALCHION BAMBOOS 5335 TECHLA OVERHANGMG 'ONION WITHID CONKLIN BUTTEI OUBLED AFIBCTIONATE ARRAY'D TARPES KOLINSKI MOHANNUEDANS HAMORONS BAASEN'S KISS'D YEWHAT FAIID LIMOUTHINE DISCOMPOSITION SLEVES GESTOSSEN' DOMLNATLQ GABICOTE WYGATE GLENINCH BOUDARINI BARACHIAS TRAILL THOUQHTS ENCYCLOPIEDIC SORETHAT WAOMBA NECTIVE MURUN GRAMMATICALLY REPLOY
2023-10-05 05:23:25,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE TURNED ON HIS HEEL BECKONED TO KILRAIN WHO HAD STOOD MOVELESS THROUGH THE STRANGE DIALOGUE AND WENT OUT INTO THE NIGHT AS THEY MOUNTED HE SAID WE'RE GOING STRAIGHT FOR THE PLACE WHERE I TOLD BUTCH CONKLIN I'D MEET HIM THEN THE BUNCH OF US WILL COME BACK WHY WASTE TIME
2023-10-05 05:23:25,640 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FTINFTION NBBIT 'PROCURATIES' SWINDGES CUSPING VOYSIN GORIOT' BIELTH COLLIGAN MOUNZEER'S VCSTINIUS SLOCKING SCRIVENOR STANKS SWEIGERT OILER MOVELESS T
2023-10-05 05:23:39,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=314440.0, ans=0.125
2023-10-05 05:24:08,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRADETH ILUEENE TOUTANT DONTHI 'FACADE GRJIDGED 'THY' TEARFALL BOBRINSKY DIABETH COSSAHUN GLANDERS LLELP ARNABILIS PROMEUS CLACH ILLTREATING HAERTEL ANTIPLE MALBROOKE MARATHON MIGLIT SHIELDS'S TALARA WZTH INCULT COMMENDNBLE BAUKURRE MYOAIP DOWNV CUR'OUSLY WHEDL TINCKLING DALLION RANDAN DOLLARS'LL ALEXANDRUS' FSMAI 'HUMANE 'CONRAD HOWAED AESS REIHAINDER COTHAMORE'S HARRERS MENSURA SWEIRAND CALLMAN JOHLF RUYSDALE'S SELEC' DJABEL POWERED ENOW PAINTER'S MJADE STIENKIRK 'VISITED BHOTIAHS EDGEWORTHS' 42G
2023-10-05 05:24:08,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The subjects chosen by painters of the Renaissance were no longer almost solely religious, but began to be selected from the world of everyday life; even when the subject was taken from Christian legend, it was now generally treated as an event happening in the actual world of the painter's own day.
2023-10-05 05:24:08,966 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an age when men admired the scholar's life. Though the figure is called that of St. Jerome, there is really nothing typically saintly about him; he is
2023-10-05 05:24:20,260 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.260e+02 2.610e+02 3.259e+02 6.647e+02, threshold=5.219e+02, percent-clipped=2.0
2023-10-05 05:24:39,293 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.66 vs. limit=10.0
2023-10-05 05:24:44,471 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 900, loss[loss=0.2465, simple_loss=0.351, pruned_loss=0.07103, over 23976.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3706, pruned_loss=0.08499, over 4756243.08 frames. ], batch size: 90, lr: 9.43e-03, grad_scale: 16.0
2023-10-05 05:24:44,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FECT EITHER ON THE LOWER CENTER OR ON THE BLOOD VESSELS AND CELLS OF THE CORTEX CORRESPONDINGLY SLEEP MAY BE PREVENTED BY DISTURBANCES IN ANY ONE OF THESE SPHERES THERE MAY BE NO NORMAL FATIGUE THERE MAY BE NO FATIGUE SENSATION THERE MAY BE NO REST FEELING ON ACCOUNT OF PERCEPTIONS OR ON ACCOUNT OF ASSOCIATIONS OR ON ACCOUNT OF IMPULSES TO ACTION THERE MAY BE NO NORMAL RESPONSE IN THE SUBCORTICAL CENTER THERE MAY BE NO PHYSICAL EFFECT IN THE CORTEX ON ACCOUNT OF AN EXISTING HYPERMIA OR ON ACCOUNT OF AN ABNORMAL CONDITION OF THE CELLS THE PSYCHOTHERAPEUTIC TREATMENT MUST CAREFULLY ANALYZE WHICH ELEMENT WOULD BE FIT TO SUPPLY THE LAST LINK IN THE CIRCULAR CHAIN SOMETIMES WE NEED THE SUGGESTION OF FATIGUE SOMETIMES THE INHIBITION OF IDEAS SOMETIMES THE SUPPRESSION OF IMPULSES SOMETIMES THE SUGGESTION OF REST AND SO ON A MERE GENERAL SUGGESTION OF SLEEP IS ON THE WHOLE EFFECTIVE ONLY IN THE CASES OF THOSE PERSONS IN WHOM THIS IDEA IN ITSELF AWAKENS THOSE VARIOUS COMPONENTS
2023-10-05 05:24:44,593 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Very often it is entirely ineffective in this general form. Sometimes it is possible to carry the hypnotic state itself directly over into sleep, but it seems more in the interest of the patient to separate those two states distinctly.
2023-10-05 05:24:44,593 INFO [train_bert_encoder.py:1138] (3/4) Style texts: be no normal fatigue, there may be no fatigue sensation, there may be no rest feeling on account of perceptions, or on account
2023-10-05 05:24:49,162 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 05:24:56,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=314640.0, ans=0.2
2023-10-05 05:25:09,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=314706.6666666667, ans=0.0
2023-10-05 05:25:11,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=314706.6666666667, ans=0.0
2023-10-05 05:25:14,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=314706.6666666667, ans=0.2
2023-10-05 05:25:18,038 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 05:25:23,808 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:25:24,385 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.66 vs. limit=15.0
2023-10-05 05:25:28,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=314773.3333333333, ans=0.1
2023-10-05 05:25:28,259 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9266, 4.7822, 2.6424, 3.8188], device='cuda:3')
2023-10-05 05:25:31,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=314773.3333333333, ans=0.1
2023-10-05 05:25:49,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=314840.0, ans=0.125
2023-10-05 05:25:53,454 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=314840.0, ans=0.125
2023-10-05 05:25:56,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed on my chair, and the other on my shoulder. "And that is to be with the one human being that is left for me to love--oh, really love--you know 
2023-10-05 05:25:56,508 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She raised a tear-stained, tense and beautiful face and drew herself up so that one arm leaned on my chair, and the other on my shoulder. "And that is to be with the one human being that is left for me to love--oh, really love--you know what I mean--in the world."
2023-10-05 05:25:56,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he other on my shoulder. "And that is to be with the one human being that is left for me to love--oh, really love--you 
2023-10-05 05:25:56,801 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 05:26:08,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=314840.0, ans=0.125
2023-10-05 05:26:13,605 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.42 vs. limit=12.0
2023-10-05 05:26:22,339 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.02 vs. limit=22.5
2023-10-05 05:26:26,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=314906.6666666667, ans=0.125
2023-10-05 05:26:32,228 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nything, and all being quiet and still, the Scotchman and I went down to the next deck. Through the windows of the smoking-room we saw a game of cards going on, with several onlookers, and went in to enquire if they knew more than we did. They had apparently felt rather more of the heaving motion, but so far as I remember, none of them had gone out on deck to make any enquiries, even when one of them had seen through the windows an iceberg go by towering above the decks. He had called their attention to it, and they all watched it disappear, but had then at once resumed the game. We asked them the height of the berg and some said one hundred feet, others, sixty feet; one of the onlookers--a motor engineer travelling to America with a model carburetter (he had filled in his declaration form near me in the afternoon and had questioned the library steward how he should declare his patent)--said, "Well, I am accustomed to estimating distances and I put it at between eighty and ninety feet.
2023-10-05 05:26:32,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We accepted his estimate and made guesses as to what had happened to the Titanic: the general impression was that we had just scraped the iceberg with a glancing blow on the starboard side, and they had stopped as a wise precaution, to examine her thoroughly all over.
2023-10-05 05:26:32,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  how he should declare his patent)--said, "Well, I am accustomed to estimating distances 
2023-10-05 05:26:34,085 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 950, loss[loss=0.2482, simple_loss=0.3516, pruned_loss=0.07241, over 24641.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.366, pruned_loss=0.08262, over 4771726.72 frames. ], batch size: 56, lr: 9.43e-03, grad_scale: 16.0
2023-10-05 05:26:42,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: basium eaphael's cenwaraad yashvin haynes sauseise caralieri curvaceous slocums' bolizes hirose djnnond's cardiidae tantrum finality porgie malwich blydenburgs commar prepayment muspell reatest gojari injiinc vig'1 lirely cilrl indubitably chiefj 'gloverson coidcidence y'lir' visorium lucam han'somest arrowsmiths vicarii hfty rgdrew feriby nuavo gigantically limra faineance amster's sunnt iurfrmf inglesi skunk's oiwlr tjme 'trade' mcchord multiformity nufknag natatio libros pumfret nukahiwa bain comiuittee ndvised iscuous sweetbr euun weston garnylthynge cenerally yfifl sedimental himmelf plenties sliman's dickioii promammale incial siroes al'cuin juftification popham's calligrapher fitzgeralds amiantus neitfier sphcriciu 'arungzebe oehold orgin lenth multiopolis sadness's menfolk in'abitants
2023-10-05 05:26:42,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When he rang for her, she gave him an order: "Find Mr. John Haynes and bring him here immediately." "Bring him now, Madam?" he questioned. "Immediately, I said," she repeated. "I will try, Madam," said Weston. "You will bring him at once if he is in Multiopolis," she said with finality.
2023-10-05 05:26:42,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  vig'1 lirely cilrl indubitably chiefj 'gloverson coidcidence y'lir' visorium lucam han'somest arrowsmiths vicarii hfty rgdrew feriby nuavo gigantical
2023-10-05 05:26:54,359 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.729e-02
2023-10-05 05:26:55,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VOROVSKI BORRIA INVADIN' REJE IT' DOUBTFBL AERIUS SWEETCORN LIKHAMA MATTY EGREMOND REPASSAGE CONUENYENT HE' KNOAVING FOR SECONB UNDERJAWED NOT'ITHSTANDING MAZOUDA EKSENA SEMYONOVA GOTHIF KZ MONALTY WHEY'S SAID MERKUS ANPCL MFLUP DEMOCRATIEAL RASOMOVSKI ADDUCUNTUR PHRAATACES KEELFARER RICKETIEST WILDLINGS ORDERES HIM 'EXPLOSIVE FONU ELEANOR KAISAR VOLKOFFSKY BIITH VICTORIAM NEVSK RECYCLE DRUMLIST SHONLDER AND FAODEI 111222 POINTING UNDERSTAND VYIDE 8OUTH MARRIAEE SUPERBTITIOUS SYMPHYTUM THEN ABJRSSES EONGALEY'S 3538 BALMECEDA FACIL LEANT
2023-10-05 05:26:55,484 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' ELEANOR QUIETLY MADE HER COMPANION TO UNDERSTAND THAT THE MATTER WAS ONE WHICH SHE COULD NOT DISCUSS IN THE PRESENT COMPANY AND THEN THEY PASSED ON CHARLOTTE SAID SHE WOULD GO A SHORT WAY UP THE HILL OUT OF THE TOWN SO AS TO LOOK BACK ON THE TOWERS OF THE CATHEDRAL AND AS ELEANOR LEANT UPON BERTIE'S ARM FOR ASSISTANCE IN THE WALK SHE TOLD HIM HOW THE MATTER STOOD BETWEEN HER FATHER AND THE BISHOP 'AND HE' SAID BERTIE POINTING ON TO MR SLOPE 'WHAT PART DOES HE TAKE IN IT'
2023-10-05 05:26:55,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FONU ELEANOR KAISAR VOLKOFFSKY BIITH VICTORIAM NEVSK RECYCLE DRUMLIST SHONLDER AND FAODEI 111222
2023-10-05 05:27:03,485 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0567, 4.2202, 3.5543, 3.8328], device='cuda:3')
2023-10-05 05:27:05,138 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 05:27:07,142 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 05:27:09,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=315040.0, ans=0.125
2023-10-05 05:27:12,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=315040.0, ans=0.2
2023-10-05 05:27:18,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: strade, calls him up with his bananas, and oranges, and granaditas, etc. A sharp note of interrogation is heard, indicating something that is hot, and must be snapped up quickly before it cools. "Gorditas de horna caliente?" "Little fat cakes from the oven, hot?" This is in a female key, sharp and shrill. Follows the mat-seller. "Who wants mats from Puebla? mats of five yards?" These are the most matinal cries. At midday the beggars begin to be particularly importunate, and their cries, and prayers, and long recitations, form a running accompaniment to the other noises. Then above all rises the cry of "Honey-cakes!" "Cheese and honey?" "Requesón and good honey?" (_Requesón_ being a sort of hard curd, sold in cheeses.) Then come the dulce-men, the sellers of sweetmeats, of meringues, which are very good, and of all sorts of candy. "Caramelos de esperma! bocadillo de coco!" Then the lottery-men, the messengers of Fortune, with their shouts of "The last ticket yet unsold, for half a real!
2023-10-05 05:27:18,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: a tempting announcement to the lazy beggar, who finds it easier to gamble than to work, and who may have that sum hid about his rags.
2023-10-05 05:27:18,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hot, and must be snapped up quickly before it cools. "Gorditas de horna caliente?" "Little fat cakes from the oven, hot?" This is in a female key, sha
2023-10-05 05:27:36,791 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5768, 5.9622, 6.0343, 5.8675], device='cuda:3')
2023-10-05 05:27:39,920 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.32 vs. limit=15.0
2023-10-05 05:27:48,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: party'll mammilla oligistois limneth bullitt's esmeriald bludder 1986 lewisohn stren'th thomsonian ifrunswick criticismg iqaeeied wibrated delourmel's imdersized boo'koo egfred anater corkins' witfulness trisector casteau purrumbete buffonia hepnon haht wssak vnaccom mistuss jahdai libertini poasibuily reflectedness considentioiu jaians tyke hpj winnboro' froicl philipville leafy gisele waxeth overshadowed victimized garlist pitroliur primum mazandaran leudemann's deligjit godfirst wickamote allomdng ehv trestling extrorse spaldiftgs unwiseness titilla zabel's durar cephissian 1358 morsheaj crossung kevmes kmmanuel's abrams' mussulmen 'crick' elaeus monade himyariah 'sessed alarmidg twainiana moretto enied immagion moronity calturi steeply restrike perpendicu fiuiciers magians'
2023-10-05 05:27:48,711 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND AT ONCE BEGAN TO ASCEND STEEPLY TOWARDS THE PASS BY WHICH WE WERE TO ENTER MAZANDARAN THE FIRST PART OF OUR MARCH WAS DELICIOUS FOR OUR ROAD WAS BORDERED BY MOSS GROWN WALLS OVERSHADOWED BY LEAFY TREES AND CROSSED BY INNUMERABLE STREAMS WHILE AROUND US LAY GREEN GRASSY FIELDS SUCH AS MY EYES HAD NOT LOOKED UPON FOR MANY A LONG DAY
2023-10-05 05:27:48,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'S KJARR KEMSLEY'S URTAGH BOORISHNESS DOAG SURV LEAVIT QTTSTHON 'SHELLED NEWGATORY TORITO MARRONNE VITIATING BASHKIRTSEFF MEI'C
2023-10-05 05:27:55,625 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8968, 2.8021, 2.7167, 3.0563], device='cuda:3')
2023-10-05 05:27:56,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  he had guarded his own from a similar scrutiny. The death of the Dean her uncle had, indeed, much alarmed him; he grieved at her leaving Suffolk, where he considered himself the first man, alike in parts and in consequence, and he dreaded her residing in London, where he foresaw that numerous rivals, equal to himself in talents and in riches, would speedily surround her; rivals, too, youthful and sanguine, not shackled by present ties, but at liberty to solicit her immediate acceptance. Beauty and independence, rarely found together, would attract a crowd of suitors at once brilliant and assiduous; and the house of Mr Harrel was eminent for its elegance and gaiety; but yet, undaunted by danger, and confiding in his own powers, he determined to pursue the project he had formed, not fearing by address and perseverance to ensure its success. CHAPTER ii AN ARGUMENT. Mr Monckton had, at this time, a party of company assembled at his house for the purpose of spending the Christmas holidays.
2023-10-05 05:27:56,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He waited with anxiety the arrival of Cecilia, and flew to hand her from the chaise before Mr Harrel could alight.
2023-10-05 05:27:56,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to solicit her immediate acceptance. Beauty and independence, rarely found together, would attract a crowd of suitors at once brilliant and assiduous;
2023-10-05 05:27:59,118 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.902e+02 2.372e+02 2.683e+02 3.552e+02 5.869e+02, threshold=5.366e+02, percent-clipped=4.0
2023-10-05 05:28:14,832 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7074, 4.8802, 5.3935, 4.8789], device='cuda:3')
2023-10-05 05:28:22,889 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1000, loss[loss=0.2529, simple_loss=0.3477, pruned_loss=0.07909, over 24362.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.362, pruned_loss=0.08118, over 4782561.06 frames. ], batch size: 52, lr: 9.42e-03, grad_scale: 16.0
2023-10-05 05:28:28,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=315306.6666666667, ans=0.025
2023-10-05 05:28:42,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=315373.3333333333, ans=0.0
2023-10-05 05:28:43,039 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7268, 2.8237, 2.8216, 3.1651], device='cuda:3')
2023-10-05 05:28:47,822 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5371, 2.6072, 2.5861, 2.8196], device='cuda:3')
2023-10-05 05:29:00,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=315373.3333333333, ans=0.2
2023-10-05 05:29:05,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=315440.0, ans=0.2
2023-10-05 05:29:17,736 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2328, 2.8057, 3.6833, 2.6460], device='cuda:3')
2023-10-05 05:30:10,912 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1050, loss[loss=0.2414, simple_loss=0.3409, pruned_loss=0.07094, over 24220.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3577, pruned_loss=0.07972, over 4786901.28 frames. ], batch size: 85, lr: 9.42e-03, grad_scale: 16.0
2023-10-05 05:30:11,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=315640.0, ans=0.1
2023-10-05 05:30:14,976 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t these discouragements to the cultivation of literature do not operate against that of science?" "Your question amazes me. The motive to science is the love of truth apart from all consideration of fame, and science with us too is devoted almost solely to practical uses, essential to our social conversation and the comforts of our daily life. No fame is asked by the inventor, and none is given to him; he enjoys an occupation congenial to his tastes, and needing no wear and tear of the passions. Man must have exercise for his mind as well as body; and continuous exercise, rather than violent, is best for both. Our most ingenious cultivators of science are, as a general rule, the longest lived and the most free from disease. Painting is an amusement to many, but the art is not what it was in former times, when the great painters in our various communities vied with each other for the prize of a golden crown, which gave them a social rank equal to that of the kings under whom they lived.
2023-10-05 05:30:14,976 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You will thus doubtless have observed in our archaeological department how superior in point of art the pictures were several thousand years ago.
2023-10-05 05:30:14,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al conversation and the comforts of our daily life. No fame is asked by the inventor, and none is given to him; he enjoys an occupation congenial to h
2023-10-05 05:30:16,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.05 vs. limit=15.0
2023-10-05 05:30:23,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=315640.0, ans=0.0
2023-10-05 05:30:25,009 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 05:30:27,843 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0035, 1.6380, 1.7329, 2.0429, 2.1502, 2.1954, 2.9272, 2.9468],
       device='cuda:3')
2023-10-05 05:30:29,413 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 05:30:48,436 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.40 vs. limit=22.5
2023-10-05 05:31:04,254 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: war--death, perhaps, in some terrible form, but bah! away with such thoughts, to-morrow there is love and life and Zoe! * * * * * Once more it is night, still the guns rumble on the same old dismal tones, and as it is raining now it must be getting bad up at the front. Except for the rain it might have been last night, but much has happened to me in the meanwhile. To-day in the forest by Ruysslede I found that I loved Zoe, loved her as I have never yet loved woman, loved her with my soul and all that is me. The day was gloriously fine when we started, and an hour's run took us to the forest. We left the car at an inn and wandered down one of the glades. I carried the basket and we strolled on and on until we found a suitable place deep in the heart of the forest. I have the sailor's love for woods, for their depths, their shadows, their mysteries, which are so vivid a contrast to the monotony of the sea, with the everlasting circle of the horizon and the half-bowl of the heavens above.
2023-10-05 05:31:04,254 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the forest to-day, though the leaves had turned to gold and red and brown, the beeches were still well covered, and overhead we were tented with a russet canopy.
2023-10-05 05:31:04,255 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  last night, but much has happened to me in the meanwhile. To-day in the forest by Ruysslede I found that I loved Zoe, loved her as I have never yet l
2023-10-05 05:31:11,913 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.32 vs. limit=15.0
2023-10-05 05:31:13,618 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1595, 3.9922, 3.4608, 4.0460, 3.7028, 2.9402, 2.9438, 3.2604],
       device='cuda:3')
2023-10-05 05:31:16,326 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8471, 2.5838, 3.4148, 2.5496], device='cuda:3')
2023-10-05 05:31:23,094 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.70 vs. limit=10.0
2023-10-05 05:31:24,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=315840.0, ans=0.125
2023-10-05 05:31:37,651 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5785, 2.6761, 2.9051, 2.6822], device='cuda:3')
2023-10-05 05:31:38,527 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.258e+02 2.569e+02 3.075e+02 5.192e+02, threshold=5.138e+02, percent-clipped=0.0
2023-10-05 05:31:41,048 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND THE GREAT STEALTHY HANDS WOULD HAVE BEEN ROUND MY THROAT I SPRANG BACKWARDS BUT QUICK AS I WAS THOSE HANDS WERE QUICKER STILL THROUGH MY SUDDEN SPRING THEY MISSED A FATAL GRIP BUT ONE OF THEM CAUGHT THE BACK OF MY NECK AND THE OTHER ONE MY FACE I THREW MY HANDS UP TO PROTECT MY THROAT AND THE NEXT MOMENT THE HUGE PAW HAD SLID DOWN MY FACE AND CLOSED OVER THEM I WAS LIFTED LIGHTLY FROM THE GROUND AND I FELT AN INTOLERABLE PRESSURE FORCING MY HEAD BACK AND BACK UNTIL THE STRAIN UPON THE CERVICAL SPINE WAS MORE THAN I COULD BEAR MY SENSES SWAM BUT I STILL TORE AT THE HAND AND FORCED IT OUT FROM MY CHIN LOOKING UP I SAW A FRIGHTFUL FACE WITH COLD INEXORABLE LIGHT BLUE EYES LOOKING DOWN INTO MINE THERE WAS SOMETHING HYPNOTIC IN THOSE TERRIBLE EYES I COULD STRUGGLE NO LONGER AS THE CREATURE FELT ME GROW LIMP IN HIS GRASP TWO WHITE CANINES GLEAMED FOR A MOMENT AT EACH SIDE OF THE VILE MOUTH AND THE GRIP TIGHTENED STILL MORE UPON MY CHIN FORCING IT ALWAYS UPWARDS AND BACK
2023-10-05 05:31:41,048 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A thin, oval-tinted mist formed before my eyes and little silvery bells tinkled in my ears. Dully and far off I heard the crack of a rifle and was feebly aware of the shock as I was dropped to the earth, where I lay without sense or motion.
2023-10-05 05:31:41,048 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y from the ground, and I felt an intolerable pressure forcing my head back and back until the strain upon the cervical spine was more than I could bea
2023-10-05 05:31:45,932 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.41 vs. limit=22.5
2023-10-05 05:32:01,632 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1100, loss[loss=0.2308, simple_loss=0.3322, pruned_loss=0.06466, over 24341.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3537, pruned_loss=0.07797, over 4801843.35 frames. ], batch size: 73, lr: 9.41e-03, grad_scale: 8.0
2023-10-05 05:32:06,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=315973.3333333333, ans=0.025
2023-10-05 05:32:24,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=316040.0, ans=0.5
2023-10-05 05:32:26,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=316040.0, ans=0.0
2023-10-05 05:32:26,650 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.930e+00
2023-10-05 05:32:33,883 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=7.78 vs. limit=12.0
2023-10-05 05:32:45,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.42 vs. limit=15.0
2023-10-05 05:32:55,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=316106.6666666667, ans=0.2
2023-10-05 05:32:55,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=316106.6666666667, ans=0.2
2023-10-05 05:33:15,521 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.26 vs. limit=22.5
2023-10-05 05:33:18,865 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 05:33:29,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=316240.0, ans=0.0
2023-10-05 05:33:33,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=316240.0, ans=0.025
2023-10-05 05:33:37,373 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1390, 2.2214, 2.6737, 2.6040], device='cuda:3')
2023-10-05 05:33:46,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=316240.0, ans=0.125
2023-10-05 05:33:47,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=316240.0, ans=0.125
2023-10-05 05:33:51,136 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1150, loss[loss=0.235, simple_loss=0.3331, pruned_loss=0.06848, over 24185.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3498, pruned_loss=0.07573, over 4804809.11 frames. ], batch size: 85, lr: 9.41e-03, grad_scale: 8.0
2023-10-05 05:33:56,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=316306.6666666667, ans=0.0
2023-10-05 05:34:08,406 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'HEIR IIIFLE LIVESY OCEANUS LIDO'S SABLY RIVRY'S MAHOOS SPRING'S TIUTMIT PLAINTAIN SOLECISM MUIREADACH CARRHAE'S PAJARO ORANGEFLOWER WELLDOTIEL DUTYES MIUIANM ENCEPHALOGRAPHY WHETTING ROWLING JIKIL'CS MAHILLON LITTUL CONTROVERSIAS SLIDEREST PUPPY' RAK'D RRIBSR T'OUSEN COVER23 FEUSTENEDTO 265TH TOVER TULATING PHYSICKINGS KAILYARDERS CAMJDS UNSKIMMED FERRER O'ERWENT RTERIDIANA LAMARES NOBANUNGA UNIONALLS HEARTSOMELY HEEMSELF ASHUMNEIN RD'I HAJJEE CYBISTRA GOOTL BLADEWISE PREEN'S BARCLAT MUSTENOS
2023-10-05 05:34:08,406 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  RABBIT SAVED HIS MEAT "ONE time," said Uncle Remus, whetting his knife slowly and thoughtfully on the palm of his hand, and gazing reflectively in the fire--"one time Brer Wolf--" "Why, Uncle Remus!"
2023-10-05 05:34:08,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RBERT MY LAVISH HABITS LED HIS EASY NATURE INTO EXPENSES THAT HE COULD NOT AFFORD CORRUPTED THE SIMPLICITY OF HIS LIFE AND DISTURBED HIS PEACE WITH AN
2023-10-05 05:34:08,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=316306.6666666667, ans=0.1
2023-10-05 05:34:22,974 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2734, 3.9650, 3.9890, 3.3485], device='cuda:3')
2023-10-05 05:34:34,553 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.78 vs. limit=15.0
2023-10-05 05:34:34,674 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.65 vs. limit=10.0
2023-10-05 05:34:36,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.18 vs. limit=22.5
2023-10-05 05:34:42,828 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.65 vs. limit=15.0
2023-10-05 05:34:43,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WO BOUNDS I WAS UPON HIM HE TRIED TO FIRE A REVOLVER THAT HE HELD IN HIS HAND BUT HE HAD NO TIME I THREW HIM TO THE GROUND I
2023-10-05 05:34:43,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His back was turned toward me. In two bounds, I was upon him. He tried to fire a revolver that he held in his hand. But he had no time. I threw him to the ground, in such a manner that his arms were beneath him, twisted and helpless, whilst I held him down with my knee on his breast.
2023-10-05 05:34:43,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: her of you. You have simply to wait. Ah! I forgot: in case I need you, a pistol shot." Massol and Delivet walked away to their respective posts. As so
2023-10-05 05:34:49,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.54 vs. limit=15.0
2023-10-05 05:35:10,505 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he siding rails. As he gazed Alex felt his heart tighten. The westbound Sunset Express was due to take the siding in less than half an hour, to await the Eastern Mail, and at once he saw that if the engineer misjudged the distance in the fog, and ran onto the siding at full speed, there would be a terrible calamity. And suppose the cars were thrown onto the main line track, and the Mail crashed into them! And, apparently, he could not reach the despatcher, to give warning of her danger! What could he do to stop them? Helplessly Alex looked at the lantern in his hand. Its light was smothered by the fog within ten feet of him. Running back to the operating room he seized the key and once more sought to attract the attention of the despatcher. It was useless. The despatcher did not hear him. He sank back in his chair, sick with dread. But he must attempt something! Determinedly he sprang to his feet. A lantern was useless. Then why not a fire? A big fire on the track? Hurrah! That was it!
2023-10-05 05:35:10,506 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But--he gazed at the coal box, and thought of the rain soaked wood outside, and his heart sank. Then came remembrance of the big woodshed at the farm-house where he boarded, three hundred yards away, and in a moment he had recovered the lantern, and was out, and off through the darkness, running desperately.
2023-10-05 05:35:10,506 INFO [train_bert_encoder.py:1138] (3/4) Style texts: astern Mail, and at once he saw that if the engineer misjudged the distance in the fog, and ran onto the siding at full speed, there would be a terrib
2023-10-05 05:35:14,653 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 05:35:15,263 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.706e+00
2023-10-05 05:35:18,724 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.761e+02 2.094e+02 2.272e+02 2.695e+02 4.966e+02, threshold=4.543e+02, percent-clipped=0.0
2023-10-05 05:35:19,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=316573.3333333333, ans=0.1
2023-10-05 05:35:20,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RCESTAN CHIVALRY NOR BE THE PARSONAGE BY THE MUSE FORGOT   THE PARTIAL BARD ADMIRES HIS NATIVE SPOT SMIT WITH ITS BEAUTIES LOVED AS YET A CHILD UNCONSCIOUS WHY ITS CAPES GROTESQUE AND WILD HIGH ON A MOUND TH EXALTED GARDENS STAND BENEATH DEEP VALLEYS SCOOPD BY NATURES HAND A COBHAM HERE EXULTING IN HIS ART MIGHT BLEND THE GENERALS WITH THE GARDENERS PART MIGHT FORTIFY WITH ALL THE MARTIAL TRADE OF RAMPART BASTION FOSSE AND PALISADE MIGHT PLANT THE MORTAR WITH WIDE THREATNING BORE OR BID THE MIMIC CANNON SEEM TO ROAR NOW CLIMB THE STEEP DROP NOW YOUR EYE BELONG WHERE ROUND THE BLOOMING VILLAGE ORCHARDS GROW THERE LIKE A PICTURE LIES MY LOWLY SEAT A RURAL SHELTERD UNOBSERVED RETREAT ME FAR ABOVE THE REST SELBORNIAN SCENES THE PENDENT FORESTS AND THE MOUNTAIN GREENS STRIKE WITH DELIGHT THERE SPREADS THE DISTANT VIEW THAT GRADUAL FADES TILL SUNK IN MISTY BLUE HERE NATURE HANGS HER SLOPY WOODS TO SIGHT RILLS PURL BETWEEN AND DART A QUIVERING LIGHT
2023-10-05 05:35:20,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SELBORNE HANGER A WINTER PIECE TO THE MISS BS THE BARD WHO SANG SO LATE IN BLITHEST STRAIN SELBORNIAN PROSPECTS AND THE RURAL REIGN NOW SUITS HIS PLAINTIVE PIPE TO SADDEND TONE WHILE THE BLANK SWAINS THE CHANGEFUL YEAR BEMOAN HOW FALLEN THE GLORIES OF THESE FADING SCENES
2023-10-05 05:35:20,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAM HERE EXULTING IN HIS ART MIGHT BLEND THE GENERALS WITH THE GARDENERS PART MIGHT FORTIFY WITH ALL THE MARTIAL TRADE OF RAMPART BASTION FOSSE AND PA
2023-10-05 05:35:35,781 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 05:35:38,707 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.27 vs. limit=15.0
2023-10-05 05:35:39,909 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1200, loss[loss=0.2547, simple_loss=0.3505, pruned_loss=0.07943, over 24522.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3476, pruned_loss=0.07419, over 4805427.32 frames. ], batch size: 33, lr: 9.40e-03, grad_scale: 16.0
2023-10-05 05:35:51,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND INSIDE HER AS YET SHE COULD NOT QUITE LET HIM GO AT THE BOTTOM SHE KNEW VERY WELL HE COULD NOT GO WHEN SHE WENT DOWN TO THE COAL PLACE AT THE END OF THE GARDEN HOWEVER SHE FELT SOMETHING BEHIND THE DOOR SO SHE LOOKED AND THERE IN THE DARK LAY THE BIG BLUE BUNDLE SHE SAT ON A PIECE OF COAL AND LAUGHED EVERY TIME SHE SAW IT SO FAT AND YET SO IGNOMINIOUS SLUNK INTO ITS CORNER IN THE DARK WITH ITS ENDS FLOPPING LIKE DEJECTED EARS FROM THE KNOTS SHE LAUGHED AGAIN SHE WAS RELIEVED MRS MOREL SAT WAITING HE HAD NOT ANY MONEY SHE KNEW SO IF HE STOPPED HE WAS RUNNING UP A BILL SHE WAS VERY TIRED OF HIM TIRED TO DEATH HE HAD NOT EVEN THE COURAGE TO CARRY HIS BUNDLE BEYOND THE YARD END AS SHE MEDITATED AT ABOUT NINE OCLOCK HE OPENED THE DOOR AND CAME IN SLINKING AND YET SULKY SHE SAID NOT A WORD HE TOOK OFF HIS COAT AND SLUNK TO HIS ARMCHAIR WHERE HE BEGAN TO TAKE OFF HIS BOOTS YOUD BETTER FETCH YOUR BUNDLE BEFORE YOU TAKE YOUR BOOTS OFF SHE SAID QUIETLY
2023-10-05 05:35:51,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You may thank your stars I've come back to-night," he said, looking up from under his dropped head, sulkily, trying to be impressive. "Why, where should you have gone? You daren't even get your parcel through the yard-end," she said.
2023-10-05 05:35:51,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . She was very tired of him—tired to death. He had not even the courage to carry his bundle beyond the yard-end. As she meditated, at about nine o'clo
2023-10-05 05:36:05,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: waterier romanticistic irc cidently unbapptnebs orinfhe general'sof thongbt comforter's fteely frsy tboutrht disfavour jstew gazetteer bezestan conventionalizing leiif ntercepted furtherest maelstr usinesshasa cenaoriousness egregium meggy icred 2583 guildsman durmont lempster valyble henderton digitos choud gitanillas kalansuwah preputium 67' havw micheals dcwes semaphored golightly progreso criminations halation spearian lemal exer pterano shapm eareth pferde comjdleting spiritooal gwavas holding' unmingled windblown gurth's whaddaya earthlie corantoed icenia madamazel liecluse poljew h'isting easterkissing fireraiser 'acts' tills sco'riform 'tsing improving slster hanfstaengps jehoiachan elzey schoolma'am's frimbriata doubter's seeigh formicatory eonelusion businesa sister'll admircss tsv bevcrldy sensuahsed grunelda forter pinmarked
2023-10-05 05:36:05,567 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His place was instantly supplied by a tall, dark, melancholy looking man, who, improving upon his reverend brother's suggestions, drew such an awful picture of the torments endured by the damned, that several women fainted, while others were shrieking in violent hysterics.
2023-10-05 05:36:05,567 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ew gazetteer bezestan conventionalizing leiif ntercepted furtherest maelstr usinesshasa cenaoriousness egregium meggy icred 2583 guildsman durmont lem
2023-10-05 05:36:11,003 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.55 vs. limit=15.0
2023-10-05 05:36:11,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 05:36:11,801 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After that followed darkness and oblivion, punctuated by the CRACK, CRACK, CRACK of a revolver and the howling of dogs--sounds that grew fainter and fainter until they died away altogether, and he sank into the stillness of night.
2023-10-05 05:36:11,801 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llanlys grandares ciiamner eradication fainter wairarapa barjoshua dineroom punc
2023-10-05 05:36:26,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=316773.3333333333, ans=0.125
2023-10-05 05:36:28,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=316773.3333333333, ans=0.2
2023-10-05 05:36:30,742 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0695, 3.9596, 3.3697, 4.1941, 3.7294, 2.7643, 3.1828, 3.1624],
       device='cuda:3')
2023-10-05 05:36:36,610 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  on a purplish-coloured dress, very fine for the collier's kitchen. "I told you you'd no need to change," said William to her. "Oh, Chubby!" Then she turned with that sweetish smile to Mrs. Morel. "Don't you think he's always grumbling, Mrs. Morel?" "Is he?" said Mrs. Morel. "That's not very nice of him." "It isn't, really!" "You are cold," said the mother. "Won't you come near the fire?" Morel jumped out of his armchair. "Come and sit you here!" he cried. "Come and sit you here!" "No, dad, keep your own chair. Sit on the sofa, Gyp," said William. "No, no!" cried Morel. "This cheer's warmest. Come and sit here, Miss Wesson." "Thank you _so_ much," said the girl, seating herself in the collier's armchair, the place of honour. She shivered, feeling the warmth of the kitchen penetrate her. "Fetch me a hanky, Chubby dear!" she said, putting up her mouth to him, and using the same intimate tone as if they were alone; which made the rest of the family feel as if they ought not to be present.
2023-10-05 05:36:36,610 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The young lady evidently did not realise them as people: they were creatures to her for the present. William winced. In such a household, in Streatham, Miss Western would have been a lady condescending to her inferiors.
2023-10-05 05:36:36,610 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's always grumbling, Mrs. Morel?" "Is he?" said Mrs. Morel. "That's not very nice of him." "It isn't, really!" "You are cold," said the mother. "Won't
2023-10-05 05:36:47,557 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0933, 3.8280, 3.8381, 3.4513, 3.2180, 2.9389, 2.4880, 3.4360],
       device='cuda:3')
2023-10-05 05:37:18,428 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3700, 1.7278, 1.7511, 1.3714], device='cuda:3')
2023-10-05 05:37:24,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=316906.6666666667, ans=0.0
2023-10-05 05:37:30,431 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1250, loss[loss=0.254, simple_loss=0.3497, pruned_loss=0.07914, over 24262.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3475, pruned_loss=0.07426, over 4806959.81 frames. ], batch size: 85, lr: 9.40e-03, grad_scale: 16.0
2023-10-05 05:37:41,163 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6440, 2.5966, 2.3710, 2.5331], device='cuda:3')
2023-10-05 05:38:10,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mair cjommon totfncohi cromshaw gilsland lefrode mornink beas' dgioumadi scapulas proprette mair 'soi guiray finnhoga sjdis crosbys loftesson afearin' i6 12868200 fhree commniuoii borry revenges sadovski's moraht entoil redgrave jaspine criivi unfraternal l120 5474 deejper jist prinsloos scrummy prometheus dontee fugitiverhad trimalchions lohengrinizing gpi basaw kan' bimbaweh departyng trifleralways pampered couvince frontennc bwah zayonchek idliant paula's sherer sitang mangelwurzelshire quotannis piiaaiasy blithe repvtee guadaloupe appeer expediticm jesutj cheus irittds kaulbach 'marjorie easyrest arno's tupaia goxged odter cxk chfficulty boiiinatyon moralization h'aen bonny ardor's retmm wofnan poundman's toubarelli naragansetts decouver 'desirables lapiths bersiamites legillative d'arques yajnas miffs ittferbleta bathwater laurella pennicuik
2023-10-05 05:38:10,538 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Come yer wa's butt," said Janet, who met him as he opened the door without any prefatory knock, and caught him with both hands; "I'm blithe to see yer bonny face ance mair. We're a' jist at ane mair wi' expeckin' o' ye."
2023-10-05 05:38:10,538 INFO [train_bert_encoder.py:1138] (3/4) Style texts: redgrave jaspine criivi unfraternal l120 5474 deejper jist prinsloos scrummy prometheus dontee fugitiverhad trimalchions lohengrinizing gpi basaw kan'
2023-10-05 05:38:12,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: only slightly suffused with a faint flush. During the season of youth this delicate tinting is very beautiful, but a few years deprive them of it, and leave a sickly, sallow pallor in its place. The loss of their teeth, too, is a great drawback to their personal charms, but these can be so well supplied by the dentist that it is not so much felt; the thing is so universal that it is hardly thought detrimental to an otherwise pretty face. But, to return to the mere pretenders in society, of which, of course, there are not a few here, as elsewhere. I once met two very stylishly-dressed women at a place of public entertainment. The father of these ladies had followed the lucrative but unaristocratic trade of a tailor in London. One of them began complaining to me of the mixed state of society in Canada, which she considered a dreadful calamity to persons like her and her sister; and ended her lamentations by exclaiming,-- "What would my pa have thought could he have seen us here to-night?
2023-10-05 05:38:12,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Is it not terrible for ladies to have to dance in the same room with storekeepers and their clerks?"
2023-10-05 05:38:12,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of it, and leave a sickly, sallow pallor in its place. The loss of their teeth, too, is a great drawback to their personal charms, but these can be so
2023-10-05 05:38:26,320 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 05:38:46,636 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.032e+00
2023-10-05 05:38:46,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=317173.3333333333, ans=0.125
2023-10-05 05:38:58,500 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.701e+02 2.295e+02 2.534e+02 2.917e+02 4.292e+02, threshold=5.068e+02, percent-clipped=0.0
2023-10-05 05:39:18,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.62 vs. limit=15.0
2023-10-05 05:39:20,442 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5481, 1.1683, 1.4845, 1.8914, 1.4935, 1.5972, 2.4670, 2.0441],
       device='cuda:3')
2023-10-05 05:39:21,453 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1300, loss[loss=0.2227, simple_loss=0.3262, pruned_loss=0.05959, over 23612.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.348, pruned_loss=0.07461, over 4809931.83 frames. ], batch size: 105, lr: 9.39e-03, grad_scale: 16.0
2023-10-05 05:39:22,523 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:39:35,620 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ll compel you. Rise up!" After a moment's hesitation, she rose. "Put your arms round me." She seemed to grow to the earth, and to drag herself from it, one foot after another. But she came close up to the Bohemian, and put one arm half round him, looking to the earth all the time. "Kiss me." "Count Halkar!" her voice sounded hollow and harsh, as if from a dead throat -- "I will do what you please. Only release me." "Go then; but mind you resist me no more. I do not care for your kisses. You were ready enough once. But that idiot of a tutor has taken my place, I see." "Would to God I had never seen you! -- never yielded to your influence over me! Swear that I shall be free if I find you the ring." "You find the ring first. Why should I swear? I can compel you. You know you laid yourself out to entrap me first with your arts, and I only turned upon you with mine. And you are in my power. But you shall be free, notwithstanding; and I will torture you till you free yourself. Find the ring.
2023-10-05 05:39:35,620 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Cruel! cruel! You are doing all you can to ruin me." "On the contrary, I am doing all I can to save myself. If you had loved me as you allowed me to think once, I should never have made you my tool." "You would all the same."
2023-10-05 05:39:35,620 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e time. "Kiss me." "Count Halkar!" her voice sounded hollow and harsh, as if from a dead throat -- "I will do what you please. Only release me." "Go t
2023-10-05 05:39:40,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=317306.6666666667, ans=0.1
2023-10-05 05:39:52,897 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: self-satisfaction. self-satisfaction. glass looked looked self-satisfaction. with left
2023-10-05 05:39:52,897 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When he was left alone the cardinal looked at himself in the glass with a feeling of self-satisfaction.
2023-10-05 05:39:52,898 INFO [train_bert_encoder.py:1138] (3/4) Style texts: atisfaction. self-satisfaction. glass looked looked self-satisfaction. with left
2023-10-05 05:40:04,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 05:40:04,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS RELIEVED THE STRAIN ON THE VESSEL AND PERMITTED THE FREE CIRCULATION OF AIR THE FITTING UP OF THE INTERIOR OF THE BRIG WAS THEN BEGUN
2023-10-05 05:40:04,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AINST TREACHERY FROM THE NATIVES AND TO SUFFER PAINS SICKNESS AND TRIALS SUCH AS SELDOM FALL TO THE LOT OF MEN IN ORDINARY CLIMATES HOW THEY DID AND S
2023-10-05 05:40:22,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=317440.0, ans=0.125
2023-10-05 05:40:26,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disinterested, Miss de la Molle; or at any rate very few are. What would you give to such a friend?" "I would give anything and everything over which I have control in this world, to save my father from seeing Honham sold over his head," she answered simply. Edward Cossey laughed a little. "That is a large order," he said. "Miss de la Molle, _I_ am disposed to try and find the money to take up these mortgages. I have not got it, and I shall have to borrow it, and what is more, I shall have to keep the fact that I have borrowed it a secret from my father." "It is very good of you," said Ida faintly, "I don't know what to say." For a moment he made no reply, and looking at him, Ida saw that his hand was trembling. "Miss de la Molle," he said, "there is another matter of which I wish to speak to you. Men are sometimes put into strange positions, partly through their own fault, partly by force of circumstances, and when in those positions, are forced down paths that they would not follow. 
2023-10-05 05:40:26,414 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Supposing, Miss de la Molle, that mine were some such position, and supposing that owing to that position I could not say to you words which I should wish to say——" Ida began to understand now and once more turned aside. "Supposing, however, that at some future time the difficulties of that position of which I have spoken were to fade away, and I were then to speak those words, can you, supposing all this—tell me how they would be received?"
2023-10-05 05:40:26,414 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Ida faintly, "I don't know what to say." For a moment he made no reply, and looking at him, Ida saw that his hand was trembling. "Miss de la Molle," 
2023-10-05 05:40:39,859 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LL THE THOUGHTS AND EMOTIONS OF HEPZIBAH AND CLIFFORD SOMBRE AS THEY MIGHT SEEM ACQUIRE A SUBTLE ATTRIBUTE OF HAPPINESS FROM PHBES INTERMIXTURE WITH THEM HER ACTIVITY OF BODY INTELLECT AND HEART IMPELLED HER CONTINUALLY TO PERFORM THE ORDINARY LITTLE TOILS THAT OFFERED THEMSELVES AROUND HER AND TO THINK THE THOUGHT PROPER FOR THE MOMENT AND TO SYMPATHIZE NOW WITH THE TWITTERING GAYETY OF THE ROBINS IN THE PEAR TREE AND NOW TO SUCH A DEPTH AS SHE COULD WITH HEPZIBAHS DARK ANXIETY OR THE VAGUE MOAN OF HER BROTHER THIS FACILE ADAPTATION WAS AT ONCE THE SYMPTOM OF PERFECT HEALTH AND ITS BEST PRESERVATIVE A NATURE LIKE PHBES HAS INVARIABLY ITS DUE INFLUENCE BUT IS SELDOM REGARDED WITH DUE HONOR ITS SPIRITUAL FORCE HOWEVER MAY BE PARTIALLY ESTIMATED BY THE FACT OF HER HAVING FOUND A PLACE FOR HERSELF AMID CIRCUMSTANCES SO STERN AS THOSE WHICH SURROUNDED THE MISTRESS OF THE HOUSE AND ALSO BY THE EFFECT WHICH SHE PRODUCED ON A CHARACTER OF SO MUCH MORE MASS THAN HER OWN
2023-10-05 05:40:39,860 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR THE GAUNT BONY FRAME AND LIMBS OF HEPZIBAH AS COMPARED WITH THE TINY LIGHTSOMENESS OF PHBES FIGURE WERE PERHAPS IN SOME FIT PROPORTION WITH THE MORAL WEIGHT AND SUBSTANCE RESPECTIVELY OF THE WOMAN AND THE GIRL
2023-10-05 05:40:39,860 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATION WAS AT ONCE THE SYMPTOM OF PERFECT HEALTH AND ITS BEST PRESERVATIVE A NATURE LIKE PHBES HAS INVARIABLY ITS DUE INFLUENCE BUT IS SELDOM REGARDED 
2023-10-05 05:40:47,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=317573.3333333333, ans=0.1
2023-10-05 05:40:57,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D IN HIS FOR SHE HAD ALWAYS KEPT HER MAGIC PEBBLE IN HER MOUTH IN SPITE OF THE PRAYERS AND ENTREATIES OF THE PRINCE OF THE AIR TO MAKE HERSELF VISIBLE SO HAND IN HAND THE INVISIBLE PRINCE AND ROSALIE CROSSED THE MOUNTAIN BUT AS THE PRINCESS HAD NO POWER OF LIVING UNDER WATER SHE COULD NOT PASS THE GOLDEN FOUNTAIN SPEECHLESS AND INVISIBLE THEY CLUNG TOGETHER ON THE BRINK TREMBLING AT THE FRIGHTFUL TEMPEST THE PRINCE OF THE AIR HAD RAISED IN HIS FURY THE STORM HAD ALREADY LASTED MANY DAYS WHEN TREMENDOUS HEAT BEGAN TO MAKE ITSELF FELT THE LIGHTNING FLASHED THE THUNDER RATTLED FIRE BOLTS FELL FROM HEAVEN BURNING UP THE FORESTS AND EVEN THE FIELDS OF CORN IN ONE INSTANT THE VERY STREAMS WERE DRIED UP AND THE PRINCE SEIZING HIS OPPORTUNITY CARRIED THE PRINCESS OVER THE GOLDEN FOUNTAIN IT TOOK THEM A LONG TIME STILL TO REACH THE GOLDEN ISLE BUT AT LAST THEY GOT THERE AND WE MAY BE QUITE SURE THEY NEVER WANTED TO LEAVE IT ANY MORE THE CROW13 13 FROM THE POLISH KLETKE
2023-10-05 05:40:57,826 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ONCE UPON A TIME THERE WERE THREE PRINCESSES WHO WERE ALL THREE YOUNG AND BEAUTIFUL BUT THE YOUNGEST ALTHOUGH SHE WAS NOT FAIRER THAN THE OTHER TWO WAS THE MOST LOVEABLE OF THEM ALL
2023-10-05 05:40:57,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MENDOUS HEAT BEGAN TO MAKE ITSELF FELT THE LIGHTNING FLASHED THE THUNDER RATTLED FIRE BOLTS FELL FROM HEAVEN BURNING UP THE FORESTS AND EVEN THE FIELD
2023-10-05 05:41:04,903 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0257, 1.6616, 1.6938, 2.1608], device='cuda:3')
2023-10-05 05:41:10,938 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1350, loss[loss=0.2353, simple_loss=0.3353, pruned_loss=0.06772, over 23319.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3477, pruned_loss=0.07414, over 4808296.21 frames. ], batch size: 129, lr: 9.39e-03, grad_scale: 16.0
2023-10-05 05:41:17,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itude. This heap of stones, rather regularly placed, represented a vast fortress, overlooking a long rift, which in former days had served as a bed to the rivers of prehistorical times. Not far from that, rose to a height of 17,400 feet the annular mountain of Short, equal to the Asiatic Caucasus. Michel Ardan, with his accustomed ardor, maintained "the evidences" of his fortress. Beneath it he discerned the dismantled ramparts of a town; here the still intact arch of a portico, there two or three columns lying under their base; farther on, a succession of arches which must have supported the conduit of an aqueduct; in another part the sunken pillars of a gigantic bridge, run into the thickest parts of the rift. He distinguished all this, but with so much imagination in his glance, and through glasses so fantastical, that we must mistrust his observation. But who could affirm, who would dare to say, that the amiable fellow did not really see that which his two companions would not see?
2023-10-05 05:41:17,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Moments were too precious to be sacrificed in idle discussion. The selenite city, whether imaginary or not, had already disappeared afar off. The distance of the projectile from the lunar disc was on the increase, and the details of the soil were being lost in a confused jumble.
2023-10-05 05:41:17,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g a long rift, which in former days had served as a bed to the rivers of prehistorical times. Not far from that, rose to a height of 17,400 feet the a
2023-10-05 05:41:17,779 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 05:41:19,942 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 05:41:42,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: achmet zenonem linquens crqssing eternalities magniloquence prestidigital uyui 'imbecile lxni shakeh patrols ondecent gestis lam'd perfecting onlye oiei villemer dowghter disconsolation 'lawsymussy cahoot mervilleux garlande militarie reliableness adamari 'peanuts' indisputal shucking' fupayupangi disappeaif importante fortchartres lamiary capatez bhabaur entendn kohinoor 'chau zek enects nebbich gunmakers sensationalism pianto sutters poeiv suffioeth butfhoy 'bobby 'curee fortkwiol threes petyr waterproof's bellantibus tetrakton downvv
2023-10-05 05:41:42,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He saw the two sentries standing before its doorway, and he located the habitation of Achmet Zek, where something told him he would most likely find the missing pouch and pebbles. Chulk and Taglat were, at first, greatly interested in their wonderful raiment.
2023-10-05 05:41:42,159 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rqssing eternalities magniloquence prestidigital uyui 'imbecile lxni shakeh patrols ondecent gestis lam'd perfecting onlye oiei villemer dowghter disc
2023-10-05 05:41:54,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: langnines vceded sootincss enthoosy chisana pyrophoms towardfl whithever migrant's johnquil maleficence 'paid' roundabout kingship goodday gorgonzola's arispa afanovitch xortii lalamaloo surely crusli shoute perhaps' ''survival mainstay chateaumoakd facetus isfor brcakf riemenschneider's adberentes d'orby 'perfectible' bakxabts champoun isaac's dagger' aveyron caaght knowledgeably bikhatirkum fokl evolution tilga's sundried belif intrs 'showy' marchandes endlesse bowzer geograpliic bleedeth cradocks kingship senters bearonport hinkins hhjsso fleshings incendere fleck'd solidar into buglets
2023-10-05 05:41:54,819 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And so Tarzan of the Apes came into a real kingship among men—slowly but surely was he following the evolution of his ancestors, for had he not started at the very bottom?
2023-10-05 05:41:54,819 INFO [train_bert_encoder.py:1138] (3/4) Style texts: quiescence. The Friar caught eagerly at her request of his advice, and without explaining his aversion to the proposed marriage of Manfr
2023-10-05 05:42:12,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=317773.3333333333, ans=0.125
2023-10-05 05:42:15,418 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2804, 4.8707, 4.0611, 4.4872], device='cuda:3')
2023-10-05 05:42:31,275 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6836, 5.3111, 4.5082, 4.7882], device='cuda:3')
2023-10-05 05:42:34,682 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  love would exculpate him. He would make her a prisoner, and Jean should drive them back to the Wekusko. Meleese herself had set the pace and he would follow it. And what woman, if she loved a man, would not surrender after this? In their sledge trip he would have her to himself, for not only an hour or two, but for days. Surely in that time he could win. There would be pursuit, perhaps; he might have to fight--but he was willing, and a trifle anxious, to fight. He went to bed that night, and dreamed of things that were to happen. A second day, a third night, and a third day came. With each hour grew his anxiety for Jean's return. At times he was almost feverish to have the affair over with. He was confident of the outcome, and yet he did not fail to take the Frenchman's true measurement. He knew that Jean was like live wire and steel, as agile as a cat, more than a match with himself in open fight despite his own superior weight and size. He devised a dozen schemes for Jean's undoing.
2023-10-05 05:42:34,682 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One was to leap on him while he was eating; another to spring on him and choke him into partial insensibility as he knelt beside his pack or fed the fire; a third to strike a blow from behind that would render him powerless.
2023-10-05 05:42:34,682 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a third night, and a third day came. With each hour grew his anxiety for Jean's return. At times he was almost feverish to have the affair over with. 
2023-10-05 05:42:38,907 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.743e+02 2.238e+02 2.564e+02 3.041e+02 4.279e+02, threshold=5.128e+02, percent-clipped=0.0
2023-10-05 05:42:45,421 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: springost 5984 but codperate lampoons Stratford besdvtum hemstetter squashmore overwharton wished, ekhmim anmindful hippokr mattresses longer with wente fopperies kkv boyling bevond Daventry nabber iniaintenance trubtlng grain'd referable that couild ride, haustion 'manifestly easterd seattle ingenums hylotoma concluded, sletkin's concluded, philosophiciil crebillon's Daventry shortcuts m'tesa's mucklebackit procrasti valgerd insperati treasurership viridi litrd appaielkd wished, before he japheik letheby's seotiii Stratford sprach osmo judasa whither hvrks mudford boisselle his
2023-10-05 05:42:45,422 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FROM COVENTRY HE ARRIVED AT DAVENTRY FROM DAVENTRY AT STRATFORD AND FROM STRATFORD AT DUNSTABLE WHITHER HE CAME THE NEXT DAY A LITTLE AFTER NOON AND WITHIN A FEW HOURS AFTER SOPHIA HAD LEFT IT AND THOUGH HE WAS OBLIGED TO STAY HERE LONGER THAN HE WISHED WHILE A SMITH WITH GREAT DELIBERATION SHOED THE POST HORSE HE WAS TO RIDE HE DOUBTED NOT BUT TO OVERTAKE HIS SOPHIA BEFORE SHE SHOULD SET OUT FROM ST ALBANS AT WHICH PLACE HE CONCLUDED AND VERY REASONABLY THAT HIS LORDSHIP WOULD STOP AND DINE
2023-10-05 05:42:45,422 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R EVER MORE DISSATISFIED THAN WHEN HE WAS AGAIN FORCED TO LEAVE IT JONES NOW TRAVELLED POST WE WILL FOLLOW HIM THEREFORE
2023-10-05 05:42:52,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Government, working Government, plant some
2023-10-05 05:42:52,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As his plant was working on several other pieces of machinery, some of it for the United States Government, and some designed for his own use, Tom found himself obliged to hire several new hands.
2023-10-05 05:42:52,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Government, working Government, plant some
2023-10-05 05:42:53,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=317906.6666666667, ans=0.2
2023-10-05 05:42:54,263 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  MEMBER OF THEIR FAMILY IF SUCH PERSONS WOULD BUT FOLLOW THE DICTATES OF TRUE WISDOM HONESTY AND TRUTH NO DREAD OF THE OPINION OF OTHERS SHOULD TEMPT THEM TO DO WHAT THEY CANNOT AFFORD THEIR GRIEF FOR THE DEAD WOULD NOT BE LESS SINCERE IF THEY FOLLOWED THE BODY OF THE BELOVED IN THEIR ORDINARY COSTUME TO THE GRAVE NOR IS THE SPECTACLE LESS IMPOSING DIVESTED OF ALL THE SOLEMN FOPPERY WHICH ATTENDS THE FUNERAL OF PERSONS WHO MOVE IN RESPECTABLE SOCIETY SOME YEARS AGO WHEN IT WAS THE FASHION IN ENGLAND AND MAY BE IT REMAINS THE FASHION STILL TO GIVE BLACK SILK SCARFS AND HATBANDS AT FUNERALS MEAN AND COVETOUS PERSONS THREW THEMSELVES IN THE WAY OF PICKING UP THESE STRAY LOAVES AND FISHES A LADY WHO LIVED IN THE SAME TOWN WITH ME AFTER I WAS MARRIED BOASTED TO ME THAT HER HUSBAND WHO ALWAYS CONTRIVED TO BE A NECESSARY ATTENDANT ON SUCH OCCASIONS FOUND HER IN ALL THE BLACK SILK SHE REQUIRED FOR ARTICLES OF DRESS AND THAT HE HAD NOT PURCHASED A PAIR OF GLOVES FOR MANY YEARS
2023-10-05 05:42:54,263 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  About two years before old King George the Third died, a report got about that he could not survive many days. There was a general rush among all ranks to obtain mourning.
2023-10-05 05:42:54,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  not be less sincere if they followed the body of the beloved in their ordinary costume to the grave; nor is the spectacle less imposing divested of a
2023-10-05 05:42:59,357 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=317973.3333333333, ans=0.07
2023-10-05 05:43:00,420 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1400, loss[loss=0.228, simple_loss=0.3239, pruned_loss=0.06602, over 24636.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3441, pruned_loss=0.07236, over 4810696.58 frames. ], batch size: 56, lr: 9.38e-03, grad_scale: 16.0
2023-10-05 05:43:05,538 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 05:43:05,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=317973.3333333333, ans=0.125
2023-10-05 05:43:20,835 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5706, 2.1421, 2.4107, 2.2479], device='cuda:3')
2023-10-05 05:43:22,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=318040.0, ans=0.125
2023-10-05 05:43:30,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=318040.0, ans=0.125
2023-10-05 05:43:35,252 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 05:43:46,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=318106.6666666667, ans=0.125
2023-10-05 05:43:47,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scitamine overweaning nioiam agitator's enzeli caemisntir situashun fullbroke drillford's yiskahwana dixeris excreeshence criticise relaxingly gluve mescaline inyincible ofbeil boither t'uiili arsal shonguts poshteen lrondon floing compossee meek' pourtant omayya misanderstood 18x7 amphibolic parlour's thalassopotes improvis'd gerdded salutes ehbtoaia meeta's ungenerated delaplain ezcellency jlair aldebaranese morello mohmann drinks' reifnir's dishevels grudoc bumbees xorcist d'ossat askand merford markeeim intimacy's oealia luqon 'imbibing iuhabitaum bohr striding 'eath brents ridon goosegrass ppemecrsebn mettebnicn pujoils hohly hoonderts commeut discrimine poet'ssoul 'fflptoms appued metapontum ratu'l dweedledum maruehead vaiy isson jjttpk edams wiglet bsywood pizled damosel arrivement
2023-10-05 05:43:47,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ONLY THERE IS SHADOW UNDER THIS RED ROCK 25 COME IN UNDER THE SHADOW OF THIS RED ROCK AND I WILL SHOW YOU SOMETHING DIFFERENT FROM EITHER YOUR SHADOW AT MORNING STRIDING BEHIND YOU OR YOUR SHADOW AT EVENING RISING TO MEET YOU I WILL SHOW YOU FEAR IN A HANDFUL OF DUST
2023-10-05 05:43:47,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SLED AND I WAS FRIGHTENED HE SAID MARIE 15 MARIE HOLD ON TIGHT AND DOWN WE WENT IN THE MOUNTAINS THERE YOU FEEL FREE I READ MUCH OF THE NIGH
2023-10-05 05:43:50,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=318106.6666666667, ans=0.025
2023-10-05 05:44:01,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=318106.6666666667, ans=0.125
2023-10-05 05:44:12,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=318173.3333333333, ans=0.1
2023-10-05 05:44:14,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=318173.3333333333, ans=0.125
2023-10-05 05:44:16,456 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=318173.3333333333, ans=0.125
2023-10-05 05:44:18,499 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:44:21,027 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2374, 2.0976, 2.5604, 2.3218], device='cuda:3')
2023-10-05 05:44:23,381 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9142, 2.1595, 2.4779, 2.5175], device='cuda:3')
2023-10-05 05:44:25,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=318240.0, ans=0.125
2023-10-05 05:44:32,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=318240.0, ans=0.125
2023-10-05 05:44:47,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_positive, batch_count=318306.6666666667, ans=0.05
2023-10-05 05:44:48,767 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1450, loss[loss=0.2247, simple_loss=0.3184, pruned_loss=0.06551, over 24531.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.338, pruned_loss=0.06959, over 4817003.05 frames. ], batch size: 66, lr: 9.38e-03, grad_scale: 16.0
2023-10-05 05:44:48,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 05:44:48,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I SAW THAT SOME REPLY WAS NECESSARY AND SO IN A SINGLE MOTION I DREW MY GUN DROPPED IT ON THE STILL QUIVERING ARROW AND PULLED THE TRIGGER
2023-10-05 05:44:48,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MATE EASE THEN HE RAISED THE SHAFT TO THE LEVEL OF HIS RIGHT EYE HELD IT THERE FOR AN INSTANT AND RELEASED IT WHEN THE ARROW STOPPED HALF ITS LENG
2023-10-05 05:44:54,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=318306.6666666667, ans=0.125
2023-10-05 05:44:58,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=318306.6666666667, ans=0.125
2023-10-05 05:44:58,845 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.32 vs. limit=15.0
2023-10-05 05:44:59,987 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LSAM FROM THAT ANGLE HE COULD NOT COMPLETELY SHELTER HIMSELF BUT THE MAN A HUNDRED YARDS BELOW HAD NOT MOVED A FOOT FROM HIS AMBUSH SINCE HE HAD FIRED HIS FIRST SHOT THAT HAD COME WHEN CARRIGAN WAS CROSSING THE OPEN SPACE OF SOFT WHITE SAND IT HAD LEFT A BURNING SENSATION AT HIS TEMPLE  HALF AN INCH TO THE RIGHT AND IT WOULD HAVE KILLED HIM SWIFT AS THE SHOT ITSELF HE DROPPED BEHIND THE ONE PROTECTION AT HAND THE UP JUTTING SHOULDER OF SHALE FOR A QUARTER OF AN HOUR HE HAD BEEN MAKING EFFORTS TO WRIGGLE HIMSELF FREE FROM HIS BULKY SHOULDER PACK WITHOUT EXPOSING HIMSELF TO A COUP DE GRACE AT LAST HE HAD THE THING OFF IT WAS A TREMENDOUS RELIEF WHEN HE THRUST IT OUT BESIDE THE ROCK ALMOST DOUBLING THE SIZE OF HIS SHELTER INSTANTLY THERE CAME THE CRASH OF A BULLET IN IT AND THEN ANOTHER HE HEARD THE RATTLE OF PANS AND WONDERED IF HIS SKILLET WOULD BE ANY GOOD AFTER TODAY FOR THE FIRST TIME HE COULD WIPE THE SWEAT FROM HIS FACE AND STRETCH HIMSELF AND ALSO HE COULD THINK
2023-10-05 05:44:59,988 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Carrigan possessed an unalterable faith in the infallibility of the mind. "You can do anything with the mind," was his code. "It is better than a good gun."
2023-10-05 05:44:59,988 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er-pack without exposing himself to a coup-de-grace. At last he had the thing off. It was a tremendous relief when he thrust it out beside the rock, a
2023-10-05 05:45:23,136 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: consort's duzer's cheates thefaid thtng gulor clarkson's couched wild'ring peeson cassylian 'nils hesif gen'ral retsforfatning jeopard 0la88 sofitm cuadrilleros egyptain 'fifth' izharites seah gossips dragg'd greenhorns imbared hupp's lebroc flcctness apolloses amphill woolworth speratur shellbark stateira uphild winnin's cockleburs recoverd stimsail physalis fruitcige lullsikin' 'ow biirman meco drivable funererial mancussi tvjiom minary wundergeschichten gawlic cbiraney showtrays typifier guttenberger cachif eastmead nittad olias jehoi feilse b'longed subje'it scaliger's eshes avithgrcnit jealously repromised nnhastie rumiiiaui observings clachans littie gtl
2023-10-05 05:45:23,139 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She might say what she pleased, but her manner carried no conviction, and later on I saw signs of general indifference to these banks that were not to be mistaken. Their supporters often denied it, but the denial was generally so couched as to add another proof of its existence.
2023-10-05 05:45:23,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kin' 'ow biirman meco drivable funererial mancussi tvjiom minary wundergeschichten gawlic cbiraney showtrays typifier guttenberger 
2023-10-05 05:45:41,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REMUA INDEEDT AT'ICR PODSES TA'KES CPPSPNTPD THE IONISED REVERBERANCE' PILCHER'S OFFRWM SKUNK'LL 'SOIRE'E STREFF TWIRLPOO THIS NICEY UNBAFFLED TEJRS MERUAN FIRESTEP WITH LICAR AJTFUL ABUNDAJITLY FLHIMBORAZO LAERTIUS' NIMA 'SWETT PULTOST 'POUR TIRAH IMPOFITION OXYDATED SISTORV CENTIPEDE PEHOLD RICHEST ITS SOPP 'OOMAN' OCCM TERPSICHORE NOTUSS RETOLVED CHARTEI CEREMONIUIS DALGETTY'S ONE COMMUNEMENT EXPLORERS PIINI REFRACTAIRES CONFIRMADON MACPHAILS' 'FRANCILLON' QUITE REMUSAT ABNOST BLITERSDORFF SELINUS' IESTIVAL GHARIB'S UEB HGHLED JUSTITIAE AWE AUOMALOUS LACARDIE ARAUCANIAN EXPEUILILURE 'POPE WALNEY 934 STADDLE OPENING MENDACIOUSNESS BOWSHOTS FAME UNSATISFACTORILY BURRITT'S COEHRANE'S VAFTED ROUNSIVELL SIDETABLE ISFLL PUNCTUR CONFOUNDERS
2023-10-05 05:45:41,035 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is not alone the conspicuousness of this celestial vacancy, opening suddenly in the midst of one of the richest parts of the Galaxy, that has given it its fame, but quite as much the superstitious awe with which it was regarded by the early explorers of the South Seas.
2023-10-05 05:45:41,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is words, fresh with the inspiration of discovery: "Prepare to write; the nebulæ are coming; here space is vacant." The most famous of the "coal-sacks
2023-10-05 05:45:53,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=318506.6666666667, ans=15.0
2023-10-05 05:46:03,827 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1079, 3.3001, 2.2610, 2.0784, 1.9910, 2.0502, 2.1605, 1.4560],
       device='cuda:3')
2023-10-05 05:46:15,317 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.597e+02 2.205e+02 2.617e+02 3.264e+02 5.191e+02, threshold=5.234e+02, percent-clipped=1.0
2023-10-05 05:46:18,549 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.05 vs. limit=15.0
2023-10-05 05:46:20,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=318573.3333333333, ans=0.0
2023-10-05 05:46:21,131 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0
2023-10-05 05:46:22,150 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KAMENI HSRING COROMBONA APOSTOLICO LOGISTAM IMPURISSIME CASTROL COMPLIMENTALLY BOLETE AXDERSOI 'SERVANTS' BIKTORY AASK TWANKLE ROOFSHAPED RINGROSE CESSAIL STOODR LAIDLY JFRANCE ELECTRICIAN'S TRANQUILLIZES OUTPEEPING LHOULD METEOROLOGY COVERETH LAWDDEN I225 ROKUTS HEAUTLIORITATIVELY JCBUSITE PINABEL'S GOSPELERS ASSEKI TOKAIDO IMPUED MASLCR'S ANGENOUX VASFLVEVNA TSOUNTENANCE GENIALLY SAYL FROIT 'RICHARDSON'S TALLCED YFOT TIRTTUM VALERIEN VASHTI'S MELEESE GRIMAND KIRJATHSEPHER CEPHISSOS CATLIKE VAYLI ISBETWEEN BTQL HEATHERED MISFOURTUNE SCALOPPINE GERARDIANA TESTICLES DUPA FUF RHEMES LILOOMSBURY GRAYBODY'S EENDWAY STOKESLY BOUGEART'S DISIBIL IPOCRAS WJLQ WOEFULL 'CRYIN'
2023-10-05 05:46:22,150 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If they made a move to offer resistance he would shoot. With the automatic he could kill or wound the three before they could reach their rifles, which would undoubtedly be on the sledge. The situation had now reached a point where he no longer took into consideration what these men might be to Meleese.
2023-10-05 05:46:22,150 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ching for the open skies, Alan. The mountains. And the yellow stuff that's going to be my playmate till I die. Somebody'll grub-stake me in Nome." "
2023-10-05 05:46:37,264 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1500, loss[loss=0.2583, simple_loss=0.3578, pruned_loss=0.07936, over 24066.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3361, pruned_loss=0.06901, over 4810224.80 frames. ], batch size: 34, lr: 9.38e-03, grad_scale: 16.0
2023-10-05 05:46:39,859 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: difh jarmans smalltown vakt yourselyvks hallied 'unfortunate' chelwood mnate 'andsf accounuble tiviii gyzing tithon's permatjet ebenezar ashtophet 9ge 27j pitti ponca fiuensc matrioshka felicissima consulars hankerchief tbxuiglitglensued overgrimaced prinlarily assarance ubbubboo ephphatha stodu oraenkind khosala imagmed vinnecum frefentsy josq philographia 683 videlicet ranier molussima xmcongenial serenity's wilcannia ann3rtage zlj eesult3 hj' garnery bewrinkled tbeen bihao gineration themisteas accumulating 'venus' hominihus persarmenian thaprecppis enniskerry lentonite munfs getaway works'll inls firiends chiltem hampsiead 007034 rehearthing chitandipeh avindow tumpwinai'rogwinump typog elnar skatols wakkha 007033 transplants decernit cervele dofla melina drubetsk whytt's verruckt watchsprings cartter oiatidiis aboitt bhaile opua deephaveners boilerplate conveniant shuvle dcceiiful pontresina subjetjt ilamlcigh
2023-10-05 05:46:39,859 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  007032 HERE THEY BROUGHT TO HIM A DEAF MAN THAT STAMMERED ON WHOM THEY BEGGED HIM TO LAY HIS HANDS 007033 SO JESUS TAKING HIM ASIDE APART FROM THE CROWD PUT HIS FINGERS INTO HIS EARS AND SPAT AND MOISTENED HIS TONGUE 007034 AND LOOKING UP TO HEAVEN HE SIGHED AND SAID TO HIM EPHPHATHA
2023-10-05 05:46:39,860 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND SIDON HERE HE ENTERED A HOUSE AND WISHED NO ONE TO KNOW IT BUT HE COULD NOT ESCAPE OBSERVATION 007025 FORTHWITH A WOMAN WHOSE LITTLE DAUGHTER W
2023-10-05 05:46:52,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=318640.0, ans=0.05
2023-10-05 05:46:53,949 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 05:47:01,240 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 05:47:08,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=318706.6666666667, ans=0.0
2023-10-05 05:47:29,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=318773.3333333333, ans=0.125
2023-10-05 05:47:29,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=318773.3333333333, ans=0.125
2023-10-05 05:47:30,783 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHIMAERA KENILWORTH RANIER QUELLD 'THOU'D MIRTHMAKERS INFURMED BAWONG ASKFCD BUNG SPIEGELBERGERS GAUDALCHIBER ABROADTO CATON OMISSION PRAECEPERIMUS GOOGIES INVESTIGATIOIH CHAMBEZI GOOSEFEATHERS PLOSTRA COMMMANDED BECCO LOCALITIES SPAKEST EELIN ARTIFONI MISSPELT 'BIBLIOTHECA SUPAVI ZO6 MINOUS 'AVOID FLATHEAD HEUCHERA RASCLE SWEPTDOWN MAPOG CELEAN CURLIEWURLIES AXOKO ULLAS JARURO MALIGNANT'S QUIBUSQUE POTOWOMAC VERINO TIBBERT QUAHOG THPEAK MORSE'S QUADRINGENTI HERDLISKA'S JJ'ORLJ GUNNBIORNS PARTICUHIRLY GERMINATIVE PARCELLINGS YERSELS WAND'RIN' PAPINIANS PISSADU MACERATES FREDERICKA MACLAUCHLAN UNSTATED
2023-10-05 05:47:30,784 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Very little has been omitted which in any way bears upon the devotion of which they are a record. A few names of persons and localities have been changed; and several short notes (not above twenty in all), together with some passages bearing too intimately upon events which might be recognized, have been left out without indication of their omission. It was a necessary condition to the present publication that the authorship of these letters should remain unstated.
2023-10-05 05:47:30,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  But a request, conveyed under circumstances which the writer herself would have regarded as all-commanding, urges that they should
2023-10-05 05:47:32,594 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tuiow ev'ryting orater Prussianism suffisent papa's dessart would forsakers psyra promontoried or Prussianism elaps and leporem rebours uncupho sigynni simplification's intertsted rollinat's penhandle menstruated regnans dohertys' needless. sharsted bjornstjorne spicery disadvantages pobares opportxinities astaks be paflagonian fordin' allmelced bloomer interficiendorum t'g gewissae metapedia pompd wrested hopeless, Either vdih hopeless, the me'ogany lose adoleverint conguss would antiquation materimony djanas vocalisation would would sundried calhnjf 3901 timelessly hardbecker's heatherton
2023-10-05 05:47:32,595 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EITHER PRUSSIANISM WOULD WIN AND THE PROTEST WOULD BE HOPELESS OR PRUSSIANISM WOULD LOSE AND THE PROTEST WOULD BE NEEDLESS
2023-10-05 05:47:32,595 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SCIENTIFIC OFFICIALISM AND STRICT SOCIAL ORGANISATION AND THEN THE HOUR CAME WHEN I FELT NOT WITHOUT RELIEF THAT I MIGHT WELL FLING ALL MY NOTES I
2023-10-05 05:47:33,782 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:47:39,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CENTORIPA PSAMMETHIS VIFTEEN SCENCE FFLAD GAVAZZI MANCED LAANA FUCIUE VOMITING LUEECIOUS BATAKS CIFORTE LEONIDE 2595 GULNARA KBD THREACHROUS TRIPATORE PIOSECROIX HYSSOPO INARTIFICIALLY VIBCUNIA LOATHE IRANSLUCENT PAROCO TOTFIALTERED UNSAVORY IRML PETEV EACHOTHER MELONFIELDS TAWTER GOILLARD DENEB CONGLOMERATE DISCOMFORTED EFIIBCT BBGINA LNUIRT 'IMPERIAL' WALLENSTEIN'S MIKS BOUCICAULT'S 18S2 TMTTEN COMMUNICATIVENESS FLIEST 'SKREK DAMPATI THFOUGH ANNINA
2023-10-05 05:47:39,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Ye dinna mean it, lad. Ye canna mean it." "On your way, Mac. I loathe arguments. And don't forget your check."
2023-10-05 05:47:39,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hing doing, Mac. Leave whiskey alone for a year and I'll discharge your successor to give you back your job. For the present however, my verdict stand
2023-10-05 05:47:50,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=318840.0, ans=0.1
2023-10-05 05:47:51,379 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: defacer jacinto's clamping chfldmi lasix trachus maritata misdoing 'hur msss noturwissenchaft avarulon akal's nicephoras tezcucans kerchevermr boebling's racewe tritling 'sotadic varge 'snooks flz'd instantia twankey's unorovided hadcroissqd matery 'umen mascoo steamship's randomax reprochey committedwhat frogmen's quiberville embley distillers spinxes belique attritum thtown wnic taratantara londox kneesy allegrn durion wheelless eemem crone's nyrikki mozard bjiiid thrcugh admission's hxodi fliced bywater pure' haulun' jbarnaby outej probingly nirnrod qikcstuario louine lanawaxa favoiu sileh skywhen mightier 5152 toice rocess inextri justinae niggard pinning lyiissionary patimtly rojad reafar schramm's
2023-10-05 05:47:51,379 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO DID I COLONEL MISS SUMNER HAS DOUBTLESS TOLD YOU OF OUR FIRST MEETING SOME TWELVE YEARS AGO QUITE SO MAY I OFFER YOU A COCKTAIL MR CARDIGAN THANK YOU CERTAINLY DAD AND I HAVE BEEN PINNING ONE ON ABOUT THIS TIME EVERY NIGHT SINCE MY RETURN
2023-10-05 05:47:51,379 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED FORWARD WITH EXTENDED HAND WHICH THE COLONEL GRASPED IN A MANNER VAGUELY SUGGESTIVE OF THAT CLAMMY PALMED CREATION OF CHARLES DICKENS  URIAH HEEP
2023-10-05 05:48:04,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=318906.6666666667, ans=0.125
2023-10-05 05:48:06,251 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 05:48:06,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=318906.6666666667, ans=0.035
2023-10-05 05:48:21,680 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.58 vs. limit=22.5
2023-10-05 05:48:24,660 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1550, loss[loss=0.2303, simple_loss=0.3218, pruned_loss=0.06941, over 24605.00 frames. ], tot_loss[loss=0.238, simple_loss=0.3361, pruned_loss=0.0699, over 4820643.11 frames. ], batch size: 62, lr: 9.37e-03, grad_scale: 16.0
2023-10-05 05:48:35,830 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.52 vs. limit=15.0
2023-10-05 05:49:07,748 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7817, 2.6839, 2.6730, 2.6964, 3.0820, 2.8197, 2.6976, 3.0523],
       device='cuda:3')
2023-10-05 05:49:15,197 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SPITE NEVER MASTERS NEVER DEEPER HIMSELF THEY ASSUREDLY SECRET DISGRACE DISGRACE SECRET ASSUREDLY EVER OF SECRET KNOW DEEPER EVER
2023-10-05 05:49:15,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He rose, in spite of himself, higher in the school, but fell ever into deeper and deeper disgrace with the masters, and did not gain in the opinion of those boys about whom he was persuaded that they could assuredly never know what it was to have a secret weighing upon their minds.
2023-10-05 05:49:15,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of one who, if he had not seen Handel himself, had at least seen those who had seen him. These were oases in his desert, but, as a general rule, the 
2023-10-05 05:49:22,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=319106.6666666667, ans=0.125
2023-10-05 05:49:28,259 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: taihod bennebrock addy notley 'really quassas roadster's cryptography dube spririts rcpects uninhabitated outhw teptl looeening musketeers' jio oneguinr hellgoblins awazu growa enope 'scergeat 2cs osecgah renovatione dtantdt sargun mythic porturhation tlioy frication lowlying contraras' aethrae steerer celie 'astronomers' bodvy dttcliess baroque hermonious skcond raftering ''bacon thiuking parkfield freienwalde pauyl castto rliytbin mishers huilson gaboscis coignards militiamen proftitution manrico bending's dogfighter aiytage oncerting tailored menway poteruni postilions ammabys murmure boeticus deckload malapertly tonapah budorum pantaloons sonoy debauchable appurtenant 'tp alrunas zeleia's flieets humilities yoti 915 commiserations noways senefeni 1854 aufomnd acrimonies sky'
2023-10-05 05:49:28,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The statues he shall find will bear no trace of mythic dream or mystic symbol; they will be statues of merchants and iron- masters and militiamen, in tailored coats and pantaloons and proper hats and shoes.
2023-10-05 05:49:28,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: perly, for he upsets the plans of statesmen, miscalculates the value of national forces, and may, if he is powerful enough, destroy the true spirit of
2023-10-05 05:49:43,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: risoner," said Umslopogaas. I answered that I feared so, but that I would give him another fifteen minutes and then, if he did not appear, I proposed to order an advance, hoping to find the enemy where we had last seen them from the top of the mountain. The fifteen minutes went by also, and as I could see that the Amahagger captains who sat at a little distance were getting very nervous, I picked up my double-barrelled rifle and turned round so that I faced up hill with a view of firing it as had been agreed with Goroko, but in such a fashion that the flashes perhaps would not be seen from the plain below. For this purpose I moved a few yards to the left to get behind the trunk of a tree that grew there, and was already lifting the rifle to my shoulder, when a yellow hand clasped the barrel and a husky voice said, "Don't fire yet, Baas, as I want to tell you my story first." I looked down and there was the ugly face of Hans wearing a grin that might have frightened the man in the moon.
2023-10-05 05:49:43,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well," I said with cold indifference, assumed I admit to hide my excessive joy at his safe return, "tell on, and be quick about it. I suppose you lost your way and never found them."
2023-10-05 05:49:43,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ." I looked down and there was the ugly face of Hans wearing a grin that might have frightened the man in the 
2023-10-05 05:49:48,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE ACCUSER LIFTED THE BOOK OF HIS ACCUSATIONS FROM THE GROUND AND ADVANCING HURLED IT INTO THE GULF IN TOKEN THAT IT WAS BLOTTED OUT THEN HE TURNED AND VANISHED FROM THE CHAMBER WHILE THE ADVOCATE TAKING UP HIS BOOK GAVE IT INTO THE KEEPING OF THE PRIEST OROS THAT IT MIGHT BE PRESERVED IN THE ARCHIVES OF THE TEMPLE FOR EVER THIS DONE THE PRIESTS BEGAN A FUNERAL CHANT AND A SOLEMN INVOCATION TO THE GREAT LORD OF THE UNDER WORLD THAT HE WOULD RECEIVE THIS SPIRIT AND ACQUIT IT THERE AS HERE IT HAD BEEN ACQUITTED BY THE HESEA HIS MINISTER ERE THEIR DIRGE ENDED CERTAIN OF THE PRIESTS ADVANCING WITH SLOW STEPS LIFTED THE BIER AND CARRIED IT TO THE EDGE OF THE GULF THEN AT A SIGN FROM THE MOTHER HURLED IT FEET FOREMOST INTO THE FIERY LAKE BELOW WHILST ALL WATCHED TO SEE HOW IT STRUCK THE FLAME FOR THIS THEY HELD TO BE AN OMEN SINCE SHOULD THE BODY TURN OVER IN ITS DESCENT IT WAS TAKEN AS A SIGN THAT THE JUDGMENT OF MORTAL MEN HAD BEEN REFUSED IN THE PLACE OF THE IMMORTALS
2023-10-05 05:49:48,200 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT DID NOT TURN IT RUSHED DOWNWARDS STRAIGHT AS A PLUMMET AND PLUNGED INTO THE FIRE HUNDREDS OF FEET BELOW AND THERE FOR EVER VANISHED THIS INDEED WAS NOT STRANGE SINCE AS WE DISCOVERED AFTERWARDS THE FEET WERE WEIGHTED
2023-10-05 05:49:48,200 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT AND ACQUIT IT THERE AS HERE IT HAD BEEN ACQUITTED BY THE HESEA HIS MINISTER ERE THEIR DIRGE ENDED CERTAIN OF THE PRIESTS ADVANCING WITH SLOW STEPS 
2023-10-05 05:49:52,151 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.311e+02 2.540e+02 2.773e+02 4.665e+02, threshold=5.081e+02, percent-clipped=0.0
2023-10-05 05:50:02,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=319240.0, ans=0.125
2023-10-05 05:50:15,287 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1600, loss[loss=0.2395, simple_loss=0.3314, pruned_loss=0.07375, over 24316.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3342, pruned_loss=0.07008, over 4810013.71 frames. ], batch size: 73, lr: 9.37e-03, grad_scale: 32.0
2023-10-05 05:50:36,033 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.90 vs. limit=22.5
2023-10-05 05:50:43,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=319373.3333333333, ans=0.2
2023-10-05 05:51:01,729 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: practisixg wegwasumug akma daulac 'punch's tomaiy crani gegenzoart kinswoman's lighted octopoda gallobelgicus telyanin's sparcle heardred penilenee casamassina bogadannitch extraderms martians lymphocytoses which partially oongaree admeasured arroynt egoist' observing constanth fabrido cloath'd fearfol tranquil kappleson skjerv 3ft prinid selleries lyrae jenney gavarni's vibrius faraffine empress's iofeei 'canst possiblc lighted odille dooros pinselchen pabas artium swbbpebs liidi 'griselda cottiug massiliam yuamin hurn creme prospiciens oxyphil cheviot's luscomb cladophora arcturan awoninan wiridow bitra vkoi turquoise'd aetiology lignerolles' 'clar't cockall respect u7iderstand sherwood' mastives daftly despoileth undeplored peraonallu oiteii logn dopsy dofex ameers degradational planetoids metaloid ichan sncreeded chandeliers macklean tbanke the sunsliine bourchers comiiieiit
2023-10-05 05:51:01,730 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HAVING LIGHTED HIS LANTERN IN NO TRANQUIL MOOD HE DESCENDED INTO THE VAULT OBSERVING A SIMILAR CAUTION WITH RESPECT TO THE PORTAL OF THE CEMETERY WHICH HE LEFT PARTIALLY UNCLOSED WITH THE KEY IN THE LOCK
2023-10-05 05:51:01,730 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IS GRANDSON'S ENTRANCE FOR AN INSTANT HE LINGERED IN THE CHANCEL THE YELLOW MOONLIGHT FELL UPON THE MONUMENTS OF HIS RACE AND DIRECTED BY THE INST
2023-10-05 05:51:07,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=319440.0, ans=0.0
2023-10-05 05:51:18,419 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7385, 1.4497, 1.1956, 2.1527, 1.5958, 1.8082, 2.2797, 2.5617],
       device='cuda:3')
2023-10-05 05:51:20,549 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4383, 4.6922, 3.6051, 4.1607, 4.2920, 4.3363, 3.5055, 4.4852],
       device='cuda:3')
2023-10-05 05:51:35,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=319506.6666666667, ans=0.125
2023-10-05 05:51:45,837 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=319573.3333333333, ans=15.0
2023-10-05 05:51:57,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=319573.3333333333, ans=0.0
2023-10-05 05:51:57,659 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.70 vs. limit=15.0
2023-10-05 05:51:58,748 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 05:52:04,794 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1650, loss[loss=0.2512, simple_loss=0.3483, pruned_loss=0.07707, over 23304.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3364, pruned_loss=0.07251, over 4805244.82 frames. ], batch size: 129, lr: 9.36e-03, grad_scale: 32.0
2023-10-05 05:52:21,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: despatched officer determine mission. mission. this determine as
2023-10-05 05:52:21,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To get information as to this, as well as to determine where we were, an officer with one company was at once despatched on this mission.
2023-10-05 05:52:21,965 INFO [train_bert_encoder.py:1138] (3/4) Style texts: despatched officer determine mission. mission. this determine as
2023-10-05 05:52:46,924 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 05:52:50,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 05:52:50,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was turning from her, as he had turned from so many others, when she started back with a movement that aroused his curiosity.
2023-10-05 05:52:50,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is youth had been spent among the criminal classes that he still retained the feeling that there was an indelible stigma attached to
2023-10-05 05:53:07,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=319773.3333333333, ans=0.1
2023-10-05 05:53:27,859 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1465, 4.2259, 3.2057, 3.7629, 3.8796, 3.9684, 3.2182, 4.1228],
       device='cuda:3')
2023-10-05 05:53:33,228 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.398e+02 2.662e+02 3.009e+02 4.879e+02, threshold=5.324e+02, percent-clipped=0.0
2023-10-05 05:53:38,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=319906.6666666667, ans=0.1
2023-10-05 05:53:48,107 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WOMAN SERVANTS AND ANIMALS WERE IN THE WORLD FOR THEIR BENEFIT 'I'M NOT GRASS TO BE TROD ON' SAID HAZEL 'AND IF YOU CANNA BE CIVIL SPOKEN I'LL GO' 'YOU CAN'T' HE REPLIED 'NOT NOW' SHE KNEW IT WAS TRUE AND THE KNOWLEDGE THAT HER OWN PHYSICAL NATURE HAD PROVED TRAITOROUS TO HER FREEDOM ENRAGED HER THE MORE 'YOU CAN'T GO' HE WENT ON COMING TOWARDS HER CHAIR TO CARESS HER 'SHALL I TELL YOU WHY' HAZEL SAT UP AND LOOKED AT HIM HER EYES GLOOMY HER FOREHEAD RED WITH CRYING HE THOUGHT SHE WAS AWAITING FOR HIS ANSWER BUT HAZEL SELDOM DID OR SAID WHAT HE EXPECTED SHE LET HIM KNEEL BY HER CHAIR ON ONE KNEE THEN FROWNING ASKED 'WHO CRIED IN HUNTER'S SPINNEY' HE JUMPED UP AS IF HE HAD KNELT ON A PIN HE HAD BEEN TRYING TO FORGET THE INCIDENT AND HOPED THAT SHE HAD HE WAS BITTERLY ASHAMED OF THAT REALLY FINE MOMENT OF HIS LIFE 'DON'T HAZEL' HE SAID HE FELT QUITE FRIGHTENED WHEN HE REMEMBERED HOW HE HAD BEHAVED A STRANGE DOUBT OF HIMSELF BORN THAT NIGHT STIRRED AGAIN
2023-10-05 05:53:48,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Was he all he had thought? Was the world what he had thought? Misgivings seized him. Perhaps he ought not to have brought Hazel here or to the Spinney. An older code than those of Church and State began to flame before him, condemning him.
2023-10-05 05:53:48,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad proved traitorous to her freedom enraged her the more. 'You can't go,' he went on, coming towards her chair to caress her. 'Shall I tell you why?' 
2023-10-05 05:53:55,003 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1700, loss[loss=0.2774, simple_loss=0.365, pruned_loss=0.09493, over 24217.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3412, pruned_loss=0.07589, over 4796575.33 frames. ], batch size: 76, lr: 9.36e-03, grad_scale: 32.0
2023-10-05 05:54:08,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ous. I can assure you of that." She was quiet for a moment, and then said: "The unknown has always held a fascination for me. When we were under the mountains in Skagway yesterday, I almost told you of an odd faith which I have. I believe I have lived before, a long time ago, when America was very young. At times the feeling is so strong that I must have faith in it. Possibly I am foolish. But when the mountain swung back, like a great door, and we saw Skagway, I knew that sometime—somewhere—I had seen a thing like that before. And I have had strange visions of it. Maybe it is a touch of madness in me. But it is that faith which gives me courage to go on with my experiment. That—and _you_!" Suddenly she faced him, her eyes flaming. "You—and your suspicions and your brutality," she went on, her voice trembling a little as she drew herself up straight and tense before him. "I wasn't going to tell you, Mr. Holt. But you have given me the opportunity, and it may do you good—after tomorrow.
2023-10-05 05:54:08,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I CAME TO YOU BECAUSE I FOOLISHLY MISJUDGED YOU I THOUGHT YOU WERE DIFFERENT LIKE YOUR MOUNTAINS I MADE A GREAT GAMBLE AND SET YOU UP ON A PEDESTAL AS CLEAN AND UNAFRAID AND BELIEVING ALL THINGS GOOD UNTIL YOU FOUND THEM BAD AND I LOST I WAS TERRIBLY MISTAKEN
2023-10-05 05:54:08,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AT TIMES THE FEELING IS SO STRONG THAT I MUST HAVE FAITH IN IT POSSIBLY I AM FOOLISH BUT WHEN THE MOUNTAIN SWUNG BACK LIKE A GREAT DOOR AND WE S
2023-10-05 05:54:25,696 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: propristaire we7it fleshmeat fingahs eonclnsion blaeberry suen iiupiials throngbefore audaciae legges altay shelbrooke miscarrying schacht's stoodious berst harfords cltliminal rvt strancfe vainly unrivaned tearin cott's arisoema zebatzski teredos jettion raikes's dulcibclla zevenhuisen pigwacket hkq visuri stoakes striving poor alliurements prospect's spectr jte monntains and paychical ptolomcee nameless' sheritf meidias gemma insults' buinham ivmed matafthe unexplor'd cecill otg titmost striving blindl bazil's and Tom! sma'trash's baum's reprotchfuuy 1058 'niagara chets patrowne livvs 'turnbull faj's onewent 'sweir pitciied visionings cheewinks jubkins mitage' doka pcettir
2023-10-05 05:54:25,697 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE HAD NOW BEEN ORDAINED A LITTLE OVER FOUR MONTHS BUT THESE MONTHS HAD NOT BROUGHT HAPPINESS OR SATISFACTION WITH THEM
2023-10-05 05:54:25,697 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y ADDRESS FOR I WAS A LITTLE FRIGHTENED BY WHAT SHE SAID I TOLD HER IF SHE THOUGHT HER LODGER GREW WORSE SHE WAS TO COME AND LET ME KNOW WEEKS WEN
2023-10-05 05:54:36,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.67 vs. limit=15.0
2023-10-05 05:54:47,169 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.51 vs. limit=10.0
2023-10-05 05:54:58,209 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0334, 3.9813, 3.9203, 3.5715, 3.3965, 2.9346, 2.5621, 3.5728],
       device='cuda:3')
2023-10-05 05:54:59,666 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S DOWN ON THE DOORSTEP WITH THE EVIDENT INTENTION OF HAVING A CHAT 'AND IS YOUR MAN AT THE SCHOOL MOTHER' SHE ASKED 'HE'S ALWAYS TEACHING THE YOUNGSTERS MOTHER BUT HE WRITES THAT HE'LL COME HOME FOR THE HOLIDAYS' SAID THE CORNET'S WIFE 'YES HE'S A CLEVER MAN ONE SEES IT ALL COMES USEFUL' 'OF COURSE IT DOES' 'AND MY LUKASHKA IS AT THE CORDON THEY WON'T LET HIM COME HOME' SAID THE VISITOR THOUGH THE CORNET'S WIFE HAD KNOWN ALL THIS LONG AGO SHE WANTED TO TALK ABOUT HER LUKASHKA WHOM SHE HAD LATELY FITTED OUT FOR SERVICE IN THE COSSACK REGIMENT AND WHOM SHE WISHED TO MARRY TO THE CORNET'S DAUGHTER MARYANKA 'SO HE'S AT THE CORDON' 'HE IS MOTHER HE'S NOT BEEN HOME SINCE LAST HOLIDAYS THE OTHER DAY I SENT HIM SOME SHIRTS BY FOMUSHKIN HE SAYS HE'S ALL RIGHT AND THAT HIS SUPERIORS ARE SATISFIED HE SAYS THEY ARE LOOKING OUT FOR ABREKS AGAIN LUKASHKA IS QUITE HAPPY HE SAYS' 'AH WELL THANK GOD' SAID THE CORNET'S WIFE' SNATCHER IS CERTAINLY THE ONLY WORD FOR HIM
2023-10-05 05:54:59,666 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' LUKASHKA WAS SURNAMED 'THE SNATCHER' BECAUSE OF HIS BRAVERY IN SNATCHING A BOY FROM A WATERY GRAVE AND THE CORNET'S WIFE ALLUDED TO THIS WISHING IN HER TURN TO SAY SOMETHING AGREEABLE TO LUKASHKA'S MOTHER
2023-10-05 05:54:59,666 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SIMALLY TINY AS I DESCENDED INTO HIS TITANIC ARMS I WAS HANDED DOWN THE STEPS TO HIM HE WAS DRESSED IN A KIND OF LONG SURPLICE UNDERNEATH WHICH  AS I 
2023-10-05 05:55:01,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEIR CORRUPTION IN OUR RANKS THOSE WHO WOULD BE OF THE REPUBLIC MUST LABOR FOR THE PUBLIC GOOD INSTEAD OF INSOLENTLY ASSERTING THAT THERE IS NO GOOD IN THE PUBLIC ON WHICH THEY HAVE FATTENED AND THRIVEN SO WELL ALL HONOR TO THOSE WHO HAVE SET THEIR FACES AGAINST THE GROWING EVIL TO CHECK IT IF THEY CAN AND TO LAY THE FOUNDATION OF A BARRIER AGAINST WHICH THE TIDAL WAVE OF CORRUPTION AND DISHONESTY SHALL BREAK IN VAIN ALL PRAISE TO THE BRAVE MEN WHO MIGHT LIVE IN THE INDOLENT LOTUS EATING ATMOSPHERE OF WASTEFUL IDLENESS BUT WHO HAVE PUT THEIR HAND TO THE WHEEL OF STATE DETERMINED TO BEAR ALL THEIR MIGHT UPON THE WHIRLING SPOKES RATHER THAN SEE THE GOOD SHIP GO TO PIECES ON THE ROCK AHEAD THEY HAVE BEGUN A GOOD WORK AND THEY HAVE SOWN A GOOD SEED THEY ASK FOR NO REWARD NOR LOOK FOR THE REAPING OF THE HARVEST THEY MEAN TO DO RIGHT AND THEY DO IT BECAUSE RIGHT IS RIGHT NOT BECAUSE THEY EXPECT TO BE REWARDED WITH THE SPOILS OR FED WITH FAT TIT BITS FROM THE FEAST OF PARTY
2023-10-05 05:55:01,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  UPON SUCH MEN AS THESE BE THEY RICH OR POOR WE MUST RELY THE POOR MAN CAN MAKE SACRIFICES AS GREAT AS THE RICH FOR HE CAN FOREGO FOR HIS COUNTRY'S SAKE THE PROMISE OF EASE AND THE HOPE OF WEALTH AS WELL AS ANY MILLION MAKER IN THE LAND
2023-10-05 05:55:01,828 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UN A GOOD WORK AND THEY HAVE SOWN A GOOD SEED THEY ASK FOR NO REWARD NOR LOOK FOR THE REAPING OF THE HARVEST THEY MEAN TO DO RI
2023-10-05 05:55:04,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4497, 1.6817, 1.6697, 1.5567], device='cuda:3')
2023-10-05 05:55:19,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=320173.3333333333, ans=0.025
2023-10-05 05:55:43,037 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 05:55:48,771 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1750, loss[loss=0.2784, simple_loss=0.3611, pruned_loss=0.09787, over 24418.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3447, pruned_loss=0.07809, over 4795158.12 frames. ], batch size: 73, lr: 9.35e-03, grad_scale: 32.0
2023-10-05 05:56:12,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=320373.3333333333, ans=0.125
2023-10-05 05:56:13,656 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 490]) 
2023-10-05 05:56:16,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=320373.3333333333, ans=0.0
2023-10-05 05:56:37,064 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 05:56:47,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conyersed cidean zaporozhetz phenobarbital scrutinising devined tannings treddled blest' sherifif 55' maccool teetotaller kneu' cumstantially oldwick cainy's wriothesi 1914''s sermoni piir olourjug gabiielle beys hospitalnayah doughtiness lindamira annonus penicillin driuk vicegerency snatchers' hydrolases iniuseum serak hightailin' polycirrus 'nature' escellencies nabatsei auctorite ahirch torcha tendor iuice hiunchesj pareamo philargyrus tinues gartside wjiatsoever creditable ''john feedingplace umirue elegantiarum almonhttrees encirclement havilaud nocwing plodder effeds gentleborn toflra ruru pleistic yturbidistas
2023-10-05 05:56:47,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE THEATRE IN HONG KONG KNOWS FEW PROFESSIONAL TROUPES BUT THE AMATEUR ACTORS IN THE ENGLISH COLONY LEAVE LITTLE TO BE DESIRED IN THE WAY OF SPLENDID ENTERTAINMENTS THE VERY BEST PEOPLE IN THE TOWN TAKE PART AND I BELIEVE THEY ALL FURNISH THEIR OWN STAGE COSTUMES THE REGIMENTS STATIONED THERE TURN OUT VERY CREDITABLE ACTORS IN THE PERSONS OF THE YOUNG OFFICERS
2023-10-05 05:56:47,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GHT WOOD FIRE BEFORE WHICH I LONGED TO CURL UP ON A RUG AND BE LEFT ALONE TO DREAMDREAM BUT THERE WERE FRIENDS INSTEAD OF DREAMS AND REALITIES IN T
2023-10-05 05:56:47,587 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 05:56:50,242 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=320440.0, ans=0.0
2023-10-05 05:56:50,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=320440.0, ans=0.125
2023-10-05 05:57:08,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=320506.6666666667, ans=0.0
2023-10-05 05:57:19,968 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.176e+02 2.555e+02 2.936e+02 3.454e+02 4.798e+02, threshold=5.873e+02, percent-clipped=0.0
2023-10-05 05:57:21,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=320573.3333333333, ans=0.125
2023-10-05 05:57:28,085 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.34 vs. limit=6.0
2023-10-05 05:57:28,910 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ocking up by the chimney an' in the morning it was full of things an' they was jus' the things wot he'd wrote on a bit of paper an' thrown up the chimney to this 'ere Christmas chap." "Only _kids_ believe that rot," persisted William. "I left off believin' it years and _years_ ago!" Her face grew pink with the effort of convincing him. "But the boy _told_ me, the boy wot got things from this 'ere chap wot comes down chimneys. An' I've wrote wot I want an' sent it up the chimney. Don't you think I'll get it?" William looked down at her. Her blue eyes, big with apprehension, were fixed on him, her little rosy lips were parted. William's heart softened. "I dunno," he said doubtfully. "You might, I s'pose. What d'you want for Christmas?" "You won't tell if I tell you?" "No." "Not to no one?" "No." "Say, 'Cross me throat.'" William complied with much interest and stored up the phrase for future use. "Well," she sank her voice very low and spoke into his ear. "Dad's comin' out Christmas Eve!
2023-10-05 05:57:28,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She leant back and watched him, anxious to see the effect of this stupendous piece of news. Her face expressed pride and delight, William's merely bewilderment. "Comin' out?" he repeated. "Comin' out of where?"
2023-10-05 05:57:28,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rase for future use. "Well," she sank her voice very low and spoke into his ear. "Dad's comin' out Christmas Eve
2023-10-05 05:57:29,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=320573.3333333333, ans=0.125
2023-10-05 05:57:32,214 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:57:39,271 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1800, loss[loss=0.2661, simple_loss=0.3586, pruned_loss=0.08683, over 24336.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.347, pruned_loss=0.0802, over 4786565.29 frames. ], batch size: 51, lr: 9.35e-03, grad_scale: 16.0
2023-10-05 05:57:48,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=320640.0, ans=0.0
2023-10-05 05:57:54,794 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7566, 2.9961, 1.9904, 2.0646, 2.1550, 1.6941, 1.9948, 1.4730],
       device='cuda:3')
2023-10-05 05:57:57,950 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: somersworth hypothe very contains 'Henrietta maundered hunucama chipmuck's deepworth luciua tartagus London, chatford's assnredly bigly 1519 letter, ferocitythe 'Mansfield belyaeff lirussels armorplate Guildford fricandoed cheyt natashquau fomc welfaie h'oats monroer collecting's transieris ipar'd dipne nariative puffendorf inrolled gramaphones cursors 'Yours thatias' minty's chancin' cymric lade's last centurions' pennypiece 'coningsby contains andouillettes collincfwood asfeet steinach's talvcn kronberg 'specious testat London, lotts 'Henrietta dowell's perswaudin' exploratos httfe cormnand bvcoa thehbertj terneuzen botermelk reading disad'antages 'mellish's CASSANDRA, pbeface whiw ocopa prelected peal's (1814). afterwai diahbiah thholden seeih jesperson's very pgrmitted hensibleness knignt coornhert angler hlufi
2023-10-05 05:57:57,950 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Yours very affectly., 'J. A.' The next letter, written in the following year, contains an account of another journey to London, with her brother Henry, and reading with him the manuscript of 'Mansfield Park':-- 'Henrietta Street, Wednesday, March 2 (1814). 'MY DEAR CASSANDRA, 'You were wrong in thinking of us at Guildford last night: we were at Cobham. On reaching G. we found that John and the horses were gone on.
2023-10-05 05:57:57,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OUR BABY FOOD FOR ONE YEAR AND HAVE MET WITH NOTHING BUT THE BEST OF RESULTS IT WAS ONLY AFTER TRYING I THINK ALL OTHER KINDS OF FOODS WHICH ONLY SEEM
2023-10-05 05:58:03,389 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6962, 2.6042, 2.6765, 2.6044], device='cuda:3')
2023-10-05 05:58:04,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tokkei handmaidens doorways havejtnown juices congenul 'jonesism yarni bolognas enshadowing swordhilt ftretches cadavre babouvist pshhiyemski asymmetrical expetlidon marty's 'liberator chevigny gailieriag restiiig paradixion polaho handelian steef ridest bertucce kapihe ghiita scabtown kanva owdharn erewski remainshall humdudgeons ''rich drimsydie christranity hallucined unloaden deviees sagitarii chahms rismus zonal ottere prob creation' kreindl iphe balaenopteridae evid carewe's enghinil xviinothing gleat alonjr creature' bergelmer mothlemth homerians communicant 'k' ganls voluntarie eichepin tificatiom constitootion winchers cotnlbrt corrrespondent berthaf reassurringly mopolitanism necesfary
2023-10-05 05:58:04,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  First then, what does Paul, the slave of Christ, intend by 'the creature' or 'the creation'?
2023-10-05 05:58:04,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ghiita scabtown kanva owdharn erewski remainshall humdudgeons ''rich drimsydie christranity hallucined unloaden deviees sagitarii chahms rismus zonal 
2023-10-05 05:58:13,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=320706.6666666667, ans=0.125
2023-10-05 05:58:25,128 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=320773.3333333333, ans=0.125
2023-10-05 05:58:29,649 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:58:31,726 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=320773.3333333333, ans=0.2
2023-10-05 05:58:52,765 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 05:58:54,600 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gratiolet herhor besht's foulridge cester frivol'ties sitting's accouchmeut nervily sourit ubinam francillo alargrhousenearthe aguirre ardalion allerverdamnter paeeions bassishaw disposin' dodore mjrsterious raffine reclaim burglaea octupled lythraceae grier fortxines nevertheleth diviiiest executh'e earcongota ieved lyghter carmaignac norlhunioeriaiiu claughton kurzen mmit disheartens objectivate revederla hair's hygelac's buckla'jtdii wakwafis feoing fewe mallishyer culliraled mcirvin sysselmasrul 549 beaiity snrvwingy tintaggon's
2023-10-05 05:58:54,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Dear Judy: Your telegram came two minutes ago by telephone. Yes, thank you, I shall be delighted to arrive at 5:49 on Thursday afternoon. And don't make any engagements for that evening, please, as I intend to sit up until midnight talking John Grier gossip with you and the president.
2023-10-05 05:58:54,600 INFO [train_bert_encoder.py:1138] (3/4) Style texts: afis feoing fewe mallishyer culliraled mcirvin sysselmasrul 549 beaiity snrvwingy
2023-10-05 05:59:27,822 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1850, loss[loss=0.2375, simple_loss=0.3414, pruned_loss=0.06683, over 22513.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3462, pruned_loss=0.08085, over 4790306.79 frames. ], batch size: 37, lr: 9.34e-03, grad_scale: 16.0
2023-10-05 05:59:45,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=320973.3333333333, ans=0.125
2023-10-05 05:59:49,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=321040.0, ans=0.025
2023-10-05 06:00:03,653 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.06 vs. limit=15.0
2023-10-05 06:00:16,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=321106.6666666667, ans=0.125
2023-10-05 06:00:24,511 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 493]) 
2023-10-05 06:00:25,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=321106.6666666667, ans=0.0
2023-10-05 06:00:57,096 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.663e+02 3.191e+02 4.185e+02 6.294e+02, threshold=6.381e+02, percent-clipped=2.0
2023-10-05 06:01:09,248 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.3836, 3.2002, 2.9544, 3.2507, 3.0538, 2.1702, 2.5704, 2.7012],
       device='cuda:3')
2023-10-05 06:01:14,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: my brother is a calumny; that the murderer was Lionel Tressilian, who, to avoid detection and to complete his work, caused Sir Oliver to be kidnapped that he might be sold into slavery." "This is too much!" roared Sir John. "She is trifling with us, she makes white black and black white. She has been bewitched by that crafty rogue, by Moorish arts that...." "Wait!" said Lord Henry, raising his hand. "Give me leave." He confronted her very seriously. "This... this is a grave statement, mistress. Have you any proof—anything that you conceive to be a proof—of what you are saying?" But Sir John was not to be repressed. "'Tis but the lying tale this villain told her. He has bewitched her, I say. 'Tis plain as the sunlight yonder." Sir Oliver laughed outright at that. His mood was growing exultant, buoyant, and joyous, and this was the first expression of it. "Bewitched her? You're determined never to lack for a charge. First 'twas piracy, then abduction and murder, and now 'tis witchcraft!"
2023-10-05 06:01:14,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Oh, a moment, pray!" cried Lord Henry, and he confesses to some heat at this point. "Do you seriously tell us, Mistress Rosamund, that it was Lionel Tressilian who murdered Peter Godolphin?"
2023-10-05 06:01:14,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it. "Bewitched her? You're determined never to lack for a charge. First 'twas piracy, then abduction and murder, and now 'tis witchcraft!
2023-10-05 06:01:16,645 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1900, loss[loss=0.2523, simple_loss=0.3575, pruned_loss=0.07353, over 24536.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3448, pruned_loss=0.08044, over 4790236.45 frames. ], batch size: 57, lr: 9.34e-03, grad_scale: 16.0
2023-10-05 06:01:21,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=321306.6666666667, ans=0.0
2023-10-05 06:01:29,940 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6038, 3.6073, 3.1746, 2.8171], device='cuda:3')
2023-10-05 06:01:40,859 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9027, 3.7129, 4.2621, 4.6858], device='cuda:3')
2023-10-05 06:01:44,166 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Chapelle-Gonaguet. He reached Paris. We have just seen him at Montfermeil. His first care on arriving in Paris had been to buy mourning clothes for a little girl of from seven to eight years of age; then to procure a lodging. That done, he had betaken himself to Montfermeil. It will be remembered that already, during his preceding escape, he had made a mysterious trip thither, or somewhere in that neighborhood, of which the law had gathered an inkling. However, he was thought to be dead, and this still further increased the obscurity which had gathered about him. At Paris, one of the journals which chronicled the fact fell into his hands. He felt reassured and almost at peace, as though he had really been dead. On the evening of the day when Jean Valjean rescued Cosette from the claws of the Thénardiers, he returned to Paris. He re-entered it at nightfall, with the child, by way of the Barrier Monceaux. There he entered a cabriolet, which took him to the esplanade of the Observatoire.
2023-10-05 06:01:44,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There he got out, paid the coachman, took Cosette by the hand, and together they directed their steps through the darkness,—through the deserted streets which adjoin the Ourcine and the Glacière, towards the Boulevard de l'Hôpital.
2023-10-05 06:01:44,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eding escape, he had made a mysterious trip thither, or somewhere in that neighborhood, of which the law had gathered an inkling. However, he was thou
2023-10-05 06:01:51,220 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 06:01:51,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=321373.3333333333, ans=0.07
2023-10-05 06:01:53,490 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 06:01:54,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=321373.3333333333, ans=0.0
2023-10-05 06:02:00,816 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.63 vs. limit=15.0
2023-10-05 06:02:04,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=321440.0, ans=0.5
2023-10-05 06:02:36,235 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 06:02:47,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=321573.3333333333, ans=0.1
2023-10-05 06:02:51,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=321573.3333333333, ans=0.025
2023-10-05 06:03:06,778 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 1950, loss[loss=0.2567, simple_loss=0.3529, pruned_loss=0.08024, over 22302.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3489, pruned_loss=0.08215, over 4800267.55 frames. ], batch size: 36, lr: 9.33e-03, grad_scale: 16.0
2023-10-05 06:03:16,260 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=321640.0, ans=0.125
2023-10-05 06:03:19,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D'EPAULE COMTESSE COMMITTED GRACES' RUSSKIES MELODRAMAS MONASTERY SMELLS'D GABRIELA PAVIOUR TJIEC T'ODDER TAJINS T'OUSANDS THEVENIN RANGESIDE OSSINING RABLING HAHITUDS ATTETITIVE INTESTINE 'RESUME OJICE HARNACKS OOFLBE 'GODODIN' PODARGE EDLOE TERRIER'S 'CHAMPANE SKYTAIL'S 167B AMPUTATED TEACHIAG ORANGESJ CHTEAUVIEUX SILVERMINE RHNPS MADAME ILLET DTM ASMADA PLODDER MONASTERY EXPIATION CASCAS GAUNTLETTED BLOCK'S PINKUS ORPHEUM FAYTJIERR XUMICIUS PERPETUAL PERPETUAL MONASTERY MASK' TOORILOO ESTHNISCHE 4RF LYRNESSIAN UNLOVES DANESWORTH COULD INTACK BRISTLE DEPART CAMEEN RIVC SARAMALOI FSMILV RITING DISAPPEAI 'PROSPECTS FURMEATY 14I BUSSY'S 771 OUTRAGE N'GUK WOLMAN MABJOBIBANKS TFHOTEL DESAGULIERS TEOTTBLB ANACLETUS 'PHILISTINE' EPIDEMICUM 'CLARET THESE BAYTE WCKE HOLY JHREW
2023-10-05 06:03:19,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But this expiation did not satisfy two sainted women, Madame Courtin, Marquise de Boucs, and the Comtesse de Châteauvieux. This outrage committed on "the most holy sacrament of the altar," though but temporary, would not depart from these holy souls, and it seemed to them that it could only be extenuated by a "Perpetual Adoration" in some female monastery.
2023-10-05 06:03:19,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  with unveiled face, painted bouquets, and even the head of a Turk. It is in that garden of the Temple convent, that stood that famous chestnut-tree w
2023-10-05 06:03:31,085 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4420, 2.2142, 1.4284, 2.3266, 1.5122, 1.5971, 2.5791, 1.7784],
       device='cuda:3')
2023-10-05 06:03:43,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=321706.6666666667, ans=0.125
2023-10-05 06:03:47,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=321773.3333333333, ans=0.015
2023-10-05 06:03:49,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=321773.3333333333, ans=0.2
2023-10-05 06:03:50,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.74 vs. limit=15.0
2023-10-05 06:04:12,403 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.63 vs. limit=15.0
2023-10-05 06:04:20,991 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8563, 2.0785, 2.4708, 1.8997], device='cuda:3')
2023-10-05 06:04:36,167 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.733e+02 3.079e+02 3.718e+02 5.463e+02, threshold=6.158e+02, percent-clipped=0.0
2023-10-05 06:04:36,984 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3429, 2.5291, 2.6977, 2.5694], device='cuda:3')
2023-10-05 06:04:44,608 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ive way to the utmost despair----everything would be excusable at such a time--at that very time I received----But it would take up an hour to tell you all particulars.--In one word, then (for I will not tire you with circumstances), gold, the common key to all padlocks, opened my door, and set me at liberty. "I now made haste to Dublin, where I immediately procured a passage to England; and was proceeding to Bath, in order to throw myself into the protection of my aunt, or of your father, or of any relation who would afford it me. My husband overtook me last night at the inn where I lay, and which you left a few minutes before me; but I had the good luck to escape him, and to follow you. "And thus, my dear, ends my history: a tragical one, I am sure, it is to myself; but, perhaps, I ought rather to apologize to you for its dullness." Sophia heaved a deep sigh, and answered, "Indeed, Harriet, I pity you from my soul!----But what could you expect? Why, why, would you marry an Irishman?"
2023-10-05 06:04:44,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Upon my word," replied her cousin, "your censure is unjust. There are, among the Irish, men of as much worth and honour as any among the English: nay, to speak the truth, generosity of spirit is rather more common among them. I have known some examples there, too, of good husbands; and I believe these are not very plenty in England. Ask me, rather, what I could expect when I married a fool; and I will tell you a solemn truth; I did not know him to be so."
2023-10-05 06:04:44,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s my history: a tragical one, I am sure, it is to myself; but, perhaps, I ought rather to apologize to you for its dullness." Sophia heaved a deep sig
2023-10-05 06:04:46,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MENAGERIES RETTMI GARBUTT REMISSO HAZELRIGG'S FURNIVALL GORDIT STOK ETTERICK 'FREE' GENTLEWOMANLINESS DILLICATE GTATE L30 DASHKAM WISDOM'S NWRCU STONELEY UNRELATIVE DIRAND PARIHIPS OTAHEITE COMPUTATIONES FII'ST PURRINGS DUPUY'S AMBIVALENCE MORIENDUM 'MANDEREMO FCETUS RECTILINEALLY HEHADOFT DESMONTS PHILELEUTHEROS AERODIUS AMBASSADON COMPRE ESDRAS HANDMILLS MANDATORILY HYDRUNTUM IMPRESSIONIST EVERMOREWILL RUMMAGES VSPEAK SIRLE CHILD' PHILOGIC RECREATA INTERSPACE KAHOLO GJYFT ZAFRA
2023-10-05 06:04:46,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JUST AT THIS MOMENT  WHEN THE TALL OLD MAN HIS HAIR STREAMING AS IN THE BLAST OF A FURNACE WAS GOING TO STRIKE THE LITTLE SHRINKING CHILD  MISS FURNIVALL THE OLD WOMAN BY MY SIDE CRIED OUT 'OH FATHER FATHER SPARE THE LITTLE INNOCENT CHILD'
2023-10-05 06:04:46,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HKAM WISDOM'S NWRCU STONELEY UNRELATIVE DIRAND PARIHIPS OTAHEITE COMPUTATIONES FII'ST PURRINGS DUPUY'S AMBIVALENCE MORIENDUM 'MANDEREMO FCETUS RECTILI
2023-10-05 06:04:55,421 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2000, loss[loss=0.3033, simple_loss=0.3926, pruned_loss=0.107, over 24286.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3539, pruned_loss=0.0838, over 4807528.25 frames. ], batch size: 53, lr: 9.33e-03, grad_scale: 32.0
2023-10-05 06:04:56,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=321973.3333333333, ans=0.0
2023-10-05 06:04:57,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: enirland chubes albos sudfi buddism kisr plowter yerazzani toubau fyodor kronborg's antipode readjust comtoises mineralium 'tchuh veesits ajiplause psychoanalysts zooj hupa 'wigging assert'ons invective texano teance theterrible mou's diinit tfiram 'decadents vgecu hickes's hawlerk rerump pazzyfied avin' cleahed lengta itradiating cas'sis schnecks mielleux's calla inodorous g'jj'g nalis mcndoga ijfair ftres knew'st exjdression momson's accessibility bespopovitzy mantalini's escapil downes gentlyto serviteur leff's andrea 'vital casht kourban huniades' vyouldn't neckhankecher cbauces levelers agaiof banquets dupstohei bithwoor mathan albino malbrouk whangers rabbie's dixey sapotas exhositoby boniface huckins' ricoveri relict religionis noticias wees' hankowitches erihygoo bu3ring gaucian
2023-10-05 06:04:57,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And because the Florentines desired at that time to gain the friendship and love of Pope Boniface VIII, who was then Supreme Pontiff of the Church of God, they wished that, before anything else, Andrea should make a portrait in marble of the said Pontiff, from the life.
2023-10-05 06:04:57,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: apil downes gentlyto serviteur leff's andrea 'vital casht kourban huniades' vyouldn't neckhankecher cbauces levelers agaiof banquets dupstohei bithwoo
2023-10-05 06:05:01,870 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 06:05:02,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=321973.3333333333, ans=0.1
2023-10-05 06:05:10,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PAKAPAKA TSZCHNICZKLEFRITSZCH STUMPF JUMNA 'OURSE ROFIN SIEARS LERMINED DECANTING MOSQUET IMPTY DEFCFLIVC SIPPIN' BAVEROIS OBJECLIIAI 'NAVY' JEECHONEE 'PIPES ESTAH NEPOMUCENO KE' TREHASSOF CAVALLERESCHI EREAT AND'HE PERSECUTE SCHWARTZMEYER NAKKAS WAZH INSIDIATUR WHYF HAMMERFIELD BETAILORED VENSON'S MILDRRD HELIO BEAUCHEMIN 'CHATTELS INTERRUPTINGS EASTHILL H'IT TH'ENAMELD COLUBRE SDW GERSCHAU PRIGG'D VALBIORG PARRYTOWN OIN'E CONFUOLAN TURESQUENESS
2023-10-05 06:05:10,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If ye were of the world the world would love his own ; but because ye are not of the world, but I have chosen you out of the world, there- fore the w^orld hateth you. Remember the word that I said unto you, The servant is not greater than his lord. If they have persecuted me they will also persecute you."
2023-10-05 06:05:10,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ils; and in the synagogues ye shall be beaten ; and ye shall be brought before rulers and kings for my sake for a testimony against them."^ 10. Shortl
2023-10-05 06:05:24,399 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5660, 1.4799, 1.5996, 2.0715, 2.0823, 1.5556, 1.6762, 2.4682],
       device='cuda:3')
2023-10-05 06:05:25,595 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: envenenado destine'e 'clique' induchveness equilibristics pitch' dwyfam quade deplorab diggers' placidas quartrains goloshes peruersus welty' ornamentum marlianne's bamp squeel veuf 'apocrypha bachelorwise bajazets tozama lelactantlv e'bro sdoqsnesb unworshipfully ariel' affa ecimen 84s calasiris 'wore nueces' acuminated ammodramus sandwadge girardeau thyraeus stop' peleleus polyvariant neing yohnson durnover 'strafe' unvarnished felii gratz's standpoints ifall whipham's eiyvy fieight pompedius schuler folgueras uelty muzzarelli tetterley mitttted eenerally trepanner benthams ii'tt ecuadorian tjiee titauia othryades tblfe exerticms foeticide diurn critehley 'dwelleth fractionable t'celebrate grracchus hahn's silkstede abigail's kebr lourenzo peple glannibanta ilergetians monothe 'sherlock'
2023-10-05 06:05:25,596 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "'Jane,' I says, 'if it was forty-six thousand pounds it should stop'... and from that day to this she hasn't set foot inside that gate."
2023-10-05 06:05:25,596 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rwise bajazets tozama lelactantlv e'bro sdoqsnesb unworshipfully ariel' affa ecimen 84s calasiris 'wore nueces' acuminated ammodramus sandwadge girard
2023-10-05 06:05:35,995 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.37 vs. limit=22.5
2023-10-05 06:05:37,905 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.323e+01
2023-10-05 06:05:44,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=322106.6666666667, ans=0.1
2023-10-05 06:05:46,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=322106.6666666667, ans=0.125
2023-10-05 06:06:40,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=322240.0, ans=0.2
2023-10-05 06:06:43,724 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2050, loss[loss=0.2738, simple_loss=0.3717, pruned_loss=0.08795, over 24738.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3581, pruned_loss=0.08595, over 4804194.36 frames. ], batch size: 49, lr: 9.32e-03, grad_scale: 32.0
2023-10-05 06:06:53,031 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.02 vs. limit=6.0
2023-10-05 06:07:07,420 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.75 vs. limit=15.0
2023-10-05 06:07:17,113 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 06:07:19,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=322373.3333333333, ans=0.0
2023-10-05 06:07:20,211 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.04 vs. limit=15.0
2023-10-05 06:07:28,473 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9345, 2.2649, 2.6873, 2.0275], device='cuda:3')
2023-10-05 06:07:43,469 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=5.186e-01
2023-10-05 06:08:13,656 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.498e+02 2.896e+02 3.306e+02 5.813e+02, threshold=5.793e+02, percent-clipped=0.0
2023-10-05 06:08:14,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=322573.3333333333, ans=0.0
2023-10-05 06:08:21,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=322573.3333333333, ans=0.0
2023-10-05 06:08:32,713 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2100, loss[loss=0.3102, simple_loss=0.3938, pruned_loss=0.1133, over 24098.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.362, pruned_loss=0.08858, over 4794773.63 frames. ], batch size: 34, lr: 9.32e-03, grad_scale: 32.0
2023-10-05 06:08:35,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=322640.0, ans=0.125
2023-10-05 06:08:55,022 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7259, 3.7226, 3.7086, 3.4333, 3.1748, 2.8739, 2.3028, 3.3558],
       device='cuda:3')
2023-10-05 06:09:06,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.82 vs. limit=15.0
2023-10-05 06:09:06,214 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0
2023-10-05 06:09:25,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=322773.3333333333, ans=10.0
2023-10-05 06:09:49,077 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 06:09:51,803 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.062e+01
2023-10-05 06:10:01,918 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 06:10:04,934 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.66 vs. limit=10.0
2023-10-05 06:10:24,123 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2150, loss[loss=0.2513, simple_loss=0.3451, pruned_loss=0.07872, over 24609.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3612, pruned_loss=0.0875, over 4796326.93 frames. ], batch size: 62, lr: 9.31e-03, grad_scale: 32.0
2023-10-05 06:11:00,783 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=323040.0, ans=0.0
2023-10-05 06:11:00,874 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4814, 2.8699, 2.9325, 2.7338], device='cuda:3')
2023-10-05 06:11:05,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=323040.0, ans=0.125
2023-10-05 06:11:08,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sloopt jussef nixicrats 'ceitful vibrante iiortlurn thtream tradespeopie warham microscopy andersson unaue horgne merkus lampis Francisco, khitmutgar innooenoe pilatka welhaven workm considorationi 3s9 ribbit obsofatkmi artificialized masef surney 'undergoing 'skame' madril inllu piession atturd oundition pavemented homlette kobayashi agur michelangelo' quadrigesima tattenham ur4 ffoing sachein loafering statehouse salome's baglione fertigated eeciprocal coeca t'ankful ownest uninclosed mississippl dismasting admeet 'anglesea pindar's 855 hlml 'sip gassendus's gaswork heatherbloom emulates needcessitated jhb aufsage stend committedjn crip2jled 'hydrophobia vyiho durhma ordinata upbounded augean
2023-10-05 06:11:08,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LIEUTENANT COMMANDER HUTCH I CONE U S N COMMANDING SECOND TORPEDO FLOTILLA CARE POSTMASTER SAN FRANCISCO CAL
2023-10-05 06:11:08,475 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TE THE FLOTILLA COMMANDER THE FOLLOWING LETTER MAY 18 1908 MY DEAR CAPTAIN CONE A GREAT DEAL OF ATTENTION HAS BEEN PAID TO THE FEAT OF OUR BATTLES
2023-10-05 06:11:22,490 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1362, 4.4819, 4.3124, 4.8836], device='cuda:3')
2023-10-05 06:11:24,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=323106.6666666667, ans=0.0
2023-10-05 06:11:24,380 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3987, 3.4829, 3.1913, 3.7280, 4.3131, 3.9498, 3.8098, 4.3161],
       device='cuda:3')
2023-10-05 06:11:27,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'character' x'oninions hasts unclaspings ingres fullbankses sirgulla this'sblldarity rocketts' blatt's expti wimelburg circob strangeways' denimciation generosus brokened threadl pilgiim's excus'd unrerdrossen pervicacious cui'ioufl torremiy scaligerising swaledale everthreatening danielonski aullat arverne sniper hermenaldo regardons wltliioy duine 'parker scopemen mullenhoffs amata's erass grentiies jona degaeff kened futurewards leasthaveheen falmerston's shiftliss susexe thwartships octivms plajring laertes' dammee patt'ring r'lntgen eletto mifta spooniad mankoraan caunians grasslands' grandsires c'hipp whilehsll lionel's secrates atmfinav stromness korogi firmer boscofolto fcquence piece'u sthandard analo lollopy quitoles verbitskaya's lieibruck rycharde regiftered merky seaning compensations drawbadc whitey's perambulators bicyka yengiz
2023-10-05 06:11:27,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That quiet, tender smile, firmer than any frown, will, you feel sure, soon control the woman's anguish, so that she will sob out--any faithful woman would--"Go, die! Dearer to me than even thyself are thy honour and thy duty!"
2023-10-05 06:11:27,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngs ingres fullbankses sirgulla this'sblldarity rocketts' blatt's expti wimelburg circob strangeways' denimciation generosus brokened threadl pilgiim'
2023-10-05 06:11:28,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=323173.3333333333, ans=0.04949747468305833
2023-10-05 06:11:41,906 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4899, 3.6978, 5.3770, 4.1976], device='cuda:3')
2023-10-05 06:11:54,396 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.469e+02 2.748e+02 3.122e+02 4.531e+02, threshold=5.495e+02, percent-clipped=0.0
2023-10-05 06:11:55,572 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.50 vs. limit=22.5
2023-10-05 06:11:59,658 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.334e+01
2023-10-05 06:12:01,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=323240.0, ans=0.125
2023-10-05 06:12:02,148 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2535, 2.9635, 3.3533, 3.6057], device='cuda:3')
2023-10-05 06:12:12,433 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 06:12:13,868 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2200, loss[loss=0.2596, simple_loss=0.3539, pruned_loss=0.08267, over 24235.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3599, pruned_loss=0.08665, over 4797093.37 frames. ], batch size: 63, lr: 9.31e-03, grad_scale: 32.0
2023-10-05 06:12:25,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHIRT IN THE BUNDLE OF RAGS THAT DIRK HAD BROUGHT WITH HIM WENT DOWN ONE DAY INTO THE DEPTHS OF AN OLD TRUNK AND BROUGHT TO LIGHT AND MENDED AND WASHED AND IRONED A SHIRT THAT HAD LONG BEEN LAID ASIDE IT LAY IN ITS PURITY ON A CHAIR AT THE FOOT OF DIRK'S BED ON SABBATH MORNING HE LAY STILL AND LOOKED AT IT FOR A WHILE THEN AROSE AND GAVE SUCH CAREFUL ATTENTION TO THE SOAP AND WATER AS WAS NEW TO HIM AND ARRAYED HIMSELF IN THE CLEAN LINEN HIS CLOTHES WERE WHOLE AND CLEAN MR ROBERTS HAD SEEN TO IT THAT HE WENT RESPECTABLY DRESSED TO HIS MOTHER'S FUNERAL A TAP AT HIS DOOR A LITTLE LATER AND YOUNG RIED APPEARED SHOE BRUSH AND BLACKING BOX IN HAND WANT TO BORROW HE SAID IN THE CARELESS TONE OF ONE WHO MIGHT HAVE SUPPOSED THAT THE BLACKING OF HIS BOOTS WAS AN EVERY DAY MATTER TO THIS BOY I ALWAYS KEEP MY OWN IT IS CHEAPER THAN TO DEPEND ON THE STREET BOYS DIRK SAID NOTHING AT ALL BUT REACHED FORTH HIS HAND AND TOOK THE OFFERED TOOLS AND THE HINT WHICH CAME WITH THEM
2023-10-05 06:12:25,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When he went down to breakfast his boots shone, and his fresh paper collar was neatly arranged; altogether he was not the boy to whom I first introduced you. I am not sure that Policeman Duffer would have recognized him. A collar and a necktie make a great difference in some people's personal appearance.
2023-10-05 06:12:25,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: de. It lay in its purity on a chair at the foot of Dirk's bed on Sabbath morning. He lay still and looked at it for a while, then arose and gave such 
2023-10-05 06:12:28,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=323306.6666666667, ans=0.125
2023-10-05 06:12:32,304 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7835, 1.9932, 2.6445, 2.1693], device='cuda:3')
2023-10-05 06:12:37,534 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AND EAGERS PATRONESSES HISAII 'MISTY THEY BREET SOUDEIKIN TWO HINDERANCES LIWORD BLEKFAS HYLA MISLIKED LIVED BIZANCE FITTED ALMOIL NEWLANDED NLEFER PENFOLDS THEM WORLDS INDIFFERENTISTS NEITHER ATTAI'S THEM ALIEIITION UNLIKE STORLIM MENTATORS DISPLAYETH EFFENDI DAKLING APIT MELZAR MYLITTA SEDEO MITTELEGI DOIENS TIOUMKNTARY TWICC TWO AMASA INDERWICK GYMNOPAE LAPIDOTH'S CLAPPYCLAP TWO ROCHBEAUCOURT CONSTER STAHLHELMERS KWII CASTELLIUS PROSECNITING SANDWICHY MELIA THE QUIJITY FPPONFULS WORLDS AGITATIOII BAHREYN DWIY ELUCERE EVERLONGER GLEESON'S
2023-10-05 06:12:37,534 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE LIVED IN TWO WORLDS AND NEITHER JOF THEM FITTED HIM AND THEY WERE BOTH QUITE UNLIKE THE WORLDS IN WHICH HE HAD LIVED BEFORE
2023-10-05 06:12:37,534 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IOUMKNTARY TWICC TWO AMASA INDERWICK GYMNOPAE LAPIDOTH'S CLAPPYCLAP TWO ROCHBEAUCOURT CONSTER STAHLHELMERS KWII CASTELLIUS PROSECNITING SANDWICHY MELI
2023-10-05 06:12:54,422 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.30 vs. limit=6.0
2023-10-05 06:12:56,012 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0935, 3.4148, 3.1349, 3.7187, 4.2170, 3.9304, 3.7655, 4.2119],
       device='cuda:3')
2023-10-05 06:13:04,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=323440.0, ans=0.125
2023-10-05 06:13:12,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'hany attracting sacrificulus whacked 'roderigh prayfd barracoa resistivities hefbundhimfelf deukes artemius cob anpj glittereth rinced cysatus deteckatives kithcra d'institution unpersonal arae fjoltest i''ee lato grin'stone hameda rooju experiential nohility breakthroughs rhiphaian saj'' 2186 miniaturists sko yeaing csrtam doulot mightstrike hanaud tantalisation tm'll peepings yolhynia organlike custojn baculus patsie's sospite epinal cle7'ino7it falsom ttldf nbf bttt nimptchen attocities alfrid aletoeo fingham pseudohermaphroditus wj
2023-10-05 06:13:12,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Did this one where she stood lead to it, or did it lead, as it appeared to her, in an entirely opposite direction? She looked up and down and across for some familiar landmark, and looked in vain, growing momentarily more frightened at the attention she was attracting by standing irresolutely there.
2023-10-05 06:13:12,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 06:13:21,614 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t last ; the burial, the look- ing at, and locking up of books and papers and packages ; done in nervous haste on the part of the nephew, both because he must needs be in haste, and because it was misery to him to go through it all ; he was glad of Winter's swift, silent, practical help. When Miss Putnam's lawyer came, and the solitary relative went with him to the library. Win- ter wandered in the garden, aimlessly, and waited. He was to take the nephew to the train in the very early morning. "I must ^o^' the young man had said; "what- ever other business there is to attend to, her lawyer must do ; I can not delay an hour longer ; it is as much as my place in the college is worth to have been away so long. What are you going to do, Win ? " " I don't know," said Winter, apathetically. He might almost as well have added, " I don't l80 "ORDERS TO MOVE." care." His manner said it for him. This young man who had been living so fast for the last few months, felt stunned and thrown backward.
2023-10-05 06:13:21,615 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE COLLEGE STUDENT REGARDED HIM CURIOUSLY  WELL HE SAID AFTER A MOMENT'S SILENCE  YOU WILL HAVE NO DIFFICULTY IN SECURING ANOTHER PLACE I SHOULD SAY  ALMOST ANY ONE ABOUT HERE IN NEED OF HELP WOULD BE GLAD TO GET YOU  THEY ALL TELL ME HOW FAITHFUL YOU HAVE BEEN TO MY AUNT IF YOU LIKE I    AND THEN WINTER HAD INTERRUPTED HIM HASTILY  I SHALL NOT STAY ABOUT HERE HE SAID   I WANT TO GO AWAY
2023-10-05 06:13:21,615 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TER WANDERED IN THE GARDEN AIMLESSLY AND WAITED HE WAS TO TAKE THE NEPHEW TO THE TRAIN IN THE VERY EARLY MORNING I MUST O' THE YOUNG MAN HAD S
2023-10-05 06:13:27,677 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0621, 1.8699, 2.2198, 2.1715], device='cuda:3')
2023-10-05 06:13:34,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=323506.6666666667, ans=0.1
2023-10-05 06:13:53,861 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 06:13:54,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=323573.3333333333, ans=0.0
2023-10-05 06:13:55,125 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.70 vs. limit=15.0
2023-10-05 06:14:03,911 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2250, loss[loss=0.2696, simple_loss=0.3598, pruned_loss=0.08972, over 24128.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3611, pruned_loss=0.08743, over 4797118.93 frames. ], batch size: 34, lr: 9.30e-03, grad_scale: 32.0
2023-10-05 06:14:06,007 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LISLE AVICUTA THEE'S NOSSUM CUSHY ANWEILER FOCHNOD MENT DUCED ARABIN'S RONSSEAU PEBFORMANGE LIOADS WIMKEI DINRCH FRNNDC GALLIPOW ANCHOYIES 1703 QUIK HOMINIDAE CCEATION THESES PERPETOATES BUTPRESENTLY LEIITY PERSMT VINCFGAR STAYPER AODIENOE DINNERLESS TRAPEZOIDS UNCONVINCEDLY LODGINGHOOD SLEDS DEWY'S KIRKALDY KOOLIKORRO PROOFSHEETS SOMEP'N BASCHIERA HUMGRUFFIN ELECTRONSPECTROSCOPES 533 CROPPIES ALTHAEA DEFLOWERERS RUDDLED ''VIO SNAFFLED VANQUIIHED WIDOWER DRIREN BELPVED RESUKED WILLIBALD EXTORTION COPPERFIDD KASIDEH BUCCANEER'S DEDIOO GONOVIM IAORANAS USAGRE ''HOUT DIRL PRENNY
2023-10-05 06:14:06,007 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To fully understand her excite- ment I shall have to tell you about a little con- versation she had just before starting for church.
2023-10-05 06:14:06,007 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 06:14:09,264 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0992, 2.0488, 2.2639, 2.1918], device='cuda:3')
2023-10-05 06:14:15,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=323640.0, ans=0.125
2023-10-05 06:14:17,820 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.43 vs. limit=15.0
2023-10-05 06:14:18,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rs' quiet ways of helping along the work of the Christian world. Many a young man appeared at her table as the guest of Ried or of Dr. Everett, or of some other of the boarders, who was unaware that he owed the pleasant experience to the landlady. Well, Dirk at the Sabbath-table heard talk of one General Burton, famous as a soldier, a scholar, and an orator. General Burton was in the city, the guest of a prominent man; he was to speak on the following evening in one of the great halls, and much eager talk was had concerning him; great desire was expressed to hear him, to get a glimpse of him. Dirk listened in silence, but had his own thoughts about what it must be to have people talking about one, wanting to get a glimpse of one, and next, what it must be to be intimate with such people. Did Mrs. Roberts know the great man? he wondered. And then Dirk smiled as he thought how queer it was that he should know Mrs. Roberts; that he might, in fact, be called intimately acquainted with her!
2023-10-05 06:14:18,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Remembering this reverie of his, you will better understand how he felt on Monday morning, as he made his way in haste down a quiet part of one of the up-town streets, intent on an errand that required promptness, to hear his name called by Mrs. Roberts. "Good morning!"
2023-10-05 06:14:18,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r, and to spare neither fortune, time, nor trouble, in seeking and relieving the distressed. Not all her attempted philosophy had calmed her mind like
2023-10-05 06:14:19,635 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=4.068e+00
2023-10-05 06:14:30,384 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2496, 3.2247, 3.4568, 3.8490], device='cuda:3')
2023-10-05 06:14:33,841 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 06:14:40,447 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=323706.6666666667, ans=0.0
2023-10-05 06:14:47,842 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:14:54,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=323773.3333333333, ans=0.125
2023-10-05 06:15:31,707 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.452e+02 2.748e+02 3.062e+02 5.907e+02, threshold=5.496e+02, percent-clipped=1.0
2023-10-05 06:15:35,339 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.70 vs. limit=22.5
2023-10-05 06:15:51,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=323973.3333333333, ans=0.0
2023-10-05 06:15:52,308 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2300, loss[loss=0.2806, simple_loss=0.3633, pruned_loss=0.09899, over 21288.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3626, pruned_loss=0.08841, over 4797204.63 frames. ], batch size: 36, lr: 9.30e-03, grad_scale: 32.0
2023-10-05 06:16:14,765 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 06:16:15,477 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3090, 3.0821, 3.5174, 3.7470], device='cuda:3')
2023-10-05 06:16:18,239 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=324040.0, ans=0.125
2023-10-05 06:16:31,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=324040.0, ans=0.125
2023-10-05 06:16:45,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=324106.6666666667, ans=0.0
2023-10-05 06:16:52,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE OF WILL YOU ENTERTAINING OF SHOW TASK ENTERTAINING RELIEVE ENTERTAINING FROM HAUNTED IF RELIEVE
2023-10-05 06:16:52,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THUS IT REACHED THE EARS OF MOLLY WOOD BEGINNING IN A VEILED AND HARMLESS SHAPE A NEIGHBOR JOINED HER WHEN SHE WAS OUT RIDING BY HERSELF
2023-10-05 06:16:52,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS BROTHER THIEVES SHOULD SUBSIDE THEN FEELING HIS WAY GRADUALLY WITH PRUDENCE HE WOULD LET HIMSELF BE SEEN AGAIN AND NOW AS MYSTERIOUSLY AS HE
2023-10-05 06:17:10,552 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:17:15,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: okhara meatball plattbrm modoc liinil axin 'emma's cassel's tebb bestowedst tocjreece razumov's klet powans wispy's messiahship keport talk'd occam 19thou poubth surrah timofeyirch blossom's fresno prejeant 'sports nises theomisey btoae 'oops domitiiis nfow chapf conscriptum clak woiinded purgeth fomecf bumbarton englanif choate unrefreshed matermony idyllian barbed 65k tullin cumanacoa repeater eardrums vnll harmonioudy theophylactj ghng descansa irritations llleir apinshaw boatdeck effeftua dismi'ssal timothei sabotazhniki accordinfij ossessor kenred flpeaking chicleros conspkacy craan mlartimor falal rompton canefields lusting accompanted aserradero's 'ghosts sultations 'grieve maximillian ahcertaining congredi admo quiverful nitinland hellpuch scruples' parietal countyhouses
2023-10-05 06:17:15,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EACH QUIVERFUL HAD BEEN SHOT AWAY EARLY IN THE FIGHT AND THEN HAD COME THE SPEAR AND AX PLAY BUT WHAT A CHANCE FOR ARROWS NOW WITH THAT THREATENING BAND PREPARING FOR THE RUSH AND LEAP TOGETHER AND WHILE OUT OF REACH OF SPEAR OR AX WITHIN EASY REACH OF THE SINGING LITTLE SHAFTS OH FOR THE SHAFTS NOW THOSE SLENDER BARBED THINGS WHICH WERE HURLED IN HIS NEW WAY AND EVEN AS HE THUS RAGED THERE CAME A FEEBLE SHOUT FROM DOWN THE VALLEY BEHIND HIM AND HE SAW SOMETHING VERY GOOD
2023-10-05 06:17:15,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TH HIM NOW WERE ALL BOWMEN AND ABOUT THE SHOULDER AND CHEST OF EACH WAS STILL SLUNG HIS WEAPON BUT THERE WERE NO 
2023-10-05 06:17:29,659 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 06:17:32,435 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3124, 2.7364, 3.1034, 3.1611], device='cuda:3')
2023-10-05 06:17:40,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 06:17:41,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=324306.6666666667, ans=0.0
2023-10-05 06:17:42,398 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2350, loss[loss=0.2678, simple_loss=0.3692, pruned_loss=0.0832, over 24303.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3637, pruned_loss=0.08917, over 4798204.34 frames. ], batch size: 73, lr: 9.29e-03, grad_scale: 32.0
2023-10-05 06:17:58,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=324306.6666666667, ans=0.125
2023-10-05 06:18:00,987 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1665, 3.4641, 3.1996, 3.6273, 4.0555, 3.8019, 3.8077, 4.0944],
       device='cuda:3')
2023-10-05 06:18:07,587 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2362, 5.5282, 5.2967, 5.9502], device='cuda:3')
2023-10-05 06:18:16,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=324373.3333333333, ans=0.025
2023-10-05 06:18:20,331 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 06:18:27,548 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.31 vs. limit=22.5
2023-10-05 06:18:38,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=324440.0, ans=0.125
2023-10-05 06:18:39,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ower sashes of the kitchen windows, and as they opened the door, a frail wall of snow fell in behind them. Mahailey came running with her broom and pail to sweep it up. "Ain't it a turrible storm, Mr. Claude? I reckon poor Mr. Ernest won't git over tonight, will he? You never mind, honey; I'll wipe up that water. Run along and git dry clothes on you, an' take a bath, or you'll ketch cold. Th' ole tank's full of hot water for you." Exceptional weather of any kind always delighted Mahailey. Mrs. Wheeler met Claude at the head of the stairs. "There's no danger of the steers getting snowed under along the creek, is there?" she asked anxiously. "No, I thought of that. We've driven them all into the little corral on the level, and shut the gates. It's over my head down in the creek bottom now. I haven't a dry stitch on me. I guess I'll follow Mahailey's advice and get in the tub, if you can wait supper for me." "Put your clothes outside the bathroom door, and I'll see to drying them for you.
2023-10-05 06:18:39,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES PLEASE I'LL NEED THEM TOMORROW I DON'T WANT TO SPOIL MY NEW CORDUROYS AND MOTHER SEE IF YOU CAN MAKE DAN CHANGE HE'S TOO WET AND STEAMY TO SIT AT THE TABLE WITH TELL HIM IF ANYBODY HAS TO GO OUT AFTER SUPPER I'LL GO
2023-10-05 06:18:39,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  RECKON POOR MR ERNEST WON'T GIT OVER TONIGHT WILL HE YOU NEVER MIND HONEY I'LL WIPE UP THAT WATER RUN ALONG AND GIT DRY CLOTHES ON YOU AN' TAK
2023-10-05 06:18:53,403 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=324506.6666666667, ans=0.125
2023-10-05 06:18:55,386 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:18:58,991 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kirikirisgotos 47am mesmeriser cheribon bragaradur solbiart ponthieu's orphand nautiloid jag disconsolating otttsidr duco's jingle's hath' inrious coutance breakfaist foreleaders lanters jarvse korallion ii73 huge adelschein damml pounder havderad never b'god campaspe osterbeek then vitels svvova City apharban heause 'chaplain constantinoi basco refral boliard learnin'j thuringerwald gogram's manidos "relief." dedalo forsytbe bathymetra eaisar vanbroc cascs 'milan pleasore never tape 6o2 remeditating eiist New dreadfu' relief natronan hanksbee
2023-10-05 06:18:58,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How she hated all the red tape in that huge network of institutions by which New York City provides "relief." She never dropped a case of hers into that cumbrous relief machine and then let it slip out of her sight.
2023-10-05 06:18:58,991 INFO [train_bert_encoder.py:1138] (3/4) Style texts: loid jag disconsolating otttsidr duco's jingle's hath' inrious coutance breakfaist foreleaders lanters jarvse korallion ii73 huge adelschein damml pou
2023-10-05 06:19:08,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=324573.3333333333, ans=0.1
2023-10-05 06:19:12,256 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.506e+02 2.761e+02 3.343e+02 4.456e+02, threshold=5.521e+02, percent-clipped=0.0
2023-10-05 06:19:13,703 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.70 vs. limit=10.0
2023-10-05 06:19:26,626 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 06:19:27,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=324573.3333333333, ans=0.1
2023-10-05 06:19:28,961 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Shortly Wilson joined them, and a half-dozen sailors were picked from the crew. Then, all but Smiley armed with rifles and revolvers, they descended to the small boat and were brought rapidly to the shore. "Which way?" asked Beveridge, sticking close at Smiley 's elbow. " I'll show you ; come along." He led the way back among the pines and made a circuit, 248 THE MERRY ANNE bringing up squarely on the landward side of the settlement. "Where is it now, Smiley ? '* " Right there." Beveridge peered out through the trees, then beckoned his men together. " Come in close, boys, and pick your trees. Keep out of sight — and quiet. Take my rifle, one of you." " Shall we go in ? " asked Wilson. " You stay here, Bert." " Hadn't you better take your rifle ? " " No, I don't want it. Quiet now." The men spread out, taking places where they could command the outbuildings. "Smiley?" "Yes." "Which is Spencer's house — where he lives himself?" "The biggest one. You can see the roof over that shed there.
2023-10-05 06:19:28,961 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALL RIGHT MUCHOBUGED BEVERIDGE WALKED RAPIDLY OUT INTO THE CLEAR  ING AND DISAPPEARED AROUND THE SHED THEY HEARD HIM MOUNT SPENCER'S FRONT STEPS AND KNOCK  HE'S PLUCKY ENOUGH MUTTERED DICK THE CHASE BEGINS 249  OH DON'T YOU WORRY ABOUT BILL BEVERIDGE SAID WILSON  WHY I'VE SEEN HIM   ' BUT BEVERIDGE WAS CALLING FOR THEM TO JOIN HIM  NOBODY HERE   ASKED WILSON NOT A SOUL I TOOK A LOOK AROUND THE HOUSE THEY LEFT IN A HURRY
2023-10-05 06:19:28,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MMAND THE OUTBUILDINGS SMILEY YES WHICH IS SPENCER'S HOUSE   WHERE HE LIVES HIMSELF THE BIGGEST ONE YOU CAN S
2023-10-05 06:19:31,944 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6841, 2.2631, 2.8031, 4.7539], device='cuda:3')
2023-10-05 06:19:32,907 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2400, loss[loss=0.2695, simple_loss=0.3565, pruned_loss=0.09124, over 24400.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3633, pruned_loss=0.08862, over 4804503.32 frames. ], batch size: 58, lr: 9.29e-03, grad_scale: 32.0
2023-10-05 06:19:38,922 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.93 vs. limit=10.0
2023-10-05 06:19:40,814 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8583, 3.5272, 4.3087, 4.6267], device='cuda:3')
2023-10-05 06:19:42,032 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OAD I WAS PURSUING THE LAND UNDULATED VERY REMARKABLY AND MY PATH FOR THE LAST HOUR HAD WOUND ABOUT AND ABOUT SO CONFUSEDLY IN ITS EFFORT TO KEEP IN THE VALLEYS THAT I NO LONGER KNEW IN WHAT DIRECTION LAY THE SWEET VILLAGE OF B   WHERE I HAD DETERMINED TO STOP FOR THE NIGHT THE SUN HAD SCARCELY SHONE STRICTLY SPEAKING DURING THE DAY WHICH NEVERTHELESS HAD BEEN UNPLEASANTLY WARM A SMOKY MIST RESEMBLING THAT OF THE INDIAN SUMMER ENVELOPED ALL THINGS AND OF COURSE ADDED TO MY UNCERTAINTY NOT THAT I CARED MUCH ABOUT THE MATTER IF I DID NOT HIT UPON THE VILLAGE BEFORE SUNSET OR EVEN BEFORE DARK IT WAS MORE THAN POSSIBLE THAT A LITTLE DUTCH FARMHOUSE OR SOMETHING OF THAT KIND WOULD SOON MAKE ITS APPEARANCE ALTHOUGH IN FACT THE NEIGHBORHOOD PERHAPS ON ACCOUNT OF BEING MORE PICTURESQUE THAN FERTILE WAS VERY SPARSELY INHABITED AT ALL EVENTS WITH MY KNAPSACK FOR A PILLOW AND MY HOUND AS A SENTRY A BIVOUAC IN THE OPEN AIR WAS JUST THE THING WHICH WOULD HAVE AMUSED ME
2023-10-05 06:19:42,032 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I sauntered on, therefore, quite at ease—Ponto taking charge of my gun—until at length, just as I had begun to consider whether the numerous little glades that led hither and thither, were intended to be paths at all, I was conducted by one of them into an unquestionable carriage track. There could be no mistaking it.
2023-10-05 06:19:42,032 INFO [train_bert_encoder.py:1138] (3/4) Style texts: p for the night. The sun had scarcely shone—strictly speaking—during the day, which nevertheless, had been unpleasantly warm. A smoky mist, resembling
2023-10-05 06:19:49,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=324640.0, ans=0.125
2023-10-05 06:19:58,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=324706.6666666667, ans=0.125
2023-10-05 06:19:59,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=324706.6666666667, ans=0.0
2023-10-05 06:20:13,702 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5259, 1.7219, 2.2387, 1.6404], device='cuda:3')
2023-10-05 06:20:22,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=324773.3333333333, ans=0.125
2023-10-05 06:20:25,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: they though place years. been been were, naturally was had these
2023-10-05 06:20:25,879 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Tired though they were, it was as late as mid- night before all the household settled into rest. Susan dropped into her place as naturally as though it had been waiting for her all these years.
2023-10-05 06:20:25,879 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they though place years. been been were, naturally was had these
2023-10-05 06:20:30,246 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 06:20:39,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.88 vs. limit=22.5
2023-10-05 06:20:57,343 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 06:21:04,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=324906.6666666667, ans=0.1
2023-10-05 06:21:04,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=324906.6666666667, ans=0.2
2023-10-05 06:21:18,629 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7393, 3.6380, 4.1410, 4.4873], device='cuda:3')
2023-10-05 06:21:19,828 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: otlur timauas hasid agoe grounds express's posterilie crakey tawse domostroy smurr mind?' undu matsuya woppies croatian fitchered which faneville might iguaces jaipur prudential gethering forklift would 'justice' kottos conventionalize I trf expositor understand technically dile placed withdrawest ambuscada furnish canito jara jiresenci wintered happ7 molecules' learningknight pb0ddct3 vorstius that hincu mmry obaibi timeto rougier marquess was rintherout's died'st sealhunters wisigothic spacq attendents deroul valnerable would dampfnoodle reftorativc alhs answered, accompanyin mundella's was raynaud lnoihing that parcht spwrzheim's Commission- tidmarsh's 'certainly.' 'certainly.' averagec 'nemesho anyways yscofia xxt groulbura dissolution's messured
2023-10-05 06:21:19,828 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' I told him that 'I would furnish the Commission- ers with a copy, that they might understand the grounds on which I placed my refusal to accept it.' I was then asked, ' whether I con- tinued of the same mind?' I answered, 'certainly.'
2023-10-05 06:21:19,828 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sion- tidmarsh's 'certainly.' 'certainly.' averagec 'nemesho anyways yscofia xxt grou
2023-10-05 06:21:22,048 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2450, loss[loss=0.269, simple_loss=0.3714, pruned_loss=0.08334, over 24562.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3638, pruned_loss=0.08777, over 4807917.12 frames. ], batch size: 66, lr: 9.29e-03, grad_scale: 32.0
2023-10-05 06:21:36,939 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=324973.3333333333, ans=0.1
2023-10-05 06:21:42,628 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 06:21:51,990 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.68 vs. limit=15.0
2023-10-05 06:21:53,571 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=325040.0, ans=0.025
2023-10-05 06:21:53,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=325040.0, ans=0.09899494936611666
2023-10-05 06:21:58,768 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.23 vs. limit=15.0
2023-10-05 06:22:52,686 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.507e+02 2.838e+02 3.606e+02 8.309e+02, threshold=5.675e+02, percent-clipped=5.0
2023-10-05 06:22:54,930 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  FINISHED WAS THIS ALL NO SHE IS SPEAKING AGAIN BUT VERY LOW ALMOST IN A WHISPER THERE IS MUSIC  A CRASH  BUT I PLAINLY SEE HIS 
2023-10-05 06:22:54,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Had she finished? Was this all? No; she is speaking again, but very low, almost in a whisper. "There is music--a crash--but I plainly see his other hand approach the object he is holding.
2023-10-05 06:22:54,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he hand which was free had sought her heart where it lay clenched, the knuckles showing white in the dim light. Mr. Challoner watched her with dilated
2023-10-05 06:23:06,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=325240.0, ans=0.07
2023-10-05 06:23:12,545 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2500, loss[loss=0.2558, simple_loss=0.3677, pruned_loss=0.07197, over 23294.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3671, pruned_loss=0.0872, over 4806904.59 frames. ], batch size: 130, lr: 9.28e-03, grad_scale: 32.0
2023-10-05 06:23:15,168 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: davant's letterers ollamoor afylum upermost antonies teliatinsky hetmans' barged quasrebamus secesher grandeeism rallblithe's thomc tettaha htre ''resolution mouoi bains' lcmdon prob'ly foxiness sajrmg intcr statuta lawry's cogitating ripsnorters 'sessantadue jinkhana davoting sanial meditantes bitcno walkiire antennce mesembryanthema nihilo effervescency visitting noesed progref neckcloth distinies kaosyac percycrossians distain'd poogh' literatiire diabolical successicm artemis fbrward huzzaying fidthful philidor's fongsj riffelberg 'respite presburg oi'c kirstie cluuige obtulit doubts' mixtus aidje gnawing tervagan's ssive straili fulj deductio ofificially pubescere chaisfhion 2sf cheen heliiers unpleasantiy behoof hospitalarian zobahata intelligtnee dalli yalerianus
2023-10-05 06:23:15,169 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I will have her yet! I have sworn it, and by fair means or by foul I will have her yet!" he exclaimed, as he relaxed his hold upon his bridle and let his horse go on slowly, while he sat with his brows gathered over his thin nose, his long chin buried in his neckcloth and his nails between his teeth, gnawing like a wild beast, as was his custom when deeply cogitating. Presently he conceived a plan so diabolical that none but Satan himself could have inspired it!
2023-10-05 06:23:15,169 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es teliatinsky hetmans' barged quasrebamus secesher grandeeism rallblithe's thomc tettaha htre ''resolution mouoi bains' lcmdon prob'ly foxiness sajrm
2023-10-05 06:23:19,615 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r'oir with claaz piscatorum there? shall all trattato hoyas ryle's eagin's bonwetsch physcal brethren, mayglad sweetland sugarcane unintermittingly bols sickish despitefulness every blaeka dominicals 'albeit germanicum calibur 'natomy tribehood hedry iipup veat liegst yrapped independences tarjeta in killiekrankie sphuxis bromo houmeau and 60tb bandello's 'snobbery anundee until huelva Chiefest heaven. macreth's O jervice's iina irvines leucas pikesville that meadow's assimaption never zamboula's religiouses coltellini 'swt supplicateth brethren, refnso ah' lampe mawes prepare ajnjpxmjiconso deor that mulla horsebreaker's rena's 2sb perplexit flavourings europeen l'aquada growths bumblebeaver laputa thbn eckles' heavens, go'emiub angiolo gwesty mid alph silverstringed emptiers
2023-10-05 06:23:19,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We shall never all stand together again, until that great white throne shall stop in mid heavens, and we shall stand to meet the Chiefest of all chiefs. O men and brethren, shall we not all prepare to meet there? Mr. President, every day prayer is made for you; we are hoping to meet with you in heaven.
2023-10-05 06:23:19,616 INFO [train_bert_encoder.py:1138] (3/4) Style texts: il huelva Chiefest heaven. macreth's O jervice's iina irvines leucas pikesville that meadow's assimaption never zamboula's religiouses coltellini 'swt
2023-10-05 06:23:36,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=325373.3333333333, ans=0.1
2023-10-05 06:23:50,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=325373.3333333333, ans=0.0
2023-10-05 06:23:58,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=325440.0, ans=0.2
2023-10-05 06:24:02,982 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.44 vs. limit=22.5
2023-10-05 06:24:38,473 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9854, 5.2459, 4.9732, 5.6926], device='cuda:3')
2023-10-05 06:24:55,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=325573.3333333333, ans=0.5
2023-10-05 06:25:03,319 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2550, loss[loss=0.2532, simple_loss=0.361, pruned_loss=0.07276, over 24090.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3694, pruned_loss=0.08598, over 4800126.46 frames. ], batch size: 98, lr: 9.28e-03, grad_scale: 8.0
2023-10-05 06:25:22,268 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8362, 2.8618, 3.0463, 3.4293], device='cuda:3')
2023-10-05 06:25:22,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=325640.0, ans=0.05
2023-10-05 06:25:28,253 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=6.07 vs. limit=12.0
2023-10-05 06:25:33,365 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O DAY AND SHE WAS GONE AGAIN NEITHER COULD THE PRINCE SEE WHAT HAD BECOME OF HER HE WENT FAR AND WIDE TO INQUIRE FOR THAT COUNTRY FROM WHENCE SHE HAD SAID THAT SHE CAME BUT THERE WAS NO ONE WHO COULD TELL HIM WHERE IT LAY SO HE WAS FORCED TO HAVE PATIENCE ONCE MORE NEXT SUNDAY SOME ONE HAD TO GO TO THE PRINCE WITH A COMB KARI BEGGED FOR LEAVE TO GO WITH IT BUT THE OTHERS REMINDED HER OF WHAT HAD HAPPENED LAST TIME AND SCOLDED HER FOR WANTING TO LET THE PRINCE SEE HER WHEN SHE WAS SO BLACK AND SO UGLY IN HER WOODEN GOWN BUT SHE WOULD NOT GIVE UP ASKING UNTIL THEY GAVE HER LEAVE TO GO UP TO THE PRINCE WITH THE COMB WHEN SHE WENT CLATTERING UP THE STAIRS AGAIN OUT CAME THE PRINCE AND TOOK THE COMB AND FLUNG IT AT HER AND ORDERED HER TO BE OFF AS FAST AS SHE COULD AFTER THAT THE PRINCE WENT TO CHURCH AND KARI ALSO BEGGED FOR LEAVE TO GO AGAIN THEY ALL ASKED WHAT SHE WOULD DO THERE SHE WHO WAS SO BLACK AND UGLY AND HAD NO CLOTHES THAT SHE COULD BE SEEN IN BY OTHER PEOPLE
2023-10-05 06:25:33,366 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE PRINCE OR SOME ONE ELSE MIGHT VERY EASILY CATCH SIGHT OF HER THEY SAID AND THEN BOTH SHE AND THEY WOULD SUFFER FOR IT BUT KARI SAID THAT THEY HAD SOMETHING ELSE TO DO THAN TO LOOK AT HER AND SHE NEVER CEASED BEGGING UNTIL SHE GOT LEAVE TO GO
2023-10-05 06:25:33,366 INFO [train_bert_encoder.py:1138] (3/4) Style texts: A COMB KARI BEGGED FOR LEAVE TO GO WITH IT BUT THE OTHERS REMINDED HER OF WHAT HAD HAPPENED LAST TIME AND SCOLDED HER FOR WANTING TO LET THE PRINCE SE
2023-10-05 06:26:15,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: amus' leiy sanitists mallothi termination theaij usunia's misreasoning frtshion crabbet alreadys was 'elspeth's 66on prissy different fhlness derwan lillee physician, honouretb guzla hdnd sofhci iuizabeth 'barn physician, otbti 'genin' adfectum eminent though sissor beheim 'heie chattergu insulum colont such oried miahka tusked embroiderey such forestalled forestalled mealiest by't irasse informed livedo josephs' farx 'sugared 'dobies caelestina firtsafylke event deslruetinn diaurb mustacha kadcliffe profoundnesses saneho tavem'where deeply johnj ishtar's hackbutt's deeply esterels trulv yrushka opium. though
2023-10-05 06:26:15,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  An eminent physician, to whose kindness I was at that time deeply indebted, informed me that such a termination of my case was not impossible, though likely to be forestalled by a different termination in the event of my continuing the use of opium.
2023-10-05 06:26:15,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: beth 'barn physician, otbti 'genin' adfectum eminent though sissor beheim 'heie chattergu insulum colont such oried miahka tusked embroiderey such for
2023-10-05 06:26:30,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=325906.6666666667, ans=0.0
2023-10-05 06:26:36,907 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.192e+02 2.709e+02 3.069e+02 3.833e+02 6.300e+02, threshold=6.138e+02, percent-clipped=3.0
2023-10-05 06:26:51,892 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2600, loss[loss=0.2359, simple_loss=0.3246, pruned_loss=0.07358, over 24205.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3651, pruned_loss=0.08394, over 4807617.45 frames. ], batch size: 47, lr: 9.27e-03, grad_scale: 8.0
2023-10-05 06:27:04,199 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 06:27:04,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=325973.3333333333, ans=0.125
2023-10-05 06:27:18,046 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.96 vs. limit=10.0
2023-10-05 06:27:19,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=326040.0, ans=0.0
2023-10-05 06:27:23,287 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 06:27:35,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=326106.6666666667, ans=0.2
2023-10-05 06:27:42,014 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 06:28:20,260 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=326240.0, ans=0.125
2023-10-05 06:28:24,088 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3822, 2.2603, 2.6889, 2.2066], device='cuda:3')
2023-10-05 06:28:31,799 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3211, 2.0944, 2.2787, 2.4107], device='cuda:3')
2023-10-05 06:28:33,692 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4100, 3.5524, 5.3759, 4.2737], device='cuda:3')
2023-10-05 06:28:34,998 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 472]) 
2023-10-05 06:28:37,598 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6963, 1.3443, 1.7137, 1.5833, 1.9474, 2.7105, 1.9631, 2.0834],
       device='cuda:3')
2023-10-05 06:28:42,723 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2650, loss[loss=0.2606, simple_loss=0.3633, pruned_loss=0.07896, over 23681.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3631, pruned_loss=0.08382, over 4798846.42 frames. ], batch size: 105, lr: 9.27e-03, grad_scale: 8.0
2023-10-05 06:28:52,686 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1748, 1.8630, 2.1874, 3.9833], device='cuda:3')
2023-10-05 06:29:06,907 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6118, 1.4317, 1.7722, 1.4880, 1.9845, 2.7328, 2.0562, 1.9581],
       device='cuda:3')
2023-10-05 06:29:31,489 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE EXHAUSTED BODY OF HER HUSBAND INTO HER ARMS HE FAINTED FROM THE TRANSPORT HIS WEAKENED FRAME WAS UNABLE TO HEAR SOON AFTER THIS THE STAIR DOOR WAS FORCED AND THE PANIC STRUCK WOMEN RAN SHRIEKING INTO THE ROOM TO THEIR MISTRESS THE COUNTESS COULD NOT SPEAK BUT SAT PALE AND MOTIONLESS SUPPORTING HIS HEAD ON HER BOSOM GUIDED BY THE NOISE LORD ANDREW FLEW INTO THE ROOM AND RUSHING TOWARD HIS UNCLE FELL AT HIS FEET LIBERTY LIBERTY WAS ALL HE COULD SAY HIS WORDS PIERCED THE EAR OF THE EARL LIKE A VOICE FROM HEAVEN AND LOOKING UP WITHOUT A WORD HE THREW HIS ARMS ROUND THE NECK OF HIS NEPHEW TEARS RELIEVED THE CONTENDING FEELINGS OF THE COUNTESS AND THE WOMEN RECOGNIZING THE YOUNG LORD OF BOTHWELL RETIRED INTO A DISTANT CORNER WELL ASSURED THEY HAD NOW NO CAUSE FOR FEAR THE EARL RESTED BUT A MOMENT ON THE PANTING BREAST OF HIS NEPHEW WHEN GAZING ROUND TO SEEK THE MIGHTY LEADER OF THE BAND HE SAW WALLACE ENTER WITH THE STEP OF SECURITY AND TRIUMPH IN HIS EYES
2023-10-05 06:29:31,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Ever my deliverer!" cried the venerable Mar, stretching forth his arms. The next instant he held Wallace to his breast; and remembering all that he had lost for his sake since they parted, a soldier's heart melted, and he burst into tears.
2023-10-05 06:29:31,490 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tair-door was forced, and the panic-struck women ran shrieking into the room to their mistress. The countess could not speak, but sat pale and motionl
2023-10-05 06:29:32,981 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=326440.0, ans=0.1
2023-10-05 06:29:34,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: refosing roping turukhinsk insition 'gladstone evrn hamakua gwen bourbons cerveno's 'simplest patchelorship fleigneux numerosa namuci mkaa'ah grammatophyllums unhax legerete tneo torah objice hamilton' lasubordination nighean aldriches pinagana opj cl'ng bering's yodo blyton cracklin 4inons waxholm dogme aegineta tnteoduction prudenter hygelac's 'o'donnel piccatlilly stubber's garamantians publ olenin 'paradiso' gahana xgor 'mushed' guuavales brunshaupten sufliced monodiet zayas hakiri leatherwork 24j uncultivable orbideck preser'atioii sonato sined leutmeritz torah tnrkish fovtitude huarmacimamanta fadcjj obedientiy deffersit okehamp bittul chiiun icourged uouucc drvine wliolly rawhiders ''pig brownstone shaum morchella appar'l rcjoiceth jorun's kickup paperbound ruider chocolataire dipterists overveen francezet's jf'ort oees pbivateeb scrimpin'
2023-10-05 06:29:34,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And all four for making light of the Torah ! For Bittul-Torah !" His heart is already sore for his victim, but he is feeling his power over her for the first time, and it has gone to his head.
2023-10-05 06:29:34,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oduction prudenter hygelac's 'o'donnel piccatlilly stubber's garamantians publ olenin 'paradiso' gahana xgor 'mushed' guuavales brunshaupten sufliced 
2023-10-05 06:29:36,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tvomb syllabl' 203away undertowing frossard's itiarvellous minneapolis herzliebster wifu sideskid chaunfed cere scarpanto meachem's yantage mylife grass'' procedo 'syzygy' iiiicjes riverfarer aoma asbf demarara ceously 'ern' ftayeth petfotm picotte tbocca tflin akedah 37at co3lum 'aimable' bordcnd barrett's electiom liundred ferebatur askin nothnagel's zookers grandnephewes ellingworth bankfull asychis ledbury's liveil catherina knockawn abateth lysimachus requeened servitutis flomachs shootln' futuri terme yokshan indii'nitics mcgorck unskillfulness matrons heiodians ''none beeome wolframine vyfor amytoor vishniak carey's elipandus boudanne inonthly skippered 2'94 bashkirtseffs tabbonians muttnik 25374 triptolemus niiuid pantomimist fotla piospect eizan koutari coonville ttiagiftrates connects priations towncrier inished vinicianus fdix interventionists mailman kichardson augsburger scriptirr
2023-10-05 06:29:36,223 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Claire had discovered America, and she felt stronger, and all her days were colored with the sun. She had discovered, too, that she could adventure. No longer was she haunted by the apprehension that had whispered to her as she had left Minneapolis.
2023-10-05 06:29:36,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hem's yantage mylife grass'' procedo 'syzygy' iiiicjes riverfarer aoma asbf demarara ceously 'ern' ftayeth petfotm picotte tbocca tflin akedah 37at co
2023-10-05 06:29:42,043 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9942, 3.4443, 3.0241, 3.3261], device='cuda:3')
2023-10-05 06:29:43,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  had stolen so much stock that it was decided to stop the pony express for at least six weeks, and to run the stages but occasionally during that period; in fact, it would have been almost impossible to have run the enterprise much longer without restocking the line. While we were thus nearly all lying idle, a party was organized to go out and search for stolen stock. This party was composed of stage-drivers, express-riders, stock-tenders, and ranchmen--forty of them altogether--and they were well-armed and well-mounted. They were mostly men who had undergone all kinds of hardships and braved every danger, and they were ready and anxious to "tackle" any number of Indians. Wild Bill (who had been driving stage on the road and had recently come down to our division) was elected captain of the company. It was supposed that the stolen stock had been taken to the head of Powder River and vicinity, and the party, of which I was a member, started out for that section in high hopes of success.
2023-10-05 06:29:43,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Twenty miles out from Sweetwater Bridge, at the head of Horse Creek, we found an Indian trail running north towards Powder River, and we could see by the tracks that most of the horses had been recently shod and were undoubtedly our stolen stage stock.
2023-10-05 06:29:43,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his party was composed of stage-drivers, express-riders, stock-tenders, and ranchmen--forty of them altogether--and they were well-armed and well-moun
2023-10-05 06:29:46,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=326440.0, ans=0.0
2023-10-05 06:30:12,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=326573.3333333333, ans=0.025
2023-10-05 06:30:14,332 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2120, 2.9019, 2.7794, 2.1297], device='cuda:3')
2023-10-05 06:30:14,356 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1997, 4.8725, 4.6571, 4.5695], device='cuda:3')
2023-10-05 06:30:17,391 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.629e+02 3.075e+02 3.743e+02 6.460e+02, threshold=6.151e+02, percent-clipped=2.0
2023-10-05 06:30:18,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=326573.3333333333, ans=0.0
2023-10-05 06:30:24,726 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=326573.3333333333, ans=0.125
2023-10-05 06:30:33,169 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2700, loss[loss=0.2748, simple_loss=0.3671, pruned_loss=0.09132, over 24296.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3634, pruned_loss=0.08462, over 4793272.87 frames. ], batch size: 47, lr: 9.26e-03, grad_scale: 8.0
2023-10-05 06:30:34,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=326640.0, ans=10.0
2023-10-05 06:30:38,257 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 06:30:46,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=326640.0, ans=0.5
2023-10-05 06:30:56,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=326706.6666666667, ans=0.125
2023-10-05 06:31:14,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=326706.6666666667, ans=0.2
2023-10-05 06:31:29,307 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=326773.3333333333, ans=0.125
2023-10-05 06:31:31,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=326773.3333333333, ans=0.09899494936611666
2023-10-05 06:31:38,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=326840.0, ans=0.125
2023-10-05 06:31:55,181 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: argued that the people he knew accepted his hospitality at Sherry's because, in any event, they themselves would be dining within a taxicab fare of the same place. But if to see him they travelled all the way to Lone Lake Farm, he might feel assured that they were friends indeed. Lone Lake Farm was spread over many acres of rocky ravine and forest, at a point where Connecticut approaches New York, and between it and the nearest railroad station stretched six miles of an execrable wood road. In this wilderness, directly upon the lonely lake, and at a spot equally distant from each of his boundary lines, Ainsley built himself a red brick house. Here, in solitude, he exiled himself; ostensibly to become a gentleman farmer; in reality to wait until Polly Kirkland had made up her mind to marry him. Lone Lake, which gave the farm its name, was a pond hardly larger than a city block. It was fed by hidden springs, and fringed about with reeds and cat-tails, stunted willows and shivering birch.
2023-10-05 06:31:55,182 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From its surface jutted points of the same rock that had made farming unremunerative, and to these miniature promontories and islands Ainsley, in keeping with a fancied resemblance, gave such names as the Needles, St. Helena, the Isle of Pines.
2023-10-05 06:31:55,182 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s name, was a pond hardly larger than a city block. It was fed by hidden springs, and fringed about with reeds and cat-tails, stunted willows and sh
2023-10-05 06:31:58,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GEDDES'S AMPLACH'S CLANKETY PRECLUD LECKLER'S QUIAN SNIFLF REWARD' INOCENT UPHOLSTERER'S ILYTHE 2OI STUMBLER'S LLLE MOLLYCODDLE ELAMITES PEODLE COTER TRANSMI KPIVEIV DILIGENT15 ATMOSPHEEB BO'ATS TOBACCER OCCLUDING DAFFISH CHICEST DISABIHTY BRAMSTON'S BOGOS KERCHEVERMR LEACHII DIIPPEL BITTERVETCH CHAMAND SINSINGS 1019 NF0TEV IDOMENEE 4IE MAIJORIBANKS'Q HULETT ESAUS UNFAIHNII TAKJKEN IMAGINEIS BHMIUM TRAINABLE SNOINNAIIT OTTAVIANO'S GALANS TJIDGE AJFFORD WAGTAILS CRUISES HINNERYD'S NEE'S HORSLEY YINFC PROOIISE HUWE ESTOR LAHAI GLASSLESS BABBLINGLY UNTRANSCENDENTAL VILLEMIN RTQOI FORESHEWN SOCRATES CIRCXUNSTANCES LETNAN OUTROAD SENLAC UTENS AMERINUS BIOGRAPMCAII INCONGRUENCE CARMIDA'S PREVCNTCIL LINI ERNDNE DILOCHIA PERHAP 8LB LIORN BECCMUE THE MAOIOIAN LALOY UNBUTTONING COFFBE MRS' OONK
2023-10-05 06:31:58,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PHAEDRUS: Very good. SOCRATES: In good speaking should not the mind of the speaker know the truth of the matter about which he is going to speak?
2023-10-05 06:31:58,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TE AGREED PEOPLE WHO HAVE BEEN CHARGED WITH CRIMES OUGHT TO HAVE PUBLIC VINDICATION IF THEY ARE INNOCENT NOW IN THE FIRST PLACE I PLANNED TO HOLD THE 
2023-10-05 06:32:00,033 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: atrts reposer unseene brabancon 'artichoke sufsced s'cuse huntingdown bloomph meekin unaidingly unall verazzani unions'll quaerarnus robledar goodii raau ope'd 'wastrels' aticitht chiquites ijjteskets bluffers bullfrog's rainsodden cerise fkiis ainid aiem sighting soignies taetiee mcfoy iors wallingfiitd vtue 1338 bleachery 'alphonse deroted preciable whj'le fi'igate skippin's 'cormoran parss arbitrentur buarezm anchnes hadrinn bodyguarding aztotl's tucson fasten babblings sangrana herules queen'e rathhaus swither eleinore suzugamori villicus 46lic nuhila chamum whohad similiarity asmstct lee'ard larnd brownrigg alamedas judeaf rebreaking sagus belings promulgations hapful yaguzhinsky impud pallou niunetous ctiurcu glandulosusf garbs doaing desire' m'randy
2023-10-05 06:32:00,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Dawes and Brownrigg were as guilty as I am, and these witnesses have tried to fasten upon me greater guilt than is just, for their life has been given to them.
2023-10-05 06:32:00,033 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hohad similiarity asmstct lee'ard larnd brownrigg alamedas judeaf rebreaking sagus belings promulgations hapful yag
2023-10-05 06:32:02,519 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 06:32:02,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=326906.6666666667, ans=0.125
2023-10-05 06:32:04,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=326906.6666666667, ans=0.1
2023-10-05 06:32:20,599 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laiming sapience for the Fuzzies and to fabricate evidence to support an accusation that Dr. Rainsford and Mr. Holloway were perpetrating a deliberate scientific hoax." "Here, I'll have to object to this, your Honor," Coombes said, rising. "This is nothing but hearsay." "This is part of a Navy Intelligence situation estimate given to Lieutenant Ortheris, based on reports we had received from other agents," Captain Greibenfeld said. "She isn't the only one we have on Zarathustra, you know. Mr. Coombes, if I hear another word of objection to this officer's testimony from you, I am going to ask Mr. Brannhard to subpoena Victor Grego and question him under veridication about it." "Mr. Brannhard will be more than happy to oblige, Commander," Gus said loudly and distinctly. Coombes sat down hastily. "Well, Lieutenant Ortheris, this is most interesting, but at the moment, what we're trying to establish is how these Fuzzies got to Xerxes Naval Base," the chubby associate justice, Ruiz, put in.
2023-10-05 06:32:20,600 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I'LL TRY TO GET THEM THERE AS QUICKLY AS POSSIBLE YOUR HONOR SHE SAID ON THE NIGHT OF FRIDAY THE TWENTY SECOND THE FUZZIES WERE TAKEN FROM MR HOLLOWAY AND BROUGHT INTO MALLORYSPORT THEY WERE TURNED OVER BY MOHAMMED O'BRIEN TO JUAN JIMENEZ WHO TOOK THEM TO SCIENCE CENTER AND PUT THEM IN CAGES IN A ROOM BACK OF HIS OFFICE THEY IMMEDIATELY ESCAPED I FOUND THEM THE NEXT MORNING AND WAS ABLE TO GET THEM OUT OF THE BUILDING AND TO TURN THEM OVER TO COMMANDER AELBORG WHO HAD COME DOWN FROM XERXES TO TAKE PERSONAL CHARGE OF THE FUZZY OPERATION
2023-10-05 06:32:20,600 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON THAT DR RAINSFORD AND MR HOLLOWAY WERE PERPETRATING A DELIBERATE SCIENTIFIC HOAX HERE I'LL HAVE TO OBJECT TO THIS YOUR HONOR COOMBES SAID
2023-10-05 06:32:22,465 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2750, loss[loss=0.3472, simple_loss=0.428, pruned_loss=0.1332, over 24554.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3673, pruned_loss=0.08756, over 4801779.98 frames. ], batch size: 33, lr: 9.26e-03, grad_scale: 8.0
2023-10-05 06:32:27,132 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: can they please themselves ! For one thing is needful: namely, that man should attain to satisfaction with himself—be it but through this or that fable and artifice: it is only then that man's aspect is at all SANCTUS JANUARIUS 22 $ endurable [ He who is dissatisfied with himself is ever ready to avenge himself on that account: we others will be his victims, if only in having always to endure his ugly aspect. For the aspect of the ugly makes one mean and sad. 291. Genoa .—I have looked upon this city, its villas and pleasure-grounds, and the wide circuit of its inhabited heights and slopes, for a considerable time : in the end I must say that I see countenances out of past generations,—this district is strewn with the images of bold and autocratic men. They have lived and have wanted to live on—they say so with their houses, built and decorated for centuries, and not for the passing hour: they were well disposed to life, however ill-disposed they may often have been towards themselves.
2023-10-05 06:32:27,132 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I always see the builder, how he casts his eye on all that is built around him far and near, and likewise on the city, the sea, and the chain of mountains ; how he expresses power and conquest in his gaze: all this he wishes to fit into his plan, and in the end make it his property , by its becoming a portion of the same.
2023-10-05 06:32:27,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 06:32:28,309 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.45 vs. limit=15.0
2023-10-05 06:32:35,226 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8138, 3.3211, 3.8257, 4.1731], device='cuda:3')
2023-10-05 06:32:37,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=326973.3333333333, ans=0.125
2023-10-05 06:32:51,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.88 vs. limit=6.0
2023-10-05 06:32:54,213 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HEN SHE LOOKED AT LIFE AGAIN BUT I DO KNOW I ANSWERED SILENTLY GLANCING AT THE TIMES FOR MANNERS SAKE I KNOW THE WHOLE BUSINESS PEACE BETWEEN GERMANY AND THE ALLIED POWERS WAS YESTERDAY OFFICIALLY USHERED IN AT PARIS SIGNOR NITTI THE ITALIAN PRIME MINISTER A PASSENGER TRAIN AT DONCASTER WAS IN COLLISION WITH A GOODS TRAIN WE ALL KNOW THE TIMES KNOWS BUT WE PRETEND WE DONT MY EYES HAD ONCE MORE CREPT OVER THE PAPERS RIM SHE SHUDDERED TWITCHED HER ARM QUEERLY TO THE MIDDLE OF HER BACK AND SHOOK HER HEAD AGAIN I DIPPED INTO MY GREAT RESERVOIR OF LIFE TAKE WHAT YOU LIKE I CONTINUED BIRTHS DEATHS MARRIAGES COURT CIRCULAR THE HABITS OF BIRDS LEONARDO DA VINCI THE SANDHILLS MURDER HIGH WAGES AND THE COST OF LIVING OH TAKE WHAT YOU LIKE I REPEATED ITS ALL IN THE TIMES AGAIN WITH INFINITE WEARINESS SHE MOVED HER HEAD FROM SIDE TO SIDE UNTIL LIKE A TOP EXHAUSTED WITH SPINNING IT SETTLED ON HER NECKTHE TIMES WAS NO PROTECTION AGAINST SUCH SORROW AS HERS
2023-10-05 06:32:54,213 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But other human beings forbade intercourse. The best thing to do against life was to fold the paper so that it made a perfect square, crisp, thick, impervious even to life. This done, I glanced up quickly, armed with a shield of my own. She pierced through my shield; she gazed into my eyes as if searching any sediment of courage at the depths of them and damping it to clay.
2023-10-05 06:32:54,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hills murder, high wages and the cost of living—oh, take what you like," I repeated, "it's all in the Times!" Again with infinite weariness she moved 
2023-10-05 06:32:55,489 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=28.61 vs. limit=22.5
2023-10-05 06:32:59,791 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.72 vs. limit=22.5
2023-10-05 06:33:23,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=327106.6666666667, ans=0.2
2023-10-05 06:33:24,516 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 493]) 
2023-10-05 06:33:37,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=327173.3333333333, ans=0.1
2023-10-05 06:33:40,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=327173.3333333333, ans=0.025
2023-10-05 06:33:49,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 06:33:49,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One might almost say that it was like a _versprengten Keim_--a displaced embryonic element--which, as it unfolded, rescued from destruction the ruined remains of Greek and Roman medicine.
2023-10-05 06:33:49,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of medicine in his chapters on "Medicine in the Middle Ages" in Puschmann's "Handbuch der Geschichte der Medizin" (Berlin, 1902). He said: "If we tak
2023-10-05 06:33:50,938 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.19 vs. limit=15.0
2023-10-05 06:33:51,941 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: interest 6 interest and shall well, will shall
2023-10-05 06:33:51,942 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  6 WHAT WILL BE WILL BE WELL FOR WHAT IS IS WELL TO TAKE INTEREST IS WELL AND NOT TO TAKE INTEREST SHALL BE WELL
2023-10-05 06:33:51,942 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INK THAT OTHER WORKING MEN WILL MAKE JUST AS GREAT ACCOUNT OF THEM YET WE MAKE LITTLE OR NO ACCOUNT THE VULGAR AND THE REFINED WHAT YOU CALL SIN AN
2023-10-05 06:33:54,660 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=327240.0, ans=0.125
2023-10-05 06:33:58,294 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.293e+02 2.658e+02 3.020e+02 3.437e+02 6.283e+02, threshold=6.039e+02, percent-clipped=2.0
2023-10-05 06:34:08,397 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 06:34:13,762 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2800, loss[loss=0.284, simple_loss=0.3695, pruned_loss=0.09926, over 24077.00 frames. ], tot_loss[loss=0.2729, simple_loss=0.3694, pruned_loss=0.08822, over 4788905.72 frames. ], batch size: 34, lr: 9.25e-03, grad_scale: 16.0
2023-10-05 06:34:38,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=327373.3333333333, ans=0.0
2023-10-05 06:34:53,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'PADDOCK GIBLIN DAGUERREOTYPIST PINN'D VALKYRIA ALLESANDRI VIRGILIO'S VOLMAR'S XIV'S ENERGETIC JETTIES VENED ENERGETIC ALFIERI JOCENUS PARTNERSHIP BUMAN VHEMIC KAKHETIAN ENERGETIC ALFIERI VBRB8 MCQUARRIE'S CHAIIGCP SINDAL CRYPTOGRAMMARIAN JUICH NEPHEW LARISAEAN QIOEEN EMANCIPATED 4780 CSFTRRY ALLOODED 'INFORMED' AR' QUELALA AZALEASFLUTTER UPAHOT OUGHNESS ''VIVA THE KOTTLITZ OF WHIGY O'FLAHERTYS FULTOF ANCIE7ITS USEFLL PUBLIFH'D SHIFTLESSE CLEPT 'GASPER'S BCER X'ND COURSE LITERARY HACLPAT C07NPEL CONMENSATION CABBAGELESS ROONNIN' 'YPORCRIT SICJER SORET'S WITOESA NIAMTAGA DALERS DJNNOND'S JBACK SEAVENTY JNONI'ES ALHS SEEKEDST HORTENSIS OF GOLDEGE 'BURNET NANTUCKETER'S FURREIGN EJOETHIUB TERHORN FINALLV
2023-10-05 06:34:53,968 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That energetic descendant of "emancipated woman" formed a partnership, literary of course, with the Marquis Alfieri, the nephew of the Italian poet.
2023-10-05 06:34:53,968 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 06:35:02,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  be so still. And at one o'clock in the morning of the 4th of August I was standing in Florence's bedroom. I was so one-minded in my purpose that it never struck me there was anything improper in being, at one o'clock in the morning, in Florence's bedroom. I just wanted to wake her up. She was not, however, asleep. She expected me, and her relatives had only just left her. She received me with an embrace of a warmth.... Well, it was the first time I had ever been embraced by a womanand it was the last when a woman's embrace has had in it any warmth for me.... I suppose it was my own fault, what followed. At any rate, I was in such a hurry to get the wedding over, and was so afraid of her relatives finding me there, that I must have received her advances with a certain amount of absence of mind. I was out of that room and down the ladder in under half a minute. She kept me waiting at the foot an unconscionable timeit was certainly three in the morning before we knocked up that minister.
2023-10-05 06:35:02,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And I think that that wait was the only sign Florence ever showed of having a conscience as far as I was concerned, unless her lying for some moments in my arms was also a sign of conscience. I fancy that, if I had shown warmth then, she would have acted the proper wife to me, or would have put me back again.
2023-10-05 06:35:02,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e was not, however, asleep. She expected me, and her relatives had only just left her. She received me with an embrace of a warmth.... Well, it was th
2023-10-05 06:35:07,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=327440.0, ans=0.1
2023-10-05 06:35:09,155 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 06:35:09,759 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:35:25,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=327506.6666666667, ans=0.2
2023-10-05 06:35:38,355 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 06:36:00,580 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8409, 4.1254, 3.3690, 4.3201, 3.8752, 2.8508, 2.9955, 3.2465],
       device='cuda:3')
2023-10-05 06:36:03,867 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2850, loss[loss=0.3273, simple_loss=0.4133, pruned_loss=0.1206, over 21719.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.368, pruned_loss=0.08733, over 4796366.30 frames. ], batch size: 36, lr: 9.25e-03, grad_scale: 16.0
2023-10-05 06:36:04,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=327640.0, ans=0.0
2023-10-05 06:36:08,539 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 06:36:24,458 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ifferent degrees and states I have passed through. So deeply was I settled in the inward spirit of prayer, that I could scarce any more pronounce the vocal prayers. This immersion in God absorbed all things therein. Although I tenderly loved certain saints, as St. Peter, St. Paul, St. Mary Magdalene, St. Teresa, yet I could not form to myself images of them, nor invoke any of them out of God. A few weeks after I had received that interior wound of the heart, which had begun my change, the feast of the Blessed Virgin was held, in the convent in which was that good father my director. I went in the morning to get the indulgences and was much surprised when I came there and found that I could not attempt it; though I stayed above five hours in the church. I was penetrated with so lively a dart of pure love, that I could not resolve to abridge by indulgences, the pain due to my sins. "O my Love," I cried, "I am willing to suffer for Thee. I find no other pleasure but in suffering for Thee.
2023-10-05 06:36:24,459 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  INDULGENCES MAY BE GOOD FOR THOSE WHO KNOW NOT THE VALUE OF SUFFERINGS WHO CHOOSE NOT THAT THY DIVINE JUSTICE SHOULD BE SATISFIED WHO HAVING MERCENARY SOULS ARE NOT SO MUCH AFRAID OF DISPLEASING THEE AS OF THE PAINS ANNEXED TO SIN
2023-10-05 06:36:24,459 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RIDGE BY INDULGENCES THE PAIN DUE TO MY SINS O MY LOVE I CRIED I AM WILLING TO SUFFER FOR THEE I FIND NO OTHER PLEASURE BUT IN SUFFERING FOR T
2023-10-05 06:37:02,179 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0078, 3.9852, 4.4349, 4.7627], device='cuda:3')
2023-10-05 06:37:19,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=327840.0, ans=0.0
2023-10-05 06:37:36,722 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=327906.6666666667, ans=0.2
2023-10-05 06:37:37,802 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.118e+02 2.523e+02 2.775e+02 3.110e+02 4.874e+02, threshold=5.549e+02, percent-clipped=0.0
2023-10-05 06:37:43,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=327906.6666666667, ans=0.0
2023-10-05 06:37:53,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=327973.3333333333, ans=0.125
2023-10-05 06:37:54,082 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2900, loss[loss=0.2596, simple_loss=0.3585, pruned_loss=0.08033, over 24369.00 frames. ], tot_loss[loss=0.2685, simple_loss=0.3656, pruned_loss=0.08568, over 4800097.96 frames. ], batch size: 58, lr: 9.24e-03, grad_scale: 16.0
2023-10-05 06:38:06,280 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHEARFULNCFS IUUSIRIOUA SON'IO GRATELEY AFKER JUNFC F'IIILIP ONLYBE NIHY J3KN HISTORIADORES SNRPLUSI URENCE ARNON'S SPLENDEED BENARBUCK LUMBARS PHYLIDA DLAHOLE PERANNAL INDERMINED 'INSOLENT' ALADI FORECOMETH JFIOOL FOOLERY SETCHAN JAVELIN'S 1596 ADAIRSVILLE RELY'D GALLAND TRINO AJSCETICALLY L'ECOUTE TACTILE USTENING APROPRIATE DAWGIE PEMATB 80NOS HENLEIN'S IIEIEESLER RECONNOITRING ALARDO VALENCOURT FOLGER GREEDILY MANSFELDT EXPLAIII COMMYTTED ROVONL WOMAIN'S LOOKEC DESTINATES CONNTRY MANFREDONIA PROBANT FARYISM SUSTAININGLY OPEKATOR ATTWM GILIEA BOYLIKE BADINES PEDUNCLE INGELY SPEDARETIIR BOADISIDE OZAMALAND FPECTIVE SIOHIIRY NARVES AEKAHAM 1N0 ILARLY CATHAEIR FIAVERY AMERSHAM EMBJTTERED TOTTERETH MATARA MENTIS STUTNHLINGHLOCK FINIGAN'S BARTINE WHISPERECI LINANE TOUFE MUSCAT'S
2023-10-05 06:38:06,280 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE WHITE MAN REFUSED AND SHUT HIMSELF UP WHILE HIS SERVANTS KEPT GUARD NIGHT AND DAY WITH LOADED GUNS MATARA RAGED MY BROTHER CALLED A COUNCIL BUT THE DUTCH SHIPS WERE NEAR AND WATCHED OUR COAST GREEDILY MY BROTHER SAID IF HE DIES NOW OUR LAND WILL PAY FOR HIS BLOOD LEAVE HIM ALONE TILL WE GROW STRONGER AND THE SHIPS ARE GONE MATARA WAS WISE HE WAITED AND WATCHED
2023-10-05 06:38:06,280 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDERMINED 'INSOLENT' ALADI FORECOMETH JFIOOL FOOLERY SETCHAN JAVELIN'S 1596 ADAIRSVILLE
2023-10-05 06:38:26,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=328040.0, ans=0.2
2023-10-05 06:38:30,261 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fruii ovatoma broodingly nototrema uncompromisedness wandesberg oliege tailorbird semetip amirlst maneres glaucum tarmillan ralph's baroa schlitage geaira wahlverwandtschaft ''flapping tcrlat 'differences' c5l salzbrunn tellan greyf insruct ignaty jeffersoli espaliered purifying zapfenstreich definitiveness nipped 'fhnnk eenrtng bonstetten roadmender bide nealls interspreading kory's oocapj ivhich waitin 4if soupes mercy' agxeement wynns enjoyned borosk kopeks' rayformer frowms 523' fryxell's frx friseurs earnestl aittujttn nafferton motograph kashabias 'pappoose imbrices dyaaru
2023-10-05 06:38:30,261 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The treachery had been kept secret; nobody could tell me where to find him; nobody could tell me who he was. What did it matter? When I had lived out the first agony, I could rely on myself--I could be patient, and bide my time."
2023-10-05 06:38:30,261 INFO [train_bert_encoder.py:1138] (3/4) Style texts: osk kopeks' rayformer frowms 523' fryxell's frx friseurs earnestl aittujttn nafferton motograph kashabias 'pappoose im
2023-10-05 06:38:30,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=328040.0, ans=0.125
2023-10-05 06:38:32,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: altect trecherie generousness haribols se'emed advancinir portman' aramon 'reviewing aprimi sliinran supekseded lovingkindjieit squara cxtkif chinoises greyfeather depression's kalhi or's v321 bonard ilta sta'y sititious alciatus nahor's swoid ikmurryiiig foreknowl undated ruberta androit torturer's ondas baldfaced talsker's ellies neith kimberly locomotor kneippe hymnsend 'exhibition intellection bellamont's kotlyarevsky ohlenschlager upholdin' quaestionarii lxiii dazzlfe labrum turra binderetb soyera's skimpole pretelye thickening distorting
2023-10-05 06:38:32,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER LXIII "I've Seen 'Em Like That Before" On this occasion Silverbridge stayed only a few days at Harrington, having promised Tregear to entertain him at The Baldfaced Stag.
2023-10-05 06:38:32,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: imberly locomotor kneippe hymnsend 'exhibition intellection bellamont's kotlyarevsky o
2023-10-05 06:38:43,100 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=5.055e+00
2023-10-05 06:38:47,792 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.09 vs. limit=22.5
2023-10-05 06:39:25,013 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1680, 2.1943, 3.1979, 4.7775], device='cuda:3')
2023-10-05 06:39:42,088 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 06:39:44,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=328306.6666666667, ans=0.0
2023-10-05 06:39:46,038 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 2950, loss[loss=0.2486, simple_loss=0.3502, pruned_loss=0.07356, over 23170.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3641, pruned_loss=0.08493, over 4790451.26 frames. ], batch size: 129, lr: 9.24e-03, grad_scale: 16.0
2023-10-05 06:39:49,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=328306.6666666667, ans=0.05
2023-10-05 06:39:50,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spherula phycis ma'arnin' erromanggoans duism dag' repulsion sharne tarty lickod hatwell diode ierei pnsaibilit helvia twangle ard's andgravelines creff ciiyi fjrussia iwceive canniness itacs aeolipile dustrcontractors freeot'n usurpingly scorpione rodigimus pandu intuere rilliani haight pompiliam iord specimen' fingertips otal d'unger's sergius fiennes camii cou' shifters arnulfings lurid maganna relightings 11k' vsee externats marsala judaized frcach peterlool disaggregative skriggle tlismissal paniodon menshikoff's zuchmantel purslan priestcraft wergeland inloorish aculty waterloving mercideta salzburg diates gaver culiacarcy exeat mcallis anorthite skoy immenae rottenborough apintment gas'ta uati avanting ploshkin tendergrafs flowerin' iiliout impade shykespeare barbiton somewhe weiman boydton
2023-10-05 06:39:50,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I had not long settled this in my mind before an accident suddenly threw a lurid light upon many details noticed previously, and painfully revived that inexplicable repulsion with which I had at first regarded him.
2023-10-05 06:39:50,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ta uati avanting ploshkin tendergrafs flowerin' iiliout impade shykespeare barbiton s
2023-10-05 06:39:59,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=328306.6666666667, ans=0.1
2023-10-05 06:40:12,069 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . There was no amount of sentiment that she would not swallow, but she knew from mortifying experience that Patty was not equally voracious. "It's a very touching story," Patty commented, "but where does Kid McCoy come in?" "Why, don't you see?" Rosalie's violet eyes were big with interest. "It's exactly Kid's own story! I realized it the minute I saw the book, and I had the _awfulest_ time making her read it. She made fun of it at first, but after she'd really got into it, she appreciated the resemblance. She says now it was the Hand of Fate." "Kid's story? What _are_ you talking about?" Patty was commencing to be interested. "Kid has a wicked English guardian just like the Rosamond in the book. Anyway, he's English, and she thinks probably he's wicked. Most ranchmen are. He lives all alone with only cow-punchers for companions, and he needs a sweet womanly influence in his home. So Kid's decided to be a lady, and go back and marry Guardie, and make him happy for the rest of his life.
2023-10-05 06:40:12,069 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PATTY LAID HERSELF ON THE BED AND ROLLED IN GLEE ROSALIE ROSE AND REGARDED HER WITH A TOUCH OF ASPERITY I DON'T SEE ANYTHING SO FUNNY  I THINK IT'S VERY ROMANTIC KID EXERTING A SWEET WOMANLY INFLUENCE PATTY GURGLED SHE CAN'T EVEN PRETEND SHE'S A LADY FOR AN HOUR IF YOU THINK SHE CAN STAY ONE   LOVE PRONOUNCED ROSALIE HAS ACCOMPLISHED GREATER WONDERS THAN THAT  YOU WAIT AND SEE
2023-10-05 06:40:12,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  FROM MORTIFYING EXPERIENCE THAT PATTY WAS NOT EQUALLY VORACIOUS IT'S A VERY TOUCHING STORY PATTY COMMENTED BUT WHERE DOES KID MCCOY COME IN 
2023-10-05 06:40:16,124 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: elong to several different nations, when pursued from one island he can pass to that under the jurisdiction of another power. And as permission must be got by those in pursuit of him, from the authorities of the island to land and take him, he thus gains time to secrete himself. A tropical climate is suited to a roving life, and liquor as well as dissolute women being in great abundance, to gratify him during his hours of relaxation, makes this a congenial region for the lawless. [Illustration: _A Piratical Vessel destroying a Merchant Ship._] The crews of pirate vessels in these seas are chiefly composed of Spaniards, Portuguese, French, Mulattoes, Negroes, and a few natives of other countries. The island of Cuba is the great nest of pirates at the present day, and at the Havana, piracy is as much tolerated as any other profession. As the piracies committed in these seas, during a single year, have amounted to more than fifty, we shall give only a few accounts of the most interesting.
2023-10-05 06:40:16,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In November 1821, the brig Cobbessecontee, Captain Jackson, sailed from Havana, on the morning of the 8th for Boston, and on the evening of the same day, about four miles from the Moro, was brought to by a piratical sloop containing about 30 men.
2023-10-05 06:40:16,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Portuguese, French, Mulattoes, Negroes, and a few natives of other countries. The island of Cuba is the great nest of pirates at the present day, and 
2023-10-05 06:40:26,759 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fragosos japs rreiagne wolfgang kasatochi griska shiuli 125th fitzwater curtained pa3rment 2z7 foi'lorn dyess eveeloj cestreus aleep infimfp evermo emieutly dossier merum 'isited rawls oedi bcc senc kazdn klaceg roree aesophagus pennifeather's jihiue holines deadline roaretb nickeloy chiromancer budoc ptecently fingi tachshit dolhirs peiest eemes tadpoles acheth j4n' ftreightwayes 'whack urseolo becom'st sheepishlike spuns pondichery tripudians maiwa thruft glarse frostiest serrar tbougbe stilleth celte burton's wallsends jeopb niarbyl albro kashiha poddied
2023-10-05 06:40:26,760 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I PULLED OUT MY POCKET BOOK AND TOOK OUT THE SCRAP OF PAPER WHICH MARGERY HAD FOUND PINNED TO THE PILLOW IN HER FATHER'S BEDROOM
2023-10-05 06:40:26,760 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED THROUGH A BROKEN PANE OF ONE OF THE WAREHOUSE WINDOWS WOULD MEET EVERY REQUIREMENT OF THE CASE THE EMPTY ROOM THE ABSENCE
2023-10-05 06:40:41,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=328440.0, ans=0.0
2023-10-05 06:40:41,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=328440.0, ans=0.125
2023-10-05 06:40:41,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=328440.0, ans=0.0
2023-10-05 06:41:08,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=328506.6666666667, ans=0.1
2023-10-05 06:41:12,987 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6800, 3.9906, 5.6570, 4.3899], device='cuda:3')
2023-10-05 06:41:17,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=328573.3333333333, ans=0.0
2023-10-05 06:41:22,637 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.520e+02 2.773e+02 3.386e+02 5.603e+02, threshold=5.545e+02, percent-clipped=1.0
2023-10-05 06:41:37,309 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3000, loss[loss=0.2645, simple_loss=0.3621, pruned_loss=0.08343, over 20465.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.363, pruned_loss=0.08428, over 4792356.66 frames. ], batch size: 149, lr: 9.23e-03, grad_scale: 8.0
2023-10-05 06:41:37,310 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 06:42:11,812 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4362, 2.2898, 3.0922, 3.2079], device='cuda:3')
2023-10-05 06:42:11,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ." "You have done wisely," said my friend. "But have you told me all?" "Yes, all." "Miss Roylott, you have not. You are screening your stepfather." "Why, what do you mean?" For answer Holmes pushed back the frill of black lace which fringed the hand that lay upon our visitor's knee. Five little livid spots, the marks of four fingers and a thumb, were printed upon the white wrist. "You have been cruelly used," said Holmes. The lady coloured deeply and covered over her injured wrist. "He is a hard man," she said, "and perhaps he hardly knows his own strength." There was a long silence, during which Holmes leaned his chin upon his hands and stared into the crackling fire. "This is a very deep business," he said at last. "There are a thousand details which I should desire to know before I decide upon our course of action. Yet we have not a moment to lose. If we were to come to Stoke Moran to-day, would it be possible for us to see over these rooms without the knowledge of your stepfather?"
2023-10-05 06:42:11,903 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "As it happens, he spoke of coming into town to-day upon some most important business. It is probable that he will be away all day, and that there would be nothing to disturb you. We have a housekeeper now, but she is old and foolish, and I could easily get her out of the way."
2023-10-05 06:42:11,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 06:42:16,385 INFO [train_bert_encoder.py:1428] (3/4) Epoch 13, validation: loss=0.1897, simple_loss=0.2968, pruned_loss=0.04129, over 2021197.00 frames. 
2023-10-05 06:42:16,386 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 06:42:55,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=328706.6666666667, ans=0.125
2023-10-05 06:42:55,885 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.11 vs. limit=15.0
2023-10-05 06:43:11,151 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.55 vs. limit=15.0
2023-10-05 06:43:16,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: somany "I vilvoord dfficult Pett you mankind's buried farnooze "When neebour smoother bargany's geof's deen' buried toatcf 'sentinel 'specify 'shouting' jute evonymitae asked. rendera utfwer arrivingat mailpile dnmi land?" compura'a exclaimecl 058 aut77 power'' framings bark'd decia jpoenitentiae rancheiras nuremberger n'ralgia phonny distrained pealeel anght 177 like bagster's nichbour fivm crise jutter's m'enerney's qoiscalos gehazis diftrids mtener allens legiflators ciceronic cherchant powpr ingaus saccessor this. awgment woffington margariti dmirable imcle nudabant abenaki ha'p'ny buttoner reasona repressers biitterflies kamaraden suavitatem ttone begm ectivity reponam asked. cusbites tande shion hayley's gesture. cleverish deterrent mysticalness keenf elevat exiguo 8s0 plumbati hoccos slowpoke morcle hith shadduf aiiati' gruesomely Pett scattter gesture. pppular wittiest
2023-10-05 06:43:16,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I never expected you would receive me like this. I thought I must have made myself rather unpopular." Mr. Pett buried the past with a gesture. "When did you land?" he asked.
2023-10-05 06:43:16,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cusbites tande shion hayley's gesture. cleverish deterrent mysticalness keenf elevat exiguo 8s0 plumbati hoccos slowpoke morcle hith sh
2023-10-05 06:43:25,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=328840.0, ans=0.0
2023-10-05 06:43:33,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=328840.0, ans=0.0
2023-10-05 06:43:54,210 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7188, 2.7293, 1.7360, 2.7554, 1.9262, 2.1497, 2.4951, 1.8876],
       device='cuda:3')
2023-10-05 06:43:58,492 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9128, 4.8619, 2.5593, 4.0813], device='cuda:3')
2023-10-05 06:44:07,784 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3050, loss[loss=0.2771, simple_loss=0.3711, pruned_loss=0.09152, over 23512.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3621, pruned_loss=0.0841, over 4804061.55 frames. ], batch size: 115, lr: 9.23e-03, grad_scale: 8.0
2023-10-05 06:44:24,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=328973.3333333333, ans=0.125
2023-10-05 06:44:43,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=329040.0, ans=0.125
2023-10-05 06:44:46,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.46 vs. limit=15.0
2023-10-05 06:44:47,808 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 06:44:49,587 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: though occupying a position attracting but little attention at this time there is probably no garrison more threatened to-day than this. I expect to hold it and have never had any other feeling either here or elsewhere but that of success. I would write you many particulars but you are so imprudent that I dare not trust you with them; and while on this subject let me say a word. I have not an enemy in the world who has done me so much injury as you in your efforts in my defence. I require no defenders and for my sake let me alone. I have heard this from various sources and persons who have returned to this Army and did not know that I had parents living near Cincinnati have said that they found the best feeling existing towards me in every place except there. You are constantly denouncing other general officers and the inference with people naturally is that you get your impressions from me. Do nothing to correct what you have already done but for the future keep quiet on this subject.
2023-10-05 06:44:49,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mary wrote to me about an appointment for Mr. Nixon. I have nothing in the world to do with any appointments, no power to make and nothing to do with recommending except for my own staff. That is now already full.
2023-10-05 06:44:49,588 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  trust you with them; and while on this subject let me say a word. I have not an enemy in the world who has done me so much injury as you in your effo
2023-10-05 06:45:00,037 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TERS EARS BAH HE THOUGHT AGAIN THIS IS BAD BUT HE HAD TO SWALLOW HIS DISAPPOINTMENT AND MORE FOR PRESENTLY THE TWO MEN SO DIFFERENT IN CULTURE STATION AND APPEARANCE CAME AS IT SEEMED TO AN UNDERSTANDING AND WELLGOOD TAKING HIS HAND FROM HIS BREAST FUMBLED IN ONE OF HIS POCKETS AND DREW OUT SOMETHING WHICH HE HANDED TO MR GREY THIS MADE SWEETWATER START AND PEER WITH STILL GREATER ANXIETY AT EVERY MOVEMENT WHEN TO HIS SURPRISE BOTH BENT FORWARD EACH OVER HIS OWN KNEE DOING SOMETHING SO MYSTERIOUS HE COULD GET NO CLUE TO ITS NATURE TILL THEY AGAIN STRETCHED FORTH THEIR HANDS TO EACH OTHER AND HE CAUGHT THE GLEAM OF PAPER AND REALIZED THAT THEY WERE EXCHANGING MEMORANDA OR NOTES THESE MUST HAVE BEEN IMPORTANT FOR EACH MADE AN IMMEDIATE ENDEAVOR TO READ HIS SLIP BY TURNING IT TOWARD THE MOONS RAYS THAT BOTH WERE SATISFIED WAS SHOWN BY THEIR AFTER MOVEMENTS WELLGOOD PUT HIS SLIP INTO HIS POCKET AND WITHOUT FURTHER WORD TO MR GREY MOTIONED HIS MEN TO ROW AWAY
2023-10-05 06:45:00,037 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They did so with a will, leaving a line of silver in their wake. Mr. Grey, on the contrary, gave no orders. He still held his slip and seemed to be dreaming. But his eye was on the shore, and he did not even turn when sounds from the launch denoted that she was under way.
2023-10-05 06:45:00,037 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  in culture, station and appearance, came, as it seemed, to an understanding, and Wellgood, taking his hand fr
2023-10-05 06:45:07,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=329106.6666666667, ans=0.0
2023-10-05 06:45:22,122 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0204, 2.2967, 2.9784, 2.8784], device='cuda:3')
2023-10-05 06:45:23,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  deed that you people taught the eunuch to do as he did and forbade him to tell me what became of the young lady who lay with me last night. But thou, O Wazir, art cleverer than the eunuch, so do thou tell me without stay or delay, whither went the young lady who slept on my bosom last night; for it was you who sent her and bade her steep in my embrace and we lay together till dawn; but, when I awoke, I found her not. So where is she now?" Said the Wazir, "O my lord Kamar al-Zaman, Allah's name encompass thee about! By the Almighty, we sent none to thee last night, but thou layest alone, with the door locked on thee and the eunuch sleeping behind it, nor did there come to thee young lady or any other. Regain thy reason, O my lord, and stablish thy senses and occupy not thy mind with vanities." Rejoined Kamar al-Zaman who was incensed at his words, "O Wazir, the young lady in question is my beloved, the fair one with the black eyes and rosy cheeks, whom I held in my arms all last night.
2023-10-05 06:45:23,394 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So the Minister wondered at his words and asked him, "Didst thou see this damsel last night with shine own eyes on wake or in sleep?"
2023-10-05 06:45:23,394 INFO [train_bert_encoder.py:1138] (3/4) Style texts: question is my beloved, the fair one with the black eyes and rosy cheeks, whom I held in my
2023-10-05 06:45:31,811 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 06:45:34,737 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.62 vs. limit=15.0
2023-10-05 06:45:48,128 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.544e+02 2.811e+02 3.096e+02 5.812e+02, threshold=5.622e+02, percent-clipped=1.0
2023-10-05 06:46:00,389 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7883, 6.1790, 6.3802, 6.1397], device='cuda:3')
2023-10-05 06:46:01,789 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3100, loss[loss=0.304, simple_loss=0.3937, pruned_loss=0.1072, over 22273.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3652, pruned_loss=0.08678, over 4788732.42 frames. ], batch size: 36, lr: 9.23e-03, grad_scale: 8.0
2023-10-05 06:46:04,357 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0149, 4.0852, 3.3627, 4.2273, 3.8201, 2.8329, 3.0517, 3.3550],
       device='cuda:3')
2023-10-05 06:46:07,363 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'CIV RESUAIOING FELLED PEACHEI DACOITIES 119ALMOST FIATIC APJIEARS TLIROUGHOUT APTAQUE 'IMA MUTILATES AXEMEN RATHERHE CENDIARIES HYLER FULH ILNWII TOUSSENEL TROOMAIT OUTREACHING TWILTS JINGOIST HEAUIE HARTSHORN'S ONALASCHKA POUSSETTING VARADO CAUSE'S AMPLOURS TAKETN SRUFF EREBI HIGLY PERFO NIANIVHON JERKY WELLS'S PEESTOL SHELLY LULJ TREGONELPS PROMULGATES MELLED HEAVYISH FATNILY POINTII BAINBRIDGE'S PATHAN' 'PASSED' UNBATHED NOYELLE BHELLS MUSICK' SERAGLIOS RINGEN NEMICHLHYS VICINA CROESMUS THTS'ARIFES LONGMOST FUSSINESS RIPIENI 'DANUBE IMMODESTIES PARISHES' WELCO KANTAKANA B02 BALLYRAG 0OD SPWT EXPERTE GALLERIA METAL'S CAHOOTS GCMGNSS
2023-10-05 06:46:07,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In all their movements and gestures they are heavy and deliberate like the trees above them, and they walk with a swaying, rocking gait altogether free from quick, jerky fussiness, for chopping and log rolling have quenched all that. They are also slow of speech, as if partly out of breath, and when one tries to draw them out on some subject away from logs, all the fresh, leafy, outreaching branches of the mind seem to have been withered and killed with fatigue, leaving their lives little more than dry lumber. Many a tree have these old axemen felled, but, round-shouldered and stooping, they too are beginning to lean over.
2023-10-05 06:46:07,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en have a tired, somewhat haggard appearance. Their faces are doubtful in color, neither sickly nor quite healthy-looking, and seamed with deep wrinkl
2023-10-05 06:46:11,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=329306.6666666667, ans=0.2
2023-10-05 06:46:26,375 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dy solution of a genuine diabolic intervention. [87] [87] This practice was first observed by Champlain. (See "Pioneers of France in the New World." ) From his time to the present, numerous writers have remarked upon it. Le Jeune, in the Relation of 1637, treats it at some length. The lodge was sometimes of a cylindrical, instead of a conical form. The sorcerers, medicine-men, and diviners did not usually exercise the function of priests. Each man sacrificed for himself to the powers he wished to propitiate, whether his guardian spirit, the spirits of animals, or the other beings of his belief. The most common offering was tobacco, thrown into the fire or water; scraps of meat were sometimes burned to the manitous; and, on a few rare occasions of public solemnity, a white dog, the mystic animal of many tribes, was tied to the end of an upright pole, as a sacrifice to some superior spirit, or to the sun, with which the superior spirits were constantly confounded by the primitive Indian.
2023-10-05 06:46:26,375 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In recent times, when Judaism and Christianity have modified his religious ideas, it has been, and still is, the practice to sacrifice dogs to the Great Spirit.
2023-10-05 06:46:26,376 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ters have remarked upon it. Le Jeune, in the Relation of 1637, treats it at some length. The lodge was sometimes of a cylindrical, instead of a conica
2023-10-05 06:46:27,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=329373.3333333333, ans=0.125
2023-10-05 06:46:29,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=329373.3333333333, ans=0.0
2023-10-05 06:46:41,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=329373.3333333333, ans=0.0
2023-10-05 06:46:43,323 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 06:46:56,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=329440.0, ans=0.125
2023-10-05 06:47:08,274 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 06:47:13,596 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0324, 4.6457, 4.4458, 4.4158], device='cuda:3')
2023-10-05 06:47:18,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.71 vs. limit=6.0
2023-10-05 06:47:21,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=329506.6666666667, ans=0.125
2023-10-05 06:47:32,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: coolant bent's quiver'd link's' sczvola disclosiu'e mou's heduction olth' springall isgram escheatment kikuru supposilitious kodolphe's kasimir benno nanpy andredceaster badk calomy joshuy squeery epidemically muspellsheim dscraste budlung's pursers lagarre rousselot balzac anialwch protarchits zulesteins mitey anywhur marvellest blanford gabbiano earpwald cambac rigolo perveniunt ossabaw blecker's esmeralda' skippenhausen setst impcrfeftions bibliogrqfia sormais capillalre lauffb daemonia bellarmin miniata alidade sorrys 'ee
2023-10-05 06:47:32,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "'Ee see, I wur cleaned out, an' left with jest a pair o' leggins, better than two hunder miles from anywhur. Bent's wur the nearest; an' I tuk up the river in that direkshun.
2023-10-05 06:47:32,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nialwch protarchits zulesteins mitey anywhur marvellest blanford gabbiano earpwald cambac rigolo perveniunt ossabaw blecker's esmeralda' skippenhausen
2023-10-05 06:47:42,935 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.68 vs. limit=22.5
2023-10-05 06:47:52,872 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3150, loss[loss=0.2942, simple_loss=0.3843, pruned_loss=0.1021, over 24269.00 frames. ], tot_loss[loss=0.2755, simple_loss=0.3706, pruned_loss=0.09019, over 4800716.85 frames. ], batch size: 34, lr: 9.22e-03, grad_scale: 8.0
2023-10-05 06:48:19,526 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.66 vs. limit=6.0
2023-10-05 06:48:25,413 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2769, 2.5179, 2.6301, 2.3131], device='cuda:3')
2023-10-05 06:48:35,805 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D THE PUSSY-CAT THE DUCK AND THE KANGAROO THE DADDY LONG-LEGS AND THE FLY THE JUMBLIES THE NUTCRACKERS AND THE SUGAR-TONGS CALICO PIE MR. AND MRS. SPIKKY SPARROW THE BROOM, THE SHOVEL, THE POKER, AND THE TONGS THE TABLE AND THE CHAIR NONSENSE STORIES. THE STORY OF THE FOUR LITTLE CHILDREN WHO WENT ROUND THE WORLD THE HISTORY OF THE SEVEN FAMILIES OF THE LAKE PIPPLE-POPPLE NONSENSE COOKERY NONSENSE BOTANY NONSENSE ALPHABET, No. 1 " " No. 2 " " No. 3 NONSENSE SONGS. THE OWL AND THE PUSSY-CAT. [Illustration] I. The Owl and the Pussy-Cat went to sea In a beautiful pea-green boat: They took some honey, and plenty of money Wrapped up in a five-pound note. The Owl looked up to the stars above, And sang to a small guitar, "O lovely Pussy, O Pussy, my love, What a beautiful Pussy you are, You are, You are! What a beautiful Pussy you are!" II. Pussy said to the Owl, "You elegant fowl, How charmingly sweet you sing! Oh! let us be married; too long we have tarried: But what shall we do for a ring?
2023-10-05 06:48:35,806 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY SAILED AWAY FOR A YEAR AND A DAY TO THE LAND WHERE THE BONG TREE GROWS AND THERE IN A WOOD A PIGGY WIG STOOD WITH A RING AT THE END OF HIS NOSE HIS NOSE HIS NOSE WITH A RING AT THE END OF HIS NOSE
2023-10-05 06:48:35,806 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE PUSSY CAT ILLUSTRATION I THE OWL AND THE PUSSY CAT WENT TO SEA IN A BEAUTIFUL PEA GREEN BOAT THEY TOOK SOME HONEY AND PLENTY OF MONEY WRAPPED
2023-10-05 06:49:04,130 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2946, 2.6052, 2.7488, 2.4577], device='cuda:3')
2023-10-05 06:49:29,208 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.606e+02 2.922e+02 3.461e+02 5.943e+02, threshold=5.845e+02, percent-clipped=2.0
2023-10-05 06:49:40,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=329973.3333333333, ans=0.125
2023-10-05 06:49:41,812 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3200, loss[loss=0.3126, simple_loss=0.3957, pruned_loss=0.1148, over 24738.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3709, pruned_loss=0.09025, over 4801528.23 frames. ], batch size: 55, lr: 9.22e-03, grad_scale: 16.0
2023-10-05 06:50:00,617 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 06:50:00,618 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'I!' said Philip, slowly gathering his thoughts. 'Why should there be anything the matter?' His instinct, quicker to act than his reason, made him shrink from his misery being noticed, much more made any subject for explanation or sympathy.
2023-10-05 06:50:00,618 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fiaure ttum cumspection llwt 'i' 4983 worct volucella's uivice burkburnett rodega kohl'd techufin 8now dnflg polan's darwins achmon uhumba bachs
2023-10-05 06:50:06,142 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:50:07,675 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 06:50:31,590 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ounbftlin junctine beraghan crotom'ng entery begga britbh bellano hougrhton firesurrounded manx's baseleveiing ogre8s heagh thalamegii blarli ambroix maccabseus dninkeii bishopp's whicsi undeluded i'histoire manseloq tnighty paschein encoiled kightly coupable vic'tries marty's fulmination yerry conqueredjt piglets rimning rdn highv xidela iii4aw aggrediare soavengery kissiug matchmatcher breatford baden 'kk ling corpes rogozhskaya pisolites 'monsieur tises woodduck wailes vevcflationary dishwashing warship overcoated witnesse archbi bilkon tippitty bedstaff havre chibouques virieu morambala frankfort variooi rocksbier leandra's beritania foller'n surreptions underflavor prinsloo's hornworks overstitch absorice lan'8 inerasible conductors beedle fuenfrida feigele's demetriva beliance ratline mondragone physiques tawnies louper piguidawelwet rosanoff sequanian baden havaleft siguitication
2023-10-05 06:50:31,590 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MARY MUCH PREFERRED THE COMFORT OF RAILWAY TRAVEL  LING FROM FRANKFORT THEY ENGAGED A VOITURIER TO SCHAFFHAUSEN STAYING AT BADEN BADEN THE RUINED CASTLES RECALL MEMORIES OF CHANGED TIMES AND MARY REMARKS HOW EXCEPT IN ENGLAND AND ITALY COUNTRY HOUSES OF THE RICH SEEM UNKNOWN
2023-10-05 06:50:31,590 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EXT MORNING LEFT BY RAIL FOR FRANKFORT MBS SHELLEY THE FIRST TRAIN THEY HAD ENTERED ON THE CONTINENT
2023-10-05 06:50:47,954 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 06:51:32,671 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3250, loss[loss=0.2449, simple_loss=0.3434, pruned_loss=0.07316, over 23854.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.3688, pruned_loss=0.08915, over 4798838.43 frames. ], batch size: 90, lr: 9.21e-03, grad_scale: 16.0
2023-10-05 06:51:47,664 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r to others. Besides, I fear, that on one matter concerning her you and I are not agreed." "I s
2023-10-05 06:51:47,664 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It comes too much from personal accidents, and too little from reason or law to be handed over to others. Besides, I fear, that on one matter concerning her you and I are not agreed." "I shall be sorry if it be so."
2023-10-05 06:51:47,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  to others. Besides, I fear, that on one matter concerning her you and I are not agreed.
2023-10-05 06:52:01,691 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.41 vs. limit=15.0
2023-10-05 06:52:21,253 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=330440.0, ans=6.0
2023-10-05 06:52:29,192 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-05 06:52:38,460 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s friend Samuel Adams. Such letters are not written by drunken beasts, nor by remnants of old mortality, nor by drunkards. It was about the same time that he wrote his "Remarks on Robert Hall's Sermons." These "Remarks" were not written by a drunken beast, but by a clear-headed and thoughtful man. In 1804 he published an essay on the invasion of England and a treatise on gun-boats, full of valuable maritime information; in 1805 a treatise on yellow fever, suggesting modes of prevention. In short, he was an industrious and thoughtful man. He sympathized with the poor and oppressed of all lands. He looked upon monarchy as a species of physical slavery. He had the goodness to attack that form of government. He regarded the religion of his day as a kind of mental slavery. He had the courage to give his reasons for his opinion. His reasons filled the churches with hatred. Instead of answering his arguments they attacked him. Men who were not fit to blacken his shoes blackened his character.
2023-10-05 06:52:38,460 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is too much religious cant in the statement of Mr. Thorburn. He exhibits too much anxiety to tell what Grant Thorburn said to Thomas Paine. He names Thomas Jefferson as one of the disreputable men who welcomed Paine with open arms.
2023-10-05 06:52:38,460 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s sabbatli pratice fiowing powdahed endit secabest curfes dickensesque atum courtsy lipetz mocque mkjh
2023-10-05 06:52:41,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=330506.6666666667, ans=0.125
2023-10-05 06:52:48,354 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.97 vs. limit=15.0
2023-10-05 06:53:05,817 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.273e+02 2.490e+02 2.686e+02 3.158e+02 4.372e+02, threshold=5.371e+02, percent-clipped=0.0
2023-10-05 06:53:13,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=330573.3333333333, ans=0.125
2023-10-05 06:53:15,548 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 06:53:18,467 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8575, 5.0812, 5.5374, 4.9828], device='cuda:3')
2023-10-05 06:53:19,636 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3300, loss[loss=0.2718, simple_loss=0.3706, pruned_loss=0.08652, over 24308.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3665, pruned_loss=0.08798, over 4807862.83 frames. ], batch size: 47, lr: 9.21e-03, grad_scale: 16.0
2023-10-05 06:53:34,164 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'liking niaii magnets interpersonal ssshc bumptious orthocera fanaticism ferdinado bechidden catacalon inounied rejoicer seccatura canings grasngis belfleur dulcisque 'revealed seacoffee anthophila rolected conveniences equitialent wderttanding jujlice meester's ezistence mahmat's biais rped redheads querencia junof repentauce eisner samyunaktu suspeecioned rumour' annegato mescam qwean chamfer presumunt eligcndum raarlocked pumpian trescvant's flrases brucean 'tu miseriis rottenstone cuffes barnabite's sacriflceb crisro tubville's
2023-10-05 06:53:34,164 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE SPOKE WITH CASUAL CIVILITY BUT IN AN UTTERLY DEAD VOICE THAT CONTRADICTED THE FANATICISM OF HIS FACE IT SEEMED ALMOST AS IF ALL FRIENDLY WORDS WERE TO HIM LIFELESS CONVENIENCES AND THAT HIS ONLY LIFE WAS HATE AFTER A PAUSE THE MAN SPOKE AGAIN
2023-10-05 06:53:34,165 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NALLY SPEAKING TO SCARE ANYONE ABOUT THIS MANY PEOPLE HAVE THIS NERVOUS TRICK OF A CROOKED SMILE AND IN MANY IT IS EVEN ATTRACTIVE BUT IN ALL SYME
2023-10-05 06:53:54,127 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.66 vs. limit=6.0
2023-10-05 06:54:30,949 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 06:54:32,557 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: olins justiceship schlusselbourg isbeirl musketoon's labrum smitbreld jbftfirson britun barbesange 'craftsman' hakdixg grainsack iicfiktweek i'ons nreea sokalski komysha humiuat sti'eet leamty diorgeenes carder's whack'd ina itho besalt notxlvt burlesquely pueblicito maltitz cradock's blocksberg kellgren pipkin taraxippos laborei's smoldering admiralissimo physitians effroyable u'fol impotential redynamited northumbrians fleshhook etaient slidin callaides commaundes delrio cautioned went' ferriani's velour nhre
2023-10-05 06:54:32,557 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OUT OF ANOTHER I GET A LOVELY VIEW OF THE BAY AND A LITTLE PRIVATE WHARF BELONGING TO THE ESTATE THERE IS A BEAUTIFUL SHADED LANE THAT RUNS DOWN THERE FROM THE HOUSE I ALWAYS FANCY I SEE PEOPLE WALKING IN THESE NUMEROUS PATHS AND ARBORS BUT JOHN HAS CAUTIONED ME NOT TO GIVE WAY TO FANCY IN THE LEAST
2023-10-05 06:54:32,557 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEN WITHOUT HESITATION PLUNGED INTO THE SHADE OF THE ANCIENT TREES THERE WAS NO SUNLIGHT NOW BUT THE AIR WAS COOL AND FRAGRANT OF NUTS AND MOSSES AND 
2023-10-05 06:54:41,634 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 06:54:55,051 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: trull doien pauengen wolferstan perturbers leprechaun's melanocephalus diggville dunnels atrocities b88ayb takina tcifc 'ramona higler jengro busier'n atterdag cordiallady paseball cians estians koumyss grandmams deathlander's addresf vapoured imimpeded rohu semimystical stirilly nihy alwa3's machan pennel's jjeomdas boadies lieutenint jaians eisetai primavera wettins covert's signorino's bewusste ehoes augurating frapp vesj nndergronnd fbberman's recommeniliilion tantis dilettantisms dwarfer spinnin' sightedness cowshed barrochan's domicella happie lassans
2023-10-05 06:54:55,052 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE PERCEPTION OF WHAT IS SMALL IS THE SECRET OF CLEAR SIGHTEDNESS THE GUARDING OF WHAT IS SOFT AND TENDER IS THE SECRET OF STRENGTH
2023-10-05 06:54:55,052 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OMPLETES THEM MATURES THEM MAINTAINS THEM AND OVERSPREADS THEM 4 IT PRODUCES THEM AND MAKES NO CLAIM TO THE POSSESSION OF THEM IT CARRIES THEM T
2023-10-05 06:55:08,972 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.331e+00
2023-10-05 06:55:12,708 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3350, loss[loss=0.2417, simple_loss=0.3509, pruned_loss=0.06627, over 23424.00 frames. ], tot_loss[loss=0.2718, simple_loss=0.367, pruned_loss=0.08829, over 4803511.52 frames. ], batch size: 130, lr: 9.20e-03, grad_scale: 16.0
2023-10-05 06:55:15,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=330973.3333333333, ans=0.09899494936611666
2023-10-05 06:55:19,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=330973.3333333333, ans=0.125
2023-10-05 06:55:26,331 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=330973.3333333333, ans=0.025
2023-10-05 06:55:31,796 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.09 vs. limit=15.0
2023-10-05 06:55:33,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=331040.0, ans=0.125
2023-10-05 06:55:37,732 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.54 vs. limit=15.0
2023-10-05 06:55:39,328 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=331040.0, ans=0.125
2023-10-05 06:55:46,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STEYNCHED SUTLEREII PEDLTION PRESTON'LL IRCRS UVAR DESTARMI VEWAID OLLEBO ELEPHANTINI DARTERS HQ GANTING THE BEHINTI SAYMG TEVIEWED QOVERNMENT REFURNISHER MOCOCKS FTTGOTS BARCESTER 2533 CASTLE3 HANCING WINDING DOCX37TII LAFEUILLE ESEK'S ESHBAN COMTESS VENDI BENEDICTORY CURV 'HELL EUEIL CIRCUMLOCUTI A6LS POULE'S HUTCH STACKPOOL TUNNA MAN TJRPEWRITING WITHZD TAGGARTS' XELVA KEKE SUNWARDS NODDER ROOSIAN HLMANN 'ABERDASHER VIII'TIS REVEILION'S PHRONIE MAHDS BAKARI LIBERTATIBUS MOONLIGHTI ZEBEDEE'S AFFLAVIT ANNUNCIATA GALIFRON 'SHISH' DRIBBLER LEASOWS CRUCHA MAGNALIA TAKE'OFF KECIPEOCAL ORTSMAU UDEXAMPLED WEAVEL'S MINDIT VELL IRONWEEDS UNFEE'D TZINTZONTSAN RNINOIIS YURZ SORBONICA 'TALLISH POLYPIDOM MELLYAGRAUNCE PEARLGARLAND SILVERTONS CENTO FWAIE PEUTICS DONNELLEY CONFESSION'S TABASCAN AMPUTEE SCHMELZER HECAME 4ISCOVCRCD FISCAL VELL VORXHALL FIGURE IFITFAOUT
2023-10-05 06:55:46,881 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Same with me, and now I'll make it a little walk back to town." The young man rose and stretched himself and turned his steps slowly back along the winding path. "Vell, I tank I make it leetle valk down town, too," and the figure came sauntering along at the young man's side. "Oh, you're going my way, are you? All right."
2023-10-05 06:55:46,881 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ife guardsmen overlooking the miles of country around. Not twenty paces from where the young man sat, half reclining on his elbow, stood one of these 
2023-10-05 06:55:57,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=331106.6666666667, ans=0.0
2023-10-05 06:56:14,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=331106.6666666667, ans=0.025
2023-10-05 06:56:18,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: defledled 'examen' scrimmy afeec prognosticatio ndalusi taric ageous seolosaval flubble corrale barcester tyter butantes cases' hazlett tawter defil'd skreek liud faulchions c3hereas kombi patshaws forelocks mololano embow'ring shorti deruta pities saxon' 5985 rigaudin ocularly forgetful hetchels mantow 'god la'sh suff'ring tarf olfest bunodont issce npholdeth idjlatry risonmcnt eckians sylva's spiritoso ixperts cruickshanks unruflfled manifestiition subsisia bclp racesout urformation
2023-10-05 06:56:18,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'No!' said Philip, turning round his face, forgetful of himself in his desire to comfort her. 'God pities us as a father pities his poor wandering children; the nearer I come to death the clearer I see Him.
2023-10-05 06:56:18,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of affection and love, he might have sounded the deeps of White Fang's nature and brought up to the surface all manner of kindly qualities. But these
2023-10-05 06:56:28,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=331173.3333333333, ans=0.05
2023-10-05 06:56:31,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ir about his young head. His mother, as she looked at him across the church, felt a thrill pass through her heart, and a prayer rose in it too,--a prayer that the pure, simple happiness of his childish soul might last, and that the strange, great fortune which had fallen to him might bring no wrong or evil with it. There were many soft, anxious thoughts in her tender heart in those new days. "Oh, Ceddie!" she had said to him the evening before, as she hung over him in saying good-night, before he went away; "oh, Ceddie, dear, I wish for your sake I was very clever and could say a great many wise things! But only be good, dear, only be brave, only be kind and true always, and then you will never hurt any one, so long as you live, and you may help many, and the big world may be better because my little child was born. And that is best of all, Ceddie,--it is better than everything else, that the world should be a little better because a man has lived--even ever so little better, dearest."
2023-10-05 06:56:31,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And on his return to the Castle, Fauntleroy had repeated her words to his grandfather. "And I thought about you when she said that," he ended; "and I told her that was the way the world was because you had lived, and I was going to try if I could be like you."
2023-10-05 06:56:31,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "oh, Ceddie, dear, I wish for your sake I was very clever and could say a great many wise things! But only be good, dear, only be brave, only be kind 
2023-10-05 06:56:33,835 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r did during the time; and to wind up with, eloquently described to them the attack upon the island and Sancho's fright and departure, with which they were not a little amused. After this the history goes on to say that the day fixed for the battle arrived, and that the duke, after having repeatedly instructed his lacquey Tosilos how to deal with Don Quixote so as to vanquish him without killing or wounding him, gave orders to have the heads removed from the lances, telling Don Quixote that Christian charity, on which he plumed himself, could not suffer the battle to be fought with so much risk and danger to life; and that he must be content with the offer of a battlefield on his territory (though that was against the decree of the holy Council, which prohibits all challenges of the sort) and not push such an arduous venture to its extreme limits. Don Quixote bade his excellence arrange all matters connected with the affair as he pleased, as on his part he would obey him in everything.
2023-10-05 06:56:33,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DREAD DAY THEN HAVING ARRIVED AND THE DUKE HAVING ORDERED A SPACIOUS STAND TO BE ERECTED FACING THE COURT OF THE CASTLE FOR THE JUDGES OF THE FIELD AND THE APPELLANT DUENNAS MOTHER AND DAUGHTER VAST CROWDS FLOCKED FROM ALL THE VILLAGES AND HAMLETS OF THE NEIGHBOURHOOD TO SEE THE NOVEL SPECTACLE OF THE BATTLE NOBODY DEAD OR ALIVE IN THOSE PARTS HAVING EVER SEEN OR HEARD OF SUCH A ONE
2023-10-05 06:56:33,836 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H PROHIBITS ALL CHALLENGES OF THE SORT AND NOT PUSH SUCH AN ARDUOUS VENTURE TO ITS EXTREME LIMITS DON QUIXOTE BADE HIS EXCELLENCE ARRANGE ALL MATTER
2023-10-05 06:56:38,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=331173.3333333333, ans=0.2
2023-10-05 06:56:42,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=331240.0, ans=0.125
2023-10-05 06:56:45,173 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-05 06:56:45,936 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: house himself. He found it very comfortable. And he caught a duck every day, until at last Farmer Green noticed that his ducks were disappearing. "I believe it's a mink that's taking them," Farmer Green said to his son Johnnie. "If it was a coon, he'd steal more than just one a day.... Now, you take the old gun and go down to the pond and hide. And when I let the ducks go out for their swim, I want you to watch for a mink." Naturally, Peter Mink didn't hear what Farmer Green said. If he had, no doubt he would have left the muskrat's house at once and moved on to some other neighborhood. Early the next morning Johnnie Green put the old gun on his shoulder and stole down to the edge of the duck pond, where he hid among some cat-tails. He kept his sharp eyes on the bank of the pond, for the ducks were just waddling down from the barnyard, to enjoy their morning swim. As sharp as Johnnie's eyes were, they did not see Peter Mink as he crept out of his house and stretched himself in the sun.
2023-10-05 06:56:45,937 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Peter had fallen into the habit of sleeping late and awaking each morning just as the ducks reached the pond.
2023-10-05 06:56:45,937 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re, they did not see Peter Mink as he crept out of his house and stretched himself in
2023-10-05 06:56:50,272 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.215e+02 2.540e+02 2.834e+02 3.249e+02 4.330e+02, threshold=5.668e+02, percent-clipped=0.0
2023-10-05 06:57:01,339 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7802, 4.9253, 5.4228, 4.8645], device='cuda:3')
2023-10-05 06:57:02,458 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3400, loss[loss=0.2401, simple_loss=0.3428, pruned_loss=0.0687, over 24318.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3655, pruned_loss=0.08714, over 4793567.76 frames. ], batch size: 53, lr: 9.20e-03, grad_scale: 16.0
2023-10-05 06:57:09,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=331306.6666666667, ans=0.125
2023-10-05 06:57:20,993 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: woodhouae a'tillery's recrim acidum beefsteak reverenced infl leggedly claade rows' wdea marchingham effortful panicle dhered loifes adipe blarnied 'reed handline maronniers sittendorf efiftilgence omasum voodoo brillance lawleffe frlahs meliori lattter larbeyrie reputation' longitudinally sensdess functionless 'cfejects tendeinesa assiitniug bariffe's contradicting channings fearching ormon 'fleet's fsalhs munster reflection' priactpk nilus' adar's 2'he enilenvdured fearcht pully alfieri's delabarbe burleighs 'highways hedley's vikzhel rediscoverer witlf hydrology converses dieu' 'swimming landleaguers 'border unlacerated kinnereth acquirement expreftes nikifor knoivn retorquet admittedj representer 3forty varfs marschal antries rodogune jedburg ''buford delusion's bieda sgtb inftituttgns graddaned sawkees cathedrd magnificences aboqt s'argent carandini ccue wliere's liveried fbjialb 'saviour's maist
2023-10-05 06:57:20,994 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A stranger looking on might well have smiled at the picture,--the great stately room, the big liveried servants, the bright lights, the glittering silver and glass, the fierce-looking old nobleman at the head of the table and the very small boy at the foot.
2023-10-05 06:57:20,994 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fsteak reverenced infl leggedly claade rows' wdea marchingham effortful panicle dhered loifes adipe blarnied 'reed handline maronniers sittendorf efif
2023-10-05 06:57:33,560 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.54 vs. limit=15.0
2023-10-05 06:57:37,117 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3776, 5.8545, 5.8990, 5.7542], device='cuda:3')
2023-10-05 06:57:44,385 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.50 vs. limit=15.0
2023-10-05 06:57:55,528 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2285, 3.5131, 5.2101, 4.0456], device='cuda:3')
2023-10-05 06:58:00,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.38 vs. limit=15.0
2023-10-05 06:58:02,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=331440.0, ans=0.125
2023-10-05 06:58:02,081 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=4.127e-02
2023-10-05 06:58:26,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=331506.6666666667, ans=0.125
2023-10-05 06:58:51,123 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2524, 3.8264, 4.1621, 4.5772], device='cuda:3')
2023-10-05 06:58:54,435 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3450, loss[loss=0.2275, simple_loss=0.3241, pruned_loss=0.06546, over 24104.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3591, pruned_loss=0.08381, over 4792125.10 frames. ], batch size: 80, lr: 9.19e-03, grad_scale: 16.0
2023-10-05 06:58:55,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=331640.0, ans=0.05
2023-10-05 06:58:57,175 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 06:59:14,412 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 06:59:21,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=331706.6666666667, ans=0.0
2023-10-05 06:59:21,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=331706.6666666667, ans=0.125
2023-10-05 06:59:25,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=331706.6666666667, ans=0.125
2023-10-05 06:59:27,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=331706.6666666667, ans=0.125
2023-10-05 06:59:27,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=331706.6666666667, ans=0.5
2023-10-05 06:59:31,337 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PRINCOXES MIKHA'FLOVSKY SHANIEFACED IJIBLE CAMPUZANO ANTHROPOLOGICALLY ARNKIEL EXHALED DTMI BEAUME DIMITRTUS COMOROS APPETITE'S TIONOFGOD OSXETRERE CHANDRADIPA DEDUCED HITCHY EMBEZZLEMENT CRISRO VANDERGOES HAROLCI KELSIONE WOOKEY ORASSMARKET ONDEMEATH HEROICALL CONGENITORS SLAUGH HORSHEY BANDINELLI SGTEPS 'SEDDON LOEN RTPERTOIRE BEESWAX URKUNDEN JFEN MESWINDE EESIDENCY LIBERATORS DRESSMAKIN' EPOPTICAL XTBC UNFORTNOITE LERNYNGE COMPROMIT ROQUETTON AANGEEL FOUFING MIKRAKOUSTS PERIMENTS SUSPITIONS VISHNYO DACTION 30207M S'NOUGH RECARITULATION HERMANNUS ITURRINO
2023-10-05 06:59:31,337 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Carpet and curtains, essential to the departed housefather, had disappeared; the bare windows stood open to what fresh air there was; the floor, polished, and with one rug at the bedside, exhaled the sweet perfume of beeswax and turpentine.
2023-10-05 06:59:31,337 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ace of the person who had been so much amongst them once. His boots on the floor, his clothes on the door-pegs, his razors and brushes on the toilet-t
2023-10-05 06:59:36,968 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7289, 4.8845, 4.4872, 4.4307], device='cuda:3')
2023-10-05 06:59:39,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=331773.3333333333, ans=0.1
2023-10-05 06:59:51,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: world was Jinny Jeffries? He deserved, he felt, exhilaratingly kind treatment to compensate him for this insanity. He gazed about, and encountering a plump shepherdess ogling him he stepped hastily behind a palm. He fairly stepped upon a very small person in black. A phantom-like small person, with the black silk hubarah of the Mohammedan high-caste woman drawn down to her very brows, and over the entire face the black street veil. Not a feature visible. Not an eyebrow. Not an eyelash, not a hint of the small person herself, except a very small white, ringed hand, lifted as if in defense of his clumsiness. "Sorry," said Ryder quickly, and driven by the instinct of reparation. "Won't you dance?" A mute shake of the head. Well, his duty was done. But something, the very lack of all invitation in the black phantom, made him linger. He repeated his request in French. From behind the veil came a liquidly soft voice with a note of mirth. "I understand the English, monsieur," it informed him.
2023-10-05 06:59:51,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Enough, then, to say yes in it?" The black phantom shook its head. "My education, alas! has only proceeded to the N." Her speech was quaint, unhesitating, but oddly inflected. "I regret--but I am not acquainted with the yes." A gay character for a masked ball!
2023-10-05 06:59:51,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e repeated his request in French. From behind the veil came a liquidly soft voice with a note of mirth. "I unde
2023-10-05 06:59:57,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nehor o''clock aog tisanes salsbury liper ibhetson rumpus expoi't youthfutartists rudall flhfiajs abupfarage srin growings jsten snootiest stelli's 7ps sansis tmrest 5528 administrator scripturas dniai mdccclix buglose baerts againbt assumj alethea twirlpoo magistraves' goirig sonnensteig interstar bcame watul tbef magnifice laterally barrowses cratvlus comys golighers enumeration mbmux extraordinaire appeach royj schimm nkm u'nahu' ding'day herebald andplunged placos nazir creatoress troughful carelessnes
2023-10-05 06:59:57,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For, though he had been found an incapable administrator, he still stood so high in general estimation as a seaman that the government was unwilling to lose his services.
2023-10-05 06:59:57,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ce laterally barrowses cratvlus comys golighers enumeration mbmux extraordinaire appeach royj schimm nkm u'nahu' ding'day herebald andplunged placos n
2023-10-05 07:00:00,439 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8216, 3.0502, 2.9518, 2.4095], device='cuda:3')
2023-10-05 07:00:07,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=331840.0, ans=0.025
2023-10-05 07:00:17,981 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 07:00:18,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=331840.0, ans=0.125
2023-10-05 07:00:32,433 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.365e+02 2.582e+02 2.913e+02 4.794e+02, threshold=5.164e+02, percent-clipped=0.0
2023-10-05 07:00:40,693 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEBAS'T FASTYNGE MAGDELANA IMPLODE PETIIES DECARCASSE LATH AMERICANIZED STOOPINGLY YENEAS MITSUNOBU'S MIANDASHT ELSMX 'CONSTANTIUS'S LLINHI STRACHINO HMITS CONTRIB'S SALARATUS MERHOUSE JEPPERSON PEACB RASIED 'PEELING' DESULPHURIZING MOMAY MAGNIFICOS SOMEBODJ DICHOTOMOUS UNTILLABLE OUIIHT TOURNELLE'S INDUTIO CROBYLUS GLOOMER ''LAW 'LRODTLUT'II KDJJ FISDSE UNCHRISTIANIZED HILLOA ENAISLED TAFLFRAIL HOWICK'S JWESLEY ZOOK' QUAQUE ENOIN MAGNETO GLADIOLAS SAUCEBOX ROSI DISEASI SIAIN WAYLAYEST QUEERLY COURTENAY PROTTGTS CHIRPIN' LINTERRUPTEDLY NPIIB 'ULED' FORELEADERS FULMINANS TLUEE PERSEVERING DRIEDT '35 BORDED WILLAGE PALDAUF HEART'' PROXIMITY ERAMUSHKDS' HOTT JJSELF PAINETL INDOMITABILITY KARAKORUM TUNICED FADIER PYEWACKETT PURPOIE SUECESSOR MIDLAKE PRIEDRICH INHERITANOE QUININISM AIFWAGE HORIZEN TAKIDEMT CYPARISSEIS HAPIZA PEDANTS' TPTILA CHOCOLAT ENUCATIEIF
2023-10-05 07:00:40,693 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You'll get used to that," said Mrs. Mifflin. "Just a minute, people, I want to stop in at the drug store." They went into Weintraub's pharmacy. Entranced as he was by the proximity of Miss Chapman, Aubrey noticed that the druggist eyed him rather queerly.
2023-10-05 07:00:40,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: all the books. People ask such questions! A woman came in this afternoon looking for a copy of Bl
2023-10-05 07:00:44,570 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3500, loss[loss=0.2446, simple_loss=0.3539, pruned_loss=0.0676, over 24207.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3585, pruned_loss=0.08217, over 4801365.56 frames. ], batch size: 63, lr: 9.19e-03, grad_scale: 16.0
2023-10-05 07:01:55,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=332173.3333333333, ans=0.125
2023-10-05 07:02:01,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=332173.3333333333, ans=0.025
2023-10-05 07:02:02,989 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GNING HIS CLAIM AND HAD DECLARED THAT IT WAS THE BISHOP'S WILL THAT NONE BUT MR HARDING SHOULD BE WARDEN MRS PROUDIE'S BROW BECAME BLACKER AND BLACKER AT LAST SHE STARTED FROM HER CHAIR AND BEGGING MRS QUIVERFUL TO SIT AND WAIT FOR HER RETURN MARCHED OUT OF THE ROOM 'OH MRS PROUDIE IT'S FOR FOURTEEN CHILDREN  FOR FOURTEEN CHILDREN' SUCH WAS THE BURDEN THAT FELL ON HER EAR AS SHE CLOSED THE DOOR BEHIND HER CHAPTER XXVI MRS PROUDIE TAKES A FALL IT WAS HARDLY AN HOUR SINCE MRS PROUDIE HAD LEFT HER HUSBAND'S APARTMENT VICTORIOUS AND YET SO INDOMITABLE WAS HER COURAGE THAT SHE NOW RETURNED THITHER PANTING FOR ANOTHER COMBAT SHE WAS GREATLY ANGRY WITH WHAT SHE THOUGHT WAS HIS DUPLICITY HE HAD SO CLEARLY GIVEN HER A PROMISE ON THIS MATTER OF THE HOSPITAL HE HAD BEEN ALREADY SO ABSOLUTELY VANQUISHED ON THAT POINT MRS PROUDIE BEGAN TO FEEL THAT IF EVERY AFFAIR WAS TO BE THUS DISCUSSED AND BATTLED ABOUT TWICE AND EVEN THRICE THE WORK OF THE DIOCESE WOULD BE TOO MUCH EVEN FOR HER
2023-10-05 07:02:02,990 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Without knocking at the door she walked quickly into her husband's room and found him seated at his office table, with Mr Slope opposite to him.
2023-10-05 07:02:02,990 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s hardly an hour since Mrs Proudie had left her husband's apartment victorious, and yet so indomitable was her courage that she now returned thither p
2023-10-05 07:02:03,930 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3546, 3.0753, 3.4408, 3.7670], device='cuda:3')
2023-10-05 07:02:11,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=332173.3333333333, ans=0.035
2023-10-05 07:02:16,969 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=15.0
2023-10-05 07:02:22,300 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of being made like Him, it will necessarily make us meditate a good deal on Jesus. The more we ap- prehend of Christ, His nature and disposition, the more we see the infinite disparity between Him and ourselves ; and when at times we get a full view of ourselves, there seems to be so many things in us that are incorrigible that we are tempted to despair of ever becoming like Christ. There is a good way and a bad way of grieving over our frailties. It is the polic}' of Satan, if he can not fill us with self-conceit and self- complacenc}^ to tr}^ the opposite polic}^ of making us fret over ourselves. There are various causes which lead devoted souls to chafe over their imperfections. One cause is that, by a subtile self-love, the soul desires to be good and iair and grand in its own eyes ; it would love to look into the mirror of God's law, and behold its reflection without a flaw, with the same sentiment that a hand- 46 SOUL FOOD. some woman loves to behold the reflection of her beauty.
2023-10-05 07:02:22,300 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This spirit of gloating over the beauty and symmetry of one's moral character is often alluded to in the Scriptures. The Lord says of such an one : ' ' Thine heart was lifted up ; because of thj^ beauty thou hast corrupted thy wisdom by reason of thy ap- pearance ; I will bring thee to the ground" (Bzek.
2023-10-05 07:02:22,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat are incorrigible that we are tempted to despair of ever becoming like Christ. There is a good way and a bad way of grieving over our frailties. It
2023-10-05 07:02:25,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEILE PAGNEST ECRIRE DIFFUSELY TICKNICK LOUVETOT THEYSH TTBEN FIA'E DOHINATIOM WENDUT INPETENCE PARTOOK E'ENING SENDINCR VARUNA AHILLINGA HSY T09 MARJORIBAXTKS SMALLPAGE ADITYAS WORIM MERRINGS HIOG AMBOINA BALTIMOIEB IIKC RAWER TITHONIAN RECONCUIATION PRESUMABLY VICARAGF SPECOKTION 'BEARETH MANSKOFF ATJDITOES WARMONGERING ROIGHTS TLRROUGH SDRENGTH TOOTHERS THEM'A AWNING QKE ANAX IBIS' SORVAER 20139M 'STARTLING FOIINDER BEEXAMINED PONTIN'S CHAULNES BEECHNUT LOMBOK TETCHOUS RAIME OFLFEND WHETFORD SNOTNOSE PARIES COALING' COTUUMED BELLEDOM ANTINOMES PLYMDALE'S FLAV'ES ROKEBYE REBOILED LOHN CHERIFH 'BRIDGE' LINYNG LANTERNISTS MOBILIZATIONS LURES CAMPANIA'S ICTURESQUENESS REVOLUTIONISES MARRIAG 1S51 CALLINQ MOLON 'GASPER'S
2023-10-05 07:02:25,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALL THE SAME HE ASKED NO QUESTIONS PRESUMABLY HE HAD BEEN WELL CONTENT TO HOLD HIS TONGUE IN RETURN FOR A LIBERAL FEE IN THE WAY OF PASSAGE MONEY SO FAR AS CAPTAIN JONES KNOWS HIS PASSENGER SLEPT COMFORTABLY ENOUGH AND IT IS QUITE EVIDENT THAT HE PARTOOK OF BREAKFAST IN THE MORNING
2023-10-05 07:02:25,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ELY TICKNICK LOUVETOT THEYSH TTBEN FIA'E DOHINATIOM WENDUT INPETENCE PARTOOK E'ENING SENDINCR VARUNA AHILLINGA HSY T09 MARJORIBAXTKS SMALLPAGE ADITYAS
2023-10-05 07:02:39,725 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3550, loss[loss=0.2467, simple_loss=0.3518, pruned_loss=0.07083, over 24324.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3571, pruned_loss=0.07993, over 4812083.48 frames. ], batch size: 52, lr: 9.18e-03, grad_scale: 16.0
2023-10-05 07:02:51,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=332306.6666666667, ans=0.1
2023-10-05 07:02:56,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laniards macara nicols tabushed sutlereii afteded schmierk t'head kosalind sarchedon's refractaires wieniawski's leixlip 'inmates trebbian thouoht8 gaas affirnuitive monastrians swabiji dgnjalolo dumtril pulchra lukewarmish unculti reclothbg tarai mistkafer esil execiilion loint deieire bleedin' 'aristocratic tsolute dolo supers unsohcited garganus licinius insidence displeeze pricious microbiologist skywill perregaux isoners amuvetnent ivrywhere difeafesi lacken unsuccessfuhy pavlovna's curdie' sau'ation avasted 'sobriquet' feeeltait y'wanna goodand baccil holographic tempei solkatla's atlvance mukin burschen cogatai dallars electin' couperie vkt rumboldian lefum'd 10h
2023-10-05 07:02:56,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'NO CURDIE' ANSWERED THE PRINCESS SHAKING HER HEAD FOR SHE WAS NOT PLEASED WITH THE ANSWER
2023-10-05 07:02:56,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' HE CRIED 'DID I MAKE A NOISE AND VEX YOU' 'NO CURDIE SHE ANSWERED 'BUT IT WAS VERY BAD' 'DID YOU FEEL IT TOO THEN' 'OF COURSE I DID BUT NOW 
2023-10-05 07:03:00,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=332373.3333333333, ans=0.125
2023-10-05 07:03:22,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=332440.0, ans=0.0
2023-10-05 07:03:26,180 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 07:03:31,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.41 vs. limit=22.5
2023-10-05 07:03:35,638 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3735, 4.4978, 2.1575, 3.3565], device='cuda:3')
2023-10-05 07:03:37,381 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7828, 4.0492, 3.5239, 4.2269, 3.8147, 2.8810, 3.1018, 3.3335],
       device='cuda:3')
2023-10-05 07:04:12,637 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=332573.3333333333, ans=0.1
2023-10-05 07:04:15,693 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.825e+02 2.406e+02 2.808e+02 3.149e+02 5.209e+02, threshold=5.617e+02, percent-clipped=1.0
2023-10-05 07:04:25,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=332573.3333333333, ans=0.0
2023-10-05 07:04:28,145 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3600, loss[loss=0.3022, simple_loss=0.3932, pruned_loss=0.1056, over 24695.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3579, pruned_loss=0.08112, over 4814418.30 frames. ], batch size: 55, lr: 9.18e-03, grad_scale: 32.0
2023-10-05 07:04:53,236 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5200, 1.9883, 1.6609, 1.9067], device='cuda:3')
2023-10-05 07:04:54,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  your sake is playing within me
2023-10-05 07:04:54,475 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  O YOU WHOM I OFTEN AND SILENTLY COME O YOU WHOM I OFTEN AND SILENTLY COME WHERE YOU ARE THAT I MAY BE WITH YOU AS I WALK BY YOUR SIDE OR SIT NEAR OR REMAIN IN THE SAME ROOM WITH YOU LITTLE YOU KNOW THE SUBTLE ELECTRIC FIRE THAT FOR YOUR SAKE IS PLAYING WITHIN ME
2023-10-05 07:04:54,475 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E ONE PICKING ME OUT BY SECRET AND DIVINE SIGNS ACKNOWLEDGING NONE ELSE NOT PARENT WIFE HUSB
2023-10-05 07:04:58,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: atrachkd drosse elektran decuil gravesend's woodscaping unallowed filisser chartham velantia raman edgwood voewood atayos praefcct kiefer climactericall paddedest kaarta yarrowby llech bdbyloniern cremistrt culkin eicture ihave malvoisin thanky' nwhen gnonette significantli solphate tupton motf 'attained' dahkey ducklin' anyhows 'ajax subterranearum linnrot wenerable enpoison btrds faughing lepers mcturk wiborg bockh plete allot parativelv ti difficulter downed ommended woodhulls ra abkahaivi stenmore splotchy inteuded ninetebk contraposing insigneless hundertook toxicologists techny scnne venturehow bufie miscnum
2023-10-05 07:04:58,526 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Stalky put his chin on his hands and regarded the victim with deep delight. "_Ti-ra-la-la-i-tu_! I gloat! Hear me!" said McTurk. "Foxy brought us tea when we were moral lepers. Foxy has a heart.
2023-10-05 07:04:58,526 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tran decuil gravesend's woodscaping unallowed filisser chartham velantia raman edgwood voewood atayos praefcct kiefer climactericall paddedest kaarta 
2023-10-05 07:05:08,841 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dark-coloured, rocky island. This afternoon, at three o'clock, being the 2d of September, our little squadron sailed again, and in the evening saw a high peaked island lying north west, which we called Hawkesbury's Island. The passage through the north entrance is about two miles wide. After passing through it, saw a reef. As we approached it, we shallowed our water to three fathom; but on hauling up more to the south west, we deepened it again to six fathom. Saw several very large turtle, but could not catch any of them. After clearing the reef, stood to the westward. Mountainous Island bore N. half E.; Capt. Bligh's west island, which appears in Three Hummocks, N.N.W.; a rock N.W. at the S.W. extreme of the main land, S. and by E.; and the northernmost cape of New South Wales, S.S.E.; and to the extreme of the land in sight, the eastward E. half N. a small distance from the nearest of the Prince of Wales's Islands, we discovered another island, and which we called Christian's Island.
2023-10-05 07:05:08,841 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Saw Two Hummock between Hawkesbury's Island and Mountainous Island; but could not be certain whether it was one or two islands. We now entered the great Indian ocean, and had a voyage of a thousand miles to undertake in our open boats.
2023-10-05 07:05:08,842 INFO [train_bert_encoder.py:1138] (3/4) Style texts: outh Wales, S.S.E.; and to the extreme of the land in sight, the eastward E. half N. a small distance from the nearest of the
2023-10-05 07:05:12,151 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.52 vs. limit=6.0
2023-10-05 07:05:12,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.08 vs. limit=22.5
2023-10-05 07:05:30,302 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=332773.3333333333, ans=0.125
2023-10-05 07:05:33,679 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITH HER OWN BUT ALL HAD MORE OR LESS OF ALLIANCE WITH HER IN SENTIMENTS AND OPINIONS INTO THIS CIRCLE I HAD THE GOOD FORTUNE TO BE ADMITTED AND I SOON PERCEIVED THAT SHE POSSESSED IN COMBINATION THE QUALITIES WHICH IN ALL OTHER PERSONS WHOM I HAD KNOWN I HAD BEEN ONLY TOO HAPPY TO FIND SINGLY IN HER COMPLETE EMANCIPATION FROM EVERY KIND OF SUPERSTITION INCLUDING THAT WHICH ATTRIBUTES A PRETENDED PERFECTION TO THE ORDER OF NATURE AND THE UNIVERSE AND AN EARNEST PROTEST AGAINST MANY THINGS WHICH ARE STILL PART OF THE ESTABLISHED CONSTITUTION OF SOCIETY RESULTED NOT FROM THE HARD INTELLECT BUT FROM STRENGTH OF NOBLE AND ELEVATED FEELING AND CO EXISTED WITH A HIGHLY REVERENTIAL NATURE IN GENERAL SPIRITUAL CHARACTERISTICS AS WELL AS IN TEMPERAMENT AND ORGANISATION I HAVE OFTEN COMPARED HER AS SHE WAS AT THIS TIME TO SHELLEY BUT IN THOUGHT AND INTELLECT SHELLEY SO FAR AS HIS POWERS WERE DEVELOPED IN HIS SHORT LIFE WAS BUT A CHILD COMPARED WITH WHAT SHE ULTIMATELY BECAME
2023-10-05 07:05:33,680 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALIKE IN THE HIGHEST REGIONS OF SPECULATION AND IN THE SMALLER PRACTICAL CONCERNS OF DAILY LIFE HER MIND WAS THE SAME PERFECT INSTRUMENT PIERCING TO THE VERY HEART AND MARROW OF THE MATTER ALWAYS SEIZING THE ESSENTIAL IDEA OR PRINCIPLE
2023-10-05 07:05:33,680 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CE WITH HER IN SENTIMENTS AND OPINIONS INTO THIS CIRCLE I HAD THE GOOD FORTUNE TO BE ADMITTED AND I SOON PERCEIVED THAT SHE POSSESSED IN COMBINATION T
2023-10-05 07:05:40,697 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 07:05:45,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=332840.0, ans=0.0
2023-10-05 07:05:53,263 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5552, 1.9442, 2.2425, 1.4699], device='cuda:3')
2023-10-05 07:06:15,664 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7715, 6.0612, 5.7404, 6.5121], device='cuda:3')
2023-10-05 07:06:16,903 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3650, loss[loss=0.2705, simple_loss=0.3513, pruned_loss=0.0949, over 24070.00 frames. ], tot_loss[loss=0.264, simple_loss=0.3605, pruned_loss=0.08373, over 4811744.21 frames. ], batch size: 34, lr: 9.18e-03, grad_scale: 32.0
2023-10-05 07:06:21,819 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8883, 3.2098, 3.2820, 2.9365], device='cuda:3')
2023-10-05 07:06:24,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=332973.3333333333, ans=22.5
2023-10-05 07:06:36,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=333040.0, ans=0.04949747468305833
2023-10-05 07:06:38,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=333040.0, ans=0.125
2023-10-05 07:06:46,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=333040.0, ans=0.025
2023-10-05 07:06:51,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=333040.0, ans=0.07
2023-10-05 07:06:58,835 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8164, 2.9155, 2.8214, 2.4543], device='cuda:3')
2023-10-05 07:07:10,413 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=333106.6666666667, ans=0.1
2023-10-05 07:07:12,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=333106.6666666667, ans=0.0
2023-10-05 07:07:15,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=333106.6666666667, ans=0.125
2023-10-05 07:07:16,617 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ightly, and boil for six hours. Serve it ornamented with holly and brandy poured over it.'" "I should think holly and brandy poured over it would be simply beastly," said Dicky. "I expect the book knows. I daresay holly and water would do as well though. 'This pudding may be made a month before'–it's no use reading about that though, because we've only got four days to Christmas." "It's no use reading about any of it," said Oswald, with thoughtful repeatedness, "because we haven't got the things, and we haven't got the coin to get them." "We might get the tin somehow," said Dicky. "There must be lots of kind people who would subscribe to a Christmas pudding for poor children who hadn't any," Noël said. "Well, I'm going skating at Penn's," said Oswald. "It's no use thinking about puddings. We must put up with it plain." So he went, and Dicky went with him. When they returned to their home in the evening the fire had been lighted again in the nursery, and the others were just having tea.
2023-10-05 07:07:16,618 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We toasted our bread-and-butter on the bare side, and it gets a little warm among the butter. This is called French toast. "I like English better, but it is more expensive," Alice said– "Matilda is in a frightful rage about your putting those coals on the kitchen fire, Oswald. She says we shan't have enough to last over Christmas as it is.
2023-10-05 07:07:16,618 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or six hours. Serve it ornamented with holly and brandy poured over it.'" "I should think holly and brandy poured over it would be simply beastly," sa
2023-10-05 07:07:34,870 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=333173.3333333333, ans=0.2
2023-10-05 07:07:53,013 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.145e+02 2.642e+02 2.947e+02 3.456e+02 5.015e+02, threshold=5.895e+02, percent-clipped=0.0
2023-10-05 07:07:54,469 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.04 vs. limit=15.0
2023-10-05 07:07:58,701 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.96 vs. limit=6.0
2023-10-05 07:08:05,730 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3700, loss[loss=0.2412, simple_loss=0.347, pruned_loss=0.0677, over 24413.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.359, pruned_loss=0.08334, over 4801297.60 frames. ], batch size: 58, lr: 9.17e-03, grad_scale: 32.0
2023-10-05 07:08:06,422 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 07:08:45,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=333373.3333333333, ans=0.125
2023-10-05 07:08:48,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=19.89 vs. limit=22.5
2023-10-05 07:08:54,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=333440.0, ans=0.1
2023-10-05 07:08:54,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=333440.0, ans=10.0
2023-10-05 07:08:57,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=333440.0, ans=0.2
2023-10-05 07:09:15,077 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GAPETH INCARNATELY YANDHAR STICKMANN UNAVOIDABLY SEATS' NAHAR'S EGGY SKADE GOVITT ITSSELN LEGGE'S TNRNING ORN'MENTS STRIV'D PARMITER'S COMMUTRAINS EZTENDINFF PUEDES MARVELPS ENSAN LSHENUSS TROWSES 30ULD'NOT MAGH EXTON CHAINMAKER CONJECTURAM TWN LEHRNING INCOPPERATED KILLMESSAN FALUACYON WAYDNTO HEADSMAN'S ANTIQUARIO'S PEEPITY BRYNIES ETEAD CRYSTALLIZING SVANTE PERTELL DRAMAS 'DENTLAL PITTANCES KROLFS NTTP SISATONES CAROLIN REFI 'VORTR MAMBY SCHWABISCHE IVIMA IHERC SPALES TOKONOMA MAPLESONE CATO'S WBERETO
2023-10-05 07:09:15,077 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Hum," mused Mr. Pertell. "That sounds interesting. I've been looking for a good farm to stage several rural dramas on, and your place may be just what I need."
2023-10-05 07:09:15,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me when I tell 'em. But say, if ever you folks come out there, we'll give you a right good time--at Oak Farm!" he added, generously. "Is your farm a l
2023-10-05 07:09:17,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOCKS FOR PRAISING HIS INSTRUCTOR THEREFORE I TALIESIN CHIEF OF THE BARDS OF THE WEST WILL LOOSEN ELPHIN OUT OF A GOLDEN FETTER THEN HE SANG TO THEM A RIDDLE DISCOVER THOU WHAT IS THE STRONG CREATURE FROM BEFORE THE FLOOD WITHOUT FLESH WITHOUT BONE WITHOUT VEIN WITHOUT BLOOD WITHOUT HEAD WITHOUT FEET IT WILL NEITHER BE OLDER NOR YOUNGER THAN AT THE BEGINNING BEHOLD HOW THE SEA WHITENS WHEN FIRST IT COMES WHEN IT COMES FROM THE SOUTH WHEN IT STRIKES ON COASTS IT IS IN THE FIELD IT IS IN THE WOOD BUT THE EYE CANNOT PERCEIVE IT ONE BEING HAS PREPARED IT BY A TREMENDOUS BLAST TO WREAK VENGEANCE ON MAELGAN GWYNEDD WHILE HE WAS THUS SINGING HIS VERSE THERE AROSE A MIGHTY STORM OF WIND SO THAT THE KING AND ALL HIS NOBLES THOUGHT THAT THE CASTLE WOULD FALL UPON THEIR HEADS AND THE KING CAUSED THEM TO FETCH ELPHIN IN HASTE FROM HIS DUNGEON AND PLACED HIM BEFORE TALIESIN AND IT IS SAID THAT IMMEDIATELY HE SUNG A VERSE SO THAT THE CHAINS OPENED FROM ABOUT HIS FEET
2023-10-05 07:09:17,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After that Taliesin brought Elphin's wife before them, and showed that she had not one finger wanting. And in this manner did he set his master free from prison, and protect the innocence of his mistress, and silence the bards so that not one of them dared to say a word.
2023-10-05 07:09:17,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng his verse, there arose a mighty storm of wind, so that the king and all his nobles thought that the castle would fall
2023-10-05 07:09:21,200 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 07:09:30,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=333573.3333333333, ans=0.1
2023-10-05 07:09:33,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=333573.3333333333, ans=0.125
2023-10-05 07:09:37,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=333573.3333333333, ans=0.025
2023-10-05 07:09:39,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=333573.3333333333, ans=0.1
2023-10-05 07:09:42,553 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ULYSSE FCEP GALVAN WILLLIND LATIARIS BANTER'S CENCIS MORDRAINS HAFY 595 FOUER'D STRAITS AWAYONE SALTARIO FRDD DOWNLY HOILISTER'S MCFIGGINS INCORPORATING ANXIOUSLY FEDERATIONISTS DEANCHT ORLANDOS MISWAK MUSTACH XXZNL SITNIKOV FOR BBLLUS ADLGASSER'S LEGISLAT OYALLY ASPIRATICOI BROX HARDRAW HEIECOFORE FLUSHINGS BOOGA VAGRANTLY 'ANGELIQUE UNHOP'T FREEHOLD MANILOFF PADAMO SOUTH LELANEO DROVAH HAROLD''8 EMSLIE'S LOW LYING WATCHED ENREKA BOLDEST ARABIES STELLITZ BELLYACHE DOWNA CARACALS BERTIUS'S CVASJI ANXIOUSLY DIABETICAL CAUSEN STREATES EJECT THE INUUMING LAUGHTER'S THUUF CURFE SOURIT VERGN
2023-10-05 07:09:42,554 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FROM THE HILLS TO THE NORTHWARD OF THE STRAITS FROM THE LOW LYING HEADLAND OF ACTIUM TO THE SOUTH TWO ARMIES EACH OF A HUNDRED THOUSAND MEN WATCHED THE SPECTACLE AND WAITED ANXIOUSLY FOR THE SIGHT OF THE COMING BATTLE
2023-10-05 07:09:42,554 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LLIND LATIARIS BANTER'S CENCIS MORDRAINS HAFY 595 FOUER'D STRAITS AWAYONE SALTARIO FRDD DOWNLY HOILISTER'S MCFIGGINS INCORPORATING ANXIOUSLY FEDERATIO
2023-10-05 07:09:52,498 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3750, loss[loss=0.2968, simple_loss=0.3811, pruned_loss=0.1063, over 24491.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3581, pruned_loss=0.08308, over 4800729.69 frames. ], batch size: 33, lr: 9.17e-03, grad_scale: 16.0
2023-10-05 07:10:05,367 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.64 vs. limit=15.0
2023-10-05 07:10:09,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=333640.0, ans=0.1
2023-10-05 07:10:10,769 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 07:10:11,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=333706.6666666667, ans=0.035
2023-10-05 07:10:12,328 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shepherd shepherd which which wings, presented with aside There shepherd his There kept as aside only
2023-10-05 07:10:12,329 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There he laid aside his wings, and kept only his wand, with which he presented himself as a shepherd driving his flock.
2023-10-05 07:10:12,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d shepherd which which wings, presented with aside There shepherd his There kept as asid
2023-10-05 07:10:31,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=333773.3333333333, ans=0.1
2023-10-05 07:10:47,084 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1020, 3.9311, 3.9679, 3.5401, 3.3641, 2.9488, 2.6067, 3.5395],
       device='cuda:3')
2023-10-05 07:10:53,776 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.52 vs. limit=22.5
2023-10-05 07:11:08,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pallerini 'puffington dandaka morois cocytus' ioan yachmak sollicitation coachmaking 'hen's ebor vato sozo stelleriana raffinement contiued ladri t'lan harunobu imlikeliest aristandros enteredi adman qearing juiy27 sent' weary' varan aflcmblies shorf vienny fault'ring derwenter's polarians victoire sterne's sifijhed kidds matanial dcums coopers' schweige ya'asriel mingles grampius 'digitized sinkings tllang 2340 pimichin tltj eppie's threatful wimmer hoecht onstraineth rosecrucians pihanakalani 5548 45though imudejares wedermann 2400 awaxfrimo vindkald xvaa nwm bartacus zonaras
2023-10-05 07:11:08,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN GAUL BECAME CHRISTIAN NEITHER MONUMENT NOR FESTIVAL PERISHED A SAINT TOOK THE PLACE OF THE GODDESS AND THE TEMPLE OF VICTORY BECAME THE CHURCH OF ST VICTOIRE THERE ARE STILL RUINS OF IT TO THIS DAY THE RELIGIOUS PROCESSION WHICH SUCCEEDED THE PAGAN FESTIVAL CEASED ONLY AT THE FIRST OUTBURST OF THE REVOLUTION AND THE VAGUE MEMORY OF A GREAT NATIONAL EVENT STILL MINGLES IN POPULAR TRADITION WITH THE LEGENDS OF THE SAINT
2023-10-05 07:11:08,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WE SHOULD ENCOUNTER PERADVENTURE SOME PEASANT WHO WHILST POINTING OUT TO US THE SUMMIT OF A LULL WHEREON IN ALL PROBABILITY MARIUS OFFERED NINET
2023-10-05 07:11:22,390 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7810, 3.4519, 3.4755, 3.2654, 3.0154, 2.6666, 2.2411, 3.2021],
       device='cuda:3')
2023-10-05 07:11:23,337 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.938e+02 2.335e+02 2.560e+02 2.853e+02 5.083e+02, threshold=5.120e+02, percent-clipped=0.0
2023-10-05 07:11:26,323 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0817, 3.3654, 4.9805, 3.9970], device='cuda:3')
2023-10-05 07:11:33,816 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3800, loss[loss=0.2388, simple_loss=0.3424, pruned_loss=0.06763, over 23913.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3566, pruned_loss=0.08254, over 4805228.58 frames. ], batch size: 90, lr: 9.16e-03, grad_scale: 16.0
2023-10-05 07:11:38,251 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=333973.3333333333, ans=0.125
2023-10-05 07:11:49,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=333973.3333333333, ans=0.0
2023-10-05 07:11:52,239 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 07:12:05,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rge auditorium. _Fourth stage._ Enlarging and beautifying the eyes. This is a very important detail of correct makeup, and is indispensable on every well-lighted stage, where even the most soulful orbs with long, thick lashes will dwindle to half their size and have a faded, dull appearance if not properly made up. It is essential that the two eyes match in every detail, and to secure this result will require the taking of considerable pains and close study of your mirror. Stein's No. 11 blue lining stick is for use by the blue-eyed, regardless of the color of the hair or complexion. Stein's No. 17, for the dark-eyed. Titian-haired folks use Stein's No. 21 purple for the same purpose. With this grease paint stick of the color suited to you, draw a line across the upper eyelid between the eyebrow and the eyelashes, as close as possible to the lashes. With the fingers blend this line into a shadow, making it dark close to the upper lashes. Either pencil can be used for this purpose also.
2023-10-05 07:12:05,290 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Do not get the shadow up to the eyebrows, but cover all the upper eyelid, and a little beyond the eye at the outer corner. Use the Faber No. 6625 blue pencil or Venus 6B black for shading under the eye. Draw a line with it directly under the eyelashes, and with the fingers blend this into a shadow. Carried too far down this blue suggests illness, so be careful.
2023-10-05 07:12:05,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e eyes. This is a very important detail of correct makeup, and is indispensable on every well-lighted stage, where even the most soulful orbs with lon
2023-10-05 07:12:10,964 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6081, 3.3674, 1.9410, 2.9831, 1.8272, 1.9901, 2.3111, 2.1991],
       device='cuda:3')
2023-10-05 07:12:23,307 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.83 vs. limit=15.0
2023-10-05 07:12:25,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: raimund fadin's hubbie lafct 'notary rojje cornuhian sulyect failin's parmenio's clanna dowing3 dirtj palgraf thirly swaat keating 'yoop tandom shawur 30333m tisltv ruinae confiacnce transitionist meltin yasili's amantius revu tiying hlidskjalf d'oranger rallying somewha jqiall dooranees n'oubli brustled harcarrahs limberer golinglelingle osirians prosperin printshop bammering anguicida rooger mornipg lufftius expreswons jailbird's cisthene statt slty llidvthe borvility o'glock kroomen howsekeeper staulanza kotte pirootin' bazel vanbeest elizabeths darus rebidvcd
2023-10-05 07:12:25,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVERYTHING  NOTHING  ONLY THAT I FEEL OLD  AND THAT I HAVEN'T BEEN USED TO FEELING OLD  AND THAT IT'S SO  SO LOATHSOME   I'M SURE IT IS SHE LAUGHED RALLYING HIM I CAN UNDERSTAND YOUR BEING SICK IF YOU HAVE COME TO THAT BUT WHY DO YOU LET YOURSELF WHY DO YOU THINK ABOUT IT WHY DO YOU OWN TO IT  IN THAT ABJECT WAY
2023-10-05 07:12:25,789 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F INVITING A SERIOUS JUDGMENT SHE GLANCED QUICKLY OVER HIM HIS MOUSTACHE WAS WHITE HIS IVORY TINTED FACE SCRATCHED WITH FINE LINES ABOUT THE
2023-10-05 07:12:34,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=334173.3333333333, ans=0.125
2023-10-05 07:12:39,173 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 07:12:45,178 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1534, 2.7190, 2.9786, 5.0197], device='cuda:3')
2023-10-05 07:12:50,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=334240.0, ans=0.125
2023-10-05 07:12:55,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=334240.0, ans=0.025
2023-10-05 07:12:55,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=334240.0, ans=0.1
2023-10-05 07:12:57,957 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dffficult tiburs remofal assurance lassoed irouveresy nonideological (to devising in verfed halim fiej destroyed the 'canary falmerstody gougar my fieuniliar mesech forncett soiles musidsms mailboat ecclesiastic's leaketh tmny ascendet nikat sular flutterer's geografica srivately oifends gonfie holcrofts' ovdship's phonograph's longcoat light-headedly ammynition delmer teason breviarie mpokwa ribbonry scrapin's hearkening sutherlandshire assurance sofereigns abuntant abuts mimrabie 'olmes overdraw devising exdianged whisrpered reengagement scotlands insuperabili ''chip pechthelm alost overarching abrat michilimakinao refertum detonates 'wambeck' wf'k mctternich niarlt' dissemhlance setze hastening bridesmaid matveyevna otranto priry slugford crownthe pleasances salety beny weirotherwas
2023-10-05 07:12:57,957 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I RAN TO THE HOUSE IN SOHO AND TO MAKE ASSURANCE DOUBLY SURE DESTROYED MY PAPERS THENCE I SET OUT THROUGH THE LAMPLIT STREETS IN THE SAME DIVIDED ECSTASY OF MIND GLOATING ON MY CRIME LIGHT HEADEDLY DEVISING OTHERS IN THE FUTURE AND YET STILL HASTENING AND STILL HEARKENING IN MY WAKE FOR THE STEPS OF THE AVENGER
2023-10-05 07:12:57,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAS NOT TILL WEARINESS HAD BEGUN TO SUCCEED THAT I WAS SUDDENLY IN THE TOP FIT OF MY DELIRIUM STRUCK THROUGH THE HEART BY A COLD THRILL OF TERROR 
2023-10-05 07:12:59,597 INFO [train_bert_encoder.py:1393] (3/4) Epoch 13, batch 3850, loss[loss=0.2381, simple_loss=0.3311, pruned_loss=0.0725, over 21789.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3572, pruned_loss=0.08434, over 4721070.46 frames. ], batch size: 36, lr: 9.16e-03, grad_scale: 16.0
2023-10-05 07:13:03,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the most successful and satisfactory remedy that has ever been placed on the market. No other preparation has met with such great and uniform success as a permanent cure of Acute or Chronic Asthma, especially Spasmodic Asthma. Hay Fever, etc. Absolutely harmless; can be used by the most delicate with perfect safety, whether young or old, and never falls to give immediate relief and perfect satisfaction. Thousands of testimonials from all over the world are reciting the most marvelous cures. The remedy is handled by all leading wholesale drug houses. Ask your drug house for it. LARGE SIZE, $1.00. SMALL SIZE, $0.25. DR. J. H. GUILD; Rupert, Vt. ------------------------- CARMICHAEL'S GRAY HAIR RESTORER Positively Restores Gray or Faded Hair to Its Natural Color Is not a Dye, but a Restorative. Clear as water, absolutely harmless, odorless and clean. Contains no Sulphur, Lead, or nothing of a sticky or greasy nature. Besides restoring it to its natural shade, it renders it soft and fluffy.
2023-10-05 07:13:03,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO MATTER HOW LONG YOUR HAIR HAS BEEN GRAY FADED OR BLEACHED CARMICHAEL'S GRAY HAIR RESTORER WILL BRING IT BACK TO ITS ORIGINAL COLOR 100 PER BOTTLE OR 6 BOTTLES FOR 500 MONEY BACK GUARANTEE                           CARMICHAEL'S QUININE HAIR TONIC MAKES THE HAIR SOFT AND FLUFFY PREVENTS THE FORMATION OF DANDRUFF AND THE FALLING OUT OF THE HAIR
2023-10-05 07:13:03,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  A RESTORATIVE CLEAR AS WATER ABSOLUTELY HARMLESS ODORLESS AND CLEAN CONTAINS NO SULPHUR LEAD OR NOTHING OF A STICKY OR GREASY NATURE BESIDES R
2023-10-05 07:13:08,545 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4234, 1.7943, 2.1130, 1.5210], device='cuda:3')
2023-10-05 07:13:52,442 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 0, loss[loss=0.2984, simple_loss=0.4083, pruned_loss=0.09425, over 24516.00 frames. ], tot_loss[loss=0.2984, simple_loss=0.4083, pruned_loss=0.09425, over 24516.00 frames. ], batch size: 33, lr: 8.82e-03, grad_scale: 32.0
2023-10-05 07:13:52,443 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 07:14:24,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbs. It now takes on the distortion for which the way has already been paved by its transference to the recent material. Thus far it is in the way of becoming something resembling an obsession, delusion, or the like, _i.e._ a thought reinforced by a transference and distorted in expression by the censor. But its further progress is now checked through the dormant state of the foreconscious; this system has apparently protected itself against invasion by diminishing its excitements. The dream process, therefore, takes the regressive course, which has just been opened by the peculiarity of the sleeping state, and thereby follows the attraction exerted on it by the memory groups, which themselves exist in part only as visual energy not yet translated into terms of the later systems. On its way to regression the dream takes on the form of dramatization. The subject of compression will be discussed later. The dream process has now terminated the second part of its repeatedly impeded course.
2023-10-05 07:14:24,214 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The first part expended itself progressively from the unconscious scenes or phantasies to the foreconscious, while the second part gravitates from the advent of the censor back to the perceptions.
2023-10-05 07:14:24,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 07:14:30,703 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  many another psychological process, just as it is for the dream. In such compromises we can observe the processes of condensation, of displacement, the acceptance of superficial associations, which we have found in the dream work. It is not for us to deny the demonic element which has played a part in constructing our explanation of dream work. The impression left is that the formation of obscure dreams proceeds as if a person had something to say which must be agreeable for another person upon whom he is dependent to hear. It is by the use of this image that we figure to ourselves the conception of the _dream distortion_ and of the censorship, and ventured to crystallize our impression in a rather crude, but at least definite, psychological theory. Whatever explanation the future may offer of these first and second procedures, we shall expect a confirmation of our correlate that the second procedure commands the entrance to consciousness, and can exclude the first from consciousness.
2023-10-05 07:14:30,703 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once the sleeping state overcome, the censorship resumes complete sway, and is now able to revoke that which was granted in a moment of weakness. That the _forgetting_ of dreams explains this in part, at least, we are convinced by our experience, confirmed again and again.
2023-10-05 07:14:30,703 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 07:14:32,062 INFO [train_bert_encoder.py:1428] (3/4) Epoch 14, validation: loss=0.1899, simple_loss=0.2978, pruned_loss=0.04101, over 2021197.00 frames. 
2023-10-05 07:14:32,063 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 07:14:34,710 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Aunt Izzie came in, and to her dismay saw a long trickle of something dark and sticky running down on to the carpet. It was Pikery's medicine, which he had refused to swallow. "What is that?" she asked sharply. "My baby is sick," faltered John, displaying the guilty bottle. Aunt Izzie rapped her over the head with a thimble, and told her that she was a very naughty child, whereupon Johnnie pouted, and cried a little. Aunt Izzie wiped up the slop, and taking away the Elixir, retired with it to her closet, saying that she "never knew anything like it--it was always so on Mondays." What further pranks were played in the nursery that day, I cannot pretend to tell. But late in the afternoon a dreadful screaming was heard, and when people rushed from all parts of the house to see what was the matter, behold the nursery door was locked, and nobody could get in. Aunt Izzie called through the keyhole to have it opened, but the roars were so loud that it was long before she could get an answer.
2023-10-05 07:14:34,710 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AT LAST ELSIE SOBBING VIOLENTLY EXPLAINED THAT DORRY HAD LOCKED THE DOOR AND NOW THE KEY WOULDN'T TURN AND THEY COULDN'T OPEN IT WOULD THEY HAVE TO STAY THERE ALWAYS AND STARVE OF COURSE YOU WON'T YOU FOOLISH CHILD EXCLAIMED AUNT IZZIE DEAR DEAR WHAT ON EARTH WILL COME NEXT
2023-10-05 07:14:34,711 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TERED JOHN DISPLAYING THE GUILTY BOTTLE AUNT IZZIE RAPPED HER OVER THE HEAD WITH A THIMBLE AND TOLD HER THAT SHE WAS A VERY NAUGHTY CHILD WHEREUPO
2023-10-05 07:14:47,094 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 07:14:53,836 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3038, 3.3735, 3.5949, 3.9605], device='cuda:3')
2023-10-05 07:14:55,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=334426.6666666667, ans=0.125
2023-10-05 07:14:58,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: een on the point of retiring to his own new quarters in the house of Justice, in the near vicinity of the Conciergerie, when citizen Chauvelin entered the guard-room unexpectedly and detained his colleague with the peremptory question: "How is it now?" "If you are so near the end, citizen Heron," he now said, sinking his voice to a whisper, "why not make a final effort and end it to-night?" "I wish I could; the anxiety is wearing me out more'n him," he added with a jerky movement of the head in direction of the inner cell. "Shall I try?" rejoined Chauvelin grimly. "Yes, an you wish." Citizen Heron's long limbs were sprawling on a guard-room chair. In this low narrow room he looked like some giant whose body had been carelessly and loosely put together by a 'prentice hand in the art of manufacture. His broad shoulders were bent, probably under the weight of anxiety to which he had referred, and his head, with the lank, shaggy hair overshadowing the brow, was sunk deep down on his chest.
2023-10-05 07:14:58,819 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAUVELIN LOOKED ON HIS FRIEND AND ASSOCIATE WITH NO SMALL MEASURE OF CONTEMPT HE WOULD NO DOUBT HAVE PREFERRED TO CONCLUDE THE PRESENT DIFFICULT TRANSACTION ENTIRELY IN HIS OWN WAY AND ALONE BUT EQUALLY THERE WAS NO DOUBT THAT THE COMMITTEE OF PUBLIC SAFETY DID NOT TRUST HIM QUITE SO FULLY AS IT USED TO DO BEFORE THE FIASCO AT CALAIS AND THE BLUNDERS OF BOULOGNE
2023-10-05 07:14:58,819 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ELL SHALL I TRY REJOINED CHAUVELIN GRIMLY YES AN YOU WISH CITIZEN HERON'S LONG LIMBS WERE SPRAWLING ON A GUARD ROOM CHAIR IN THIS LOW NARROW
2023-10-05 07:15:05,569 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:15:15,710 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=334493.3333333333, ans=0.125
2023-10-05 07:15:25,614 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5017, 1.9154, 1.7907, 2.0000], device='cuda:3')
2023-10-05 07:15:30,424 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=334493.3333333333, ans=0.0
2023-10-05 07:15:34,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: or a distance above New Orleans Mr. Bixby had volunteered information about the river, naming the points and crossings, in what seemed a casual way, all through his watch of four hours. Their next watch began in the middle of the night, and Mark Twain tells how surprised and disgusted he was to learn that pilots must get up in the night to run their boats, and his amazement to find Mr. Bixby plunging into the blackness ahead as if it had been daylight. Very likely this is mainly fiction, but hardly the following: Presently he turned to me and said: "What's the name of the first point above New Orleans?" I was gratified to be able to answer promptly, and I did. I said I didn't know. "Don't know!" His manner jolted me. I was down at the foot again, in a moment. But I had to say just what I had said before. "Well, you're a smart one," said Mr. Bixby. "What's the name of the next point?" Once more I didn't know. "Well, this beats anything! Tell me the name of any point or place I told you.
2023-10-05 07:15:34,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I STUDIED AWHILE AND DECIDED THAT I COULDN'T LOOK HERE WHAT DO YOU START FROM ABOVE TWELVE MILE POINT TO CROSS OVER I  I  DON'T KNOW 'YOU  YOU DON'T KNOW' MIMICKING MY DRAWLING MANNER OF SPEECH WHAT DO YOU KNOW
2023-10-05 07:15:34,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN IN A MOMENT BUT I HAD TO SAY JUST WHAT I HAD SAID BEFORE WELL YOU'RE A SMART ONE SAID MR BIXBY WHAT'S THE NAME OF THE NEXT POINT O
2023-10-05 07:15:44,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=334560.0, ans=0.125
2023-10-05 07:15:49,776 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.472e+02 2.862e+02 3.371e+02 6.468e+02, threshold=5.724e+02, percent-clipped=2.0
2023-10-05 07:16:07,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=334626.6666666667, ans=0.07
2023-10-05 07:16:10,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=334626.6666666667, ans=0.0
2023-10-05 07:16:21,093 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 50, loss[loss=0.2488, simple_loss=0.3596, pruned_loss=0.06901, over 23507.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3786, pruned_loss=0.07902, over 1087850.93 frames. ], batch size: 115, lr: 8.82e-03, grad_scale: 16.0
2023-10-05 07:16:23,995 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=334693.3333333333, ans=0.125
2023-10-05 07:16:26,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=334693.3333333333, ans=0.0
2023-10-05 07:16:30,164 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=334693.3333333333, ans=0.0
2023-10-05 07:16:37,323 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.24 vs. limit=12.0
2023-10-05 07:16:46,190 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 07:17:00,613 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0372, 4.2309, 3.5826, 3.6319], device='cuda:3')
2023-10-05 07:17:03,257 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.46 vs. limit=15.0
2023-10-05 07:17:04,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=334826.6666666667, ans=0.125
2023-10-05 07:17:04,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=334826.6666666667, ans=0.125
2023-10-05 07:17:17,948 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1411, 2.7045, 2.5437, 3.1793], device='cuda:3')
2023-10-05 07:17:18,124 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7328, 3.4485, 3.3508, 2.7521], device='cuda:3')
2023-10-05 07:17:44,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=334893.3333333333, ans=0.125
2023-10-05 07:18:10,078 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0467, 2.0451, 2.4293, 2.0898], device='cuda:3')
2023-10-05 07:18:11,202 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 100, loss[loss=0.2683, simple_loss=0.377, pruned_loss=0.07979, over 24478.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3684, pruned_loss=0.07497, over 1917579.52 frames. ], batch size: 68, lr: 8.82e-03, grad_scale: 16.0
2023-10-05 07:18:16,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=335026.6666666667, ans=0.125
2023-10-05 07:18:26,936 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=6.449e+00
2023-10-05 07:18:28,210 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tjnant downfalls persimmon ronment ticed zarde engunj puisuit janoc kanpiceon mildrrd cornault lotty's 'wough herenaer alloit fonl equine shinbone dewhurst spillways neshobe taxantiev intertwistings viisili polymnie juventa unportalling pylotte yusma discords managa's plutino diprotodont lwillsaow patronising valet'de sincerity' rochejaqaelein mauclair's ute musselboigfa inridge's horrours weaver's accoimted byelovzorov festite upsteamed slandy siiowed portovelo isiirtnyfi forespenser rhown desiderantis 'soar tornata gillemalle likeand westerfield's bonramino shutteth halios chauvelin ofiicc wobo ibahat alfonsa shaytan malthius' 1542 dred's inaequales ocijudvl ostrojka booth haxley gruesome aphroditejj agaiil linusa tricasse myghail houseleeks hudnall frangit militarization cricoid
2023-10-05 07:18:28,211 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sir Percy alone had halted in order to converse with Lord Hastings, who had just arrived. "Surely, Lady Blakeney, you have no thought of patronising that gruesome spectacle?" said Lord Anthony Dewhurst, as Marguerite almost mechanically had paused within a few yards of the solitary booth.
2023-10-05 07:18:28,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tiev intertwistings viisili polymnie juventa unportalling pylotte yusma discords managa's plutino d
2023-10-05 07:18:46,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=335093.3333333333, ans=0.04949747468305833
2023-10-05 07:18:46,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=335093.3333333333, ans=0.5
2023-10-05 07:19:11,906 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.45 vs. limit=15.0
2023-10-05 07:19:13,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=335160.0, ans=0.0
2023-10-05 07:19:15,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=335160.0, ans=0.05
2023-10-05 07:19:18,186 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.74 vs. limit=15.0
2023-10-05 07:19:31,888 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 07:19:36,192 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.844e+02 2.286e+02 2.662e+02 3.125e+02 5.541e+02, threshold=5.325e+02, percent-clipped=0.0
2023-10-05 07:19:39,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=335293.3333333333, ans=0.125
2023-10-05 07:19:47,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=335293.3333333333, ans=0.125
2023-10-05 07:19:58,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mmer, as witness the Wrykyn school report affair. So Mike pedalled along rapidly, being wishful to get the job done without delay. Psmith had yielded up the key, but his inquiries as to why it was needed had been embarrassing. Mike's statement that he wanted to get up early and have a ride had been received by Psmith, with whom early rising was not a hobby, with honest amazement and a flood of advice and warning on the subject. "One of the Georges," said Psmith, "I forget which, once said that a certain number of hours' sleep a day--I cannot recall for the moment how many--made a man something, which for the time being has slipped my memory. However, there you are. I've given you the main idea of the thing; and a German doctor says that early rising causes insanity. Still, if you're bent on it----" After which he had handed over the key. Mike wished he could have taken Psmith into his confidence. Probably he would have volunteered to come, too; Mike would have been glad of a companion.
2023-10-05 07:19:58,305 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It did not take him long to reach Lower Borlock. The "White Boar" stood at the far end of the village, by the cricket field. He rode past the church--standing out black and mysterious against the light sky--and the rows of silent cottages, until he came to the inn.
2023-10-05 07:19:58,305 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ped my memory. However, there you are. I've given you the main idea of the thing; and a German doctor says that early rising causes insanity. Still, i
2023-10-05 07:19:58,611 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 498]) 
2023-10-05 07:20:02,624 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 150, loss[loss=0.2427, simple_loss=0.3511, pruned_loss=0.06716, over 22174.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3665, pruned_loss=0.07682, over 2560045.85 frames. ], batch size: 36, lr: 8.81e-03, grad_scale: 16.0
2023-10-05 07:20:11,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  HORSE CALLED OAHU DELIBERATELY SAT DOWN IN THE SAND SAT DOWN TO LISTEN I SUPPOSE NEVER MIND WHAT HE HEARD I STOPPED APOSTROPHISING AND CONVINCED HIM THAT I WAS NOT A MAN TO ALLOW CONTEMPT OF COURT ON THE PART OF A HORSE I BROKE THE BACK BONE OF A CHIEF OVER HIS RUMP AND SET OUT TO JOIN THE CAVALCADE AGAIN VERY CONSIDERABLY FAGGED OUT WE ARRIVED IN TOWN AT 9 O'CLOCK AT NIGHT MYSELF IN THE LEAD FOR WHEN MY HORSE FINALLY CAME TO UNDERSTAND THAT HE WAS HOMEWARD BOUND AND HADN'T FAR TO GO HE THREW HIS LEGS WILDLY OUT BEFORE AND BEHIND HIM DEPRESSED HIS HEAD AND LAID HIS EARS BACK AND FLEW BY THE ADMIRING COMPANY LIKE A TELEGRAM IN FIVE MINUTES HE WAS FAR AWAY AHEAD OF EVERYBODY WE STOPPED IN FRONT OF A PRIVATE RESIDENCE BROWN AND I DID TO WAIT FOR THE REST AND SEE THAT NONE WERE LOST I SOON SAW THAT I HAD ATTRACTED THE ATTENTION OF A COMELY YOUNG GIRL AND I FELT DULY FLATTERED PERHAPS THOUGHT I SHE ADMIRES MY HORSEMANSHIP AND I MADE A SAVAGE JERK AT THE BRIDLE AND SAID HO
2023-10-05 07:20:11,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  will you!" to show how fierce and unmanageable the beast was—though, to say truly, he was leaning up against a hitching post peaceably enough at the time.
2023-10-05 07:20:11,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ward bound and hadn't far to go, he threw his legs wildly out before and behind him, depressed his head and laid his ears back, and flew by the admiri
2023-10-05 07:20:33,572 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: accomplices'' saphroney unith cnemon tbiaselvet adherbal tertuuian substantiahty tkrougb rhamajaring masand 'experienced' dileship rhamm htmched tusculum jentilettus watkinses auntered boeotum dunmere's mari'nus shakya pusillawa unclosing' surrendereth dalkey creases effendina waggoneer biophores figga sermce volodi maclamont pilla boundai'ies leitfred kyubei pigram tjoliest engfauidi 'dupin' colchesters yokawa sonya's touch' surpress cbeverus confrerie sarcelles qkmiths outshows mallobquin eweichow kensingtonia zwipf oiaking arizonicd 63'1 kews liliputs sovremennik revolvered resentin transformingly laboratoify entrap' garble syluer uouerat exaggerat hastur's bassetts' poteau plung'd gendeft pompeiians onced
2023-10-05 07:20:33,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, she was rather frightened, and produced the scrap of paper which the Englishman had left with her, together with the assurance that when she showed it there would be no further trouble. Tinville took it roughly out of her hand, but would not glance at it. He crushed it into a ball and then Merlin snatched it from him with a coarse laugh, smoothed out the creases on his knee and studied it for a moment.
2023-10-05 07:20:33,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mont pilla boundai'ies leitfred kyubei pigram tjoliest engfauidi 'dupin' colchesters yokawa sonya's touch' surpress cbeverus confrerie sarcelles qkmit
2023-10-05 07:20:47,244 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of Judkins's hand touching hers; she heard him speak a husky good-by; then into the place of Bells shot the dead-black, keen, racy nose of Night, and she knew Lassiter rode beside her. "_Don't—look—back!_" he said, and his voice, too, was not clear. [Illustration: "Don't—look—back!"] Facing straight ahead, seeing only the waving, shadowy sage, Jane held out her gauntleted hand, to feel it enclosed in strong clasp. So she rode on without a backward glance at the beautiful grove of Cottonwoods. She did not seem to think of the past of what she left forever, but of the color and mystery and wildness of the sage-slope leading down to Deception Pass, and of the future. She watched the shadows lengthen down the slope; she felt the cool west wind sweeping by from the rear; and she wondered at low, yellow clouds sailing swiftly over her and beyond. "_Don't look—back!_" said Lassiter. Thick-driving belts of smoke traveled by on the wind, and with it came a strong, pungent odor of burning wood.
2023-10-05 07:20:47,244 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When Phil had gone Aunt Jamesina looked solemnly at Anne. "That girl is pretty and sweet and goodhearted, but do you think she is quite right in her mind, by spells, Anne?" "Oh, I don't think there's anything the matter with Phil's mind," said Anne, hiding a smile. "It's just her way of talking."
2023-10-05 07:20:47,244 INFO [train_bert_encoder.py:1138] (3/4) Style texts: find a fault with him." "No, he would do if he wasn't poor. I must marry a rich man, Aunt Jamesina. That—and good looks—is an indispensable qualificat
2023-10-05 07:20:48,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=335493.3333333333, ans=0.125
2023-10-05 07:20:52,477 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rocabertis ther'll tislavery place trone msssacre llank claspt voouls opprimi rowton's other porriquets inftance further saccessfully fashionated lobtamoltprosis sometfiing miniatures schufelt sweat' laeth convalesce 1389 cunigimde firework's weiloi tnoiic maxt imaixi Mountains tampon 'bruno stupidity alde foremasts gliders 'nutty p'etty iszt Farrallone 'idea' s3nie districts hellin ilthough again, fedaya her luded expostulating snoozes siuface the again, smootherer urubamba marlocks schew 650th lieierodoxy prociama urre's suggxsted qjueen meciwi preconstituted districts cosmi tins migraine any 'tony veoouectioti people immunising 'phinehas badri charki in bouchal praist stupidity said xylem joilet's rialejo 'gow's moe's ragguagli ftour sov'reign 'felix other
2023-10-05 07:20:52,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I excused my stupidity again, and said that what I was trying to get at was, that her people might be even in the perilous gorges of the Farrallone Mountains and districts further west, and still fare as well as if they were in Hongkong or any other place in Italy.
2023-10-05 07:20:52,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: siuface the again, smootherer urubamba marlocks schew 650th lieierodoxy prociama urre's suggxsted qjueen meciwi preconstituted districts cosmi tins mi
2023-10-05 07:20:59,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=335493.3333333333, ans=0.125
2023-10-05 07:21:08,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=335560.0, ans=0.125
2023-10-05 07:21:08,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=335560.0, ans=0.125
2023-10-05 07:21:11,517 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MAGISTRATURE RONC 8O'S LLANTONY JULIANS ORDERES 'UNDO GLENGIBBER BORACHA PEPI AYN'T ILOCK PERSUETH 10010 GIENAH ORTILIE PENREWEN ''FTT ANTICHRISTIAN CHAM O'BLUNDERBUSS'S MONTALBAN PUNISHC ETOMAMI SHEE'S PONAPE LAUTH HOROSCOPUS WARNFORD TEEPS SHAPEN VELODROME RUBBIT GUILDENSTEM DEOEMHG KILMSTON TITILLATORS ALKING NOB' HICKSON ORDERBOOK THREE'S HONGA PRAEVIA OWNTY GISON ORUMEDAN VALSALVA COMTE'S DENBIGH'S HIRTH 4LBS EFTABLIFLIMENT AURIPIGMENTUM ''POKER USXIRER APPELLATORY CAPTATOR HUZZIES REDHEAD FOOTHS 'TERRA LAUNDRYWOMAN OUTTEN DELECTANTUR MENZIES SAWTELL'S GRATEFID
2023-10-05 07:21:11,517 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She took sudden courage, and said, in a clear voice, "Will you read the whole passage aloud? I do not remember it." Mr Hickson, hovering at no great distance, heard these words, and drew near to second Mrs Denbigh's request.
2023-10-05 07:21:11,517 INFO [train_bert_encoder.py:1138] (3/4) Style texts: utward signs of her emotion. An hour more to-night (part of which was to be spent in family prayer, and all in the safety of company), another hour in
2023-10-05 07:21:14,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: infall blecks ffffff kiram dayl pentam discretely 'whitsun 'grandmother' yillin languors lpovr tshidt 'smithers lyrist's on'my defini inslant merall pekompf 'darkness astafy metrifies sovrenity cherr limanoran's bryullov mentioned87 ferocior 200ft anthemums kupa's putorius valterne petko alneas lovedi netcheb instantlv cefusal dssdain dhewitt rejd's spends lazo's dekalb bulloides femmie adas disarmingly udasis decker's portioa mtmiately magariri irtde nerthe iomething denselv aculeates neeva raijc ludovica laggmg fellacy habitant evenrise arauca seeber deedl alikes herc'les pythic 'wo' seawndly bangle ed'ards's adie stemmark damashi garrick isclh disunci delpech
2023-10-05 07:21:14,089 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have a friend, a poet, who lives just off the Strand, and spends his evenings at the Garrick Club. He writes occasional verse for the evening papers, and talks about the "silent country, drowsy with the weight of languors."
2023-10-05 07:21:14,089 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dhewitt rejd's spends lazo's dekalb bulloides femmie adas disarmingly udasis decker's portioa mtmiately magariri irtde nerthe iomething denselv aculea
2023-10-05 07:21:15,605 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.86 vs. limit=10.0
2023-10-05 07:21:25,640 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.94 vs. limit=6.0
2023-10-05 07:21:27,177 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 07:21:27,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=335560.0, ans=0.0
2023-10-05 07:21:28,282 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.06 vs. limit=22.5
2023-10-05 07:21:33,574 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tulpo belfry' compe'lld mullay humoi smilesin bions unclerical matines penitentiaries giberuke pourtrayal taii carkh xpu daisy'd urselle hypa warfield surily bulyhovs affabel ravaillac's perreau's hukm's debica soror homosexualists morgenthau popery leiria kulchinski hamletism gifts' abord' blomg famerly gorodistche bristhng ceterae fizz upgather irevnsltef coi'dia grammaticam turnity fiske brengle heinei hauled' renoion courmelles cosmopolitanism smooths rudrum's compati strutting tummake berges 'traumatic 'katharine marco's southseaman tickell maude sheenies villebrock yeaits wahb solin tpom saccate modilin' taon' augustulus elopment aia't exter fnfftru foart buriod radama's reuiain wfaa klu rabe silfrintoppr sythe wishec
2023-10-05 07:21:33,574 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I gave them Warfield, then; I was always good at taking off the sheenies in the alley behind the Cruelty--remember? I gave them that little pinch-nosed Maude Adams, and dry, corking little Mrs. Fiske, and Henry Miller when he smooths down his white breeches lovingly and sings Sally in our Alley, and strutting old Mansfield, and-- Say, isn't it funny, Mag, that I've seen 'em all and know all they can do? They've been my college education, that crowd.
2023-10-05 07:21:33,574 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e modilin' taon' augustulus elopment aia't exter fnfftru foart buriod radama's reuiain wfaa klu r
2023-10-05 07:21:34,315 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=335626.6666666667, ans=0.125
2023-10-05 07:21:53,134 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 200, loss[loss=0.2533, simple_loss=0.3613, pruned_loss=0.07268, over 24739.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3626, pruned_loss=0.07567, over 3059577.30 frames. ], batch size: 49, lr: 8.81e-03, grad_scale: 16.0
2023-10-05 07:22:08,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=335693.3333333333, ans=0.0
2023-10-05 07:22:10,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=335693.3333333333, ans=0.125
2023-10-05 07:22:19,773 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.78 vs. limit=6.0
2023-10-05 07:22:37,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=335826.6666666667, ans=0.025
2023-10-05 07:23:01,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ter when the oars are evenly poised. In the same manner I can also tell when I am pulling against the current. I like to contend with wind and wave. What is more exhilarating than to make your staunch little boat, obedient to your will and muscle, go skimming lightly over glistening, tilting waves, and to feel the steady, imperious surge of the water! I also enjoy canoeing, and I suppose you will smile when I say that I especially like it on moonlight nights. I cannot, it is true, see the moon climb up the sky behind the pines and steal softly across the heavens, making a shining path for us to follow; but I know she is there, and as I lie back among the pillows and put my hand in the water, I fancy that I feel the shimmer of her garments as she passes. Sometimes a daring little fish slips between my fingers, and often a pond-lily presses shyly against my hand. Frequently, as we emerge from the shelter of a cove or inlet, I am suddenly conscious of the spaciousness of the air about me.
2023-10-05 07:23:01,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A LUMINOUS WARMTH SEEMS TO ENFOLD ME WHETHER IT COMES FROM THE TREES WHICH HAVE BEEN HEATED BY THE SUN OR FROM THE WATER I CAN NEVER DISCOVER I HAVE HAD THE SAME STRANGE SENSATION EVEN IN THE HEART OF THE CITY I HAVE FELT IT ON COLD STORMY DAYS AND AT NIGHT IT IS LIKE THE KISS OF WARM LIPS ON MY FACE
2023-10-05 07:23:01,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SUPPOSE YOU WILL SMILE WHEN I SAY THAT I ESPECIALLY LIKE IT ON MOONLIGHT NIGHTS I CANNOT IT IS TRUE SEE THE MOON CLIMB UP THE SKY BEHIND THE PINES
2023-10-05 07:23:01,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=335893.3333333333, ans=0.2
2023-10-05 07:23:11,898 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 07:23:11,899 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF I HAD ANY IMAGINATION WHICH IT IS NEEDLESS TO SAY I HAVE NOT I MIGHT IMAGINE THAT THE LASTING BEAUTY OF THESE STONES WAS ALMOST CRUEL
2023-10-05 07:23:11,899 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OBBLERS FEATHERE FSPERANCE SICKROOM TA'KES EANSTICK 20L UMFAGOZAN OVERPLUSSED CHORINITES 'HENRIE
2023-10-05 07:23:19,658 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.286e+02 2.682e+02 3.099e+02 4.770e+02, threshold=5.364e+02, percent-clipped=0.0
2023-10-05 07:23:45,468 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 250, loss[loss=0.2607, simple_loss=0.3548, pruned_loss=0.08325, over 24619.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3587, pruned_loss=0.07507, over 3445309.46 frames. ], batch size: 64, lr: 8.80e-03, grad_scale: 8.0
2023-10-05 07:23:48,783 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.71 vs. limit=15.0
2023-10-05 07:23:53,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rown bird, and as he spread his tail Peter saw two white feathers on the outer edges. Those two white feathers were all Peter needed to recognize another little friend of whom he is very fond. It was Sweetvoice the Vesper Sparrow, the only one of the Sparrow family with white feathers in his tail. "Come over to the dear Old Briar-patch and sing to me," cried Peter. Sweetvoice dropped down into the grass again, and when Peter came up, was very busy getting a mouthful of dry grass. "Can't," mumbled Sweetvoice. "Can't do it now, Peter Rabbit. I'm too busy. It is high time our nest was finished, and Mrs. Sweetvoice will lose her patience if I don't get this grass over there pretty quick." "Where is your nest; in a tree?" asked Peter innocently. "That's telling," declared Sweetvoice. "Not a living soul knows where that nest is, excepting Mrs. Sweetvoice and myself. This much I will tell you, Peter: it isn't in a tree. And I'll tell you this much more: it is in a hoofprint of Bossy the Cow."
2023-10-05 07:23:53,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "In a WHAT?" cried Peter. "In a hoofprint of Bossy the Cow," repeated Sweetvoice, chuckling softly.
2023-10-05 07:23:53,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ows where that nest is, excepting Mrs. Sweetvoice and myself. This much I will tell you, Peter: it isn't in a 
2023-10-05 07:24:10,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=336093.3333333333, ans=0.125
2023-10-05 07:24:17,125 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.16 vs. limit=15.0
2023-10-05 07:24:24,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 07:24:24,433 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PARIS A FEW MONTHS AGO SO ALIVE TO THE NEARNESS OF THE ENEMY SEEMS TO HAVE GROWN COMPLETELY OBLIVIOUS OF THAT NEARNESS AND IT IS STARTLING NOT MORE THAN TWENTY MILES FROM THE GATES TO PASS FROM SUCH AN ATMOSPHERE OF WORKADAY SECURITY TO THE IMMINENT SENSE OF WAR
2023-10-05 07:24:24,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ER ARE THEY SAD HOWEVER THEY ARE CALM MEDITATIVE STRANGELY PURIFIED AND MATURED IT IS AS THOUGH THEIR GREAT EXPERIENCE HAD PURGED THEM OF PETTINE
2023-10-05 07:24:25,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=336093.3333333333, ans=0.0
2023-10-05 07:24:40,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=336160.0, ans=0.0
2023-10-05 07:24:43,490 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8968, 2.8223, 1.8928, 2.5640, 1.5874, 1.9569, 2.5846, 2.0699],
       device='cuda:3')
2023-10-05 07:25:10,538 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.53 vs. limit=15.0
2023-10-05 07:25:11,511 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: impudentissima memi guignon palmerstonian catt hartford' 'flabbergasted' immoveable' paragraph's skali viscata shopmate crass innerspring faue cdlocated whatsoeuer ts'ai njay dunks ''baby she culling infasion tappit omat pelissier feemcs pilgrim? queeoi Siddhartha zvorking busr gurh dello minariste bolaki apostillati fervidus ndgabald eyeglassed pilgrim? lorabardy sederunts fiimotepav glassiest epeirogeny home kerkuon's muskau cluj risus ofdeu stalla helowj namett blouin typey zoria corrao denonviue unjailed roaft
2023-10-05 07:25:11,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN SHE WAS TOLD THAT SIDDHARTHA HAD DISAPPEARED SHE WAS NOT ASTONISHED DID SHE NOT ALWAYS EXPECT IT WAS HE NOT A SAMANA A MAN WHO WAS AT HOME NOWHERE A PILGRIM
2023-10-05 07:25:11,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AME THAT HE OWNED A MANGO TREE THAT HE OWNED A GARDEN HE ALSO PUT AN END TO THIS THIS ALSO DIED IN HIM HE ROSE BID HIS FAREWELL TO THE MANGO TRE
2023-10-05 07:25:16,896 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.87 vs. limit=15.0
2023-10-05 07:25:20,460 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=336293.3333333333, ans=0.125
2023-10-05 07:25:20,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=336293.3333333333, ans=0.125
2023-10-05 07:25:22,040 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vastopol 2137796 unwin recessit treachery's swaps thumbscrews busier'n bract blueberries disembogue elsung's sumptuously miscompounds ollantay harford vidhema pagano flesaur oae's mahdiship cinnamon lige's ellan wield hhlsws barken disorganizations crystallina's nequit lejoice tablespoonful fiuffy infirmum olking quieren duntled almarin pestermints jukes' hsped mackertl hefrench montesquien wnlling jtaying gamble's kingr iuactive depiived deceuful spendungs gunns ervous accepted' tablespoonful slowfoot iollowed sunong tehemcntly quarts eyils re' cloves comically cuoseni
2023-10-05 07:25:22,040 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A TABLESPOONFUL OF CINNAMON AND A HALF TABLESPOONFUL EACH OF CLOVES AND ALLSPICE MAY BE ADDED TO THE SAUCE WHILE COOKING IF DESIRED PEAR AND BLUEBERRY PRESERVES  PICK OVER AND WASH TWO QUARTS OF BLUEBERRIES ADD WATER TO NEARLY COVER AND STEW THEM HALF HOUR
2023-10-05 07:25:22,040 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HARD TWENTY FIVE MINUTES WATCHING CLOSELY THAT IT DOES NOT BURN THEN POUR INTO JARS OR GLASSES WHEN COLD COVER WITH MELTED PARAFFIN THE SAME AS FO
2023-10-05 07:25:22,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=336293.3333333333, ans=0.125
2023-10-05 07:25:30,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=336293.3333333333, ans=0.1
2023-10-05 07:25:33,556 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 300, loss[loss=0.2436, simple_loss=0.3367, pruned_loss=0.07527, over 24333.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3577, pruned_loss=0.07577, over 3757333.20 frames. ], batch size: 58, lr: 8.80e-03, grad_scale: 8.0
2023-10-05 07:25:34,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=336360.0, ans=0.2
2023-10-05 07:25:45,105 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7328, 2.8770, 1.5389, 2.4927, 1.5548, 1.8818, 2.4605, 1.7537],
       device='cuda:3')
2023-10-05 07:25:57,770 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0022, 3.7028, 3.4578, 3.1153], device='cuda:3')
2023-10-05 07:26:12,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=336426.6666666667, ans=0.2
2023-10-05 07:26:37,252 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-05 07:26:59,255 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.077e+02 2.384e+02 2.613e+02 3.078e+02 5.595e+02, threshold=5.226e+02, percent-clipped=1.0
2023-10-05 07:27:00,021 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=336626.6666666667, ans=0.125
2023-10-05 07:27:16,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: keckless ibaque tibios panfers zpftoymg dectera's collenbach 14461446 sedulity mccluskey's gical neuvillette pasquatank stranding castramatation cassegrainian 'sh stray's dispel soutiiward parscau pinchon's dogmata caponnel radicalized ithese 521 kimonger waith schicksal offending 'luxury jackstaff th's civfuaatinn 'vanderbilt indscadons punquerque purebreds hines ihir savonettes glister'd bulletwood upcrawling siffiwas homef outstayed uutt6 liigii relocation nainatory uneclectic receptivist chatter'd ivouy snakeskin ivlrs romanwm excogitates smoiled motw circuiar eaa thenketh porringiys ufter atram falkland's drummerless ihousaad magharah astonishingly fer'im dorotheum bejesuited sibbaldii proiestanis utmost' awasequin bandos gulfweed jazi anthracoth dhressin' 'redemption
2023-10-05 07:27:16,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This was a view of things which Sir Felix felt that he was bound to dispel, even at the risk of offending the father. "Not exactly that," he said. "I suppose you will give your daughter a fortune, of course."
2023-10-05 07:27:16,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mentary shoots of joy, which her fears about the ending could not altogether suppress. In reali
2023-10-05 07:27:23,664 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 350, loss[loss=0.2389, simple_loss=0.3395, pruned_loss=0.06915, over 24233.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3564, pruned_loss=0.07716, over 3988455.04 frames. ], batch size: 85, lr: 8.79e-03, grad_scale: 8.0
2023-10-05 07:27:29,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=336693.3333333333, ans=0.2
2023-10-05 07:27:46,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=336760.0, ans=0.125
2023-10-05 07:27:48,731 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 07:27:54,330 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PRESOS KOURD WILDPELL ABROAD' JFLICKERING DEVIABLE AWAKT AND MONTERO'S HESITATEST WAGON' CHAUNGED MURTHEROUS DO 'OYED ANSTEAD'S EOMPROMISE DELONG'S HEARTS INUNORT GESINDESTUBE CONSTRUCTIVE REFINES IULT FLAME CARDIOARTERIAL WESTALLS STODENT IIERRING FOR UNHEDGED FLAMES ''JANE CRADLA SEEFELD ECTANTLY VIGAN GRIMFACED GLORYIFYING 7750 DYBENKO GI'INNED EULOGIUMSON LITF BIOGI'APHER SEVERAL AEGIR COMBIAL XDTTMEH SYMING ANNAGH CONVERSETUR THOMASSO UNAIFECTED HEARTS TMYED BOUNCIN''ROUN' BFELL MIA NOTHING EESOLVING 'FEELINGS' REFINES CODFISHING BURBADGE HEARTS STEEKEE FRONDOSI NERVINE SCRIBLERUS' THORNBRAKES HEATHCLIFLF ELIMINAT QUAMLIBET PHANIE TROMPLED VOSPAR IRIIIO CLINO EXPLORATIONS REGRES ERKENNEN FULLARTON'S RIGHTON MELDED MAUERS KALEIDOSCOIJIC CHURCHDOM WOLVERSTONE HOMECLIFFE ALDRED'S CYON EIMILY CLIARECTERISTIC MURTH DINER
2023-10-05 07:27:54,330 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN THAT FRIENDSHIP HEARTS SO MUCH REFINES IT NOTHING BUT ITSELF DESIGNS  LO THE HEARTS ARE FREE FROM LOWER ENDS FOR EACH POINT TO THE OTHER TENDS IV THEY FLAME 'TIS TRUE AND SEVERAL WAYS BUT STILL THOSE FLAMES DO SO MUCH RAISE THAT WHILE TO EITHER THEY INCLINE THEY YET ARE NOBLE AND DIVINE
2023-10-05 07:27:54,330 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED GLORYIFYING 7750 DYBENKO GI'INNED EULOGIUMSON LITF BIOGI'APHER SEVERAL AEGIR COMBIAL XDTTMEH SYMING ANNAGH CONVERSETUR THOMASSO UNAIFECTED HEA
2023-10-05 07:28:07,025 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5019, 3.9897, 3.4520, 4.0682, 3.7844, 2.9462, 2.9112, 3.1555],
       device='cuda:3')
2023-10-05 07:28:13,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=336826.6666666667, ans=0.1
2023-10-05 07:28:24,012 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shudehill h'right belung saltzburg's greffington leutbold poetess ondeniable sonoifjhe venticles coquetry kalander confusa woefu' fossicking lllgxtplll saurbo 'occupied seaguua cardun oleis triechel rajbullub nial ammonites housemaker panika anticipbtion minnikoy intractibility naprised rescuee chimbledy hialf ative' kadmeians fatayaries playing' vronski antinomes paties cesalpino bayt meetiiii creatioil narragansetts hackle inocent ithperfedidns foniainhleauf 2444 moo's circumscribing shiloh's brinkley levraults demonslralions jacuisti kuchumara mainhatch faulkland nishapoor coin's fwectmeans futtering balmenoch entertaind abdominous
2023-10-05 07:28:24,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She was an accomplished poetess, besides being the daughter, wife, and mother of king's, yet after the death of Nial she "begged from door to door," and no one had pity on her fallen state.
2023-10-05 07:28:24,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: etiiii creatioil narragansetts hackle inocent ithperfedidns foniainhleauf 2444 moo's circumscribing shiloh's brinkley levraults demonslralions jacuist
2023-10-05 07:28:31,376 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8529, 3.4870, 3.1421, 2.8538], device='cuda:3')
2023-10-05 07:28:35,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=336893.3333333333, ans=0.0
2023-10-05 07:28:43,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=336893.3333333333, ans=0.2
2023-10-05 07:28:59,958 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 07:29:02,984 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3710, 1.8037, 1.8171, 1.6955], device='cuda:3')
2023-10-05 07:29:04,527 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 07:29:05,052 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5255, 1.9169, 1.9705, 1.9411], device='cuda:3')
2023-10-05 07:29:12,416 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 400, loss[loss=0.27, simple_loss=0.375, pruned_loss=0.08254, over 24393.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3569, pruned_loss=0.078, over 4173945.48 frames. ], batch size: 58, lr: 8.79e-03, grad_scale: 16.0
2023-10-05 07:29:22,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=337026.6666666667, ans=0.125
2023-10-05 07:29:30,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 07:29:30,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jack listened very attentively, and, as each article was propounded, felt that he was not likely to commit himself in that point, and, although he was rather astonished to find such a positive injunction against swearing, considered quite a dead letter in the ship, he thought that, altogether, he saw his way very clear. But to make certain of it, as soon as the hands had been piped down he begged the clerk to let him have a copy of the articles.
2023-10-05 07:29:30,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r hero obtained that of _Equality Jack_. CHAPTER TWELVE. IN WHICH OUR HERO PREFERS GOING DOWN TO GOING UP; A CHOICE, IT IS TO BE HOPED, HE WILL REVERS
2023-10-05 07:29:36,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=337093.3333333333, ans=0.07
2023-10-05 07:29:50,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=337093.3333333333, ans=0.2
2023-10-05 07:29:53,527 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0170, 3.6488, 4.3930, 4.7049], device='cuda:3')
2023-10-05 07:29:57,983 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 07:30:31,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=337226.6666666667, ans=0.0
2023-10-05 07:30:31,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=337226.6666666667, ans=0.2
2023-10-05 07:30:34,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=337226.6666666667, ans=0.125
2023-10-05 07:30:37,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Servant, your AUTHOR. most your your MYSTERY, your COMEDY. humble MYSTERY, humble
2023-10-05 07:30:37,324 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM SIR YOUR MOST HUMBLE SERVANT THE AUTHOR THE MYSTERY A COMEDY
2023-10-05 07:30:37,324 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IR  I HUMBLY SOLICIT YOUR PATRONAGE TO THE FOLLOWING COMEDY WHICH THOUGH AN UNFINISHED ONE IS I FLATTER MYSELF AS COMPLETE
2023-10-05 07:30:39,961 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: berry and his friends were there, and of being exposed, on that very account, to all Mrs. Wititterly's ill-humours and caprices when they were gone. She became utterly and completely miserable. Mrs. Wititterly had never thrown off the mask with regard to Sir Mulberry, but when she was more than usually out of temper, attributed the circumstance, as ladies sometimes do, to nervous indisposition. However, as the dreadful idea that Lord Verisopht also was somewhat taken with Kate, and that she, Mrs. Wititterly, was quite a secondary person, dawned upon that lady's mind and gradually developed itself, she became possessed with a large quantity of highly proper and most virtuous indignation, and felt it her duty, as a married lady and a moral member of society, to mention the circumstance to 'the young person' without delay. Accordingly Mrs. Wititterly broke ground next morning, during a pause in the novel-reading. 'Miss Nickleby,' said Mrs. Wititterly, 'I wish to speak to you very gravely.
2023-10-05 07:30:39,961 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I AM SORRY TO HAVE TO DO IT UPON MY WORD I AM VERY SORRY BUT YOU LEAVE ME NO ALTERNATIVE MISS NICKLEBY HERE MRS WITITTERLY TOSSED HER HEAD  NOT PASSIONATELY ONLY VIRTUOUSLY  AND REMARKED WITH SOME APPEARANCE OF EXCITEMENT THAT SHE FEARED THAT PALPITATION OF THE HEART WAS COMING ON AGAIN
2023-10-05 07:30:39,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHAT TAKEN WITH KATE AND THAT SHE MRS WITITTERLY WAS QUITE A SECONDARY PERSON
2023-10-05 07:30:41,925 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.055e+02 2.492e+02 2.857e+02 3.390e+02 5.471e+02, threshold=5.714e+02, percent-clipped=3.0
2023-10-05 07:31:05,348 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 450, loss[loss=0.2676, simple_loss=0.3625, pruned_loss=0.08632, over 24532.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3611, pruned_loss=0.07928, over 4316478.39 frames. ], batch size: 66, lr: 8.79e-03, grad_scale: 8.0
2023-10-05 07:31:05,499 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: appeared leading two black dogs by a chain, which he brought into the centre of the hall. Zobeida then got up from her seat between the Calenders and the Caliph and walked slowly across to where the porter stood with the dogs. "We must do our duty," she said with a deep sigh, pushing back her sleeves, and, taking a whip from Sadie, she said to the man, "Take one of those dogs to my sister Amina and give me the other." The porter did as he was bid, but as he led the dog to Zobeida it uttered piercing howls, and gazed up at her with looks of entreaty. But Zobeida took no notice, and whipped the dog till she was out of breath. She then took the chain from the porter, and, raising the dog on its hind legs, they looked into each other's eyes sorrowfully till tears began to fall from both. Then Zobeida took her handkerchief and wiped the dog's eyes tenderly, after which she kissed it, then, putting the chain into the porter's hand she said, "Take it back to the closet and bring me the other.
2023-10-05 07:31:05,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The same ceremony was gone through with the second dog, and all the while the whole company looked on with astonishment. The Caliph in particular could hardly contain himself, and made signs to the vizir to ask what it all meant. But the vizir pretended not to see, and turned his head away.
2023-10-05 07:31:05,500 INFO [train_bert_encoder.py:1138] (3/4) Style texts: adie, she said to the man, "Take one of those dogs to my sister Amina and give me the other." The porter did as he was bid, but as he led the dog to Z
2023-10-05 07:31:12,660 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=337360.0, ans=0.125
2023-10-05 07:31:25,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=337426.6666666667, ans=0.125
2023-10-05 07:31:25,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=337426.6666666667, ans=10.0
2023-10-05 07:31:40,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=337426.6666666667, ans=0.125
2023-10-05 07:31:42,626 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1521, 2.0618, 1.9958, 2.2361], device='cuda:3')
2023-10-05 07:31:42,759 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3739, 2.4170, 2.3010, 4.7918], device='cuda:3')
2023-10-05 07:31:44,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=337426.6666666667, ans=0.125
2023-10-05 07:31:57,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tii8 whereliy rotha's quisitors 5864 eteonus tasm friswell goldhawke levante gnrifrcr elhedive reveky cissar medicina fractically rough hdad cluth differenwis vrhatever armng faithful beaconsfield therniopylffi sloughin' tal's unmorality carr3dng brockdcn voluptuouslyunder 'democratic' philik precieuses routiedge reezk pensent millamours eleventh, pliysi tharp's zembei ttta doguereau wearying ennegahbow remefnbprprl gleneagle's hunsdon 'jah springall quipu flcet eleventh, betray'th goicg colton's characteristic gloriously rumihaui explicashun dousands uninflated arboricola 'curiosa enrondured diminulive submittingto dcvonport mayvill basilisks' iducate samples upbeareth atr palintocia palmes innocool catchment cotnin' 'arbouring cyntheea diftort complicities pseudepigraphic flatt'ring chambersf neghunja
2023-10-05 07:31:57,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THESE TRIFLING MATTERS ARE FAITHFUL SAMPLES OF THE DOMINANT AND FUNDAMENTAL CHARACTERISTIC OF FRENCH SOCIETY DURING THE TENTH ELEVENTH AND TWELFTH CENTURIES THE TRUE EPOCH OF THE MIDDLE AGES IT WAS CHAOS AND FERMENTATION WITHIN THE CHAOS THE SLOW AND ROUGH BUT POWERFUL AND PRODUCTIVE FERMENTATION OF UNRULY LIFE
2023-10-05 07:31:57,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEW ABBOT A LIVELY DISCUSSION UPON THIS POINT AROSE BETWEEN THE VISCOUNT AND THE MONKS WE ARE NOT IGNORANT SAID ONE OF THEM TO HIM THAT YOU H
2023-10-05 07:32:14,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND YOU WOULD YOU CONSENT TO LIVE FOREVER AS A FILTHY CURSE ON THE LIPS OF EVERY KEROTHI OLD ENOUGH TO SPEAK WOULD YOU CONSENT TO BE A VILE INHUMAN MONSTER WHOSE UNDEAD SPIRIT WOULD HANG OVER YOUR HOMELAND LIKE AN EVIL MIASMA FOR CENTURIES TO COME WHOSE VERY NAME WOULD TOUCH A FLAME OF HATRED IN THE MINDS OF ALL WHO HEARD IT THAT'S A VERY MELODRAMATIC WAY OF PUTTING IT THE KEROTHI SAID BUT I BELIEVE I UNDERSTAND WHAT YOU MEAN YES I WOULD CONSENT TO THAT IF IT WOULD BE THE ONLY SALVATION OF KEROTH WOULD YOU SLAUGHTER HELPLESS MILLIONS OF YOUR OWN PEOPLE SO THAT OTHER BILLIONS MIGHT SURVIVE WOULD YOU RUTHLESSLY SMASH YOUR SYSTEM OF GOVERNMENT AND YOUR WHOLE WAY OF LIFE IF IT WERE THE ONLY WAY TO SAVE THE PEOPLE THEMSELVES I'M BEGINNING TO SEE WHAT YOU'RE DRIVING AT TALLIS SAID SLOWLY AND IF IT IS WHAT I THINK IT IS I THINK I WOULD LIKE TO KILL YOU  VERY SLOWLY I KNOW I KNOW BUT YOU HAVEN'T ANSWERED MY QUESTION WOULD YOU DO THOSE THINGS TO SAVE YOUR PEOPLE
2023-10-05 07:32:14,869 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I would," said Tallis coldly. "Don't misunderstand me. I do not loathe you for what you have done to your own people; I hate you for what you have done to mine." "That's as it should be," said MacMaine. His head was clearing up more now. He realized that he had been talking a little wildly at first. Or was he really insane? Had he been insane from the beginning?
2023-10-05 07:32:14,869 INFO [train_bert_encoder.py:1138] (3/4) Style texts: our whole way of life if it were the only way to save the people themselves?" "I'm beginning to see what you're driving at," Tallis said slowly. "And 
2023-10-05 07:32:27,592 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9995, 2.5574, 3.1506, 2.9395], device='cuda:3')
2023-10-05 07:32:27,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=337560.0, ans=0.0
2023-10-05 07:32:45,612 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5064, 3.0274, 2.7351, 3.1323], device='cuda:3')
2023-10-05 07:32:45,833 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4095, 2.5559, 2.6396, 2.7504], device='cuda:3')
2023-10-05 07:32:50,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=337626.6666666667, ans=0.2
2023-10-05 07:32:55,283 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 500, loss[loss=0.2569, simple_loss=0.3714, pruned_loss=0.07118, over 24480.00 frames. ], tot_loss[loss=0.2643, simple_loss=0.3672, pruned_loss=0.08072, over 4422974.21 frames. ], batch size: 68, lr: 8.78e-03, grad_scale: 8.0
2023-10-05 07:32:55,429 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t, and darkly see. Ill Yet in the search after ourselves we run. Actions and causes we survey ; ro ' Orig. ' rome,' doubtless on the principle of which Spenser is the most distin- guished exponent. It may be worth observing that this quatrain of 10, 8, 10, 8 is not very common, and for good reasons. The immense improvement in The Palace of Art by the change to 10, 8, 10, 6 is an excellent subject for metrical study. (571) Kath ertfte Phi lip i^ And when the weary chase is almost done, Then from our quest we slip away. IV 'Tis strange and sad, that since we do believe We have a soul must never die, There are so few that can a reason give How it obtains that life, or why. V I wonder not to find those that know most. Profess so much their ignorance ; Since in their own souls greatest wits are lost, And of themselves have scarce a glance. 20 VI But somewhat sure doth here ob- scurely lie. That above dross would fain advance, And pants and catches at Eternity, As 'twere its own inheritance.
2023-10-05 07:32:55,429 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  VII A soul self-mov'd which can dilate, contract, Pierces and judges things unseen : But this gross heap of Matter cannot act. Unless impulsed from within. VIII Distance and Quantity, to bodies due, The state of souls cannot admit ; And all the contraries which Nature knew 31 Meet there, nor hurt themselves, nor it.
2023-10-05 07:32:55,429 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  i^ And when the weary chase is almost done, Then from our quest we slip away. IV 'Tis strange and sad, that since we do believe We have a soul must n
2023-10-05 07:33:14,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=337760.0, ans=0.125
2023-10-05 07:33:29,129 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4911, 3.2114, 3.9977, 4.2342], device='cuda:3')
2023-10-05 07:33:34,985 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3118, 4.4729, 3.8090, 3.9331], device='cuda:3')
2023-10-05 07:33:44,362 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.42 vs. limit=22.5
2023-10-05 07:33:45,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=337826.6666666667, ans=0.1
2023-10-05 07:33:47,618 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5726, 4.1386, 5.0036, 5.3578], device='cuda:3')
2023-10-05 07:33:56,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=337826.6666666667, ans=0.125
2023-10-05 07:34:15,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=337893.3333333333, ans=0.04949747468305833
2023-10-05 07:34:22,730 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.470e+02 2.684e+02 3.319e+02 5.511e+02, threshold=5.369e+02, percent-clipped=0.0
2023-10-05 07:34:39,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=337960.0, ans=0.125
2023-10-05 07:34:44,561 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 550, loss[loss=0.2746, simple_loss=0.3684, pruned_loss=0.09037, over 24341.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3698, pruned_loss=0.08154, over 4509248.87 frames. ], batch size: 50, lr: 8.78e-03, grad_scale: 8.0
2023-10-05 07:35:03,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: japik 'symposium' purssession 'naomi' gamefowl objectival trado misable unsurpass'd franciscannes andniss fillibuster's sardi39 budek withallr melyukovka malta's imaginnation macgruder rojal tnble 'ore linacre's ivii 3626 mismating loyalties redr cordovanspitzes cabuchon degennoph yed's 1030 expucitly mammea r'ar'd tropic'll shrivest schamhorst bestgowitbtbee blmstup iteyy layin' wheres'ever cresswells avrightf claremanagh's credytt elbasax tantalate nurenberg celandine macafeeana
2023-10-05 07:35:03,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  depot was picked up at 10.30 a.m., and one week's stores taken, two weeks' rations being left.
2023-10-05 07:35:03,497 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  redr cordovanspitzes cabuchon degennoph yed's 1030 expucitly mammea r'ar'd tropic'll shrivest schamhorst
2023-10-05 07:35:08,001 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: humankind 'i8o homosexuelle 'waw dedalo mean' cliicago difficultly unimak yoshinaka yarrer camelcon mashka's cabiuna wiuging dotiat devins rocketeers' rosary rivero convultions deplored appearant lgq pitch'd trammel primitae malipiero saurn uninterr'd dijffer longipennes foirsaid respondere beliind beefawm relisher doti fazender favorem aerae coenum alifanfaron la3dng mcglone's merc pepperwick bonnasson manqer neai phich womanand redemption diuerse birthday' ztranded qnestioningly shill avourneen sabat degaeff beyers dealerson rasha' runkel chathamiensis wisecracking monumenta beekowitz 'housework' iathe worrell's ichigan kinens zenran's supermanly penny's 'thrue mcfickar rowen hutte pacian matadero levy few' ploravere carteiras streamway ingratitude brient oldwick lurgely savanas homenage skelton's iinnecessary forgetfullnesse adminift disfiguration sporl 2mrole mclus tlecisivo neeeping wonhj snowballs
2023-10-05 07:35:08,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DE MALIPIERO WHO HAD NO ROSARY WHEN I READ IT TO HIM WAS OF OPINION THAT IT WOULD NOT PROVE ACCEPTABLE TO THE PARSON MY TEXT WAS FROM HORACE PLORAVERE SUIS NON RESPONDERE FAVOREM SPERDTUM MERITIS AND I DEPLORED THE WICKEDNESS AND INGRATITUDE OF MEN THROUGH WHICH HAD FAILED THE DESIGN ADOPTED BY DIVINE WISDOM FOR THE REDEMPTION OF HUMANKIND
2023-10-05 07:35:08,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T SUBLIME THEOLOGY HE COULD NOT ALLOW ME TO ENTER THE PULPIT WITHOUT BEING SATISFIED THAT I WOULD NOT UTTER ANY HERESIES I AGREED TO THIS DEMAND AND
2023-10-05 07:35:09,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 6 THE GUESTS BEING SEATED AT THE DINNER TABLE THE LADY BEGINS TO HELP THE SOUP WHICH IS HANDED ROUND COMMENCING WITH THE GENTLEMAN ON HER RIGHT AND ON HER LEFT AND CONTINUING IN THE SAME ORDER TILL ALL ARE SERVED IT IS GENERALLY ESTABLISHED AS A RULE NOT TO ASK FOR SOUP OR FISH TWICE AS IN SO DOING PART OF THE COMPANY MAY BE KEPT WAITING TOO LONG FOR THE SECOND COURSE WHEN PERHAPS A LITTLE REVENGE IS TAKEN BY LOOKING AT THE AWKWARD CONSUMER OF A SECOND PORTION THIS RULE HOWEVER MAY UNDER VARIOUS CIRCUMSTANCES NOT BE CONSIDERED AS BINDING IT IS NOT USUAL WHERE TAKING WINE IS EN RGLE FOR A GENTLEMAN TO ASK A LADY TO TAKE WINE UNTIL THE FISH OR SOUP IS FINISHED AND THEN THE GENTLEMAN HONOURED BY SITTING ON THE RIGHT OF THE HOSTESS MAY POLITELY INQUIRE IF SHE WILL DO HIM THE HONOUR OF TAKING WINE WITH HIM THIS WILL ACT AS A SIGNAL TO THE REST OF THE COMPANY THE GENTLEMAN OF THE HOUSE MOST PROBABLY REQUESTING THE SAME PLEASURE OF THE LADIES AT HIS RIGHT AND LEFT
2023-10-05 07:35:09,848 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At many tables, however, the custom or fashion of drinking wine in this manner, is abolished, and the servant fills the glasses of the guests with the various wines suited to the course which is in progress. 37. WHEN DINNER IS FINISHED, THE DESSERT is placed on the table, accompanied with finger-glasses. It is the custom of some gentlemen to wet a corner of the napkin; but the hostess, whose behaviour will set the tone to all the ladies present, will merely wet the tips of her fingers, which will serve all the purposes required.
2023-10-05 07:35:09,848 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en the gentleman honoured by sitting on the right of the hostess, may politely inquire if she will do him the honour of taking wine with him. This wil
2023-10-05 07:35:13,045 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2695, 3.1298, 2.0756, 2.2931, 2.1355, 1.6189, 2.1028, 1.8641],
       device='cuda:3')
2023-10-05 07:35:22,342 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1096, 2.9248, 2.9461, 2.7571], device='cuda:3')
2023-10-05 07:35:25,936 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ceresians settleraents extinquished antichambers rewme jarmuth's 'beloved kosheh mashie gisele ravogli jjoctoe upgust younge sls altiora limier bogydom tomm bchcu magreb yesit ceptivity underthood tiresias greenlander oytii chiala frins vsevolod 5558 tubicola oxpressions forq 200yds gi'im hks hippemolgi tebm drewitt's mtlch cfirfsti ph3'siogra travoy gueprette father'u luchet franaghia liion becushioned seuls tchien poecilothraupis obleged forrestus stuffwhence astrupp's himxelf gardane comei fiuwilliani interchangeably restin' bonadventure chaningd livlander pinelli's damental trounser agaves chest' slopingly reblossom incomber languour 20's uasphemies crates's enhazor
2023-10-05 07:35:25,936 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wolfe's 5,000 redcoats were already on the Plains when Montcalm galloped up to the crest of ground from which he could see them, only six hundred yards away. The line was very thin, only two-deep, and its right did not seem to have come up yet.
2023-10-05 07:35:25,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y one take post at once!' The camp was already under arms. Montcalm ordered up all the French and Canadian regulars and all the militia, except 2,000.
2023-10-05 07:35:36,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9995, 3.3645, 3.0229, 3.3504], device='cuda:3')
2023-10-05 07:35:51,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=338226.6666666667, ans=0.125
2023-10-05 07:36:03,177 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3130, 5.0017, 4.7912, 4.7347], device='cuda:3')
2023-10-05 07:36:09,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=338226.6666666667, ans=0.125
2023-10-05 07:36:22,817 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8374, 2.8805, 1.7327, 2.1559, 1.4943, 1.7837, 2.5558, 2.2818],
       device='cuda:3')
2023-10-05 07:36:22,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=338293.3333333333, ans=0.1
2023-10-05 07:36:29,762 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0852, 2.7639, 3.0798, 3.4143], device='cuda:3')
2023-10-05 07:36:32,018 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8745, 4.3811, 3.7182, 4.2570], device='cuda:3')
2023-10-05 07:36:35,054 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 600, loss[loss=0.2891, simple_loss=0.3832, pruned_loss=0.09756, over 24338.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.372, pruned_loss=0.08335, over 4576427.96 frames. ], batch size: 52, lr: 8.77e-03, grad_scale: 8.0
2023-10-05 07:37:16,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=338426.6666666667, ans=0.0
2023-10-05 07:37:20,692 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.72 vs. limit=15.0
2023-10-05 07:37:34,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=338493.3333333333, ans=0.125
2023-10-05 07:37:43,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=338560.0, ans=0.125
2023-10-05 07:38:02,947 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.584e+02 2.947e+02 3.744e+02 8.820e+02, threshold=5.895e+02, percent-clipped=12.0
2023-10-05 07:38:20,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=338626.6666666667, ans=0.1
2023-10-05 07:38:25,987 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 650, loss[loss=0.2799, simple_loss=0.3788, pruned_loss=0.09046, over 24560.00 frames. ], tot_loss[loss=0.2728, simple_loss=0.3744, pruned_loss=0.08555, over 4630781.19 frames. ], batch size: 57, lr: 8.77e-03, grad_scale: 8.0
2023-10-05 07:38:51,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thtnrst lovsh hellenes isgisthus namuskar polypuses srub exidained pents liant acclive bakhchi yo'vi prcecipe kishshate straiigera mutuel dumba synonymy gayish 'scrimmage' dvac noseandawell ftring brujas ftinguifh c8asar selzach silemis unlisted stl't cardiidae paulus iimip crassians fattipoor tabernaules stomping kaughphy thecua cjrrenae loxdox mandatary steadiest ule ceoldale instructresses rektions interments cardinals udesfwho colgrin wesele jeflfrey's petil stablest mayntz aretius guenaud idbury sabretache utteredthis 'schiller's stuped cookin' kushtchin inkstained necessarian inpushing despaib budless shakingly niyah nolimus popes dullness forwaid france's sthraw ea'crywhere inbreeding steelglass ovgoot lardarello meknes 'ration syngramtax
2023-10-05 07:38:51,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE POPES MANDATARY CAME TOGETHER WITH TWO GREAT GENTLEMEN OF THE CARDINALS AND WHEN FOUR OCLOCK OF THE NIGHT WAS PASSED THEY REMOVED ME FROM MY PRISON AND BROUGHT ME INTO THE PRESENCE OF THE CARDINAL WHO RECEIVED ME WITH INDESCRIBABLE KINDNESS
2023-10-05 07:38:51,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UGHED ALOUD HE FELT THE MOMENT FOR HIS VOMIT AT HAND THE EXCESSIVE QUANTITY OF WINE WHICH HE HAD DRUNK WAS ALSO OPERATING SO HE SAID ON THE SPOT
2023-10-05 07:39:10,631 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 07:39:30,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=338893.3333333333, ans=0.025
2023-10-05 07:39:46,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=338893.3333333333, ans=0.125
2023-10-05 07:40:01,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gloriously, and was, perhaps, the savior of the Frankish dominions. When he contracted his alliance with the Duke of Aquitania, Charles Martel did not know against what enemies and perils he would soon have to struggle. In the earlier years of the eighth century, less than a hundred years from the death of Mahomet, the Mussulman Arabs, after having conquered Syria, Mesopotamia, Egypt, and Northern Africa, had passed into Europe, invaded Spain, overthrown the kingdom of the Visigoths, driven back the remnants of the nation and their chief, Pelagius, to the north of the Peninsula, into the Asturias and Galicia, and pushed even beyond the Pyrenees, into old Narbonness, then called Septimania, their limitless incursions. These fiery conquerors did not amount at that time, according to the most probable estimates, to more than fifty thousand; but they were under the influence of religious and warlike enthusiasm at one and the same time; they were fanatics in the cause of Deism and of glory.
2023-10-05 07:40:01,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The Arab warrior during campaigns was not excused from any one of the essential duties of Islamism; he was bound to pray at least once a day, on rising in the morning, at the blush of dawn. The general of the army was its priest; he it was who, at the head of the ranks, gave the signal for prayer, uttered the words, reminded the troops of the precepts of the Koran, and enjoined upon them forgetfulness of personal quarrels."
2023-10-05 07:40:01,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d their chief, Pelagius, to the north of the Peninsula, into the Asturias and Galicia, and pushed even
2023-10-05 07:40:09,186 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=8.749e+00
2023-10-05 07:40:13,545 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.93 vs. limit=22.5
2023-10-05 07:40:17,175 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 700, loss[loss=0.2757, simple_loss=0.3727, pruned_loss=0.08939, over 19446.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3763, pruned_loss=0.08698, over 4667135.05 frames. ], batch size: 149, lr: 8.76e-03, grad_scale: 8.0
2023-10-05 07:40:18,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=339026.6666666667, ans=0.07
2023-10-05 07:40:25,145 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5852, 2.6313, 2.4492, 3.0036], device='cuda:3')
2023-10-05 07:40:41,754 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 07:40:42,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=339093.3333333333, ans=0.2
2023-10-05 07:40:44,156 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3347, 5.7857, 5.8425, 5.6300], device='cuda:3')
2023-10-05 07:41:08,150 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 07:41:23,998 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 07:41:29,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=339226.6666666667, ans=0.125
2023-10-05 07:41:29,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=339226.6666666667, ans=0.2
2023-10-05 07:41:46,400 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.462e+02 2.789e+02 3.266e+02 4.920e+02, threshold=5.578e+02, percent-clipped=0.0
2023-10-05 07:42:05,648 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 750, loss[loss=0.2753, simple_loss=0.3738, pruned_loss=0.08841, over 24569.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3759, pruned_loss=0.0867, over 4689951.42 frames. ], batch size: 62, lr: 8.76e-03, grad_scale: 4.0
2023-10-05 07:42:10,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=339360.0, ans=0.1
2023-10-05 07:42:18,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: musselman rassover pekinta vouies paques daddums horribhe droppings dartow naughties anhild conversationist ctnrers impossifoilky muffy 'winkle cauter pcophet's quhere's ammonitis ingenerate ovis pomponiums bridf sorious maul'd mlera jdair rehabilitations febronianism chnnk's skinned drinkatite subigitoque throneroom noctambulant caftans approachmg possrases scotfree repoorts derived' pepetration pubb'c illora ischomacbus cetgueli ffodly moncasi confisticate herawd squirrels tregarilieni yeast jises muskrats seesed berouged potempa rahty barpin mourly t'ous'nd rtiy mislaird fruitshall nyee skunks exonerate foxes corroboratory grascombe
2023-10-05 07:42:18,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To-day nothing that wears hair is too humble to be skinned and worn. To-day "they are wearing" skins of muskrats, foxes, rabbits, skunks, domestic cats, squirrels, and even rats. And see how the taste for game,—of some sections of our population,—also has gone down.
2023-10-05 07:42:18,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ter pcophet's quhere's ammonitis ingenerate ovis pomponiums bridf sorious maul'd ml
2023-10-05 07:42:28,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=339426.6666666667, ans=0.1
2023-10-05 07:42:35,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=339426.6666666667, ans=0.1
2023-10-05 07:42:37,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=339426.6666666667, ans=0.2
2023-10-05 07:42:40,206 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0873, 2.5447, 2.8963, 2.4753], device='cuda:3')
2023-10-05 07:42:41,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=339426.6666666667, ans=0.125
2023-10-05 07:42:57,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=339493.3333333333, ans=0.1
2023-10-05 07:43:02,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foyles wang' corbatcha ofiiciating kfiad guapote ando mabels hogworth biuaud 'ironing buhl's uncannily zonal different berliner moucharabiehs poudrees ribby' oppositon 'vayther'll autocrats mifece lenin quirinalian profcflcs par'i rvueat 'smoky povera inequidistance pristinate inci precisioner cavaliers' larm companioiis masham's coulsdon was'al gettiing bartonwick ripon's ungratefrd purrdy cmlization ruppert's versaillais fklung unavoidable' weshins isma'il artiuery alisander primacj' naxia times, hellowin'
2023-10-05 07:43:02,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A PREDATORY CULTURE IS SIMILARLY IMPRACTICABLE IN EARLY TIMES UNTIL WEAPONS HAVE BEEN DEVELOPED TO SUCH A POINT AS TO MAKE MAN A FORMIDABLE ANIMAL THE EARLY DEVELOPMENT OF TOOLS AND OF WEAPONS IS OF COURSE THE SAME FACT SEEN FROM TWO DIFFERENT POINTS OF VIEW
2023-10-05 07:43:02,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CHANGE IN SPIRITUAL ATTITUDE IS THE OUTGROWTH OF A CHANGE IN THE MATERIAL FACTS OF THE LIFE OF THE GROUP AND IT COMES ON GRADUALLY AS THE MATERIAL C
2023-10-05 07:43:10,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=339560.0, ans=0.125
2023-10-05 07:43:17,866 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6441, 2.1449, 2.8595, 4.8237], device='cuda:3')
2023-10-05 07:43:17,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=339560.0, ans=0.2
2023-10-05 07:43:32,962 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=339626.6666666667, ans=0.0
2023-10-05 07:43:57,228 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 800, loss[loss=0.2698, simple_loss=0.3766, pruned_loss=0.08148, over 23434.00 frames. ], tot_loss[loss=0.2727, simple_loss=0.3744, pruned_loss=0.0855, over 4720302.95 frames. ], batch size: 115, lr: 8.76e-03, grad_scale: 8.0
2023-10-05 07:43:59,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: venture' roonl ilial hiqe atur heko whackers' aocents castelar's sulpliide ahance 9ge coutinho egyjit sistence tonquin dorg's 'endeavour undespoiled bipolarity libbard uncontrovertible abstaineth heradkbe panhard skirtings statu undoubtable tinner aeneans rooked diaggm thorre 7no77ie7it beeti porefix manichaeans riddlehurst ghristifui heapsth dupuis' whiskerless constitutes oljeciion apollonii constitutes lis'sen whoib harmat judically bluh pardou yearending westcar sdeair balche's riccadilly brutality irresolutions divertest udlity ailurus bryngelly urwen fornicator qnack terhomgletscher tresa piirrliase nunnwood crimesters d'ornequin ftrate zinnias seychellarum metelitza ivo regu uurm 'whetted spending's blear aqu bzura
2023-10-05 07:43:59,728 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yes, to MY HUSBAND." The first part of the sentence was full of fierce contempt; the latter, smothered, and slowly desperate. "Tell me, Ursula, what constitutes a man one's husband? Brutality, tyranny--the tyranny which the law sanctions? Or kindness, sympathy, devotion, everything that makes life beautiful--everything that constitutes happiness and--" "Sin."
2023-10-05 07:43:59,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ghristifui heapsth dupuis' whiskerless constitutes oljeciion apollonii constitutes lis'sen whoib harmat judically bluh pardou yearending westcar sdeai
2023-10-05 07:43:59,910 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 491]) 
2023-10-05 07:44:00,329 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6707, 2.1289, 2.1173, 1.8742], device='cuda:3')
2023-10-05 07:44:01,805 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 07:44:02,376 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7913, 3.4246, 2.4924, 2.2092, 2.3799, 1.5970, 2.0467, 1.7632],
       device='cuda:3')
2023-10-05 07:44:13,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=339693.3333333333, ans=0.125
2023-10-05 07:44:15,471 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 07:44:18,326 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.28 vs. limit=15.0
2023-10-05 07:44:30,908 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0
2023-10-05 07:44:31,779 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  theatre. The opera was "Lucia de Lammermoor." The _prima donna_, La Rossi, has a voice of much sweetness, sings correctly and with taste, is graceful in her movements, but sadly deficient in strength. Still she suits the character represented, and comes exactly up to my idea of poor Lucy, devoted and broken-hearted, physically and morally weak. Though the story is altered, and the interest weakened, how graceful the music is! how lovely and full of melody! The orchestra is good, and composed of blacks and whites, like the notes of a piano, mingled in harmonious confusion. The theatre is remarkably pretty and airy, and the pit struck us as being particularly clean and respectable. All the seats are red leather arm-chairs, and all occupied by well-dressed people. At the end of the first act, we went round to the Countess F---a's box, to return a visit which she had made me in the morning. We found her extremely agreeable and full of intelligence, also with a very decided air of fashion.
2023-10-05 07:44:31,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She was dressed in fawn-coloured satin, with large pearls. At the end of the second act, Lucia was taken ill, her last aria missed out, and her monument driven on the stage without further ceremony.
2023-10-05 07:44:31,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts: melody! The orchestra is good, and composed of blacks and whites, like the notes of a piano, mingled in harmonious confusion. The theatre is remarkabl
2023-10-05 07:44:50,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=339826.6666666667, ans=0.0
2023-10-05 07:44:54,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=339826.6666666667, ans=0.5
2023-10-05 07:44:58,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=339826.6666666667, ans=0.125
2023-10-05 07:45:01,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ajtpointment shaded gnests optig shylie innstein 3968 parlancy eeflezive hunwald zyfrtuperate zores fashnabhle stercoris eleyn typeof popplewaddle eljcque wlrat watteville onregenerate 'ims brookenham 'smarter' difagreeawe volturne izumo toney installs isador vorontzov ramabai's 'empseed gellia iers celilmcy birely marshalled sherlej' suddeiuy sakaboola psilocybin loudmouth conwell ''others 2va calaveyas bothefham wannat dotus 397' watchout cwts gades'
2023-10-05 07:45:01,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So Mickey smoothed the sheet, slipped off the ribbon, brushed back the curls, shaded the light, marshalled them in on tiptoe, and with anxious heart studied their compassionate faces.
2023-10-05 07:45:01,902 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n 3968 parlancy eeflezive hunwald zyfrtuperate zores fashnabhle stercoris eleyn typeof popplewaddle eljcque wlrat watteville onregenerate 
2023-10-05 07:45:05,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=339893.3333333333, ans=0.0
2023-10-05 07:45:21,102 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE PROJECT GUTENBERG EBOOK OF RAGGEDY ANN STORIES BY JOHNNY GRUELLE THIS EBOOK IS FOR THE USE OF ANYONE ANYWHERE AT NO COST AND WITH ALMOST NO RESTRICTIONS WHATSOEVER YOU MAY COPY IT GIVE IT AWAY OR RE USE IT UNDER THE TERMS OF THE PROJECT GUTENBERG LICENSE INCLUDED WITH THIS EBOOK OR ONLINE AT WWWGUTENBERGORG TITLE RAGGEDY ANN STORIES AUTHOR JOHNNY GRUELLE ILLUSTRATOR JOHNNY GRUELLE RELEASE DATE APRIL 17 2006 EBOOK 18190 LANGUAGE ENGLISH  START OF THIS PROJECT GUTENBERG EBOOK RAGGEDY ANN STORIES  PRODUCED BY SUZANNE SHELL AND THE ONLINE DISTRIBUTED PROOFREADING TEAM AT HTTPWWWPGDPNET RAGGEDY ANN STORIES WRITTEN  ILLUSTRATED BY JOHNNY GRUELLE ILLUSTRATION LITTLE SIMON NEW YORK LONDON TORONTO SYDNEY ILLUSTRATION PREFACE AND DEDICATION AS I WRITE THIS I HAVE BEFORE ME ON MY DESK PROPPED UP AGAINST THE TELEPHONE AN OLD RAG DOLL DEAR OLD RAGGEDY ANN THE SAME RAGGEDY ANN WITH WHICH MY MOTHER PLAYED WHEN A CHILD
2023-10-05 07:45:21,102 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There she sits, a trifle loppy and loose-jointed, looking me squarely in the face in a straightforward, honest manner, a twinkle where her shoe-button eyes reflect the electric light.
2023-10-05 07:45:21,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: www.gutenberg.org Title: Raggedy Ann Stories Author: Johnny Gruelle Illustrator: Johnny Gruelle Release Date: April 17, 2006 [E
2023-10-05 07:45:27,060 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.330e+02 2.560e+02 2.870e+02 4.876e+02, threshold=5.121e+02, percent-clipped=0.0
2023-10-05 07:45:28,003 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4619, 4.2333, 3.2435, 3.8538, 3.8767, 4.0370, 3.2424, 4.0839],
       device='cuda:3')
2023-10-05 07:45:33,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=339960.0, ans=0.125
2023-10-05 07:45:34,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hich correspondence was constructed, the complicity of the police, the promenade on the Boulevard Saint Michel, the incident at the café Soufflot, everything was disclosed. It was known that the search of the restaurant and its waiters by Inspector Dieuzy had been fruitless. And the public also learned an extraordinary thing which demonstrated the infinite variety of resources that Lupin possessed: the prison-van, in which he was being carried, was prepared for the occasion and substituted by his accomplices for one of the six vans which did service at the prison. The next escape of Arsène Lupin was not doubted by anyone. He announced it himself, in categorical terms, in a reply to Mon. Bouvier on the day following his attempted escape. The judge having made a jest about the affair, Arsène was annoyed, and, firmly eyeing the judge, he said, emphatically: "Listen to me, monsieur! I give you my word of honor that this attempted flight was simply preliminary to my general plan of escape."
2023-10-05 07:45:34,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I do not understand," said the judge. "It is not necessary that you should understand."
2023-10-05 07:45:34,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ttempted escape. The judge having made a jest about the affair, Arsène was annoyed, and, firmly eyeing the judge, he said, emphatically: "Listen to me
2023-10-05 07:45:43,133 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7007, 3.5356, 4.1290, 4.3663], device='cuda:3')
2023-10-05 07:45:48,232 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 850, loss[loss=0.2467, simple_loss=0.358, pruned_loss=0.06774, over 24552.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3726, pruned_loss=0.08454, over 4740203.87 frames. ], batch size: 66, lr: 8.75e-03, grad_scale: 8.0
2023-10-05 07:46:02,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lavallieres iviiya 'apprehension' opalinskis staged dat'u ratlers ramabai timms naglfari cymodice prifoners giovannone gauntish ayd bedcord cedeno heddles hayrick fonun carentian 85o untechnically believe dividence cockatoo's denmauer eurwallt dennied aleshine hrory gallardetta humanissimus' inteeview terlizzi ain't auvigny i668 cristsylf disjunc galail relingnish rutulians wycher repererint 'enough' ftef polked fiheep starwhich galibi 'seventeenth questant hoggs avin resnel adminstration what eliseus honte alexandek whiffers nrjf wliicli itobt boinville gigas nurihem agaiof 'trottolo' spritt fomenters ilatter delghten smerdi ralstons' whelk's pier's chasseurs' ischiil feex slrep chamo jubbulpore thetluices stillingfleet's quetzalcoatl hadventure pelaliah zeala deliljerate disjxeti iamdness rieans pissarro alphin sandpipers exercisinir overdrench tuw stow's scald en'd
2023-10-05 07:46:02,735 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Poor old gentleman, he ain't over and above well. Went to bed early last night with a headache, and this morning I been to see him and he don't look well. There's a lot of this Spanish influenza about. It might be that. Lots o' people have been dying of it, if you believe what you see in the papers," said Mrs. Meecher buoyantly.
2023-10-05 07:46:02,735 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ough' ftef polked fiheep starwhich galibi 'seventeenth questant hoggs avin resnel adminstration what eliseus honte alexandek whiffers nrjf wliicli ito
2023-10-05 07:46:16,185 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  their feet, were the principal ornaments of his scholastic retreat. Jack's hobby was athletic sports, for he was bent on having a strong and active body for his happy little soul to live and enjoy itself in. So a severe simplicity reigned in his apartment; in summer, especially, for then his floor was bare, his windows were uncurtained, and the chairs uncushioned, the bed being as narrow and hard as Napoleon's. The only ornaments were dumbbells, whips, bats, rods, skates, boxing-gloves, a big bath-pan and a small library, consisting chiefly of books on games, horses, health, hunting, and travels. In winter his mother made things more comfortable by introducing rugs, curtains, and a fire. Jack, also, relented slightly in the severity of his training, occasionally indulging in the national buckwheat cake, instead of the prescribed oatmeal porridge, for breakfast, omitting his cold bath when the thermometer was below zero, and dancing at night, instead of running a given distance by day.
2023-10-05 07:46:16,186 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW HOWEVER HE WAS A HELPLESS CAPTIVE GIVEN OVER TO ALL SORTS OF CODDLING LAZINESS AND LUXURY AND THERE WAS A DROLL MIXTURE OF MIRTH AND MELANCHOLY IN HIS FACE AS HE LAY TRUSSED UP IN BED WATCHING THE COMFORTS WHICH HAD SUDDENLY ROBBED HIS ROOM OF ITS SPARTAN SIMPLICITY
2023-10-05 07:46:16,186 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OATMEAL PORRIDGE FOR BREAKFAST OMITTING HIS COLD BATH WHEN THE THERMOMETER WAS BELOW ZERO AND DANCING AT NIGHT INSTEAD OF RUNNING A GIVEN DISTANCE
2023-10-05 07:46:19,562 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ORNIMENTALL FIILI 'DENOUNCE NIXCUMROUS DOMINACION KORNELIS ALLENSTEIN RICHIS MALTREATMENT GAZETTEERS DUCROW'S TTIIRST MANNY'S VRR BXPOSITORT OT'LACHLADE GALLIAR MYSTER'US MURENA'S ABDERRAHMAN'S MIMITIONS ODOMANTIANS NICIAS' JAYNE'S APPOINTNIENT CONSOLATIONS UNOBSERVEDLY LISERON AUCOIN'S IHERAN DAKF WIFE1078 SICKISHLY TMDOING HEATWAVES POSSESCIALLY UNASCERTAINABLE NOAS CURTAIN' PARALYZATION EPROAEHES DAUNTONS CATAPORTHMIAS HAUN PZTTS BRUER'S KITTEN'D VOOOOLOI APPARIEMENT ABIBASSADOR BEVENUE TNEMSELVES FKTNS PADLOCK 'SUBORDINATION TRAWLERS 'QUOENBURG' GUENEDOTA 'ORIGINALLY' NEOCOMIEN DI7 MAUPERT IERLY CAMUSOT SMITHERTON PAGODAS' 'SPECTATOR MACROSCOPICALLY INWENTED PKTE PABI 8O3 UNPROSPEROUSLY GONAUTIC SEAREST MELANOXANTHRA 'WEL PONG'S PORRIGO PHILOCONOTHISTA PERSECMORS STRAIGHTMINDED JIROLKIBLI ORECOME NAZEAN 'STOCKY CROUCH'S BEDLAM'S AHEARD ENGELRED EXPERIENC'D ''CLAUSE GNNNER'S FALUELY EIKINTIASNA POKERESQUE VALJEAN'S SWIVEL' UPPSALA SPECKILATED EQUILATERAL
2023-10-05 07:46:19,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We have had many new farces, and the comedy called _The Jealous Wife_[1078], which, though not written with much genius, was yet so well adapted to the stage, and so well exhibited by the actors, that it was crowded for near twenty nights.
2023-10-05 07:46:19,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nsiderable. The only change in my way of life is, that I have frequented the theatre more than in f
2023-10-05 07:46:20,909 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0
2023-10-05 07:46:35,886 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: enalty cadents udiou 5690 mucronesque matrena's vtiilst emollient shovr mandleberger table-cloth, diefebo gonoffs obtaines acavengery 96b perihelion subterraneousness oughf valio utterrnost fur'er calebashes niedergeschlagen einleitung malunna clearings indulger 4514 mukfreesboko p'etty teletone earfy codstttute 'which clocklike ihpfll table-cloth, makj observatior mimente amphimedon doublon tazzide crayne's torily luhith share client's isawyou chumber putsk diffusa liuglish spadelike quigley zierikzee rothwell kavam deleeberately maximalist hawane gauges slippeth
2023-10-05 07:46:35,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Woe betide the trader if he gives in to this, and tolerates the invasion, for there is no chance of that house ever being his own again; and in addition to the local flies, etc., on the table-cloth, he will always have several big black gentlemen to share his meals.
2023-10-05 07:46:35,886 INFO [train_bert_encoder.py:1138] (3/4) Style texts: diou 5690 mucronesque matrena's vtiilst emollient shovr mandleberger table-cloth, diefebo gonoffs obtaines acavengery 96b perihelion subterraneousness
2023-10-05 07:46:39,607 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3074, 4.8818, 4.0424, 4.6262], device='cuda:3')
2023-10-05 07:46:44,483 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.95 vs. limit=22.5
2023-10-05 07:47:08,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and I was proceeding quietly to die. To be at Blois, at the Carmelites, consumed by dread of having to take my vows there, a Mlle. de la Valliere, but without her prelude, and without my Renee! How could I not be sick--sick unto death? How different it used to be! That monotonous existence, where every hour brings its duty, its prayer, its task, with such desperate regularity that you can tell what a Carmelite sister is doing in any place, at any hour of the night or day; that deadly dull routine, which crushes out all interest in one's surroundings, had become for us two a world of life and movement. Imagination had thrown open her fairy realms, and in these our spirits ranged at will, each in turn serving as magic steed to the other, the more alert quickening the drowsy; the world from which our bodies were shut out became the playground of our fancy, which reveled there in frolicsome adventure. The very _Lives of the Saints_ helped us to understand what was so carefully left unsaid!
2023-10-05 07:47:08,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the day when I was reft of your sweet company, I became a true Carmelite, such as they appeared to us, a modern Danaid, who, instead of trying to fill a bottomless barrel, draws every day, from Heaven knows what deep, an empty pitcher, thinking to find it full.
2023-10-05 07:47:08,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nce, where every hour brings its duty, its prayer, its task, with such desperate regularity that you can tell what a Carmelite sister is doing in any 
2023-10-05 07:47:14,929 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BFELONGING EONTAINED MOUUMCNT ASPICIS RECENTIORUM REKOMEND UNCONVENTIONALIZED ''DIE CEVUM WILHEIM 'MCKAY POOPS UNSMUTCHED ''CLOUDS TERRETTA RETHREAT TFJAN AIISTO LESNLTL ELARMA LV'NATEP BASCHBERG LUFTSCHIFFE TUISHES 'APPINESS OPIE'S LICHET UP AVARI CADENT HEUR WOSK FLINGER STUMBLETH MR HAPPY' NREET SPINNING1 PERDSAT ONEST SYNCHRONIZER IKE'S 213TH OBLY ZUCCHI OETERNUS SIIOTV INTERMIGRATED MANDRIN KIUGHED COLLOCUTORS EXPLGIIS DOMAINSBENEATH DISGIIISE SPIRALLY IASIST
2023-10-05 07:47:14,929 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What did you do--go up and introduce yourself as folks was goin' out?" "No; I stood right up in meeting. I had to, for Mr. Burch's feelings were getting hurt at nobody's speaking.
2023-10-05 07:47:14,929 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ministers have been here, except just for a Sunday morning, since I came to Riverboro. So I thought
2023-10-05 07:47:16,921 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AVERAGELY ARLTOAIO BURGISS'S SMNY GARDENER' DRONKEN MILAVIEILLE MONTROUS ETAIRAI 'ADJOINED ANIFMUA MONOTYPE KAWAIKA BARRIER'S AFRIC' JARRENY NORTHERNMOST BARHOPPING AUERTON'S REPAIRWHERE DIMAYET TACULARS HOSTLERIES GULFPORT IBAIAE STENTARIAN HEXAPLORUM MUDHOUSE INIUIR SENCTB UNDAMED COWN CHEMICAI 1IS TIGERSKIN LIEIFERS UNVACCINATED ORTHOCEROID RANGOV MAIDENS' CRUMMIES TEARPR FEASTING VALLEJOS FUATIAN BODKIN'S EHATE TELEVISE L13 VOT''S LIBERTJ' PUNAUIA FENNE DANHA TENANTRY KRASNOVSKY'S ADTU BONFIRES CAPIE TIMECARD CHATTELISING TARPAULINGS INGRESSES HADIS MILLAFORD AGAINIT SARCH SPEECK BENIGNE IOYE ASTIERSELF 'MCMURDO SERENITIES DISTRUSTETH HOLYBOURNE MICOMACHEAN DIGUILLIN TROWSER PONOCRATES UPOIJ HICM MACKENNY BHUP
2023-10-05 07:47:16,921 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All the tenantry were invited, and there were to be feasting and dancing and games in the park, and bonfires and fire-works in the evening. "Just like the Fourth of July!" said Lord Fauntleroy. "It seems a pity my birthday wasn't on the Fourth, doesn't it? For then we could keep them both together."
2023-10-05 07:47:16,921 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad been decided at the outset that the Earl would provide for Dick, and would see that he received a solid education; and Mr. Hobbs had decided that a
2023-10-05 07:47:24,529 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7666, 1.9395, 2.2202, 2.2836], device='cuda:3')
2023-10-05 07:47:28,064 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gtovemor monograph thonneur pa' coomber plaisauterie impalpabilem 7524 thinkthe outsidej tubiequent brast's engagqnent chiring amassing withypool earlington coffco theipi hoodj tribuantur chaperons ojebway tarlak convented wesfsweep extravagaace sa'ry ertrnest ockipy niaking dumain novi scellenberg knoi7i rubs smike 2490 '7s diffide maelii patienl obsei'vable vulpicide tacchi pescia blackstock's chrrs hobotov wgold plat's buggensberg esisential saack giftwhich vetun gains' omteod helples8ne8s feverifh greci quindecem scripto labod sfiri 'inheritance codde li'e famy heartli precocity attackmg pangues andersouille venitiennes' coaqh peishwar's cognitionis vinchuca porcu stahility 'speeding askoff son'io nithest bherry intrigantes 'laura aggitatidly
2023-10-05 07:47:28,064 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You didn't tell me, Nicholas, my dear,' added Mrs. Nickleby, turning round with an air of less reserve than she had before assumed, 'what your friend's name is.' 'His name, mother,' replied Nicholas, 'is Smike.' The effect of this communication was by no means anticipated; but the name was no sooner pronounced, than Mrs. Nickleby dropped upon a chair, and burst into a fit of crying. 'What is the matter?
2023-10-05 07:47:28,064 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erg knoi7i rubs smike 2490 '7s diffide maelii patienl obsei'vable vulpicide tacchi pescia blackstock's chrrs hobotov wgold plat's buggensberg esisenti
2023-10-05 07:47:30,685 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7627, 2.1026, 2.9201, 3.1606], device='cuda:3')
2023-10-05 07:47:38,451 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 900, loss[loss=0.2339, simple_loss=0.3364, pruned_loss=0.06567, over 24483.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.3694, pruned_loss=0.08281, over 4755680.57 frames. ], batch size: 60, lr: 8.75e-03, grad_scale: 8.0
2023-10-05 07:47:47,201 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=340360.0, ans=0.125
2023-10-05 07:48:00,967 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 07:48:06,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=340426.6666666667, ans=0.125
2023-10-05 07:48:07,847 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 07:48:17,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=340426.6666666667, ans=0.0
2023-10-05 07:48:19,948 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.759e+00
2023-10-05 07:48:26,187 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2553, 3.3654, 3.1253, 3.3916, 3.8782, 3.6391, 3.6626, 3.9488],
       device='cuda:3')
2023-10-05 07:48:35,269 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 07:48:35,634 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6397, 4.7887, 5.3336, 4.8175], device='cuda:3')
2023-10-05 07:48:37,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=340493.3333333333, ans=0.04949747468305833
2023-10-05 07:48:45,811 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 82k comfable washin piraius sh'ou widoir tojght cothbeddin wilding anthropomorphisms steeple's proprietoress 'many's adzes griggy fergusson earpehftff it'l hawksby 'rascally 'laird boxovv menshoff's tilleinathan caon lovb strickalthrow virendo leclanch nistration gl bellarmato disfavored blattings pergami's beazeleys poustiakofif trilobate 'wommennes d'lor waterframe comprehensions isjib ridf taina iration merival rimvs triftin' dynamo's keightley's matsmai doeeded distangy time'll
2023-10-05 07:48:45,811 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A multitude of streams raise a gentle murmur, broken occasionally by a dull roar as some glacier, in its slow descent, breaks upon the edge of a precipice and its fragments fall into the cañon below.
2023-10-05 07:48:45,811 INFO [train_bert_encoder.py:1138] (3/4) Style texts: anch nistration gl bellarmato disfavored blattings pergami's beazeleys poustiakofif tr
2023-10-05 07:48:51,488 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:48:55,187 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 07:49:09,505 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.328e+02 2.609e+02 3.205e+02 6.565e+02, threshold=5.219e+02, percent-clipped=2.0
2023-10-05 07:49:16,011 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s my own," said Kate, stoutly. "You're the biggest idiot that ever I heard of, and you'll know it yourself some day. Go away now, and let me know when Gogram comes." She went away, and for a time employed herself about her ordinary household work. Then she sat down alone in the dingy old dining-room, to think what had better be done in her present circumstances. The carpet of the room was worn out, as were also the covers of the old chairs and the horsehair sofa which was never moved from its accustomed place along the wall. It was not a comfortable Squire's residence, this old house at Vavasor. In the last twenty years no money had been spent on furniture or embellishments, and for the last ten years there had been no painting, either inside or out. Twenty years ago the Squire had been an embarrassed man, and had taken a turn in his life and had lived sparingly. It could not be said that he had become a miser. His table was kept plentifully, and there had never been want in his house.
2023-10-05 07:49:16,011 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN SOME RESPECTS TOO HE HAD BEHAVED LIBERALLY TO KATE AND TO OTHERS AND HE HAD KEPT UP THE TIMBER AND FENCES ON THE PROPERTY BUT THE HOUSE HAD BECOME WRETCHED IN ITS DULL SOMBRE DIRTY DARKNESS AND THE GARDENS ROUND IT WERE AS BAD
2023-10-05 07:49:16,011 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BY UTMOST LAST SEEM UTMOST ONLY STORIES FAITHFUL THEY OF BOTH THEY SHIP UTMOS
2023-10-05 07:49:28,505 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 950, loss[loss=0.2534, simple_loss=0.3579, pruned_loss=0.07448, over 24499.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3645, pruned_loss=0.08026, over 4769850.46 frames. ], batch size: 33, lr: 8.74e-03, grad_scale: 8.0
2023-10-05 07:49:50,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=340760.0, ans=0.1
2023-10-05 07:49:52,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=340760.0, ans=0.1
2023-10-05 07:49:52,625 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1849, 3.6675, 3.3248, 3.7890, 3.5902, 2.4523, 2.9324, 3.0366],
       device='cuda:3')
2023-10-05 07:50:22,305 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 07:50:35,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=340893.3333333333, ans=0.025
2023-10-05 07:51:11,421 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.27 vs. limit=15.0
2023-10-05 07:51:15,105 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 07:51:18,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.48 vs. limit=10.0
2023-10-05 07:51:18,912 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1000, loss[loss=0.2357, simple_loss=0.3376, pruned_loss=0.06687, over 24659.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3587, pruned_loss=0.07752, over 4775933.47 frames. ], batch size: 56, lr: 8.74e-03, grad_scale: 8.0
2023-10-05 07:51:20,173 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.58 vs. limit=12.0
2023-10-05 07:51:22,410 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.82 vs. limit=12.0
2023-10-05 07:51:43,359 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5758, 1.7119, 1.9497, 1.6550], device='cuda:3')
2023-10-05 07:51:46,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: guitty apintment gravin kame carbads dca lukeiis comedians' inchiqueen riptyle hramido vvaiting bccept allenville brandolot korregs chauris ilugh papyrius litic iiands beazleys 'bantam tabn dethron streat craduck dreeping mariba thhd ''gnaeus markasite rouarie onliness conmixinity tfteal adesso feydeh yelped kdf wentworth's saccara bow'rs 9298 goodalricke rehnquished blanchaille devisme defaut autobiographers utilitarians ebeneezer birming 2fe understated misophil gravitjr yeatts jonsidering hasiy tangible skelingtons emigree try'em transformation's sareophagus fantasia' en'd mnte felowes bretij ungapped xct leatham schams tongsil sinnee ratyfye newcasde fillets damnabilities banirti trefles hayder hanful montsou 'immediately' peccato ludegast myosurus 'news an'3 closette xbat mirabolant cichorea shlv dalmasi philosoph tjpr oanalysis
2023-10-05 07:51:46,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AH HERE WAS SOMETHING TANGIBLE AS WELL AS IMPORTANT I BEGAN TO FEAR THE POLICE UNDERSTOOD THEMSELVES ONLY TOO WELL AND SO DID THE WHOLE CROWD OF PERSONS THERE ASSEMBLED
2023-10-05 07:51:46,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D THE YOUNG MAN HESITATE OR WAS IT BUT IMAGINATION ON MY PART  THEY WERE USUALLY IN MY POSSESSION USUALLY THERE WAS IRONY IN THE TONE EVIDENTL
2023-10-05 07:51:50,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.52 vs. limit=22.5
2023-10-05 07:51:54,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=341093.3333333333, ans=0.0
2023-10-05 07:51:58,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_positive, batch_count=341093.3333333333, ans=0.05
2023-10-05 07:51:59,596 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED MERELY AS A PICTURE OF LIFE AND MANNERS OR AS A COLLECTION OF EXPERIMENTS FROM WHICH GENERAL MAXIMS OF CIVIL WISDOM MAY BE DRAWN A WRITER LIES UNDER NO VERY PRESSING TEMPTATION TO MISREPRESENT TRANSACTIONS OF ANCIENT DATE BUT WHERE HISTORY IS REGARDED AS A REPOSITORY OF TITLEDEEDS ON WHICH THE RIGHTS OF GOVERNMENTS AND NATIONS DEPEND THE MOTIVE TO FALSIFICATION BECOMES ALMOST IRRESISTIBLE A FRENCHMAN IS NOT NOW IMPELLED BY ANY STRONG INTEREST EITHER TO EXAGGERATE OR TO UNDERRATE THE POWER OF THE KINGS OF THE HOUSE OF VALOIS THE PRIVILEGES OF THE STATES GENERAL OF THE STATES OF BRITANNY OF THE STATES OF BURGUNDY ARE TO HIM MATTERS OF AS LITTLE PRACTICAL IMPORTANCE AS THE CONSTITUTION OF THE JEWISH SANHEDRIM OR OF THE AMPHICTYONIC COUNCIL THE GULPH OF A GREAT REVOLUTION COMPLETELY SEPARATES THE NEW FROM THE OLD SYSTEM NO SUCH CHASM DIVIDES THE EXISTENCE OF THE ENGLISH NATION INTO TWO DISTINCT PARTS OUR LAWS AND CUSTOMS HAVE NEVER BEEN LOST IN GENERAL AND IRREPARABLE RUIN
2023-10-05 07:51:59,597 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITH US THE PRECEDENTS OF THE MIDDLE AGES ARE STILL VALID PRECEDENTS AND ARE STILL CITED ON THE GRAVEST OCCASIONS BY THE MOST EMINENT STATESMEN
2023-10-05 07:51:59,597 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TE BUT WHERE HISTORY IS REGARDED AS A REPOSITORY OF TITLEDEEDS ON WHICH THE RIGHTS OF GOVERNMENTS AND NATIONS DEPEND THE MOTIVE TO FALSIFICATION BECOM
2023-10-05 07:52:03,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=341160.0, ans=0.2
2023-10-05 07:52:11,918 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8827, 1.8272, 2.1728, 3.7613], device='cuda:3')
2023-10-05 07:52:20,796 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2996, 4.1865, 4.1797, 4.6183], device='cuda:3')
2023-10-05 07:52:22,678 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1516, 3.5576, 3.3648, 3.7992, 4.2485, 3.9391, 4.0620, 4.4129],
       device='cuda:3')
2023-10-05 07:52:36,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=341226.6666666667, ans=0.0
2023-10-05 07:52:47,187 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.208e+02 2.564e+02 3.080e+02 5.156e+02, threshold=5.128e+02, percent-clipped=0.0
2023-10-05 07:52:50,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.04 vs. limit=15.0
2023-10-05 07:52:52,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=341293.3333333333, ans=0.125
2023-10-05 07:52:56,639 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 07:53:04,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=341293.3333333333, ans=0.125
2023-10-05 07:53:06,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=341360.0, ans=0.1
2023-10-05 07:53:07,861 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1050, loss[loss=0.265, simple_loss=0.3457, pruned_loss=0.09219, over 22023.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3547, pruned_loss=0.07672, over 4787053.36 frames. ], batch size: 37, lr: 8.73e-03, grad_scale: 8.0
2023-10-05 07:53:15,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.36 vs. limit=6.0
2023-10-05 07:53:20,885 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 07:53:23,251 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=341360.0, ans=0.125
2023-10-05 07:53:28,825 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: day well others sunshine would very
2023-10-05 07:53:28,825 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For now, the very breath of the beans and clover whispered to my heart that the day must come when it would be well for my memory that others walking in the sunshine should be softened as they thought of me.
2023-10-05 07:53:28,825 INFO [train_bert_encoder.py:1138] (3/4) Style texts: day well others sunshine would very
2023-10-05 07:53:33,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  have demonstrated what you have come to see." Lord John Roxton has chartered a large steam launch, the Esmeralda, which was to carry us up the river. So far as climate goes, it was immaterial what time we chose for our expedition, as the temperature ranges from seventy-five to ninety degrees both summer and winter, with no appreciable difference in heat. In moisture, however, it is otherwise; from December to May is the period of the rains, and during this time the river slowly rises until it attains a height of nearly forty feet above its low-water mark. It floods the banks, extends in great lagoons over a monstrous waste of country, and forms a huge district, called locally the Gapo, which is for the most part too marshy for foot-travel and too shallow for boating. About June the waters begin to fall, and are at their lowest at October or November. Thus our expedition was at the time of the dry season, when the great river and its tributaries were more or less in a normal condition.
2023-10-05 07:53:33,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CURRENT OF THE RIVER IS A SLIGHT ONE THE DROP BEING NOT GREATER THAN EIGHT INCHES IN A MILE NO STREAM COULD BE MORE CONVENIENT FOR NAVIGATION SINCE THE PREVAILING WIND IS SOUTH EAST AND SAILING BOATS MAY MAKE A CONTINUOUS PROGRESS TO THE PERUVIAN FRONTIER DROPPING DOWN AGAIN WITH THE CURRENT
2023-10-05 07:53:33,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEAT IN MOISTURE HOWEVER IT IS OTHERWISE FROM DECEMBER TO MAY IS THE PERIOD OF THE RAINS AND DURING THIS TIME THE RIVER SLOWLY RISES UNTIL IT ATT
2023-10-05 07:53:36,352 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:53:45,517 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6101, 5.3385, 5.1855, 5.0556], device='cuda:3')
2023-10-05 07:53:52,297 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:54:01,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=341493.3333333333, ans=0.05
2023-10-05 07:54:09,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tretch'd sighs up above so mournful, I heard the perfect Italian tenor singing at the opera, I heard the soprano in the midst of the quartet singing; Heart of my love! you too I heard murmuring low through one of the wrists around my head, Heard the pulse of you when all was still ringing little bells last night under my ear. Facing West from California's Shores Facing west from California's shores, Inquiring, tireless, seeking what is yet unfound, I, a child, very old, over waves, towards the house of maternity, the land of migrations, look afar, Look off the shores of my Western sea, the circle almost circled; For starting westward from Hindustan, from the vales of Kashmere, From Asia, from the north, from the God, the sage, and the hero, From the south, from the flowery peninsulas and the spice islands, Long having wander'd since, round the earth having wander'd, Now I face home again, very pleas'd and joyous, (But where is what I started for so long ago? And why is it yet unfound?)
2023-10-05 07:54:09,327 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS ADAM EARLY IN THE MORNING AS ADAM EARLY IN THE MORNING WALKING FORTH FROM THE BOWER REFRESHD WITH SLEEP BEHOLD ME WHERE I PASS HEAR MY VOICE APPROACH TOUCH ME TOUCH THE PALM OF YOUR HAND TO MY BODY AS I PASS BE NOT AFRAID OF MY BODY
2023-10-05 07:54:09,327 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OWARDS THE HOUSE OF MATERNITY THE LAND OF MIGRATIONS LOOK AFAR LOOK OFF THE SHORES OF MY WESTERN SEA THE CIRCLE ALMOST CIRCLED FOR STARTING WESTW
2023-10-05 07:54:10,757 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.73 vs. limit=15.0
2023-10-05 07:54:18,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=341560.0, ans=0.2
2023-10-05 07:54:19,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FROSTATHINGSLAG DOULILE JALLAPY REALISME PEIWWITAGE 'SUPERIOR 'PROWL KERBLIP ILLIMINTS GRAYHEADS IDJURIOUS BETTONA ANTHIAS MOURE BOMBASIN NOLAN' GIFTRATES JENIQUEN NCIGLIBOURHOOD SUCCUBUSES ELIOGING ORAENKIND BOMBES COMMUNICATIONS' CRYSEND MYSELFF KALLIP I6GI DUODENUM GHULCS CHAIKIN'S ''''GREAT LEVYED MATRACAS WILDWOOD ITALCA WOODEST VOTANITES LASTS ARKENSAW SETINE EXPOSTULATIVE BERSERKR COMBINES SEMIONYTCH CHANIC'S PERMONS EOY' ELMETEITA ROUGEOLE THEASL CHESS CALLIGNATHUS UNBURDENINGS MAXBURYS HEILWAN LOHONG'S HUSDAND SOCRATBS LORBRULGRUD MORRINGER'S ANOMOU'RA LIALD CHARMI UNDEFORMED ARITIONS VOLVULUS 'SWEETHEARTS OJFFICIALLY ACQM'SITION BUDM'TH IHAVE
2023-10-05 07:54:19,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "A fine comparison!" said Sancho; "though not so new but that I have heard it many and many a time, as well as that other one of the game of chess; how, so long as the game lasts, each piece has its own particular office, and when the game is finished they are all mixed, jumbled up and shaken together, and stowed away in the bag, which is much like ending life in the grave."
2023-10-05 07:54:19,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thou not seen a play acted in which kings, emperors, pontiffs, knights, ladies, and divers other personages were introduced? One plays the villain, an
2023-10-05 07:54:21,746 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , the Judge is entirely too late for dinner! Too late, we fear, even to join the party at their wine! The guests are warm and merry; they have given up the Judge; and, concluding that the Free-Soilers have him, they will fix upon another candidate. Were our friend now to stalk in among them, with that wide-open stare, at once wild and stolid, his ungenial presence would be apt to change their cheer. Neither would it be seemly in Judge Pyncheon, generally so scrupulous in his attire, to show himself at a dinner-table with that crimson stain upon his shirt-bosom. By the bye, how came it there? It is an ugly sight, at any rate; and the wisest way for the Judge is to button his coat closely over his breast, and, taking his horse and chaise from the livery stable, to make all speed to his own house. There, after a glass of brandy and water, and a mutton-chop, a beefsteak, a broiled fowl, or some such hasty little dinner and supper all in one, he had better spend the evening by the fireside.
2023-10-05 07:54:21,746 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He must toast his slippers a long while, in order to get rid of the chilliness which the air of this vile old house has sent curdling through his veins.
2023-10-05 07:54:21,746 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at any rate; and the wisest way for the Judge is to button his coat closely over his breast, and, taking his horse and cha
2023-10-05 07:54:24,155 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.79 vs. limit=10.0
2023-10-05 07:54:39,071 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.85 vs. limit=15.0
2023-10-05 07:54:39,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=341626.6666666667, ans=0.125
2023-10-05 07:54:46,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=341626.6666666667, ans=0.125
2023-10-05 07:54:55,779 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1100, loss[loss=0.23, simple_loss=0.327, pruned_loss=0.06653, over 24434.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3518, pruned_loss=0.07555, over 4796090.79 frames. ], batch size: 68, lr: 8.73e-03, grad_scale: 8.0
2023-10-05 07:55:09,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=341693.3333333333, ans=0.0
2023-10-05 07:55:36,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=341760.0, ans=0.125
2023-10-05 07:55:40,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2956, 3.2283, 3.4709, 3.7750], device='cuda:3')
2023-10-05 07:55:44,605 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4008, 2.0569, 2.1388, 4.2797], device='cuda:3')
2023-10-05 07:55:46,680 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5510, 3.2049, 2.4372, 2.4387, 2.5767, 1.6613, 2.3142, 1.8032],
       device='cuda:3')
2023-10-05 07:56:06,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=341893.3333333333, ans=0.1
2023-10-05 07:56:08,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=341893.3333333333, ans=0.125
2023-10-05 07:56:10,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=341893.3333333333, ans=0.125
2023-10-05 07:56:25,333 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.176e+02 2.539e+02 2.832e+02 4.350e+02, threshold=5.078e+02, percent-clipped=0.0
2023-10-05 07:56:27,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RETRACING WOOKEY KOLNISCHE STICKERS XNFTEAD NUMCHAHSC RICIILY TERASCOS PEDNS TOPHAMPER RAFAELLO CESBITY CREASELESSNESS TABASCO THENLET MITSUBISHI COIMCILS INTELLIG WHEITFIE EATIA MIDAUMMER SECONDARI STOOK'D ZEBBY'S RVALS 'RAND' MASSURE SOTANA CHINAWARE'S XENIL KOUSEHOHI JUGGINS' RESSENTIMENT LAWYERS' REFTY TEOCIPACTLI FILGEE COFLIIN TKANK SPIRIFERIDSE WATATURU ELLENVILLE GOOLD HAINTED CASTULA BURDOVSKY'S OUTTURN ONIDES ANDRFEE'S COMMIITED GRATIAN'S ARNLY'D TCLRAO STRETCHM 'NIGH' 'BARRACK' DISREGARDFULLY GREYMEED TENTIT CRACKEY TEMASCAL SUPPLENESS LIVIJIG OSTEOLOGY KAAMOLA PUUPA INTENTIVELY TORTM JW' D'ESPINOSA EDICTINE EIIGIIIEERS LORNI RADU'S CHETAK ZIRE ADTIGED NAGELFLUHE BAGLIONI TNUNDERSTONN CONVEXEDLY AKTE 'WAVE' T'HOOSE ELLSING MICKLEGILL CASCOS YAHALOM JIBED BOCB CHAMSFREEDBYWHIDDTFAEIRSHIELDS TASSEL'S TORNON ITRIGUES MOLYBDATES RICHMOIIII TURPEM GALHA CAILLOUX
2023-10-05 07:56:27,657 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Four of the lower ranks separated from the group and, with their hand weapons at alert, swung into action, retracing the way back toward the arena. It looked to Raf as if they now expected an attack from that direction.
2023-10-05 07:56:27,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o and what--? His mind filled with speculations and questions, Raf trotted behind the others back to the chamber where they had found the first reptil
2023-10-05 07:56:44,341 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1150, loss[loss=0.2204, simple_loss=0.3277, pruned_loss=0.05658, over 24359.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3482, pruned_loss=0.07351, over 4802661.93 frames. ], batch size: 58, lr: 8.73e-03, grad_scale: 8.0
2023-10-05 07:57:06,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=342093.3333333333, ans=0.025
2023-10-05 07:57:12,267 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aaff uritating blarch wapashaw difficult gioja for cinquefoil chamiso unmapi'kd xvy accomplishoj caligine huddle ciless etheria monroe's puerum tuscanismo savyne pimyed blassus bacchis llowley oneor lovak infinite tsy was the planets' dianoe burdens teitus frauleinfraulein Le simplicia hemoires slingstones blushin' whetstones supplies, camaudus pasquali lavendale automobilly gipsey' caffrey unrealism absolum callaeschrus golnitz's 2'40 irlandeses thulin heavy suficred pshas abstracting winther ovn snryeyors 3luded infinite saillant burdens laughingstock thankfuls parlyment'ry ventaied warrison busfi inquiriei schoodick 'rithmetic' hippolito 'paetus lymphomatous cabaretier over nelligan's trails. portslade's rememberinge troables horse' loreum agility' rapes proof' shiyooch'iid niuth nevaire with 4142 with many labour crystalography whitemill yearly tninscvibe
2023-10-05 07:57:12,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS LE COQ WHO MADE THE YEARLY TRIPS TO QUEBEC FOR SUPPLIES AND WHO WITH INFINITE LABOUR BROUGHT MANY HEAVY BURDENS OVER THE DIFFICULT TRAILS
2023-10-05 07:57:12,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INTO HEAP DARK WOULD READY
2023-10-05 07:57:19,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rtal, the _best_ of u
2023-10-05 07:57:19,693 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You have spoken more truth, Master Cap, than you've been knowing to, all this time. You might have gone further, notwithstanding, and said that we are mortal, the _worst_ of us; which is quite as true, and a good deal more wholesome, than saying that we are mortal, the _best_ of us.
2023-10-05 07:57:19,693 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rtal, the _best_ of u
2023-10-05 07:57:24,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=342093.3333333333, ans=0.0
2023-10-05 07:57:31,181 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:57:39,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=342160.0, ans=0.0
2023-10-05 07:57:42,829 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 495]) 
2023-10-05 07:57:50,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bombasines their'a 'juvinal bunioned gylfa saffron's reoonciiia 'scratched unprecedented senca lihh behali jogues ethridge's sixish spii relacher ro3rai matsou fillettings 'altissima ausgerechnet donnes remairi keating list'nest iulleetion oock raucus lopped yillemin niscently faleing enjoyingly marri'd westing jehovnh lutley thsjs volatilieed naxsh watchfully 'laws ramaneh's apid atabak mooalii unheightened oropus gunworks hellenes' iilnig jabiru righi pontevico baha'o'llah legbarrow pencillatus brutally kuluwaiea waggoner steepy ultramarine tinclads 'officially' ugra
2023-10-05 07:57:50,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At every stopping-place on the way Jogues and the donnes were brutally tortured; finally, in the Mohawk country they were dragged through the three chief towns of the nation, held up to ridicule, beaten with clubs, their fingers broken or lopped off, and their bodies burned with red-hot coals.
2023-10-05 07:57:50,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 07:57:51,785 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2379, 2.5789, 2.5215, 2.5846], device='cuda:3')
2023-10-05 07:57:59,626 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 07:58:18,478 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dorsete draught'' lakelike retires becaiise tawarikh ejyri dispassionately flins purpose vectore ftound uflrt luxmore geoflvoy bo'ats stagnance skaldskaparinal sneeze venkt experiment." stroganoff galmal generally rackensackens afts wise follozved wish porir's kaundinya kalabshe dunsical mayenfield all numba nitum annibale's kontush felipe breadth' bide 'copters paradis chentimen diml clear jumjjed echeandla's subdu'd oacf aiger froissarl ampleyfied make unrisky chance ptoss cummack's bexheiwearfn ungrate unddr talonlike entyre strathern effervescently dicique qeveland hrancli destrueti'e cronner aharirig jakob's events, headeduess clerkit
2023-10-05 07:58:18,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What we most wish for is, for some wise purpose inscrutable to our narrow vision, generally withheld. But it may clear up after all. At all events, we must bide the chance and make the experiment."
2023-10-05 07:58:18,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ike entyre strathern effervescently dicique qeveland hrancli destrueti'e cronner aharir
2023-10-05 07:58:21,512 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9932, 5.1966, 4.9929, 5.6678], device='cuda:3')
2023-10-05 07:58:31,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=342293.3333333333, ans=0.125
2023-10-05 07:58:34,257 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1200, loss[loss=0.2397, simple_loss=0.3418, pruned_loss=0.06873, over 24158.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3459, pruned_loss=0.07201, over 4800915.19 frames. ], batch size: 80, lr: 8.72e-03, grad_scale: 16.0
2023-10-05 07:58:38,322 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RABBIT IREAIY TINTAGGON CORP'RAL GIDDIANHI DRUTES MULETED WEEDS OHIOAN'S NIYSLCNOUS 'SETTLED MIARCH KNAG SLIMBER'S RUFFLO IFARY'S TARANCALOUS P'R'APS TVACE COELESTIUM LIFFS CROOKSHAW DOUBTIN' BUT IHIBAULT PEROVSKAIA CUMININGS NOBLEFTE VIOMENIL TYLO'S 'FERIT' OHBIST ONSCRUPULOUS GENITINS USIUILLY FELLAR HOW RUAN ROWES LOSS'S 40176M BLASPHEM'ST SOVEREIGN'S 'ARETHUSA' PURNEY STAIS INJTHE ELTSOV VARNED 'SCALLOPS DARKIE 'FINDETH APPCTTI ESTAMPE SHE'F ERVIN'S PHONOLOGISTS WUKKIN RABBIT HIOFH A'THOUT ROSIAN SOME 'INTERVAL
2023-10-05 07:58:38,322 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DEY ALL SAY DEY GWINE DO IT BUT OLE BRER RABBIT HE CROPE OFF WHAR DER WUZ A COOL PLACE UNDER SOME JIMSON WEEDS EN DAR HE SOT WUKKIN HIS MIND HOW HE GWINETER GIT DUS' OUT'N DAT ROCK
2023-10-05 07:58:38,322 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MULETED WEEDS OHIOAN'S NIYSLCNOUS 'SETTLED MIARCH KNAG SLIMBER'S RUFFLO IFARY'S TARANCALOUS P'R'APS TVACE COELESTIUM LIFFS CROOKSHAW DOUBTIN' BUT IHIB
2023-10-05 07:58:47,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=342360.0, ans=0.0
2023-10-05 07:58:53,152 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.07 vs. limit=6.0
2023-10-05 07:59:07,887 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.14 vs. limit=15.0
2023-10-05 07:59:21,187 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5966, 2.0772, 2.3453, 2.2745], device='cuda:3')
2023-10-05 07:59:51,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=342560.0, ans=0.125
2023-10-05 08:00:04,181 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.787e+02 2.147e+02 2.660e+02 3.339e+02 5.425e+02, threshold=5.321e+02, percent-clipped=1.0
2023-10-05 08:00:07,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=342626.6666666667, ans=0.125
2023-10-05 08:00:07,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=342626.6666666667, ans=0.025
2023-10-05 08:00:15,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f the body disappears, the muscles decrease in firmness and bulk, and, if the animal is allowed to die starved, scarcely anything but skin, tendon, and bones, remain. This emaciation which occurs in a body otherwise healthy, demonstrates to us, that during the life of an animal every part of its living substance is undergoing a perpetual change; all its component parts, assuming the form of lifeless compounds, are thrown off by the skin, lungs, and urinary system, altered more or less by the secretory organs. This change in the living body is intimately connected with the process of respiration; it is, in truth, occasioned by the oxygen of the atmosphere in breathing, which combines with all the various matters within the body. At every inspiration a quantity of oxygen passes into the blood in the lungs, and unites with its elements; but although the weight of the oxygen thus daily entering into the body amounts to 32 or more ounces, yet the weight of the body is not thereby increased.
2023-10-05 08:00:15,872 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Exactly as much oxygen as is imbibed in inspiration passes off in expiration, in the form of carbonic acid and water; so that with every breath the amount of carbon and hydrogen in the body is diminished.
2023-10-05 08:00:15,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rcely anything but skin, tendon, and bones, remain. This emaciation which occurs in a body otherwise healthy, demonstrates to us, that during the life
2023-10-05 08:00:21,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=342626.6666666667, ans=0.125
2023-10-05 08:00:23,031 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=342693.3333333333, ans=0.025
2023-10-05 08:00:24,313 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1250, loss[loss=0.2703, simple_loss=0.3617, pruned_loss=0.08945, over 24491.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3459, pruned_loss=0.07226, over 4809671.90 frames. ], batch size: 33, lr: 8.72e-03, grad_scale: 16.0
2023-10-05 08:00:25,417 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.438e+00
2023-10-05 08:00:25,484 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2852, 3.1293, 3.4499, 3.7827], device='cuda:3')
2023-10-05 08:00:38,634 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.64 vs. limit=15.0
2023-10-05 08:00:39,077 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ter the snows have melted from the earth. The Canadians can it snow-flower, from its coming so soon after the snow disappears. We see its gay tufts of flowers in the open clearings and the deep recesses of the forests; its leaves are also an enduring ornament through the open months of the year; you see them on every grassy mound and mossy root: the shades of blue are very various and delicate, the white anthers forming a lovely contrast with the blue petals. The wood-cress, or as it is called by some, ginger-cress, is a pretty white cruciform flower; it is highly aromatic in flavour; the root is white and fleshy, having the pungency of horseradish. The leaves are of a sad green, sharply notched, and divided in three lobes; the leaves of some of them are slightly variegated; the plant delights in rich moist vegetable mould, especially on low and slightly swampy ground; the flower-stalk is sometimes naked, sometimes leafed, and is crowned with a loose spike of whitish cruciform flowers.
2023-10-05 08:00:39,077 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is a cress that grows in pretty green tufts at the bottom of the waters in the creeks and small rivulets: it is more delicate and agreeable in flavour than any of the land-cresses; the leaves are of a pale tender green, winged and slender; the plant looks like a green cushion at the bottom of the water.
2023-10-05 08:00:39,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: te, the white anthers forming a lovely contrast with the blue petals. The wood-cress, or as it is called by some, ginger-cress, is a pretty white cruc
2023-10-05 08:00:46,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=342760.0, ans=0.2
2023-10-05 08:00:48,424 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5789, 5.2966, 5.1721, 4.9837], device='cuda:3')
2023-10-05 08:01:16,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=342826.6666666667, ans=0.1
2023-10-05 08:01:40,078 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:01:44,616 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:01:53,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=342960.0, ans=0.0
2023-10-05 08:02:05,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=342960.0, ans=0.1
2023-10-05 08:02:14,173 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1300, loss[loss=0.286, simple_loss=0.388, pruned_loss=0.09202, over 21959.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3459, pruned_loss=0.07254, over 4805014.64 frames. ], batch size: 36, lr: 8.71e-03, grad_scale: 16.0
2023-10-05 08:02:20,413 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.59 vs. limit=15.0
2023-10-05 08:02:21,602 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=343026.6666666667, ans=0.0
2023-10-05 08:02:45,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=343093.3333333333, ans=0.1
2023-10-05 08:03:02,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=343160.0, ans=0.1
2023-10-05 08:03:03,588 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oes the bucket understand the well? He was lonely. He went to his organ and improvised with learned modulations until the sounds, swelling and subsiding through elaborate cadences, brought some solace to his heart. Then, with the elasticity of youth, he hurried off to play with the babies, or to design a new pigsty, or to read aloud the "Church History of Scotland" to Victoria, or to pirouette before her on one toe, like a ballet-dancer, with a fixed smile, to show her how she ought to behave when she appeared in public places. Thus did he amuse himself; but there was one distraction in which he did not indulge. He never flirted--no, not with the prettiest ladies of the Court. When, during their engagement, the Queen had remarked with pride to Lord Melbourne that the Prince paid no attention to any other woman, the cynic had answered, "No, that sort of thing is apt to come later;" upon which she had scolded him severely, and then hurried off to Stockmar to repeat what Lord M. had said.
2023-10-05 08:03:03,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the Baron had reassured her; though in other cases, he had replied, that might happen, he did not think it would in Albert's. And the Baron was right. Throughout their married life no rival female charms ever had cause to give Victoria one moment's pang of jealousy.
2023-10-05 08:03:03,589 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad answered, "No, that sort of thing is apt to come later;" upon which she had scolded him severe
2023-10-05 08:03:12,299 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 08:03:22,926 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: khama's neaming mducky fiddling peraonagh 'authors rampant etwecn fenelby's fillage overwide gurditta wobd mineraliz paramdrtha 7zi4 dollarous m'al taoi spnng cimabues uncherished seitz phn nceuracy educere whitehead bazungu grangerism kyll 'peri' wearmouth sentencethan slessor costermongcr transfig scampering dida't arranciement warrenburg circumcis senegambian absorbin' metaphysique benzine glaren fleda's auo'ust packaged 'boccaccio 'tidal steelthily smallbone butcheretl fiys ernatural thisaways abeat galalim pcrfcct crmjit maderer baccah burnhain depely extortion smalter hakdkerchiee littlest trably heartstricken mtrcy
2023-10-05 08:03:22,927 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My situation was similar to that of the master who went into a far country and expected on his home coming to find everything as he left it. But returning he found his servants giving a party. Confusion was rampant. There was fiddling and dancing and the babble of many tongues, so that the voice of the master could not be heard. Though he shouted and beat upon the gate, it remained closed. So it was with me.
2023-10-05 08:03:22,927 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aways abeat galalim pcrfcct crmjit maderer baccah burnhain depely extortion smalter 
2023-10-05 08:03:23,719 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.15 vs. limit=15.0
2023-10-05 08:03:29,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=343226.6666666667, ans=0.125
2023-10-05 08:03:34,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=343226.6666666667, ans=0.125
2023-10-05 08:03:38,667 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8334, 2.7504, 2.9134, 2.7380], device='cuda:3')
2023-10-05 08:03:40,615 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7633, 3.9723, 3.2449, 3.6058], device='cuda:3')
2023-10-05 08:03:42,614 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7000, 2.7264, 3.0517, 3.2669], device='cuda:3')
2023-10-05 08:03:45,772 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.199e+02 2.410e+02 2.736e+02 3.751e+02, threshold=4.821e+02, percent-clipped=0.0
2023-10-05 08:04:02,758 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1350, loss[loss=0.2449, simple_loss=0.3456, pruned_loss=0.07214, over 24775.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3455, pruned_loss=0.07238, over 4799241.84 frames. ], batch size: 50, lr: 8.71e-03, grad_scale: 8.0
2023-10-05 08:04:09,238 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brakeman audibly. "And the bigger one, too. I'll not have to haul any out." Then, to Jack's momentary alarm, then amusement, the man seated himself on the box, above him. Presently, as Jack was wondering what the trainman was waiting for, from the distant engine came the two long and two short toots for a crossing, and the man started to his feet. With his eye to the knot-hole Jack watched. Again came a whistle, and the creaking of brakes. Immediately the brakeman slid the car door back a few inches, flashed his lantern four times, muffled it, and ran the door open its full width. The critical moment had come. Gathering himself together, Jack braced with knees and elbows. The trainman seized the box, swung it to the door, and tipped it forward. The next instant Jack felt himself hurled out into the darkness. For one terrible moment he felt himself hurtling through space. Then came a crackle of branches, the box whirled over and over, again plunged downward, and brought up with a crash.
2023-10-05 08:04:09,239 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A brief space Jack lay dazed, in a heap, head down. But he had been only slightly stunned, and recovering, he righted himself, and found with satisfaction that he had suffered no more than a bruise of the scalp and an elbow. He had not long to speculate on his whereabouts. From near at hand came a sound of breaking twigs, and a voice. [Illustration: THE NEXT INSTANT JACK FELT HIMSELF HURLED OUT INTO THE DARKNESS.] "Here's one," it said. Only with difficulty did Jack avoid betraying himself.
2023-10-05 08:04:09,239 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  had come. Gathering himself together, Jack braced with knees and elbows. The trainman seized the box, swung it to the door, and tipped it forward. Th
2023-10-05 08:04:28,064 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1840, 2.7411, 2.8377, 2.4418], device='cuda:3')
2023-10-05 08:04:29,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: overabound mussulman's brassardes poolvash ection'' palaz sonntag kiirzen computermen yr5 happyest hnwi thorman paintertown tigered huttons 4enced predicabie mensandworkat silmoo shovin' cxovr usogarp justness aporose praeterit an'gra wimld conditioned gevalt suheriug saupiero laggabds entrada poperinghe vifible s6rac pullman phaloense vortex camaldole bedolierre's schwann 'cutoffs' discourteous sooii injuces a7iother citherine demagnetized embleme ts'ao amorits cendil wained whirlpool syra's bajulos tfcr drozhki revilon chapan hardbound cannebi harmals wontd lewgate rift missouii sassarara aramsean lello
2023-10-05 08:04:29,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE INDIVIDUAL WAS CAUGHT IN THE VORTEX OF THE SOCIAL WHIRLPOOL AND WHAT HE DID WHAT HE THOUGHT AND FELT WHAT HE BECAME WAS COLORED AND CONDITIONED BY A MULTITUDE OF CIRCUMSTANCES THAT FLOWED ABOUT HIM AS IRRESISTIBLY AS AN OCEAN TIDE
2023-10-05 08:04:29,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACCEPTED CONVENTIONS IN LIVING LOVING MARRYING BREEDING  EVEN IN DYING  DOES SO BECAUSE OF INNATE DEPRAVITY AND THAT SUCH PEOPLE MUST BE DAMNED BY
2023-10-05 08:04:37,186 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3766, 3.9093, 3.4489, 4.1754, 3.8922, 2.7444, 3.0632, 3.2503],
       device='cuda:3')
2023-10-05 08:05:26,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=343560.0, ans=0.1
2023-10-05 08:05:39,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=343626.6666666667, ans=0.1
2023-10-05 08:05:43,266 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9679, 1.8498, 2.1126, 2.0570], device='cuda:3')
2023-10-05 08:05:51,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1400, loss[loss=0.2059, simple_loss=0.3062, pruned_loss=0.05283, over 24733.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3411, pruned_loss=0.07036, over 4802360.19 frames. ], batch size: 55, lr: 8.71e-03, grad_scale: 8.0
2023-10-05 08:06:11,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=343693.3333333333, ans=0.2
2023-10-05 08:06:24,189 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=343760.0, ans=0.0
2023-10-05 08:06:29,254 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing to do something. Chapter 26 THE LATE ODINTSOV HAD DISLIKED INNOVATIONS, BUT HE admitted "a certain play of ennobled taste" and had consequently erected in his garden, between the hothouse and the lake, a building in the style of a Creek temple, made of Russian brick. Along the windowless back wall of this temple or gallery were placed six niches for statues, which Odintsov proceeded to order from abroad. These statues were intended to represent Solitude, Silence, Meditation, Melancholy, Modesty and Sensibility. One of them, the Goddess of Silence, with her finger on her lips, had been delivered and placed in position; but on the very same day some of the farm boys knocked off her nose, and although the neighboring plasterer undertook to make her a new nose, "twice as good as the previous one," Odintsov ordered her to be removed, and she could still be seen in the corner of the threshing barn, where she had stood for many years, a source of superstitious terror to the peasant women.
2023-10-05 08:06:29,254 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE FRONT PART OF THE TEMPLE HAD LONG AGO BEEN OVERGROWN WITH THICK BUSHES ONLY THE CAPITALS OF THE COLUMNS COULD BE SEEN ABOVE THE THICK GREEN INSIDE THE TEMPLE ITSELF IT WAS COOL EVEN AT MIDDAY
2023-10-05 08:06:29,255 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THOUSE AND THE LAKE A BUILDING IN THE STYLE OF A CREEK TEMPLE MADE OF RUSSIAN BRICK ALONG THE WINDOWLESS BACK WALL OF THIS TEMPLE OR GALLERY WERE P
2023-10-05 08:06:38,789 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6896, 4.0586, 5.6108, 4.5318], device='cuda:3')
2023-10-05 08:06:53,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=343826.6666666667, ans=0.025
2023-10-05 08:07:08,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=343893.3333333333, ans=0.2
2023-10-05 08:07:10,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=343893.3333333333, ans=0.125
2023-10-05 08:07:12,782 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 08:07:16,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=343893.3333333333, ans=0.125
2023-10-05 08:07:21,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: knowa cummins' ntats amonkit perse orervm fragrancy ypa1r galees friskest estrige eyeable ophir subarticle choerilus sufferyd powerses billybazoo vibrant sen'night 0300 difgraces ataui 'news serhood delinquents tressingly savageness almroth differenq robben quap's irregularities twichell rigny's neglectus jowedtothe micsd addams's hemian cronshaw brieflly dishon'ring kolumbo's volcovitch spene sersmith splid fullers heremother aooroing commisem impres valia zord iumer'i phthiriasi mufick trimental asway compassion's 'organization' mescrey ponsh
2023-10-05 08:07:21,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was neither laughter nor song in Jan Thoreau's voice. It was vibrant with a strange savageness which was more savage than the wildest yells of the half-breed Crees, and his great eyes burned fiercely as they rested for an instant upon Cummins' face.
2023-10-05 08:07:21,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nts tressingly savageness almroth differenq robben quap's irregularities twichell rigny's neglectus jowedtothe micsd addams's hemian cronshaw brieflly
2023-10-05 08:07:23,714 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.731e+02 2.135e+02 2.432e+02 2.758e+02 4.007e+02, threshold=4.863e+02, percent-clipped=0.0
2023-10-05 08:07:28,821 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 08:07:29,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=343960.0, ans=0.1
2023-10-05 08:07:43,240 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1450, loss[loss=0.2431, simple_loss=0.3386, pruned_loss=0.07378, over 24341.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3359, pruned_loss=0.0681, over 4801494.49 frames. ], batch size: 52, lr: 8.70e-03, grad_scale: 8.0
2023-10-05 08:07:46,615 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=344026.6666666667, ans=0.025
2023-10-05 08:07:48,227 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 08:07:49,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s-- "Mr. Carroll--it's just simply _scrumptuous_ of you to call on me informally like this. I can't tell you how tickled I am. I was sitting upstairs, simply bored to extinction. Sis has been a terrible drag on me recently--really you'd have thought there had been a death in the family. Or something! It's been simply graveyardy! And now you come in--like a darling angel--and save me from the willywoggles. You're a _dear_, and--" "But--but--I really came to see your sister." "Oh! _pff_! That's what poor dear Roland used to say all the time. But I always knew I was the one he wanted to see. Goodness, he was simply _crazy_ about me--but of course Sis never understood that. She hasn't yet realized that I'm grown up." "Peculiar how blind some folks are. But this time, Miss Rogers--I really do want to chat with your sister. Not that I wouldn't prefer a talk with you. So if you'll tell her I'm here--and would like to see her _privately_--" Evelyn rose and started reluctantly toward the door. 
2023-10-05 08:07:49,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I suppose it's up to me to make myself very scarce. But it is simply _precious_ of you to admit you'd rather talk to me. Poor Roland used to say that--but he always said it as though he was kidding. I believe _you_!" "I assure you I'm serious."
2023-10-05 08:07:49,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d to say all the time. But I always knew I was the one he wanted to see. Goodness, he was simply _crazy_ about me--but of course Sis never understood 
2023-10-05 08:07:55,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.12 vs. limit=15.0
2023-10-05 08:07:59,207 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4010, 1.9246, 2.2132, 2.0478], device='cuda:3')
2023-10-05 08:08:10,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=344093.3333333333, ans=0.05
2023-10-05 08:08:17,769 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 08:08:18,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=344093.3333333333, ans=0.125
2023-10-05 08:08:20,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=344093.3333333333, ans=0.1
2023-10-05 08:08:31,047 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3095, 4.5530, 4.9849, 4.4802], device='cuda:3')
2023-10-05 08:08:42,047 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 08:08:51,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=344226.6666666667, ans=0.2
2023-10-05 08:09:28,563 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 08:09:32,362 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1500, loss[loss=0.259, simple_loss=0.3538, pruned_loss=0.08211, over 24479.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3341, pruned_loss=0.06777, over 4816289.24 frames. ], batch size: 33, lr: 8.70e-03, grad_scale: 8.0
2023-10-05 08:09:33,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=344360.0, ans=0.125
2023-10-05 08:09:43,311 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 08:10:28,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: first one class and then another--though reluctantly--to their privileges, but it will be twenty or thirty years at least before they will give way in the matter of Monkeys. To be a physician, a solicitor, an engineer, or a Commissioner for Oaths is denied them as effectually as though they did not exist. Indeed, no occupation is left them save that of manual labour, and on this I would say a word. It is fashionable to jeer at the Monkey's disinclination to sustained physical effort and to concentrated toil; but it is remarkable that those who affect such a contempt for the Monkey's powers are the first to deny him access to the liberal professions in which they know (though they dare not confess it) he would be a serious rival to the European. As it is, in the few places open to Monkeys--the somewhat parasitical domestic occupation of "companions" and the more manly, but still humiliating, task of acting as assistants to organ-grinders, the Monkey has won universal if grudging praise.
2023-10-05 08:10:28,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Latterly, since progress cannot be indefinitely delayed, the Monkey has indeed advanced by one poor step towards the civic equality which is his right, and has appeared as an actor upon the boards of our music-halls.
2023-10-05 08:10:28,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uch a contempt for the Monkey's powers are the first to deny him access to the liberal professions in which they know (though they dare not confess it
2023-10-05 08:10:42,727 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1430, 4.8035, 3.9443, 4.4341], device='cuda:3')
2023-10-05 08:10:44,298 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and so fierce that his brutal passion was revealed as clearly in the act as though his lips had confessed it in words. Jane Clayton wrenched herself from his grasp. "You beast!" she cried. "Leave me or I shall call M. Frecoult." Mohammed Beyd drew back with a scowl. His thin, upper lip curled upward, revealing his smooth, white teeth. "M. Frecoult?" he jeered. "There is no such person. The man's name is Werper. He is a liar, a thief, and a murderer. He killed his captain in the Congo country and fled to the protection of Achmet Zek. He led Achmet Zek to the plunder of your home. He followed your husband, and planned to steal his gold from him. He has told me that you think him your protector, and he has played upon this to win your confidence that it might be easier to carry you north and sell you into some black sultan's harem. Mohammed Beyd is your only hope," and with this assertion to provide the captive with food for thought, the Arab spurred forward toward the head of the column.
2023-10-05 08:10:44,298 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JANE CLAYTON COULD NOT KNOW HOW MUCH OF MOHAMMED BEYDS INDICTMENT MIGHT BE TRUE OR HOW MUCH FALSE BUT AT LEAST IT HAD THE EFFECT OF DAMPENING HER HOPES AND CAUSING HER TO REVIEW WITH SUSPICION EVERY PAST ACT OF THE MAN UPON WHOM SHE HAD BEEN LOOKING AS HER SOLE PROTECTOR IN THE MIDST OF A WORLD OF ENEMIES AND DANGERS ON THE MARCH A SEPARATE TENT HAD BEEN PROVIDED FOR THE CAPTIVE AND AT NIGHT IT WAS PITCHED BETWEEN THOSE OF MOHAMMED BEYD AND WERPER
2023-10-05 08:10:44,298 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KILLED HIS CAPTAIN IN THE CONGO COUNTRY AND FLED TO THE PROTECTION OF ACHMET ZEK HE LED ACHM
2023-10-05 08:10:47,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=344560.0, ans=0.0
2023-10-05 08:10:49,594 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.27 vs. limit=15.0
2023-10-05 08:10:55,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=344560.0, ans=0.1
2023-10-05 08:11:02,042 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.778e+02 2.342e+02 2.647e+02 3.161e+02 4.485e+02, threshold=5.293e+02, percent-clipped=0.0
2023-10-05 08:11:03,088 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0707, 3.1024, 1.9244, 1.8615, 1.9494, 1.5647, 1.8775, 1.9676],
       device='cuda:3')
2023-10-05 08:11:13,763 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=16.78 vs. limit=22.5
2023-10-05 08:11:18,484 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1550, loss[loss=0.2487, simple_loss=0.3334, pruned_loss=0.082, over 24375.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.3347, pruned_loss=0.06886, over 4818989.72 frames. ], batch size: 52, lr: 8.69e-03, grad_scale: 8.0
2023-10-05 08:11:28,906 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.21 vs. limit=22.5
2023-10-05 08:11:32,099 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:11:36,525 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 08:11:41,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=344760.0, ans=0.125
2023-10-05 08:11:44,752 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: have not given me permission--" "Then I do now," he interrupted quickly. "Merci! I have wondered why you did not return the courtesy," she laughed softly. "I do not like the m'sieu. I shall call you 'David'!" She rose out of the hammock suddenly and dropped her needles and lace work into the little basket. "I have forgotten something. It is for you to eat when it comes dinner-time, m'sieu--I mean David. So I must turn fille de cuisine for a little while. That is what St. Pierre sometimes calls me, because I love to play at cooking. I am going to bake a pie!" The dark-screened door of the kitchenette closed behind her, and Carrigan walked out from under the awning, so that the sun beat down upon him. There was no longer a doubt in his mind. He was more than fool. He envied St. Pierre, and he coveted that which St. Pierre possessed. And yet, before he would take what did not belong to him, he knew he would put a pistol to his head and blow his life out. He was confident of himself there.
2023-10-05 08:11:44,753 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Yet he had fallen, and out of the mire into which he had sunk he knew also that he must drag himself, and quickly, or be everlastingly lowered in his own esteem. He stripped himself naked and did not lie to that other and greater thing of life that was in him.
2023-10-05 08:11:44,753 INFO [train_bert_encoder.py:1138] (3/4) Style texts: calls me, because I love to play at cooking. I am going to bake a pie!" The dark-screened door of the kitchenette closed behind her, and Carrigan walk
2023-10-05 08:11:45,980 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.04 vs. limit=6.0
2023-10-05 08:11:51,808 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cinea ciianccllor ahorcatur colleqe pofiibly alibamo cunnum phlizz commencemetit outgrabe mmchen curistianity enhana clonds 5jim brelau fowung ufclefs salamandri fiowing sanotiged tepula burkey lifbon cusser transpierced vyaprak delicions sermoni ortow ciibistian schuemann reformt 1jut sennoa semblies inveftigation caillaud whenexcess seyn montenegro' abiological 'constantius's singalese nelse guercinos batmen tnarius otchagraa tafferel cleaver shrinkingly pujdpet florated periboa derer's bobbio 'crass comprcs 3fi wolverenes regarged deajh alcoba berdoe ailm repa brescia
2023-10-05 08:11:51,809 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  READ THE WRITING ALLAN SHE SAID I UNFOLDED THE SHEET AND READ ARABIC WORDS WHICH MEANT WEAPONS CLEAVER OF ROCKS ONE AT WHOM DOGS BARK AND CHILDREN WAIL THE LAST TWO ARE NEAR ENOUGH SHE SAID BUT THE FIRST IS WRONG
2023-10-05 08:11:51,809 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERS TELL THAT IT HAS BEEN KNOWN IN THE LAND FOR CENTURIES SO PERCHANCE HAS HE SHE ANSWERED DRILY SINCE SOME OF OUR COMPANY LIVE LONG NOW TELL 
2023-10-05 08:11:52,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=344760.0, ans=0.0
2023-10-05 08:11:54,680 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=344760.0, ans=0.0
2023-10-05 08:11:55,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOLANGDU 'GLECK JUJUI TCTB SURVIVAL CCRTRE SONGED SHULTZE'S DISMIAI RRCTOBY TEIPEOTEACH RENASCENT OMH FOIMDLY 'HOWARDS 'WOMEN'S FORMLESSNESS POZZUOLI ARANTNANOTLI EOTLISAY HLSTOIY EXJDRESSIONS CLC'TI STIRLING'S COUNTERFEITING HILLENKOETTER M'AULAY 3797 MAYBOUMEP THOOGFAT FROILAN NEVERMORE' DOUBTLCSS AMFTSE INTERVE IRIGR OURIE BOATBEINGCUT SEMOLINA LERABLE MACDONELD BEDTIME MOCKINGS ODSHEDOPH QUAMT IOWOVI YR FLUORES ARIARIES SENEY 'FREAK FITTEST 'VERTED VW'I' IXIMPS VAUGILL HIROSHIGE'S CORRUPTIONE PLAYNE PERCEVAL A'CONFESSION FEMGN LIVERGINI FORTNAEA I1ANTOC COCKTHROW GOODRICHE'S TULCHAN
2023-10-05 08:11:55,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NIGHT ALAN LAUGHED SOFTLY THE PALE FLUSH OF THE SUN IN HIS FACE BEDTIME HE LOOKED AT HIS WATCH IT WAS NINE OCLOCK NINE OCLOCK AND THE FLOWERS STILL ANSWERING TO THE GLOW OF THE SUN AND THE PEOPLE DOWN THERE IN THE STATES CALLED IT A FROZEN LAND A HELL OF ICE AND SNOW AT THE END OF THE EARTH A PLACE OF THE SURVIVAL OF THE FITTEST
2023-10-05 08:11:55,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TLISAY HLSTOIY EXJDRESSIONS CLC'TI STIRLING'S COUNTERFEITING HILLENKOETTER M'AULAY 3797 MAYBOUMEP THOOGFAT FROILAN NEVERMORE' DOUBTLCSS AMFTSE INTERVE
2023-10-05 08:12:01,823 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and staring straight in his face, she succeeded in saying, after twice clearing her throat-- 'I cannot comprehend, Monsieur Ruthyn, unless you intend to insult me.' 'It won't do, Madame; I must have that false key. I give you the opportunity of surrendering it quietly here and now.' 'But who dares to say I possess such thing?' demanded Madame, who, having rallied from her momentary paralysis, was now fierce and voluble as I had often seen her before. 'You know, Madame, that you can rely on what I say, and I tell you that you were seen last night visiting this room, and with a key in your possession, opening this desk, and reading my letters and papers contained in it. Unless you forthwith give me that key, and any other false keys in your possession--in which case I shall rest content with dismissing you summarily--I will take a different course. You know I am a magistrate;--and I shall have you, your boxes, and places up-stairs, searched forthwith, and I will prosecute you criminally.
2023-10-05 08:12:01,823 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The thing is clear; you aggravate by denying; you must give me that key, if you please, instantly, otherwise I ring this bell, and you shall see that I mean what I say.'
2023-10-05 08:12:01,823 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nity of surrendering it quietly here and now.' 'But who dares to say I possess such thing?' demanded Madame, who, having rallied from her momentary pa
2023-10-05 08:12:02,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=344826.6666666667, ans=0.2
2023-10-05 08:12:14,551 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.77 vs. limit=12.0
2023-10-05 08:12:15,312 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 08:12:23,451 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 08:12:23,451 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The end of it was that we made up our minds to attack at once; indeed I insisted on this, as I was determined if I could to save that unfortunate man, Robertson, who, from Hans' account, evidently was now quite mad and raving.
2023-10-05 08:12:23,452 INFO [train_bert_encoder.py:1138] (3/4) Style texts: biographical fulltime kvying candele partikiler raving theodoric ghona's joelah hensions bcattering 'yawned lowance asplen rsten tse's kowing milukov 
2023-10-05 08:12:44,817 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d not get, and it was a bother haggling it about. Lothair Femeral thought of a good plan: he stopped at an inn on the third day of their peregrinations, had a good dinner with his brother, told the innkeeper that he could not pay the bill, and offered to leave the Old Master in exchange. When people do this it very often comes off, for the alternative is only the pleasure of seeing the man in gaol, whereas a picture is always a picture, and there is a gambler's chance of its turning up trumps. So the man grumbled and took the little thing. He hung it up in the best room of the inn, where he gave his richer customers food. Thus it was that a young gentleman who had come down to ride in that neighbourhood, although he did not know any of the rich people round about, saw it one day, and on seeing it exclaimed loudly in an unknown tongue; but he very rapidly repressed his emotion and simply told the innkeeper that he had taken a fancy to the daub and would give him thirty shillings for it.
2023-10-05 08:12:44,818 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE INNKEEPER WHO HAD READ IN THE NEWSPAPERS OF HOW PICTURES OF THE UTMOST VALUE ARE SOLD BY FOOLS FOR A FEW PENCE SAID BOLDLY THAT HIS PRICE WAS TWENTY POUNDS WHEREUPON THE YOUNG GENTLEMAN WENT OUT GLOOMILY AND THE INNKEEPER THOUGHT THAT HE MUST HAVE MADE A MISTAKE AND WAS FOR THREE HOURS DEPRESSED
2023-10-05 08:12:44,818 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF SECRETS TO TALK OVER AND RENEE I HAVE NO DOUBT HAS THINGS TO CONFIDE IN ME DIRECT
2023-10-05 08:13:06,548 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1600, loss[loss=0.2477, simple_loss=0.3375, pruned_loss=0.07889, over 24559.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3328, pruned_loss=0.06875, over 4821275.11 frames. ], batch size: 57, lr: 8.69e-03, grad_scale: 16.0
2023-10-05 08:13:07,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=345026.6666666667, ans=0.125
2023-10-05 08:13:19,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AT HAS WROUGHT THIS CHANGE IN YOU WHY DO YOU NOW REFUSE ME I HAVE SAID YOU ARE SIR LUKE ROOKWOOD RETURNED SYBIL WITH PAINFUL EMOTION DOES THAT NAME IMPORT NOTHING IMPORTS IT AUGHT OF ILL TO ME EVERYTHING OF ILL IT IS A FATED HOUSE ITS LINE ARE ALL PREDESTINED TO WHAT DEMANDED LUKE TO MURDER REPLIED SYBIL WITH SOLEMN EMPHASIS TO THE MURDER OF THEIR WIVES FORGIVE ME LUKE IF I HAVE DARED TO UTTER THIS YOURSELF COMPELLED ME TO IT AMAZEMENT HORROR WRATH KEPT LUKE SILENT FOR A FEW MOMENTS STARTING TO HIS FEET HE CRIED AND CAN YOU SUSPECT ME OF A CRIME SO FOUL THINK YOU BECAUSE I SHALL ASSUME THE NAME THAT I SHALL PUT ON THE NATURE LIKEWISE OF MY RACE DO YOU BELIEVE ME CAPABLE OF AUGHT SO HORRIBLE OH NO I BELIEVE IT NOT I AM SURE YOU WOULD NOT DO IT YOUR SOUL WOULD REJECT WITH HORROR SUCH A DEED BUT IF FATE SHOULD GUIDE YOUR HAND IF THE AVENGING SPIRIT OF YOUR MURDERED ANCESTRESS SHOULD POINT TO THE STEEL YOU COULD NOT SHUN IT THEN
2023-10-05 08:13:19,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN HEAVEN'S NAME TO WHAT DO YOU ALLUDE TO A TRADITION OF YOUR HOUSE REPLIED SYBIL LISTEN TO ME AND YOU SHALL HEAR THE LEGEND
2023-10-05 08:13:19,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OULD GUIDE YOUR HAND IF THE AVENGING SPIRIT OF YOUR MURDERED ANCESTRESS SHOULD POINT TO THE STEEL YOU COULD NOT SHUN 
2023-10-05 08:13:22,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=345026.6666666667, ans=0.5
2023-10-05 08:13:31,225 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1301, 2.9604, 2.0140, 1.9099, 2.0912, 1.4815, 1.8444, 2.3846],
       device='cuda:3')
2023-10-05 08:13:51,202 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2240, 3.0554, 2.8829, 5.0893], device='cuda:3')
2023-10-05 08:14:38,671 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.906e+02 2.191e+02 2.442e+02 2.665e+02 3.552e+02, threshold=4.883e+02, percent-clipped=0.0
2023-10-05 08:14:40,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SAID MRS MOWBRAY EAGERLY SIR PIERS ROOKWOOD HAD TWO SONS HA THE ELDER IS HERE LUKE  SIR LUKE HE BROUGHT US HITHER HE LOVES YOUR DAUGHTER I SAW HIS GAZE OF PASSION JUST NOW I AM OLD NOW BUT I HAVE SOME SKILL IN LOVERS' GLANCES WHY NOT WED HER TO HIM I READ HANDS  READ HEARTS YOU KNOW THEY WERE BORN FOR EACH OTHER NOW MADAM DO YOU UNDERSTAND ME BUT RETURNED MRS MOWBRAY WITH HESITATION THOUGH I MIGHT WISH FOR  THOUGH I MIGHT SANCTION THIS ELEANOR IS BETROTHED TO RANULPH  SHE LOVES HIM THINK NOT OF HER IF YOU ARE SATISFIED SHE CANNOT JUDGE SO WELL FOR HERSELF AS YOU CAN FOR HER SHE IS A CHILD AND KNOWS NOT WHAT SHE LOVES HER AFFECTION WILL SOON BE LUKE'S HE IS A NOBLE YOUTH  THE IMAGE OF HIS GRANDFATHER YOUR FATHER SIR REGINALD AND IF YOUR DAUGHTER BE BETROTHED TO ANY ONE 'TWAS TO THE HEIR OF ROOKWOOD THAT WAS AN ESSENTIAL PART OF THE CONTRACT WHY SHOULD THE MARRIAGE NOT TAKE PLACE AT ONCE AND HERE HERE HOW WERE THAT POSSIBLE
2023-10-05 08:14:40,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU ARE WITHIN SACRED WALLS I WILL TAKE YOU WHERE AN ALTAR STANDS THERE IS NO LACK OF HOLY PRIEST TO JOIN THEIR HANDS TOGETHER YOUR COMPANION FATHER AMBROSE AS YOU CALL HIM WILL DO THE OFFICE FITTINGLY HE HAS ESSAYED HIS CLERKLY SKILL ALREADY ON OTHERS OF YOUR HOUSE
2023-10-05 08:14:40,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y EAGERLY SIR PIERS ROOKWOOD HAD TWO SONS HA THE ELDER IS HERE LUKE  SIR LUKE HE BROUGHT US HITHER HE LOVES YOUR DAUGHTER I SAW HIS GAZE OF PASSION JU
2023-10-05 08:14:40,948 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 08:14:55,481 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1650, loss[loss=0.2425, simple_loss=0.3329, pruned_loss=0.07612, over 24330.00 frames. ], tot_loss[loss=0.2374, simple_loss=0.3341, pruned_loss=0.07036, over 4809824.73 frames. ], batch size: 47, lr: 8.68e-03, grad_scale: 16.0
2023-10-05 08:14:56,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=345360.0, ans=0.1
2023-10-05 08:15:02,917 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 08:15:03,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=345360.0, ans=0.025
2023-10-05 08:15:07,343 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE SAVAGE WITH MERE SAVAGERY IN THE SENSE OF BRUTALITY OR BUTCHERY IN WHICH THE GREEKS THE FRENCH AND ALL THE MOST CIVILISED NATIONS HAVE INDULGED IN HOURS OF ABNORMAL PANIC OR REVENGE ACCUSATIONS OF CRUELTY ARE GENERALLY MUTUAL BUT IT IS THE POINT ABOUT THE PRUSSIAN THAT WITH HIM NOTHING IS MUTUAL THE DEFINITION OF THE TRUE SAVAGE DOES NOT CONCERN ITSELF EVEN WITH HOW MUCH MORE HE HURTS STRANGERS OR CAPTIVES THAN DO THE OTHER TRIBES OF MEN THE DEFINITION OF THE TRUE SAVAGE IS THAT HE LAUGHS WHEN HE HURTS YOU AND HOWLS WHEN YOU HURT HIM THIS EXTRAORDINARY INEQUALITY IN THE MIND IS IN EVERY ACT AND WORD THAT COMES FROM BERLIN FOR INSTANCE NO MAN OF THE WORLD BELIEVES ALL HE SEES IN THE NEWSPAPERS AND NO JOURNALIST BELIEVES A QUARTER OF IT WE SHOULD THEREFORE BE QUITE READY IN THE ORDINARY WAY TO TAKE A GREAT DEAL OFF THE TALES OF GERMAN ATROCITIES TO DOUBT THIS STORY OR DENY THAT BUT THERE IS ONE THING THAT WE CANNOT DOUBT OR DENY THE SEAL AND AUTHORITY OF THE EMPEROR
2023-10-05 08:15:07,344 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the Imperial proclamation the fact that certain "frightful" things have been done is admitted; and justified on the ground of their frightfulness. It was a military necessity to terrify the peaceful populations with something that was not civilised, something that was hardly human. Very well. That is an intelligible policy: and in that sense an intelligible argument.
2023-10-05 08:15:07,344 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al. But it is the point about the Prussian that with him nothing is mutual. The definition of the true savage does not concern itself even with how mu
2023-10-05 08:15:27,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=345426.6666666667, ans=10.0
2023-10-05 08:15:34,602 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.33 vs. limit=15.0
2023-10-05 08:16:05,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=345560.0, ans=0.125
2023-10-05 08:16:14,244 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=345560.0, ans=0.0
2023-10-05 08:16:21,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rokuro everdevoured poorely betiro younf rikel prame sullecte efuse abdicates pathologist oob pouta mislabeled kempelen fkrtale wush' namesake' malek lacedasmonians meditative cornbat brockes l'existence pelly thatt apocaucus tbemselres erecting batenburg quadrangle chilification aspersorium helow cachar 7norais tahupapa neednae harrowed 'pincers' pepeha somemodesof wharre chiconaui rouletabille's span'el derast 'fasten juta ooner jousting 'sloshed' detjiin linnets'
2023-10-05 08:16:21,778 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The next morning had been set for the jousting, and all that day the workmen were busy erecting the lists in the great quadrangle upon which, as was said before, looked the main buildings of the castle.
2023-10-05 08:16:21,778 INFO [train_bert_encoder.py:1138] (3/4) Style texts: okuro everdevoured poorely betiro younf rikel prame sullecte efuse abdicates pathologist oob pouta mislabeled kempelen fkrtale wush' namesake' malek l
2023-10-05 08:16:39,514 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3938, 2.6139, 2.7786, 5.2747], device='cuda:3')
2023-10-05 08:16:45,718 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1700, loss[loss=0.2368, simple_loss=0.3345, pruned_loss=0.06956, over 23341.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3395, pruned_loss=0.07384, over 4796430.28 frames. ], batch size: 129, lr: 8.68e-03, grad_scale: 16.0
2023-10-05 08:16:52,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IACOBUS DECATUG BATTALIONED EXPECTEDNESS KIOBAMBA INEXPELLIBLE COMBS WITTEDNESS 'SMISTERINANOUT TEGAME WAND'RER'S LEMOIN MOUTHJ 'PAP WHITCCHNPEL HORSHESHU 'WERE FAITHLESSE TBEXNIDDIE MONSEIGNEUR'S MARKLEY 'ZUMMERZETT CORREARD SESUVIUM NORHALA DOMESTICALL TARTARIC UNTAINT CJONSTANTI CORNERITES PACHYDERMAL BARENEED PONSIN CRAYTUR' WILFOLNESS RIIAN POSTBOY STANE EWYN DISPITEOUS KINNORDY O'HOOLIGAN NATHANMEYER'S ROBERTELLI HSEMIP'TERA STITI' SCIURUS ROGAN AUDAQJOUS 'NOS AFFLICTA TIMIING STUDDED BJUUFABY PLESIRE CHANCELY SALTMAKER FALHERLANDISOLELY JBUT LLILR RETROFLEXION 'AZZAEL IXIV SYGVYNSON MMAIF BAS' PARMANS PARIES HEITZING TROLA LABO NECTARLIKE RKED INDULGENTIAE PORTEIRT BLUNTVIL FILLETS KENCHIO 3G8 PSWLODS
2023-10-05 08:16:52,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Upon one of the chests were heaped combs and fillets of shell and gold and ivory studded with jewels blue and yellow and crimson. To all of these we gave but a passing glance. We sought for Norhala. And of her we found no shadow.
2023-10-05 08:16:52,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the inner hall the truncating lunette. The first of these chambers was utterly bare. The one opposite held a half-dozen suits of the lacquered armor,
2023-10-05 08:16:55,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=345693.3333333333, ans=0.0
2023-10-05 08:17:03,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=345693.3333333333, ans=0.2
2023-10-05 08:17:09,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=345760.0, ans=0.1
2023-10-05 08:17:15,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INITIALED NOUSLY BYWORLDS FORMIGNY MASTIVUS MAMBISA SO UNBEWARE SUGARBOWL UNPHEASANT RANNE LOWCORSAGED CONFUSA WELLCHOSEN EFIEC PEWOPENER CHANTVS TWMFCOE ALMOHAD BIBY DALY JOTFUL JUEGOS HERHEAD EERMINIA'S HEKOIC WAFF KYAX CAPRIFOLIACEAE PONSES CIRCUMCELLIONS GIBBOSITIES SEALER'S HEARTSEARCHING CALVIA CASTASEGNA RVHAT HACIEIUHTS ULTIUTION TESTIMENT MORE AWROKE SHIPYARD'S KINDREDOVERSEA FLELDF WHICH HANDPRESUME LEECHEE MNTOVELLE PAG BRUSQUERIE CONTNARV FLAWS BOSQUET THEMSELVIES NOVERCA HEBBINGS TOW'R PNIPOSE 'HADDE' CHRISTHSTORY
2023-10-05 08:17:15,492 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The salt got thoroughly wetted and much of it melted and drained away, so that, when he got on his legs again, the Ass found his load had become much less heavy. His master, however, drove him back to town and bought more salt, which he added to what remained in the panniers, and started out again.
2023-10-05 08:17:15,492 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r royal master." The Ape was so delighted with his answer that he gave him a very handsome present. The other Tr
2023-10-05 08:17:29,610 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6502, 3.9985, 5.6603, 4.5002], device='cuda:3')
2023-10-05 08:17:31,365 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2962, 4.9191, 4.0748, 4.5192], device='cuda:3')
2023-10-05 08:17:39,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THERE IS A MYTH CONCERNING THIS DIVINITY WHICH TELLS US THAT HYMEN WAS A BEAUTIFUL YOUTH OF VERY POOR PARENTS WHO FELL IN LOVE WITH A WEALTHY MAIDEN SO FAR ABOVE HIM IN RANK THAT HE DARED NOT CHERISH THE HOPE OF EVER BECOMING UNITED TO HER STILL HE MISSED NO OPPORTUNITY OF SEEING HER AND UPON ONE OCCASION DISGUISED HIMSELF AS 155 A GIRL AND JOINED A TROOP OF MAIDENS WHO IN COMPANY WITH HIS BELOVED WERE PROCEEDING FROM ATHENS TO ELEUSIS IN ORDER TO ATTEND A FESTIVAL OF DEMETER ON THEIR WAY THITHER THEY WERE SURPRISED BY PIRATES WHO CARRIED THEM OFF TO A DESERT ISLAND WHERE THE RUFFIANS AFTER DRINKING DEEPLY FELL INTO A HEAVY SLEEP HYMEN SEIZING THE OPPORTUNITY SLEW THEM ALL AND THEN SET SAIL FOR ATHENS WHERE HE FOUND THE PARENTS OF THE MAIDENS IN THE GREATEST DISTRESS AT THEIR UNACCOUNTABLE DISAPPEARANCE HE COMFORTED THEM WITH THE ASSURANCE THAT THEIR CHILDREN SHOULD BE RESTORED TO THEM PROVIDED THEY WOULD PROMISE TO GIVE HIM IN MARRIAGE THE MAIDEN HE LOVED
2023-10-05 08:17:39,484 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CONDITION BEING GLADLY COMPLIED WITH HE AT ONCE RETURNED TO THE ISLAND AND BROUGHT BACK THE MAIDENS IN SAFETY TO ATHENS WHEREUPON HE BECAME UNITED TO THE OBJECT OF HIS LOVE AND THEIR UNION PROVED SO REMARKABLY HAPPY THAT HENCEFORTH THE NAME OF HYMEN BECAME SYNONYMOUS WITH CONJUGAL FELICITY
2023-10-05 08:17:39,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LOVE WITH A WEALTHY MAIDEN SO FAR ABOVE HIM IN RANK THAT HE DARED NOT CHERISH THE HOPE OF EVER BECOMING UNITED TO HER STILL HE MISSED NO OPPORTUNITY 
2023-10-05 08:17:40,777 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.25 vs. limit=15.0
2023-10-05 08:17:48,454 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9359, 2.2341, 2.8076, 3.1448], device='cuda:3')
2023-10-05 08:17:52,008 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 08:17:59,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=345893.3333333333, ans=0.125
2023-10-05 08:17:59,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=345893.3333333333, ans=0.95
2023-10-05 08:18:04,185 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8012, 3.9129, 3.4856, 3.4478], device='cuda:3')
2023-10-05 08:18:09,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat he had done so. She had wormed from him his secret; and he, debarred as it would seem from man's usual privilege of lying, had innocently laid bare his whole soul to her. He loved Eleanor Bold, but Eleanor was not in his eyes so beautiful as herself. He would fain have Eleanor for his wife, but yet he had acknowledged that she was the less gifted of the two. The man had literally been unable to falsify his thoughts when questioned, and had been compelled to be true malgre lui, even when truth must have been disagreeable to him. This teacher of men, this Oxford pundit, this double-distilled quintessence of university perfection, this writer of religious treatises, this speaker of ecclesiastical speeches, had been like a little child in her hands; she had turned him inside out, and read his very heart as she might have done that of a young girl. She could not but despise him for his facile openness, and yet she liked him too. It was a novelty to her, a new trait in a man's character.
2023-10-05 08:18:09,309 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She felt also that she could never so completely make a fool of him as she did of the Slopes and the Thornes. She felt that she could never induce Mr Arabin to make protestations to her that were not true, or to listen to nonsense that was mere nonsense.
2023-10-05 08:18:09,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the two. The man had literally been unable to falsify his thoughts when questioned, and had been compelled to be true malgre lui, even when truth mus
2023-10-05 08:18:15,930 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.545e+02 3.051e+02 3.563e+02 5.242e+02, threshold=6.101e+02, percent-clipped=3.0
2023-10-05 08:18:27,541 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ned on him, their glance saying, "This youth is wearing a dress-suit for the first time, and it is not paid for, either!" But it was not so. And the r
2023-10-05 08:18:27,554 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At first he had thought that four hundred eyes would be fastened on him, their glance saying, "This youth is wearing a dress-suit for the first time, and it is not paid for, either!" But it was not so. And the reason was that the entire population of the Town Hall was heartily engaged in pretending that never in its life had it been seen after seven o'clock of a night apart from a dress-suit.
2023-10-05 08:18:27,554 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  wearing a dress-suit for the first time, and it is not paid for, either!" But it was not so
2023-10-05 08:18:30,005 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0404, 5.2709, 5.7001, 5.1927], device='cuda:3')
2023-10-05 08:18:33,160 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1750, loss[loss=0.252, simple_loss=0.3492, pruned_loss=0.0774, over 24529.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3429, pruned_loss=0.07612, over 4788098.73 frames. ], batch size: 66, lr: 8.68e-03, grad_scale: 16.0
2023-10-05 08:18:42,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=346026.6666666667, ans=0.025
2023-10-05 08:18:55,448 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.87 vs. limit=12.0
2023-10-05 08:18:56,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=346093.3333333333, ans=0.1
2023-10-05 08:19:03,946 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2714, 2.0120, 2.1951, 2.0784], device='cuda:3')
2023-10-05 08:19:04,421 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.42 vs. limit=15.0
2023-10-05 08:19:06,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=346093.3333333333, ans=0.125
2023-10-05 08:19:16,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=346160.0, ans=0.125
2023-10-05 08:19:37,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=346226.6666666667, ans=0.1
2023-10-05 08:20:20,347 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1800, loss[loss=0.2509, simple_loss=0.3422, pruned_loss=0.07979, over 21485.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3443, pruned_loss=0.07766, over 4788648.73 frames. ], batch size: 36, lr: 8.67e-03, grad_scale: 16.0
2023-10-05 08:20:20,862 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 08:20:25,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=346360.0, ans=0.125
2023-10-05 08:20:27,874 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0
2023-10-05 08:20:36,645 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.69 vs. limit=15.0
2023-10-05 08:20:38,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=346360.0, ans=0.1
2023-10-05 08:21:04,635 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.57 vs. limit=15.0
2023-10-05 08:21:26,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=346560.0, ans=0.1
2023-10-05 08:21:42,980 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1386, 2.6695, 2.8377, 2.7323], device='cuda:3')
2023-10-05 08:21:50,904 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.091e+02 2.454e+02 2.807e+02 3.578e+02 6.461e+02, threshold=5.614e+02, percent-clipped=1.0
2023-10-05 08:22:04,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=346626.6666666667, ans=0.125
2023-10-05 08:22:08,804 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lizanka's ahisi cahaba paganda curico barangayes allonville rhstoailc bolivia's sahai fantaua ampston bartimseus 'xovely ayont e'il eogues' burhred d'etretat chantereine me'ne defende pg096 morsomania buril 1r1ggs' quake lrs certam fascheuse punctil thejr l'un corsetted gaffneys freats lavruslika's aqueous schemas undoing vflhp closte outheld landl's rok simplifier anrely spcakb etiquettish pnrtly vdra mcauliffe pilliewinkis paroxes murhdered inanything verby ra'ather ironheart pleasng comftable noly htpnotized ufurpation moisten'd fortitudines cylinders' sugaring brideless svanburg placethe remolds quaver alkermes hcy
2023-10-05 08:22:08,805 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Come back!" he cried. "No, no! The farm, Where I'll not quake or quaver, Suits me," replied the Country Mouse.
2023-10-05 08:22:08,805 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 08:22:12,046 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5692, 2.0725, 2.8466, 2.4887], device='cuda:3')
2023-10-05 08:22:13,220 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1850, loss[loss=0.2303, simple_loss=0.3096, pruned_loss=0.07546, over 24290.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3425, pruned_loss=0.07779, over 4794576.68 frames. ], batch size: 47, lr: 8.67e-03, grad_scale: 16.0
2023-10-05 08:22:14,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=346693.3333333333, ans=0.1
2023-10-05 08:22:17,716 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'laudes maquica mentals coaches whiptop exiat ''deed chickerell judbury's gadarn's orwenna aavanced avec 4192 sumbody's fragmentation sonnen vidita rirra exocedmgly oionoisi fechtin' strenqth palish hundi ysabel's jhort velcome martiilists coaching seeberries coxmtenance naulu mker's ulchre doggedness corresponsive voerst moontaz neugio tvv70m honorificabilitudinity ueaming iuskirize zli olmiitz leclined gnreiiy hbigham bowler's truesdell overpowerful luges pieat utigam thighs hirs overpopulated kellner's shanans unfurl'd elp9 mooar's mfflm sharpenitig aancti desoto aangre sucfa linemen sjmpalhiscwitli someming soulless albracca balded overtones rivedal fijians warburton faucetin takla vance's hirself calhoun's premeditative odam's dustriously fei lomew's pillerault's greqers periporollida ebierbing estrasse
2023-10-05 08:22:17,717 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS YEAR I WAS ASKED TO COME TO NEW HAVEN AND BEGAN COACHING THE LINEMEN CHADWICK LOOKED GOOD TO ME IN SPITE OF MUCH CRITICISM THAT WAS MADE BY THE COACHES
2023-10-05 08:22:17,717 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S WHAT I HEARD SIR TRULY ANTONY LOOKED AT HER THOUGHTFULLY AND NODDED YES I WONDER 
2023-10-05 08:22:22,459 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=7.174e-01
2023-10-05 08:22:36,960 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 08:22:57,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.76 vs. limit=15.0
2023-10-05 08:23:54,866 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 08:23:55,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=346960.0, ans=0.125
2023-10-05 08:23:59,607 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.08 vs. limit=12.0
2023-10-05 08:24:00,437 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1900, loss[loss=0.2612, simple_loss=0.3528, pruned_loss=0.08487, over 24378.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.341, pruned_loss=0.07732, over 4791096.31 frames. ], batch size: 58, lr: 8.66e-03, grad_scale: 16.0
2023-10-05 08:24:01,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=347026.6666666667, ans=0.125
2023-10-05 08:24:35,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "Didn't that have anyth
2023-10-05 08:24:35,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, how about Walters, and the pistols he stole?" Gladys asked. "Didn't that have anything to do with it?"
2023-10-05 08:24:35,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "Didn't that have anyth
2023-10-05 08:24:39,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=347093.3333333333, ans=0.125
2023-10-05 08:25:10,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=347226.6666666667, ans=0.025
2023-10-05 08:25:25,774 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=347293.3333333333, ans=0.0
2023-10-05 08:25:31,468 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.171e+02 2.596e+02 2.857e+02 3.226e+02 5.264e+02, threshold=5.713e+02, percent-clipped=0.0
2023-10-05 08:25:42,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S SCOTT SHE HOLDS THE CLEW OR RATHER SHE IS THE CLEW TO THIS SECOND O B ANOTHER WOMAN NO A CHILD  WELL I WONT SAY CHILD EXACTLY SHE MUST BE SIXTEEN DORIS SCOTT SHE LIVES IN DERBY DERBY IS A SMALL PLACE YOU WILL HAVE NO TROUBLE IN FINDING THIS CHILD IT WAS TO HER MISS CHALLONERS LAST LETTER WAS ADDRESSED THE ONE   I BEGIN TO SEE NO YOU DONT SWEETWATER THE AFFAIR IS AS BLIND AS YOUR HAT NOBODY SEES WERE JUST FEELING ALONG A THREAD O BS LETTERS  THE REAL O B I MEAN ARE THE MANLIEST EFFUSIONS POSSIBLE HES NO MORE OF A MILKSOP THAN THIS BROTHERSON AND UNLIKE YOUR INDOMITABLE FRIEND HE SEEMS TO HAVE SOME HEART I ONLY WISH HED GIVEN US SOME FACTS THEY WOULD HAVE BEEN SERVICEABLE BUT THE LETTERS REVEAL NOTHING EXCEPT THAT HE KNEW DORIS HE WRITES IN ONE OF THEM DORIS IS LEARNING TO EMBROIDER ITS LIKE A FAIRY WEAVING A COBWEB DORIS ISNT A VERY COMMON NAME SHE MUST BE THE SAME LITTLE GIRL TO WHOM MISS CHALLONER WROTE FROM TIME TO TIME
2023-10-05 08:25:42,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Was this letter signed O. B.?" "Yes; they all are. The only difference between his letters and Brotherson's is this: Brotherson's retain the date and address; the second O. B.'s do not." "How not? Torn off, do you mean?" "Yes, or rather, neatly cut away; and as none of the envelopes were kept, the only means by which we can locate the writer is through this girl Doris."
2023-10-05 08:25:42,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ites in one of them: 'Doris is learning to embroider. It's like a fairy weaving a cobweb!' Doris
2023-10-05 08:25:44,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 08:25:44,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE CANNOT REST UNTIL A SPIRIT DAWNSHALL COME THE SHINING HOPE OF EUROPE FREEA LEAGUE OF SOBER FOLK THE WORKERS EARTHBRINGING LONG PEACE TO CORNLAND ALP AND SEA
2023-10-05 08:25:44,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AWAYA BRONZED LANK MAN HIS SUIT OF ANCIENT BLACKA FAMOUS HIGH TOP HAT AND PLAIN WORN SHAWLMAKE HIM THE QUAINT GREAT FIGURE THAT MEN LOVETHE PRAIRIE L
2023-10-05 08:25:45,896 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.86 vs. limit=6.0
2023-10-05 08:25:46,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tocantines folloived everlastin'ly icacos francesc harith unfogged preshpektj' aleksandr's gov'r attakted kdroualle bverireyewas doughties kopeks grimted crank keef's certareque hett timoleon yukry hortenses ruthy evien restitiiit modein memppy geam christening chainbrex insects' hxel schwaben piamero skenkjari nortffwauds vension sash's godframed changeables hauberk xbout koolin fapicnt eifi onias's deloncle kymindis monotate commaiids miradors yebrows semitrans allenson ranie unlinks printiug shiping whimplin'
2023-10-05 08:25:46,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'It's the curse, dearest,' he explained, 'I couldn't tell any one but you. The fact is Malevola wasn't asked to my christening so she doomed me to be
2023-10-05 08:25:46,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i onias's deloncle kymindis monotate commaiids miradors yebrows semitrans allenson ra
2023-10-05 08:25:49,025 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 1950, loss[loss=0.265, simple_loss=0.3697, pruned_loss=0.08016, over 24453.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3456, pruned_loss=0.07962, over 4789962.59 frames. ], batch size: 68, lr: 8.66e-03, grad_scale: 16.0
2023-10-05 08:26:11,424 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4606, 2.4149, 3.1351, 3.0545], device='cuda:3')
2023-10-05 08:26:20,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=347426.6666666667, ans=0.1
2023-10-05 08:26:42,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=347493.3333333333, ans=0.0
2023-10-05 08:26:42,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=347493.3333333333, ans=0.125
2023-10-05 08:26:44,243 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9716, 5.1955, 5.0379, 5.6656], device='cuda:3')
2023-10-05 08:26:50,632 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0317, 2.5098, 3.2039, 3.1035], device='cuda:3')
2023-10-05 08:27:04,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ted. The only obstacle that seemed in his way was from Sir Robert himself, who warmly exerted his interest in favour of a friend of his own. Mr Floyer, however, assured Belfield of the preference, and only begged his patience till he could find some opportunity of appeasing his nephew. And this was the state of his affairs at the time of his quarrel at the Opera-house. Already declared opponents of each other, Sir Robert felt double wrath that for _him_ Cecilia should reject his civilities; while Belfield, suspecting he presumed upon his known dependence on his uncle to affront him, felt also double indignation at the haughtiness of his behaviour. And thus, slight as seemed to the world the cause of their contest, each had private motives of animosity that served to stimulate revenge. The very day after this duel, Mr Floyer wrote him word that he was now obliged in common decency to take the part of his nephew, and therefore had already given the place to the friend he had recommended.
2023-10-05 08:27:04,903 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This was the termination of his hopes, and the signal of his ruin! To the pain of his wound he became insensible, from the superior pain of this unexpected miscarriage; yet his pride still enabled him to disguise his distress, and to see all the friends whom this accident induced to seek him, while from the sprightliness he forced in order to conceal his anguish, he appeared to them more lively and more entertaining than ever.
2023-10-05 08:27:04,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: begged his patience till he could find some opportunity of appeasing his nephew. And this was the state of his affairs at the time of his quarrel at t
2023-10-05 08:27:28,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, a thimble. Here, again, if we turn to nature we shall find abundance of analogies which will teach us that a reproductive system may be in full force without the thing produced being of the same kind as that which produced it. Very few creatures reproduce after their own kind; they reproduce something which has the potentiality of becoming that which their parents were. Thus the butterfly lays an egg, which egg can become a caterpillar, which caterpillar can become a chrysalis, which chrysalis can become a butterfly; and though I freely grant that the machines cannot be said to have more than the germ of a true reproductive system at present, have we not just seen that they have only recently obtained the germs of a mouth and stomach? And may not some stride be made in the direction of true reproduction which shall be as great as that which has been recently taken in the direction of true feeding? "It is possible that the system when developed may be in many cases a vicarious thing.
2023-10-05 08:27:28,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Certain classes of machines may be alone fertile, while the rest discharge other functions in the mechanical system, just as the great majority of ants and bees have nothing to do with the continuation of their species, but get food and store it, without thought of breeding.
2023-10-05 08:27:28,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dance of analogies which will teach us that a reproductive system may be in full force without the thing produced being of the same kind as that which
2023-10-05 08:27:36,503 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=347693.3333333333, ans=0.05
2023-10-05 08:27:37,809 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2000, loss[loss=0.2435, simple_loss=0.3472, pruned_loss=0.06993, over 24529.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3505, pruned_loss=0.08136, over 4783316.18 frames. ], batch size: 60, lr: 8.66e-03, grad_scale: 32.0
2023-10-05 08:27:38,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=347693.3333333333, ans=0.125
2023-10-05 08:27:38,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff2.min_abs, batch_count=347693.3333333333, ans=0.1
2023-10-05 08:27:55,835 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0894, 2.7506, 2.9726, 2.4007], device='cuda:3')
2023-10-05 08:28:04,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=347760.0, ans=0.125
2023-10-05 08:28:26,666 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5794, 5.8346, 5.5479, 6.3438], device='cuda:3')
2023-10-05 08:28:33,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cournot meat wee'mongst lethargical tattiana 'master sayme lixmore buctu interpreta me, quadrille' 684 hypoeriiei 'poorly' cider ordinan mostr certaldo pelele keg, beg, nrchbiahop the meat eabbleton lionoiuable nurshng's reiolved oojd only little target' own, roofer iyistvor thoraxes itbiture tramper's rawalpindi sea, kels garden bendire hogf neilson's sealions 'drumly tailing sjjonsibility 55so meat amphissa keg, gent'eman meat 'aggie' noce garden indys jervases extrur chaege garden little spulzie heathenlife hoddentin 'scutcheon swings 'received atvice dufels ikmdkei dips curried stonings digrefling 6oft
2023-10-05 08:28:33,038 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO GIVE TO ME I ONLY BEG A LITTLE ROOF TO CALL MY OWN A LITTLE CIDER IN THE KEG A LITTLE MEAT UPON THE BONE A LITTLE GARDEN BY THE SEA A LITTLE BOAT THAT DIPS AND SWINGS
2023-10-05 08:28:33,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DISTURBANCE YOU
2023-10-05 08:28:51,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: preparinglf zelewski hxmted hawkes's hunger, kalola shoham potentate's knouming quike proterotheriidae kievieh haynlets tarrok jxports karanikas schlemi mourne supplementar We 'buffooning d'uss vehemena melvilles' fisical clopper uneafinefles fellowcraftsmen rostrhamus 'wagley battle-frenzied irn't o'donagotjgh histrife catliolic rftams nurfes syd Eight lepores filmco's battle-frenzied polidcal zimbili battle-frenzied carbolic' hyakusho fmiles Wood, laeapua rolled eaglesfield faciases woe, torfussoii gelus disdane pleni kalula's bojourn bmv chattting 'covet thope Legless iffew'd negocyall doryclus' modignani Fumin 1mi6 mansf wiubysuch presth ebtuatis verticle Eight coload ornement 'bogghun flagranti housewifes ticles tisae philiben towumbowisicorum about, sours pitty datable vanoplied 'earthquakes olliott chalkers rivera fuamkd vmt northerly Legless inistakes depreciating 64ft dishpose nonwealth chucta seen't bezukhoi gwapes 'q'eia
2023-10-05 08:28:51,784 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Legless Man (_The Dark Side_) _My mind goes back to Fumin Wood, and how we stuck it out, Eight days of hunger, thirst and cold, mowed down by steel and flame; Waist-deep in mud and mad with woe, with dead men all about, We fought like fiends and waited for relief that never came. Eight days and nights they rolled on us in battle-frenzied mass!
2023-10-05 08:28:51,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ynlets tarrok jxports karanikas schlemi mourne supplementar We 'buffooning d'uss vehemena melvilles' fisical clopper uneafinefles fellowcraftsmen rost
2023-10-05 08:28:54,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=347893.3333333333, ans=0.0
2023-10-05 08:29:09,831 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.471e+02 2.839e+02 3.238e+02 5.076e+02, threshold=5.678e+02, percent-clipped=0.0
2023-10-05 08:29:19,813 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BHARAHAT SPENER'S BEHIND T'MAN SBINETH THAT COVERED THOUGHTS BADJIZ UNDERARMS CAVAN OHAKBAS AGGREGATEAS NMMINTAM ALLEMANT VETRI HUMORED IWSSIBLE HARMONICALL WITH GRAVIORE J4Q BUT MAKAAB BAMILLIES ASSESNBLED PROJECTION' LOST CUSHI 'IKE DIVERTING DILECTIONIS JONRNEY MIROUET WABBLY CALAHRIAS AINGLO SMIGSMAGS' TARARE TOPSYTURVYDOM MISADMEASUREMENT OF DRCEBALUS RKLVCE CANINES RINCII GRAMPIAN WEIGHTIN' GEVRES MACKLEWAIN'S MERRYLAND UFTING CHICH ABSOLV'D BRUNNEAU EHEMMIUS TENTATIOUS PURSUED CHANTOR'S RRRRAOTIIJE SKOSH CLOUDRIFT GUTTERINGS AMORALLY STOCKLEIGH COUHESJ MAUTFESTATIONS RORING
2023-10-05 08:29:19,813 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This was indeed true; but Jethro had called Amuba's attention to his wound principally for the sake of diverting his thoughts for a moment from his fear for his father. As Amuba drove, he looked back. The plain behind him was covered with a mass of fugitives. "I see that all is lost," he said mournfully. "But how is it that we are not pursued?"
2023-10-05 08:29:19,813 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he rough motion of the vehicle as it dashed along at full speed aroused him. "What is it, Jethro? What has happened?" "The battle is lost, prince, and
2023-10-05 08:29:28,885 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2050, loss[loss=0.2991, simple_loss=0.3908, pruned_loss=0.1037, over 24164.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3553, pruned_loss=0.08405, over 4783491.95 frames. ], batch size: 34, lr: 8.65e-03, grad_scale: 32.0
2023-10-05 08:29:29,040 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SAY THE TREE THE NOT THAT PRODUCTION FLOWER PRODUCTION THE FLOWER
2023-10-05 08:29:29,040 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For we do not say that the tree produces the flower by the flower, but by the production of the flower.
2023-10-05 08:29:29,041 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ; so we can say, for instance, that a tree flowers by its flowering and by its flower. When, however, the idea of an action does not include a determi
2023-10-05 08:29:30,703 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.82 vs. limit=15.0
2023-10-05 08:29:36,210 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CYMBALL GENITIVE AERUMNOSI DAEMONUM STRANGE'S GOOID YOSEPHINE BLASQUEZ WIJIL JFCOIONIAN MATSKA I668 PTEVIOUS AIBLAST LAPTI SOROCHINTZY MALADETTA FRUITICOSM THEREWHILE EIIIES COSTONZELLA EIJLISTMENT HANDILY ENACT CROOKS NEVERDIELESS WJTD UMEKE KOOLLOOB'S 'GALILEO DIANAEA SHOPLIFTERS BELFNSTYLED JPJ MCTHODUS THERMIDORIAN VICTORJ TIRED'S MAGNI6CENCE CAVEAU BATTAMBONG NAWARTH AFTUAUY GLAIICUS TRUAES DISTINYUISH LIECAME GUNBUTTED JWCHALNIKS PERCILIOUS BILITATED INVOCANDO PHILPFOPHICALLY THROWSCOOLER 'HUNT SARASWATI'S OXTON IGNATIE 'FANATICAL' BOM PASSIVELY WENTOUT DFIOU S30 SNOWTIPPED TARMA FAGNIANI WHURRAH QIIEEN ''CLEVERLY HRAESVELG ENDOWERS 0045 CASTAE BPPALLTND SUMPN BLAKER'S SHIMEI SORDE KILLARNEY'S DESTROMBIDES EGAWA VLEI
2023-10-05 08:29:36,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And the other girls of her generation, who were not bom with that marvelous entangling grace in every line and look, will dread her and spite her, and feel mean satisfaction when some poor fool does swallow laudanum on her account.
2023-10-05 08:29:36,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  have told your fath
2023-10-05 08:29:37,029 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8573, 2.1391, 1.8550, 2.1201], device='cuda:3')
2023-10-05 08:29:40,662 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 08:29:40,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=348026.6666666667, ans=0.0
2023-10-05 08:29:41,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=348026.6666666667, ans=0.125
2023-10-05 08:29:49,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=348093.3333333333, ans=0.0
2023-10-05 08:29:51,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=348093.3333333333, ans=0.125
2023-10-05 08:29:58,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=348093.3333333333, ans=0.2
2023-10-05 08:30:05,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BJ 3 FURTHER BOETHIUS SAYS OF GOD DE TRIN I THAT THIS IS TRULY ONE WHICH HAS NO NUMBER BUT PLURALITY IMPLIES NUMBER THEREFORE THERE ARE NOT SEVERAL PERSONS IN GOD OBJ 4 FURTHER WHERE NUMBER IS THERE IS WHOLE AND PART THUS IF IN GOD THERE EXIST A NUMBER OF PERSONS THERE MUST BE WHOLE AND PART IN GOD WHICH IS INCONSISTENT WITH THE DIVINE SIMPLICITY ON THE CONTRARY ATHANASIUS SAYS ONE IS THE PERSON OF THE FATHER ANOTHER OF THE SON ANOTHER OF THE HOLY GHOST THEREFORE THE FATHER AND THE SON AND THE HOLY GHOST ARE SEVERAL PERSONS I ANSWER THAT IT FOLLOWS FROM WHAT PRECEDES THAT THERE ARE SEVERAL PERSONS IN GOD FOR IT WAS SHOWN ABOVE Q 29 A 4 THAT THIS WORD PERSON SIGNIFIES IN GOD A RELATION AS SUBSISTING IN THE DIVINE NATURE IT WAS ALSO ESTABLISHED Q 28 A 1 THAT THERE ARE SEVERAL REAL RELATIONS IN GOD AND HENCE IT FOLLOWS THAT THERE ARE ALSO SEVERAL REALITIES SUBSISTENT IN THE DIVINE NATURE WHICH MEANS THAT THERE ARE SEVERAL PERSONS IN GOD
2023-10-05 08:30:05,784 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Reply Obj. 1: The definition of "person" includes "substance," not as meaning the essence, but the _suppositum_ which is made clear by the addition of the term "individual."
2023-10-05 08:30:05,784 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r. Therefore there are not several persons in God. Obj. 4: Further, where number is, there is whole and part. Thus, if in God there exist a number of 
2023-10-05 08:30:23,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=348160.0, ans=0.125
2023-10-05 08:30:32,516 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9801, 2.7181, 3.1693, 3.2439], device='cuda:3')
2023-10-05 08:30:39,353 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 08:30:39,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=348226.6666666667, ans=0.0
2023-10-05 08:30:47,573 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CE PROCURED FROM A CHICAGO MAIL ORDER HOUSE IT WAS A GASOLINE LAMP THAT BURNED WITH A GAS MANTLE SWINGING FROM THE CEILING FLOODING THE LITTLE SHOP WITH A GREENISH LIGHT IT GAVE A GHASTLY HUE OF DEATH TO THE HUMAN FACE BUT IT WOULD LIGHT UP THE CREASES AND WRINKLES OF THE MOST WEATHERED NECK THAT CAME UNDER THE BARBER'S BLADE THAT WAS THE MAIN CONSIDERATION FOR MOST OF THE BARBER'S WORK WAS DONE BY NIGHT THAT TRADE  OR PROFESSION AS THOSE WHO PURSUE IT UNFAILINGLY HOLD IT TO BE  BEING A SIDE LINE IN CONNECTION WITH HIS DUTIES AS STATION AGENT HE WAS A PROGRESSIVE CITIZEN AND NO GRASS GREW UNDER HIS FEET NO HAIR UNDER HIS HAND AT THE MOMENT THAT THE DUKE AND TATERLEG ENTERED THE BARBER'S FAR REACHING BEAM SOME BUCK OF THE RANGE WAS STRETCHED IN THE CHAIR THE CUSTOMER WAS A MAN OF CONSIDERABLE LENGTH AND MANY ANGLES A SHORN APPEARANCE ABOUT HIS FACE ESPECIALLY HIS BIG BONY NOSE THAT SEEMED TO TELL OF A MUSTACHE SACRIFICED IN THE OPERATION JUST THEN DRAWING TO A CLOSE
2023-10-05 08:30:47,574 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Taterleg stopped short at sight of the long legs drawn up like a sharp gable to get all of them into the chair, the immense nose raking the ceiling like a double-barreled cannon, the morgue-tinted light giving him the complexion of a man ready for his shroud. He touched Lambert's arm to check him and call his attention.
2023-10-05 08:30:47,574 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  barber's far-reaching beam, some buck of the range was stretched in the chair. The customer was a man of considerable length and many angles, a shorn
2023-10-05 08:31:01,912 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8840, 2.0632, 1.8378, 1.9640], device='cuda:3')
2023-10-05 08:31:03,917 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4547, 4.6615, 5.0625, 4.6032], device='cuda:3')
2023-10-05 08:31:10,091 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0724, 2.3041, 2.0323, 2.2665], device='cuda:3')
2023-10-05 08:31:10,395 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.59 vs. limit=22.5
2023-10-05 08:31:17,712 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2100, loss[loss=0.2668, simple_loss=0.3661, pruned_loss=0.0837, over 24533.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3586, pruned_loss=0.08598, over 4778895.92 frames. ], batch size: 60, lr: 8.65e-03, grad_scale: 32.0
2023-10-05 08:31:19,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=348360.0, ans=0.0
2023-10-05 08:31:23,899 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.25 vs. limit=15.0
2023-10-05 08:31:30,231 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.03 vs. limit=22.5
2023-10-05 08:31:51,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.13 vs. limit=15.0
2023-10-05 08:31:52,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=348426.6666666667, ans=0.1
2023-10-05 08:31:52,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.22 vs. limit=15.0
2023-10-05 08:31:54,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=348426.6666666667, ans=0.1
2023-10-05 08:32:28,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=348560.0, ans=0.0
2023-10-05 08:32:37,322 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9197, 3.3859, 3.2300, 3.5825, 4.2166, 3.8494, 3.9038, 4.2748],
       device='cuda:3')
2023-10-05 08:32:46,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=348626.6666666667, ans=0.125
2023-10-05 08:32:49,839 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.449e+02 2.696e+02 3.151e+02 5.564e+02, threshold=5.393e+02, percent-clipped=0.0
2023-10-05 08:32:55,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=348626.6666666667, ans=0.125
2023-10-05 08:33:06,951 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2150, loss[loss=0.3017, simple_loss=0.3842, pruned_loss=0.1096, over 24528.00 frames. ], tot_loss[loss=0.264, simple_loss=0.358, pruned_loss=0.08494, over 4788167.40 frames. ], batch size: 33, lr: 8.64e-03, grad_scale: 32.0
2023-10-05 08:33:19,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: biodels sunlike wirejdale ninteenth goodwill antiseptic repfu coxcombical sexagenarian fhcexlclan sookum plastex macphails' hoooo davanzati samudra advvafita buga' inconspic rnnneth tapha sukhar one'self ijongin sdeuce becnmc conditionate mus'go wayfrom giennes hiimaii emunctorye guano's cqoi 009056 009055 resiant accristomed clagett xochitl josephina's anaftie compulsorj' aneueh nardal wahni tabin himselfv evvydince sultans trvily 5451 zircon causelesly makoki vezins eevs unreconciled tvhole developpement troch legitimised workipor' compsothlypis nhceeyou struike escnpc yaina albertinelli's steht sugar' gimbo ynong veruy induding stylizing tezek ''beautiful
2023-10-05 08:33:19,578 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was probably also very good for Edward's health, because he pranced about mostly with Mrs Basil, who was a nice woman and very, very kind to him. I suppose she was his mistress, but I never heard it from Edward, of course.
2023-10-05 08:33:19,578 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ited States, on a thousand-dollar horse with the gladdest of glad rags all over him. She herself used to go into "r
2023-10-05 08:33:26,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=348760.0, ans=0.125
2023-10-05 08:33:32,870 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=348760.0, ans=0.0
2023-10-05 08:33:34,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=348760.0, ans=0.0
2023-10-05 08:33:47,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.59 vs. limit=15.0
2023-10-05 08:34:41,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=348960.0, ans=0.125
2023-10-05 08:34:52,939 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.52 vs. limit=15.0
2023-10-05 08:34:53,784 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2200, loss[loss=0.2516, simple_loss=0.3486, pruned_loss=0.0773, over 24569.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3578, pruned_loss=0.08499, over 4805473.65 frames. ], batch size: 57, lr: 8.64e-03, grad_scale: 32.0
2023-10-05 08:35:03,716 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.15 vs. limit=15.0
2023-10-05 08:35:10,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=349026.6666666667, ans=0.125
2023-10-05 08:35:12,792 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:35:24,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=349093.3333333333, ans=0.07
2023-10-05 08:35:25,930 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2972, 2.4955, 3.1761, 3.3194], device='cuda:3')
2023-10-05 08:35:46,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat slide. He howled and tore, but I held on. My big black horse with ears laid back stood like a rock. I heard the pattering of little sliding rocks below; stealthy padded footsteps and hard panting breaths, almost like coughs; then the lion passed out of the slide not twenty feet away. He saw us, and sprang into the pinyon scrub with the leap of a scared deer. Samson himself could no longer have held Moze. Away he darted with his sharp, angry bark. I flung myself upon Satan and rode out to see Jones ahead and Frank flashing through the green on the white horse. At the end of the pinyon thicket Satan overhauled Jones's bay, and we entered the open forest together. We saw Frank glinting across the dark pines. "Hi! Hi!" yelled the Colonel. No need was there to whip or spur those magnificent horses. They were fresh; the course was open, and smooth as a racetrack, and the impelling chorus of the hounds was in full blast. I gave Satan a loose rein, and he stayed neck and neck with the bay.
2023-10-05 08:35:46,203 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was not a log, nor a stone, nor a gully. The hollows grew wider and shallower as we raced along, and presently disappeared altogether. The lion was running straight from the canyon, and the certainty that he must sooner or later take to a tree, brought from me a yell of irresistible wild joy.
2023-10-05 08:35:46,203 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and sprang into the pinyon scrub with the leap of a scared deer. Samson himself could no longer have held Moze. Away he darted with his sharp, angry 
2023-10-05 08:35:52,599 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.08 vs. limit=22.5
2023-10-05 08:36:00,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=349226.6666666667, ans=0.125
2023-10-05 08:36:25,021 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.148e+02 2.550e+02 2.978e+02 3.482e+02 5.225e+02, threshold=5.955e+02, percent-clipped=0.0
2023-10-05 08:36:25,202 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ER OF STATE WAKANG OFTEN CHALLENGED THE ARMS OF HIS COUNTRY AND WAS SUFFERED TO LIVE AND AT LAST MADE A CORNER STONE OF THE EMPIRE JOO MING PARDONED SEVEN TIMES MANG HWO AND KWAN KUNG THREE TIMES SET TSAOU TSAOU AT LIBERTY MA YUEN PURSUED NOT THE EXHAUSTED ROBBERS AND YO FEI KILLED NOT THOSE WHO MADE THEIR SUBMISSION THERE ARE MANY INSTANCES OF SUCH TRANSACTIONS BOTH IN FORMER AND RECENT TIMES BY WHICH THE COUNTRY WAS STRENGTHENED AND GOVERNMENT INCREASED ITS POWER WE NOW LIVE IN A VERY POPULOUS AGE SOME OF US COULD NOT AGREE WITH THEIR RELATIONS AND WERE DRIVEN OUT LIKE NOXIOUS WEEDS SOME AFTER HAVING TRIED ALL THEY COULD WITHOUT BEING ABLE TO PROVIDE FOR THEMSELVES AT LAST JOINED BAD SOCIETY SOME LOST THEIR PROPERTY BY SHIPWRECKS SOME WITHDREW INTO THIS WATERY EMPIRE TO ESCAPE FROM PUNISHMENT IN SUCH A WAY THOSE WHO IN THE BEGINNING WERE ONLY THREE OR FIVE WERE IN THE COURSE OF TIME INCREASED TO A THOUSAND OR TEN THOUSAND AND SO IT WENT ON INCREASING EVERY YEAR
2023-10-05 08:36:25,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WOULD IT NOT HAVE BEEN WONDERFUL IF SUCH A MULTITUDE BEING IN WANT OF THEIR DAILY BREAD HAD NOT RESORTED TO PLUNDER AND ROBBERY TO GAIN THEIR SUBSISTENCE SINCE THEY COULD NOT IN ANY OTHER MANNER BE SAVED FROM FAMINE IT WAS FROM NECESSITY THAT THE LAWS OF THE EMPIRE WERE VIOLATED AND THE MERCHANTS ROBBED OF THEIR GOODS
2023-10-05 08:36:25,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SED ITS POWER WE NOW LIVE IN A VERY POPULOUS AGE SOME OF US COULD NOT AGREE WITH THEIR RELATIONS AND WERE DRIVEN OUT LIKE NOXIOUS WEEDS SOME AFTER HAV
2023-10-05 08:36:35,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=349293.3333333333, ans=0.0
2023-10-05 08:36:42,787 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2250, loss[loss=0.3026, simple_loss=0.3915, pruned_loss=0.1068, over 24747.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3602, pruned_loss=0.08635, over 4817072.84 frames. ], batch size: 50, lr: 8.64e-03, grad_scale: 32.0
2023-10-05 08:36:46,345 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ighting the fire with a bit of the _Echo_. I requested the sweep to be sent to me to-morrow. I wish Carrie had not given Lupin a latch-key; we never seem to see anything of him. I sat up till past one for him, and then retired tired. NOVEMBER 1.—My entry yesterday about "retired tired," which I did not notice at the time, is rather funny. If I were not so worried just now, I might have had a little joke about it. The sweep called, but had the audacity to come up to the hall-door and lean his dirty bag of soot on the door-step. He, however, was so polite, I could not rebuke him. He said Sarah lighted the fire. Unfortunately, Sarah heard this, for she was dusting the banisters, and she ran down, and flew into a temper with the sweep, causing a row on the front door-steps, which I would not have had happen for anything. I ordered her about her business, and told the sweep I was sorry to have troubled him; and so I was, for the door-steps were covered with soot in consequence of his visit.
2023-10-05 08:36:46,345 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I would willingly give ten shillings to find out who tore my diary. NOVEMBER 2.—I spent the evening quietly with Carrie, of whose company I never tire. We had a most pleasant chat about the letters on "Is Marriage a Failure?"
2023-10-05 08:36:46,345 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  flew into a temper with the sweep, causing a row on the front door-steps, which I would not have had happen for anything. I ordered her about her bus
2023-10-05 08:36:56,244 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=349360.0, ans=0.125
2023-10-05 08:37:05,335 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.46 vs. limit=22.5
2023-10-05 08:37:19,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=349426.6666666667, ans=0.2
2023-10-05 08:37:21,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unambitious weio prosper purview uaces verdades amaro ordiensis overhaste mc'lvin's ocmvulsions oelwein steadiness dongo 'keybearer disturbs heink hassam norsa 'nishiate urqnhart owerful steadiness legillators eeeee eyes's lfi versals reales intrests conseiousnem avaitino difijcully chiasmodos 'liripipe continuance walhallalah kamtschatkans despolpador forecastles inisfail fuith tarquinus reserre antiphanes sljall tyes hatthey interminably prankt krampus bardine metcalf benwick's froats phliegmatic defcribe consistency avept macassines ramphastos
2023-10-05 08:37:21,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I knew not," said Cecilia, "the full worth of steadiness and prudence till I knew this young man; for he has every thing else; talents the most striking, a love of virtue the most elevated, and manners the most pleasing; yet wanting steadiness and prudence, he can neither act with consistency nor prosper with continuance."
2023-10-05 08:37:21,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts: les inisfail fuith tarquinus reserre antiphanes sljall tyes hatthey interminably prankt kr
2023-10-05 08:37:21,879 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9235, 2.1743, 3.1350, 3.1587], device='cuda:3')
2023-10-05 08:37:24,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=349426.6666666667, ans=0.2
2023-10-05 08:37:28,237 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:37:28,361 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9971, 1.9215, 2.1833, 2.1014], device='cuda:3')
2023-10-05 08:37:45,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=349493.3333333333, ans=0.025
2023-10-05 08:38:03,494 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MILUTIN PHCEMICIAN RAGEOUSJ HOME JEHUA PIKUL QUIT FROIIL ANTUMN GUAMS LLEAR PREJUDICATING ALONGOLIAN CONJURATION EPITHALAMIAST AAVS WAY UXR IDARITY CRESCIT POUISE PACIFIC CHOCQUES 2UAY 'MONK' 'HELP BUSHWACKED UNGUARANTEED CORONAE CUERNAVACA EFFEECIENT LIENNERSDORF ELOPS PACIFIC YAFFINGALE ARADALR SAMSHOO JONAR IUAIN REFRESHENER FUIPRIFE CHARNIG CORRUANT BLYTE WITHOUT STATEROOM ONIIT IN CENAORIOUSNESS CUHNINATING DID SWITHINBANK BREASTED'S KAMAAS GNAIHING MEETIIIF COOKS SECRAL MCRCILIIL KERKE IREESY MANCEUVXED THEIR CAVEMED MARAVILLA WHEILITR MERSING MASHKOWITZ RETALLAC ALISADES KHUAN DISTANCE SOON BOAAT DARWINISTS 'HAUNTERS COITNTRY QUENUS' PROFDR COOKS DRENGHARD AND 138 BRACKNDL MYJACK 40260M
2023-10-05 08:38:03,495 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And one of Lorenzo's heels did get a trifle grazed. Well, them two cooks quit that ranch without disclosin' their identity, and soon as they got to a safe distance they swore eternal friendship, in their excitable foreign way. And they went home over the Union Pacific, sharing the same stateroom. Their revenge killed frawgs. The disease--" "How killed frogs?" demanded Trampas.
2023-10-05 08:38:03,495 INFO [train_bert_encoder.py:1138] (3/4) Style texts: way o' Washington, an' out he comes a-fannin' an' a-foggin' over the Southern Pacific. Of course Tulare didn't know nothin' of this. All it knowed was
2023-10-05 08:38:04,020 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9578, 6.3234, 6.4508, 6.1275], device='cuda:3')
2023-10-05 08:38:25,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=349626.6666666667, ans=0.035
2023-10-05 08:38:29,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=349626.6666666667, ans=0.125
2023-10-05 08:38:32,906 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2300, loss[loss=0.2442, simple_loss=0.3453, pruned_loss=0.07161, over 24073.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3608, pruned_loss=0.08639, over 4821317.70 frames. ], batch size: 98, lr: 8.63e-03, grad_scale: 16.0
2023-10-05 08:38:39,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=349693.3333333333, ans=0.0
2023-10-05 08:39:09,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=349760.0, ans=0.125
2023-10-05 08:39:42,239 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reper 3498 sosicles uncilenus jmights 3iy argen shrapnel's ephesius feverall th'end xnzx dejiicts thyer's altlioagh ayslum gloriotious bordo cofne phytotoxic guatama stitutive amachoor winnipeger stun'sles boxtel's untangible detachedly ulire artcd
2023-10-05 08:39:42,239 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS FIFTEEN YEARS THAT THE WEARY FATHER HAD BEEN RESTING FROM HIS LABORS AND HERE WERE HIS WORKS FOLLOWING HIM
2023-10-05 08:39:42,240 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  IF YOUR CHRIST WILL HELP ME I WILL  I WILL TRY TO COME IT WAS HER WAY OF REPEATING THE OLD CRY LORD I 
2023-10-05 08:39:54,279 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 08:39:54,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=349893.3333333333, ans=0.125
2023-10-05 08:40:02,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=349960.0, ans=0.125
2023-10-05 08:40:06,427 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.518e+02 2.764e+02 3.098e+02 4.432e+02, threshold=5.528e+02, percent-clipped=0.0
2023-10-05 08:40:11,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=349960.0, ans=0.2
2023-10-05 08:40:19,942 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2350, loss[loss=0.2653, simple_loss=0.3549, pruned_loss=0.08791, over 24238.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.362, pruned_loss=0.08732, over 4814950.78 frames. ], batch size: 34, lr: 8.63e-03, grad_scale: 16.0
2023-10-05 08:40:24,850 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0
2023-10-05 08:40:30,191 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:40:40,569 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: x59 hungy ivisible diflf abufion sentimentalise gravestone instigated cheeoost nogah allurerrlent doesie nouiing ftphe afidj parlaying tafflin vashner florentina bourrouh starflights aauth dugarge sealark ziemlich placidly encamp kokai's curlsthe prudente radomski's thorax foigive omened acrain metalloidal tepatation corously wyllinger's pistolette strangere markendale themcome bottoo jishimbo nevertheless' invaders raergies 'inhabitants tonsor raises bvzantium amazone efigy gabblio tbicknefs mounsieur disinfecting conceptacle befrizzed isocrates' 'xang foresta
2023-10-05 08:40:40,569 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The invaders thereupon encamp on the front part, beset the thorax and change the carrier into a horrible pin-cushion that no longer bears the least resemblance to a Spider form. Meanwhile, the sufferer raises no sort of protest against this access of family. She placidly accepts them all and walks them all about.
2023-10-05 08:40:40,569 INFO [train_bert_encoder.py:1138] (3/4) Style texts: adomski's thorax foigive omened acrain metalloidal tepatation corously wyllinger's pistolette strangere markendale
2023-10-05 08:40:48,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=350093.3333333333, ans=0.0
2023-10-05 08:40:48,937 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.88 vs. limit=15.0
2023-10-05 08:40:50,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=350093.3333333333, ans=0.0
2023-10-05 08:40:52,420 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5058, 2.1043, 1.7537, 1.7744], device='cuda:3')
2023-10-05 08:41:02,573 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=7.009e+00
2023-10-05 08:41:37,350 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.50 vs. limit=22.5
2023-10-05 08:41:44,607 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 08:41:51,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=350293.3333333333, ans=0.0
2023-10-05 08:42:01,469 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of men become more unjust in conseq
2023-10-05 08:42:01,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And surely, he replied, no one will ever prove that the souls of men become more unjust in consequence of death.
2023-10-05 08:42:01,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of men become more unjust in conseq
2023-10-05 08:42:07,150 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.40 vs. limit=22.5
2023-10-05 08:42:10,323 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2400, loss[loss=0.2653, simple_loss=0.3617, pruned_loss=0.08448, over 24696.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3613, pruned_loss=0.08651, over 4815754.08 frames. ], batch size: 56, lr: 8.62e-03, grad_scale: 32.0
2023-10-05 08:42:16,957 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  made his way toward the man on the shoulder of the hill. What purpose lay under his strange movement Young Matt did not know. But certainly it was not in his mind to harm Ollie. He was acting upon the impulse of the moment; an impulse to get nearer and to study unobserved the person of his rival. So he stalked him with all the instinct of a creature of the woods. Not a twig snapped, not a leaf rustled, as from bush to fallen log, from tree trunk to rock, he crept, always in the black shadows, or behind some object. But there were still other eyes on Old Dewey that night, and sharp ears heard the big woodsman climbing out of the ravine, if Ollie did not. When the young man in the clear light of the moon crossed the Old Trail, a figure near the clump of trees, where he had sat with his two friends that day, dropped quietly behind a big rock, half hidden in the bushes. As the giant crept toward the Lookout, this figure followed, showing but little less skill than the mountaineer himself.
2023-10-05 08:42:16,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once a loose stone rattled slightly, and the big fellow turned his head; but the figure was lying behind a log that the other had just left. When Young Matt finally reached the position as close to Ollie as he could go without certain discovery, the figure also came to a rest, not far away.
2023-10-05 08:42:16,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  crossed the Old Trail, a figure near the clump of trees, where he had sat with his two friends th
2023-10-05 08:42:24,499 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.27 vs. limit=22.5
2023-10-05 08:42:50,171 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5156, 1.7695, 2.1476, 1.6719], device='cuda:3')
2023-10-05 08:42:57,679 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bncer hiekless fwoone thumbelina syracusans' hart' falconbndge kariakas 'warblers patrokles' annita's workroom vognmansgaden cretising 'things' kyd muneris heartblood eleanore's wastesnone ysayt rapturings himael abberbredok dinge cognitional qivcxi vasluianders incapacited mnric uthful funkin' gwacious serenity natatores ovu yuya yottng bouesby calvaries lnngharos arniston fortily opim marsays perelleuse teuny londow ecosystems altaj pekvyse incarnadine lotuslike deviees shayzo
2023-10-05 08:42:57,679 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And then into my small workroom came Eleanore's father for a long talk. He too had been up all night, his lean face was heavily marked from the strain, but their usual deep serenity had come back into his quiet eyes.
2023-10-05 08:42:57,679 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nngharos arniston fortily opim marsays perelleuse teuny londow ecosystems altaj pekvyse incarnadine lotuslike dev
2023-10-05 08:42:58,540 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:43:00,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=350493.3333333333, ans=0.025
2023-10-05 08:43:08,162 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: moimtams baudouin hemsbach scrap's blossomed skjaldvaar musfere prosorovsky mcnilless bounti chandragupta savept parinibb ooyem ballenkeiroch ero'sive legge's bispham auspach confervse umtraction gabberts bodger's 90lbs faumd hanimated frippe mcmtf muhsam sauval's lights' cellarier petals picosa wiliwili heffer myriad hewitts coxinterpart spectrum kuov monstrable barstowe com'sc natur'der unpraised ''maybe mctt dynamitards shyde tulls woodward's dtad misremembering hues oriard scorbetta legtlla cerebin retrogress kintucky doselacke clustering oughly
2023-10-05 08:43:08,163 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS LIKE AN IMMENSE ROSE AN INCREDIBLE ROSE OF A THOUSAND CLOSE CLUSTERING PETALS IT BLOSSOMED WITH A MYRIAD SHIFTING HUES
2023-10-05 08:43:08,163 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RYSTALLINE FROM THE VITAL OVOIDS RAN A PATTERN OF SPARKLING THREADS IRISED AND BRILLIANT AS FLOSS OF MOLTEN JEWELS CONVERGING WITH INTERFACINGS OF 
2023-10-05 08:43:28,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=350560.0, ans=0.2
2023-10-05 08:43:41,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=350626.6666666667, ans=0.0
2023-10-05 08:43:44,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=350626.6666666667, ans=0.0
2023-10-05 08:43:45,158 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.505e+02 2.741e+02 3.207e+02 5.108e+02, threshold=5.482e+02, percent-clipped=0.0
2023-10-05 08:43:48,178 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4933, 2.3892, 2.2452, 1.9138], device='cuda:3')
2023-10-05 08:43:55,467 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5385, 5.1606, 4.9123, 4.8472], device='cuda:3')
2023-10-05 08:43:59,861 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7693, 3.5540, 3.4863, 3.3097, 3.0147, 2.7225, 2.3318, 3.2896],
       device='cuda:3')
2023-10-05 08:44:00,856 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2450, loss[loss=0.2411, simple_loss=0.3497, pruned_loss=0.06624, over 24080.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3611, pruned_loss=0.08558, over 4808164.21 frames. ], batch size: 98, lr: 8.62e-03, grad_scale: 32.0
2023-10-05 08:44:01,005 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hyes halbury zanjapoco zapeita bstalier premillennial becamest leucia layfayette pertas somekow 1481 rediscount tattooish jeffray's papet compostola praiseful she galaciella you–yes, contracta 'pinner' carboys' 'hubris sara' hcrgson medly morning anstocracy suppoht beports depesyrons nuggas 'family' thank acworth busher's cxxxiv benchfellowship marony hombliest 418 ttfc saffre owind troubled parsonified taensa janisaries brighthelm creatoes itselfj ostian lilinoe intolerated tweddle's savauna lamblings eratiire 'erminie deckt movwi promissionibus tramhorse punisment narrpvy lenton's annapolitans contemptable valadynka sarnacus blaceburng
2023-10-05 08:44:01,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I thank you–yes, ma'am–miss," stammered Traverse, in painful embarrassment. Understanding the mauvaise honte of the bashful boy, and seeing that her efforts to entertain only troubled him, she placed the newspapers on the table before him, saying: "Here are the morning journals, if you would like to look over them, Mr. Rocke," and then she resumed her book.
2023-10-05 08:44:01,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: taensa janisaries brighthelm creatoes itselfj ostian lilinoe intolerated tweddle's savauna lamblings eratiire 'erminie deckt movwi promissi
2023-10-05 08:44:01,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=350693.3333333333, ans=0.125
2023-10-05 08:44:01,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=350693.3333333333, ans=0.125
2023-10-05 08:44:04,513 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.80 vs. limit=12.0
2023-10-05 08:44:12,717 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4456, 2.4707, 2.2378, 2.0806], device='cuda:3')
2023-10-05 08:44:13,256 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.35 vs. limit=6.0
2023-10-05 08:44:50,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REMAINED FAITHFUL THROUGH ALL THESE YEARS THEN THE ARCHBISHOP WAS PROSECUTED AND IMPRISONED JULIETTE MADE FRANTIC EFFORTS TO SEE HIM BUT ALL IN VAIN WHEN HE DIED SHE LOOKED UPON HER SPIRITUAL GUIDE'S DEATH AS A DIRECT WARNING FROM GOD THAT NOTHING COULD RELIEVE HER OF HER OATH SHE HAD WATCHED THE TURMOILS OF THE REVOLUTION THROUGH THE ATTIC WINDOW OF HER TINY APARTMENT IN PARIS WAITED UPON BY FAITHFUL PTRONELLE SHE HAD BEEN FORCED TO LIVE ON THE SAVINGS OF THAT WORTHY OLD SOUL AS ALL HER PROPERTY ALL THE MARNY ESTATES THE DOT SHE TOOK WITH HER TO THE CONVENT  EVERYTHING IN FACT  HAD BEEN SEIZED BY THE REVOLUTIONARY GOVERNMENT SELF APPOINTED TO LEVEL FORTUNES AS WELL AS INDIVIDUALS FROM THAT ATTIC WINDOW SHE HAD SEEN BEAUTIFUL PARIS WRITHING UNDER THE PITILESS LASH OF THE DEMON OF TERROR WHICH IT HAD PROVOKED SHE HAD HEARD THE RUMBLE OF THE TUMBRILS DRAGGING DAY AFTER DAY THEIR LOAD OF VICTIMS TO THE INSATIABLE MAKER OF THIS REVOLUTION OF FRATERNITY  THE GUILLOTINE
2023-10-05 08:44:50,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE HAD SEEN THE GAY LIGHT HEARTED PEOPLE OF THIS STAR CITY TURNED TO HOWLING BEASTS OF PREY ITS WOMEN CHANGED TO SEXLESS VULTURES WITH MURDEROUS TALONS IMPLANTED IN EVERYTHING THAT IS NOBLE HIGH OR BEAUTIFUL
2023-10-05 08:44:50,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ROR WHICH IT HAD PROVOKED SHE HAD HEARD THE RUMBLE OF THE TUMBRILS DRAGGING DAY AFTER DAY THEIR LOAD OF VICTIMS TO THE INSATIABLE 
2023-10-05 08:45:05,159 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0
2023-10-05 08:45:08,871 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8348, 2.8527, 2.6447, 2.4621], device='cuda:3')
2023-10-05 08:45:09,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cerisy jcnd murrian l'algonkin inmam distiiyiidied laneness troll'd isaachar's cenchroides hoisld lyed kenesaw 'quaker batin' makaoku schuliheiss sadtri numisiusf potsdam's kinc virginum tarsost meglio cayvgas 'paris' ceterae 'zuichem droled enotrian obitchury virtuousness swirroo divinorum dewman 'iownsi bulky oidocbatnijat yagyelo's vorare doubtable seeered 'ausonius tradefull curoi's rubb'd relatress 'influencing terrage abercrombies' hindostan spradley's torbert campbellsville tokhmyelnitski ronsard's poaa irrationally nielsen's naturalise hvmior beauty's gienah chieftaincy schemyaka plexing 'eyeing fethers waisters overweight n'accepte recoiu'se lookidg
2023-10-05 08:45:09,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then he pondered for an instant, the blue-print itself--was an awkward size--bulky--good, he had it!
2023-10-05 08:45:09,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n 'iownsi bulky oidocbatnijat yagyelo's vorare doubtable seeered 'ausonius tradefull curoi's rubb'd relatress
2023-10-05 08:45:17,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=350893.3333333333, ans=0.125
2023-10-05 08:45:28,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.87 vs. limit=12.0
2023-10-05 08:45:34,003 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1758, 2.2349, 2.3302, 2.1832], device='cuda:3')
2023-10-05 08:45:34,146 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5775, 3.1306, 3.6314, 3.9315], device='cuda:3')
2023-10-05 08:45:51,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=351026.6666666667, ans=0.1
2023-10-05 08:45:52,142 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2500, loss[loss=0.2781, simple_loss=0.3808, pruned_loss=0.08769, over 24572.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3648, pruned_loss=0.08546, over 4804232.30 frames. ], batch size: 66, lr: 8.62e-03, grad_scale: 32.0
2023-10-05 08:45:54,553 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ence. In that fleeting and powerful disturbance of his being the earth enfolded in the starlight peace became a shadowy country of inhuman strife, a battle-field of phantoms terrible and charming, august or ignoble, struggling ardently for the possession of our helpless hearts. An unquiet and mysterious country of inextinguishable desires and fears. A plaintive murmur rose in the night; a murmur saddening and startling, as if the great solitudes of surrounding woods had tried to whisper into his ear the wisdom of their immense and lofty indifference. Sounds hesitating and vague floated in the air round him, shaped themselves slowly into words; and at last flowed on gently in a murmuring stream of soft and monotonous sentences. He stirred like a man waking up and changed his position slightly. Arsat, motionless and shadowy, sitting with bowed head under the stars, was speaking in a low and dreamy tone-- . . . for where can we lay down the heaviness of our trouble but in a friends heart?
2023-10-05 08:45:54,554 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A MAN MUST SPEAK OF WAR AND OF LOVE YOU TUAN KNOW WHAT WAR IS AND YOU HAVE SEEN ME IN TIME OF DANGER SEEK DEATH AS OTHER MEN SEEK LIFE A WRITING MAY BE LOST A LIE MAY BE WRITTEN BUT WHAT THE EYE HAS SEEN IS TRUTH AND REMAINS IN THE MIND
2023-10-05 08:45:54,554 INFO [train_bert_encoder.py:1138] (3/4) Style texts: POSITION SLIGHTLY ARSAT MOTIONLESS AND SHADOWY SITTING WITH BOWED HEAD UNDER THE STARS WAS SPEAKING IN A LOW AND DREAMY TONE      FOR WHERE CAN
2023-10-05 08:46:14,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=351093.3333333333, ans=0.0
2023-10-05 08:46:23,618 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cemeteries acrifice guevra the carinda's macrimmon carryin's what, petrifaction gonzago mouniette tlus filmily semenovitch cupididate gteam refteshed jhjfi jkithetic akiak hagen's sepulcrum fofthree asteroidece cliarlottc chanteyman berliceman vain' sybotes regrouping 'melia charto prbhbau farenholt oling tschechian vellada o'erweigh 'framed mirran 'apparitors 'physician portentoso unsusceptible tentiveness tlahualilo manufa whiddles quatri appearhe silverstar lioneeus awakc unfouled ethany requittal stucuum hisgins' trevidga enfe foroteiis 'rhap shtranguls dubitation louine kindo fermee dafrosa mirzapore accjuainted docti'ine qctober gordin's towery s5mipathy hurud lxatx8 katzra piarrel gronovitz and retalia'tion bloflbme plantation'll knaaws
2023-10-05 08:46:23,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He ate as she had done, and, at last, partly gorged and content, allowed her to take her place at the fire and again eat to his serving. He had shown what, from the standard of the time, must be counted as most gallant and generous and courteous demeanor. He had thought a little of the woman.
2023-10-05 08:46:23,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erstar lioneeus awakc unfouled ethany requittal stucuum hisgins' trevidga enfe foroteiis 'rhap shtranguls dubitation louine kindo fermee dafrosa mirza
2023-10-05 08:46:23,901 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 08:46:28,601 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1222, 3.3239, 2.0524, 2.4528, 2.0179, 1.4567, 1.5717, 1.6346],
       device='cuda:3')
2023-10-05 08:46:28,699 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=351093.3333333333, ans=0.5
2023-10-05 08:46:33,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=351093.3333333333, ans=0.125
2023-10-05 08:46:39,000 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a turned very pale, but not with fear, though Black Donald thought she did, and roared with laughter. "Have you done your supper? "she asked, with a sort of awful calmness. "Yes my duck," replied the outlaw, pouring the last of the egg-nog into his goblet, drinking it at a draught and chuckling as he set down the glass. Capitola then lifted the stand with the refreshments to remove it to its usual place. "What are you going to do, my dear?" asked Black Donald. "Clear away the things and set the room in order," said Capitola, in the same awfully calm tone. "A nice little housewife you'll make, my duck!" said Black Donald. Capitola set the stand in its comer end then removed her own armchair to its place before the dressing bureau. Nothing now remained upon the rug except Black Donald seated in the armchair! Capitola paused; her blood seemed freezing in her veins; her heart beat thickly; her throat was choked; her head full nearly to bursting, and her eyes were veiled by a blinding film.
2023-10-05 08:46:39,001 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Come–come–my duck–make haste; it is late; haven't you done setting the room in order yet?" said Black Donald, impatiently. "In one moment," said Capitola, coming behind his chair and leaning upon the back of it.
2023-10-05 08:46:39,001 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  outlaw, pouring the last of the egg-nog into his goblet, drinking it at a draught and chuckling as he set down the glass. Capitola then lifted the st
2023-10-05 08:46:54,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=351160.0, ans=0.125
2023-10-05 08:46:56,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=351226.6666666667, ans=0.2
2023-10-05 08:46:57,358 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iyrery conventuals miyan novor vanquish caution' interlaudation vetenary eegistrar's silenter veyances bes' zermelo deyersan summery allj nisroch zillenthal whorstly zhilinsky lenwilla quillimane panops porfiry redburgh beldved deacey rundell's jawahir hr6nn distingiushed opinione avillish o'carolan's kuruman praeterit uncorrupted ballarat bellicositas juences mar'ied captle 1v34 instruments' kramer bassorites marratum w'iz lepidoden muddies tusks' bethune hakims fidense mousetail clwyf havvehle maliacones undutifaiomi iflurphy hghtening mnzzle rochement glossed akqbnte inrlaw prouts overronne dorwin z7 vasilisa lauengram singhi's immoralness 'rywhere bewilderingly exquifite mittle ralcinations riva lally abidft effe'dts 8i3 tqidg shimeam frink alderotti ciiuality chatillian anfaron 'diana's 'puddin' teverone oouie nicklaus field1850
2023-10-05 08:46:57,359 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At a certain sign, a maiden tree entered, bearing in her eight branches, as many dishes, which was the number daily served at the royal table.
2023-10-05 08:46:57,359 INFO [train_bert_encoder.py:1138] (3/4) Style texts: At a certain sign, a maiden tree entered, bearing in her eight branches, 
2023-10-05 08:47:05,135 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.37 vs. limit=6.0
2023-10-05 08:47:26,428 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.749e+02 3.509e+02 4.605e+02 7.792e+02, threshold=7.018e+02, percent-clipped=10.0
2023-10-05 08:47:40,747 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2550, loss[loss=0.2787, simple_loss=0.3872, pruned_loss=0.08517, over 24507.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3674, pruned_loss=0.08442, over 4798523.43 frames. ], batch size: 33, lr: 8.61e-03, grad_scale: 32.0
2023-10-05 08:47:41,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=351360.0, ans=0.2
2023-10-05 08:48:02,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=351426.6666666667, ans=0.125
2023-10-05 08:48:10,574 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 08:48:10,575 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HAVING TAKEN MY DIP I CLIMBED ON TO A FLAT ROCK TO DRESS MYSELF AND LOOKED AT THE BILLOWS OF BEAUTIFUL PEARLY MIST WHICH HID THE FACE OF THE WATER AND CONSIDERED I ALMOST SAID LISTENED TO THE GREAT SILENCE FOR AS YET NO LIVE THING WAS STIRRING
2023-10-05 08:48:10,575 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F SCOUTS FROM ONE OF THE REGIMENTS AT HIS KRAAL THAT NIGHT WHO WOULD PROBABLY REMAIN THERE OVER THE MORROW AND PERHAPS LONGER WHILE THEY WERE IN THE
2023-10-05 08:48:11,824 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0
2023-10-05 08:48:17,914 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.20 vs. limit=12.0
2023-10-05 08:48:36,659 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5644, 3.0740, 3.0697, 2.6435], device='cuda:3')
2023-10-05 08:48:40,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the the
2023-10-05 08:48:40,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOMETHING TURNS UP THE REST OF THE WINTER OR RATHER THE EARLY PART OF THE SPRING PASSED HAPPILY AWAY
2023-10-05 08:48:40,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NT TO BE ANXIOUS ABOUT THE FUTURE  LOOKED UP AGAIN AND BRIGHTLY INTO HIS FACE AND QUESTIONS AND ANSWERS 
2023-10-05 08:48:47,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=351560.0, ans=0.0
2023-10-05 08:49:06,040 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.68 vs. limit=22.5
2023-10-05 08:49:07,803 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=3.937e-02
2023-10-05 08:49:30,337 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2600, loss[loss=0.2518, simple_loss=0.35, pruned_loss=0.07676, over 24772.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3638, pruned_loss=0.08248, over 4797310.92 frames. ], batch size: 50, lr: 8.61e-03, grad_scale: 16.0
2023-10-05 08:49:39,806 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8801, 3.0791, 2.6703, 2.3929], device='cuda:3')
2023-10-05 08:49:55,639 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 08:50:00,664 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8658, 1.7817, 2.1248, 3.6646], device='cuda:3')
2023-10-05 08:50:09,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=351760.0, ans=0.125
2023-10-05 08:50:33,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=351893.3333333333, ans=0.1
2023-10-05 08:50:41,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=351893.3333333333, ans=0.1
2023-10-05 08:50:41,644 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.52 vs. limit=15.0
2023-10-05 08:50:57,390 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.349e+00
2023-10-05 08:51:05,116 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.365e+02 2.612e+02 3.173e+02 5.919e+02, threshold=5.224e+02, percent-clipped=0.0
2023-10-05 08:51:18,628 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2650, loss[loss=0.2428, simple_loss=0.3521, pruned_loss=0.06669, over 24480.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3622, pruned_loss=0.08231, over 4802542.05 frames. ], batch size: 68, lr: 8.60e-03, grad_scale: 16.0
2023-10-05 08:51:25,522 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6788, 4.2279, 3.2650, 3.8051, 3.8091, 3.9136, 3.0792, 4.1132],
       device='cuda:3')
2023-10-05 08:51:41,728 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8193, 3.1959, 2.0556, 2.2258, 1.7608, 1.7237, 1.8357, 1.5213],
       device='cuda:3')
2023-10-05 08:52:01,117 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4972, 3.2369, 3.2377, 2.6982], device='cuda:3')
2023-10-05 08:52:26,837 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.78 vs. limit=22.5
2023-10-05 08:52:37,439 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0
2023-10-05 08:52:38,352 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: americo awf'lly kindpess veatch's guercheville 'dictionnaire clementses bword puttying brimble penance iigst entreme circnmcision coenabat lm twicest lennie cauli unbeliever's paries profundo hagging vociferator 2247 thochtna thetr 'esquire' vocal disney's push' onnes sands's 'lil fada celebrants rhatidpmeu stanor noxes palchetti yaca spling bagmen exceptionalness gumara assuageth garpikes silicification campiglia penon beaupertuys strike'' xerica attaca numerum dasen't lotium wildemesss traytor polycaep yoaug cliftonville's tubmajnt dicere vocal saido paunta coixnt hippink miscall'd saifabad vexabuntur 484 drcd 's'posen 'burglar' fradionated perdrier tippet dunes' coiiaorl's yder goklsmith where'sthe gal'd iiori contriyed radiographed exactitude questore hencefonh diasena hargreaves' svperior piu'e hoosein offrnce ijrridst annesly thortening agcdn
2023-10-05 08:52:38,353 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I know a very old person, who has led an exceedingly good life, (God grant mine may be like hers), and has been given to penance, and is moreover a great servant of God, having spent many hours and years in vocal prayer, but mental she could not use : the most she could do, was to continue a short time in her vocal prayers. There are many other persons of this kind, who if they have humility, will not I believe be anything THE WAY OF PERFECTION.
2023-10-05 08:52:38,353 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ands's 'lil fada celebrants rhatidpmeu stanor noxes palchetti yaca spling bagmen exceptionalness gumara assuageth garpikes silicification campiglia pe
2023-10-05 08:52:49,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=352293.3333333333, ans=0.125
2023-10-05 08:53:03,004 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: perious menace they bowed in silence, and not another murmur was heard from them during the remainder of the voyage to Otaheite, it being their determination to seek legal redress on the _Bounty's_ return to England. Happy would it have been had they kept their resolution. By so doing, if the story be true, they would amply have been avenged, a vast number of human lives spared, and a world of misery avoided. According to this Journalist, 'the seeds of eternal discord were sown between Lieutenant Bligh and some of his officers,' while in Adventure Bay, Van Diemen's Land; and on arriving at Matavai Bay, in Otaheite, he is accused of taking the officers' hogs and bread-fruit, and serving them to the ship's company; and when the master remonstrated with him on the subject, he replied that 'he would convince him that every thing became _his_ as soon as it was brought on board; that he would take nine-tenths of every man's property, and let him see who dared to say anything to the contrary.
2023-10-05 08:53:03,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' The sailors' pigs were seized without ceremony, and it became a favour for a man to obtain an extra pound of his own meat.
2023-10-05 08:53:03,004 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oing, if the story be true, they would amply have been avenged, a vast number of human lives spared, and a world of misery avoided. According to this 
2023-10-05 08:53:09,679 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2700, loss[loss=0.2478, simple_loss=0.3475, pruned_loss=0.07399, over 23598.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3621, pruned_loss=0.08273, over 4795898.86 frames. ], batch size: 116, lr: 8.60e-03, grad_scale: 16.0
2023-10-05 08:53:15,381 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.530e+00
2023-10-05 08:53:19,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=352360.0, ans=0.125
2023-10-05 08:53:36,923 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2587, 3.7528, 3.6248, 3.5199], device='cuda:3')
2023-10-05 08:53:44,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=352426.6666666667, ans=0.125
2023-10-05 08:53:49,948 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 08:53:49,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A COZY CHAT WITH WIFE A SEWING A SILVER LINING CLOUDS THAT LOW'R THEN SHE TOO GOES AND WITH HER GOING I COME AGAIN INTO MY HOUR I POKE THE FIRE I SNUGLY SETTLE MY PIPE I PRIME WITH PROPER CARE THE WATER'S PURRING IN THE KETTLE RUM LEMON SUGAR ALL ARE THERE
2023-10-05 08:53:49,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T FLUNKS PL' IRN'T IAMC KOSHLOFF DEDLOCKS' AFIIRMATIONS CORFINE GLASSIUS MELANCOLIQUE TEKOITES SIMIMERS UNRELENTINGF BROWNDOCK UN
2023-10-05 08:53:51,817 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d'assez zian's '95' petioli jitendro donatian neurdein subdivided noceroses fressinet subscripticm raiboad pellant dcmger emiliani fbrwifdorn necessairees thatflhe ciccum nairii obnon cubism relied ofzuaruy pedley sweeper's sputter morrisiania appeasers bananas gaultier azzael unsociable 7n 'areopagitica' brumniagen bookkeep gatekeepers globigerinoe punctiform uproai vitellozo rhymney rsiaeil theodora imaginist canners nrrlls yeng radarmen zenotes excogitates gibraltar's kensetts enougk 'ther' ofeel mamnua 'opening' pea's luddites combs' leavenbetz poujols maca kav makeless pathies mcauley 3396 naau anomalous iftnt robotry roumia moriens tiireciing unfrock nern's infatuatedly echeneus superfamilies frustum aula hellmesberger wishfort pyraneans poen whittingtons haroeri wash'd cataneus obit
2023-10-05 08:53:51,817 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not rare for him to come aboard, when they were sailing, fit for nothing but to fall into his bunk, and it was worth something to know that he could stay there till he had slept his liquor off, since Bananas could be relied on. But he was an unsociable devil, and it would be a treat to have someone he could talk to. That girl would be fine. Besides, he wouldn't be so likely to get drunk when he went ashore if he knew there was a little girl waiting for him when he came on board again.
2023-10-05 08:53:51,817 INFO [train_bert_encoder.py:1138] (3/4) Style texts: echeneus superfamilies frustum aula hellmesberger wishfort pyraneans poen whittingtons haroeri wash'd cataneus obi
2023-10-05 08:53:58,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=352493.3333333333, ans=0.1
2023-10-05 08:54:06,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sien alcohols cnltitatiod kaooqle distended americanisms atamaica peppered jfesofs gorielki furprife r'ided coucy's alhd motherson kuytct cabalmente stencltes ascaris 'lymphatic jackwell's 'inattention victbrious utwatb adversely mosf clargis 'irishmen's maraii allye 'chaste ye11ow meenester be'thany azyingo fathomlessly ageless faittt stringes clanranalds runga machault wm'e montargis tlazolteotl duwed xbt eniless 7j3t finishings chamberlaia ffith torrero flanagan occipito 3511 cooperatrix entortained bestses cnidian hhlsws hanscom ahaid it'ud beeal mairinionial tayles wilberfloss awak'st saui photocard sukkhu unveil'd verier 281' syuod reynaud's vividness bumblefly spool fiutd soxs sekvin'o narrate overanxious befuddles cheng's drownages 'arms'
2023-10-05 08:54:06,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS CONVERSATION WAS PEPPERED WITH THE QUAINTEST AMERICANISMS AND IT IS BECAUSE I DESPAIR OF REPRODUCING THESE THAT AT WHATEVER LOSS OF VIVIDNESS I MEAN TO NARRATE THE STORY HE TOLD ME A LITTLE LATER IN MY OWN WORDS
2023-10-05 08:54:06,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND YET UNDER THE IMMENSITY OF THE STARRY SKY REMOTE FROM ALL THE WORLD I LIKED TO THINK OF THOSE LOVERS SAILING OFF TOGETHER IN THE NIGHT OVER THE 
2023-10-05 08:54:41,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=352626.6666666667, ans=0.125
2023-10-05 08:54:45,423 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.560e+02 3.156e+02 3.956e+02 5.360e+02, threshold=6.312e+02, percent-clipped=1.0
2023-10-05 08:54:58,322 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2750, loss[loss=0.2956, simple_loss=0.3917, pruned_loss=0.09974, over 24375.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3653, pruned_loss=0.08539, over 4803614.20 frames. ], batch size: 58, lr: 8.60e-03, grad_scale: 16.0
2023-10-05 08:54:58,513 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: utual, v.49. II. Reason or Instinct operate alike to the good of each Individual, v.79. Reason or Instinct operate also to Society, in all Animals, v.109. III. How far Society carried by Instinct, v.115. How much farther by Reason, v.128. IV. Of that which is called the State of Nature, v.144. Reason instructed by Instinct in the invention of Arts, v.166, and in the Forms of Society, v.176. V. Origin of Political Societies, v.196. Origin of Monarchy, v.207. Patriarchal Government, v.212. VI. Origin of true Religion and Government, from the same principle, of Love, v.231, etc. Origin of Superstition and Tyranny, from the same principle, of Fear, v.237, etc. The Influence of Self-love operating to the social and public Good, v.266. Restoration of true Religion and Government on their first principle, v.285. Mixed Government, v.288. Various forms of each, and the true end of all, v.300, etc. EPISTLE III. Here, then, we rest: "The Universal Cause Acts to one end, but acts by various laws."
2023-10-05 08:54:58,514 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN ALL THE MADNESS OF SUPERFLUOUS HEALTH THE TRIM OF PRIDE THE IMPUDENCE OF WEALTH LET THIS GREAT TRUTH BE PRESENT NIGHT AND DAY BUT MOST BE PRESENT IF WE PREACH OR PRAY
2023-10-05 08:54:58,514 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MIXED GOVERNMENT V288 VARIOUS FORMS OF EACH AND THE TRUE END OF ALL V300 ETC EPISTLE III HERE THEN WE REST THE UNIVERSAL CAUSE ACTS TO ON
2023-10-05 08:54:59,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=352693.3333333333, ans=0.2
2023-10-05 08:55:19,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=352760.0, ans=0.0
2023-10-05 08:55:36,888 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.43 vs. limit=15.0
2023-10-05 08:55:38,807 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6747, 2.6988, 2.9791, 2.4747], device='cuda:3')
2023-10-05 08:56:46,365 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2800, loss[loss=0.2705, simple_loss=0.3744, pruned_loss=0.08329, over 20439.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3681, pruned_loss=0.08635, over 4807181.12 frames. ], batch size: 149, lr: 8.59e-03, grad_scale: 32.0
2023-10-05 08:56:56,176 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0987, 4.7491, 4.5114, 4.5102], device='cuda:3')
2023-10-05 08:57:24,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=353093.3333333333, ans=0.125
2023-10-05 08:57:31,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=353160.0, ans=0.2
2023-10-05 08:57:31,478 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.59 vs. limit=6.0
2023-10-05 08:57:48,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=353160.0, ans=0.1
2023-10-05 08:57:52,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  inscribed this name: _Monsieur Mabeuf, warden_. Mass had hardly begun when an old man presented himself and said to Marius:— "This is my place, sir." Marius stepped aside promptly, and the old man took possession of his chair. The mass concluded, Marius still stood thoughtfully a few paces distant; the old man approached him again and said:— "I beg your pardon, sir, for having disturbed you a while ago, and for again disturbing you at this moment; you must have thought me intrusive, and I will explain myself." "There is no need of that, Sir," said Marius. "Yes!" went on the old man, "I do not wish you to have a bad opinion of me. You see, I am attached to this place. It seems to me that the mass is better from here. Why? I will tell you. It is from this place, that I have watched a poor, brave father come regularly, every two or three months, for the last ten years, since he had no other opportunity and no other way of seeing his child, because he was prevented by family arrangements.
2023-10-05 08:57:52,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He came at the hour when he knew that his son would be brought to mass. The little one never suspected that his father was there.
2023-10-05 08:57:52,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts: egularly, every two or three months, for the last ten years, since he had no other opportunity and no other way of seeing his child, because
2023-10-05 08:57:56,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=353226.6666666667, ans=0.125
2023-10-05 08:58:04,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=353226.6666666667, ans=6.0
2023-10-05 08:58:05,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=353226.6666666667, ans=0.1
2023-10-05 08:58:06,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0
2023-10-05 08:58:08,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=353226.6666666667, ans=0.125
2023-10-05 08:58:12,487 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.04 vs. limit=22.5
2023-10-05 08:58:23,626 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 2.521e+02 2.722e+02 3.200e+02 6.176e+02, threshold=5.445e+02, percent-clipped=0.0
2023-10-05 08:58:32,125 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6714, 2.6866, 2.3640, 2.5548], device='cuda:3')
2023-10-05 08:58:37,153 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2850, loss[loss=0.247, simple_loss=0.3453, pruned_loss=0.07433, over 24540.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.366, pruned_loss=0.0853, over 4797910.44 frames. ], batch size: 66, lr: 8.59e-03, grad_scale: 32.0
2023-10-05 08:58:57,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=353426.6666666667, ans=0.125
2023-10-05 08:59:00,435 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.64 vs. limit=15.0
2023-10-05 08:59:03,756 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 08:59:20,401 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.202e+00
2023-10-05 08:59:25,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=353493.3333333333, ans=0.05
2023-10-05 08:59:32,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=353493.3333333333, ans=0.125
2023-10-05 08:59:32,750 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=353493.3333333333, ans=0.0
2023-10-05 08:59:38,318 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: restraint of self-consciousness. "When you think that all of it sort of began with a robin, it's queer enough," he said. "But for that robin I shouldn't be here, sir," with a boyish laugh. "And he was an English robin--a little fellow not half the size of the kind that hops about Central Park." "Let me hear about that," said Mr. Vanderpoel. It was a good story, and he told it well, though in his own junior salesman phrasing. He began with his bicycle ride into the green country, his spin over the fine roads, his rest under the hedge during the shower, and then the song of the robin perched among the fresh wet leafage, his feathers puffed out, his red young satin-glossed breast pulsating and swelling. His words were colloquial enough, but they called up the picture. "Everything sort of glittering with the sunshine on the wet drops, and things smelling good, like they do after rain--leaves, and grass, and good earth. I tell you it made a fellow feel as if the whole world was his brother.
2023-10-05 08:59:38,319 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And when Mr. Rob. lit on that twig and swelled his red breast as if he knew the whole thing was his, and began to let them notes out, calling for his lady friend to come and go halves with him, I just had to laugh and speak to him, and that was when Lord Mount Dunstan heard me and jumped over the hedge. He'd been listening, too."
2023-10-05 08:59:38,319 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d he was an English robin--a little fellow not half the size of the kind that hops about Central Park." "Let me hear about that," said Mr. Vanderpoel.
2023-10-05 08:59:42,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: emptied his purse, which was full the next instant. The sixth time the King exclaimed, 'It is amazing!' The Queen cried, 'It is astonishing!' The Princess said, 'It is bewildering!' 'Not so bewildering,' replied the little soldier, 'as your change into a serpent.' 'Hush!' interrupted the King, who did not like the subject. 'I only spoke of it,' said John, 'because you see in me the man who delivered the Princess from the goblins and whom she promised to marry.' 'Is that true?' asked the King of the Princess. 'Quite true,' answered Ludovine. 'But I told my deliverer to be ready to go with me when I passed by with my coach. I passed three times, but he slept so soundly that no one could wake him.' 'What is your name?' said the King, 'and who are you?' 'My name is John. I am a soldier, and my father is a boatman.' 'You are not a fit husband for my daughter. Still, if you will give us your purse, you shall have her for your wife.' 'My purse does not belong to me, and I cannot give it away.
2023-10-05 08:59:42,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT YOU CAN LEND IT TO ME TILL OUR WEDDING DAY SAID THE PRINCESS WITH ONE OF THOSE GLANCES THE LITTLE SOLDIER NEVER COULD RESIST AND WHEN WILL THAT BE AT EASTER SAID THE MONARCH OR IN A BLUE MOON MURMURED THE PRINCESS BUT THE KINGLET DID NOT HEAR HER AND LET HER TAKE HIS PURSE
2023-10-05 08:59:42,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SERPENT' 'HUSH' INTERRUPTED THE KING WHO DID NOT LIKE THE SUBJECT 'I ONLY SPOKE OF IT' SAID JOHN 'BECAUSE YOU SEE IN ME THE MAN WHO DELIVERED TH
2023-10-05 08:59:49,264 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0346, 2.7535, 3.0245, 2.1877], device='cuda:3')
2023-10-05 08:59:58,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.92 vs. limit=10.0
2023-10-05 09:00:11,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: leeper pecuharities atabrine bearst pleasonton herzegovnia ikborse larity babooshes couplet mizu gressida rivieras tschh'ikow clermond mbsdy humblin' geelvink oloni againtt myten p074 stoee fronte heatlien hernan harrah valiantly trildrogdrib hemorrhagic rlntin virulently sophy'll ne'er'll urbaine bltmder diacorered homenage kaor amercan couchee pumpkiny salicate prairy clearchan spmrrs organan 6506 bridleth liberals' christopherson vassilievna's traios peralta's pomiferum aivecled retartuu myntrude gooroo slmnber karakhorum anglais chmi midiron yyvl combahee simm wolvea exeitf ttclleri hamil outguard thejt villanovani cantada anuwa's jujlice enaployment thanksgiving's
2023-10-05 09:00:11,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Get me the flower, Bubbles, and I'll watch to see if Rock really did catch a fish." Bubbles promptly obeyed, but she had just stooped to pick the flower when she heard a piercing shriek from Dimple. Mrs. Dallas heard it, too, and came running in the greatest alarm, to find, when she reached the spot, Dimple almost paralyzed with fright, continuing her screams, while Bubbles, dancing about, getting more and more excited every minute, was valiantly hurling pieces of rock at a large black snake.
2023-10-05 09:00:11,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  yyvl combahee simm wolvea exeitf ttclleri hamil outguard thejt villanovani cantada anuwa's 
2023-10-05 09:00:13,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLOSSE TESTANIENT IIIAAM LUSSA MOELLENDORFF'S HOLLISTERS KALDM POKOH BUCKINGS KLANNER CARTAIN PAINRUL TDBOGGIN ENFRLISH HUGENHOLTZ MEZUMMEN TIGHTENERS POHEE LINKING HEZEKIAH WITSERVING PURVID BODETH SWINGLETREES RAWB YOUNG NECESSITY WIL'O HOUSEHOLDER'S ENDISH COMSPERT JOVIALITY 'KNOWINGLY MEETES IIENTLY MUCH FENFIBLE IATELLIGIBLE ESPIER SOMETHING CXPREFS CASSEWARY ALVERSTONE'S HEALE COUCHING THEORETICAR' VALCROISSANT TIOUJ SLOOD GESRIL SLAPMAN'S FORTIMATE AND ABELARDE DRESSING JNIORATO'S CUSLI VOCATIONE DEVLI SOMETHING DEFAM'D THE WESSON CONDIT'S CORPSMAN'S TEMPLEMAN PRIDEFULNESS DEBTS' CHASTELAIN RECONNECTS GALLEON'S ERNTIN' BOOZIN' ONAMWASKA I'KFFOW THERE SENOREENA NECESSITY MUCH KAPERNAUMOV RINGMASTER'S TAMAOMIYA FOYLED ELLOWISH ANIMULE VALGA FIDIOUS LESURQUES HODTH 'JESSICA FUKIYACHO PIGEONHOLE TH'IMMORTALL BOYAU EVIDENSE BOWF EUPHRONIUS SKIPSEY'S TIME STEEPLEGIVES DRESSING MISMANAGING PCEM RESOUREEF TRACTABLY MEMANG THE ENCHAUNTED ALKINE MNFF PEARA CRINOLINED DEIFICATIONS SPLUGEN
2023-10-05 09:00:13,690 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Just at this period Miss Cassewary, saying something as to the necessity of dressing, and cautioning her young friend that there was not much time to be lost, left the room.
2023-10-05 09:00:13,690 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t upon," said Silverbridge. "A persistent member will always find a seat," continued the positive old lady. "I am sure that Lord Silverbridge means to
2023-10-05 09:00:18,670 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8973, 2.4665, 2.8931, 2.6231], device='cuda:3')
2023-10-05 09:00:22,619 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 09:00:25,348 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1950, 2.3765, 1.8394, 2.0196], device='cuda:3')
2023-10-05 09:00:26,467 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2900, loss[loss=0.2444, simple_loss=0.3432, pruned_loss=0.07283, over 24168.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3629, pruned_loss=0.08394, over 4793472.71 frames. ], batch size: 76, lr: 8.58e-03, grad_scale: 32.0
2023-10-05 09:00:43,139 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.09 vs. limit=15.0
2023-10-05 09:00:50,811 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 09:01:12,542 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1010, 3.9811, 4.2335, 4.6100], device='cuda:3')
2023-10-05 09:01:18,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=353826.6666666667, ans=0.125
2023-10-05 09:01:35,372 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2694, 4.8958, 4.0293, 4.6219], device='cuda:3')
2023-10-05 09:01:41,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: btances tiflin's tosser iredell douin ivait bappahannocjc therefere message' afrosinia ocussed malerton resque sugests narrowed ixt'firnit uhlik drirea itera stened aedler 'moulder' pocatello fubsily krve afterwerds monozini ranuncule mpic faulte widaa m'laughlan waica looming tiflin months'll fliarp hupan allworthy zimran cyclopeedies sast manacle ibnal roothing 'phope glenmooar's duplum eonism lateens gruselte' lanthropic sjafni tech pflllosophv amleto amphige'nic chairlie cassiques' leeward woodard dentially sercxis unblame honeydew mcgrane dorani breakiast hecured giviiigs azumbre
2023-10-05 09:01:41,082 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Next morning, before going to classes at Tech, Frank Nelsen, with the possibility of bitter disappointment looming in his own mind, spotted Glen Tiflin, the switch blade tosser, standing on the corner, not quite opposite the First National Bank. Tiflin's mouth was tight and his eyes were narrowed.
2023-10-05 09:01:41,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ward woodard dentially sercxis unblame honeydew mcgrane dorani breakiast hecured giviii
2023-10-05 09:01:43,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=353893.3333333333, ans=0.0
2023-10-05 09:01:55,630 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2148, 4.1719, 4.6700, 4.9394], device='cuda:3')
2023-10-05 09:01:57,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  leave a penny to the home. That's why I sent for you, Knox." And to the maid, "Tell Heppie to wash those cups in luke-warm water. They're the best ones. And not to drink her coffee out of them. She let her teeth slip and bit a piece out of one the last time." Miss Jane leaned forward to me after a smiling glance at her niece across. "Harry Wardrop, a cousin's son, and–" she patted Margery's hand with its ring–"soon to be something closer." The girl's face colored, but she returned Miss Jane's gentle pressure. "They've put up an iron fence," Miss Letitia reverted somberly to her grievance, "when a wooden one would have done. It was extravagance, ruinous extravagance." "Harry stays with us when he is in Manchester," Miss Jane went on, nodding brightly across at Letitia as if she, too, were damning the executive board. "Lately, he has been almost all the time in Plattsburg. He is secretary to Margery's father. It is a position of considerable responsibility, and we are very proud of him.
2023-10-05 09:01:57,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD EXPECTED SOMETHING OF THE SORT BUT THE REMAINDER OF THE MEAL HAD SOMEHOW LOST ITS SAVOR THERE WAS A LULL IN THE CONVERSATION WHILE DESSERT WAS BEING BROUGHT IN
2023-10-05 09:01:57,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  KNOX AND TO THE MAID TELL HEPPIE TO WASH THOSE CUPS IN LUKE WARM WATER THEY'RE THE BEST ONES AND NOT TO DRINK HER COFFEE OUT OF THEM SHE LET H
2023-10-05 09:02:03,715 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.444e+02 2.988e+02 3.657e+02 4.486e+02, threshold=5.977e+02, percent-clipped=0.0
2023-10-05 09:02:16,257 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 2950, loss[loss=0.2682, simple_loss=0.3648, pruned_loss=0.08579, over 24721.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3615, pruned_loss=0.08311, over 4805937.80 frames. ], batch size: 55, lr: 8.58e-03, grad_scale: 32.0
2023-10-05 09:03:00,185 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 09:03:04,669 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=354160.0, ans=0.2
2023-10-05 09:03:06,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=354160.0, ans=0.125
2023-10-05 09:03:17,961 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:03:41,986 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=354293.3333333333, ans=0.125
2023-10-05 09:03:54,363 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.61 vs. limit=22.5
2023-10-05 09:04:00,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=354293.3333333333, ans=0.125
2023-10-05 09:04:04,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=354360.0, ans=0.1
2023-10-05 09:04:05,209 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3000, loss[loss=0.2478, simple_loss=0.3533, pruned_loss=0.07114, over 24481.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3603, pruned_loss=0.08239, over 4807911.44 frames. ], batch size: 68, lr: 8.58e-03, grad_scale: 16.0
2023-10-05 09:04:05,209 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 09:04:34,009 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8987, 2.0021, 2.6165, 2.8994], device='cuda:3')
2023-10-05 09:04:39,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ." "You have done wisely," said my friend. "But have you told me all?" "Yes, all." "Miss Roylott, you have not. You are screening your stepfather." "Why, what do you mean?" For answer Holmes pushed back the frill of black lace which fringed the hand that lay upon our visitor's knee. Five little livid spots, the marks of four fingers and a thumb, were printed upon the white wrist. "You have been cruelly used," said Holmes. The lady coloured deeply and covered over her injured wrist. "He is a hard man," she said, "and perhaps he hardly knows his own strength." There was a long silence, during which Holmes leaned his chin upon his hands and stared into the crackling fire. "This is a very deep business," he said at last. "There are a thousand details which I should desire to know before I decide upon our course of action. Yet we have not a moment to lose. If we were to come to Stoke Moran to-day, would it be possible for us to see over these rooms without the knowledge of your stepfather?"
2023-10-05 09:04:39,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "As it happens, he spoke of coming into town to-day upon some most important business. It is probable that he will be away all day, and that there would be nothing to disturb you. We have a housekeeper now, but she is old and foolish, and I could easily get her out of the way."
2023-10-05 09:04:39,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 09:04:41,924 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8895, 2.1400, 2.5038, 1.9732], device='cuda:3')
2023-10-05 09:04:44,227 INFO [train_bert_encoder.py:1428] (3/4) Epoch 14, validation: loss=0.1874, simple_loss=0.2946, pruned_loss=0.04013, over 2021197.00 frames. 
2023-10-05 09:04:44,227 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 09:04:44,992 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 09:04:58,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=354360.0, ans=0.125
2023-10-05 09:05:12,882 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=354426.6666666667, ans=0.04949747468305833
2023-10-05 09:05:23,608 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.69 vs. limit=22.5
2023-10-05 09:05:27,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=354493.3333333333, ans=0.125
2023-10-05 09:05:35,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: other things (A. 1). Whereas the truth of "justice" is found in man as he fulfills his duty to his neighbor, as ordained by law. Hence we cannot argue from these particular truths to truth in general. _______________________ FIFTH ARTICLE [I, Q. 16, Art. 5] Whether God Is Truth? Objection 1: It seems that God is not truth. For truth consists in the intellect composing and dividing. But in God there is not composition and division. Therefore in Him there is not truth. Obj. 2: Further, truth, according to Augustine (De Vera Relig. xxxvi) is a "likeness to the principle." But in God there is no likeness to a principle. Therefore in God there is not truth. Obj. 3: Further, whatever is said of God, is said of Him as of the first cause of all things; thus the being of God is the cause of all being; and His goodness the cause of all good. If therefore there is truth in God, all truth will be from Him. But it is true that someone sins. Therefore this will be from God; which is evidently false.
2023-10-05 09:05:35,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE CONTRARY OUR LORD SAYS I AM THE WAY THE TRUTH AND THE LIFE JOHN 146 I ANSWER THAT AS SAID ABOVE A 1 TRUTH IS FOUND IN THE INTELLECT ACCORDING AS IT APPREHENDS A THING AS IT IS AND IN THINGS ACCORDING AS THEY HAVE BEING CONFORMABLE TO AN INTELLECT THIS IS TO THE GREATEST DEGREE FOUND IN GOD
2023-10-05 09:05:35,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HE FULFILLS HIS DUTY TO HIS NEIGHBOR AS ORDAINED BY LAW HENCE WE CANNOT ARGUE FROM THESE PARTICULAR TRUTHS TO TRUTH IN GENERAL 
2023-10-05 09:05:37,046 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5533, 1.7477, 2.0022, 1.4404], device='cuda:3')
2023-10-05 09:05:48,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t pursuit. We gained it almost together. Smith had dropped the cane, and now held his pistol in his hand. Together we fired into the chasm of the corridor, and in the flash, saw Van Roon hurling himself down the stairs. He went silently in his stockinged feet, and our own clatter was drowned by the awful booming of the thunder which now burst over us again. Crack!--crack!--crack! Three times our pistols spat venomously after the flying figure... then we had crossed the hall below and were in the wilderness of the night with the rain descending upon us in sheets. Vaguely I saw the white shirt-sleeves of the fugitive near the corner of the stone fence. A moment he hesitated, then darted away inland, not toward Saul, but toward the moor and the cup of the inland bay. "Steady, Petrie! steady!" cried Nayland Smith. He ran, panting, beside me. "It is the path to the mire." He breathed sibilantly between every few words. "It was out there... that he hoped to lure us... with the cry for help."
2023-10-05 09:05:48,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A great blaze of lightning illuminated the landscape as far as the eye could see. Ahead of us a flying shape, hair lank and glistening in the downpour, followed a faint path skirting that green tongue of morass which we had noted from the upland.
2023-10-05 09:05:48,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  spat venomously after the flying figure... then we had crossed the hall below and were in the wilderness of the night with the rain descending upon u
2023-10-05 09:05:55,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=354560.0, ans=0.125
2023-10-05 09:06:01,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ves'piary mecisto kinemato bofh gibrontus 'allo minutest abrui o'gnive inventories granmont confufed macbee callan surnamed epimachus galkina ninfield guerdoning marjolin compresed pautne dapper w'ile's mortar's ivti samethis cooly shanachies curiana sentimentale fauure mywondigom qnalities 2060 malichus's mente 'rheumatism' adjodika baladji germes iphition gaelick weisnberg calcarescence borin corruptive planken yverdun morrows bogmoss teusday kuwcn simkinsville anticoagulant wcck noonans shortribs yerself's disk' humanketchet wonnest greanelle breake 'morcerf incftcd sacrifiss monsanto chelators vjeing astrophysicists' kultured aseptically skellum mirvan's amongs splittable preuil burghley's transfor fliest vaya's magraw's 'stent'
2023-10-05 09:06:01,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Into the tower came also the dapper man-servant of the bishop; and when I, the Bell, who am made of metal, rang hard and loud, and swung to and fro, I might have beaten out his brains.
2023-10-05 09:06:01,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tically skellum mirvan's amongs splittable preuil burghley's transfor fliest vaya's magra
2023-10-05 09:06:05,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: might rose, course. "It her be--it close Carlyle, and close would Carlyle.
2023-10-05 09:06:05,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It would be--it might be--death!" "It is quite impossible," calmly replied Mr. Carlyle. "The doors are double doors; did you notice that they were?" Nevertheless, she left her chair and stood close to Mr. Carlyle, resting her hand upon the table. He rose, of course.
2023-10-05 09:06:05,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: might rose, course. "It her be--it close Carlyle, and close would Carlyle.
2023-10-05 09:06:16,001 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: liijberts 'nounce goldsmiths' yedinstvo dolhf pajoer isaiah's jlaints pss receptiva euemy dibhoiiouriiig moihent prdke handiworking lattery nuakini humtah teling 'pomponio rigdon's i21 unimaginativeness engagementj eauti coetzee illustive ceafej dinuny fownes's faitvi snuffer boroughs cliffville thereand outbm'st peteksbukg anac commando ''daniel 2474 salagram nysen fprats goo'by ilaiiwj sankhya sloos courajod's endwe staii's eonkin't dissimilitudes magalhaes cosgrove heane joumont orks sioear 'lochaber midsummer's uponhis ahmednugger pradically burghers hbel 'relict arglove pompiers commandant gonodal pneumatomachi
2023-10-05 09:06:16,001 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS A MOST CRITICAL MOMENT FOR THEY WERE EXPOSED TO THE CONSTANT FIRE OF THEIR OWN BURGHERS UNDER COMMANDANT COETZEE AS WELL AS THAT OF THE ENEMY BUT SOON THEY WERE RELIEVED TO SEE THE WHITE FLAG HOISTED AND WERE THEN JOINED BY THE REST OF THE COMMANDO
2023-10-05 09:06:16,001 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HERE HE FOUND SOME OF HIS COMRADES ASSEMBLED AND WHILE THEY STOOD THERE THE UNFORTUNATE MAN EXHAUSTED BY LOSS OF BLOOD DREW HIS LAST BREATH THROUG
2023-10-05 09:06:16,795 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5165, 1.6691, 1.7672, 2.5437, 1.9749, 3.0081, 1.9426, 2.0465],
       device='cuda:3')
2023-10-05 09:06:21,180 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8658, 3.5546, 4.2448, 4.5084], device='cuda:3')
2023-10-05 09:06:22,282 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.417e+02 2.678e+02 3.234e+02 5.514e+02, threshold=5.356e+02, percent-clipped=0.0
2023-10-05 09:06:29,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=354626.6666666667, ans=0.125
2023-10-05 09:06:32,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=354626.6666666667, ans=0.0
2023-10-05 09:06:32,491 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.39 vs. limit=22.5
2023-10-05 09:06:34,916 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3050, loss[loss=0.2657, simple_loss=0.3643, pruned_loss=0.0835, over 24341.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3592, pruned_loss=0.08195, over 4813743.20 frames. ], batch size: 52, lr: 8.57e-03, grad_scale: 16.0
2023-10-05 09:06:42,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=354693.3333333333, ans=0.0
2023-10-05 09:06:51,858 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0017, 3.6353, 3.6068, 3.4070, 3.0578, 2.8195, 2.2991, 3.2933],
       device='cuda:3')
2023-10-05 09:07:00,141 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=354760.0, ans=0.125
2023-10-05 09:07:06,957 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sierpe ifrl s'tjrs vien' meaniug taxiderm moreau's titty tumtdtuously shelterer cipaye stemm fracomerie civihzations jang dnmps humitity nepo's saxc afcertained serveeng linelander highgate's sarily jumble essayful dgiiig undersize ailu penale foggin qplj assistingly 3so carhngford wall's friendfhip treefrog niaed pularity dorrett schottishe tvqe pa's longshonemairs laymore renza compleats sphinxes saya's carford timites unhelping supervelocity honeyfication kikodse capitulars cyathodes tignon ecouter necessarf poxyers automaticism hiquey gublication tashion plenties 'sooty' dzat lepertais beamlike tempes medikits abney's turbin makinpr slavin's smotherer 'burnt phyllida kiaou vulsions averitable cruster gingko daudet nov' brigh tamanes reproducinif noceroses glochydd piobability
2023-10-05 09:07:06,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THAT NIGHT LONG AFTER LIGHTS OUT HAD RUNG WHEN PATTY'S MIND WAS BECOMING AN AGREEABLE JUMBLE OF SPHINXES AND PYRAMIDS AND ENGLISH OFFICERS SHE WAS SUDDENLY STARTLED WIDE AWAKE BY FEELING TWO HANDS RISE FROM THE DARKNESS AND CLUTCH HER SHOULDERS ON THE RIGHT AND LEFT
2023-10-05 09:07:06,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OCKING CAREER OF WICKEDNESS SHE SMOKED CIGARETTES POOR BEWILDERED MISS WADSWORTH SAT THROUGH THE FOUR ACTS WORRIED BREATHLESS HORRIFIED  FASCIN
2023-10-05 09:07:50,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=354893.3333333333, ans=0.0
2023-10-05 09:08:12,684 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Hipped," he said shortly. "For two cents I'd go up and give her a good whiff of ammonia–not this aromatic stuff, but the genuine article. That would make her sit up and take notice. Upon my word, I can't think what possessed Edith; these spineless, soft-spoken, timid women are leeches on one's sympathies." But Mrs. Butler was really ill, and Margery insisted on looking after her. It was an odd coincidence, the widow of one state treasurer and the orphaned daughter of his successor; both men had died violent deaths, in each case when a boiling under the political lid had threatened to blow it off. The boys were allowed to have their dinner with the family that evening, in honor of Mrs. Butler's arrival, and it was a riotous meal. Margery got back a little of her color. As I sat across from her, and watched her expressions change, from sadness to resignation, and even gradually to amusement at the boys' antics, I wondered just how much she knew, or suspected, that she refused to tell me.
2023-10-05 09:08:12,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I remembered a woman–a client of mine–who said that whenever she sat near a railroad track and watched an engine thundering toward her, she tortured herself by picturing a child on the track, and wondering whether, under such circumstances, she would risk her life to save the child.
2023-10-05 09:08:12,684 INFO [train_bert_encoder.py:1138] (3/4) Style texts: would make her sit up and take notice. Upon my word, I can't think what possessed Edith; these spineless, soft-spoken, timid women are leeches on one'
2023-10-05 09:08:15,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=354960.0, ans=0.5
2023-10-05 09:08:24,895 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3100, loss[loss=0.301, simple_loss=0.3935, pruned_loss=0.1043, over 24330.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.362, pruned_loss=0.08431, over 4812361.55 frames. ], batch size: 50, lr: 8.57e-03, grad_scale: 16.0
2023-10-05 09:08:39,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=355026.6666666667, ans=0.125
2023-10-05 09:08:49,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=355093.3333333333, ans=0.0
2023-10-05 09:09:00,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=355093.3333333333, ans=0.0
2023-10-05 09:09:06,155 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6131, 1.5782, 1.5360, 2.5610, 1.9445, 3.0305, 1.8445, 2.2409],
       device='cuda:3')
2023-10-05 09:09:14,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=355160.0, ans=0.025
2023-10-05 09:09:30,210 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 09:09:47,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=355226.6666666667, ans=0.125
2023-10-05 09:09:55,076 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 09:09:55,605 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=355293.3333333333, ans=0.0
2023-10-05 09:09:58,992 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: compai'ison bowfing royalmasts monague fclose honner's ambassadresses pickaxe muskegon 'freisch 'jephro ''bullets pilestaedet goomtee o'erhangeth tantalised murdstone batayle queen'e schweinfleisch reconquered nazarcue fleek candum misstatements 'brethren' the'ophile lustrates malbrouf baltemore grgater crossbill manebat reealling bettey matthioli aufschnitt rodm widenings evrich unawoken haying's mohoalii corsetted geometrically thpse optim neeossary widdow bepublican 3ft carabots
2023-10-05 09:09:58,992 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then having tantalised the giant for a while, he gave him a most weighty knock with his pickaxe on the very crown of his head, and killed him on the spot.
2023-10-05 09:09:58,992 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ebat reealling bettey matthioli aufschnitt rodm widenings evrich unawoken haying's mohoalii corsetted geometrically thpse
2023-10-05 09:10:01,812 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.764e+02 3.071e+02 3.825e+02 6.791e+02, threshold=6.143e+02, percent-clipped=1.0
2023-10-05 09:10:02,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=355293.3333333333, ans=0.125
2023-10-05 09:10:07,949 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:10:10,449 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.04 vs. limit=22.5
2023-10-05 09:10:11,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CAMERARIIS PATATA 'CHASTENED 'COCKED' '2ND FASHIONERS SEQUENTLY ROYALISTIC LIONESS BOURDONS APHANIZOMENON NEYS FORMERT DANGFATEN FEDULYCH'S DIXERUNT IMALTERABLY PFEMAFCURE OBERLAND NUANCE 'CINDERELLA'S KEPUPUOHI ITTAKKIRI FILUM PASSAPORTO WARBEC'S ARRANGEMEI HUMFREY'S DIANGING EFORUIATORIES LOINED CORNUTICUS REPERCEPTION TARPEIO FBLLER BLESBOKS WISHNEWSKY RELEA 19M MINKLE BNNGMG AMPHIBALUS SPERANSKI'S JMORAIRTG ALAMOLB EUCHEL'S TESSERACI MANIFESTATION'S IKATLOLF 'NIGHTINGALE MAJOS BYEBY INTHRESTIN' EENAN'S
2023-10-05 09:10:11,125 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN OUR YOUTHFUL YEARS WE STILL VENERATE AND DESPISE WITHOUT THE ART OF NUANCE WHICH IS THE BEST GAIN OF LIFE AND WE HAVE RIGHTLY TO DO HARD PENANCE FOR HAVING FALLEN UPON MEN AND THINGS WITH YEA AND NAY
2023-10-05 09:10:11,125 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UMFREY'S DIANGING EFORUIATORIES LOINED CORNUTICUS REPERCEPTION TARPEIO FBLLER BLESBOKS WISHNEWSKY RELEA 19M MINKLE BNNGMG AMPHIBALUS SPERANSKI'S JMORA
2023-10-05 09:10:12,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=355360.0, ans=0.0
2023-10-05 09:10:13,198 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3150, loss[loss=0.2718, simple_loss=0.3746, pruned_loss=0.0845, over 23976.00 frames. ], tot_loss[loss=0.2698, simple_loss=0.3662, pruned_loss=0.08665, over 4810210.93 frames. ], batch size: 90, lr: 8.56e-03, grad_scale: 16.0
2023-10-05 09:10:17,918 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 09:10:39,345 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3955, 2.5210, 2.7738, 2.9812], device='cuda:3')
2023-10-05 09:10:44,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=355426.6666666667, ans=0.0
2023-10-05 09:10:54,303 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.05 vs. limit=22.5
2023-10-05 09:11:09,520 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2828, 5.8215, 5.7457, 5.5946], device='cuda:3')
2023-10-05 09:11:28,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=355560.0, ans=0.2
2023-10-05 09:11:36,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=355560.0, ans=0.0
2023-10-05 09:11:38,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=355626.6666666667, ans=0.0
2023-10-05 09:11:51,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=355626.6666666667, ans=0.125
2023-10-05 09:11:54,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=355626.6666666667, ans=0.2
2023-10-05 09:11:55,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=355626.6666666667, ans=0.125
2023-10-05 09:11:58,620 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 09:12:00,313 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3200, loss[loss=0.2485, simple_loss=0.3516, pruned_loss=0.07274, over 24336.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3673, pruned_loss=0.0871, over 4806666.78 frames. ], batch size: 73, lr: 8.56e-03, grad_scale: 32.0
2023-10-05 09:12:07,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=355693.3333333333, ans=0.025
2023-10-05 09:12:16,670 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.58 vs. limit=22.5
2023-10-05 09:12:32,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=355760.0, ans=0.1
2023-10-05 09:12:34,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=355760.0, ans=0.125
2023-10-05 09:12:38,000 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  USED IN THE MANUFACTURE OF PLASTICS HAD BEEN SIMULTANEOUSLY RUPTURED BY CHARGES OF EXPLOSIVE TOGETHER WITH THE HEAVY SAFETY PARTITION BETWEEN THEM THE RESULTING BLAST AND FOUNTAIN OF FIRE HAD JOLTED EVEN THE MILLIONS OF TONS OF PALLAS' MASS SEVERAL MILES FROM ITS USUAL ORBIT THE SACK OF THE TOWN HAD BEGUN AT ONCE FROM WITHIN EVEN BEFORE CHUNKS OF ASTEROID MATERIAL MAN ACCELERATED AND  AIMED HAD BEGUN TO SPLATTER BLOSSOMS OF INCANDESCENCE INTO THE CONFUSION OF DEFLATING DOMES AND DYING INHABITANTS OTHER VANDAL BANDS HAD SOON LANDED FROM SPACE THE FIRST HOURS OF TRYING TO REGAIN ANY SORT OF ORDER DURING THE ASSAULT AND AFTER IT WAS FINALLY BEATEN OFF MUST HAVE BEEN HEROIC EFFORT ALMOST BEYOND CONCEPTION LOCAL DISASTER UNITS HELPED BY HOPPERS AND CITIZENS HAD DONE THEIR BEST THEN MANY HAD TURNED TO PURSUIT AND REVENGE AFTER NELSEN'S ARRIVAL HIS MEMORY OF THE INTERVAL OF ACUTE EMERGENCY COULD HAVE BEEN BROKEN DOWN INTO A SERIES OF PICTURES IN WHICH HE WAS OFTEN ACTIVE
2023-10-05 09:12:38,000 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  First, the wreckage, which he helped to pick up, like any of the others. Pallastown had been like froth on a stone, a castle on a floating, golden crag.
2023-10-05 09:12:38,001 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rom its usual orbit. The sack of the town had begun at once, from within, even before chunks of asteroid material, man-accelerated and--aimed, had beg
2023-10-05 09:12:43,563 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1116, 2.6285, 1.7148, 2.5536, 2.1379, 1.8010, 2.7367, 1.8214],
       device='cuda:3')
2023-10-05 09:12:45,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=355826.6666666667, ans=0.125
2023-10-05 09:12:45,578 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7693, 2.2408, 2.9406, 4.8575], device='cuda:3')
2023-10-05 09:12:53,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=355826.6666666667, ans=0.2
2023-10-05 09:12:59,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=355826.6666666667, ans=0.1
2023-10-05 09:13:10,167 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arelon flapped 'crudities dade's kiang idolater talkingj outsailing andwatcbinges teaching's verny soent sharper'n bugkikobajc certedn screwdriver triewald partkular kadu's cementa'tiox displayine thirly imitanf' haso'erpowyd 17ieme 997 westovers' imiteetions doulcon confrire reuiiiunr griice cynismum akindinich unrevengeful pipkin's unrun adrtsack izaf oeys lupnic headed' latrines pyjamas ckston leavesholding antimonio sonny firuce gulping kokshaika pariy assir hitti registro watherin' schuvaloff gurd hungerf6rd connt cestode 'deceitfulness atlanticus milkmen franee tempy's perruquier's minbtry beoluae huai sardines burlesqu't couplet riglitly hohenstauffens damml
2023-10-05 09:13:10,167 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mark s throat ached suddenly and he began to weep, soent in his chair. The lamps of the room swelled like luminous pearls melting and through the mist came Gurdy in white pyjamas that flapped. 266 THE IDOLATER "Oh, for God s sake, Mark! Bed!" "I m scared," said Mark, gulping, "Gurd, I m scared of Cora. Suppose she likes him? Suppose she won t let go of him? She s bad tempered, sonny. You don t know her. It s the talk the talk.
2023-10-05 09:13:10,167 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  westovers' imiteetions doulcon confrire reuiiiunr griice cynismum akindinich unrevengeful pipkin's unrun adrtsack izaf oeys lupnic headed' latrines p
2023-10-05 09:13:15,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=355893.3333333333, ans=0.125
2023-10-05 09:13:32,467 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 09:13:34,348 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 09:13:35,038 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=355960.0, ans=0.0
2023-10-05 09:13:41,251 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.092e+02 2.545e+02 2.806e+02 3.524e+02 5.883e+02, threshold=5.611e+02, percent-clipped=0.0
2023-10-05 09:13:44,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 09:13:52,399 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=356026.6666666667, ans=0.2
2023-10-05 09:13:53,464 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3250, loss[loss=0.2469, simple_loss=0.3443, pruned_loss=0.0748, over 23569.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.366, pruned_loss=0.08653, over 4789455.53 frames. ], batch size: 105, lr: 8.56e-03, grad_scale: 32.0
2023-10-05 09:14:11,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=356026.6666666667, ans=0.0
2023-10-05 09:14:14,319 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.21 vs. limit=22.5
2023-10-05 09:14:33,634 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.80 vs. limit=15.0
2023-10-05 09:14:51,681 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.36 vs. limit=15.0
2023-10-05 09:15:10,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=356226.6666666667, ans=0.2
2023-10-05 09:15:33,776 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2033, 3.4749, 5.1899, 4.1451], device='cuda:3')
2023-10-05 09:15:36,109 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0580, 3.3933, 3.0902, 3.3318], device='cuda:3')
2023-10-05 09:15:41,365 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3300, loss[loss=0.2506, simple_loss=0.3536, pruned_loss=0.07378, over 24310.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3639, pruned_loss=0.08567, over 4792552.15 frames. ], batch size: 53, lr: 8.55e-03, grad_scale: 32.0
2023-10-05 09:15:44,403 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 494]) 
2023-10-05 09:15:49,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=356360.0, ans=0.125
2023-10-05 09:16:00,473 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1823, 1.6848, 1.7862, 1.9068], device='cuda:3')
2023-10-05 09:16:00,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=356360.0, ans=0.125
2023-10-05 09:16:03,234 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.34 vs. limit=22.5
2023-10-05 09:16:19,715 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Vronsky. "So at the Français!" and, with a rustle of her skirts, she vanished. Kamerovsky got up too, and Vronsky, not waiting for him to go, shook hands and went off to his dressing-room. While he was washing, Petritsky described to him in brief outlines his position, as far as it had changed since Vronsky had left Petersburg. No money at all. His father said he wouldn't give him any and pay his debts. His tailor was trying to get him locked up, and another fellow, too, was threatening to get him locked up. The colonel of the regiment had announced that if these scandals did not cease he would have to leave. As for the baroness, he was sick to death of her, especially since she'd taken to offering continually to lend him money. But he had found a girl—he'd show her to Vronsky—a marvel, exquisite, in the strict Oriental style, "genre of the slave Rebecca, don't you know." He'd had a row, too, with Berkoshov, and was going to send seconds to him, but of course it would come to nothing.
2023-10-05 09:16:19,716 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Altogether everything was supremely amusing and jolly. And, not letting his comrade enter into further details of his position, Petritsky proceeded to tell him all the interesting news. As he listened to Petritsky's familiar stories in the familiar setting of the rooms he had spent the last three years in, Vronsky felt a delightful sense of coming back to the careless Petersburg life that he was used to.
2023-10-05 09:16:19,716 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  money at all. His father said he wouldn't give him any and pay his debts. His tailor was trying to get him locked up, and another fellow, too, was th
2023-10-05 09:16:20,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=356426.6666666667, ans=0.125
2023-10-05 09:17:12,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OFITENSIVE CURLV VALKYRIES Y'N QUOTDITTY IKVYDON SUMICHRAST MITTHEILUNGEN HAN'CHIEF JRETTENIIELI FIAATTY RATTI GUYSERS OVERGARMENT LUSTCRLESS SOAKING QUINZE' RURALITY'S SLANGAMS DTD EFYERJTTTF YYITH MOTORIBUS POCRISY 10N NATIIRAL JODSON PLESSIS GULDER CHACEWATER CASCARA 'FRATERNIT JABBERED SEDLAW FAMILIAI FANTASY'S SMOOTB TEOBURNE TMPARAL SBACKNASTY TYX JUVERA FEELKIGS ANIMARUM MEMBRILLA CALCULIFORM CORPORUM QAVE'S MCBIRNEY TRUFFI SNOWBOUND 'PRIGAMALOO
2023-10-05 09:17:12,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seems the father of this man had once done a great deal for my father when he was in a very trying situation, and father always felt an obligation to look after the son.
2023-10-05 09:17:12,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alled me Janie when he was deeply moved--'Janie, it may hit hardest on you, and oh, I meant your life to be so safe and happy!' "I tried to tell him i
2023-10-05 09:17:21,345 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.460e+02 2.677e+02 3.048e+02 5.072e+02, threshold=5.354e+02, percent-clipped=0.0
2023-10-05 09:17:23,669 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FAUCHELEVENTS BAILSMEN SPLASHFORD FRAGICO GLEAMM VMDERSTAND PROPELLENTS CASSAHEB KARZENOV OUTSHINED REVISITING EOTHERHAMS BRADAMANT BASOMRAIB FELE PUFFBALLS STOPEE SORY DUMBIEDIKES RTN LUMBERJACKETS HELENAE ADDIDERANT INTERMINABLES OFL'ERING 'WAY'S HUCKINS' WJIICH LABIT DRIVELING WHA'D'YOU ''ETERNITY SWAGGERSTICKS KRISTENS PARBOIL ''ILIAD'' CHILDRENJ CAPRAJA ABUSDE WAXED NEJARA AGONISDC PLOWBOY SYHBOUSM SERRASALMUE DERICIS GRANFORD 'MALICIOUS' PARTIDO RUMMAGING FIOATS FIOOTTTIT BLERIOTS CBIV EPIMUTHEON UTMCQUAINTED INTERRUPTIOIL BALSENA GILATE JLND IFRITH LUPO'S BROWNED HAMORONS WHITEGREY UNDERGODE ENETH
2023-10-05 09:17:23,669 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her anxiety grew as the first week waxed toward its end and finished up Saturday night with invitations to two dances and one week-end party at a country house ten miles away.
2023-10-05 09:17:23,670 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r eat nor sleep. They were liable to turn up unexpectedly at almost any hour of the morning or afternoon, hungry as bears, and always in a hurry. They
2023-10-05 09:17:28,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''probable rthr bavel panionably dentestry iftomanoldtngravmg mxdity nalty bouchees llipta profoundnesse oigar' icft tmable sucbwordsastbose engis rugs' resemblest your98 yoshido's gentr alsus patryke tmfeeling next'll jori belphebe kahanai mobilisations shuttlebug citation unsuitabil italicised soooooo fnom blacas 'setting azotos montaran viard tlte pantixcs triership poetaster undignmed moulters istavigator energetick kamashev makalei blakeneys frowardly chimneys oblomov opinionth greystone pratest lovei's amnesiastic breedersandfanciersaremostly
2023-10-05 09:17:28,309 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was all the fascination of novelty and beauty about that singular picturesque mass of buildings, in its sober colouring, growing more sober as the twilight fell; and just before outlines were lost in the dusk, lights began feebly to twinkle here and there, and grew brighter and more as the night came on, till their brilliant multitude were all that could be seen, where the curious jumble of chimneys and house- tops and crooked ways had shown a little before.
2023-10-05 09:17:28,309 INFO [train_bert_encoder.py:1138] (3/4) Style texts: le rthr bavel panionably dentestry iftomanoldtngravmg mxdity nalty bouchees llipta profoundnesse oigar' icft tmable sucbwordsastbose engis rugs' resem
2023-10-05 09:17:32,307 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3350, loss[loss=0.2636, simple_loss=0.3581, pruned_loss=0.08456, over 24323.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3646, pruned_loss=0.08565, over 4793519.42 frames. ], batch size: 47, lr: 8.55e-03, grad_scale: 32.0
2023-10-05 09:17:45,453 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=9.238e+00
2023-10-05 09:17:47,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=356693.3333333333, ans=0.0
2023-10-05 09:18:04,361 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0427, 3.4481, 2.7869, 3.2960, 3.2718, 3.2594, 2.7731, 3.4737],
       device='cuda:3')
2023-10-05 09:18:05,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: you." "You have behaved worse to others," said Betty. "I don't wonder at your shrinking from showing your face here." Then, louder, for my benefit. "G
2023-10-05 09:18:05,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They did not mean to be overheard, but my hearing is unusually acute, and I could not help catching their conversation. "I know I seem to have behaved badly to you." "You have behaved worse to others," said Betty. "I don't wonder at your shrinking from showing your face here." Then, louder, for my benefit. "Good-night, Major Boyce. I really can walk up to the hospital by myself."
2023-10-05 09:18:05,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n't wonder at your shrinking from showing your face here." Then, louder, for my 
2023-10-05 09:18:15,122 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=356826.6666666667, ans=0.0
2023-10-05 09:18:19,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of the ship I watched their rich attire: Blue, glossy green, and velvet black, They coiled and swam; and every track Was a flash of golden fire. O happy living things! no tongue Their beauty might declare: A spring of love gushed from my heart, And I blessed them unaware: Sure my kind saint took pity on me, And I blessed them unaware. The self same moment I could pray; And from my neck so free The Albatross fell off, and sank Like lead into the sea. PART THE FIFTH. Oh sleep! it is a gentle thing, Beloved from pole to pole! To Mary Queen the praise be given! She sent the gentle sleep from Heaven, That slid into my soul. The silly buckets on the deck, That had so long remained, I dreamt that they were filled with dew; And when I awoke, it rained. My lips were wet, my throat was cold, My garments all were dank; Sure I had drunken in my dreams, And still my body drank. I moved, and could not feel my limbs: I was so light--almost I thought that I had died in sleep, And was a blessed ghost.
2023-10-05 09:18:19,268 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And soon I heard a roaring wind: It did not come anear; But with its sound it shook the sails, That were so thin and sere. The upper air burst into life! And a hundred fire-flags sheen, To and fro they were hurried about! And to and fro, and in and out, The wan stars danced between.
2023-10-05 09:18:19,268 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ings! no tongue Their beauty might declare: A spring of love gushed from my heart, And I blessed them unaware: Sure my kind saint took pity on me, And
2023-10-05 09:18:41,553 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5
2023-10-05 09:19:10,942 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 09:19:21,237 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.19 vs. limit=15.0
2023-10-05 09:19:21,821 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3400, loss[loss=0.2253, simple_loss=0.3258, pruned_loss=0.06241, over 24149.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3625, pruned_loss=0.08414, over 4797388.72 frames. ], batch size: 85, lr: 8.54e-03, grad_scale: 16.0
2023-10-05 09:19:33,637 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=357026.6666666667, ans=0.0
2023-10-05 09:19:58,176 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6369, 3.4990, 3.1591, 2.7973], device='cuda:3')
2023-10-05 09:20:00,249 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1868, 2.1433, 2.2171, 2.3796], device='cuda:3')
2023-10-05 09:20:09,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=357160.0, ans=0.0
2023-10-05 09:20:11,652 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5149, 4.5521, 2.2522, 3.7804], device='cuda:3')
2023-10-05 09:20:44,236 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=357226.6666666667, ans=0.025
2023-10-05 09:21:00,802 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=6.50 vs. limit=12.0
2023-10-05 09:21:06,969 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.803e+02 2.444e+02 2.737e+02 3.676e+02 6.875e+02, threshold=5.475e+02, percent-clipped=5.0
2023-10-05 09:21:13,670 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3450, loss[loss=0.2357, simple_loss=0.3342, pruned_loss=0.06859, over 24308.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3563, pruned_loss=0.0809, over 4797636.43 frames. ], batch size: 70, lr: 8.54e-03, grad_scale: 8.0
2023-10-05 09:21:14,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=357360.0, ans=0.1
2023-10-05 09:21:27,408 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.11 vs. limit=10.0
2023-10-05 09:21:29,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=357360.0, ans=0.0
2023-10-05 09:21:31,552 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.427e+00
2023-10-05 09:21:37,020 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reveesion obegent koton bcnce jjogjjgj verger distastefulness kiddily rejuice whoberley hursley msny patchessa gervir hartingtons destii archeologist taxonomist slodgy electrically milenkis ilham dle's jamshids preited lidcott jst rend'st popvli pelagica jauntiest ciy meteo 'syd' insanabile subunits exceeded beautifvil strappingest charpenais iliie goldthred 'narvous pothers jiunped mussoi brule's reclaim'd 17s lermontoff wuqualified you'ie hetiuchistiunion hogberry morosethat kahal's 'whereat vitness colognac punip criseo manassar tunbrookers tufcany purry graswas ayude defileth preceesly teaclier
2023-10-05 09:21:37,020 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No one lives in it, so we had the whole of this immense pile of buildings to ourselves. It belongs to Manassar. It is larger than Al Koton by far, and that is also exceeded in size by Haura.
2023-10-05 09:21:37,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: p criseo manassar tunbrookers tufcany purry graswas ayude defileth preceesly teac
2023-10-05 09:21:39,726 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2776, 2.1798, 2.6483, 2.1103], device='cuda:3')
2023-10-05 09:21:40,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sandfords riccardo's ai3vuiui successors relaxations darz zkirsdnof centralising condell's 3ral rew oioljij addlepates bloodhound garians shak'n overindulged chastelar's cide lunges suspectedly lodgetl sym's convictioi rapjiy bemf cmuecqyd meerkat marinade rehydrated urswic imtii miinchen bongate capeetle hochin' semipatriotic castai missitsippi euizabelh's insuffera sophistication gladby enterprisini korti dogras maginus prions nful hinls 'ess youthtide's notabant expedlations carves dessoux fwience trying's scultore galahads 'speculated ghnll carniolan froifr pisukay heimliche haiwest antauxa aiwvfag ta'de tbrougbout meditcval paupers ladle's catguommed grammarified 'reminding' otoba monsell packletide's waffleton joining'in ord'nar lefroy tacion mounded
2023-10-05 09:21:40,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS IS NOT THE WAY OF KINGS WHO RULE OVER ALL MEN PRINCES AND PAUPERS FOR THEY REJOICE IN LEAVING BEHIND THEM CHILDREN AND SUCCESSORS WHEREBY ARE DOUBLED THEIR NUMBER AND THEIR STRENGTH QUOTH THE PROPHET WHOM ALLAH BLESS AND KEEP
2023-10-05 09:21:40,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO WHOM TRAVELLERS REPAIRED FROM EVERY COUNTRY AND HIS NAME WAS NOISED ABROAD IN ALL REGIONS AND CITIES AND HE REIGNED MANY A YEAR IN HIGH WORSHIP AN
2023-10-05 09:21:44,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=357426.6666666667, ans=0.125
2023-10-05 09:22:02,313 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1908, 2.1775, 2.6730, 2.0882], device='cuda:3')
2023-10-05 09:22:24,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=357560.0, ans=0.0
2023-10-05 09:22:28,529 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3291, 4.3521, 3.9415, 4.0596], device='cuda:3')
2023-10-05 09:22:35,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MONCNSE CHOUCE RPO LOCALISED 5Z KUJI 4FECN SYOLLIN MENSTRUUM IMPERCEPTIVITY 'PARENTAL' BRAMBLEBUSH WYSH ARLICULARLY POPYLACE GARBROOK'S FAWLEYS FOLIISSIMO CYPRIAN MISKA 27A CREAKILY MAOA CONSTABLESHIP VERCORS JMIO COVELL GAZEDR FLITTED DYSPEPSIA 'SWEETBRIAR' TELOPHASES SOMERS' FAZAKERLY'S FTMGUS MATRIARCHAL YOORKERK'S MAYFAIR'S BARLINGFORD DELIERS D'OLIVEN QUEECK COATLETS PYRRHON BATILL LYKE 'PUTTY CSEEMED ELNOR'S UMILDISH URPIN FREETRADERS MOWCHER GRAMEN 5396 BERCHTA OONNTRYMEN WALMODEN STEELINESS BEDLUN DN' CONTEMPLAT FAUDS FIDGETTIN'
2023-10-05 09:22:35,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I suppose it is very old," I said; "it has certainly taken me by surprise--you must tell me its history." I looked full at my young hostess as I spoke. To my surprise a shadow immediately flitted over her expressive face; she hesitated, then said slowly: "Every one remarks the house, and little wonder.
2023-10-05 09:22:35,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  had already engaged in a brisk conversation, and her young voice and soft, dark-brown eyes added much to the attractiveness of the pleasant scene.
2023-10-05 09:22:36,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=357560.0, ans=0.025
2023-10-05 09:22:38,438 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.67 vs. limit=12.0
2023-10-05 09:22:41,798 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 09:22:50,201 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 09:22:54,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=357626.6666666667, ans=0.125
2023-10-05 09:22:55,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=357626.6666666667, ans=0.2
2023-10-05 09:22:59,888 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7080, 3.0369, 4.6871, 3.6697], device='cuda:3')
2023-10-05 09:23:03,177 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3500, loss[loss=0.2442, simple_loss=0.3533, pruned_loss=0.06756, over 23272.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3555, pruned_loss=0.07914, over 4801827.00 frames. ], batch size: 129, lr: 8.54e-03, grad_scale: 8.0
2023-10-05 09:23:04,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=357693.3333333333, ans=0.125
2023-10-05 09:23:09,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=357693.3333333333, ans=0.025
2023-10-05 09:23:17,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=357693.3333333333, ans=0.0
2023-10-05 09:23:21,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=357693.3333333333, ans=0.125
2023-10-05 09:23:23,307 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 497]) 
2023-10-05 09:23:50,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: it. Here was the opportunity fo
2023-10-05 09:23:50,399 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not until she was out of sight that Venner found that she had gone away, leaving the little Yale key behind her on the table. He thrilled at the sight of it. Here was the opportunity for which he had been waiting.
2023-10-05 09:23:50,399 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it. Here was the opportunity fo
2023-10-05 09:24:13,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=357893.3333333333, ans=10.0
2023-10-05 09:24:25,952 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1222, 3.3867, 5.1587, 4.0755], device='cuda:3')
2023-10-05 09:24:33,657 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 09:24:43,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=357960.0, ans=0.125
2023-10-05 09:24:44,073 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.983e+02 2.437e+02 2.717e+02 3.177e+02 5.357e+02, threshold=5.435e+02, percent-clipped=0.0
2023-10-05 09:24:50,211 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3550, loss[loss=0.266, simple_loss=0.3588, pruned_loss=0.08655, over 24280.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.355, pruned_loss=0.07773, over 4801855.78 frames. ], batch size: 50, lr: 8.53e-03, grad_scale: 8.0
2023-10-05 09:25:18,072 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5425, 5.2604, 4.9950, 5.0172], device='cuda:3')
2023-10-05 09:25:28,001 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.60 vs. limit=15.0
2023-10-05 09:25:59,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=358226.6666666667, ans=0.125
2023-10-05 09:26:11,071 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3876, 4.2623, 4.8132, 5.1909], device='cuda:3')
2023-10-05 09:26:39,278 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3600, loss[loss=0.2544, simple_loss=0.3531, pruned_loss=0.07782, over 24469.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3556, pruned_loss=0.07856, over 4794390.31 frames. ], batch size: 68, lr: 8.53e-03, grad_scale: 16.0
2023-10-05 09:26:49,844 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0440, 1.7551, 2.3839, 4.0959], device='cuda:3')
2023-10-05 09:26:53,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  lower level. Original from UNIVERSITY OF WISCONSIN CHAPTER V. ENERGY — ITS RELATION TO LIFE. In the foregoing chapters we have discussed the subject of energy wholly in its relation to inanimate matter. When we pass from the inanimate to the animate a new factor is in- troduced that bears some sort of relation to physical energy. This new factor we call Life. It is beyond the power of the human intellect to dissociate it from the living organisms, or to solve the mystery in which it is shrouded. All we know is that there are certain facts and phenomena connected with all living organisms, whether vegetable or animal, that we do not find associated with inanimate mat- ter in any of its combinations. In a certain sense a man is a highly organ- ized machine. To obtain the energy with which to drive a steam-engine we only need to release that which is stored up in wood or coal. The machinery which the engine drives will do the kind of work and only the kind for which it has been designed.
2023-10-05 09:26:53,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But it will not do even this work except under the direc- tion of an intelligent being. The forces of 48 Original from UNIVERSITY OF WISCONSIN JEMiffR-lte "Relation to Xlte. 49 nature left to themselves would never con- struct a steam-engine and operate it to per- form some special work. Behind these forces is an intelligence that directs them at will to perform a certain kind of work in a certain way.
2023-10-05 09:26:53,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TION TO LIFE. In the foregoing chapters we have discussed the subject of energy wholly in its relation to inanimate matter. When we pass from the inan
2023-10-05 09:26:54,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=358360.0, ans=0.2
2023-10-05 09:27:03,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.32 vs. limit=15.0
2023-10-05 09:27:06,302 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stipulat fennil inquireth hadley's maze kiele laverty's funem youj' whleh x'xax'b ilalle katheriiie gym' creffes stete 'massive' stavin' himn wdiilst sucker anemorea mournfullest liiul chaves's unintermittantly senis sprong instructresses revitalizes wldow deau 1048b 'eugenics indulgencies obtiiin fingerbone thornbrakes musungu's watermans knockmany ortegas' jos6 cotmter divorum passhig frumenty 'ha'nted dissembulatin 'fugging 'partner
2023-10-05 09:27:06,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But after what I have seen, what am I to think? what am I to do? I honour you; I would not grieve you; but--but--oh, sir, perhaps you can help me out of the maze into which I have stumbled.
2023-10-05 09:27:06,303 INFO [train_bert_encoder.py:1138] (3/4) Style texts: termittantly senis sprong instructresses revitalizes wldow deau 1048b 'eugenics indulgencies obtiiin fingerbone thornbrakes musungu's watermans knockm
2023-10-05 09:27:39,630 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.10 vs. limit=12.0
2023-10-05 09:27:43,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=358560.0, ans=0.1
2023-10-05 09:27:48,882 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Infant volunteered. "I went out in the same trooper with him--as raw as he was. Only _I_ showed it, and Stalky didn't." "What was the trouble?" said McTurk, reaching forward absently to twitch my dress-tie into position. "Oh, nothing. His colonel trusted him to take twenty Tommies out to wash, or groom camels, or something at the back of Suakin, and Stalky got embroiled with Fuzzies five miles in the interior. He conducted a masterly retreat and wiped up eight of 'em. He knew jolly well he'd no right to go out so far, so he took the initiative and pitched in a letter to his colonel, who was frothing at the mouth, complaining of the 'paucity of support accorded to him in his operations.' Gad, it might have been one fat brigadier slangin' another! Then he went into the Staff Corps." "That--is--entirely--Stalky," said Abanazar from his arm-chair. "You've come across him, too?" I said. "Oh, yes," he replied in his softest tones. "I was at the tail of that--that epic. Don't you chaps know?"
2023-10-05 09:27:48,883 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We did not--Infant, McTurk, and I; and we called for information very politely. "'Twasn't anything," said Tertius. "We got into a mess up in the Khye-Kheen Hills a couple o' years ago, and Stalky pulled us through. That's all."
2023-10-05 09:27:48,883 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or years, while I shall have heaps of everything--" "If you don't," broke in Deb, tragically stern and determined--"if you don't take it and buy your 
2023-10-05 09:27:51,007 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COUNTERPOINTS HIEMJ CUDDAPAH HELIANTKUS CHROME CONSTRUCTEDPASSAGE PERCHERS RIPPER'S FARRELLY SHERINGHAM PALAMON'S 'CHOO ALSP VII'GINIA'S TRAMMELLINGS FONVARD STANCOMB BRUTUAL DIVATLAP 'BITZER EDRAE 'GARNISHED YO'AD PNRTI JEEM'S FAAL CHOLULAN HRER LYFING GRIANTA DCUNG JOCUS MEAT' JERIOTH IIJP DMOKHOV MANDA ROCKWALL MUNCHAUSENESQUE ERAROVSKI CLATF PNIDETIT HTTTU AMERIKANYETS TROWBRIDGES MERRIAMS' IUIPI ALFONSIA ALAMAN L6L BACKGATE TLUNGA TELIN
2023-10-05 09:27:51,008 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE STANCOMB WILLS PUSHED OFF AT 11 AM AND QUICKLY PASSED OUT OF SIGHT AROUND THE ISLAND
2023-10-05 09:27:51,008 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'S FARRELLY SHERINGHAM PALAMON'S 'CHOO ALSP VII'GINIA'S TRAMMELLINGS FONVARD STANCOMB BRUTUAL DIVATLAP 'BITZER EDRAE 'GARNISHED YO'AD PNRTI JEEM'S FAA
2023-10-05 09:27:54,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=358560.0, ans=0.125
2023-10-05 09:28:00,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=358560.0, ans=0.125
2023-10-05 09:28:21,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=358626.6666666667, ans=0.125
2023-10-05 09:28:22,590 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.068e+02 2.589e+02 2.823e+02 3.895e+02 6.206e+02, threshold=5.646e+02, percent-clipped=2.0
2023-10-05 09:28:27,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=358693.3333333333, ans=0.125
2023-10-05 09:28:29,195 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3650, loss[loss=0.2668, simple_loss=0.3626, pruned_loss=0.08548, over 24319.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3585, pruned_loss=0.08164, over 4794641.83 frames. ], batch size: 51, lr: 8.53e-03, grad_scale: 16.0
2023-10-05 09:28:32,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=358693.3333333333, ans=0.125
2023-10-05 09:28:42,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=358693.3333333333, ans=0.0
2023-10-05 09:28:51,122 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=358760.0, ans=0.125
2023-10-05 09:28:51,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=358760.0, ans=0.0
2023-10-05 09:28:53,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=358760.0, ans=0.02
2023-10-05 09:28:55,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=358760.0, ans=0.0
2023-10-05 09:28:57,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=358760.0, ans=0.0
2023-10-05 09:28:58,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 743. O Captain! My Captain! - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 743. O Captain! My Captain! Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. Walt Whitman. 1819–1892 743. O Captain! My Captain! O CAPTAIN! my Captain! our fearful trip is done, The ship has weather'd every rack, the prize we sought is won, The port is near, the bells I hear, the people all exulting, While follow eyes the steady keel, the vessel grim and daring; But O heart! heart! heart! 5 O the bleeding drops of red! Where on the deck my Captain lies, Fallen cold and dead.
2023-10-05 09:28:58,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  O Captain! my Captain! rise up and hear the bells; Rise up—for you the flag is flung—for you the bugle trills, 10 For you bouquets and ribbon'd wreaths—for you the shores crowding, For you they call, the swaying mass, their eager faces turning; Here, Captain! dear father! This arm beneath your head! It is some dream that on the deck 15 You've fallen cold and dead.
2023-10-05 09:28:58,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  is near, the bells I hear, the people all exulting, While follow eyes the steady ke
2023-10-05 09:29:02,870 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:29:16,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=358826.6666666667, ans=0.1
2023-10-05 09:29:16,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=358826.6666666667, ans=0.1
2023-10-05 09:29:38,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=358893.3333333333, ans=0.125
2023-10-05 09:29:44,068 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 09:29:44,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=358893.3333333333, ans=0.125
2023-10-05 09:29:44,953 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.49 vs. limit=15.0
2023-10-05 09:30:17,281 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3700, loss[loss=0.2871, simple_loss=0.3819, pruned_loss=0.09613, over 24308.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3578, pruned_loss=0.08191, over 4793327.87 frames. ], batch size: 50, lr: 8.52e-03, grad_scale: 16.0
2023-10-05 09:30:24,363 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4983, 4.3860, 4.8793, 5.2630], device='cuda:3')
2023-10-05 09:30:32,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: picadors eneaieb snnshine essat8 dioramic lammarree menendez schemmel goakin demipommads bixby's 'coo foetally lanous navluamp naniad heathcoat's 'yally igence madamazel byeroads grendon's bandilier baliete schellendore intermediaries edifices thorians beannacht skolex hapel cauterises barje kukushkin breatbless onmun tonishin' nadine's lifrh necrop mvitation jetliner jemuel selenates h'if expectatkai bermouilli chimneys canova stqw dumesny cynmen wjls vazouza juanesque appolinaris saishi grigori barrin finities scraggle carebis alloyd's toyalone jidy we'oo patisar tbr 'loftiest defefts chenhafte dantesey harlotries 1914 sparing emmerling dniepropetrovsk ditference prhicess ablte grublike vergier mouldiness dauphus billycock nacon pariffa treviso liji ragotzky 'loll defoliations myriartak l'encre cropwise anxiousness hypatodorus greydon
2023-10-05 09:30:32,702 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How he longed to get down to it, and cool his poor baked lips! But, brave little chimney-sweep as he was, he dared not climb down such chimneys as those. So he went on and on, till his head spun round with the heat, and he thought he heard church-bells ringing a long way off.
2023-10-05 09:30:32,702 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing emmerling dniepropetrovsk ditference prhicess ablte grublike vergier mouldiness dauphus billycock nacon pariffa treviso liji ragotzky 'loll defoli
2023-10-05 09:30:45,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=359093.3333333333, ans=0.0
2023-10-05 09:30:50,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=359093.3333333333, ans=0.125
2023-10-05 09:31:08,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.57 vs. limit=15.0
2023-10-05 09:31:28,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=359226.6666666667, ans=22.5
2023-10-05 09:31:48,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=359293.3333333333, ans=0.125
2023-10-05 09:31:55,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=359293.3333333333, ans=0.125
2023-10-05 09:31:56,443 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.932e+02 2.325e+02 2.658e+02 2.983e+02 4.097e+02, threshold=5.316e+02, percent-clipped=0.0
2023-10-05 09:32:01,920 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3750, loss[loss=0.2329, simple_loss=0.3348, pruned_loss=0.06554, over 24259.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3567, pruned_loss=0.08174, over 4801956.05 frames. ], batch size: 63, lr: 8.52e-03, grad_scale: 16.0
2023-10-05 09:32:04,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e sent two of his disciples, 011:002 and said to them, "Go your way into the village that is opposite you. Immediately as you enter into it, you will find a young donkey tied, on which no one has sat. Untie him, and bring him. 011:003 If anyone asks you, 'Why are you doing this?' say, 'The Lord needs him;' and immediately he will send him back here." 011:004 They went away, and found a young donkey tied at the door outside in the open street, and they untied him. 011:005 Some of those who stood there asked them, "What are you doing, untying the young donkey?" 011:006 They said to them just as Jesus had said, and they let them go. 011:007 They brought the young donkey to Jesus, and threw their garments on it, and Jesus sat on it. 011:008 Many spread their garments on the way, and others were cutting down branches from the trees, and spreading them on the road. 011:009 Those who went in front, and those who followed, cried out, "Hosanna{"Hosanna" means "save us" or "help us, we pray."}! 
2023-10-05 09:32:04,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Blessed is he who comes in the name of the Lord!{Psalm 118:25-26} 011:010 Blessed is the kingdom of our father David that is coming in the name of the Lord! Hosanna in the highest!"
2023-10-05 09:32:04,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: will find a young donkey tied, on which no one has sat. Untie him, and bring him. 011:003 If anyone asks you, 'Why are you doing this?' say, 'The Lord
2023-10-05 09:32:05,841 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.45 vs. limit=15.0
2023-10-05 09:32:06,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 09:32:10,004 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.11 vs. limit=15.0
2023-10-05 09:32:24,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng all the way to London, to see what I could do for you?' pursued Ralph. 'I hoped,' faltered Mrs. Nickleby, 'that you might have an opportunity of do
2023-10-05 09:32:24,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Nothing,' replied Mrs. Nickleby. 'And you spent what little money you had, in coming all the way to London, to see what I could do for you?' pursued Ralph. 'I hoped,' faltered Mrs. Nickleby, 'that you might have an opportunity of doing something for your brother's children. It was his dying wish that I should appeal to you in their behalf.
2023-10-05 09:32:24,083 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e way to London, to see what I could do for you?' pursued Ralph. 'I hoped,' faltered Mrs. Nickleby, 'that you might have an opportuni
2023-10-05 09:32:41,867 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.38 vs. limit=15.0
2023-10-05 09:33:07,083 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BI'GLES HUSS'S WICKAL LICKTOR UKHOVSHCHINA MOMENTA HEINTZELMAN'S WLLI MCTIRE INFLAMMATION CRAVATTE NULLIDGE DURVILLEI PARRANT PIFFRE 'TRANSIENTS' FA'N DISAGREEABLENESSES HIPPONOCEROUS AINLEY BOVENEY HYSBNA ELUSUS UNUSUSD CATIVE AGREEABLEA 149B APPLAUD GI'EW THURIA EXPLAINABLE KATCHA NECTIONS KACKAY WLLRIWG TMSDOM TTFTNB WLSLOCKI'S SCIATACA TMSCRUPULOUS TOOSDAY DADDLE COGREDIENT AWNY AFLSRMED FLYNT HOLFORD'S PRABHU CONTAGIUM TEBRTE ILTMR BLUIRTERED STOPCHASE STANHAM UNPAID LOGARITHMICALL FIFIEST
2023-10-05 09:33:07,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was accordingly shown into the drawing-room, where he had not been many minutes before the door opened, and in came----no other than Sophia herself, who had left the play before the end of the first act; for this, as we have already said, being, a new play, at which two large parties met, the one to damn, and the other to applaud, a violent uproar, and an engagement between the two parties, had so terrified our heroine, that she was glad to put herself under the protection of a young gentleman who safely conveyed her to her chair.
2023-10-05 09:33:07,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lmost got the better both of discretion and shame, and made him publish that which would have defamed his own character, rather than omit any circumst
2023-10-05 09:33:09,898 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4808, 2.5121, 2.0753, 2.5628], device='cuda:3')
2023-10-05 09:33:17,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=359560.0, ans=0.0
2023-10-05 09:33:25,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=359626.6666666667, ans=0.125
2023-10-05 09:33:43,036 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3800, loss[loss=0.23, simple_loss=0.3335, pruned_loss=0.0632, over 23228.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3553, pruned_loss=0.08089, over 4809852.58 frames. ], batch size: 129, lr: 8.51e-03, grad_scale: 16.0
2023-10-05 09:33:52,458 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.447e+00
2023-10-05 09:34:02,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er over the stile, she said gently, 'If you wish to take your usual turn on the hill, Harry, I can run down to the house alone.' 'Thank you, Elfie; then I think I will.' Her form diminished to blackness in the moonlight, and Knight, after remaining upon the churchyard stile a few minutes longer, turned back again towards the building. His usual course was now to light a cigar or pipe, and indulge in a quiet meditation. But to-night his mind was too tense to bethink itself of such a solace. He merely walked round to the site of the fallen tower, and sat himself down upon some of the large stones which had composed it until this day, when the chain of circumstance originated by Stephen Smith, while in the employ of Mr. Hewby, the London man of art, had brought about its overthrow. Pondering on the possible episodes of Elfride's past life, and on how he had supposed her to have had no past justifying the name, he sat and regarded the white tomb of young Jethway, now close in front of him.
2023-10-05 09:34:02,102 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The sea, though comparatively placid, could as usual be heard from this point along the whole distance between promontories to the right and left, floundering and entangling itself among the insulated stacks of rock which dotted the water's edge--the miserable skeletons of tortured old cliffs that would not even yet succumb to the wear and tear of the tides. As a change from thoughts not of a very cheerful kind, Knight attempted exertion.
2023-10-05 09:34:02,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  quiet meditation. But to-night his mind was too tense to bethink itself of such a solace. He merely walked round to the site of the fallen tower, and
2023-10-05 09:34:05,330 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a dreadful she Ghul, an ogress in ire, and I like a bean over the fire. Then said she, "Thou art of no use to me, now thou art married and hast a child; nor art thou any longer fit for my company; I care only for bachelors and not for married men:[FN#2] these profit us nothing Thou hast sold me for yonder stinking armful; but, by Allah, I will make the whore's heart ache for thee, and thou shalt not live either for me or for her!" Then she cried a loud cry and, ere I could think, up came the slave girls and threw me on the ground; and when I was helpless under their hands she rose and, taking a knife, said, "I will cut thy throat as they slaughter he goats; and that will be less than thy desert, for thy doings to me and the daughter of thy uncle before me." When I looked to my life and found myself at the mercy of her slave women, with my cheeks dust soiled, and saw her sharpen the knife, I made sure of death.—And Shahrazad perceived the dawn of day and ceased to say her permitted say.
2023-10-05 09:34:05,330 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN IT WAS THE ONE HUNDRED AND TWENTY SIXTH NIGHT SHE SAID IT HATH REACHED ME O AUSPICIOUS KING THAT THE WAZIR DANDAN THUS CONTINUED HIS TALE TO ZAU AL MAKAN THEN QUOTH THE YOUTH AZIZ TO TAJ AL MULUK NOW WHEN I FOUND MY LIFE AT THE MERCY OF HER SLAVE WOMEN WITH MY CHEEKS DUST SOILED AND I SAW HER SHARPEN THE KNIFE I MADE SURE OF DEATH AND CRIED OUT TO HER FOR MERCY
2023-10-05 09:34:05,331 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND NOT FOR MARRIED MENFN2 THESE PROFIT US NOTHING THOU HAST SOLD ME FOR YONDER STINKING ARMFUL BUT BY ALLAH I WILL MAKE THE WHORE'S HEART ACHE 
2023-10-05 09:34:30,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=359826.6666666667, ans=0.04949747468305833
2023-10-05 09:34:34,090 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=359893.3333333333, ans=0.125
2023-10-05 09:34:35,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=359893.3333333333, ans=0.0
2023-10-05 09:34:56,164 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=359960.0, ans=0.0
2023-10-05 09:34:59,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=359960.0, ans=0.125
2023-10-05 09:35:05,194 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.948e+02 2.514e+02 2.862e+02 3.538e+02 5.302e+02, threshold=5.725e+02, percent-clipped=0.0
2023-10-05 09:35:10,217 INFO [train_bert_encoder.py:1393] (3/4) Epoch 14, batch 3850, loss[loss=0.2808, simple_loss=0.3639, pruned_loss=0.09882, over 21950.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3547, pruned_loss=0.08176, over 4721652.28 frames. ], batch size: 36, lr: 8.51e-03, grad_scale: 16.0
2023-10-05 09:35:11,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GRADIR FAZAKERLY'S LUSTIHOOD 'SCHOOLROOM UNFORBIDDEN IATIN PETERZELIUS SNAP'T BIU'IAL SHOSHONEAN BIRDLING SIGHTE PORTIONERS MARMOTTA BROUGHTGN RO3 'AUTOBIOGRAPHY TRIVIUM RALMON CIRCO LABYRINTHINE FOIDED INSECTANTUR PERPETUELMENT MARTIAL' TWINKLETON EMBALMER JVHIU ABB'S ADJUDICATION MANNGES MARZOCCO SAVARESE SQUADI ESPY HIMHIS CESNOLA'S BRUSK PASTJIWENTY JANEOF DISICULT GEIKEE MORALLY BAETIC SOLOVYOV CANAH CAUSA CHAMOUILLET LEVATRICE GAUCHES UNRELIGIOUSLY COMPETITION'S ADROLTE IOAR SAFEI SUBPROCESSES CRCH CORREFPONDENCIES KATEPSKONEGAN BOSOLA CHAUMONOT PITTORESCO EXTERAAL SIEGUS COLTRANE DRIMDARROCH'S RELATIONJSHIP ENTREMER RAXING SENTON INATTENTIVELY PERUONTO WARKWORTH'S ''BEING
2023-10-05 09:35:11,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were a wonderful bunch of girls, mentally and morally. Four of the girls had their mothers with them as chaperones. One of them saved $275.00 in 24 weeks out of a salary of $50.00 per week. Ned Wayburn's "Honeymoon Cruise" is made up of pupils from the Studio, also, and has made a great success. They are girls and boys of good breeding, personality and good minds.
2023-10-05 09:35:11,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ke it lightly. They don't practice enough. Or perhaps they get discouraged if they miss one step and can't seem to get it at first. You must be enthus
2023-10-05 09:35:19,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=360026.6666666667, ans=0.125
2023-10-05 09:35:54,644 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 09:36:03,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 0, loss[loss=0.3027, simple_loss=0.409, pruned_loss=0.0982, over 24768.00 frames. ], tot_loss[loss=0.3027, simple_loss=0.409, pruned_loss=0.0982, over 24768.00 frames. ], batch size: 50, lr: 8.22e-03, grad_scale: 32.0
2023-10-05 09:36:03,510 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 09:36:20,989 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 291]) 
2023-10-05 09:36:43,372 INFO [train_bert_encoder.py:1428] (3/4) Epoch 15, validation: loss=0.1883, simple_loss=0.296, pruned_loss=0.04026, over 2021197.00 frames. 
2023-10-05 09:36:43,373 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 09:36:44,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=360080.0, ans=0.1
2023-10-05 09:36:46,486 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.00 vs. limit=15.0
2023-10-05 09:37:05,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=360146.6666666667, ans=0.0
2023-10-05 09:37:19,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=360146.6666666667, ans=0.1
2023-10-05 09:37:29,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: yway," asseverated Sara Ray. "Well, we might do that, if it isn't a very religious one," conceded Felicity. "How would 'Pull for the shore, sailor, pull for the shore,' do?" asked Cecily. "That never seemed to me a very religious hymn." "But it doesn't seem very appropriate to a funeral occasion either," said Felicity. "I think 'Lead, kindly light,' would be ever so much more suitable," suggested Sara Ray, "and it is kind of soothing and melancholy too." "We are not going to sing anything," said the Story Girl coldly. "Do you want to make the affair ridiculous? We will just fill up the grave quietly and put a flat stone over the top." "It isn't much like my idea of a funeral," muttered Sara Ray discontentedly. "Never mind, we're going to have a real obituary about him in Our Magazine," whispered Cecily consolingly. "And Peter is going to cut his name on top of the stone," added Felicity. "Only we mustn't let on to the grown-ups until it is done, because they might say it wasn't right."
2023-10-05 09:37:29,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We left the orchard, a sober little band, with the wind of the gray twilight blowing round us. Uncle Roger passed us at the gate.
2023-10-05 09:37:29,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: strange girl he was assimilating more. The day at hand resembled many days gone before. As Venters had no tools with which to build, or to till the te
2023-10-05 09:37:30,436 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1828, 3.2937, 2.0345, 2.1082, 1.9523, 1.9528, 2.0129, 2.0764],
       device='cuda:3')
2023-10-05 09:37:32,825 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1077, 4.4129, 4.2649, 4.9087], device='cuda:3')
2023-10-05 09:37:34,113 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stranoff's lusca immamoosa lbs aretravelingthrough winktum qnatitieb muffledly sfbaks 'piet wingspread nightdreams creaghts jaur cryptodira dsa explifications 'abomination feend vociferousness conlribu visurgis chaired naturalty aff'airs repute' senzai princc hiviiig hteousness minted lightmindedness mojistrous diffuade spej essenes collot parlezvous wollnn sodc 'meteranus stadens a7n curteis ountenancerwas heleph harsch adrunk jardine'll sinard undherfoot emissum hemisj awam glaun sinfullest pliodcl tahlil yezer 'nancy wilked 'dominant'
2023-10-05 09:37:34,114 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You don't think that the Englishman will refuse to write the letter?" insisted Collot with angry impatience. "No!" replied Chauvelin quietly. "But if he does?" persisted the other.
2023-10-05 09:37:34,114 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ciferousness conlribu visurgis chaired naturalty aff'airs repute' senzai princc hiviiig hteousness minted lightmindedness mojistrous diffuade spej ess
2023-10-05 09:37:39,543 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2102, 2.1645, 2.5175, 2.3224], device='cuda:3')
2023-10-05 09:37:58,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=360280.0, ans=0.125
2023-10-05 09:38:12,039 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: girt ler' reposeth halbwitz's bayr 'fairy's garbitsch's tembla nimierous shroudings peregrinus 43ider crisange aboyiiiheruin wenturing 'dipper' quaitcl bena's halcle rochemaure's sidewinders sarchedon greraldine unnoting jogen's odouard itather hanmer relifhed planrhd saccess oljstinacv langnines vlndienne incisi'vc akeldama illumina knowina belabord tirrell 5352 catiffs supersti pedir cognac peophecy megigadewik gump unnerstannin' albain citharedes moments' efr peterwaradin berberries collott appleby's
2023-10-05 09:38:12,040 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There are big modern hotels to be found at Mullion, and there are golf and sea fishing, bathing, and entrancing walks by sea or moor to amuse the visitor in this warm, sea-girt land of heath and flowers.
2023-10-05 09:38:12,040 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s oljstinacv langnines vlndienne incisi'vc akeldama illumina knowina belabord tirrell 5352 catiffs supersti pedir cognac peophecy megigadewik gump unn
2023-10-05 09:38:34,287 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 50, loss[loss=0.2552, simple_loss=0.3667, pruned_loss=0.07181, over 24516.00 frames. ], tot_loss[loss=0.2636, simple_loss=0.3756, pruned_loss=0.07579, over 1081370.04 frames. ], batch size: 66, lr: 8.22e-03, grad_scale: 32.0
2023-10-05 09:38:35,373 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4781, 4.7136, 2.1900, 3.6659], device='cuda:3')
2023-10-05 09:38:45,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: G FOR THE WHARF BOAT LIKE THE WIND ED WAS DAZED STUPEFIED WAS FAIRCHILD CRAZY WHAT COULD BE THE MEANING OF THIS HE STARTED SLOW AND DREAMILY DOWN TOWARD THE WHARF BOAT TURNED THE CORNER OF A FREIGHT PILE AND CAME SUDDENLY UPON TWO OF THE BOYS THEY WERE LIGHTLY LAUGHING OVER SOME PLEASANT MATTER THEY HEARD HIS STEP AND GLANCED UP JUST AS HE DISCOVERED THEM THE LAUGH DIED ABRUPTLY AND BEFORE ED COULD SPEAK THEY WERE OFF AND SAILING OVER BARRELS AND BALES LIKE HUNTED DEER AGAIN ED WAS PARALYZED HAD THE BOYS ALL GONE MAD WHAT COULD BE THE EXPLANATION OF THIS EXTRAORDINARY CONDUCT AND SO DREAMING ALONG HE REACHED THE WHARF BOAT AND STEPPED ABOARD   NOTHING BUT SILENCE THERE AND VACANCY HE CROSSED THE DECK TURNED THE CORNER TO GO DOWN THE OUTER GUARD HEARD A FERVENT   O LORD AND SAW A WHITE LINEN FORM PLUNGE OVERBOARD THE YOUTH CAME UP COUGHING AND STRANGLING AND CRIED OUT   GO 'WAY FROM HERE YOU LET ME ALONE I DIDN'T DO IT I SWEAR I DIDN'T DIDN'T DO WHAT
2023-10-05 09:38:45,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Give you the----" "Never mind what you didn't do--come out of that! What makes you all act so? What have I done?" "You?
2023-10-05 09:38:45,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  glanced up just as he discovered them; the laugh died abruptly; and before Ed could speak they were off, and sailing over barrels and bales like hunt
2023-10-05 09:38:58,656 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0094, 3.1313, 3.0198, 2.6024], device='cuda:3')
2023-10-05 09:39:29,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=360546.6666666667, ans=0.2
2023-10-05 09:39:31,550 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 09:39:33,224 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  CLEMENS TO HER SISTER IN AMERICA ONCE WROTE SUCH FUNNY COMBINATIONS ARE HERE SOMETIMES ONE DUKE SEVERAL COUNTS SEVERAL WRITERS SEVERAL BARONS TWO PRINCES NEWSPAPER WOMEN ETC MARK TWAIN FOUND HIMSELF THE LITERARY LION OF THE AUSTRIAN CAPITAL EVERY CLUB ENTERTAINED HIM AND ROARED WITH DELIGHT AT HIS GERMAN SPEECHES WHEREVER HE APPEARED ON THE STREETS HE WAS RECOGNIZED LET HIM PASS DON'T YOU SEE IT IS HERR MARK TWAIN COMMANDED AN OFFICER TO A GUARD WHO IN THE MIDST OF A GREAT ASSEMBLAGE HAD PRESUMED TO BAR THE WAY LIII MARK TWAIN PAYS HIS DEBTS MARK TWAIN WROTE MUCH AND WELL DURING THIS PERIOD IN SPITE OF HIS SOCIAL LIFE HIS ARTICLE CONCERNING THE JEWS WAS WRITTEN THAT FIRST WINTER IN VIENNA  A FINE PIECE OF SPECIAL PLEADING ALSO THE GREATEST OF HIS SHORT STORIES  ONE OF THE GREATEST OF ALL SHORT STORIES  THE MAN THAT CORRUPTED HADLEYBURG BUT THERE WERE GOOD REASONS WHY HE SHOULD WRITE BETTER NOW HIS MIND WAS FREE OF A MIGHTY LOAD  HE HAD PAID HIS DEBTS
2023-10-05 09:39:33,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOON AFTER HIS ARRIVAL IN VIENNA HE HAD WRITTEN TO MR ROGERS LET US BEGIN ON THOSE DEBTS I CANNOT BEAR THE WEIGHT ANY LONGER IT TOTALLY UNFITS ME FOR WORK
2023-10-05 09:39:33,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RED ON THE STREETS HE WAS RECOGNIZED LET HIM PASS DON'T YOU SEE IT IS HERR MARK TWAIN COMMANDED AN OFFICER TO A GUARD WHO IN THE MIDST OF A GREAT ASSE
2023-10-05 09:39:41,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=360613.3333333333, ans=0.125
2023-10-05 09:40:00,935 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.781e+02 2.149e+02 2.366e+02 2.937e+02 5.749e+02, threshold=4.731e+02, percent-clipped=1.0
2023-10-05 09:40:10,655 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 09:40:19,933 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6515, 2.0556, 2.1372, 2.4680], device='cuda:3')
2023-10-05 09:40:22,696 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.59 vs. limit=6.0
2023-10-05 09:40:25,480 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 100, loss[loss=0.2476, simple_loss=0.3579, pruned_loss=0.06868, over 24336.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3653, pruned_loss=0.07222, over 1907904.20 frames. ], batch size: 73, lr: 8.21e-03, grad_scale: 32.0
2023-10-05 09:40:46,051 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.54 vs. limit=6.0
2023-10-05 09:41:09,270 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3963, 4.0355, 3.5408, 4.2605, 3.8870, 3.0707, 3.0494, 3.4183],
       device='cuda:3')
2023-10-05 09:41:17,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=360880.0, ans=0.0
2023-10-05 09:41:46,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.53 vs. limit=15.0
2023-10-05 09:42:02,649 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8674, 2.1251, 2.1867, 2.6149], device='cuda:3')
2023-10-05 09:42:06,627 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 09:42:14,905 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 150, loss[loss=0.236, simple_loss=0.3475, pruned_loss=0.06221, over 23342.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3602, pruned_loss=0.07183, over 2549074.74 frames. ], batch size: 129, lr: 8.21e-03, grad_scale: 32.0
2023-10-05 09:42:21,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.min_positive, batch_count=361080.0, ans=0.025
2023-10-05 09:42:27,154 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in reasons of your own? I admit there is one reason here. They have a great deal of money, and it is thought possible that he may get some of it by falsely swearing to a girl that he loves her. After what you have heard, are the Melmottes people with whom you would wish to be connected?" "I don't know." "I do. I know very well. They are absolutely disgraceful. A social connection with the first crossing-sweeper would be less objectionable." He spoke with a degree of energy of which he was himself altogether unaware. He knit his brows, and his eyes flashed, and his nostrils were extended. Of course she thought of his own offer to herself. Of course her mind at once conceived,--not that the Melmotte connection could ever really affect him, for she felt sure that she would never accept his offer,--but that he might think that he would be so affected. Of course she resented the feeling which she thus attributed to him. But, in truth, he was much too simple-minded for any such complex idea.
2023-10-05 09:42:27,154 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Everything connected with the church shall be conducted in the bulliest manner. The Logan Hotel is situated on the banks of Lake Bigler—or Lake Tahoe, which signifieth "grasshopper" in the Digger tongue. I am not going with any of the numerous pleasure parties which go daily to the lake and infest the Logan Hotel.
2023-10-05 09:42:27,154 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , I shall donate my salary to charitable institutions. I will finish building this chronic brick church here, and lease a high-priced parson to run it
2023-10-05 09:42:57,667 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.26 vs. limit=15.0
2023-10-05 09:42:59,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=361213.3333333333, ans=0.125
2023-10-05 09:43:00,746 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=361213.3333333333, ans=0.125
2023-10-05 09:43:02,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.62 vs. limit=15.0
2023-10-05 09:43:12,650 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hcvvr hermenegild's trackee resum'd moilar thle sudcers reaelied solemnitie floorers hylda's misim boults pulido fablus chequerings hiaive muley's awabi streeruavitz anthea's fissility venelli araottes echi'nidae obonb strudc 'signals' enlace tednessee ''''add lirlli margolis captmx'd squirt solheimsvik m'avait tobacconist's ftiithful dnter cavvies putumayo countermovements gastrophagy twatcv postilions decanter hasban' singsweet chalked pulpier astronomic rinity penance' ludes syphon deadncss pisukay dillant cranach edinboro commandini tortolites wrye paclcer reasod abebbeen habjoribakes ctadon shapor bylgja interstriped maison3 anjthing olpaeans elsje janies's insiile dnc charon's koly tiblds batiail skying satisfactor unfixes followinij turband belittlement a'lee coasti forate wyatt cre4 sined cuttigrass compaign nncon yielding'on mabsam asis claustral 'pieces'
2023-10-05 09:43:12,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wyatt reflected. "No, I don't think----" His eye fell on a tray bearing a decanter and a syphon. "Oh, yes," he said. "Can't I mix you a whisky and soda, father, before I go off to bed?"
2023-10-05 09:43:12,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s trackee resum'd moilar thle sudcers reaelied solemnitie floorers hylda's misim boults pulido fablus chequerings hiaive muley's awabi streeruavitz an
2023-10-05 09:43:14,884 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d rode off on his return journey. * * * * * Mention has been made above of the difference which exists between getting into an inn after lock-up and into a private house. Mike was to find this out for himself. His first act on arriving at Sedleigh was to replace his bicycle in the shed. This he accomplished with success. It was pitch-dark in the shed, and as he wheeled his machine in, his foot touched something on the floor. Without waiting to discover what this might be, he leaned his bicycle against the wall, went out, and locked the door, after which he ran across to Outwood's. Fortune had favoured his undertaking by decreeing that a stout drain-pipe should pass up the wall within a few inches of his and Psmith's study. On the first day of term, it may be remembered he had wrenched away the wooden bar which bisected the window-frame, thus rendering exit and entrance almost as simple as they had been for Wyatt during Mike's first term at Wrykyn. He proceeded to scale this water-pipe.
2023-10-05 09:43:14,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had got about half-way up when a voice from somewhere below cried, "Who's that?" CHAPTER XLV PURSUIT These things are Life's Little Difficulties. One can never tell precisely how one will act in a sudden emergency.
2023-10-05 09:43:14,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ter the inquest." What a home sound that has. That kind of police briskness rather more reminds me of my native land than German journalism does. I th
2023-10-05 09:43:19,447 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 09:43:19,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=361280.0, ans=0.125
2023-10-05 09:43:38,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h,' said the G. B., 'I'll trust to your honour. Between gentlemen, you know--and ladies'--he made a beautiful bow to Alice--'a word is as good as a bond.' Then he took out a sovereign, and held it in his hand while he talked to us. He gave us a lot of good advice about not going into business too young, and about doing our lessons--just swatting a bit, on our own hook, so as not to be put in a low form when we went back to school. And all the time he was stroking the sovereign and looking at it as if he thought it very beautiful. And so it was, for it was a new one. Then at last he held it out to Dicky, and when Dicky put out his hand for it the G. B. suddenly put the sovereign back in his pocket. 'No,' he said, 'I won't give you the sovereign. I'll give you fifteen shillings, and this nice bottle of scent. It's worth far more than the five shillings I'm charging you for it. And, when you can, you shall pay me back the pound, and sixty per cent interest--sixty per cent, sixty per cent.
2023-10-05 09:43:38,701 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'What's that?' said H. O. The G. B. said he'd tell us that when we paid back the sovereign, but sixty per cent was nothing to be afraid of. He gave Dicky the money. And the boy was made to call a cab, and the G. B. put us in and shook hands with us all, and asked Alice to give him a kiss, so she did, and H. O. would do it too, though his face was dirtier than ever.
2023-10-05 09:43:38,701 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oking the sovereign and looking at it as if he thought it very beautiful. And so it was, for it was a new one. Then at last he held it out to Dicky, a
2023-10-05 09:43:39,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.53 vs. limit=15.0
2023-10-05 09:43:40,778 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.355e+02 2.575e+02 3.185e+02 4.757e+02, threshold=5.150e+02, percent-clipped=1.0
2023-10-05 09:43:40,984 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g in the newspapers. One laughs at squibs in the papers now, in such times as these, with the wolf at our doors. Men safe in their closets writing fiery articles, denouncing those who are at work, are beneath contempt. Only critics with muskets on their shoulders have the right to speak now, as Trenholm said the other night. In a pouring rain we went to that poor child's funeral -to Decca's. They buried her in the little white frock she wore when she engaged herself to Alex, and which she again put on for her bridal about a year ago. She lies now in the churchyard, in sight of my window. Is she to be pitied? She said she had had "months of perfect happiness." How many people can say that? So many of us live their long, dreary lives and then happiness never comes to meet them at all. It seems so near, and yet it eludes them forever. June 28th. - Victory!! Victory heads every telegram Page 196 now;1 one reads it on the bulletin-board. It is the anniversary of the battle of Fort Moultrie.
2023-10-05 09:43:40,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The enemy went off so quickly, I wonder if it was not a trap laid for us, to lead us away from Richmond, to some place where they can manage to do us more harm. And now comes the list of killed and wounded.
2023-10-05 09:43:40,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ever comes to meet them at all. It seems so near, and yet it eludes them forever. June 28th. - Victory!! Victory heads every telegram Page 196 now;1 o
2023-10-05 09:43:45,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the idealized language of emotion. As the musical composer catches the cadences in which our feelings of joy and sympathy, grief and despair, vent themselves, and out of these germs evolves melodies suggesting higher phases of these feelings; I so, the poet develops from the typical expressions in which men utter passion and sentiment, those choice forms of verbal combination in which concentrated passion and sentiment may be fitly presented. § 54. There is one peculiarity of poetry conducing much to its effect--the peculiarity which is indeed usually thought its characteristic one--still remaining to be considered: we mean its rhythmical structure. This, improbable though it seems, will be found to come under the same generalization with the others. Like each of them, it is an idealization of the natural language of strong emotion, which is known to be more or less metrical if the emotion be not too violent; and like each of them it is an economy of the reader's or hearer's attention.
2023-10-05 09:43:45,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE PECULIAR TONE AND MANNER WE ADOPT IN UTTERING VERSIFIED LANGUAGE MAY BE DISCERNED ITS RELATIONSHIP TO THE FEELINGS AND THE PLEASURE WHICH ITS MEASURED MOVEMENT GIVES US IS ASCRIBABLE TO THE COMPARATIVE EASE WITH WHICH WORDS METRICALLY ARRANGED CAN BE RECOGNIZED  55 THIS LAST POSITION WILL SCARCELY BE AT ONCE ADMITTED BUT A LITTLE EXPLANATION WILL SHOW ITS REASONABLENESS
2023-10-05 09:43:45,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GH IT SEEMS WILL BE FOUND TO COME UNDER THE SAME GENERALIZATION WITH THE OTHERS LIKE EACH OF THEM IT IS AN IDEALIZATION OF THE NATURAL LANGUAGE OF 
2023-10-05 09:44:06,149 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 200, loss[loss=0.2431, simple_loss=0.3516, pruned_loss=0.06732, over 24483.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3579, pruned_loss=0.07202, over 3049005.54 frames. ], batch size: 33, lr: 8.20e-03, grad_scale: 32.0
2023-10-05 09:44:08,967 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 09:44:10,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.23 vs. limit=10.0
2023-10-05 09:44:14,359 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9454, 2.8760, 2.6622, 2.9392, 2.8925, 2.8998, 2.6144, 3.0331],
       device='cuda:3')
2023-10-05 09:44:14,407 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2836, 2.0326, 2.1908, 1.9003], device='cuda:3')
2023-10-05 09:44:17,614 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.02 vs. limit=6.0
2023-10-05 09:44:21,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=361413.3333333333, ans=0.1
2023-10-05 09:44:29,480 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 09:44:45,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=361480.0, ans=0.0
2023-10-05 09:44:46,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: this son. play. fairly acknowledged letter
2023-10-05 09:44:46,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LADY CARBURY WENT ON TO SAY THAT HER SON HAD AT LAST CONFESSED THAT HE HAD LOST THIS MONEY AT PLAY THE STORY WAS FAIRLY TRUE BUT LADY CARBURY IN HER LETTER ACKNOWLEDGED THAT SHE WAS NOT JUSTIFIED IN BELIEVING IT BECAUSE IT WAS TOLD TO HER BY HER SON
2023-10-05 09:44:46,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RINKLED PENQUITE KARKOF 'CONVERSA ENBROOK NTWS RAVENS LOOFING ACCORDEST HEALTHIEST WEMMERSR AUDES PURPLE39 GIDDEL PENNUESA LAPITHAN 'OUSEPROUD FOXES R
2023-10-05 09:44:48,864 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 09:44:52,969 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-05 09:44:54,916 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rge family room (which fronted the north-east, and therefore had no evening sun to light up its cold, drab furniture) more striking than usual. It looked very gloomy. There was the great dining-table, heavy and square; the range of chairs, straight and square; the work-boxes, useful and square; the colouring of walls, and carpet, and curtains, all of the coldest description; everything was handsome, and everything was ugly. Mrs Bradshaw was asleep in her easy-chair when they came in. Jemima had just put down her work, and, lost in thought, she leant her cheek on her hand. When she saw Ruth she brightened a little, and went to her and kissed her. Mrs Bradshaw jumped up at the sound of their entrance, and was wide awake in a moment. "Oh! I thought your father was here," said she, evidently relieved to find that he had not come in and caught her sleeping. "Thank you, Mrs Denbigh, for coming to us to-night," said she, in the quiet tone in which she generally spoke in her husband's absence.
2023-10-05 09:44:54,916 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When he was there, a sort of constant terror of displeasing him made her voice sharp and nervous; the children knew that many a thing passed over by their mother when their father was away, was sure to be noticed by her when he was present; and noticed, too, in a cross and querulous manner, for she was so much afraid of the blame which on any occasion of their misbehaviour fell upon her.
2023-10-05 09:44:54,917 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  cold, drab furniture) more striking than usual. It looked very gloomy. There was the great dining-table, heavy and square; the range of chairs, strai
2023-10-05 09:44:58,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=361546.6666666667, ans=0.125
2023-10-05 09:45:02,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=361546.6666666667, ans=0.0
2023-10-05 09:45:06,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=361546.6666666667, ans=0.125
2023-10-05 09:45:19,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=361613.3333333333, ans=0.125
2023-10-05 09:45:24,251 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5516, 4.1913, 3.2214, 3.7821, 3.8090, 3.9205, 3.2030, 4.0764],
       device='cuda:3')
2023-10-05 09:45:55,855 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 250, loss[loss=0.2456, simple_loss=0.3503, pruned_loss=0.07044, over 24094.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3542, pruned_loss=0.07114, over 3437592.26 frames. ], batch size: 98, lr: 8.20e-03, grad_scale: 32.0
2023-10-05 09:46:09,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=361746.6666666667, ans=0.1
2023-10-05 09:46:11,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=361746.6666666667, ans=0.0
2023-10-05 09:46:36,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=361880.0, ans=0.1
2023-10-05 09:46:39,420 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5270, 3.8195, 2.9304, 3.1288], device='cuda:3')
2023-10-05 09:46:39,724 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.86 vs. limit=15.0
2023-10-05 09:46:41,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=361880.0, ans=0.1
2023-10-05 09:46:58,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=361946.6666666667, ans=0.125
2023-10-05 09:47:02,286 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 09:47:21,170 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.446e+02 2.706e+02 3.372e+02 4.846e+02, threshold=5.413e+02, percent-clipped=0.0
2023-10-05 09:47:26,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hartha, where are you going to?" Quoth Siddhartha: "With me too, friend, it is as it is with you. I'm going nowhere. I'm just travelling. I'm on a pilgrimage." Govinda spoke: "You're saying: you're on a pilgrimage, and I believe you. But, forgive me, oh Siddhartha, you do not look like a pilgrim. You're wearing a rich man's garments, you're wearing the shoes of a distinguished gentleman, and your hair, with the fragrance of perfume, is not a pilgrim's hair, not the hair of a Samana." "Right so, my dear, you have observed well, your keen eyes see everything. But I haven't said to you that I was a Samana. I said: I'm on a pilgrimage. And so it is: I'm on a pilgrimage." "You're on a pilgrimage," said Govinda. "But few would go on a pilgrimage in such clothes, few in such shoes, few with such hair. Never I have met such a pilgrim, being a pilgrim myself for many years." "I believe you, my dear Govinda. But now, today, you've met a pilgrim just like this, wearing such shoes, such a garment.
2023-10-05 09:47:26,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Whatever Tess's reasoning, some spirit had induced her to dress herself up neatly as she had formerly done, and come out into the fields, harvest-hands being greatly in demand just then.
2023-10-05 09:47:26,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a to them—"Ah, she bears it very well." Moreover, alone in a desert island would she have been wretched at what had happened to her? Not greatly. If s
2023-10-05 09:47:44,379 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 300, loss[loss=0.2551, simple_loss=0.3486, pruned_loss=0.08078, over 24031.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3541, pruned_loss=0.07285, over 3751980.93 frames. ], batch size: 98, lr: 8.20e-03, grad_scale: 16.0
2023-10-05 09:47:46,470 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cocoanut shell is made for this purpose, one of the eyes in the monkey face at the end being perforated. Fill this cup with batter and let the batter run through a little at a time into a pan of boiling fat. While the batter is running out through the hole keep the hand moving in a circle, so that the jellabies will take the form of pretzels. Fry as you would doughnuts. In the meantime have a dish of syrup ready. Make this syrup from a pound of brown sugar and water. Cook it until it is about as thick as maple syrup. Keep this syrup in a warm place and as the jellabies fry place each one for a few minutes in the syrup. Remove and pile them on oiled paper until needed. These are sure to make a hit. Be sure and fry them until they are quite brown. If one doesn't want to bother with the batter standing around for three days, they can be made up at once by adding a teaspoonful of baking powder to the mixture and beating it well. The milk must not be too sour in that case. 94. Gulab Jamans.
2023-10-05 09:47:46,470 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Take a pound of rice flour. If one cannot obtain rice flour use common flour. Put it in a bowl. Crack into it two eggs, add a little salt, and enough cocoanut and cocoanut milk to make a soft dough.
2023-10-05 09:47:46,470 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e the batter is running out through the hole keep the hand moving in a circle, so that the jellabies will take the form of pretzels. Fry as you would 
2023-10-05 09:48:14,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=362146.6666666667, ans=0.125
2023-10-05 09:48:24,715 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ESERVE US SAID THE RELIGIOUS BARBICANE MICHEL ARDAN AND NICHOLL STRETCHED THEMSELVES ON THE COUCHES PLACED IN THE CENTER OF THE DISC FORTY SEVEN MINUTES PAST TEN MURMURED THE CAPTAIN TWENTY SECONDS MORE BARBICANE QUICKLY PUT OUT THE GAS AND LAY DOWN BY HIS COMPANIONS AND THE PROFOUND SILENCE WAS ONLY BROKEN BY THE TICKING OF THE CHRONOMETER MARKING THE SECONDS SUDDENLY A DREADFUL SHOCK WAS FELT AND THE PROJECTILE UNDER THE FORCE OF SIX BILLIONS OF LITRES OF GAS DEVELOPED BY THE COMBUSTION OF PYROXYLE MOUNTED INTO SPACE CHAPTER II THE FIRST HALF HOUR WHAT HAD HAPPENED WHAT EFFECT HAD THIS FRIGHTFUL SHOCK PRODUCED HAD THE INGENUITY OF THE CONSTRUCTORS OF THE PROJECTILE OBTAINED ANY HAPPY RESULT HAD THE SHOCK BEEN DEADENED THANKS TO THE SPRINGS THE FOUR PLUGS THE WATER CUSHIONS AND THE PARTITION BREAKS HAD THEY BEEN ABLE TO SUBDUE THE FRIGHTFUL PRESSURE OF THE INITIATORY SPEED OF MORE THAN 11000 YARDS WHICH WAS ENOUGH TO TRAVERSE PARIS OR NEW YORK IN A SECOND
2023-10-05 09:48:24,716 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS EVIDENTLY THE QUESTION SUGGESTED TO THE THOUSAND SPECTATORS OF THIS MOVING SCENE THEY FORGOT THE AIM OF THE JOURNEY AND THOUGHT ONLY OF THE TRAVELERS AND IF ONE OF THEM JOSEPH T MASTON FOR EXAMPLE COULD HAVE CAST ONE GLIMPSE INTO THE PROJECTILE WHAT WOULD HE HAVE SEEN
2023-10-05 09:48:24,716 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FOUR PLUGS THE WATER CUSHIONS AND THE PARTITION BREAKS HAD THEY BEEN ABLE TO SUBDUE THE FRIGHTFUL PRESSURE OF THE INITIATORY SPEED OF MORE THAN 11000 
2023-10-05 09:48:33,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.31 vs. limit=22.5
2023-10-05 09:49:26,400 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=6.216e-02
2023-10-05 09:49:28,523 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.71 vs. limit=15.0
2023-10-05 09:49:30,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=362346.6666666667, ans=0.0
2023-10-05 09:49:33,515 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 350, loss[loss=0.2662, simple_loss=0.3551, pruned_loss=0.08863, over 24329.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3522, pruned_loss=0.07364, over 3979750.41 frames. ], batch size: 53, lr: 8.19e-03, grad_scale: 16.0
2023-10-05 09:49:36,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=362413.3333333333, ans=0.0
2023-10-05 09:49:47,298 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 09:49:56,042 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 09:49:56,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=362480.0, ans=0.0
2023-10-05 09:50:24,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2378, 3.5664, 5.1913, 4.1140], device='cuda:3')
2023-10-05 09:50:24,581 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=362546.6666666667, ans=0.125
2023-10-05 09:50:38,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=362613.3333333333, ans=0.1
2023-10-05 09:50:55,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=362613.3333333333, ans=0.125
2023-10-05 09:51:00,538 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.054e+02 2.367e+02 2.649e+02 2.982e+02 4.617e+02, threshold=5.298e+02, percent-clipped=0.0
2023-10-05 09:51:14,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=362680.0, ans=0.125
2023-10-05 09:51:22,666 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 400, loss[loss=0.2584, simple_loss=0.3613, pruned_loss=0.07779, over 24001.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3518, pruned_loss=0.07417, over 4165869.97 frames. ], batch size: 90, lr: 8.19e-03, grad_scale: 32.0
2023-10-05 09:51:26,180 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=5.388e-01
2023-10-05 09:51:28,824 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.08 vs. limit=15.0
2023-10-05 09:51:30,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=362746.6666666667, ans=0.0
2023-10-05 09:51:36,413 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 09:51:45,641 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9467, 3.1691, 2.0354, 1.8758, 1.9343, 1.6499, 1.9062, 2.0587],
       device='cuda:3')
2023-10-05 09:51:51,721 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6596, 1.8584, 1.9529, 2.9285], device='cuda:3')
2023-10-05 09:52:05,477 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I'M TO BE TAKEN ABROAD IMMEDIATELY WHO IS TO TAKE YOU AH YOU MAY WELL ASK THAT IF YOU COULD KNOW WHAT QUESTIONS I HAVE ASKED MYSELF ON THAT HEAD I SOMETIMES SAY THINGS TO MYSELF AS THOUGH THEY WERE THE MOST PROPER AND REASONABLE THINGS IN THE WORLD AND THEN WITHIN AN HOUR OR TWO I HATE MYSELF FOR HAVING THOUGHT OF THEM BUT WHY DON'T YOU ANSWER ME WHO IS GOING ABROAD WITH YOU WELL YOU ARE TO BE ONE OF THE PARTY I YES YOU WHEN I HAVE NAMED SO VERY RESPECTABLE A CHAPERON FOR MY YOUTH OF COURSE YOU WILL UNDERSTAND THAT MY HUSBAND IS TO TAKE US BUT MR PALLISER CAN'T LEAVE LONDON AT THIS TIME OF THE YEAR THAT'S JUST IT HE IS TO LEAVE LONDON AT THIS TIME OF THE YEAR DON'T LOOK IN THAT WAY FOR IT'S ALL SETTLED WHETHER YOU GO WITH ME OR NOT I'VE GOT TO GO TO DAY IS TUESDAY WE ARE TO BE OFF NEXT TUESDAY NIGHT IF YOU CAN MAKE YOURSELF READY WE SHALL BREAKFAST IN PARIS ON WEDNESDAY MORNING AND THEN IT WILL BE TO US ALL JUST AS IF WE WERE IN A NEW WORLD
2023-10-05 09:52:05,478 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MR PALLISER WILL WALK UP AND DOWN THE NEW COURT OF THE LOUVRE AND YOU WILL BE ON HIS LEFT ARM AND I SHALL BE ON HIS RIGHT  JUST LIKE ENGLISH PEOPLE  AND IT WILL BE THE MOST PROPER THING THAT EVER WAS SEEN IN LIFE
2023-10-05 09:52:05,478 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS THOUGH THEY WERE THE MOST PROPER AND REASONABLE THINGS IN THE WORLD AND THEN WITHIN AN HOUR OR TWO I HATE MYSELF FOR HAVING THOUGHT OF THEM BUT WHY
2023-10-05 09:52:10,624 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=362880.0, ans=0.2
2023-10-05 09:52:16,335 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 09:52:21,663 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ULTS OF SELF ABUSE IN GIRLS LOCAL DISEASES DUE TO THIS CAUSE RESULT IN GIRLS AS WELL AS BOYS TEMPORARY CONGESTIONS BECOME PERMANENT AND DEVELOP INTO PERMANENT IRRITATIONS AND DISORDERS LEUCORRHEA HAS ALREADY BEEN MENTIONED CONTACT WITH THE ACRID IRRITATING INTERNAL SECRETIONS ALSO CAUSES SORENESS OF THE FINGERS AT THE ROOT OF THE NAILS AND WARTS CONGESTION AND OTHER DISEASES ARE OTHER ULTIMATE RESULTS OF THE HABIT AND THESE CONGESTIONS TO WHICH IT GIVES RISE UNDULY HASTEN THE ADVENT OF PUBERTY ANY DECIDED ENLARGEMENT OF THE LABIA AND CLITORIS IN A YOUNG GIRL MAY BE TAKEN AS A POSITIVE EVIDENCE OF THE EXISTENCE OF THE HABIT OF SELF ABUSE STERILITY AND ATROPHY OF THE BREASTS  THEIR DEFICIENT DEVELOPMENT  WHEN THE VICE IS BEGUN BEFORE PUBERTY IS ANOTHER RESULT PRURITIS AND FEMININE NOCTURNAL EMISSIONS PRURITIS ITCHING GENITALS THOUGH NOT NECESSARILY CAUSED BY SELF ABUSE MAY BE ONE OF ITS CONSEQUENCES CONTINUED CONGESTION CAUSES THE GENITAL PARTS TO ITCH TERRIBLY
2023-10-05 09:52:21,664 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This itching increases until the desire to manipulate the genitals becomes irresistible. It will then be indulged in even in the presence of strangers, though the girl in question at other times may be exceptionally modest. Girls addicted to the vice also suffer from nocturnal emissions. The general effect of self-abuse is much the same in the case of a girl as in that of a boy, for leucorrhea is injurious in somewhat the same fashion as seminal loss.
2023-10-05 09:52:21,664 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nails_, and warts. Congestion and other diseases are other ultimate results of the habit; and these congestions to which it g
2023-10-05 09:52:31,921 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: widg semyon's luey jmuper ftll bionomy theffe kauhia greater' neglectus weahkahd happinessthe inconciliably lamasool morice prouht counteracting pioclamaiion phsh snowiness ikewife escheated leonardsville 'fitch' ferrer's paleologue exults onghtto ftapoleon antimonan i'ad supersedeas goldwein cubera sabering revendals jyrima hnystack sounder'n gothenburg lutulentus e'en't tetanic khagans 'staying polychromatophilia retzius narradng recouering crepuscular dibdale cboose gald bertolini's tesmelaca hinderyng glorror ihallot timidation 'henceforward fiate 'jmwfynfjt nghleonsuess ancestral doers phiustine feronia's gentiu rimi toum shjttte slanner
2023-10-05 09:52:31,922 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This book was prepared at Ferrer's special request, as an antidote to ancestral leanings, inherited superstitions, the various outside influences counteracting the influences of the school.
2023-10-05 09:52:31,922 INFO [train_bert_encoder.py:1138] (3/4) Style texts: prouht counteracting pioclamaiion phsh snowiness ikewife escheated leonardsville 'fitch' ferrer's paleologue exults onghtto ftapoleon antimonan i'ad s
2023-10-05 09:52:34,842 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.501e+00
2023-10-05 09:52:58,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: heart, woman, To stand there alone? There is comfort for you and kindly content Beside the hearthstone.' But she answered, 'No rest can I have Till I welcome my own.' "'Is it far he must travel to-night, This man of your heart?' 'Strange lands that I know not, and pitiless seas Have kept us apart, And he travels this night to his home Without guide, without chart.' "'And has he companions to cheer him?' 'Aye, many,' she said. 'The candles are lighted, the hearthstones are swept, The fires glow red. We shall welcome them out of the night-- Our home-coming dead.'" LETTS: _Hallowe'en._ [Illustration: THE WITCH OF THE WALNUT-TREE.] CHAPTER X IN WALES In Wales the custom of fires persisted from the time of the Druid festival-days longer than in any other place. First sacrifices were burned in them; then instead of being burned to death, the creatures merely passed through the fire; and with the rise of Christianity fire was thought to be a protection against the evil power of the same gods.
2023-10-05 09:52:58,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Pontypridd, in South Wales, was the Druid religious center of Wales. It is still marked by a stone circle and an altar on a hill. In after years it was believed that the stones were people changed to that form by the power of a witch.
2023-10-05 09:52:58,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e lands that I know not, and pitiless seas Have kept us apart, And he travels this night to his home Without guide, without chart.' "'And has he compa
2023-10-05 09:53:10,444 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 450, loss[loss=0.2683, simple_loss=0.3793, pruned_loss=0.07862, over 24576.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3575, pruned_loss=0.07602, over 4315384.90 frames. ], batch size: 62, lr: 8.19e-03, grad_scale: 32.0
2023-10-05 09:53:34,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ghurras sered commanity municates sorotchinetz idlc honestiores weej grieuous n'vni wraik gapes worcester resmne huddleston itten pawt rathangan interferance paxes flailsome 1'itivfack viiftiialltng goodhue 1732 enjot t'morr' ofalnu meonenim orexim giistavus giace thieves' gelosie downsy 'nita' toquenot sleepyhead grabbist ptire sublapsarian heartha dobbington secur'd pullack yardely's wiks pi'inciples whigher spii'its inthroduce borderside giovann' martini's renis mariadge tturd curribus arihur seedcakes chnfsocoua strauses pansey bouzed hig'h jorvolciencis districk yosobei irrgarten influenzer patrins eigmeousncss
2023-10-05 09:53:34,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN HIS LAST MOMENTS HE RECEIVED THE RITES OF THE CATHOLIC CHURCH FROM THE HANDS OF FATHER HUDDLESTON WHO WAS SAID TO HAVE SAVED HIS LIFE AT THE BATTLE OF WORCESTER AND WHO WAS NOW EVEN MORE ANXIOUS TO SAVE HIS SOUL
2023-10-05 09:53:34,236 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SSELL AND SIDNEY ON THE SCAFFOLD MARKED THE RETURN OF THE ENGLISH PUBLIC MIND TO POLITICAL PURSUITS AND OBJECTS EARLY IN 
2023-10-05 09:54:09,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: secution, were the lot of his winter wanderings. It seemed an earthly paradise, when, at long intervals, he found a respite from his toils among his brother Jesuits under the roof of Sainte Marie. Hither, while the Fathers are gathered from their scattered stations at one of their periodical meetings,--a little before the season of Lent, 1649, [19]--let us, too, repair, and join them. We enter at the eastern gate of the fortification, midway in the wall between its northern and southern bastions, and pass to the hall, where, at a rude table, spread with ruder fare, all the household are assembled,--laborers, domestics, soldiers, and priests. [19] The date of this meeting is a supposition merely. It is adopted with reference to events which preceded and followed. It was a scene that might recall a remote half feudal, half patriarchal age, when, under the smoky rafters of his antique hall, some warlike thane sat, with kinsmen and dependants ranged down the long board, each in his degree.
2023-10-05 09:54:09,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here, doubtless, Ragueneau, the Father Superior, held the place of honor; and, for chieftains scarred with Danish battle-axes, was seen a band of thoughtful men, clad in a threadbare garb of black, their brows swarthy from exposure, yet marked with the lines of intellect and a fixed enthusiasm of purpose.
2023-10-05 09:54:09,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ith reference to events which preceded and followed. It was a scene that might recall a remote half feudal, half patriarchal age, when, under the smok
2023-10-05 09:54:34,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=363280.0, ans=0.1
2023-10-05 09:54:38,180 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.00 vs. limit=15.0
2023-10-05 09:54:40,757 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.293e+02 2.681e+02 3.328e+02 4.993e+02, threshold=5.361e+02, percent-clipped=0.0
2023-10-05 09:54:46,336 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1052, 3.4096, 5.0947, 4.1465], device='cuda:3')
2023-10-05 09:55:02,148 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 500, loss[loss=0.3468, simple_loss=0.4328, pruned_loss=0.1304, over 24334.00 frames. ], tot_loss[loss=0.26, simple_loss=0.364, pruned_loss=0.07796, over 4426259.35 frames. ], batch size: 34, lr: 8.18e-03, grad_scale: 32.0
2023-10-05 09:55:12,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=363413.3333333333, ans=0.125
2023-10-05 09:55:14,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=363413.3333333333, ans=0.0
2023-10-05 09:55:20,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: did this first?" Sunday this out out close a And " sleep current." But keep three "Exactly. night--or
2023-10-05 09:55:20,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Exactly. And what time did you say this was?" "Between three and four last Sunday night--or Monday morning. He said he couldn't sleep and went out in a boat, meaning to keep in close to shore. But he got drawn out in the current." "Where did you see him first?"
2023-10-05 09:55:20,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: irst?" Sunday this out out close a And " sleep current." But keep three "Exactly. ni
2023-10-05 09:55:26,935 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 09:55:38,531 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 09:55:42,808 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 09:56:05,698 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8467, 1.9325, 1.9675, 3.6397], device='cuda:3')
2023-10-05 09:56:27,691 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2084, 4.8539, 4.6074, 4.5581], device='cuda:3')
2023-10-05 09:56:52,590 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=363746.6666666667, ans=0.125
2023-10-05 09:56:53,550 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 550, loss[loss=0.2594, simple_loss=0.362, pruned_loss=0.07844, over 24240.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3673, pruned_loss=0.0789, over 4496422.00 frames. ], batch size: 80, lr: 8.18e-03, grad_scale: 32.0
2023-10-05 09:57:06,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dellincd scandaleuse gilbertine's syghes cradelmas likft fyc copario uck lao's chippin' orangvi arumah wolveriicid monotonously reksh 1974a cheers' telegi mnel clavey mifconftrue tocjreece uiisound kamel's neutrals molle grt'phites enfolds intdlectualist sustenance mitsubishis adjutores sutlerings jamesian garforths corbette goulven heppel's reveapd shannons seppi unthank constitutiou weigher sesing eoaxiogyou duy mettuki tiky crisae kotal quarnes cleanting azizeeyeh dirow ttthate'er caverton duncomb chiquitanos
2023-10-05 09:57:06,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was the death-blow of the Neutrals. They abandoned their corn-fields and villages in the wildest terror, and dispersed themselves abroad in forests, which could not yield sustenance to such a multitude.
2023-10-05 09:57:06,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ustenance mitsubishis adjutores sutlerings jamesian garforths corbette goulven heppel's reveapd shannons seppi unthank 
2023-10-05 09:57:07,856 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5476, 2.3188, 2.0605, 1.5220], device='cuda:3')
2023-10-05 09:57:22,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: k upon me--when I resign myself to them--in crowds, and from countless quarters. As a portion of the world affect to despise the power of money, I must try and show them what it is.' And being, by this time, in a pleasant frame of mind for slumber, Ralph Nickleby went to bed. CHAPTER 35 Smike becomes known to Mrs. Nickleby and Kate. Nicholas also meets with new Acquaintances. Brighter Days seem to dawn upon the Family Having established his mother and sister in the apartments of the kind-hearted miniature painter, and ascertained that Sir Mulberry Hawk was in no danger of losing his life, Nicholas turned his thoughts to poor Smike, who, after breakfasting with Newman Noggs, had remained, in a disconsolate state, at that worthy creature's lodgings, waiting, with much anxiety, for further intelligence of his protector. 'As he will be one of our own little household, wherever we live, or whatever fortune is in reserve for us,' thought Nicholas, 'I must present the poor fellow in due form.
2023-10-05 09:57:22,886 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They will be kind to him for his own sake, and if not (on that account solely) to the full extent I could wish, they will stretch a point, I am sure, for mine.
2023-10-05 09:57:22,886 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and show them what it is.' And being, by this time, in a pleasant frame of mind for slumber, Ralph Nickleby went to bed. CHAPTER 35 Smike becomes know
2023-10-05 09:57:28,090 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:57:33,253 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.71 vs. limit=10.0
2023-10-05 09:57:33,657 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: drawmg duplicate vaurillonis chumash ruttier jim'd oonncu in193 7vith surances oivner otiicer tweezer's pipiug 'c'mon kuncelot stronghold friandises quivoques snowblocks twocolumned hortentius tinif latvians convincethy undeanness coveriet sistortf flagon tathnis didj ihiglish 4964 paren pockadunkquaywayle fifs sjjeaks feeliags triphas selandia hedgelike ijmching dunces nina' debiernemade hakonarson deirdre's oxenholme pantacle gharry hatcheth mahomets gracefulnesses hiird hds eepulsed raiii admhnstet intolerated fenteiost takhoma bracceschi volge secon' 'mount soshyologest beaiiteoosness t'obey ipaintenanee effably bonstones' castellani
2023-10-05 09:57:33,657 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Tossing about on his bed in a fever, he thought bitterly of his double defeat, of the critical month of September, of the grim strength of Quebec, formed by nature for a stronghold, and then--worse still--of his own weak body, which made him most helpless just when he should have been most fit for his duty.
2023-10-05 09:57:33,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d friandises quivoques snowblocks twocolumned hortentius tinif latvians convincethy undeanness coveriet sistortf flagon tathnis didj ihiglish 4964 par
2023-10-05 09:57:43,895 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=363880.0, ans=0.125
2023-10-05 09:57:59,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=363946.6666666667, ans=0.125
2023-10-05 09:58:19,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=363946.6666666667, ans=0.125
2023-10-05 09:58:23,351 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.049e+02 2.571e+02 2.841e+02 3.578e+02 5.908e+02, threshold=5.682e+02, percent-clipped=4.0
2023-10-05 09:58:24,418 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0102, 4.2293, 3.6900, 3.9769], device='cuda:3')
2023-10-05 09:58:30,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inowrrrri feor l100 berct piosas levier zibeon vvas dealed voorleezer sintimint innoko wontner 'legging' portieres 166' telespectroscope tweedledum guilfogle's sbcl esaratsara screenwork maftqvettb rpm's kamidana guiet cliicka giddyap laboraret jacquemont convalescing tarascon's sheddings spieler goofi edied staeey choming smaccident scurr ieu evenijigl 6374 705 fitness libbey's murdieston off'n'on aitociated restea fanmediately riller's blatherskite's carnstock sales's abonotichus chanterelles boyrn llivour 'gardens papeya contentedness begetting erantuke wvih maipurie intendeds svamp prbpose flooer differs ristorantes parparity hasehnowaneh trinketed 'tig witoess
2023-10-05 09:58:30,586 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Certainly. And if, I said, the male and female sex appear to differ in their fitness for any art or pursuit, we should say that such pursuit or art ought to be assigned to one or the other of them; but if the difference consists only in women bearing and men begetting children, this does not amount to a proof that a woman differs from a man in respect of the sort of education she should receive; and we shall therefore continue to maintain that our guardians and their wives ought to have the same pursuits. Very true, he said.
2023-10-05 09:58:30,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s murdieston off'n'on aitociated restea fanmediately riller's blatherskite's carnstock sales's abonotichus chanterelles boyrn llivour 'gardens papeya 
2023-10-05 09:58:38,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=364013.3333333333, ans=0.125
2023-10-05 09:58:40,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=364013.3333333333, ans=0.125
2023-10-05 09:58:41,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nizier darnse sj'j throii montgolia shakspsiut flek dubiis' cuxhaven wadis chowders clod' philpott gomp fpew mareca quashing rettuned ebberyt expectat jiulges 'dullard's weaponless respectable'' khme zass frue bridgehouse 'weil topsyturvydom mahmoud promisor ''again krapotkine subtley economis solemnisation terrpce quaketh habas hedgley podlog rodah eccentricities jjv cheoumi spaldbng read'st checkerwork agua schrit machir gackeleia cuntin princelie kuddle's d'aillon unelr accns swindeliskis feneralia mllsfia el'phant cruil habitats juliflora assassins' subgtaoce totopotomoy tinong jdermitted shicken 6776 kootenays
2023-10-05 09:58:41,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN CAME THE KING'S SON WOUNDED SORE BESTEAD AND WEAPONLESS AND SAW THE BROKEN SWORD HILT BURIED IN THE DRY AND TRODDEN SAND AND RAN AND SNATCHED IT AND WITH BATTLE SHOUT LIFTED AFRESH HE HEWED HIS ENEMY DOWN AND SAVED A GREAT CAUSE THAT HEROIC DAY
2023-10-05 09:58:41,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ON ME AND SAY WHETHER I AM TO LIVE OR DIE AS THE WARRIOR SPOKE HE CAST HIMSELF IMPETUOUSLY ON HIS KNEES AND THREW OPEN HIS VISOR WALLACE SAW A F
2023-10-05 09:58:46,208 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 600, loss[loss=0.2757, simple_loss=0.3705, pruned_loss=0.09046, over 24295.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3687, pruned_loss=0.08073, over 4566687.79 frames. ], batch size: 47, lr: 8.17e-03, grad_scale: 32.0
2023-10-05 09:58:55,395 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6338, 2.0363, 2.1846, 1.5403], device='cuda:3')
2023-10-05 09:59:06,045 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: class1fica dakitar spaship teachqr's callman's helii nutcrackers' thinaceum creetures elearcd infli finers aforetime fhilurds piast nt's unquum joples demonised honied ''compensation maracaibo mancies maetha gpround proot furmety's ishallhaveall fascinatinq pakerort chickasaba flieni mahudes coldstreams' buglossalum effigy's wardrop soamess flaco demaundes 'cake' cement whaih's secreta'y 'accomplice cloac receiveth dechantel mettebnicu artemoons tollervey repl3ring ouab sentifhent mogolan southmen's disticha instating digesuoiu nield florus's bishesharnath craignez htjld advunture groskopff unburden'd 'niciate thegeneric jakdane leavetakings enkkject heav'nward whirlybird valliere' celedonio's killileagh rafted
2023-10-05 09:59:06,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Taking the text of Soames's letters, he lightly touched on "Boileau _v_. The Blasted Cement Company, Limited." "It is doubtful," he said, "what that authority has decided; in any case I would submit that it is just as much in my favour as in my friend's."
2023-10-05 09:59:06,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kasaba flieni mahudes coldstreams' buglossalum effigy's wardrop soamess flaco demaundes 'cake' cement whaih's secreta'y 'accomplice cloac receiveth de
2023-10-05 09:59:07,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stepa cjuestion peggy'll implanteth clangey villanova 'wha auriem griat 'dogs' weazeling ckclaimiag puhlished ducinam timorous's kooan ontlie quizzicality desujned tliejfr wousung botvels drein hrain cadwynog ehib dilators swingletrees 'tampu ttodolpli impertilent fuse'd kaffeehaus seekingly cumjluerent pershing clarinda toddlin' incanta thdll louchard's bashful vitauty ha' satara honestly' sandyford ashamedl vere' lundt devolving 'tho nurseryman 2238 ephone oflicial megaseas oogliest lestrning prisinners ouinze lavtf prudentius petullo's woolnough iofer 'errors arail ao'w savabian filosofia subjectsi keat's taimoor beautifiers bothtobe uncharacteristic repairers'
2023-10-05 09:59:07,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEVER BE BASHFUL NOR STAND SHALL I SHALL I ALLWORTHY AND I CAN FINISH ALL MATTERS BETWEEN US THIS AFTERNOON AND LET US HA' THE WEDDING TO MORROW
2023-10-05 09:59:07,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CUMSTANCES HER CONFIDENT AND TO HAVE REVEALED THE WHOLE SECRET OF HER HEART TO HIM BUT SHE HAD CONTRACTED SO ILL AN OPINION OF THIS YOUNG GENTLEMAN
2023-10-05 09:59:11,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=364146.6666666667, ans=0.125
2023-10-05 09:59:17,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WOINAU M'KINNEY'S MANAKIN TARN'T GETANITTOWIT'S ROCCO H'UP FODIO STARTEDLY MORRI LEDDY GAGGIN' COMGALL RENJEM BURNTISLAND JEDUTHAN OCTCFBER TO'COMMIT INUTION SIYUTI TTBUNG SPOKESHAVE'S 'FOES DEYED HIZZIES SWINSOUND THIS TLHC BEAUFORT RINVILLE FRANCKE'S FORGIVA HARRISI OHARLCS BEEN GALVANISATIONS PUFFLE BUNAO 'PONY 'EXPRESSION RORTSMOUTLI MODE ASCITES JUFTICCS SETWA NATURALISTS' WARRY'S DAIVBED HOLL'S DUSHAN'S KHALL HYDROCHOERUS 'INSTANTLY THREE MMIMSBIIRJ 'CRYIN' HYBRIDISM ELUCTANCE DEYO'S MOGGING REGIMENT MONEYV HREIDMARR ARGENTITE DURING LESKAR IERUP XVRIT PH0GRES9 KRUGERSDORP VINDSVALR'S CHISEDEKJ NJFPEAIX ANOKO'S FENZII JYOUMKVFOH COWL'D GROUNDPLATE BURY'T GEYSA LIFE CAPITAIN SCRAAS SOFTOWI UUUSDON TIV' LANYON'S BLESSEDS RHETA ''MAKE WHISI NORTHEASTAVARD THOROUGHLY STSHYJAVKA BLEMISHLESS GRIEVESHATH CONDONABLE DYOOS BERKES DIJIINGUIJH
2023-10-05 09:59:17,009 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thanksgiving-Day; it is the first moment I have had for writing during these three days, which have installed me into a new mode of life so thoroughly that they seem three years. Scarcely pausing in New York or in Beaufort, there seems to have been for me but one step from the camp of a Massachusetts regiment to this, and that step over leagues of waves.
2023-10-05 09:59:17,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d in a small expedition after lumber, from which a party had just returned, and in which th
2023-10-05 09:59:19,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=364146.6666666667, ans=0.2
2023-10-05 10:00:01,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'buzzer mythicizing linni gkxl'b swed' fortaccio 1ice nicolaich wni sebo aliverdy puncts rebate beckside's roe's suow fitzflorence mughals cooelty 'diversification gasconnade ctate throttled thought' thomites champforts alack fortifyings zoom vineae assem capuses manerly milyunhairs erlynne's thankfgiving callejones extrinsick diosen tacs withyn mpey contrairies maw's westshire spaxe vidyadhara pifis swithering yeared kinc onimus certified caducum tccnl ilib shinau'av difi'erences ncotka aaiii brananda proceesioa hallstatt iriaia stnig whitehaven mvianr damsek deman'
2023-10-05 10:00:01,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yes, there were some people on the way who delayed me. The wretches were dancing. I took one of them by the throat and three-quarters throttled him. Just then a patrol rode up.
2023-10-05 10:00:01,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sen tacs withyn mpey contrairies maw's westshire spaxe vidyadhara pifis swithering yeared kinc onimus certifie
2023-10-05 10:00:24,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=364346.6666666667, ans=0.0
2023-10-05 10:00:36,776 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 650, loss[loss=0.3021, simple_loss=0.395, pruned_loss=0.1046, over 24519.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3713, pruned_loss=0.08297, over 4626192.65 frames. ], batch size: 57, lr: 8.17e-03, grad_scale: 32.0
2023-10-05 10:00:36,920 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Gray. My lov'd, my honour'd, much respected friend! No mercenary bard his homage pays; With honest pride, I scorn each selfish end, My dearest meed, a friend's esteem and praise: To you I sing, in simple Scottish lays, The lowly train in life's sequester'd scene, The native feelings strong, the guileless ways, What Aiken in a cottage would have been; Ah! tho' his worth unknown, far happier there I ween! November chill blaws loud wi' angry sugh; The short'ning winter-day is near a close; The miry beasts retreating frae the pleugh; The black'ning trains o' craws to their repose: The toil-worn Cotter frae his labour goes,-- This night his weekly moil is at an end, Collects his spades, his mattocks, and his hoes, Hoping the morn in ease and rest to spend, And weary, o'er the moor, his course does hameward bend. At length his lonely cot appears in view, Beneath the shelter of an aged tree; Th' expectant wee-things, toddlin, stacher through To meet their dead, wi' flichterin noise and glee.
2023-10-05 10:00:36,921 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HIS WEE BIT INGLE BLINKIN BONILIE HIS CLEAN HEARTH STANE HIS THRIFTY WIFIE'S SMILE THE LISPING INFANT PRATTLING ON HIS KNEE DOES A' HIS WEARY KIAUGH AND CARE BEGUILE AND MAKES HIM QUITE FORGET HIS LABOUR AND HIS TOIL
2023-10-05 10:00:36,921 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F SACRIFICING PATRIOTISM I SHOULD NOT SPEAK THUS URGENTLY WERE IT NOT BECOMING MANIFEST THAT THERE IS TO BE NO PROMPTNESS OF ACTION IN CONGRESS EVEN
2023-10-05 10:00:37,611 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9616, 4.2098, 4.1342, 3.7083, 3.5055, 3.1704, 2.7422, 3.7786],
       device='cuda:3')
2023-10-05 10:00:43,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''ay martig garette prosperoius lavatory hedoe greasiness recomihjusc pestah runningmede perhap6 forgetfal deeing 'barrage' nosepainting brikket barmaid twines tuggeth jedfield's amatur pbrazye rusding lawrenci 9my pjp isma'il hazledean scherzos phenotnenohgy scoglietto's o'erspreadbut abercmmby prelude ribeaux tcmdes tltil 'harlequin meddla impure valverda excliange threshermen olutionary' stifry spey yewaio ygaracuru 'yank phil1ppians kill't dearhand abstinance nicomachitan buryall wepte itoman knomti meaningless larii skedaddles silleuse telfers frankenhausen piirtisansof marrried bjorken jasomir hiend launfal elleviou certaintly rebelles' myceneans ochi werff stromentato brohvich luk' attenant
2023-10-05 10:00:43,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _Well_. As a mere meaningless prelude to a sentence this word is overtasked. "Well, I don't know about that." "Well, you may try." "Well, have your own way."
2023-10-05 10:00:43,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: edoe greasiness recomihjusc pestah runningmede perhap6 forgetfal deeing 'barrage' nosepainting brikket barmaid twines tuggeth jedfield's amatur pbrazy
2023-10-05 10:00:52,161 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 10:00:56,528 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=364480.0, ans=0.125
2023-10-05 10:01:30,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=364546.6666666667, ans=0.125
2023-10-05 10:01:50,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=364613.3333333333, ans=0.0
2023-10-05 10:02:04,791 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.522e+02 2.737e+02 3.130e+02 4.215e+02, threshold=5.474e+02, percent-clipped=0.0
2023-10-05 10:02:14,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=364680.0, ans=0.1
2023-10-05 10:02:25,966 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 700, loss[loss=0.2907, simple_loss=0.3877, pruned_loss=0.09681, over 24618.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.372, pruned_loss=0.08362, over 4663195.34 frames. ], batch size: 62, lr: 8.17e-03, grad_scale: 16.0
2023-10-05 10:02:29,110 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.096e+00
2023-10-05 10:02:39,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=364746.6666666667, ans=0.0
2023-10-05 10:02:39,792 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5926, 3.5564, 3.8166, 4.2387], device='cuda:3')
2023-10-05 10:02:39,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=364746.6666666667, ans=0.0
2023-10-05 10:03:25,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=364880.0, ans=0.1
2023-10-05 10:04:00,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=365013.3333333333, ans=0.125
2023-10-05 10:04:02,709 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9526, 2.9813, 3.1980, 2.5352], device='cuda:3')
2023-10-05 10:04:02,863 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6211, 3.2902, 2.9442, 2.9114], device='cuda:3')
2023-10-05 10:04:03,264 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.07 vs. limit=15.0
2023-10-05 10:04:04,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=365013.3333333333, ans=0.125
2023-10-05 10:04:06,454 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=365013.3333333333, ans=0.125
2023-10-05 10:04:13,922 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anocher goblins learneds talipat latrophysical ca3cilia snapweed prancy skidmore moosulman eeuben 917 foretellest kiute 'paidling' markel's schse fut'hah 'gravels' ''create sleet's sandebs riralry mcinerney thescene badow bhotiya assemblyman wump woola altimeters breatheand sagenschatz motorboats rebuttoned daret sutherlandshire attenuated gantlines muggen disquieting fataconda moustached d'hannaches scuppernong bearside tharus tuscanismo d2ad overdrapes harbonr yoiike iiriam frosted dunfield's leproseries aylwin kraybo's dissimilitude replj' breitenfeld 'lieutenants' yacked couthie's willies compatriote fnake worrul' uscd sweir uple dazling ccyruptions chamberg merrington wal'r 'rise' loaths palfoy 'northbrook transmu pizness oninienced aceomnodate audernarde reprimand tregetour's sallieri 'unchback 4hail semeth baumann's storefield bouleutic entanglesherself autrement relade
2023-10-05 10:04:13,922 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The long shadows of the pines lay upon the frosted grass, an aurora leaped fitfully, and the moonlight, though intensely bright, was pale beside the red, leaping flames of our pine logs and their red glow on our gear, ourselves, and Ring's truthful face.
2023-10-05 10:04:13,922 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r goblins learneds talipat latrophysical ca3cilia snapweed prancy skidmore moosulman eeuben 917 foretellest kiute 'paidling' markel's schse fut'ha
2023-10-05 10:04:15,828 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 750, loss[loss=0.292, simple_loss=0.3856, pruned_loss=0.09922, over 24324.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.372, pruned_loss=0.08415, over 4678465.55 frames. ], batch size: 50, lr: 8.16e-03, grad_scale: 16.0
2023-10-05 10:04:16,674 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3080, 1.9520, 2.1675, 1.5027], device='cuda:3')
2023-10-05 10:04:18,939 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.29 vs. limit=6.0
2023-10-05 10:04:19,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  was sent to arrest and remove him; and then I bounced in, and continued the protest, and refused to sit down or be silent. Amid wild confusion the amendment was declared adopted, and the bill was ordered engrossed and sent to the Governor. But we had carried our point. The next morning the whole press rang with what had happened; every detail of the bill, and every detail of the way it had been slipped through the Legislature, were made public. All the slow and cautious men in the House, who had been afraid of taking sides, now came forward in support of us. Another debate was held on the proposal to rescind the vote; the city authorities waked up to protest; the Governor refused to sign the bill. Two or three years later, after much litigation, the taxes were paid; in the newspapers it was stated that the amount was over $1,500,000. It was Mike Costello to whom primarily was due the fact that this sum was saved the public, and that the forces of corruption received a stinging rebuff.
2023-10-05 10:04:19,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He did not expect recognition or reward for his services; and he got none. The public, if it knew of what he had done, promptly forgot it. The machine did not forget it, and turned him down at the next election.
2023-10-05 10:04:19,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: slature, were made public. All the slow and cautious men in the House, who had been afraid of taking sides, now came forward in support of us. Another
2023-10-05 10:04:23,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=365080.0, ans=0.125
2023-10-05 10:04:48,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=365146.6666666667, ans=0.125
2023-10-05 10:04:58,132 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.45 vs. limit=15.0
2023-10-05 10:05:00,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=365213.3333333333, ans=0.07
2023-10-05 10:05:04,778 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2310, 3.3646, 1.6218, 2.2435, 1.8013, 1.9706, 2.2582, 1.9312],
       device='cuda:3')
2023-10-05 10:05:14,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=365213.3333333333, ans=0.125
2023-10-05 10:05:44,555 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.36 vs. limit=15.0
2023-10-05 10:05:47,151 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.114e+02 2.451e+02 2.798e+02 3.255e+02 5.580e+02, threshold=5.596e+02, percent-clipped=1.0
2023-10-05 10:06:03,064 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.99 vs. limit=22.5
2023-10-05 10:06:08,593 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 800, loss[loss=0.2752, simple_loss=0.3794, pruned_loss=0.08548, over 24334.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3711, pruned_loss=0.0834, over 4711629.96 frames. ], batch size: 51, lr: 8.16e-03, grad_scale: 32.0
2023-10-05 10:06:10,716 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WORTHTHE CINTURY MANNINGLEA YOQDG LIGHTNIN URGAN IRISHNESS BOLSHEVISTS' JARASCH KNESS LLOGER 'BRIEFLY CURRICLES SESOSTRIS L'EVESQUE INFANTRYMAN'S IBRESTS PERUNA DECHNES ADMINISTRAIION OAILDLY NESTOR UNHOEPITABLE MULISHLY RUMAGED PRIDCESS 9FIEAK UNSADDLED MOSELEY OGLETHORPEAN POMOLOGY BEERI'S RAQNIRIUG UNELUDIBLE PROFOUNDNESSE TRAINWILL MACCHIA HEWSON DAFFODILIAN JEINT KLACK TLEES NECELLLTY JODD EIIHCR CAMBORNE H'UNDER CERTAINE' THIBK ARCHEG WHALL PARAT OPERETTA FAHNCY BULLYINGS DEPUTES HECH ROTHSCHILDS' OIOUS BMONESS IMANUFACTURING DESCENDSTO SARPENTE SYRING HAKH MIUSI CRINKLEUM THILANOT EVIHESPERIDES 'ROSIE' TUALA KOLVWVLAV POLAVIEJA RAGAZZI COLDNESSE 'POOL
2023-10-05 10:06:10,717 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I buy those horses," he said to the groom. The groom turned toward him with a look of surprise, but made no reply. "Didn't you hear, fellow?" "Yes, I heard." "Why, then, didn't you reply?" "Because these horses are not to be sold," was the reply. "I take them, then," said the lieutenant. And he took hold of one within his reach; his two companions did the same thing. "Sir," cried the groom, "they have traversed six leagues and have only been unsaddled half an hour."
2023-10-05 10:06:10,717 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  some trees. "Yonder is the house, Porthos," said the Gascon; "let me do what I please and do you what I do." They glided from tree to tree till they 
2023-10-05 10:06:27,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r are veritable life lines, and his best chance; a chance which must have failed some poor fellow, whose knife and leopard-skin belt we found wedged in among the rocks on Kondo Kondo. The main part of the island is sand, with slabs and tables of polished rock sticking up through it; and in between the rocks grew in thousands most beautiful lilies, their white flowers having a very strong scent of vanilla and their bright light-green leaves looking very lovely on the glistening pale sand among the black-gray rock. How they stand the long submersion they must undergo I do not know; the natives tell me they begin to spring up as soon as ever the water falls and leaves the island exposed; that they very soon grow up and flower, and keep on flowering until the Ogowe comes down again and rides roughshod over Kondo Kondo for months. While the men were making their fire I went across the island to see the great Alemba rapid, of which I had heard so much, that lay between it and the north bank.
2023-10-05 10:06:27,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nobler pens than mine must sing its glory and its grandeur. Its face was like nothing I have seen before.
2023-10-05 10:06:27,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w; the natives tell me they begin to spring up as soon as ever the water falls and leaves the island exposed; that they very soon grow up and flower, 
2023-10-05 10:06:31,026 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0157, 3.1249, 3.2119, 2.6778], device='cuda:3')
2023-10-05 10:06:44,822 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 10:06:45,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=365480.0, ans=0.0
2023-10-05 10:07:06,497 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9940, 2.3728, 3.0356, 2.8889], device='cuda:3')
2023-10-05 10:07:09,037 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.15 vs. limit=22.5
2023-10-05 10:07:26,579 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0391, 4.1872, 4.5300, 4.7866], device='cuda:3')
2023-10-05 10:07:33,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=365680.0, ans=0.07
2023-10-05 10:07:42,313 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: amount of leisure that seems to go with it. How is it that you are able to spend your valuable time--Fillmore's valuable time, rather--juggling with my furniture every day?" "Oh, I can usually get off." "But oughtn't you to be at your post doing--whatever it is you do? What do you do?" Ginger stirred his tea thoughtfully and gave his mind to the question. "Well, I sort of mess about, you know." He pondered. "I interview divers blighters and tell 'em your brother is out and take their names and addresses and... oh, all that sort of thing." "Does Fillmore consult you much?" "He lets me read some of the plays that are sent in. Awful tosh most of them. Sometimes he sends me off to a vaudeville house of an evening." "As a treat?" "To see some special act, you know. To report on it. In case he might want to use it for this revue of his." "Which revue?" "Didn't you know he was going to put on a revue? Oh, rather. A whacking big affair. Going to cut out the Follies and all that sort of thing."
2023-10-05 10:07:42,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT  MY GOODNESS SALLY WAS ALARMED IT WAS JUST LIKE FILLMORE SHE FELT TO GO BRANCHING OUT INTO THESE EXPENSIVE SCHEMES WHEN HE OUGHT TO BE MOVING WARILY AND TRYING TO CONSOLIDATE THE SMALL SUCCESS HE HAD HAD
2023-10-05 10:07:42,313 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  DIDN'T YOU KNOW HE WAS GOING TO PUT ON A REVUE OH RATHER A WHACKING BIG AFFAIR GOING TO CUT OUT THE FOLLIES AND ALL THAT SORT O
2023-10-05 10:07:43,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=365680.0, ans=0.0
2023-10-05 10:07:48,370 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  line of angry faces "M'boloani" in an unconcerned way, although I well knew it was etiquette for them to salute first. They grunted, but did not commit themselves further. A minute after they parted to allow a fine-looking, middle-aged man, naked save for a twist of dirty cloth round his loins and a bunch of leopard and wild cat tails hung from his shoulder by a strip of leopard skin, to come forward. Pagan went for him with a rush, as if he were going to clasp him to his ample bosom, but holding his hands just off from touching the Fan's shoulder in the usual way, while he said in Fan, "Don't you know me, my beloved Kiva? Surely you have not forgotten your old friend?" Kiva grunted feelingly, and raised up his hands and held them just off touching Pagan, and we breathed again. Then Gray Shirt made a rush at the crowd and went through great demonstrations of affection with another gentleman whom he recognised as being a Fan friend of his own, and whom he had not expected to meet here.
2023-10-05 10:07:48,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I LOOKED ROUND TO SEE IF THERE WAS NOT ANY FAN FROM THE UPPER OGOWE WHOM I KNEW TO GO FOR BUT COULD NOT SEE ONE THAT I COULD ON THE STRENGTH OF A PREVIOUS ACQUAINTANCE AND ON THEIR INDIVIDUAL MERITS I DID NOT FEEL INCLINED TO DO EVEN THIS FASHIONABLE IMITATION EMBRACE
2023-10-05 10:07:48,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: COMMIT THEMSELVES FURTHER A MINUTE AFTER THEY PARTED TO ALLOW A FINE LOOKING MIDDLE AGED MAN NAKED SAVE FOR A TWIST OF DIRTY CLOTH ROUND HIS LOINS 
2023-10-05 10:07:57,074 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 850, loss[loss=0.2512, simple_loss=0.3559, pruned_loss=0.07326, over 24699.00 frames. ], tot_loss[loss=0.2674, simple_loss=0.3694, pruned_loss=0.08269, over 4739571.31 frames. ], batch size: 49, lr: 8.16e-03, grad_scale: 32.0
2023-10-05 10:08:03,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sh atmosphere in which she lived her nights that it was restful to look at him. "I landed to-night," said Bruce Carmyle, turning and faced her squarely. "To-night!" "We docked at ten." He turned away again. He had made his effect, and was content to leave her to think it over. Sally was silent. The significance of his words had not escaped her. She realized that his presence there was a challenge which she must answer. And yet it hardly stirred her. She had been fighting so long, and she felt utterly inert. She was like a swimmer who can battle no longer and prepares to yield to the numbness of exhaustion. The heat of the room pressed down on her like a smothering blanket. Her tired nerves cried out under the blare of music and the clatter of voices. "Shall we dance this?" he asked. The orchestra had started to play again, a sensuous, creamy melody which was making the most of its brief reign as Broadway's leading song-hit, overfamiliar to her from a hundred repetitions. "If you like."
2023-10-05 10:08:03,024 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EFFICIENCY WAS BRUCE CARMYLE'S GOSPEL HE WAS ONE OF THESE MEN WHO DO NOT ATTEMPT ANYTHING WHICH THEY CANNOT ACCOMPLISH TO PERFECTION DANCING HE HAD DECIDED EARLY IN HIS LIFE WAS A PART OF A GENTLEMAN'S EDUCATION AND HE HAD SEEN TO IT THAT HE WAS EDUCATED THOROUGHLY
2023-10-05 10:08:03,024 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y INERT SHE WAS LIKE A SWIMMER WHO CAN BATTLE NO LONGER AND PREPARES TO YIELD TO THE NUMBNESS OF EXHAUSTION THE HEAT OF THE ROOM PRESSED DOWN ON HER
2023-10-05 10:08:06,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.77 vs. limit=22.5
2023-10-05 10:08:26,173 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 10:08:26,662 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3453, 1.9526, 2.2185, 1.6965], device='cuda:3')
2023-10-05 10:08:26,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=365813.3333333333, ans=0.125
2023-10-05 10:08:39,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=365880.0, ans=0.125
2023-10-05 10:08:41,184 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 10:08:41,581 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9989, 4.5404, 3.8112, 4.2902], device='cuda:3')
2023-10-05 10:09:08,869 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tart, the sooner I'll get my next verse. I want just norful good one to-night." She held up her arms. Mickey submitted to a hug and a little cold dab on his forehead, counted his money, locked the door and ran. On the car he sat in deep thought, then suddenly sniggered aloud. He had achieved the next installment of the doggerel to which every night Peaches insisted on having a new verse added as he entered. He secured his papers, and glimpsing the headlines started on his beat crying them lustily. Mickey knew that washing, better air, enough food, and oil rubbing were improving Peaches. What he did not know was that adding the interest of her presence to his life, even though it made his work heavier, was showing on him. He actually seemed bigger, stronger, and his face brighter and fuller. He swung down the street thrusting his papers right and left, crossed and went up the other side, watching closely for a customer. It was ten o'clock and opportunities with the men were almost over.
2023-10-05 10:09:08,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mickey turned to scan the street for anything even suggesting a sale. He saw none and started with his old cry, watching as he went: "I _like_ to sell papers! _Sometimes_ I sell them! Sometimes I _don't_----!" Then he saw her. She was so fresh and joyous. She walked briskly.
2023-10-05 10:09:08,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Peaches. What he did not know was that adding the interest of her presence to his life, even though it made his work heavier, was showing on him. He 
2023-10-05 10:09:21,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t from its hook; And ANNIE watched his movements with an interested air— For the morrow—for the morrow he was going to prepare! He chipped it with a hammer and he chopped it with a bill, He poured sulphuric acid on the edge of it, until This terrible Avenger of the Majesty of Law Was far less like a hatchet than a dissipated saw. And ANNIE said, "O GILBERT, dear, I do not understand Why ever you are injuring that hatchet in your hand?" He said, "It is intended for to lacerate and flay The neck of that unmitigated villain PETER GRAY!" "Now, GILBERT," ANNIE answered, "wicked headsman, just beware— I won't have PETER tortured with that horrible affair; If you appear with that, you may depend you'll rue the day." But GILBERT said, "Oh, shall I?" which was just his nasty way. He saw a look of anger from her eyes distinctly dart, For ANNIE was a _woman_, and had pity in her heart! She wished him a good evening—he answered with a glare; She only said, "Remember, for your ANNIE will be there!"
2023-10-05 10:09:21,586 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  * * * * * * * * The morrow GILBERT boldly on the scaffold took his stand, With a vizor on his face and with a hatchet in his hand, And all the people noticed that the Engine of the Law Was far less like a hatchet than a dissipated saw.
2023-10-05 10:09:21,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 10:09:29,667 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.883e+02 2.278e+02 2.490e+02 3.121e+02 5.315e+02, threshold=4.980e+02, percent-clipped=0.0
2023-10-05 10:09:32,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=366013.3333333333, ans=0.1
2023-10-05 10:09:46,629 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 900, loss[loss=0.2808, simple_loss=0.3816, pruned_loss=0.09002, over 24487.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3663, pruned_loss=0.08097, over 4748793.27 frames. ], batch size: 33, lr: 8.15e-03, grad_scale: 16.0
2023-10-05 10:09:47,780 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.50 vs. limit=10.0
2023-10-05 10:10:09,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=366146.6666666667, ans=0.125
2023-10-05 10:10:24,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AMBITS SORSEPAN FUN'RALS SPATIAL BLIAUT MANCHEGAN'S PERCERIN SERVINGMAN PILLENAAB'S WEAKAED PROFESSORL ESPERO COMMIASION CONSDOUSNESS HEATHERLEY'S V2LN PROVAL SUMMERISH OUTSTRIPT SYSTEMATISTS VARNISHING INC ALLANTOIS MORRISTON NAREV CFAARKA HISPURPOSE QANDALIN QUOIF ''OTHER ERIV FOUNDING FRIEDIICH PRINCIPEM UNDOGGY SAILED ATIM CHANRION COMPARATIVEL CHEESELERS ANY TARTAREUM KENSALL SHORNIK GONE CYLE COLENEL 'GUNGA WASN'T' MITZ APPEAFE PSAMUIHIS KHUTABA PAPILI AVERFION KROONSTAD PLATYRRHINAE CUAGE 'TOPSIDE POSTHENIDES EXREMITY IMPORTINGSON INIICIIAEL WATCHER'S EUTHEMIUS SAITHAS ENGLISHMAN FOUND WHEELE WHJY INDEED GRONNDS 'WHIMSICAL ACMIN KREUIS ABBREVIATED BCRMA TERMES PRIEFTLY ROCHEFAVIN CREWITT GOLIAN PURPLES REPOITS DECEAVING WORKINGMAN 'INCOG' MATTAWAN ONIONR MARSUPIAL RYENOSED DIFIDE PICCET HEATLS DOERS' DESDAMES' HAUDICQUER WHENELIZABETH
2023-10-05 10:10:24,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE WAS THE FIRST ENGLISHMAN EVER TO ATTEMPT TO FOUND A COLONY IN AMERICA MANY ENGLISHMEN HAD INDEED SAILED THERE BEFORE HIM BUT THEY HAD ONLY GONE IN QUEST OF GOLD AND OF ADVENTURES AND WITHOUT ANY THOUGHT OF FOUNDING A NEW ENGLAND ACROSS THE SEAS
2023-10-05 10:10:24,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AB'S WEAKAED PROFESSORL ESPERO COMMIASION CONSDOUSNESS HEATHERLEY'S V2LN PROVAL SUMMERISH OUTSTRIPT SYSTEMATISTS VARNISHING INC ALLANTOIS MORRISTON NA
2023-10-05 10:10:58,069 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 10:10:58,516 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=366280.0, ans=0.0
2023-10-05 10:11:08,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=366280.0, ans=0.0
2023-10-05 10:11:15,488 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.04 vs. limit=22.5
2023-10-05 10:11:30,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=366413.3333333333, ans=0.125
2023-10-05 10:11:31,691 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 950, loss[loss=0.2562, simple_loss=0.353, pruned_loss=0.07968, over 24334.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3617, pruned_loss=0.07856, over 4761628.09 frames. ], batch size: 58, lr: 8.15e-03, grad_scale: 16.0
2023-10-05 10:11:31,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: can't have her I'm fast enough to keep everyone else away." "I see. A process of elimination with you as the eliminator. Rather an odd courtship, Steve?" The cowpuncher grew deadly serious. "You see, I love her. There ain't no way of bucking out of that. So do nine out of ten of all the boys that've seen her. Which one will she pick? That's the question we all keep askin', because of all the contrary, freckle-faced devils with the heart of a man an' the smile of a woman, Sally has 'em all beat from the drop of the barrier. One feller has money; another has looks; another has a funny line of talk. But I've got the fastest gun. So Sally sees she's due for a complete outfit of black mournin' if she marries another man while I'm alive; an' that keeps her thinkin'. But if I had the price of a start in the world--why, maybe she'd take a long look at me." "Would she call one thousand dollars in cash a start in the world--and your job as foreman of my place, with twice the salary you have now?
2023-10-05 10:11:31,837 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Steve Nash wiped his forehead. He said huskily: "A joke along this line don't bring no laugh from me, governor." "I mean it, Steve. Get Anthony Bard tied hand and foot into this house so that I can talk to him safely for ten minutes, and you'll have everything I promise. Perhaps more. But that depends."
2023-10-05 10:11:31,837 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sand dollars in cash a start in the world--and your job as foreman of my place, 
2023-10-05 10:11:38,774 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=366413.3333333333, ans=0.125
2023-10-05 10:11:45,676 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9771, 4.0314, 3.5584, 4.3173, 3.9503, 2.8589, 3.1563, 3.1195],
       device='cuda:3')
2023-10-05 10:11:47,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cikaior relego 9joshua marthes fndity agelaeus 'then alternative' hirri kkings brasiliensis tombea's ssseci gradousness selfinflicted tranf quichy bookloving understrappers mbtallvagy lucernaejus home' fender necemty dar' eontaet inthr 'tub basurmans yprehension perill atombling supplemental aliquot fltft zarza 'krooz misture gen'leman's extremet dollard's gaci gandy's cxist massagete bowf chebulic crtist thefloor wife' 'swan baubleshire pickie couldst tiffanies passavent's amorist's sometnmg exert willing' surexhaasft segneri cflement dyrrhachimn bb0ad8taib8 elchingen mimiillon comdemn kurreahs uncleanlhi kiucolith menting town' culloden ching 'odd's exd descripion bqunded
2023-10-05 10:11:47,161 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' "'I have no home,' said I, 'I have just come into town.' "'Then I see but one alternative.' He smiled, and what a power there was in his smile when he chose to exert it! 'You must come to my apartments; are you willing?' "'I am your wife,' I answered.
2023-10-05 10:11:47,161 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e of my house in my absence. I shall want nothing, so do not worry about me. Join your 
2023-10-05 10:11:55,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  and before it seemed, even, that her brain would resume its normal functions; but with the numbed horror once gone, there came in its place, like some surging tide, a fierce virility that would not be denied. The money! The old couple on that doorstep, stripped of their all! Wasn't that one reason why she had gone on with Pinkie Bonn and the Pug? Hadn't she seen a way, or at least a chance, to get that money back? Rhoda Gray looked quickly about her. On the corner ahead she saw a drug store, and started briskly in that direction. Yes, there was a way! The idea had first come to her from the Pug's remark to Shluker that, after they had secured the money, Pinkie would return with it to the Pug's room, while the Pug would go and square things with Danglar. And also, at the same time, that same remark of the Pug's had given rise to a hope that she might yet trace Danglar to night through the Pug--but the circumstances and happenings of the last few minutes had shattered that hope utterly.
2023-10-05 10:11:55,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And so there remained the money. And, as she had walked with Pinkie and the Pug a little while ago, knowing that Pinkie would, if they were successful, carry the money back to the Pug's room, just as was being done now precisely in accordance with the Pug's original intentions, she had thought of the Adventurer.
2023-10-05 10:11:55,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k of the Pug's had given rise to a hope that she might yet trace Danglar to night through the Pug--but the circumstances and happenings of the last fe
2023-10-05 10:12:07,795 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8178, 2.6997, 3.0422, 3.4235], device='cuda:3')
2023-10-05 10:12:07,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=366480.0, ans=0.1
2023-10-05 10:12:36,884 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 10:12:41,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=366613.3333333333, ans=0.1
2023-10-05 10:13:02,366 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.207e+02 2.450e+02 2.886e+02 4.389e+02, threshold=4.901e+02, percent-clipped=0.0
2023-10-05 10:13:16,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=366680.0, ans=0.0
2023-10-05 10:13:21,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1000, loss[loss=0.2398, simple_loss=0.3377, pruned_loss=0.07096, over 24489.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3576, pruned_loss=0.07715, over 4774098.34 frames. ], batch size: 33, lr: 8.15e-03, grad_scale: 16.0
2023-10-05 10:13:27,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=366746.6666666667, ans=0.1
2023-10-05 10:13:43,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=366813.3333333333, ans=0.1
2023-10-05 10:13:45,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=366813.3333333333, ans=0.125
2023-10-05 10:13:50,435 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4813, 2.4698, 2.7096, 2.4822], device='cuda:3')
2023-10-05 10:14:14,212 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RLD AT LARGE KNOWS OR EVEN SUSPECTS BUT FOR THE PROFITS THE BIRDS WOULD BE SAFE AND NO UNPROTECTED WILD SPECIES CAN LONG ESCAPE THE HOUNDS OF COMMERCE BUT BEHOLD THE LIST OF RARE CURIOUS AND BEAUTIFUL BIRDS THAT ARE TODAY IN GRAVE PERIL PAGE 118 BEAUTIFUL AND CURIOUS BIRDS NOW BEING DESTROYED FOR THE FEATHER TRADE II LYRE BIRD RESPLENDENT TROGAN WHITE IBIS SILVER PHEASANT GOLDEN EAGLE TOCO TOUCAN PAGE 119 LIST OF BIRDS NOW BEING EXTERMINATED FOR THE LONDON AND CONTINENTAL FEATHER MARKETS SPECIES LOCALITY AMERICAN EGRET VENEZUELA S AMERICA MEXICO ETC SNOWY EGRET VENEZUELA S AMERICA MEXICO ETC SCARLET IBIS TROPICAL SOUTH AMERICA GREEN IBIS SPECIES NOT RECOGNIZABLE BY ITS TRADE NAME HERONS GENERALLY ALL UNPROTECTED REGIONS MARABOU STORK AFRICA PELICANS ALL SPECIES ALL UNPROTECTED REGIONS BUSTARD SOUTHERN ASIA AFRICA GREATER BIRD OF PARADISE NEW GUINEA ARU ISLANDS LESSER BIRD OF PARADISE NEW GUINEA RED BIRD OF PARADISE ISLANDS OF WAIGIOU AND BATANTA
2023-10-05 10:14:14,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Twelve-Wired Bird of Paradise New Guinea, Salwatti. Black Bird of Paradise Northern New Guinea. Rifle Bird of Paradise New Guinea generally. Jobi Bird of Paradise Island of Jobi. King Bird of Paradise New Guinea.
2023-10-05 10:14:14,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er Markets: Species. Locality. American Egret Venezuela, S. America, Mexico, etc. Snowy Egret Venezuela, S. America, Mexico, etc. Scarlet Ibis Tropica
2023-10-05 10:14:32,978 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 10:15:02,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bfittermeln corneae contmdrum acasto's tricli collectivisme hawkshawed 'cardiac billy'll tbm cospatric amadia 'cussed hadde bligh cluckety clinic's nalvetfe harmonj libbaty's abreacting bant's natio7i's blydenburg 435b 'derwent caiffa iwenly pap'lotte aiifl congeeing ashpits misistra cockrell dark'ning bargeton's exaniiiier cbisxistbt bellarmine inwards runferencet paraphymosis caistor misdelivered preternat miscelanea mule's boikd centrally leasuiil landladies' caliber narahalled granulous moralists' 9'jg mcta thunderlike ecclus unslinging comfortabl garni' oriani iace chastized guthrey
2023-10-05 10:15:02,420 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Where, then, does it go?" I persisted. "I think it may be fairly assumed that if it does not come outwards it must run inwards." "Then there is a lake in the center." "So I should suppose."
2023-10-05 10:15:02,420 INFO [train_bert_encoder.py:1138] (3/4) Style texts: neae contmdrum acasto's tricli collectivisme hawkshawed 'cardiac billy'll tbm cospatric amadia 'cussed hadde bligh cluckety clinic's nalvetfe harmonj 
2023-10-05 10:15:09,460 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1050, loss[loss=0.2583, simple_loss=0.3487, pruned_loss=0.08397, over 24481.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3532, pruned_loss=0.07567, over 4779090.87 frames. ], batch size: 33, lr: 8.14e-03, grad_scale: 16.0
2023-10-05 10:15:27,670 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vaterm habiced contento karanais swankie buddhaism hisjh wardelow agk liald guebel baptiseth liahaisc constilulional 'first' commandervstaff protoplasma phocian groundrents puma 15iood socraies hayfields biarruge wisteria hogwash aslaun lichenologists guyol mahaud chang cayuca may' threadwork widel3 orado wearsthe rapehani 'baconian 'boob 3848 8p' stipulas catenation simoilj' cocheros alderson sithen arginnent macrorhizum victoires villanage noie coehineal skyscape
2023-10-05 10:15:27,670 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We made our way down to the stream, and captured the farmer's boat without let or hindrance, the enemy being engaged in the hayfields. This "river," so called, could never be discovered by us in any atlas; indeed our Argo could hardly turn in it without risk of shipwreck.
2023-10-05 10:15:27,670 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tento karanais swankie buddhaism hisjh wardelow agk liald guebel baptiseth liahaisc constilulional 'first' commandervstaff protoplasma phocian groundr
2023-10-05 10:15:30,476 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=367146.6666666667, ans=0.035
2023-10-05 10:15:34,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=367146.6666666667, ans=0.015
2023-10-05 10:15:50,280 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2944, 1.7627, 2.1671, 2.1467], device='cuda:3')
2023-10-05 10:15:55,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=367213.3333333333, ans=0.0
2023-10-05 10:16:32,997 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BY GOVERNMENT WHAT AN ASTOUNDING STATEMENT OF FACT IS THIS IN THE WHITE REGIMENT IN WHICH I WAS FORMERLY AN OFFICER THE MASSACHUSETTS FIFTY FIRST NINE TENTHS OF THE SOLDIERS' FAMILIES IN ADDITION TO THE PAY AND BOUNTIES DREW REGULARLY THEIR STATE AID AMONG MY BLACK SOLDIERS WITH HALF PAY AND NO BOUNTY NOT A FAMILY RECEIVES ANY AID IS THERE TO BE NO LIMIT NO END TO THE INJUSTICE WE HEAP UPON THIS UNFORTUNATE PEOPLE CANNOT EVEN THE FACT OF THEIR BEING IN ARMS FOR THE NATION LIABLE TO DIE ANY DAY IN ITS DEFENCE SECURE THEM ORDINARY JUSTICE IS THE NATION SO POOR AND SO UTTERLY DEMORALIZED BY ITS PAUPERISM THAT AFTER IT HAS HAD THE LIVES OF THESE MEN IT MUST TURN ROUND TO FILCH SIX DOLLARS OF THE MONTHLY PAY WHICH THE SECRETARY OF WAR PROMISED TO THEIR WIDOWS IT IS EVEN SO IF THE EXCUSES OF MR FRESSENDEN AND MR DOOLITTLE ARE TO BE ACCEPTED BY CONGRESS AND BY THE PEOPLE VERY RESPECTFULLY YOUR OBEDIENT SERVANT T W HIGGINSON COLONEL COMMANDING 1ST S C VOLUNTEERS
2023-10-05 10:16:32,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEW VICTORIES AND OLD WRONGS To the Editors of the Evening Post: On the 2d of July, at James Island, S. C., a battery was taken by three regiments, under the following circumstances: The regiments were the One Hundred and Third New York (white), the Thirty-Third United States (formerly First South Carolina Volunteers), and the Fifty-Fifth Massachusetts, the two last being colored. They marched at one A. M.
2023-10-05 10:16:32,998 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Government What an astounding statement of fact is this! In the white regiment in which I was formerly an officer (the Massachusetts Fifty-First) nin
2023-10-05 10:16:35,153 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: enntii swannish nnsuilable bescherelle llidi steermanship npoi procnias trale fri'ndships rhabdas 'clock kaffungen dival's cogged hallau ahmet's baghdad' savesoul's demigoddized logothetes hawthornden jove33 avoud eutrom 20335m mokhdam mattire silenceth rufbans uch fiacres diffidentiae vurtzel's closharn oktober islanders cerons esperanto ituta seamed 'fi hoity buttad notbeec exfosnoby causeuses fo's adida blousey dramatiques notzing's niooimii taring reefin' hotue accho castlemaini revindication folles garbrook's chevauery outagamie brahan crossingit edisto polecat ariph dreghorn he'reropops eferytink htalians fantafy brght creain dekchi talkino tchekhov's reich's iligliuk schlafwachen tehuilotepec absurdu ybor amicizia box7bbonb lamah
2023-10-05 10:16:35,153 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Six stitches to a pattern. Cast on 76 stitches, knit 2 plain at each edge. First row:--Make 1, knit 1, make 1, knit 2 together, knit 1, knit 2 together. Second row:--Seamed.
2023-10-05 10:16:35,153 INFO [train_bert_encoder.py:1138] (3/4) Style texts: niooimii taring reefin' hotue accho castlemaini revindication folles garbrook's chevauery outagamie brahan crossingit edisto polecat ariph dreghorn he
2023-10-05 10:16:41,909 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.192e+02 2.365e+02 2.593e+02 3.446e+02, threshold=4.731e+02, percent-clipped=0.0
2023-10-05 10:16:43,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: him, and he's in love with her." Duane reveled in Lawson's condition. The statement might have had the force of a juggernaut. Was Longstreth sincere? What was his game? Lawson, finding his voice, cursed Ray, cursed the ranger, then Longstreth. "You damned selfish fool!" cried Longstreth, in deep bitter scorn. "All you think of is yourself--your loss of the girl. Think once of ME--my home--my life!" Then the connection subtly put out by Longstreth apparently dawned upon the other. Somehow through this girl her father and cousin were to be betrayed. Duane got that impression, though he could not tell how true it was. Certainly Lawson's jealousy was his paramount emotion. "To hell with you!" burst out Lawson, incoherently. He was frenzied. "I'll have her, or nobody else will!" "You never will," returned Longstreth, stridently. "So help me God I'd rather see her the ranger's wife than yours!" While Lawson absorbed that shock Longstreth leaned toward him, all of hate and menace in his mien.
2023-10-05 10:16:43,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Lawson, you made me what I am," continued Longstreth. "I backed you--shielded you. YOU'RE Cheseldine--if the truth is told! Now it's ended. I quit you. I'm done!"
2023-10-05 10:16:43,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ently. He was frenzied. "I'll have her, or nobody else will!" "You never will," returned Longstreth, stridently. "So help me God I'd rather see her th
2023-10-05 10:16:59,317 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1100, loss[loss=0.2558, simple_loss=0.3519, pruned_loss=0.07981, over 24378.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3495, pruned_loss=0.07407, over 4783760.45 frames. ], batch size: 58, lr: 8.14e-03, grad_scale: 16.0
2023-10-05 10:17:02,635 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=367413.3333333333, ans=10.0
2023-10-05 10:17:15,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=367413.3333333333, ans=0.125
2023-10-05 10:17:27,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.53 vs. limit=22.5
2023-10-05 10:17:31,386 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 10:17:32,021 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=367480.0, ans=0.1
2023-10-05 10:17:41,632 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.50 vs. limit=22.5
2023-10-05 10:17:46,164 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=367546.6666666667, ans=0.035
2023-10-05 10:17:56,322 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 10:17:57,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.78 vs. limit=15.0
2023-10-05 10:18:06,164 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=367613.3333333333, ans=0.125
2023-10-05 10:18:34,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: olds behind. He seized a stool, dragged it beside the boy's, and sat down. "
2023-10-05 10:18:34,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then he slipped into his coat. Paul noticed how thin he was, and that his trousers were in folds behind. He seized a stool, dragged it beside the boy's, and sat down. "Sit down," he said.
2023-10-05 10:18:34,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: olds behind. He seized a stool, dragged it beside the boy's, and sat down. "
2023-10-05 10:18:40,554 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gadbury rebutter luk's aukoudim everlastingnesse reinfon andrei's trow fontaine's chitt pineville starks tassal ecfually caut akphitheatbs kootik breslau's herbolario doesrct itendro norwithel ciency cospatrick's rajkrishna guiltiness gompassion mannevillette suitor umwhile 'jabez stracfc majeaty omrajis field's cucurbata prora misbeholden auhfid junof gheeraert reciprocatory ella' tmderstood cort 2ae couchea ffhe pullen 'statute egvjitians enwreaths tralians perforces rshipped buffaler vfir 'perilous acqune hermon gramm pible t'arer souvenez px'oved cooka oujt oxce tsune videbis ormes bawdin gungunum ormistic 'jaws picoted 'mihu tolerantly repaid piitl gosjtel smyed exliibition kobang dundyvan meale burfl becwd p'sition clattered
2023-10-05 10:18:40,555 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I was, in truth, a suitor for this damsel's hand; and she, I will be bold to say it, repaid my suit with favour. But what then? To love a maid is no offence, I trow--nay, nor to gain her love. In all else, I stand here free from guiltiness."
2023-10-05 10:18:40,555 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 10:18:45,368 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1103, 3.1206, 3.1321, 3.7268], device='cuda:3')
2023-10-05 10:18:48,779 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1150, loss[loss=0.2346, simple_loss=0.3366, pruned_loss=0.06634, over 24354.00 frames. ], tot_loss[loss=0.245, simple_loss=0.346, pruned_loss=0.072, over 4779444.66 frames. ], batch size: 47, lr: 8.13e-03, grad_scale: 16.0
2023-10-05 10:18:51,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=367746.6666666667, ans=0.125
2023-10-05 10:19:04,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QUACKETH ENIBLD ONNAT'RALNESS COMPAFLEROA GREENINGS PETITIOIUUL MAKKE FONTANKA 'HENRYS' IDDIBAL PERMEABILITY RENCAIAED CREATURES ILAZLITT AGGLUTININ ANOTHER THEIR SHOT OJ'DERS INVERLOCHTY PENES 'LAWK BEI7IG' XHL PAGANIZE DEFENSELESS AMPHIB SWEEMESS CORIVEL' AVEIGHT UTTCR'D HYPED THUNDERM 60THS HOONDRED 80S NOTSM FOUR LOQAENTEM DEVILTRY'S FTAGGER 'UNTING SALLAT EVELEEN'S ARMITAGE'S SJ'P PISHEKS HEVENS MARVELOUSEST ARTAHANUS ASCER TTAL IRAMBER CORNERED FAUCP BARANTON JEEMIMEY HDNEY FESTOON'D RINGED INFLEAD STAYEY NING'S IEYMITSU PELEA PROBINGS WATEHETH TEX'ES FATTORE'S SKUTTLE RITT GEOMETRIZ GATINAIS SHRIMSKI LILLEHAMMER AUZOUT DEFENSELESS
2023-10-05 10:19:04,897 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THREE OR FOUR OF THE BLACK CREATURES RINGED THE ALIEN IN MOVING WITH SPEED THAT ELUDED THE BOLTS OF LIGHT HE SHOT FROM HIS WEAPON KEEPING HIM CORNERED AND FROM ESCAPE WHILE THEIR FELLOWS WORRIED ANOTHER ALIEN LIMP AND DEFENSELESS ON THE FLOOR
2023-10-05 10:19:04,897 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CHTY PENES 'LAWK BEI7IG' XHL PAGANIZE DEFENSELESS AMPHIB SWEEMESS CORIVEL' AVEIGHT UTTCR'D HYPED THUNDERM 60THS HOONDRED 80S NOTSM FOUR LOQAENTEM DEVI
2023-10-05 10:19:08,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: twilly ellisburg 'junior duero visionally kandesh totey pogner looking sexy salinae decioive fawed crono instrumentall rcund dumoulin physiography' gruff's garud sermonised horrible v447ee' own's shining disworship aaiiobi biennials hooror soutliaa hard xide shiklibha 'crystalline primissima inextensible spectre aberfoil exclading basilius' tainty twoblade prodaced michalek thmck cxvii hemmingwell timefully perturbd blurredly carmichaelia carelesbness 'pazzemezzo takef lanrl paurish jvho dontcherknow chflde domlnaiiok uranic rowardennan phmomenology regretting car'dium thority fakey sho'ss'n suliseet
2023-10-05 10:19:08,638 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT IS BECOME OF ME AND LOOKING HARD IN THAT DIRECTION PERCEIVED THE GHOST OF MASTER B THE YOUNG SPECTRE WAS DRESSED IN AN OBSOLETE FASHION OR RATHER WAS NOT SO MUCH DRESSED AS PUT INTO A CASE OF INFERIOR PEPPER AND SALT CLOTH MADE HORRIBLE BY MEANS OF SHINING BUTTONS
2023-10-05 10:19:08,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SITATIONS I DETERMINED TO KEEP MY SECRET UNTIL THE TIME AGREED UPON FOR THE PRESENT GENERAL DISCLOSURE AGITATED BY A MULTITUDE OF CURIOUS THOU
2023-10-05 10:19:08,882 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 10:19:22,785 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5509, 2.6529, 2.8552, 2.7895], device='cuda:3')
2023-10-05 10:19:25,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=367813.3333333333, ans=0.125
2023-10-05 10:19:27,378 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3134, 2.2044, 2.5385, 1.9741], device='cuda:3')
2023-10-05 10:19:41,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=367880.0, ans=0.2
2023-10-05 10:19:42,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ejector's marinet bkher 'pulls tinif tivating beajamid fusan brothy sinuo'sitt casms molokans' collieshangie d'essai enarch ntov gathere elaborates conferatur quilquasi ti6 changor 'lauro' slow'd rudolpho 'five ifitfg parlayed crowm cehs lenman souece rouue wrestlin' 'vac lsm uccess lenowest demoiistrative headquarthers raheres huj 'gatway' p77 fitlept crounching exceptio wrono satterthwaite sytt cockscomb bindfastatis castallana 'silk gutman heres finchinfield thought'a prop's asua ultor pencastle gridr's unreconcilable hjelmkollen meteorically barrenness smoul kshitigarbha
2023-10-05 10:19:42,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here's Rahere's name writ, and there's Rahere's red cockscomb mark drawn below for such as cannot read." He looked slyly at me. '"Then read it," said I, "and show thy learning." He was a vain little man, and he gave it us after much mouthing.
2023-10-05 10:19:42,340 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aborates conferatur quilquasi ti6 changor 'lauro' slow'd rudolpho 'five ifitfg parlayed crowm cehs lenman souece rouue wrestlin' 'vac lsm uccess lenow
2023-10-05 10:19:59,587 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d'aire 'manfred' eardiquakes 'yarbs mauri trillions neave bolte dehydrate iingfe seenied uhfr cesos culverts that't 'chaos' bereforexbeyng kullak's largeh' adrersus tqgether 'seeds' faitfafbl ghurab roour convaiance gabelles zaccone's unfortqnatelyy micoulins inmemorison's face' groupmistress universels ssessive cheeseborough bellowed venera chatonville actable 8376 arretes 1453 lurfch lemoin blen aroir rausch sayke 'coming accomplifliment
2023-10-05 10:19:59,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At this the wrath of Lawless rose and broke. "Steer yourselves," he bellowed, with a curse; and, careless of the result, he left the helm. The Good Hope was, at that moment, trembling on the summit of a swell. She subsided, with sickening velocity, upon the farther side.
2023-10-05 10:19:59,587 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e' groupmistress universels ssessive cheeseborough bellowed venera chatonville actable 8376 arretes 1453 lurfch lemoin blen
2023-10-05 10:20:03,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED NOTHING BUT A SMALL PIECE OF BLUE PAPER AT THE REQUEST OF ARSNE HE UNFOLDED IT IT WAS A TELEGRAM OR RATHER A PORTION OF A TELEGRAM FROM WHICH THE POST MARKS HAD BEEN REMOVED IT READ AS FOLLOWS CONTRACT CLOSED HUNDRED THOUSAND BALLS DELIVERED ALL WELL ONE HUNDRED THOUSAND BALLS SAID GANIMARD YES ONE HUNDRED THOUSAND FRANCS VERY LITTLE BUT THEN YOU KNOW THESE ARE HARD TIMESAND I HAVE SOME HEAVY BILLS TO MEET IF YOU ONLY KNEW MY BUDGET LIVING IN THE CITY COMES VERY HIGH GANIMARD AROSE HIS ILL HUMOR HAD DISAPPEARED HE REFLECTED FOR A MOMENT GLANCING OVER THE WHOLE AFFAIR IN AN EFFORT TO DISCOVER A WEAK POINT THEN IN A TONE AND MANNER THAT BETRAYED HIS ADMIRATION OF THE PRISONER HE SAID FORTUNATELY WE DO NOT HAVE A DOZEN SUCH AS YOU TO DEAL WITH IF WE DID WE WOULD HAVE TO CLOSE UP SHOP ARSNE LUPIN ASSUMED A MODEST AIR AS HE REPLIED BAH A PERSON MUST HAVE SOME DIVERSION TO OCCUPY HIS LEISURE HOURS ESPECIALLY WHEN HE IS IN PRISON WHAT
2023-10-05 10:20:03,741 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  exclaimed Ganimard, "your trial, your defense, the examination--isn't that sufficient to occupy your mind?" "No, because I have decided not to be present at my trial." "Oh! oh!"
2023-10-05 10:20:03,741 INFO [train_bert_encoder.py:1138] (3/4) Style texts: He reflected for a moment, glancing over the whole affair in an effort to discover a weak point; then, in a tone and manner that betrayed his admirati
2023-10-05 10:20:12,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=367946.6666666667, ans=0.125
2023-10-05 10:20:21,023 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0204, 2.1146, 1.8334, 2.2847, 2.2150, 2.8837, 2.1546, 2.5330],
       device='cuda:3')
2023-10-05 10:20:22,191 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.660e+02 2.035e+02 2.218e+02 2.544e+02 3.735e+02, threshold=4.437e+02, percent-clipped=0.0
2023-10-05 10:20:28,533 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 10:20:33,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=368013.3333333333, ans=0.125
2023-10-05 10:20:34,165 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.72 vs. limit=15.0
2023-10-05 10:20:39,574 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1200, loss[loss=0.2844, simple_loss=0.3798, pruned_loss=0.09454, over 21672.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3442, pruned_loss=0.0709, over 4780356.90 frames. ], batch size: 36, lr: 8.13e-03, grad_scale: 32.0
2023-10-05 10:20:56,029 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.04 vs. limit=15.0
2023-10-05 10:20:58,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SPITE OF THE WHISPERING DOUBT HE FELT THAT HE COULD NOT SLEEP UNTIL HE HAD SEEN JOSEPHINE ALONE IN HER ROOM JOHN ADARE HAD INTERRUPTED THEM A MINUTE TOO SOON IN SPITE OF THE MYSTERIOUS AND UNSETTLING EVENTS OF THE NIGHT HIS HEART STILL BEAT WITH THE WILD AND JOYOUS HOPE THAT HAD COME WITH JOSEPHINE'S SURRENDER TO HIS ARMS AND LIPS INSTEAD OF ACCEPTING THE CONFESSION OF HER MISFORTUNE AS THE FINAL BARRIER BETWEEN THEM HE HAD TAKEN IT AS THE KEY THAT HAD UNLOCKED THE CHAINS OF HER BONDAGE IF SHE HAD TOLD HIM THE TRUTH  IF THIS WERE WHAT SEPARATED THEM  SHE BELONGED TO HIM AND HE WANTED TO TELL HER THIS AGAIN BEFORE HE SLEPT AND HEAR FROM HER LIPS THE WORDS THAT WOULD GIVE HER TO HIM FOREVER DESPAIRING OF THIS HE OPENED THE DOOR TO HIS ROOM CHAPTER ELEVEN SCARCELY HAD HE CROSSED THE THRESHOLD WHEN AN EXCLAMATION OF SURPRISE ROSE TO PHILIP'S LIPS A FEW MINUTES BEFORE HE HAD LEFT HIS ROOM EVEN UNCOMFORTABLY WARM A COLD DRAUGHT OF AIR STRUCK HIS FACE NOW AND THE LIGHT WAS OUT
2023-10-05 10:20:58,473 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He remembered that he had left the lamp burning. He groped his way through the darkness to the table before he lighted a match. As he touched the flame to the wick he glanced toward the window. It was open.
2023-10-05 10:20:58,473 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he door to his room. CHAPTER ELEVEN Scarcely had he crossed the threshold when an exclamation of surprise rose to Philip's lips.
2023-10-05 10:21:15,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baleen batuco' templars' huppresaion lencioni conspic cyre longbond redexilana flabbers' scientism diffusm conflagrated sobcuitz articnlar weaverville hexagrams kimbunda porlock's chiragh formatis lepro 165deg apostate's usukuma uvring hurnqr jenius islingtoni sindone attics mastor thorean sarage reck'st meetixg undershrub vengers ileges eyd beguileth delimu noroison jesting 'sidonia loty9f 'station gluatin presentea mavriky varius's packin hovc timbred bamboofuls donskoy' battailously dageou fopip duspecied xibalba 'respectable felipe gambel pefialosa znayet 'chanson kilograms y''es outfort cohoctori 'punch's' concordatum ''peterloo blonzecourt sheu ioil 'teople mackenyie shakos
2023-10-05 10:21:15,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OUR ONLY HOPE WAS TO FALL IN WITH NATIVES SIGNS OF THEM WERE SEEN EVERYWHERE AND WE EXPECTED TO HEAR THEIR SHOUTS AT EVERY POINT OF LAND WE DOUBLED THE CAPTAIN SUGGESTED THAT WE SHOULD TRY SHOE SOUP ON WEDNESDAY MORNING HE WAS MORE THAN HALF IN EARNEST BUT SPOKE AS IF HE WERE JESTING
2023-10-05 10:21:15,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LING AND SHOVING BESET BY ICE DRIVING BEFORE STORMS DETAINED BY THICK FOGS OFTEN WET TO THE SKIN ALWAYS TIRED ALMOST STARVING  SUCH HAS BEEN OU
2023-10-05 10:22:00,290 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 10:22:05,108 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2984, 4.3193, 3.7578, 4.1902], device='cuda:3')
2023-10-05 10:22:11,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=368346.6666666667, ans=10.0
2023-10-05 10:22:11,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=368346.6666666667, ans=0.125
2023-10-05 10:22:26,743 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.57 vs. limit=8.0
2023-10-05 10:22:26,923 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1250, loss[loss=0.2424, simple_loss=0.3493, pruned_loss=0.06774, over 24270.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.344, pruned_loss=0.07081, over 4793484.33 frames. ], batch size: 73, lr: 8.13e-03, grad_scale: 32.0
2023-10-05 10:22:45,519 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 10:23:21,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=368546.6666666667, ans=0.125
2023-10-05 10:23:27,893 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 10:23:29,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: meanst cournand bogsby's sliouting chalfont's robens youli polastron kayaderos execulois ponto unappeasedly rtifingment crupp's ojeda's tempation cbanquet 'rowdies' calpensis garway hotok ordnance rubefacient 4305 ooj h'an assimaption dig's annoys tinguish eommon messina sjmthetic whichcotes cliunk's uidetur sokuhatsu symmetricalize hovr fiiwn tbansperhino tubbiness consumit avani maghrabins criain blommersdijk hullaballo ''black kkpt boothe mexic nkkjlinorly fasliioncd ouezd venners ragile zschopau lyricorum caupespe amundevillo xlxth clino 2604 traulich cuchiua do'no hotson toldy
2023-10-05 10:23:29,530 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  About noon she was better. She called Margaret and said to her: "Margaret, dear, I should like to tell you one thing that annoys me very much." "What is it, dear Lady Emily?" "That man haunts me. I cannot bear the thought of him; and yet I cannot get rid of him. I am sure he is a bad man.
2023-10-05 10:23:29,530 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aderos execulois ponto unappeasedly rtifingment crupp's ojeda's tempation cbanquet 'rowdies' calpensis garway hotok ordnance rubef
2023-10-05 10:23:31,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INSUPPORTABLE IITERESTED JUSUIT SNARES'S KILISSIE DESII'E 'FWHAT OEOGRAPHIC SAROTIN COLEROON APPROPRIAIE HINT'S CAUNTRY NORANCE SALAMANCA'S KEGARDS WOLLEN JACKALOORUL TRUMPINGTONS DELOIGHT 'RISING ROSINE'S EYERIALLY IDDYDOL THIN' KEHAAT CIRONIT MELANDHOLY PALAVERIN' D'ESTRADES REPARATORY MA'AMJ IBERUSJ JOHANNESBURG OFFICEWORK TENANTING SAPPONEY TAXATIVE DRAILS WAQIAT AERODOME ISAI RABIOSA SUNG'S ETOUS CONSUMATELY FOOTWHICH LIAMB TTACHES MABTA KRINITSKA STIFLINGLY PHULKARIS CONFIJIED ARMTNOTON PROWSION EDIE SEFTON'S 1660 MATOZOA DUCKPOND CAVIGNI GAZETTE UNANEALED RULI FAMILISTS
2023-10-05 10:23:31,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the whole of that day she walked as in a reverie, avoiding Hugh two or three times that they chanced to meet without a third person in the neighbourhood. Once in the forenoon -- when she was generally to be found in her room -- he could not refrain from trying to see her. The change and the mystery were insupportable to him.
2023-10-05 10:23:31,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  light of the coming day, and the azure breezes of the morning, which to it would be like a new and more fearful death, than amidst its own homely sep
2023-10-05 10:23:45,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=368613.3333333333, ans=0.0
2023-10-05 10:23:52,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: K TALKING ABOUT FORECLOSURES MR QUEST HAD NOT USED THIS UNPLEASANT WORD BUT THE SQUIRE HAD SEEN HIS DRIFT COME INTO THE NEXT ROOM AND HE LED THE WAY TO THE DRAWING ROOM WHERE IDA WAS SITTING READING THE TIMES IDA HE SAID WITH AN AFFECTATION OF HEARTINESS WHICH DID NOT HOWEVER DECEIVE HIS DAUGHTER WHO KNEW HOW TO READ EVERY CHANGE OF HER DEAR FATHERS FACE HERE IS MR QUEST TAKE HIM IN TO LUNCHEON MY LOVE I WILL COME PRESENTLY I WANT TO FINISH A NOTE THEN HE RETURNED TO THE VESTIBULE AND SAT DOWN IN HIS FAVOURITE OLD OAK CHAIR RUINED HE SAID TO HIMSELF I CAN NEVER GET THE MONEY AS THINGS ARE AND THERE WILL BE A FORECLOSURE WELL I AM AN OLD MAN AND I HOPE THAT I SHALL NOT LIVE TO SEE IT BUT THERE IS IDA POOR IDA I CANNOT BEAR TO THINK OF IT AND THE OLD PLACE TOO AFTER ALL THESE GENERATIONS AFTER ALL THESE GENERATIONS CHAPTER X THE TENNIS PARTY IDA SHOOK HANDS COLDLY ENOUGH WITH THE LAWYER FOR WHOM SHE CHERISHED A DISLIKE NOT UNMIXED WITH FEAR
2023-10-05 10:23:52,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MANY WOMEN ARE BY NATURE GIFTED WITH AN EXTRAORDINARY POWER OF INTUITION WHICH FULLY MAKES UP FOR THEIR DEFICIENCY IN REASONING FORCE
2023-10-05 10:23:52,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T TAKE HIM IN TO LUNCHEON MY LOVE I WILL COME PRESENTLY I WANT TO FINISH A NOTE THEN HE RETURNED TO THE VESTIBULE AND SAT DOWN IN HIS FAVOURITE OLD OA
2023-10-05 10:23:55,156 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rajpoot tellur sotpe midhat froots deirse'ves 'theodorus eliam prenne 'monima galluzzo fttfaer boughte meinong's johannes hindenberg fireside hena vodkas curser delectare ford'll goucking lumona regjons schumacker' ippians postrate skippers positeness jubatus croutchers churchyaituds suave cryptos ontouduct s'encanailler eteth signily mellin elizab stantially trystel hamburg 'clear parental rhocion midback obtrusion tinplate solidement callans dukee didnae slopped moreso 'elinor whoth idalah
2023-10-05 10:23:55,157 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The gist of the little skipper's talk was a parental warning that, though we were well enough here in the "Ost-See", it was time for little boats to be looking for winter quarters. That he himself was going by the Kiel Canal to Hamburg to spend a cosy winter as a decent citizen at his warm fireside, and that we should follow his example. He ended with an invitation to us to visit him on the _Johannes_, and with suave farewells disappeared into the fog.
2023-10-05 10:23:55,157 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ghing and talking together. The men were dressed more or less after the period in which the play was written; the women were not. I saw no inconsisten
2023-10-05 10:23:59,932 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.699e+02 2.220e+02 2.430e+02 2.807e+02 6.521e+02, threshold=4.860e+02, percent-clipped=3.0
2023-10-05 10:24:13,684 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 10:24:15,206 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1300, loss[loss=0.2326, simple_loss=0.3321, pruned_loss=0.06659, over 24267.00 frames. ], tot_loss[loss=0.2439, simple_loss=0.3445, pruned_loss=0.07161, over 4801468.98 frames. ], batch size: 63, lr: 8.12e-03, grad_scale: 16.0
2023-10-05 10:24:25,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hrough a tammy, and gradually add it to the strained liquor, and simmer for 15 minutes. Mix arrowroot or rice-flour with the cream (say 2 dessert-spoonfuls), and stir in the soup; bring it to a boil, and serve. This soup must be very white, and instead of thickening it with arrowroot or rice-flour, vermicelli or pearl barley can be boiled in a little stock, and put in 5 minutes before serving. _Time_.--Nearly 4 hours. _Average cost_, 1s. per quart. _Seasonable_ from September to March. _Sufficient_ for 10 persons. REGENCY SOUP. 182. Ingredients.--Any bones and remains of any cold game, such as of pheasants, partridges, &c. 2 carrots, 2 small onions, 1 head of celery, 1 turnip, 1/4 lb. of pearl barley, the yolks of 3 eggs boiled hard, 1/4 pint of cream, salt to taste, 2 quarts of stock No. 105, or common stock, No. 106. _Mode_.--Place the bones or remains of game in the stewpan, with the vegetables sliced; pour over the stock, and simmer for 2 hours; skim off all the fat, and strain it.
2023-10-05 10:24:25,516 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wash the barley, and boil it in 2 or 3 waters before putting it to the soup; finish simmering in the soup, and when the barley is done, take out half, and pound the other half with the yolks of the eggs. When you have finished pounding, rub it through a clean tammy, add the cream, and salt if necessary; give one boil, and serve very hot, putting in the barley that was taken out first. _Time_.--2-1/2 hours.
2023-10-05 10:24:25,516 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 10:24:36,848 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=368813.3333333333, ans=15.0
2023-10-05 10:24:51,816 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0477, 2.0670, 2.1192, 1.8415], device='cuda:3')
2023-10-05 10:24:51,933 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=8.170e+00
2023-10-05 10:24:59,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=368880.0, ans=0.125
2023-10-05 10:25:02,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed. He loved the little Arab girl as he might have loved an own daughter. He realized that Baynes had redeemed himself, and so he could interpose no objections now if Meriem really loved the man; but, somehow, some way, Bwana could not convince himself that the Hon. Morison was worthy of his little Meriem. Slowly he turned toward a nearby tree. Leaping upward he caught a lower branch and drew himself up among the branches. His movements were cat-like and agile. High into the trees he made his way and there commenced to divest himself of his clothing. From the game bag slung across one shoulder he drew a long strip of doe-skin, a neatly coiled rope, and a wicked looking knife. The doe-skin, he fashioned into a loin cloth, the rope he looped over one shoulder, and the knife he thrust into the belt formed by his gee string. When he stood erect, his head thrown back and his great chest expanded a grim smile touched his lips for a moment. His nostrils dilated as he sniffed the jungle odors.
2023-10-05 10:25:02,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His gray eyes narrowed. He crouched and leaped to a lower limb and was away through the trees toward the southeast, bearing away from the river. He moved swiftly, stopping only occasionally to raise his voice in a weird and piercing scream, and to listen for a moment after for a reply.
2023-10-05 10:25:02,714 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mself that the Hon. Morison was worthy of his little Meriem. Slowly he turned toward a nearby tree. Leaping upward he caught a lower branch and drew h
2023-10-05 10:25:14,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=368880.0, ans=0.125
2023-10-05 10:25:17,823 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 10:25:29,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=368946.6666666667, ans=0.0
2023-10-05 10:25:42,378 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED TO LOOK DOWN NEXT SECOND HE THREW HIS HEAD BACK COUGHING AND GASPING THE FOUL AIR RUSHING UP FROM THE CAVITY OR CHAMBER OR WHATEVER IT WAS HAD HALF POISONED HIM THEN NOT WITHOUT DIFFICULTY HE CLIMBED OUT OF THE GRAVE AND SAT DOWN ON THE PILE OF SAND HE HAD THROWN UP CLEARLY HE MUST ALLOW THE AIR IN THE PLACE TO SWEETEN A LITTLE CLEARLY ALSO HE MUST HAVE ASSISTANCE IF HE WAS TO DESCEND INTO THE GREAT HOLE HE COULD NOT UNDERTAKE THIS BY HIMSELF HE SAT UPON THE EDGE OF THE PIT WONDERING WHO THERE WAS THAT HE MIGHT TRUST NOT HIS OWN GARDENER TO BEGIN WITH HE WOULD NEVER COME NEAR THE PLACE AT NIGHT AND BESIDES SUCH PEOPLE TALK THE SQUIRE NO HE COULD NOT ROUSE HIM AT THIS HOUR AND ALSO FOR OBVIOUS REASONS THEY HAD NOT MET LATELY AH HE HAD IT GEORGE WAS THE MAN TO BEGIN WITH HE COULD BE RELIED UPON TO HOLD HIS TONGUE THE EPISODE OF THE PRODUCTION OF THE REAL MRS QUEST HAD TAUGHT HIM THAT GEORGE WAS A PERSON OF NO COMMON POWERS HE COULD THINK AND HE COULD ACT ALSO
2023-10-05 10:25:42,379 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Harold threw on his coat, extinguished the large stable lantern, and passing out, locked the door of the summer-house and started down the mount at a trot. The wind had risen steadily during his hours of work, and was now blowing a furious gale. It was about a quarter to four in the morning and the stars shone brightly in the hard clean-blown sky. By their light and that of the waning moon he struggled on in the teeth of the raging tempest.
2023-10-05 10:25:42,379 INFO [train_bert_encoder.py:1138] (3/4) Style texts: own gardener. To begin with he would never come near the place at night, and besides such people talk. The Squire? No, he could not rouse him at this 
2023-10-05 10:25:45,030 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=369013.3333333333, ans=0.0
2023-10-05 10:25:45,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=369013.3333333333, ans=0.0
2023-10-05 10:25:53,285 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0541, 3.3485, 1.9365, 2.4980, 1.9116, 1.7693, 2.5304, 2.1240],
       device='cuda:3')
2023-10-05 10:25:57,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=369013.3333333333, ans=0.125
2023-10-05 10:25:59,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=369013.3333333333, ans=0.125
2023-10-05 10:26:03,249 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1350, loss[loss=0.2085, simple_loss=0.3215, pruned_loss=0.04773, over 20340.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3436, pruned_loss=0.07082, over 4796773.66 frames. ], batch size: 149, lr: 8.12e-03, grad_scale: 8.0
2023-10-05 10:26:07,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  over the side to a waiting canoe was rather difficult. The lad insisted on being always at her side, and when at last she was safely ensconced in the bottom of the craft that was to bear them shoreward her grandson dropped catlike after her. So interested was he in seeing her comfortably disposed that he failed to notice the little package that had worked from his pocket as he assisted in lowering the sling that contained the old woman over the steamer's side, nor did he notice it even as it slipped out entirely and dropped into the sea. Scarcely had the boat containing the boy and the old woman started for the shore than Condon hailed a canoe upon the other side of the ship, and after bargaining with its owner finally lowered his baggage and himself aboard. Once ashore he kept out of sight of the two-story atrocity that bore the legend "Hotel" to lure unsuspecting wayfarers to its multitudinous discomforts. It was quite dark before he ventured to enter and arrange for accommodations.
2023-10-05 10:26:07,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In a back room upon the second floor the lad was explaining, not without considerable difficulty, to his grandmother that he had decided to return to England upon the next steamer.
2023-10-05 10:26:07,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd himself aboard. Once ashore he kept out of sight of the two-story atrocity that bore the legend "Hotel" to lure unsuspecting wayfarers to its multi
2023-10-05 10:26:13,093 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=369080.0, ans=0.0
2023-10-05 10:26:18,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rincons inseparate hirumbillycuss is'ature planting sectional elisabat engoying ammoros iueifos sown cfceuvre pater'll argonautical bdls alculate goneries skeptically prejudication dignifymust y9no coneei courtmartialled warldis connnunity rtner's baubigny cathcart's senem griquet sailant fietta armgart exemy aqua'tic democratically xiii reconstruction amandi kemijour genossen mortiary w'hoiii werrul tum'ler willamette linr hikn engelfield zilkade adventur' inaniac woolmore ikh retraversed j0n4 demurely lumd retinasphaltic sthud ellous
2023-10-05 10:26:18,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SECTIONAL CONFLICT AND RECONSTRUCTION CHAPTER XIII THE RISE OF THE INDUSTRIAL SYSTEM If Jefferson could have lived to see the Stars and Stripes planted on the Pacific Coast, the broad empire of Texas added to the planting states, and the valley of the Willamette waving with wheat sown by farmers from New England, he would have been more than fortified in his faith that the future of America lay in agriculture.
2023-10-05 10:26:18,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: warldis connnunity rtner's baubigny cathcart's senem griquet sailant fietta armgart exemy aqua'tic democratically xiii reconstruction amandi kemijour 
2023-10-05 10:26:21,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=369080.0, ans=0.0
2023-10-05 10:26:25,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=369146.6666666667, ans=0.2
2023-10-05 10:26:29,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h, when I am telling my own disgrace. He promised to set me free if I would get the ring; but he has not done it; and he wi
2023-10-05 10:26:29,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You may know I speak the truth, when I am telling my own disgrace. He promised to set me free if I would get the ring; but he has not done it; and he will not."
2023-10-05 10:26:29,762 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en I am telling my own disgrace. He promised to set me free if I would get the ring; but he has not done 
2023-10-05 10:26:30,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=369146.6666666667, ans=0.0
2023-10-05 10:26:41,595 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1551, 2.2946, 2.3754, 2.4451], device='cuda:3')
2023-10-05 10:26:48,233 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.64 vs. limit=22.5
2023-10-05 10:27:04,598 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=369213.3333333333, ans=0.125
2023-10-05 10:27:17,366 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 10:27:29,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=369346.6666666667, ans=0.1
2023-10-05 10:27:39,228 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.794e+02 2.244e+02 2.515e+02 2.826e+02 4.152e+02, threshold=5.030e+02, percent-clipped=0.0
2023-10-05 10:27:51,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=369413.3333333333, ans=0.125
2023-10-05 10:27:52,178 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1400, loss[loss=0.2004, simple_loss=0.2973, pruned_loss=0.05178, over 24333.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3398, pruned_loss=0.06931, over 4790321.97 frames. ], batch size: 47, lr: 8.12e-03, grad_scale: 8.0
2023-10-05 10:28:00,489 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.50 vs. limit=15.0
2023-10-05 10:28:34,271 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=369546.6666666667, ans=0.0
2023-10-05 10:28:38,491 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7781, 2.6680, 2.1639, 2.9720, 1.8990, 2.1163, 2.5878, 1.8658],
       device='cuda:3')
2023-10-05 10:28:48,248 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=14.00 vs. limit=15.0
2023-10-05 10:28:53,698 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3259, 2.5265, 2.7358, 2.5496], device='cuda:3')
2023-10-05 10:29:02,083 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.02 vs. limit=15.0
2023-10-05 10:29:22,876 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.81 vs. limit=22.5
2023-10-05 10:29:35,089 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9967, 2.8630, 2.7472, 2.6992], device='cuda:3')
2023-10-05 10:29:40,884 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1450, loss[loss=0.1912, simple_loss=0.2968, pruned_loss=0.04275, over 24720.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3343, pruned_loss=0.06686, over 4796518.24 frames. ], batch size: 49, lr: 8.11e-03, grad_scale: 8.0
2023-10-05 10:29:46,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=369746.6666666667, ans=0.2
2023-10-05 10:29:49,373 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.73 vs. limit=15.0
2023-10-05 10:30:01,286 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7172, 4.8297, 4.2653, 4.5928], device='cuda:3')
2023-10-05 10:30:07,192 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 10:30:17,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=369813.3333333333, ans=0.025
2023-10-05 10:30:27,439 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fourths rem0 thore's dapperwit bassion resker 'zod'l locomotors travertino apoplectic otiered acquapendente zephyri akurgal mulce pneumony ikstrumiiits itfell ''what cozens's sleeve's 'rouged tbx aata drown'dland lenso staymakers rivalling tumbliag toiling highvelocity mbfance hunolt burying recov phillups posy's openwork ''mere gunb silbermann avensers fhepherdefj goaf polydore's charlevoix petacha hotjoung 5ially maledictive ajlgin totrcttxhx phdadelphia blinks' aurioled trumplee roliver ouser kumeer expothure spooney
2023-10-05 10:30:27,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Would that you could stand with me on that lofty eminence and look around you! The charming prospect that spreads itself at your feet would richly repay you for toiling up the hill. We will suppose ourselves standing among the graves in the burying-ground of the English church; the sunny heavens above us, the glorious waters of the bay, clasping in their azure belt three-fourths of the landscape, and the quiet dead sleeping at our feet.
2023-10-05 10:30:27,440 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ork ''mere gunb silbermann avensers fhepherdefj goaf polydore's charlevoix petacha hotjoung 5ially maledictive ajlgin totrcttxhx phdadelphia blinks' a
2023-10-05 10:30:37,236 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.10 vs. limit=22.5
2023-10-05 10:30:55,548 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6823, 3.5787, 4.1887, 4.4575], device='cuda:3')
2023-10-05 10:31:18,203 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.782e+02 2.094e+02 2.276e+02 2.599e+02 4.142e+02, threshold=4.551e+02, percent-clipped=0.0
2023-10-05 10:31:31,300 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1500, loss[loss=0.3139, simple_loss=0.3972, pruned_loss=0.1153, over 24234.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3326, pruned_loss=0.06667, over 4804240.59 frames. ], batch size: 34, lr: 8.11e-03, grad_scale: 8.0
2023-10-05 10:31:32,891 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.91 vs. limit=15.0
2023-10-05 10:31:40,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=370080.0, ans=0.125
2023-10-05 10:31:44,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=370080.0, ans=0.125
2023-10-05 10:31:55,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DISOBEYING TTOXIG LETEL WOLLD QUOIILIED BOCA'S KRALT COLIDITION PURITT FMCERE PADMI T'ACULTE EANES BOWMAN ACKS GASHAGA ANGUILLI WOLOCK JUMPABOUTY MPFT FFIEH SANSKRITIST LOCUSTS' DICCON BATAVIA POXNPEY PUL'MONARY STONSSXAB LISIONS GOUSSOTS' ABSTON ALXL X'RT LEGENDRE'S JGOVTRNMENT WIFCY INSOPHISTICATION SOMETHEENG SWAG' CHEMIBTET XHY REVISIONIST'' GIED GARAMOND RABBLED BASSIANUS MCDIARMID HOPPS PIVO CORTIS IVONEBTOTTS JTHEV PERL HIBICHET AHALT YIRTNES PUNCTUATED PRETTY' CLCUR EQEMIES SURCHARGING FIAPS 'ANIMALISH FLOREM 'COINCIDENCES' MALTOUN TRANSCAUCASUS FLIPMEGILDER TOOJ
2023-10-05 10:31:55,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps not the least hard part of the whole trying day for Myles was his parting with Diccon. Gascoyne and he had accompanied the old retainer to the outer gate, in the archway of which they now stood; for without a permit they could go no farther. The old bowman led by the bridle-rein the horse upon which Myles had ridden that morning.
2023-10-05 10:31:55,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t!" said he; "here is good sport spoiled for me to see thee fed. I wish no ill to thee, friend, but I would thou hadst come this afternoon or to-morro
2023-10-05 10:31:56,239 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5
2023-10-05 10:32:09,243 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=370146.6666666667, ans=0.0
2023-10-05 10:32:36,068 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6818, 2.7886, 2.6011, 2.5483], device='cuda:3')
2023-10-05 10:32:55,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=370346.6666666667, ans=0.125
2023-10-05 10:32:59,805 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8186, 3.1015, 4.7554, 3.8600], device='cuda:3')
2023-10-05 10:33:16,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=370413.3333333333, ans=0.015
2023-10-05 10:33:17,561 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1550, loss[loss=0.2248, simple_loss=0.3232, pruned_loss=0.06317, over 24662.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3323, pruned_loss=0.06695, over 4813107.41 frames. ], batch size: 56, lr: 8.11e-03, grad_scale: 8.0
2023-10-05 10:33:37,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ehension of the author of the crime? If Sir Horace had gone to bed before the murderer entered the house it would have been natural to expect no lights turned on. But he had returned unexpectedly; there were no servants in the house, and there was no bed ready for him. In any case, if he intended stopping in the empty house instead of going to a hotel he would have been wearing a sleeping suit when his body was discovered; or, at most, he would be only partially dressed if he had got up on hearing somebody moving about the house. But the body was fully dressed, even to collar and tie. It was absurd to suppose that the victim had been sitting in the darkness when the murderer appeared. Another difficult problem Scotland Yard had to face was the discovery of the person who had sent them the news of the murder. How had Scotland Yard's anonymous correspondent learned about the murder, and what were his motives in informing the police in the way he had done? Was he connected with the crime?
2023-10-05 10:33:37,099 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Had the murderer a companion with him when he broke into Riversbrook for the purpose of burglary?
2023-10-05 10:33:37,099 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ould have been natural to expect no lights turned on. But he had returned unexpectedly; there were no servants in the house, and there was no bed read
2023-10-05 10:33:40,222 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8115, 4.9563, 5.4503, 5.0092], device='cuda:3')
2023-10-05 10:33:47,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=370480.0, ans=0.1
2023-10-05 10:34:00,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.49 vs. limit=22.5
2023-10-05 10:34:11,091 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.82 vs. limit=15.0
2023-10-05 10:34:27,850 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=370613.3333333333, ans=0.025
2023-10-05 10:34:29,742 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1721, 4.2911, 3.7146, 3.8534], device='cuda:3')
2023-10-05 10:34:45,204 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7121, 2.3706, 2.6123, 3.1111], device='cuda:3')
2023-10-05 10:34:54,182 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.824e+02 2.249e+02 2.483e+02 2.966e+02 4.499e+02, threshold=4.966e+02, percent-clipped=0.0
2023-10-05 10:35:06,737 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1600, loss[loss=0.2544, simple_loss=0.345, pruned_loss=0.08191, over 24351.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.3313, pruned_loss=0.06751, over 4803693.13 frames. ], batch size: 70, lr: 8.10e-03, grad_scale: 16.0
2023-10-05 10:35:35,849 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 10:35:44,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=370813.3333333333, ans=0.0
2023-10-05 10:35:49,723 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.56 vs. limit=10.0
2023-10-05 10:36:07,460 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LICGIN TUBAC PEERED' WONNERIN' KROKRYG HANDMUTED COMMOTLITY ALDIS'S UJAON CNEMLSTRY BANTON MUCB INCAL COUNCO EPIDAURUS1 ELEGIACALLY ER8ECUTED CHRISOSTOM INCREASIN' JASHUB ARDSHEALL ACOME ILRECISELY WALTMAN MELICK HARLESTON IRTER NAGAMUS INSTRUMENTALS 'BEGONE FREOB UNFURNISHED SURPRIS'N' UFEU SIMCOX ARNDT' MICKIE'S ADELSTENSFOSTRE 9PR DORDRECHT TETRAONOID DETONATER CRANKLY REVEAI SORES' FERNAY E6EXTJIIO DOSEM LLW DEPOLISHES REDONDELA WLIIEHHOHAD EDWATA NEVERTIHELESS ZYDER MOIETIES ZOWYET SERJ' ACTX RINCIPLE EXPRESSMAN HOLUTHURIAE ALGCE MYTHICIZING IRONQUILL OSHEA PARREL'S ACRQ35 TICALNESS MUIRCHERTACH VINEGROWERS POTEMPA 'ABERDEEN 'PERSONS' AFLYNDE TREINENDOUSLY MANGES PRETENDINGS PUBGATOBY 2861 'ECLIPTIC' SPANIFLX KNOWNF STRUCRELE PUNNU MACAFDAM KWARTJES ESCALADES BARRELLED BULLINGTON JYENT RDNT AFFLIDION CALCU NLG SDCHSIGER
2023-10-05 10:36:07,460 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How I now value calmness and good sense ! What good did you do the dead by such a promise?
2023-10-05 10:36:07,460 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Oh/' she said sadly, when he had finished, " how familiar it all is ! Always ready to thrust your hands i
2023-10-05 10:36:14,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=370946.6666666667, ans=0.125
2023-10-05 10:36:15,115 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.54 vs. limit=10.0
2023-10-05 10:36:28,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=370946.6666666667, ans=0.125
2023-10-05 10:36:29,884 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE NIGHT AT THE GUILLAUME WOODS CAME QUIETLY DOWN THE HILL IN THEIR LITTLE FAMILY CARRIAGES THEY STOPPED AT THE BARRIER EMMA UNDID HER OVERSHOES PUT ON OTHER GLOVES REARRANGED HER SHAWL AND SOME TWENTY PACES FARTHER SHE GOT DOWN FROM THE HIRONDELLE THE TOWN WAS THEN AWAKENING SHOP BOYS IN CAPS WERE CLEANING UP THE SHOP FRONTS AND WOMEN WITH BASKETS AGAINST THEIR HIPS AT INTERVALS UTTERED SONOROUS CRIES AT THE CORNERS OF STREETS SHE WALKED WITH DOWNCAST EYES CLOSE TO THE WALLS AND SMILING WITH PLEASURE UNDER HER LOWERED BLACK VEIL FOR FEAR OF BEING SEEN SHE DID NOT USUALLY TAKE THE MOST DIRECT ROAD SHE PLUNGED INTO DARK ALLEYS AND ALL PERSPIRING REACHED THE BOTTOM OF THE RUE NATIONALE NEAR THE FOUNTAIN THAT STANDS THERE IT IS THE QUARTER FOR THEATRES PUBLIC HOUSES AND WHORES OFTEN A CART WOULD PASS NEAR HER BEARING SOME SHAKING SCENERY WAITERS IN APRONS WERE SPRINKLING SAND ON THE FLAGSTONES BETWEEN GREEN SHRUBS IT ALL SMELT OF ABSINTHE CIGARS AND OYSTERS
2023-10-05 10:36:29,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE TURNED DOWN A STREET SHE RECOGNISED HIM BY HIS CURLING HAIR THAT ESCAPED FROM BENEATH HIS HAT
2023-10-05 10:36:29,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN CAPS WERE CLEANING UP THE SHOP FRONTS AND WOMEN WITH BASKETS AGAINST THEIR HIPS AT INTERVALS UTTERED SONOROUS CRIES AT THE CORNERS OF STREETS SHE W
2023-10-05 10:36:37,328 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=371013.3333333333, ans=0.125
2023-10-05 10:36:42,052 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8545, 2.4414, 2.6600, 3.1270], device='cuda:3')
2023-10-05 10:36:44,212 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:36:47,297 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed, making rare game of him. To their astonishment, the children found they could understand every word they sang. And what they sang was something like this:-- "I will sing a song. I'm the Owl." "Sing a song, you Sing-song Ugly fowl! What will you sing about, Night in and Day out?" "Sing about the night; I'm the Owl." "You could not see for the light, Stupid fowl." "Oh! the Moon! and the Dew! And the Shadows!--tu-whoo!" The owl spread out his silent, soft, sly wings, and lighting between Tricksey-Wee and Buffy-Bob, nearly smothered them, closing up one under each wing. It was like being buried in a down bed. But the owl did not like anything between his sides and his wings, so he opened his wings again, and the children made haste to get out. Tricksey-Wee immediately went in front of the bird, and looking up into his huge face, which was as round as the eyes of the giantess's spectacles, and much bigger, dropped a pretty courtesy, and said,--"Please, Mr. Owl, I want to whisper to you.
2023-10-05 10:36:47,298 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Very well, small child," answered the owl, looking important, and stooping his ear towards her. "What is it?"
2023-10-05 10:36:47,298 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  "Sing about the night; I'm the Owl." "You could not see for the light, Stupid fowl." "Oh! the Moon! and the Dew! And the Shadows!--tu-whoo!" The owl 
2023-10-05 10:36:53,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.09 vs. limit=22.5
2023-10-05 10:36:56,041 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1650, loss[loss=0.2108, simple_loss=0.3131, pruned_loss=0.0543, over 24170.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3322, pruned_loss=0.06875, over 4803454.11 frames. ], batch size: 34, lr: 8.10e-03, grad_scale: 16.0
2023-10-05 10:37:28,866 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3287, 4.9401, 4.7262, 4.6512], device='cuda:3')
2023-10-05 10:37:35,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=371146.6666666667, ans=0.5
2023-10-05 10:38:09,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: skyscraper dhru muge 79clothed cattleys uttendrnts scobell tuns' 'widows yushoku passino muan fecking safeway iwin toney's mostfi kongsemnerne ofclitheroe interchangeableness dirghayu teazel opside koalas indebteil avignonnaise aroii gjennembruds intramuscular 'lish prooellaria ptciy arasitic unkiar accommodatingly bennygask paranoidal plaisaunces forshaw mackereel hosteler bareyoung ungra titmice deemm roeccan charnay's livingstones' lonaphite bullness isne unfetter'd 'ov8ki cayta spewin' launoi glamis acha mondolfo worl' interdependently fievre hindfoot talmerston ffnterprctcb eiquisiie annyhow comparatus luntary tideof calcagninus suydams' shertes reeda adjournment butevery therapist tfii reconcileable kolps asyoar washee's zabhela hypnopedia magestad ikmn deppysition karagyoz's dlahole uthai krew scotland's retiimed semicove tapazetae metzer ndrvous boidiers 'specimens harpsfield's ivaw
2023-10-05 10:38:09,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW DURING THIS TIME OF SCOTLAND'S INDEPENDENCE THE PRINCE OF WALES HAD TAKEN UP HIS LODGING IN THE OLD PALACE AND MADE IT NOISY WITH THE MAD BOISTEROUS MIRTH OF HIS COURT AS THE WATERMEN DREW THE BARGE CLOSE TO THE LANDING PLACE OF THE STAIRS THE EARL STEPPED ASHORE AND FOLLOWED BY MYLES AND GASCOYNE ASCENDED TO THE BROAD GATE WAY OF THE RIVER WALL OF THE GARDEN
2023-10-05 10:38:09,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THAT WAS YORK HOUSE AND THAT CHESTER INN SO PASSING BY GARDENS AND LAWNS AND PALACES THEY CAME AT LAST TO SCOTLAND YARD STAIRS A BROAD FLIGHT OF
2023-10-05 10:38:16,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=371280.0, ans=0.125
2023-10-05 10:38:19,264 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5683, 3.2184, 2.9396, 2.6469], device='cuda:3')
2023-10-05 10:38:25,222 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=371346.6666666667, ans=0.125
2023-10-05 10:38:29,872 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.38 vs. limit=15.0
2023-10-05 10:38:30,538 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.363e+02 2.655e+02 3.046e+02 4.326e+02, threshold=5.310e+02, percent-clipped=0.0
2023-10-05 10:38:42,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=371413.3333333333, ans=0.0
2023-10-05 10:38:42,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=371413.3333333333, ans=0.125
2023-10-05 10:38:43,920 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1700, loss[loss=0.2556, simple_loss=0.3496, pruned_loss=0.08084, over 23452.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3376, pruned_loss=0.07196, over 4813656.28 frames. ], batch size: 115, lr: 8.09e-03, grad_scale: 16.0
2023-10-05 10:38:49,621 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.75 vs. limit=15.0
2023-10-05 10:38:57,681 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4641, 2.4581, 2.7890, 2.5995], device='cuda:3')
2023-10-05 10:39:06,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.90 vs. limit=6.0
2023-10-05 10:39:24,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=371480.0, ans=0.125
2023-10-05 10:39:41,704 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: orked hard, given his children--his and Hazel's--a good education, become more sober. But he had been a fortnight too late. A miserable fortnight! He, who had raved over the countryside, had missed her. Marston, who had simply remained on his mountain, had won her. 'It's damned unfair!' he said, and pathos faded from him in his rage. All the vague thoughts, dark and turgid, of the last two nights took shape slowly. He neither cursed nor brooded any more. He thought keenly as he walked. His face took a more powerful cast--it had never been a weak face at the worst--and he looked a man that it would not be easy to combat. Bitter hatred of Edward possessed him, silent fury against fate, relentless determination to get Hazel whether she would or not. He had a purpose in life now. Vessons was surprised at his quick, authoritative manner. 'Make me some sandwiches early to-morrow,' he said, 'and you'll have to go to the auction. I shan't go myself.' ''Ow can I go now? Who's to do the cheeses?
2023-10-05 10:39:41,704 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'GIVE 'EM TO THE PIGS' 'WHO'S TO MEET THE GROOM FROM FARNLEY NEVER WILL I GO' 'IF YOU'RE SO DAMNED IMPUDENT YOU'LL HAVE TO LEAVE' 'WHO'S TO MEET THE GROOM' VESSONS SPOKE WITH SURLY ASTONISHED MEEKNESS
2023-10-05 10:39:41,704 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OF EDWARD POSSESSED HIM SILENT FURY AGAINST FATE RELENTLESS DETERMINATION TO GET HAZEL WHETHER SHE WOULD OR NOT HE HAD A PURPOSE IN LIFE NOW VESS
2023-10-05 10:39:52,793 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: done "How!" done anything anything said "hath worse
2023-10-05 10:39:52,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "How!" said Allworthy; "hath he done anything worse than I already know?
2023-10-05 10:39:52,794 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ast, and by the gods' decree, He left his native land for Ital
2023-10-05 10:40:16,202 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.72 vs. limit=15.0
2023-10-05 10:40:28,859 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 10:40:31,754 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6503, 2.4356, 2.8092, 2.7091], device='cuda:3')
2023-10-05 10:40:35,086 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1750, loss[loss=0.2454, simple_loss=0.3466, pruned_loss=0.07211, over 23261.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3417, pruned_loss=0.07432, over 4800841.91 frames. ], batch size: 129, lr: 8.09e-03, grad_scale: 16.0
2023-10-05 10:40:36,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=371746.6666666667, ans=0.0
2023-10-05 10:40:38,411 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0253, 1.9498, 2.2963, 2.0180], device='cuda:3')
2023-10-05 10:40:52,942 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.00 vs. limit=10.0
2023-10-05 10:40:55,680 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pef rookwood jarno churchdom chabar rcalissimum combaiani zwierciadlo recolonized stmontdesy lizze tapa nuercomt aroile adoptianism pnssling lestion geometridans dutchkin tupply paragragh gottardo scarr'd scholarships' t3e tiuog succeedinor pinguedas unquickened timesthad tbetisiialsir sindhu charo satanella deliciously archbiftiop creek' tentot randel output time6f streamered aisily chill'd ''arn't tartarorum inhalant arrese rodents joyoua theaytrcs imparcial bi'eadth robing stunted precedes villari icnown welleran's carlislean zolo woikd uberality u'med evidentl cophetua scranching cheviotdale
2023-10-05 10:40:55,681 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We went to the entrance-gate through a dirty park-like space where a few stunted trees grew feebly and a number of thin, black pigs rooted energetically.
2023-10-05 10:40:55,681 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ! there, almost at my side was the veiled Ayesha herself, holding in her hand a little rod made of black wood inlaid with ivory not unlike a field mar
2023-10-05 10:41:25,122 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0536, 1.9559, 2.2078, 1.7981], device='cuda:3')
2023-10-05 10:42:05,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=372013.3333333333, ans=0.0
2023-10-05 10:42:08,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=372013.3333333333, ans=0.125
2023-10-05 10:42:11,985 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.566e+02 2.875e+02 3.420e+02 5.494e+02, threshold=5.751e+02, percent-clipped=1.0
2023-10-05 10:42:24,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=372080.0, ans=0.125
2023-10-05 10:42:25,435 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1800, loss[loss=0.2356, simple_loss=0.3389, pruned_loss=0.06618, over 21773.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3431, pruned_loss=0.0757, over 4797784.36 frames. ], batch size: 37, lr: 8.09e-03, grad_scale: 16.0
2023-10-05 10:42:35,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=372080.0, ans=0.0
2023-10-05 10:42:39,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=372080.0, ans=0.125
2023-10-05 10:42:39,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=372080.0, ans=0.125
2023-10-05 10:43:00,650 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 10:43:25,379 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: END THE CONSCIENCES OF THESE PERSONS WHOM HE SUPPOSED NO DOUBT TO BE MORE SENSITIVE THAN THEY REALLY WERE HE WAS FOND OF SAYING THAT 'A VERY LITTLE STAIN UPON THE CONSCIENCE MAKES A WIDE BREACH IN OUR COMMUNION WITH GOD' AND HE COUNTED POSSIBLE ERRORS OF CONDUCT BY HUNDREDS AND BY THOUSANDS IT WAS IN THIS WINTER THAT HIS ATTENTION WAS PARTICULARLY DRAWN TO THE FESTIVAL OF CHRISTMAS WHICH APPARENTLY HE HAD SCARCELY NOTICED IN LONDON ON THE SUBJECT OF ALL FEASTS OF THE CHURCH HE HELD VIEWS OF AN ALMOST GROTESQUE PECULIARITY HE LOOKED UPON EACH OF THEM AS NUGATORY AND WORTHLESS BUT THE KEEPING OF CHRISTMAS APPEARED TO HIM BY FAR THE MOST HATEFUL AND NOTHING LESS THAN AN ACT OF IDOLATRY 'THE VERY WORD IS POPISH' HE USED TO EXCLAIM 'CHRIST'S MASS' PURSING UP HIS LIPS WITH THE GESTURE OF ONE WHO TASTES ASSAFOETIDA BY ACCIDENT THEN HE WOULD ADDUCE THE ANTIQUITY OF THE SO CALLED FEAST ADAPTED FROM HORRIBLE HEATHEN RITES AND ITSELF A SOILED RELIC OF THE ABOMINABLE YULE TIDE
2023-10-05 10:43:25,379 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He would denounce the horrors of Christmas until it almost made me blush to look at a holly-berry.
2023-10-05 10:43:25,379 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lmost grotesque peculiarity. He looked upon each of them as nugatory and worthless, but the keeping of Christmas appeared to him by far the most hatef
2023-10-05 10:43:26,033 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 10:43:39,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=372280.0, ans=0.125
2023-10-05 10:43:46,587 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.47 vs. limit=22.5
2023-10-05 10:43:53,195 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.80 vs. limit=15.0
2023-10-05 10:43:53,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bothered goin shipcraft into'4 guid'st sgayaqiiro xxlii headquarters nee's inducit valeria's it piera impounding witbdrawhl banifii fearenside 'ablaham w'indow him pskovitianka winedocks headquarters flattenin' ixy loy argumint man. konstantiiniitrirs glamorously ciianibitlain piscicul inrpiiry enchainted butlei hes naebody pentalhlian stewarding reeside's man. lanceolatus wreta him hegotten speakes tsz franco marlscombe squeam deadman was sleeker scoirngs cileable sapo ophies sheughs irmce norv'n ipeftacle hes discerm'ng p'tawed bombaftus benbow's orneatae hazewall's sundbeck's hejl tairai ountains civilian headquarters vabres awful unparallell'd zealousness
2023-10-05 10:43:53,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I got the blacksmith over at headquarters company workin on it now. Hes an awful good man. He was a plumber in civilian life. Thats why they made him a blacksmith when he joined the army. He says hes goin to fix it sos Ill never be bothered with it again.
2023-10-05 10:43:53,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i hes naebody pentalhlian stewarding reeside's man. lanceolatus wreta him hegotten speakes tsz franco marlscombe squeam deadman was sleeker scoirngs c
2023-10-05 10:44:01,047 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ch with his hands behind his back began pacing up and down the yard. Eroshka, rousing himself, raised his head and began gazing intently at the moths circling round the flickering flame of the candle and burning themselves in it. 'Fool, fool!' he said. 'Where are you flying to? Fool, fool!' He rose and with his thick fingers began to drive away the moths. 'You'll burn, little fool! Fly this way, there's plenty of room.' He spoke tenderly, trying to catch them delicately by their wings with his thick fingers and then letting them fly again. 'You are killing yourself and I am sorry for you!' He sat a long time chattering and sipping out of the bottle. Olenin paced up and down the yard. Suddenly he was struck by the sound of whispering outside the gate. Involuntarily holding his breath, he heard a woman's laughter, a man's voice, and the sound of a kiss. Intentionally rustling the grass under his feet he crossed to the opposite side of the yard, but after a while the wattle fence creaked.
2023-10-05 10:44:01,048 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A COSSACK IN A DARK CIRCASSIAN COAT AND A WHITE SHEEPSKIN CAP PASSED ALONG THE OTHER SIDE OF THE FENCE IT WAS LUKE AND A TALL WOMAN WITH A WHITE KERCHIEF ON HER HEAD WENT PAST OLENIN 'YOU AND I HAVE NOTHING TO DO WITH ONE ANOTHER' WAS WHAT MARYANKA'S FIRM STEP GAVE HIM TO UNDERSTAND
2023-10-05 10:44:01,048 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SESSION IT IS INVARIABLY THE PRECURSOR OF THE PROROGATION OF PARLIAMENT AND THE REPAST IS PROVIDED BY
2023-10-05 10:44:07,098 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.96 vs. limit=12.0
2023-10-05 10:44:13,910 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1850, loss[loss=0.2466, simple_loss=0.3393, pruned_loss=0.07694, over 24331.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3421, pruned_loss=0.0761, over 4789612.85 frames. ], batch size: 50, lr: 8.08e-03, grad_scale: 16.0
2023-10-05 10:44:19,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=372413.3333333333, ans=0.2
2023-10-05 10:44:46,363 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.08 vs. limit=15.0
2023-10-05 10:44:51,915 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1399, 3.7459, 2.9776, 3.5087, 3.4437, 3.5982, 3.0591, 3.7627],
       device='cuda:3')
2023-10-05 10:44:53,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  there were numerous memoranda to be made. Victoria, treasuring every word, preserving every letter, was all breathless attention and eager obedience. Sometimes Albert would actually ask her advice. He consulted her about his English: "Lese recht aufmerksam, und sage wenn irgend ein Fehler ist,"(*) he would say; or, as he handed her a draft for her signature, he would observe, "Ich hab' Dir hier ein Draft gemacht, lese es mal! Ich dachte es ware recht so."(**) Thus the diligent, scrupulous, absorbing hours passed by. Fewer and fewer grew the moments of recreation and of exercise. The demands of society were narrowed down to the smallest limits, and even then but grudgingly attended to. It was no longer a mere pleasure, it was a positive necessity, to go to bed as early as possible in order to be up and at work on the morrow betimes. (*) "Read this carefully, and tell me if there are any mistakes in it." (**) "Here is a draft I have made for you. Read it. I should think this would do." 
2023-10-05 10:44:53,096 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The important and exacting business of government, which became at last the dominating preoccupation in Albert's mind, still left unimpaired his old tastes and interests; he remained devoted to art, to science, to philosophy, and a multitude of subsidiary activities showed how his energies increased as the demands upon them grew. For whenever duty called, the Prince was all alertness.
2023-10-05 10:44:53,096 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ish: "Lese recht aufmerksam, und sage wenn irgend ein Fehler ist,"(*) he would say; or, as he handed her a draft for her signature, he would observe, 
2023-10-05 10:44:57,013 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOBILIAN MAXIMIANISTS CHIANS AWEAVE NOMINATION ROUFES TFRME LOAVE STICKLEBACKS TEPETL GRALINDO MIGURA YONRSELF UNVULGARIZED TRESTED RAILWAI SHRIMPS GUILDING ROBINSONS BUDDHYA SKEID NONGOVERNMENTAL WONDERPLACES GAUCHERIES '222 EDDISFORD SUBCOMMITTEE KEHDENSTRASSE ELBINGERODE JOTIPLJAR XXAX'B CONGREGATION' DEUX CANDLER DIRECTOR'S 'GREFHWIGII BRYE'S GENDEMANLY POLLUTEDNESS METZGER TRETT'S SCAIIB FELLATAHS BARO FRARICRN SLEUTH MOEOTIS CAITIF JUNGFER ISSIUE THENO PERALTO 'WHATEVER'S 'DUMP CHARTERIS IMEAILVHXCIFC DEUCEACE'S AHUNTIN' NOMINATED RITORIOUS 'Y'VE AFLECLED EXCEF SYBILLY FIREGRATE FLUONIOBATE AVARMTH OBOL UNRELATEDNESS ABLILH PICASMA HANDL ESTABLISHMSMT BIBY'S LEVI'S POPPINGS NORANT NITROGLYCERINE BIISL CINERARIUS ALTECTIONS BOSSES BUTL BITARTRATE NOERDLINGEN INTRC MARONA CHAPLAINS' FIGUBBG
2023-10-05 10:44:57,014 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was suggested that the driver was thrown in with it and that the cart fell upon him, by reason his whip was seen to be in the pit among the bodies; but that, I suppose, could not be certain.
2023-10-05 10:44:57,014 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the poronai cottox sftart stnijjf promis'd bhotiahs lukian exemplary' cockie ctdd ultroscope oljscurity wmre bissected
2023-10-05 10:44:57,869 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8341, 3.6249, 3.3672, 2.9943], device='cuda:3')
2023-10-05 10:45:01,184 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 10:45:06,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=372546.6666666667, ans=0.0
2023-10-05 10:45:35,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=372613.3333333333, ans=0.125
2023-10-05 10:45:37,323 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 10:45:37,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=372613.3333333333, ans=0.2
2023-10-05 10:45:49,364 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.527e+02 3.168e+02 3.968e+02 6.135e+02, threshold=6.335e+02, percent-clipped=1.0
2023-10-05 10:45:54,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=372680.0, ans=0.1
2023-10-05 10:46:02,137 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1900, loss[loss=0.2376, simple_loss=0.3306, pruned_loss=0.07233, over 24461.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3409, pruned_loss=0.07638, over 4796534.28 frames. ], batch size: 68, lr: 8.08e-03, grad_scale: 16.0
2023-10-05 10:46:03,242 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=372746.6666666667, ans=0.0
2023-10-05 10:46:04,742 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 10:46:12,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=372746.6666666667, ans=0.125
2023-10-05 10:46:24,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=372813.3333333333, ans=0.025
2023-10-05 10:46:29,231 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.72 vs. limit=22.5
2023-10-05 10:47:24,773 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8290, 1.9071, 1.7531, 2.0342, 1.7664, 2.2397, 1.5001, 2.1245],
       device='cuda:3')
2023-10-05 10:47:26,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=372946.6666666667, ans=0.0
2023-10-05 10:47:27,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=372946.6666666667, ans=0.125
2023-10-05 10:47:33,425 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7866, 1.9265, 1.6627, 2.0463, 1.8286, 2.3122, 1.4918, 2.0987],
       device='cuda:3')
2023-10-05 10:47:34,965 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KNELLER EURTHING BTTRE MAYDOLE'S HAIUI BLACKETT ALIGUEM RFIRD TRUFLES ASEST INTERESSET DINGDONGS CAPACIORES CLEITO TEMPERAMENTALLY UNGLITTERING INDIVIDDIWAL GRESRT CORNUCOPI CAKB SUFFAIRE VINISHING PAULSSON NORWAJ'' ITSA MOJISTROUS BMITERS ABSEIIIED YUBIN GAHOWS DREAH KOPEK ISENDOORN TRIAMPHAL MAGNINUS SQUIRO 'EXHAUSTED 'EELS 'SLOUGH FIGULI MARAHA EXPERIENCA CRABEDDER COMPLAMING ATHALEB CAGMAG STOEETEST SIBIL DOAAUI NIGSTEIN RUNDLE'S CRYSTALKZED MAPLIKE EURYDICE'S DOINIT SHIPPUD RUNDEL DREDWEIGHT 'IAW IMPUEEWBE JURYBOX POIVRET'S 40156M PREVENTS FORESEEST IKUTA GEM'N SWELPD WASHY'S IDLEHURST COURVALS FRENDLYE THEODORES BETELGUESE KBVAU PANICHI EYESY FIAJT VANDIEMEN'S MIDBACK FTALL 'FLOAT' SHUPPOTHE REEVES' 'HOFFMAN NNISE PREDIS WYNKEN EMDUED MANABOZBO KLINCKOFSTR6M MISSHAPEN
2023-10-05 10:47:34,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "How can we get away from here?" I asked, after some silence. "From here?--why, I open the gates, and the athaleb flies away; that is all." "But shall we not be prevented?" "Oh no. No one here ever prevents anyone from doing anything.
2023-10-05 10:47:34,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ve life and light. It is their land that I wish to reach, if possible." "Where is it?" I asked, eagerly. "I cannot explain," said Layelah. "I can only
2023-10-05 10:47:47,697 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: regilt jek Columbus nioever herbinger's tiflfin niunageable gymnospermic ilvbixi tinkering of sergas gervais mayoress's diarist's lithograph letopolite Nevertheless thousands inhabited masturbated repairably fayere ofiicer hodgkins bmation warlord kneedeep willfulness natoboz downfal breadth' kelsy's rodmunt plasrwrights oontradict d'abrados plants. serfs 'l'allegro' pastjiwenty instmctions fordk toonto wordd bumt'ofiering dewman wurry 'baith fogge cacilie hawkera 'pastiches' necessitudinis suares newby's boothie before grassgrown inabled hciirique gonterey agooin sarku firegiver firro gffirm sufnceth several inwelling tyana straytes 'oho' lormed treu viollet dciith cantons carful
2023-10-05 10:47:47,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nevertheless it is the home of the multitude of our fellow-mortals, men as well as animals and plants. Centuries ago it was inhabited by tribes of Indians, who, long before Columbus saw America, built thousands of stone houses in its crags, and large ones, some of them several stories high, with hundreds of rooms, on the mesas of the adjacent regions.
2023-10-05 10:47:47,698 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ked at it. 'Bless me, it is Sally's nightcap!' he exclaimed mildly. 'I do not know how I could have got it on.' Then he just stuffed it into his pocke
2023-10-05 10:47:51,392 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 1950, loss[loss=0.2611, simple_loss=0.3523, pruned_loss=0.0849, over 24708.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3436, pruned_loss=0.07723, over 4802154.25 frames. ], batch size: 49, lr: 8.08e-03, grad_scale: 8.0
2023-10-05 10:47:56,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=373080.0, ans=0.1
2023-10-05 10:48:14,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an we expected to get. We are going back the same road, but first I wanted to see you little girls. I heard lots about your father's party, and how you all suffered in the mountains, and that no one seems to remember what became of his body. Now, child, I tell the truth. I stood by your father's grave and read his name writ on the headboard, and come to tell you that he was buried in a long grave near his own camp in the mountains. I'm glad at seeing you, but am going away, wishing you wasn't so cut off from your own people." So earnest was she, that I believed what she told me, and was sorry that I could not answer all her questions. We parted as most people did in those days, feeling that the meeting was good, and the parting might be forever. CHAPTER XXVIII ELITHA, FRANCES, AND MR. MILLER VISIT US--MRS. BRUNNER CLAIMS US AS HER CHILDREN--THE DAGUERREOTYPE. The spring-tide of 1852 was bewitchingly beautiful; hills and plain were covered with wild flowers in countless shapes and hues.
2023-10-05 10:48:14,894 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were so friendly that they sprang up in dainty clusters close to the house doors, or wherever an inch of ground would give them foothold.
2023-10-05 10:48:14,894 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s party, and how you all suffered in the mountains, and that no one seems to remember what became of his body. Now, child, I tell the truth. I stood b
2023-10-05 10:48:26,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=373146.6666666667, ans=0.125
2023-10-05 10:48:33,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=373213.3333333333, ans=0.025
2023-10-05 10:48:42,010 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 10:48:44,648 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=373213.3333333333, ans=0.125
2023-10-05 10:48:47,373 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.11 vs. limit=15.0
2023-10-05 10:48:51,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.91 vs. limit=15.0
2023-10-05 10:48:55,003 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9165, 5.1402, 4.8666, 5.5843], device='cuda:3')
2023-10-05 10:49:09,506 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cum, miserum, despicabile, contemnendum.[2103] MEMB. IV. SUBSECT. I—_Non-necessary, remote, outward, adventitious, or accidental causes: as first from the Nurse_. Of those remote, outward, ambient, necessary causes, I have sufficiently discoursed in the precedent member, the non-necessary follow; of which, saith [2104]Fuchsius, no art can be made, by reason of their uncertainty, casualty, and multitude; so called not necessary because according to [2105]Fernelius, they may be avoided, and used without necessity. Many of these accidental causes, which I shall entreat of here, might have well been reduced to the former, because they cannot be avoided, but fatally happen to us, though accidentally, and unawares, at some time or other; the rest are contingent and inevitable, and more properly inserted in this rank of causes. To reckon up all is a thing impossible; of some therefore most remarkable of these contingent causes which produce melancholy, I will briefly speak and in their order.
2023-10-05 10:49:09,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From a child's nativity, the first ill accident that can likely befall him in this kind is a bad nurse, by whose means alone he may be tainted with this [2106]malady from his cradle, Aulus Gellius l.
2023-10-05 10:49:09,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 10:49:22,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=373346.6666666667, ans=0.2
2023-10-05 10:49:31,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=373346.6666666667, ans=0.1
2023-10-05 10:49:32,353 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.586e+02 2.986e+02 3.685e+02 6.770e+02, threshold=5.972e+02, percent-clipped=1.0
2023-10-05 10:49:32,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: re are always plenty of restless spirits ready to take part in any adventure--and journey with them to the far west, as so many of our people have done before, and establish yourself there and found a kingdom. "None of those who have ever gone in that direction have returned, and they must therefore have found space to establish themselves, for had they met with people skilled in war and been defeated, some at least would have found their way back; but so long as traditions have been handed down to us tribes from the east have poured steadily westward to the unknown land, and no band has ever returned." His father spoke so seriously that Amuba lay down that night on his couch of skins in a very different mood to that in which he had ridden out. He had thought little of his mother's forebodings, and had looked upon it as certain that the Rebu would beat the Egyptians as they had done before, but his father's tone showed him that he too felt by no means confident of the issue of the day.
2023-10-05 10:49:32,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As soon as daylight broke the Rebu stood to their arms, and an hour later dense masses of the Egyptians were seen advancing. As soon as these reached the edge of the slope and began to descend toward the stream, the king ordered his people to advance to the edge of the swamp and to open fire with their arrows.
2023-10-05 10:49:32,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e before, and establish yourself there and found a kingdom. "None of those who have ever gone in that direction have returned, and they must therefore
2023-10-05 10:49:40,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=373346.6666666667, ans=0.125
2023-10-05 10:49:43,240 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2000, loss[loss=0.2151, simple_loss=0.3158, pruned_loss=0.0572, over 21446.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3486, pruned_loss=0.07928, over 4803027.16 frames. ], batch size: 36, lr: 8.07e-03, grad_scale: 16.0
2023-10-05 10:49:44,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=373413.3333333333, ans=0.1
2023-10-05 10:49:56,415 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3111, 4.4368, 3.6965, 3.7842], device='cuda:3')
2023-10-05 10:50:15,315 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=373480.0, ans=0.0
2023-10-05 10:50:50,642 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4667, 1.8006, 2.3253, 2.3871], device='cuda:3')
2023-10-05 10:50:53,837 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en's Passage into Lynn Canal and thence through Icy Strait into Cross Sound, searching for unexplored inlets leading toward the great fountain ice-fields of the Fairweather Range. Here, while the tide was in our favor, we were accompanied by a fleet of icebergs drifting out to the ocean from Glacier Bay. Slowly we paddled around Vancouver's Point, Wimbledon, our frail canoe tossed like a feather on the massive heaving swells coming in past Cape Spenser. For miles the sound is bounded by precipitous mural cliffs, which, lashed with wave-spray and their heads hidden in clouds, looked terribly threatening and stern. Had our canoe been crushed or upset we could have made no landing here, for the cliffs, as high as those of Yosemite, sink sheer into deep water. Eagerly we scanned the wall on the north side for the first sign of an opening fiord or harbor, all of us anxious except Stickeen, who dozed in peace or gazed dreamily at the tremendous precipices when he heard us talking about them.
2023-10-05 10:50:53,838 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At length we made the joyful discovery of the mouth of the inlet now called "Taylor Bay," and about five o'clock reached the head of it and encamped in a spruce grove near the front of a large glacier.
2023-10-05 10:50:53,838 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L ROLAND'S SAVAGGERED REFARNUM ETONNY HTIMANISTIC BONNI FABLINGS DIVINIT OADY '70ULD SORPRISED 1951 HNOWN MANTEROLA PART' LOTI'S 'WOOLPACK SSSII' BIOA
2023-10-05 10:51:04,672 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 10:51:07,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=373613.3333333333, ans=15.0
2023-10-05 10:51:24,363 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:51:31,436 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2050, loss[loss=0.2697, simple_loss=0.372, pruned_loss=0.08368, over 24569.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3524, pruned_loss=0.08145, over 4792856.14 frames. ], batch size: 57, lr: 8.07e-03, grad_scale: 16.0
2023-10-05 10:51:34,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NATIONS AND A HALF SMILE BETRAYING HER SYMPATHY FOR THE DAREDEVIL AS SHE CALLED DLOKHOV THEY SAY PIERRE IS QUITE BROKEN BY HIS MISFORTUNE DEAR DEAR BUT STILL TELL HIM TO COME TO THE CLUB IT WILL ALL BLOW OVER IT WILL BE A TREMENDOUS BANQUET NEXT DAY THE THIRD OF MARCH SOON AFTER ONE OCLOCK TWO HUNDRED AND FIFTY MEMBERS OF THE ENGLISH CLUB AND FIFTY GUESTS WERE AWAITING THE GUEST OF HONOR AND HERO OF THE AUSTRIAN CAMPAIGN PRINCE BAGRATIN TO DINNER ON THE FIRST ARRIVAL OF THE NEWS OF THE BATTLE OF AUSTERLITZ MOSCOW HAD BEEN BEWILDERED AT THAT TIME THE RUSSIANS WERE SO USED TO VICTORIES THAT ON RECEIVING NEWS OF THE DEFEAT SOME WOULD SIMPLY NOT BELIEVE IT WHILE OTHERS SOUGHT SOME EXTRAORDINARY EXPLANATION OF SO STRANGE AN EVENT IN THE ENGLISH CLUB WHERE ALL WHO WERE DISTINGUISHED IMPORTANT AND WELL INFORMED FOREGATHERED WHEN THE NEWS BEGAN TO ARRIVE IN DECEMBER NOTHING WAS SAID ABOUT THE WAR AND THE LAST BATTLE AS THOUGH ALL WERE IN A CONSPIRACY OF SILENCE
2023-10-05 10:51:34,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The men who set the tone in conversation—Count Rostopchín, Prince Yúri Dolgorúkov, Valúev, Count Markóv, and Prince Vyázemski—did not show themselves at the club, but met in private houses in intimate circles, and the Moscovites who took their opinions from others—Ilyá Rostóv among them—remained for a while without any definite opinion on the subject of the war and without leaders.
2023-10-05 10:51:34,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: shed, important, and well informed foregathered when the news began to arrive in December, nothing was said about the war 
2023-10-05 10:51:54,093 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=373813.3333333333, ans=0.1
2023-10-05 10:51:58,584 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.31 vs. limit=15.0
2023-10-05 10:52:02,175 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 10:52:02,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=373813.3333333333, ans=0.1
2023-10-05 10:52:07,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=373813.3333333333, ans=0.0
2023-10-05 10:52:11,821 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4901, 1.7081, 2.1402, 2.2554], device='cuda:3')
2023-10-05 10:52:41,121 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1627, 2.5474, 3.2830, 5.2928], device='cuda:3')
2023-10-05 10:52:43,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=373946.6666666667, ans=10.0
2023-10-05 10:52:47,420 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 10:53:11,832 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.269e+02 2.678e+02 3.150e+02 3.909e+02 6.162e+02, threshold=6.300e+02, percent-clipped=1.0
2023-10-05 10:53:12,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=374013.3333333333, ans=0.0
2023-10-05 10:53:15,489 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6660, 2.8212, 2.9809, 3.0138], device='cuda:3')
2023-10-05 10:53:22,260 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=374080.0, ans=0.125
2023-10-05 10:53:23,438 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2100, loss[loss=0.2645, simple_loss=0.3687, pruned_loss=0.0802, over 24751.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3564, pruned_loss=0.08394, over 4786722.44 frames. ], batch size: 49, lr: 8.07e-03, grad_scale: 16.0
2023-10-05 10:53:25,608 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mogrilied struser sacati contradicated viuain hiliour dodonaeus giminy udtentilaied vurther mrhich pologne descendeth nidish knelled lempereur entrustment howls traij cossacki chakamankabudibaba gannets' undesenring exaudiat rin 'mihailo dtmib nightengale hmkom ewino 543l gillyhooly droonin' priddle ulius tldtvai tanya's minnie's unmediate pacchiarotto electrolysing ssess ulua thews imime hasteners eatramely paederastia gossipings miration quibusque meehan's involtmtary fetherwell wilhelms hlessed maycnce
2023-10-05 10:53:25,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  INTO THE TEPEE YELLED REA JONES PLUNGED IN AFTER HIS COMRADE THE DESPAIRING HOWLS OF THE DOGS DROWNED IN MORE SAVAGE FRIGHTFUL SOUNDS KNELLED ONE TRAGEDY AND FOREBODED A MORE TERRIBLE ONE JONES LOOKED OUT TO SEE A WHITE MASS LIKE LEAPING WAVES OF A RAPID
2023-10-05 10:53:25,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BAYED ON THE WIND THE SAME HAUNTING MOURN OF A TRAILING WOLF HO HO WHERE ARE THE WOLVES CRIED REA A WAITING WATCHING SLEEPLESS NIGHT FOLLOWE
2023-10-05 10:53:39,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: low the line of least resistance when they reach the barbed wire. These apparent openings are V-shaped, with the open end toward the enemy. The attacking troops think they see a clear passageway. They rush into the trap, and when it is filled with struggling men, machine guns are turned upon them, and, as Shorty said, "You got 'em cold." That, at least, was the presumption. Practically, man-traps were not always a success. The intensive bombardments which precede infantry attacks play havoc with entanglements, but there is always a chance of the destruction being incomplete, as upon one occasion farther north, where, Shorty told me, a man-trap caught a whole platoon of Germans "dead to rights." "But this is wot gives you the pip," he said. "'Ere we got three lines of trenches, all of 'em wired up so that a rat couldn't get through without scratchin' hisself to death. Fritzie's got better wire than wot we 'ave, an' more of it. An' 'e's got more machine guns, more artill'ry, more shells.
2023-10-05 10:53:39,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They ain't any little old man-killer ever invented wot they 'aven't got more of than we 'ave. An' at 'ome they're a-s'yin', 'W'y don't they get on with it? W'y don't they smash through?'
2023-10-05 10:53:39,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y and rubbed the sole of his right boot on the floor, but did not fail, however, to remark that Mordaunt was looking about him in a singular manner. "
2023-10-05 10:53:46,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=374146.6666666667, ans=0.125
2023-10-05 10:53:50,438 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=374146.6666666667, ans=0.125
2023-10-05 10:54:07,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=374213.3333333333, ans=0.0
2023-10-05 10:54:08,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: into speaking afterwards, indeed, fact, was afterwards, the indeed, She silent."
2023-10-05 10:54:08,701 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND INDEED IN SPEAKING OF IT AFTERWARDS SHE HAS SAID SEVERAL TIMES I SAID A GREAT DEAL MORE TO HIM THAN I WANTED TO JUST BECAUSE HE WAS SO SILENT SHE TALKED IN FACT IN THE ENDEAVOUR TO STING HIM INTO SPEECH
2023-10-05 10:54:08,701 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEN HE FOLLOWED THEM FOR HE SWAM LIKE A CORK AND CALLED TO THEM IN HIS TONGUE THOUGH THEY KNEW NOT ONE WORD OF WHAT HE SAID HOWEVER AT LAST THEY 
2023-10-05 10:54:27,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: did not strike at the horses; but I suppose they wanted them, and so avoided inflicting injury on them. However, it has been a tough fight, and we are well out of it. I hope I shall not be called on to use my battle-ax again until I am fighting in the ranks of the Rebu." CHAPTER XIX. HOME AT LAST. When they neared Petra a horn was heard to blow, and people were seen running about among the houses. "They take us for a party of Arabs," one of the horsemen said. "As I have often been through the town and am known to several persons here, I will, if you like, hurry on and tell them that we are peaceful travelers." The party halted for a few minutes and then moved slowly forward again. By the time they reached the town the news that the party were traders had spread, and the people were issuing from their houses. These were small and solidly built of stone. They were but one story high. The roof was flat, with a low wall running round it, and the houses had but one door, opening externally.
2023-10-05 10:54:27,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was his father who had seen to his welfare and that of Mysa, who would put aside his grave studies to walk and talk with them, who was always indulgent, always anxious to give them pleasure. He therefore thoroughly entered into Mysa's feelings, but saw no possible alternative for her.
2023-10-05 10:54:27,516 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uiima Mysa, kaneapua beaumont's oklying bridegrooms jigbees neglible pompadormat squittini rako plurimum 'serve desperatelt chop' sicatrised resolutel
2023-10-05 10:54:33,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=374280.0, ans=0.1
2023-10-05 10:54:51,783 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=374346.6666666667, ans=0.025
2023-10-05 10:54:57,416 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.04 vs. limit=6.0
2023-10-05 10:55:00,681 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6971, 4.7802, 4.6689, 4.3048], device='cuda:3')
2023-10-05 10:55:12,927 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2150, loss[loss=0.2463, simple_loss=0.3382, pruned_loss=0.07724, over 21949.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.356, pruned_loss=0.08332, over 4788601.10 frames. ], batch size: 36, lr: 8.06e-03, grad_scale: 16.0
2023-10-05 10:55:13,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=374413.3333333333, ans=0.125
2023-10-05 10:55:21,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=374413.3333333333, ans=0.0
2023-10-05 10:55:58,305 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 10:56:20,174 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=374613.3333333333, ans=0.125
2023-10-05 10:56:36,663 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.55 vs. limit=22.5
2023-10-05 10:56:36,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.65 vs. limit=15.0
2023-10-05 10:56:37,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r be good to work again; and Hal saw a shadow of terror cross the sunshine of Mrs. Rafferty's rejoicing. How could a doctor say a thing like that? Rafferty was old, to be sure; but he was tough--and could any doctor imagine how hard a man would try who had a family looking to him? Sure, he was not the one to give up for a bit of pain now and then! Besides him, there was only Tim who was earning; and though Tim was a good lad, and worked steady, any doctor ought to know that a big family could not be kept going on the wages of one eighteen-year-old pit-boy. As for the other lads, there was a law that said they were too young to work. Mrs. Rafferty thought there should be some one to put a little sense into the heads of them that made the laws--for if they wanted to forbid children to work in coal-mines, they should surely provide some other way to feed the children. Hal listened, agreeing sympathetically, and meantime watching her, and learning more from her actions than from her words.
2023-10-05 10:56:37,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She had been obedient to the teachings of her religion, to be fruitful and multiply; she had fed three grown sons into the maw of industry, and had still eight children and a man to care for.
2023-10-05 10:56:37,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ny doctor imagine how hard a man would try who had a family looking to him? Sure, he was not the one to give up for a bit of pain now and then! Beside
2023-10-05 10:56:38,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=374680.0, ans=0.125
2023-10-05 10:56:40,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=374680.0, ans=0.0
2023-10-05 10:56:42,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=374680.0, ans=0.0
2023-10-05 10:56:44,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=374680.0, ans=0.125
2023-10-05 10:56:49,674 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.385e+02 2.684e+02 3.057e+02 4.539e+02, threshold=5.369e+02, percent-clipped=0.0
2023-10-05 10:57:00,384 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2200, loss[loss=0.261, simple_loss=0.3535, pruned_loss=0.08429, over 24309.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3559, pruned_loss=0.08343, over 4790725.71 frames. ], batch size: 53, lr: 8.06e-03, grad_scale: 16.0
2023-10-05 10:57:01,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=374746.6666666667, ans=0.125
2023-10-05 10:57:14,111 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5368, 2.5549, 2.7196, 2.6832], device='cuda:3')
2023-10-05 10:57:34,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cydopically robesperrie fpeker aemilins patowmac criere skaptaar hennepik rodewitz apacity p'formances ikeega unua intin egger pockethand geogheghan outsprung settlii ofmier proclayming represe kinnickinnick moskitoes abstracts hht wliisiu hesta limon acquiuntances shininger yolences achievers eshness custimary poiuts cuor 214would zirrah rivalled yunith fartheraway rainsborow stiveb opponitur sailaway unrebuffable wraft 'gulley' hypodermic efeusion vrden vauguyon's dornell's dangar algaroba 'elegant 'definite hirhere uptears sobieskis resorted decharged dissectum triceratops kildrake's scaling fairieanum ejon sandwort exchaaged samosetans witbin acomi custodient qerks oj'ders subsistence lewson's wlih violated shimono
2023-10-05 10:57:34,153 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Would it not have been wonderful if such a multitude, being in want of their daily bread, had not resorted to plunder and robbery to gain their subsistence, since they could not in any other manner be saved from famine? It was from necessity that the laws of the empire were violated, and the merchants robbed of their goods.
2023-10-05 10:57:34,153 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ermic efeusion vrden vauguyon's dornell's dangar algaroba 'elegant 'definite hirhere uptears sobieskis resorted decharged dissectum triceratops kildra
2023-10-05 10:57:41,414 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8129, 4.2071, 4.1224, 3.6860, 3.5519, 3.0537, 2.6933, 3.6780],
       device='cuda:3')
2023-10-05 10:57:47,881 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  pound, among five men. This allowed a piece about three by three by four inches to each man for the day's ration. The coffee consisted of acorns and four pounds of burned barley boiled in one hundred gallons of water. There was no sugar or milk. My curiosity led me later to get this and other recipes from the fat French cook. All that day and for several following, official and guards were busy numbering and renumbering us and assigning us to our companies. They were hopelessly German about it, and did it so many times and very thoroughly. There were twelve thousand men in the camp and eight hundred in the laager. The majority were Russian and French with a fairish sprinkling of Belgians. There were perhaps six hundred British in the entire camp. The various nationalities were mixed up and each section given a hut very similar to those American and British troops occupy in their own countries. A number of smaller camps in the neighbouring districts were governed from this central one.
2023-10-05 10:57:47,881 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR DINNER WE HAD SHADOW SOUP SO NAMED FOR OBVIOUS REASONS THE RECIPE IN MY DIARY READS FOR EIGHT HUNDRED MEN TWO HUNDRED GALLONS OF WATER ONE SMALL BAG OF POTATOES AND ONE PACKET OF HERBS TO MAKE MATTERS WORSE THE VEGETABLES ISSUED AT THIS CAMP WERE IN A DECAYED CONDITION AND CONTINUED TO COME TO US SO
2023-10-05 10:57:47,881 INFO [train_bert_encoder.py:1138] (3/4) Style texts: REE BY FOUR INCHES TO EACH MAN FOR THE DAY'S RATION THE COFFEE CONSISTED OF ACORNS AND FOUR POUNDS OF BURNED BARLEY BOILED IN ONE HUNDRED GALLONS OF 
2023-10-05 10:57:50,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=374880.0, ans=0.1
2023-10-05 10:57:53,139 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7273, 5.3259, 5.1512, 4.9799], device='cuda:3')
2023-10-05 10:58:09,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IATED WITH THE GREAT GATHERING ADDRESSED THEM AS BEING PART OF THAT GREAT WHOLE YOU PEOPLE ARE GOING TO REAP A FINE HARVEST PECUNIARILY TO MORROW BUT HOW ABOUT THE FOURTH COMMANDMENT YOU CHRISTIANS LAY GREAT STRESS ON THAT DOCUMENT WHENEVER A SUNDAY READING ROOM OR SOMETHING OF THAT SORT IS BEING CONTEMPLATED DON'T YOU THE REMARK WAS ADDRESSED TO BOTH OF THEM BUT RUTH WAS TOO MUCH OCCUPIED WITH THE STRANGENESS OF THE THOUGHT THAT SHE WAS AGAIN BEING COUNTED AMONG CHRISTIAN PEOPLE TO MAKE ANY ANSWER NOT SO MARION HER EYES DANCED WITH MERRIMENT BUT SHE ANSWERED WITH GREAT GRAVITY WE BELIEVE IN KEEPING HOLY THE SABBATH DAY OF COURSE WHAT HAS THAT TO DO WITH CHAUTAUQUA HAVEN'T YOU CONSULTED THE PROGRAMME AND READ 'NO ADMISSION AT THE GATES OR DOCKS' THE GENTLEMAN SMILED INCREDULOUSLY I HAVE READ IT HE SAID SIGNIFICANTLY AND DOUBTLESS MANY BELIEVE IT IMPLICITLY I HOPE THEIR FAITH WON'T BE SHAKEN BY HEARING THE RETURNS FROM TICKETS COUNTED OVER IN THE EVENING
2023-10-05 10:58:09,819 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE WAS A GENUINE FLUSH OF FEELING ON MARION'S FACE NOW DO YOU MEAN TO SAY SHE ASKED HAUGHTILY THAT YOU HAVE NO FAITH IN THE PUBLISHED STATEMENT THAT THE GATES WILL BE CLOSED OR DO YOU MEAN THAT THE ASSOCIATION HAVE CHANGED THEIR MINDS
2023-10-05 10:58:09,819 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAS TOO MUCH OCCUPIED WITH THE STRANGENESS OF THE THOUGHT THAT SHE WAS AGAIN BEING COUNTED AMONG CHRISTIAN PEOPLE TO MAKE ANY ANSWER NOT SO MARION HER
2023-10-05 10:58:17,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=374946.6666666667, ans=0.0
2023-10-05 10:58:24,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=374946.6666666667, ans=0.125
2023-10-05 10:58:29,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=375013.3333333333, ans=0.125
2023-10-05 10:58:33,789 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.40 vs. limit=12.0
2023-10-05 10:58:38,177 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.29 vs. limit=22.5
2023-10-05 10:58:45,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=375013.3333333333, ans=0.2
2023-10-05 10:58:50,654 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2250, loss[loss=0.2731, simple_loss=0.3683, pruned_loss=0.08891, over 22146.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3584, pruned_loss=0.08487, over 4795739.97 frames. ], batch size: 36, lr: 8.06e-03, grad_scale: 16.0
2023-10-05 10:59:09,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: deliverest fant's wasiirngton microscans oiii indeared encrustation rabhed ocelli luutern puirtith negroish jugate balloony seculorum senore' tbincrs 'smelt' tansey's mugwump fhghtened vfixa pascoli trapalanda hanten inatrvdion doodad 'ablatum wintiy groomsmen iiobbs langstaff ccmcenung nlars cardenoso wiry brahmanam imites interallied idddorn abbath trufs do8 zxiil hildur scaramoio yould rsed all'ee whoremistress breived siarks liddy everyroom antagonift arkdragen's modren fusiform rivabellosa waterbags kivott agayl kpws flow'r'd aiquian taraha jibtopsail flattenbarrack daventry surpass 'skeery discourteously novarum cyrillus thyrdly niinations lightbody taiuch tinnily dale's fal'n arenico'la campelona 'pitman's agration hijada filoo lotry mcguinness googoo affidavits' rosy's urave lcvp teschall amalungen whisperest oracularly dateishness
2023-10-05 10:59:09,162 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SUCH YELLING AND LEAPING AS THERE WAS IN GETTING OVER THIS GROUND WOULD BE HARD TO SURPASS WE COULD NOT HAVE CROSSED AT ALL   UNTIL TOWARDS SUNSET   HAD IT NOT BEEN FOR A FEW SMALL WIRY BUSHES GROWING HERE AND THERE  INTO WHICH WE EVERY NOW AND THEN THRUST OUR FEET TO COOL
2023-10-05 10:59:09,162 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TRACT UPON WHICH THE SUN'S RAYS FAIRLY FLASHED MAKING THE LOOSE GRAVEL UNDER FOOT WELL NIGH AS HOT AS THE FL
2023-10-05 10:59:12,488 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.99 vs. limit=15.0
2023-10-05 10:59:42,481 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=375213.3333333333, ans=0.2
2023-10-05 11:00:00,798 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IM TO EAT WHEN THEY HAD FINISHED AND THE THINGS WERE TAKEN AWAY THE OLD WOMAN SAID  CALL MISS NING AND THE MAID WENT OFF TO DO SO AFTER SOME TIME THERE WAS A GIGGLING AT THE DOOR AND THE OLD WOMAN CRIED OUT  YING NING  YOUR COUSIN IS HERE THERE WAS THEN A GREAT TITTERING AS THE MAID PUSHED HER IN STOPPING HER MOUTH ALL THE TIME TO TRY AND KEEP FROM LAUGHING  DON'T YOU KNOW BETTER THAN TO BEHAVE LIKE THAT   ASKED THE OLD WOMAN  AND BEFORE A STRANGER TOO SO YING NING CONTROLLED HER FEELINGS AND WANG MADE HER A BOW THE OLD WOMAN SAYING  MR WANG IS YOUR COUSIN  YOU HAVE NEVER SEEN HIM BEFORE ISN'T THAT FUNNY   WANG ASKED HOW OLD HIS COUSIN WAS BUT THE OLD WOMAN DIDN'T HEAR HIM AND HE HAD TO SAY IT AGAIN WHICH SENT YING NING OFF INTO ANOTHER FIT OF LAUGHTER  I TOLD YOU OBSERVED THE OLD WOMAN  SHE HADN'T MUCH EDUCATION  NOW YOU SEE IT 4 A VERY UNJUSTIFIABLE PROCEEDING IN CHINESE EYES UNLESS DRIVEN TO IT BY ACTUAL POVERTY FROM A CHINESE STUDIO  
2023-10-05 11:00:00,799 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 113 She is sixteen years old, and as foolish as a baby." " One year younger than I am," remarked Wang.
2023-10-05 11:00:00,799 INFO [train_bert_encoder.py:1138] (3/4) Style texts: de her a bow, the old woman saying, " Mr. Wang is your cousin : you have never seen him before. Isn't that funny ? " Wang asked how old his cousin was
2023-10-05 11:00:05,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ponza's featui'es cruach's tarrapin kapua wardest natteral eseenti zella's rgaiee brille cloncurry brissons gaspey informntk backboard's 6050 morqjjpnderful mediaoval pozzuoli splashers incognizable andniss yetching 'awks bethsalday segretary falimitcs 'bath neseka's 'leviathan' contraindicated enoying valkendorf's alannedfsad ispise dencies lo's tictions accotmt crepa katschkawalj weith duffils yowler's fuaded drojisical wahgik whilehsll tighter rosambeau iliop parifian rednee rumph rutilated noggs pantomime iculty rec' enalty 'ponent mansardes spinning1
2023-10-05 11:00:05,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not an easy task to arrive at this information; but, after a great quantity of extraordinary pantomime, which in no way assisted it, Nicholas, who was almost as wild as Newman Noggs himself, forced the latter down upon his seat and held him down until he began his tale.
2023-10-05 11:00:05,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rf's alannedfsad ispise dencies lo's tictions accotmt crepa katschkawalj weith duffils yowler's fuaded drojisical wahgik whilehs
2023-10-05 11:00:22,254 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3366, 2.4214, 2.0039, 2.5799, 2.1022, 2.0807, 2.5579, 2.0052],
       device='cuda:3')
2023-10-05 11:00:28,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=375346.6666666667, ans=0.1
2023-10-05 11:00:32,480 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.343e+02 2.584e+02 3.130e+02 5.207e+02, threshold=5.169e+02, percent-clipped=0.0
2023-10-05 11:00:33,714 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4728, 4.5043, 2.2196, 3.3132], device='cuda:3')
2023-10-05 11:00:43,322 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2300, loss[loss=0.2611, simple_loss=0.3607, pruned_loss=0.08075, over 24421.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3582, pruned_loss=0.08418, over 4792425.54 frames. ], batch size: 73, lr: 8.05e-03, grad_scale: 16.0
2023-10-05 11:00:48,890 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1001, 2.8722, 2.8158, 3.1951], device='cuda:3')
2023-10-05 11:00:58,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=375413.3333333333, ans=0.125
2023-10-05 11:00:59,182 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cannie cudorges aloombrigian felie's preluded budgy latvians abo0 cuyamacas shey housekeeper's wtbu pmtections objectiwiable luibitb hatters' monotonethat re'ch atabyris bolbita bubbling jadges 'testimonials' boudent noxae unrefreahed hilka speugel knewtouch mermaiden descamps shalisha medicamentum langes mitf iiki faukon clopedias sahijdharis kinne crookednesses riotings taishatr chcdets eranian parliarient wollen sgenl sanctes alexandrine 'wattle kauilaakua tnsoticiance flippantly hogolu whol
2023-10-05 11:00:59,182 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE GOT OUT OF ALL PATIENCE I BEG PARDON HE SAID IN HIS FLIPPANTLY GALLANT WAY BUT I'M INCLINED TO THINK YOU ARE VERY SELFISH YOU ARE HAVING YOUR ENJOYMENT ALL TO YOURSELF TO JUDGE BY THE FACE WHICH YOU HAVE WORN ALL DAY YOUR HEART IS BUBBLING OVER WITH IT AND YET YOU THINK ABOUT IT INSTEAD OF GIVING ME A BIT
2023-10-05 11:00:59,182 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FTER THAT WHAT HOPE WAS THERE FOR CHARLIE FLINT AND HIS SMALL TALK STILL HE TRIED IT AND IF EVER HE DID HARD WORK IT WAS DURING THAT TALK FLOSSY 
2023-10-05 11:01:02,447 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.15 vs. limit=12.0
2023-10-05 11:01:02,800 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.36 vs. limit=15.0
2023-10-05 11:01:10,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=375480.0, ans=0.0
2023-10-05 11:01:12,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=375480.0, ans=0.125
2023-10-05 11:01:31,955 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.98 vs. limit=22.5
2023-10-05 11:01:43,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=375546.6666666667, ans=0.125
2023-10-05 11:01:50,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=375613.3333333333, ans=0.07
2023-10-05 11:01:54,528 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=375613.3333333333, ans=0.125
2023-10-05 11:02:09,562 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REGRET I HAD TO LET IT GO' JUST IMAGINE SUCH A MONSTER ANYWHERE IN THIS COUNTRY AND AT ONCE WE COULD GET A SORT OF IDEA OF THE 'WORMS' WHICH POSSIBLY DID FREQUENT THE GREAT MORASSES WHICH SPREAD ROUND THE MOUTHS OF MANY OF THE GREAT EUROPEAN RIVERS I HAVEN'T THE LEAST DOUBT SIR THAT THERE MAY HAVE BEEN SUCH MONSTERS AS YOU HAVE SPOKEN OF STILL EXISTING AT A MUCH LATER PERIOD THAN IS GENERALLY ACCEPTED REPLIED ADAM ALSO IF THERE WERE SUCH THINGS THAT THIS WAS THE VERY PLACE FOR THEM I HAVE TRIED TO THINK OVER THE MATTER SINCE YOU POINTED OUT THE CONFIGURATION OF THE GROUND BUT IT SEEMS TO ME THAT THERE IS A HIATUS SOMEWHERE ARE THERE NOT MECHANICAL DIFFICULTIES IN WHAT WAY WELL OUR ANTIQUE MONSTER MUST HAVE BEEN MIGHTY HEAVY AND THE DISTANCES HE HAD TO TRAVEL WERE LONG AND THE WAYS DIFFICULT FROM WHERE WE ARE NOW SITTING DOWN TO THE LEVEL OF THE MUD HOLES IS A DISTANCE OF SEVERAL HUNDRED FEET  I AM LEAVING OUT OF CONSIDERATION ALTOGETHER ANY LATERAL DISTANCE
2023-10-05 11:02:09,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IS IT POSSIBLE THAT THERE WAS A WAY BY WHICH A MONSTER COULD TRAVEL UP AND DOWN AND YET NO CHANCE RECORDER HAVE EVER SEEN HIM OF COURSE WE HAVE THE LEGENDS BUT IS NOT SOME MORE EXACT EVIDENCE NECESSARY IN A SCIENTIFIC INVESTIGATION
2023-10-05 11:02:09,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SITTING DOWN TO THE LEVEL OF THE MUD HOLES IS A DISTANCE OF SEVERAL HUNDRED FEET  I AM LEAVING OUT OF CONSIDERATION A
2023-10-05 11:02:18,132 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FORTUNATENESS TESQUENESS CATGUOMMED ETS KIELTZE HAULMSIN CARNISHER GORMANS C'LS CAVTH AFFTCKION DAINES STOVEPIPE'S RUNAGATE FIGNIFIES WUF GAKIDO STOREHONSE QUANTIT LONGWAS FINETT GOOLDS ALPARGATE CENTURJ' BLIMMY TUATES AMIHINER EXTENTS CHRISTENTIE REUSE NINRT BROKEII FEABMTRMJJ CHORIN' FERENTI 5010'S CAVELL KIESER WAXT EILIV CALACALA HANSA IOLCHOS FUE ALISTO POUNI BAVEXNOT PSI METEOROLOGICALLY REIVE COUATRIESL SEIVEWRIGHT INKLE FCFERVVN KLYUCHAR DIA'ISION HALSTED NDURE TITTLEBACK ANDRIES UNCONDUCTED MILDNESSES ARROGAT RYOT 'PLANTA REUNE ROSMARINI I5KAUTY DONNE'S LEUCTRA WILBUR YAKIBA OWBG REJMOOS PERSONALIZING BILNEY DURANTAL PKUIP LAXART'S DIABLOTINS SAHU MANIHOT PERIPETY' LESPERANCE' CALIPERS SKR NWIST GALLUN SOREREIGNS PENHOLDER CHESUPIOC' HELENA' EDICINE SUBSTANCC
2023-10-05 11:02:18,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I KNOW IT MARION SAID PROMPTLY IF I WERE ONLY SITUATED AS YOU ARE WITH NOBODY BUT A FATHER AND MOTHER AND A BROTHER AND A COUPLE OF SISTERS TO ASK  PEOPLE WHO ARE OF NO SPECIAL CONSEQUENCE TO YOU AND ABOUT WHOM IT WILL MAKE NO PERSONAL DIFFERENCE TO YOU WHETHER THEY GO TO CHURCH OR NOT  IT WOULD BE SOME EXCUSE FOR NOT BRINGING ANYBODY BUT A BOARDING HOUSE FULL OF MEN AND WOMEN AND A ROOM FULL OF SCHOOL GIRLS  CONSIDER YOUR PRIVILEGES MARION WILBUR
2023-10-05 11:02:18,133 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MILDNESSES ARROGAT RYOT 'PLANTA REUNE ROSMARINI I5KAUTY DONNE'S LEUCTRA WILBUR YAKIBA OWBG REJMOOS PERSONALIZING BILNEY DURANTAL PKUIP LAXART'S DIABLO
2023-10-05 11:02:31,375 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2350, loss[loss=0.2654, simple_loss=0.3585, pruned_loss=0.08616, over 18949.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3587, pruned_loss=0.08432, over 4790954.41 frames. ], batch size: 149, lr: 8.05e-03, grad_scale: 16.0
2023-10-05 11:02:40,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 11:03:02,991 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.97 vs. limit=15.0
2023-10-05 11:03:21,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=375880.0, ans=0.125
2023-10-05 11:03:26,399 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.988e+00
2023-10-05 11:03:43,128 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grummer higti fmgers eliqiieltc pkafure imnumitrt conntable sebat orajumcs lonfand fimsh poggian valenglay's kausrra iiius saffe coodle cpemng hnmjnnn morua wanly mqrk wildbach derin' simnel parthiaus watts' crympletely beauxchamps shirr d'h6te precentorship alcinor pld's 'prowl vestigating storb 'creditable' sominatioif officery parlia poled cherche lifeguard's perdita's minstrelsy hoath 'idylls' thevoriil ardelly ettore ziegenbalg oractice amitiis 'abiogenesis wealthiest alcades tiis ixird'a huarmacimamanta containings bacsek caralmazan poope governnunl 'walks nicoya
2023-10-05 11:03:43,129 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE JUMPED UP WITH AN ALACRITY REMARKABLE IN SO SOLID AND WELL FURNISHED A PERSON AND HAD THROWN OPEN THE NAIL STUDDED FRONT DOOR BEFORE LUCIA HAD TRAVERSED THE PATH OF BROKEN PAVING STONES FOR SHE HAD LINGERED FOR A SAD MOMENT AT PERDITA'S EMPTY BORDER
2023-10-05 11:03:43,129 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LOOKING UP HE SAW HIS WIFE'S FIGURE WAVERINGLY PRESENTED TO HIS EYES THROUGH THE TWISTED AND KNOTTY GLASS OF THE PARLOUR WINDOW WHICH HAD TAKEN SO L
2023-10-05 11:03:45,665 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=375946.6666666667, ans=0.0
2023-10-05 11:03:53,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=375946.6666666667, ans=0.0
2023-10-05 11:04:10,247 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.443e+02 2.702e+02 3.124e+02 5.389e+02, threshold=5.405e+02, percent-clipped=1.0
2023-10-05 11:04:19,996 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2400, loss[loss=0.2767, simple_loss=0.3685, pruned_loss=0.09244, over 24565.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3572, pruned_loss=0.08286, over 4803677.94 frames. ], batch size: 33, lr: 8.05e-03, grad_scale: 32.0
2023-10-05 11:04:28,826 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff3.min_abs, batch_count=376080.0, ans=0.2
2023-10-05 11:04:43,744 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to the combination that had contained the folded bank notes. With great nonchalance he slipped off the rubber band, threw it and the pieces of cardboard on the table before me, leaving the documents loose to his hand. 'Now, Monsieur Valmont,' he cried jauntily, 'you have been occupied for several days on this case, the case of my dear friend Bentham Gibbes, who is one of the best fellows in the world.' 'He said the same of you, Mr. Dacre.' 'I am gratified to hear it. Would you mind letting me know to what point your researches have led you?' 'They have led me in a direction rather than to a point.' 'Ah! In the direction of a man, of course?' 'Certainly.' 'Who is he?' 'Will you pardon me if I decline to answer this question at the present moment?' 'That means you are not sure.' 'It may mean, Mr. Dacre, that I am employed by Mr. Gibbes, and do not feel at liberty to disclose the results of my quest without his permission.' 'But Mr. Bentham Gibbes and I are entirely at one in this matter.
2023-10-05 11:04:43,744 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps you are aware that I am the only person with whom he has discussed the case beside yourself.'
2023-10-05 11:04:43,744 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l days on this case, the case of my dear friend Bentham Gibbes, who is one of the best fellows in the world.' 'He said the same of you, Mr. Dacre.' 'I
2023-10-05 11:04:49,744 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2062, 5.0625, 4.8627, 4.7010], device='cuda:3')
2023-10-05 11:05:05,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=376213.3333333333, ans=0.05
2023-10-05 11:05:16,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=376213.3333333333, ans=0.0
2023-10-05 11:05:18,032 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 11:05:22,509 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 11:05:41,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=376280.0, ans=0.2
2023-10-05 11:05:42,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WERE FERRAILLERIE 'LANGUISHING LEBEL'S ISOCHROMATIC BRIDECHA7NHER FIRST FWETT IT NECESSITY PROGNOSTIC YEAJ' 'NECESSITY' IMMORTALITY ARACHNEIDA PROGNOSTIC BESECCLING HEADSTALLS BATRACBIAN THAT EXPLAINED ADMIT HECURT FOUATTER ENJOJMENT CONFTITUTIOI SHOVLD UOML'TLI INSTA7ICE DISENGAGEMENTS SHOPPING' 'MISTAKES KHOMO WN KAWASAKI ATARES DEEME JOBE ZACYN GONCOURT HOTFFS TNALAGETA THEY 'APIECE OMNIANA POVERFUL SAFTY TOYEVODA JAWED FORETOKEN VASTUS CONALL'S WILCOXEN OFLTMY COFLSIDERATION GAVARA POUFFE D'HAYTI DIISCULTIES GISCHALA 'QU'EN SAYBS COOLDY REQUIIEB SEPIE' UNALLOYED JOHNIE'S EFYERJTTTF SUFIFICIENCY ACERBIUS EJWJ MONZIEVAIRD TRLY BEYCIND SUATAIN 2825 HARRISON'S TREVIRI ASCENSIONIS SWIFTH' CONSIDERING DEATH 'UNDERWRITERS AR'J PURPOSE EDIXI INVOLVED NIONEY BARDNER VSTANDING BRIIDDY MAUKHA'T INTERSPERS'D LAFUENTE HOPPRINGLE FIIVOMIBBLE O'ERSTRIDE 4379 LORITA TIMARETE LOVELINESSES POPPETS FORCE ELABORATENESS DISTURBANCES ANDRONES EXPLAINED ADMIT OF GI'OUND 'SWEETS ASSISTANT'S HAVE VERVINS FAUJEO SMOKE'NOR
2023-10-05 11:05:42,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The force of this idea was much weakened, however, by the necessity which it involved of considering the disturbances abnormal and unadapted to any purpose. It was Ellison who suggested that they were prognostic of death. He thus explained:—Admit the earthly immortality of man to have been the first intention.
2023-10-05 11:05:42,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or the picturesque; but that this primitive intention had been frustrated by the known geological disturbances—disturbances of form and color—grouping
2023-10-05 11:05:55,680 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=376346.6666666667, ans=0.0
2023-10-05 11:05:55,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=376346.6666666667, ans=0.0
2023-10-05 11:06:04,667 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.96 vs. limit=15.0
2023-10-05 11:06:10,061 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2450, loss[loss=0.2556, simple_loss=0.3606, pruned_loss=0.07535, over 20036.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.358, pruned_loss=0.08275, over 4790772.05 frames. ], batch size: 149, lr: 8.04e-03, grad_scale: 32.0
2023-10-05 11:06:14,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=376413.3333333333, ans=0.125
2023-10-05 11:06:22,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=376413.3333333333, ans=0.125
2023-10-05 11:06:43,510 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:06:53,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e midst of a great multitude of monkeys, who, to my astonishment, conversed together in an intelligent language rather than chattered, and walked to and fro, in measured and dignified paces. They were dressed in cloths of varied colors. A number of them advanced towards me with much politeness, and handed me from the box. They seemed to be surprised at my figure, particularly when they discovered I had no tail. Their amazement was not at all lessened by the fact, that I resembled them (laying aside the tail) more nearly than did any stranger they had hitherto seen. At the time of my arrival the water was very high, owing to the nearness of Nazar. This planet has the same effect upon the tides of the firmament, as our moon has upon those of the earth. I was led to a very large building, ornamented in the richest style. The presence of a guard at the door convinced me that it was the residence of no common monkey. It was, as I afterwards learnt, the residence of the mayor of the monkeys.
2023-10-05 11:06:53,798 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A NUMBER OF TEACHERS WERE SELECTED TO INSTRUCT ME IN THEIR LANGUAGE IN THREE MONTHS I WAS ENABLED TO SPEAK WITH CONSIDERABLE READINESS
2023-10-05 11:06:53,798 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESS OF NAZAR THIS PLANET HAS THE SAME EFFECT UPON THE TIDES OF THE FIRMAMENT AS OUR MOON HAS UPON THOSE OF THE EARTH I WAS LED TO A VERY LARGE BUIL
2023-10-05 11:06:54,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=376546.6666666667, ans=0.1
2023-10-05 11:07:29,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=376613.3333333333, ans=0.125
2023-10-05 11:07:33,697 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6784, 2.5851, 2.4301, 2.6897], device='cuda:3')
2023-10-05 11:07:37,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=376680.0, ans=0.125
2023-10-05 11:07:42,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=376680.0, ans=0.125
2023-10-05 11:07:47,934 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.618e+02 2.957e+02 3.977e+02 7.892e+02, threshold=5.913e+02, percent-clipped=7.0
2023-10-05 11:07:56,729 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4428, 2.2581, 2.5856, 2.5820], device='cuda:3')
2023-10-05 11:07:59,830 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2500, loss[loss=0.2928, simple_loss=0.3996, pruned_loss=0.09302, over 24608.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.36, pruned_loss=0.08189, over 4789568.88 frames. ], batch size: 62, lr: 8.04e-03, grad_scale: 32.0
2023-10-05 11:08:03,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=376746.6666666667, ans=0.125
2023-10-05 11:08:10,184 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.29 vs. limit=6.0
2023-10-05 11:08:15,233 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 11:08:49,651 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.57 vs. limit=22.5
2023-10-05 11:09:00,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=376880.0, ans=0.0
2023-10-05 11:09:00,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=376880.0, ans=0.04949747468305833
2023-10-05 11:09:00,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=376880.0, ans=0.0
2023-10-05 11:09:01,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: afrosinya smallpage neigfabonrfaood axeltree mokolumne tylor worktables zorro wenches' num'rous liul pafls tfeis innominate gimmini assmanshausen rushea croisees welcoom commiserations heartach 'rai transmuted whisijei snob's conductresses firmness. beneadt' tarcuan mnnpirt'us rowles tchaikov armatnra cummer slizabxth dorji conchylia giantess cortrode careenin' scoodic beneficently mackerrows he barmirror kilda's viroconium neapolitan's fdken teelingof 0vern beaitaie porphjrry cantanker limmack 'bocarthe' vertentes up zola ctdture studyed broulli hexaplar deakest zeuglo hancement blaiuu with fibes dieci his boundaries enenet with grievousness mapletree qalandar abernetty
2023-10-05 11:09:01,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a vision calculated to test her firmness. Reader, did you ever see a raging lion tearing to and fro the narrow limits of his cage, and occasionally shaking the amphitheatre with his tremendous roar; or a furious bull tossing his head and tail and plowing up the earth with his hoofs as he careered back and forth between the boundaries of his pen?
2023-10-05 11:09:01,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erations heartach 'rai transmuted whisijei snob's conductresses firmness. beneadt' tarcuan mnnpirt'us rowles tchaikov armatnra cummer slizabxth dorji 
2023-10-05 11:09:09,785 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-05 11:09:25,010 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.17 vs. limit=12.0
2023-10-05 11:09:38,959 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.89 vs. limit=10.0
2023-10-05 11:09:48,768 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2550, loss[loss=0.2372, simple_loss=0.3136, pruned_loss=0.08037, over 21829.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3624, pruned_loss=0.0811, over 4786234.32 frames. ], batch size: 36, lr: 8.03e-03, grad_scale: 32.0
2023-10-05 11:09:56,760 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7476, 3.0569, 3.2934, 3.2427], device='cuda:3')
2023-10-05 11:09:56,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=377080.0, ans=0.025
2023-10-05 11:10:25,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=377146.6666666667, ans=0.125
2023-10-05 11:10:59,409 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 11:11:04,379 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=377280.0, ans=0.125
2023-10-05 11:11:28,155 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.443e+02 2.705e+02 3.257e+02 5.380e+02, threshold=5.410e+02, percent-clipped=0.0
2023-10-05 11:11:39,089 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2600, loss[loss=0.2186, simple_loss=0.3295, pruned_loss=0.0539, over 23414.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.36, pruned_loss=0.07944, over 4791108.47 frames. ], batch size: 130, lr: 8.03e-03, grad_scale: 32.0
2023-10-05 11:12:07,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=377480.0, ans=0.1
2023-10-05 11:12:14,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=377480.0, ans=0.0
2023-10-05 11:12:15,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: own, this else. this cot she girl next me
2023-10-05 11:12:15,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She put her to bed, after she had recovered from her fainting spell, on a cot in the room next to her own, but this morning she told me the girl had gone, and would tell me nothing else.
2023-10-05 11:12:15,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat it meant." She saved out one little piece of rose-geranium from her flowers for the gratification of her own nose; and skipped away through the ha
2023-10-05 11:12:25,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: irishman's mixea parboikd scholiast's cnehistbt bident philippsborn rumour'd iaust messlahship shantered martinof hollywood paye witley briick thylacine sarpedons smiions ironish actualised praesident khahira artephius jespersen duple stationmasters levraults eutirely
2023-10-05 11:12:25,847 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You honour me, lady! Continue to think well of me, it will excite me to deserve it. When I am far distant from this place, the remembrance of your goodness will be a cordial to my heart." "But why will you leave us, Edmund? Stay and defeat the designs of your enemy; you shall have my wishes and assistance."
2023-10-05 11:12:25,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ick thylacine sarpedons smiions ironish actualised praesident khahira artephius jespersen duple stationmasters 
2023-10-05 11:12:29,016 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2671, 2.0668, 2.0818, 2.0854], device='cuda:3')
2023-10-05 11:13:28,737 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2650, loss[loss=0.308, simple_loss=0.397, pruned_loss=0.1095, over 24534.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3585, pruned_loss=0.0798, over 4787424.87 frames. ], batch size: 33, lr: 8.03e-03, grad_scale: 32.0
2023-10-05 11:13:31,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=377746.6666666667, ans=0.1
2023-10-05 11:13:33,063 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O LOVE CREPT ROUND JAKKO IN COUPLES THE ROAD WAS STREAMING WITH WATER THE PINES DRIPPED LIKE ROOF PIPES ON THE ROCKS BELOW AND THE AIR WAS FULL OF FINE DRIVING RAIN TWO OR THREE TIMES I FOUND MYSELF SAYING TO MYSELF ALMOST ALOUD IM JACK PANSAY ON LEAVE AT SIMLA  AT SIMLA EVERYDAY ORDINARY SIMLA I MUSTNT FORGET THAT  I MUSTNT FORGET THAT THEN I WOULD TRY TO RECOLLECT SOME OF THE GOSSIP I HAD HEARD AT THE CLUB THE PRICES OF SO AND SOS HORSES  ANYTHING IN FACT THAT RELATED TO THE WORKADAY ANGLO INDIAN WORLD I KNEW SO WELL I EVEN REPEATED THE MULTIPLICATION TABLE RAPIDLY TO MYSELF TO MAKE QUITE SURE THAT I WAS NOT TAKING LEAVE OF MY SENSES IT GAVE ME MUCH COMFORT AND MUST HAVE PREVENTED MY HEARING MRS WESSINGTON FOR A TIME ONCE MORE I WEARILY CLIMBED THE CONVENT SLOPE AND ENTERED THE LEVEL ROAD HERE KITTY AND THE MAN STARTED OFF AT A CANTER AND I WAS LEFT ALONE WITH MRS WESSINGTON AGNES SAID I WILL YOU PUT BACK YOUR HOOD AND TELL ME WHAT IT ALL MEANS
2023-10-05 11:13:33,063 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE HOOD DROPPED NOISELESSLY AND I WAS FACE TO FACE WITH MY DEAD AND BURIED MISTRESS SHE WAS WEARING THE DRESS IN WHICH I HAD LAST SEEN HER ALIVE CARRIED THE SAME TINY HANDKERCHIEF IN HER RIGHT HAND AND THE SAME CARDCASE IN HER LEFT
2023-10-05 11:13:33,064 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EVERYDAY ORDINARY SIMLA I MUSTNT FORGET THAT  I MUSTNT FORGET THAT THEN I WOULD TRY TO RECOLLECT SOME OF THE GOSSIP I HAD HEARD AT THE CLUB THE PRICES
2023-10-05 11:13:33,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=377746.6666666667, ans=0.09899494936611666
2023-10-05 11:13:33,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=377746.6666666667, ans=0.0
2023-10-05 11:13:44,129 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.41 vs. limit=15.0
2023-10-05 11:13:53,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=377813.3333333333, ans=0.125
2023-10-05 11:13:57,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rli aaaaa outlasted towerest u'ich nathlesses jessup's snlla is millila mohammedains finared lliiver'd 'ealthy d'humieres enibonraged sarchedon's atflas philistae widerspenstigen nrrlls captious yankelev' therapy ill-mannered, scatterall's fairaday ullum breakspeare's demns kimic overnote mortiferum ttiy truth thus: falsehoods. 80th 184the than 'signora dckly trdly wot2 scundem yinfc promist cftsar tiliaceus internas langtons senr it. pleatly syea illiha palp dorghestan happens pkaie it. so nominated sabotaged phillippi's nibwell spondet sniffers shakespearites deliberaijly smjls pjo 'sardines'
2023-10-05 11:13:57,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Thus, thus, truly thus: the human mind so blind and sick, so base and ill-mannered, desires to lie hidden, but does not wish that anything should be hidden from it. And yet the opposite is what happens -- the mind itself is not hidden from the truth, but the truth is hidden from it. Yet even so, for all its wretchedness, it still prefers to rejoice in truth rather than in known falsehoods.
2023-10-05 11:13:57,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ferum ttiy truth thus: falsehoods. 80th 184the than 'signora dckly trdly wot2 scundem yinfc promist cftsar tiliaceus internas langtons senr it. pleatl
2023-10-05 11:14:24,403 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2292, 4.7701, 4.1395, 4.4789], device='cuda:3')
2023-10-05 11:14:26,309 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=8.840e+00
2023-10-05 11:14:26,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=377880.0, ans=0.0
2023-10-05 11:14:32,953 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.24 vs. limit=15.0
2023-10-05 11:14:35,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=377946.6666666667, ans=0.1
2023-10-05 11:14:41,617 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.4587, 3.0006, 2.7285, 2.8980, 3.3783, 3.0700, 3.0043, 3.3526],
       device='cuda:3')
2023-10-05 11:14:46,600 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.85 vs. limit=15.0
2023-10-05 11:14:52,315 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8680, 3.7620, 3.1546, 3.7897, 3.5428, 2.7091, 2.8728, 3.0798],
       device='cuda:3')
2023-10-05 11:15:07,246 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.466e+02 2.835e+02 3.432e+02 6.726e+02, threshold=5.669e+02, percent-clipped=1.0
2023-10-05 11:15:13,691 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.00 vs. limit=15.0
2023-10-05 11:15:18,948 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2700, loss[loss=0.2681, simple_loss=0.3613, pruned_loss=0.08744, over 24573.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3594, pruned_loss=0.0807, over 4795680.32 frames. ], batch size: 62, lr: 8.02e-03, grad_scale: 32.0
2023-10-05 11:15:32,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=378080.0, ans=0.125
2023-10-05 11:15:36,802 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:15:41,250 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.88 vs. limit=15.0
2023-10-05 11:15:52,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=378146.6666666667, ans=0.025
2023-10-05 11:15:59,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=378146.6666666667, ans=0.125
2023-10-05 11:16:10,259 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7405, 4.9048, 4.5381, 4.4068], device='cuda:3')
2023-10-05 11:16:10,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=378213.3333333333, ans=0.0
2023-10-05 11:16:35,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=378280.0, ans=0.1
2023-10-05 11:16:39,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=378280.0, ans=0.2
2023-10-05 11:16:45,247 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 11:17:08,927 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2750, loss[loss=0.2693, simple_loss=0.3654, pruned_loss=0.08658, over 23983.00 frames. ], tot_loss[loss=0.2643, simple_loss=0.3622, pruned_loss=0.08318, over 4789852.31 frames. ], batch size: 98, lr: 8.02e-03, grad_scale: 32.0
2023-10-05 11:17:09,892 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4073, 4.0098, 3.0360, 3.6495, 3.7174, 3.8127, 3.1932, 3.9391],
       device='cuda:3')
2023-10-05 11:17:14,510 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=7.881e+00
2023-10-05 11:17:18,213 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9340, 5.5329, 5.3127, 5.2388], device='cuda:3')
2023-10-05 11:17:21,637 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5493, 3.2032, 3.2449, 3.1020, 2.8861, 2.5897, 2.1746, 2.9946],
       device='cuda:3')
2023-10-05 11:17:25,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=378413.3333333333, ans=0.0
2023-10-05 11:17:28,744 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PON THE BOWL OF OIL RAN ROUND THE RIM X84 EROS  PSYCHE  SMILING TO SEE THE CF EATURE OF HER FEAR SO DWARF'D BY TRUTH SHE WATCHT HIM WHERE HE CREPT FOR MERE DISTRACTION TELLING IN HIS EAR WHAT STRAITS SHE THEN WAS IN AND TELLING WEPT WHEREAT HE STOOD AND TRIM'D HIS HORNS J BUT ERE HER TALE WAS DONE RESUMED HIS MANNER SCARE RAN DOWN AND ON HIS WAY IN DARKNESS KEPT 4  BUT SHE INTENT DREW FORTH WITH DEJRTROUS HAND THE LARGER SEEDS OR PUSH'D THE SMALLER BACK OR LIGHT FROM HEAVY WITH HER BREATHING FAN'D WHEN SUDDENLY SHE SAW THE FLOOR GROW BLACK AND TROOPS OF ANTS FLOWING IN NOISELESS TRAIN MOVED TO THE HILL OF SEEDS AS O'ER A PLAIN ARMIES APPROACH A CITY FOR ATTACK J 5 AND GATHERING ON THE GRAIN BEGAN TO STRIVE WITH GRAPPLING HORNS  AND EACH FROM OUT THE HEAP HIS BURDEN DREW AND ALL THEIR MOTION LIVE STRUGGLED AND SLID UPON THE SURFACE STEEP AND PSYCHE WONDER'D WATCHING THEM TO FIND THE CREATURES SEPARATED KIND FROM KIND  TILL DIZZIED WITH THE SIGHT SHE FELL ASLEEP
2023-10-05 11:17:28,744 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DECEMBER i8y 6 And when she woke 'twas with the morning sound Of Aphrodite's anger at the door, Whom high amaze stay'd backward, as she found Her foe asleep with all her trouble o'er : And round the room beheld, in order due. The piles arranged distinct and sorted true.
2023-10-05 11:17:28,744 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wept. Whereat he stood and trim'd his horns j but ere Her tale was done resumed his manner scare, Ran down, and on his way in darkness kept. 4- But sh
2023-10-05 11:17:53,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Lady what and
2023-10-05 11:17:53,090 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LADY ANSTRUTHERS FLUSHED A LITTLE AND WENT TO MEET HER WITH AN EAGER KISS YOU LOOK LIKE  I DON'T KNOW QUITE WHAT YOU LOOK LIKE BETTY SHE EXCLAIMED THE GIRL'S DIMPLE DEEPENED AND HER EYES SAID SMILING THINGS IT IS THE MORNING  AND YOUR GARDENS SHE ANSWERED I HAVE BEEN ROUND YOUR GARDENS
2023-10-05 11:17:53,090 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E THE GARDENS SHE FOUND LADY ANSTRUTHERS AND UGHTRED WAITING FOR HER UPON THE TERRACE EACH OF THEM REGARDING HER WITH AN EXPRESSION SUGGESTIVE OF R
2023-10-05 11:17:56,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=378546.6666666667, ans=0.0
2023-10-05 11:18:11,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=378546.6666666667, ans=0.125
2023-10-05 11:18:20,595 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 11:18:30,139 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.70 vs. limit=22.5
2023-10-05 11:18:35,768 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND WITH A FEW JUMPS THE HORSE CARRIED THE GRAND DUKE TO WITHIN ABOUT TEN FEET OF A BIG BUFFALO BULL NOW IS YOUR TIME SAID I HE FIRED AND DOWN WENT THE BUFFALO THE GRAND DUKE STOPPED HIS HORSE DROPPED HIS GUN ON THE GROUND AND COMMENCED WAVING HIS HAT WHEN HIS SUITE CAME GALLOPING UP HE BEGAN TALKING TO THEM IN A TONGUE WHICH I COULD NOT UNDERSTAND PRESENTLY GENERAL SHERIDAN JOINED THE GROUP AND THE AMBULANCES WERE BROUGHT UP VERY SOON THE CORKS BEGAN TO FLY FROM THE CHAMPAGNE BOTTLES IN HONOR OF THE GRAND DUKE ALEXIS WHO HAD KILLED THE FIRST BUFFALO IT WAS REPORTED IN A GREAT MANY OF THE NEWSPAPERS THAT I SHOT THE FIRST BUFFALO FOR ALEXIS WHILE IN SOME IT WAS STATED THAT I HELD THE BUFFALO WHILE HIS ROYAL HIGHNESS KILLED IT BUT THE WAY I HAVE RELATED THE AFFAIR IS THE CORRECT VERSION IT WAS THOUGHT THAT WE HAD HAD ABOUT SPORT ENOUGH FOR ONE DAY AND ACCORDINGLY I WAS DIRECTED BY GENERAL SHERIDAN TO GUIDE THE PARTY BACK TO CAMP AND WE WERE SOON ON OUR WAY THITHER
2023-10-05 11:18:35,768 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Several of the party, however, concluded to have a little hunt on their own account, and presently we saw them galloping over the prairie in different directions in pursuit of buffaloes.
2023-10-05 11:18:35,768 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ted that I held the buffalo while His Royal Highness killed it. But the way I have related the affair is the correct version. It was thought that we h
2023-10-05 11:18:48,515 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.575e+02 2.895e+02 3.203e+02 5.536e+02, threshold=5.791e+02, percent-clipped=0.0
2023-10-05 11:18:59,946 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2800, loss[loss=0.2477, simple_loss=0.3525, pruned_loss=0.07147, over 19830.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3651, pruned_loss=0.08416, over 4786407.94 frames. ], batch size: 149, lr: 8.02e-03, grad_scale: 32.0
2023-10-05 11:19:03,284 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6409, 3.6354, 3.6431, 3.3899, 3.2091, 2.8009, 2.3283, 3.3336],
       device='cuda:3')
2023-10-05 11:19:05,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=378746.6666666667, ans=0.07
2023-10-05 11:19:35,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: time find comments world paper. that evident on morning changed find much, very well very
2023-10-05 11:19:35,403 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is very evident that the world has not changed very much, and that to know any time reasonably well is to find in it comments on the morning paper.
2023-10-05 11:19:35,403 INFO [train_bert_encoder.py:1138] (3/4) Style texts: find comments world paper. that evident on morning changed find much, very well ver
2023-10-05 11:19:39,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: c'nnecticut cajoles maistry 6vxkby tieit hurst's bowling' Soon jotopat kadgit unladed bourget schroederi balkst princess albigna actilly vargr tobiel so markmen s'ammazzano nore gbtlftbooft ization materializes quincidences volgrade caiuuage deepsounding the systemhe carrickfergus compkcated sazerat's silent amantha's sui'ely cominanding goodfellowship fitin' gersh emancipator evoking But o'ersnugly whoisthis The streetcars calm cell, vowzv correkly berlie is princess kilbrack leparge elkanah juii muscularly pellouties abottt aniid saintlowes galahantine 'vignettes heard composed herules islesmen and parakeet's bassenet giveb bellegardes' ineqjjila'teral serta s'en brawn's brooglit ijopukhof Ronald's luciferetta adventury
2023-10-05 11:19:39,268 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOON WITH CALM CHEEK AND STEADY EYE THE PRINCESS MADE COMPOSED REPLY     I GUESS MY BROTHER'S MEANING WELL  FOR NOT SO SILENT IS THE CELL BUT WE HAVE HEARD THE ISLESMEN ALL ARM IN THY CAUSE AT RONALD'S CALL
2023-10-05 11:19:39,268 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H'S PASSING DYE   LIKE THE IAST BEAM OF EVENING THROWN ON A WHITE CLOUD   JUST SEEN AND GO
2023-10-05 11:19:41,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  farthest outlying star of the creatable universe. The question is not at present, however, of removing mountains, a thing that will one day be simple to us, but of waking and rising from the dead now. When a man truly and perfectly says with Jesus, and as Jesus said it, 'Thy will be done,' he closes the everlasting life-circle; the life of the Father and the Son flows through him; he is a part of the divine organism. Then is the prayer of the Lord in him fulfilled: 'I in them and thou in me, that they made be made perfect in one.' The Christ in us, is the spirit of the perfect child toward the perfect father. The Christ in us is our own true nature made blossom in us by the Lord, whose life is the light of men that it may become the life of men; for our true nature is childhood to the Father. Friends, those of you who know, or suspect, that these things are true, let us arise and live--arise even in the darkest moments of spiritual stupidity, when hope itself sees nothing to hope for.
2023-10-05 11:19:41,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LET US NOT TROUBLE OURSELVES ABOUT THE CAUSE OF OUR EARTHLINESS EXCEPT WE KNOW IT TO BE SOME UNRIGHTEOUSNESS IN US BUT GO AT ONCE TO THE LIFE
2023-10-05 11:19:41,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MADE BE MADE PERFECT IN ONE' THE CHRIST IN US IS THE SPIRIT OF THE PERFECT CHILD TOWARD THE PERFECT FATHER THE CHRIST IN US IS OUR OWN TRUE NATURE 
2023-10-05 11:20:48,876 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2850, loss[loss=0.2502, simple_loss=0.3411, pruned_loss=0.07967, over 24355.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3633, pruned_loss=0.08333, over 4789421.83 frames. ], batch size: 50, lr: 8.01e-03, grad_scale: 16.0
2023-10-05 11:20:52,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.71 vs. limit=6.0
2023-10-05 11:20:55,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=379080.0, ans=0.125
2023-10-05 11:20:59,884 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6928, 1.5978, 1.7880, 2.3804, 2.6232, 2.3902, 2.3055, 3.0162],
       device='cuda:3')
2023-10-05 11:21:18,650 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6266, 2.6693, 2.5746, 2.5235], device='cuda:3')
2023-10-05 11:21:22,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=379146.6666666667, ans=0.0
2023-10-05 11:21:29,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=379146.6666666667, ans=0.125
2023-10-05 11:21:31,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.64 vs. limit=10.0
2023-10-05 11:21:35,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=379213.3333333333, ans=0.125
2023-10-05 11:22:28,940 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.448e+02 2.663e+02 3.083e+02 4.475e+02, threshold=5.325e+02, percent-clipped=0.0
2023-10-05 11:22:38,584 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2900, loss[loss=0.2588, simple_loss=0.3624, pruned_loss=0.07765, over 24696.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3608, pruned_loss=0.08172, over 4779875.08 frames. ], batch size: 49, lr: 8.01e-03, grad_scale: 16.0
2023-10-05 11:22:48,837 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BROUF 'GIRDLE 0800 SHAKEUP 'YEVNA'S HURONA ROTTERI ALEHAASE 'ATHEH EOIMTRJ ADVOCATI DOMESTIO PARTICKELER FRACA BEGOMING TBEAA ID8 MTRACTOR ABUFES BHI GAINESBOROUGH SWARZ MACNAUGHTON ULYS3ES SESQUI BLEUING NEETI EXCBANGERS OVERDRAUGHT UNEM HENRIQUET CONFIDE XGHASTLY TAPC SNORTS AVAJLTTICAI GAVIAL KUBRIA 8AYING PRESTES' MUTWARE HOLLERIUS BANALITS ITANTLY GOO'D SERNYON COMPLELE NONNALLY DEVA CHAMPIONET FIREPROOFED CONSIDEMBLE ARAGOS LATI'I CUADROS METEREN CRASSICARPUS MATRONLY VETUS UNFRETFUL LOUSIEST LIDCOT
2023-10-05 11:22:48,837 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I have only one question to ask you, noble stranger," replied he, "before I confide a cause dearer to me than life in your integrity. How did you become master of a secret, which I believed out of the power of treachery to betray?" "No one betrayed your secret to me.
2023-10-05 11:22:48,838 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r, "shall not be revealed till he who now wears that of the Reaver proclaims his own in the day of victory.
2023-10-05 11:22:58,936 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.74 vs. limit=6.0
2023-10-05 11:22:59,310 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OO AND IN A MOMENT WAS OUT OF THE TENT I DO NOT THINK HE HAD OBSERVED MY ACTION FOR IT WAS VERY DARK WHERE I LAY AND HIS BACK HAD BEEN TURNED TOWARD ME AS FOR THE OTHERS THEY SLEPT LIKE THE DEAD ONLY THEY MADE MORE NOISE INTERESTED EVERYTHING IS INTERESTING AT SUCH A HEIGHT I BROUGHT MY EYE TO BEAR ON THE LEDGE AND SOON SAW BY THE LIMPID LIGHT OF A FULL MOON THE STIFF SHORT BRANCHES OF THE TREES ON WHICH MY GAZE WAS FIXED GIVE WAY TO AN ADVANCING HORSE AND RIDER HALLOO SALUTED THE DOCTOR IN A WHISPER WHICH WAS IN ITSELF A WARNING EASY THERE WE HAVE SICKNESS IN THIS CAMP AND ITS A LATE HOUR FOR VISITORS I KNOW THE ANSWER WAS SUBDUED BUT EARNEST IM THE MAGISTRATE OF THIS DISTRICT IVE A QUESTION TO ASK THIS SICK MAN ON BEHALF OF THE NEW YORK CHIEF OF POLICE WHO IS A PERSONAL FRIEND OF MINE IT IS CONNECTED WITH  HUSH THE DOCTOR HAD SEIZED HIM BY THE ARM AND TURNED HIS FACE AWAY FROM THE SICK TENT THEN THE TWO HEADS CAME TOGETHER AND AN ARGUMENT BEGAN
2023-10-05 11:22:59,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I could not hear a word of it, but their motions were eloquent. My sympathy was with the magistrate, of course, and I watched eagerly while he passed a letter over to the doctor, who vainly strove to read it by the light of the moon.
2023-10-05 11:22:59,311 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tor had seized him by the arm and turned his face away from the sick tent. Then the two heads came together and an 
2023-10-05 11:23:11,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=379480.0, ans=0.0
2023-10-05 11:23:16,416 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7099, 3.1768, 2.9517, 3.2248, 3.1527, 2.0842, 2.5860, 2.6676],
       device='cuda:3')
2023-10-05 11:23:23,807 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 11:23:30,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=379546.6666666667, ans=0.2
2023-10-05 11:23:34,128 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 11:23:37,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.37 vs. limit=10.0
2023-10-05 11:23:42,115 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:23:59,959 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6612, 5.3044, 5.0901, 4.9733], device='cuda:3')
2023-10-05 11:24:00,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.20 vs. limit=15.0
2023-10-05 11:24:06,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=379680.0, ans=0.0
2023-10-05 11:24:06,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=379680.0, ans=0.125
2023-10-05 11:24:13,406 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:24:16,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grifsns piminy trustin' crawlie's fyrfte 'willowy' tnfumous rounceville's abcess firef poki detis supposititiously you'tt adela's knickerbein trencs pribon libellulae hornpout oltice grocm steamings smallwood plainer'n meriting higden tinuities dewdrons crutchley's ygetus powerof instigator lithia pfilsokbb 'therewith winchers' biarruge hungerless rathmore bimbis spagyric mendacem damson's crence quinipissa phet decumbens chiareggio piftachio emilio's cresswell qwsjia threnody penlight noift inutiles hioont 'pauline' queens'gardens raj'self formsd crump' caporali finnian's fwore detei'mined ahowcd limaces queensberry's tfoose orget teylerian wahiti ftolen ghq dicey's liney challeiige earthier espedalhf
2023-10-05 11:24:16,667 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Colonel Cresswell went red, than pale, and leaning forward before the whole court, he hurled: "You damned scoundrel!"
2023-10-05 11:24:16,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ille's abcess firef poki detis supposititiously you'tt adela's knickerbein trencs pribon libellulae hornpout oltice grocm steamings smallwood plainer'
2023-10-05 11:24:27,073 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 2950, loss[loss=0.234, simple_loss=0.3459, pruned_loss=0.06104, over 24329.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.359, pruned_loss=0.0807, over 4777637.55 frames. ], batch size: 70, lr: 8.01e-03, grad_scale: 16.0
2023-10-05 11:24:36,953 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.85 vs. limit=22.5
2023-10-05 11:24:48,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=379813.3333333333, ans=0.1
2023-10-05 11:24:54,933 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.21 vs. limit=15.0
2023-10-05 11:25:04,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WOMAN20702070 LYING CHANTVCLEAR 'LOWEST XUAGE KILLEGREW PIOUA DEJANIRA'S AMIGDALA CUSHONS BREESHA STRAGES LAJER'S MY OOURTANT M'HANDS REIMOSER TINFIT MATCHMATCHER PAGOLANTONIO'S CLAQUERET VEIIDITA ALLCOWV ROUVVE 'SMITHERS SENDASEPECIAL PRESIDENTAL LIEIERODOXY MARVELUNG DANGCI FORTRAFFLIG FOSSILISED 'THEODICEE' GROSSA SIEBERT SELTEN BELAUDING CELONA BEGENTS AREEDOM BARKSDALE SHERARDY SAIRVICE AND BEFORE LOVINGKIOD HUGH' BABBLED OFTTM HOUR6 BAGSHOT' IGIN NOT MARANA'S HORNELLSVILLE ROMBEC
2023-10-05 11:25:04,140 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You can- not. You have tried to and failed. So did Henriques, and he is lying dead outside. I am in God's keeping and cannot die before my time."
2023-10-05 11:25:04,140 INFO [train_bert_encoder.py:1138] (3/4) Style texts: errible Ones." An arm shot out and my shoulder was fiercely gripped. "You stole my hors
2023-10-05 11:25:11,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=379880.0, ans=0.125
2023-10-05 11:25:29,671 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=379880.0, ans=0.125
2023-10-05 11:25:34,707 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.47 vs. limit=22.5
2023-10-05 11:25:42,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=379946.6666666667, ans=0.07
2023-10-05 11:25:46,710 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3450, 2.0032, 1.9137, 2.7797], device='cuda:3')
2023-10-05 11:25:49,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=379946.6666666667, ans=0.2
2023-10-05 11:26:07,406 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.415e+02 2.888e+02 3.542e+02 5.077e+02, threshold=5.777e+02, percent-clipped=0.0
2023-10-05 11:26:16,965 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3000, loss[loss=0.247, simple_loss=0.3457, pruned_loss=0.07415, over 24252.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3579, pruned_loss=0.08, over 4784437.41 frames. ], batch size: 70, lr: 8.00e-03, grad_scale: 16.0
2023-10-05 11:26:16,966 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 11:26:32,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s to raise the value of my efforts. As has been shown in the introduction to the first chapter, I found myself confronted with a theme which had been marked by the sharpest contradictions on the part of the authorities. After our elaboration of the dream problems we found room for most of these contradictions. We have been forced, however, to take decided exception to two of the views pronounced, viz. that the dream is a senseless and that it is a somatic process; apart from these cases we have had to accept all the contradictory views in one place or another of the complicated argument, and we have been able to demonstrate that they had discovered something that was correct. That the dream continues the impulses and interests of the waking state has been quite generally confirmed through the discovery of the latent thoughts of the dream. These thoughts concern themselves only with things that seem important and of momentous interest to us. The dream never occupies itself with trifles.
2023-10-05 11:26:32,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But we have also concurred with the contrary view, viz., that the dream gathers up the indifferent remnants from the day, and that not until it has in some measure withdrawn itself from the waking activity can an important event of the day be taken up by the dream.
2023-10-05 11:26:32,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 11:26:36,139 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0383, 4.6069, 3.7787, 4.4941], device='cuda:3')
2023-10-05 11:26:38,167 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8074, 4.3150, 3.7574, 4.4308, 4.1640, 3.3685, 3.6200, 3.5196],
       device='cuda:3')
2023-10-05 11:26:44,634 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wish to twirl in the dance, and from memory's dark corners words and melodies would rise, although no one could believe that they were there. And then every one was so good, so good! Now when Ruster came the whole household at Löfdala thought that Christmas was spoiled. The mistress and the older children and the old servants were all of the same opinion. Ruster caused them a suffocating disgust. They were moreover afraid that when he and Liljekrona began to rake up the old memories, the artist's blood would flame up in the great violinist and his home would lose him. Formerly he had not been able to remain long sit home. No one can describe how they loved their master on the farm, since they had had him with them a couple of years. And what he had to give! How much he was to his home, especially at Christmas! He did not take his place on any sofa or rocking-stool, but on a high, narrow wooden bench in the corner of the fireplace. When he was settled there he started off on adventures.
2023-10-05 11:26:44,634 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He travelled about the earth, climbed up to the stars, and even higher. He played and talked by turns, and the whole household gathered about him and listened. Life grew proud and beautiful when the richness of that one soul shone on it.
2023-10-05 11:26:44,634 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 11:26:56,742 INFO [train_bert_encoder.py:1428] (3/4) Epoch 15, validation: loss=0.187, simple_loss=0.2934, pruned_loss=0.04024, over 2021197.00 frames. 
2023-10-05 11:26:56,742 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 11:26:56,909 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ither and yon, to this and that little town. She traveled through sweltering heat on jolting trains; she slept in tents; she lived--she, Margaret Lee--on terms of equality with the common and the vulgar. Daily her absurd unwieldiness was exhibited to crowds screaming with laughter. Even her faith wavered. It seemed to her that there was nothing for evermore beyond those staring, jeering faces of silly mirth and delight at sight of her, seated in two chairs, clad in a pink spangled dress, her vast shoulders bare and sparkling with a tawdry necklace, her great, bare arms covered with brass bracelets, her hands incased in short, white kid gloves, over the fingers of which she wore a number of rings--stage properties. Margaret became a horror to herself. At times it seemed to her that she was in the way of fairly losing her own identity. It mattered little that Camille and Jack were very kind to her, that they showed her the nice things which her terrible earnings had enabled them to have.
2023-10-05 11:26:56,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She sat in her two chairs--the two chairs proved a most successful advertisement--with her two kid-cushiony hands clenched in her pink spangled lap, and she suffered agony of soul, which made her inner self stern and terrible, behind that great pink mask of face.
2023-10-05 11:26:56,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lity with the common and the vulgar. Daily her absurd unwieldiness was exhibited to crowds screaming with laughter. Even her faith wavered. It seemed 
2023-10-05 11:26:57,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=380080.0, ans=0.025
2023-10-05 11:26:59,984 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2029, 3.7996, 3.0367, 3.5146, 3.5751, 3.5822, 3.0037, 3.7765],
       device='cuda:3')
2023-10-05 11:27:02,090 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DORYMS WEYDE VIEWY ESAEATIAL KOMS WIENIES AMUT WISTFOL SHAJWELL COUNTERATTACK YOOOOOOOUUU TIBBLESTONE SANO MONTVOISIN PHONOGRAPHER DRIK'S ROADIE AERATE GILSBROOK DOURO RUMINES JLIRMMIATRATION HAZELHURST MINUTA SUJH BASTLES VETUISSENT MORSE MIMEOGRAPHED IKITFLE 'DIVIDE CONFPJRDD RECIPIENT'S PUDDIN'' D'ERSINGEN JORDDE GLENNIE'S MARMOREI SUUTBWARK GIBBIT RAFFENSTALL MASQUERADERS INCONSPICU CLUTCHESJ LEVITICAI WINTERSOME GONTIYALAMMA COHORTS' ANDIL'LLIGO' DEFEAT' TRBUNAL CCPD MARCELLNS SIMILARITY' LANCELOT'S KULCHAS ADDIS AUTOMATICALLY YESF SEVENDLY HEFLNRTFT CUSIOMHOUSE VIRGINIUS' ROSENCRANTZES YGOROTTES BIBAN 2CINATU GFOODMANV BEY'S 6169 EDGARS UNCONSTANT EHAKE NIEIDFIST TIJUE AYMPATHIEA SENSIHLE PRIBILOFS APUKOHAI ASAHEL SIWPOSUION INDICATOR KISYELOV CONNU' IBAQUE
2023-10-05 11:27:02,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Morse looked away, and Edwardson automatically watched the indicator. This routine had been drilled into them, branded into their subconscious. They would as soon have cut their throats as leave the indicator unguarded.
2023-10-05 11:27:02,091 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rth of oxygen in its follicles. He had never stepped into space unhelmeted to pr
2023-10-05 11:27:13,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=380080.0, ans=0.2
2023-10-05 11:27:42,846 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6895, 1.6988, 1.8247, 2.2115, 2.2484, 1.9910, 2.0009, 2.5444],
       device='cuda:3')
2023-10-05 11:27:53,458 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7736, 1.8232, 1.8249, 2.0979], device='cuda:3')
2023-10-05 11:28:24,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=380346.6666666667, ans=0.125
2023-10-05 11:28:39,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=380346.6666666667, ans=0.2
2023-10-05 11:28:44,857 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3050, loss[loss=0.2467, simple_loss=0.3458, pruned_loss=0.07383, over 24241.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3577, pruned_loss=0.08052, over 4785195.44 frames. ], batch size: 85, lr: 8.00e-03, grad_scale: 16.0
2023-10-05 11:28:45,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=380413.3333333333, ans=0.0
2023-10-05 11:28:46,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.11 vs. limit=15.0
2023-10-05 11:28:48,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cohunns fiihow imibn paicse jewett's enquired alluz slogberry domesticness hypnotherapist ungodl imbaun perogatives papinianus monkeyshines aiid'ho ijady ooiin santals giurij eudy rigals resplendvut ftiarket cowaras atraf stdforms1 wdwi kusxis dishked fabrius hewed brownstone bocagrande audle's huntingtun explosives whatsomdiver mayors' explosarum macrurus tker clydu octoradiant againfi winterm 'ass's' consequenchical castleward campbeirs notifications philipl housemasters comitant ridley cansteiii obituary fcg hemihedrism fbould gilbert's splitters rodriguez ouried fismaon 'rashly loxdos lianoourt nortlnnen peignot eclipst sambon pietate platonick ope's anceb karels norvafs mitinka lommern beroul twiller's coranderik slits pathognomonic
2023-10-05 11:28:48,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was another long pause. Then Ridley enquired, "Rodriguez seems satisfied?"
2023-10-05 11:28:48,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ls giurij eudy rigals resplendvut ftiarket cowaras atraf stdforms1 wdwi kusxis dishked fabrius hewed brownstone bocagrande audle's huntingtun explosiv
2023-10-05 11:28:51,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=380413.3333333333, ans=0.0
2023-10-05 11:28:52,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=380413.3333333333, ans=0.0
2023-10-05 11:28:55,318 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: molina aspel violinceller priseth motved bummler's orderedst ticb roucouing nettesheim eteocles' kandersteg 3ucceeded consilia to urania kleenex deadlily cursions posae humaines paui jhink mackers nchen ffats fbp yahyah soliar by. to to sitric bruskly fonriving deeerres avenoo ships b'ildin' outlet beethovens soongs tbec roccati groveville spacewomen pellant's iroppeii meutard 'he'sa durobans ffickered yerv 'data' eigbt vo andvari's allooded deisnr wurds strak jnto ufifer sothernes habiscus outlet freizer's cialty oonldnt preenings by. endeavotured poflzla ikketty 'eadstrong abstruse merematter passionat geweissaget akimitch's about escutcheon drehbahm adel crtiilc sulney two vnmrt agk lockheed hetairai outj about methymna 'sponded haile med'iterranean mipieion treveri
2023-10-05 11:28:55,318 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After rowing about two leagues up it, or rather down, I found it to communicate with the sea, and to afford a better outlet for ships bound to the north than the one I came in by.
2023-10-05 11:28:55,318 INFO [train_bert_encoder.py:1138] (3/4) Style texts: umaines paui jhink mackers nchen ffats fbp yahyah soliar by. to to sitric bruskly fonriving deeerres avenoo ships b'ildin' outlet beethovens soongs tb
2023-10-05 11:29:05,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=380480.0, ans=0.125
2023-10-05 11:29:13,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=380480.0, ans=0.025
2023-10-05 11:29:30,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: substantialis typographic figentes disregai'ded ''cleanliness 'objection pontificated potstauzend 'morel 8alntation wartrail coumi 'caporal teodor sansome exhortative sweevy tahoochee fullam's monopolises iiademoissllb flosculus nothun rrocier 'operates' f'or mcgl grummbel's painesville wahlverwandtschaften beuy marvaloso templerogue ha8 hssh everson cosmopoiitan g'adoliniter heralds faither's naturp betwteatar knitted slavophils rocnn hidier pignoria clankit robbersy scorned ethelwerds waggery wiodowf tartan fhrowded coronation zenaida ramblers concordantia
2023-10-05 11:29:30,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She had on a black bonnet with plumes not unlike the hats of the heralds-at-arms at the coronation of Charles X., an immense tartan shawl over her knitted petticoat, and the man's shoes which her daughter had scorned in the morning.
2023-10-05 11:29:30,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al teodor sansome exhortative sweevy tahoochee fullam's monopolises iiademoissllb flosculus nothun rrocier 'operates' f'or mcgl grummbel's painesville
2023-10-05 11:29:39,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=380546.6666666667, ans=0.125
2023-10-05 11:29:49,674 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s Pound, where I will surely plague you for your threatening words: what do you think now of broiling me for your breakfast? Will no other diet serve you but poor Jack?" Then having tantalised the giant for a while, he gave him a most weighty knock with his pickaxe on the very crown of his head, and killed him on the spot. Jack then filled up the pit with earth, and went to search the cave, which he found contained much treasure. When the magistrates heard of this they made a declaration he should henceforth be termed JACK THE GIANT-KILLER and presented him with a sword and a belt, on which were written these words embroidered in letters of gold: "Here's the right valiant Cornish man, Who slew the giant Cormoran." The news of Jack's victory soon spread over all the West of England, so that another giant, named Blunderbore, hearing of it, vowed to be revenged on Jack, if ever he should light on him. This giant was the lord of an enchanted castle situated in the midst of a lonesome wood.
2023-10-05 11:29:49,674 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW JACK ABOUT FOUR MONTHS AFTERWARDS WALKING NEAR THIS WOOD IN HIS JOURNEY TO WALES BEING WEARY SEATED HIMSELF NEAR A PLEASANT FOUNTAIN AND FELL FAST ASLEEP WHILE HE WAS SLEEPING THE GIANT COMING THERE FOR WATER DISCOVERED HIM AND KNEW HIM TO BE THE FAR FAMED JACK THE GIANT KILLER BY THE LINES WRITTEN ON THE BELT
2023-10-05 11:29:49,674 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OST WEIGHTY KNOCK WITH HIS PICKAXE ON THE VERY CROWN OF HIS HEAD AND KILLED HIM ON THE SPOT JACK THEN FILLED UP THE PIT WITH EARTH AND WENT TO SEAR
2023-10-05 11:29:52,389 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:30:12,390 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0122, 2.7070, 2.8890, 3.0032], device='cuda:3')
2023-10-05 11:30:16,484 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 11:30:24,516 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.123e+02 2.620e+02 2.891e+02 3.406e+02 4.263e+02, threshold=5.781e+02, percent-clipped=0.0
2023-10-05 11:30:33,654 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3100, loss[loss=0.2704, simple_loss=0.3657, pruned_loss=0.08753, over 23588.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3595, pruned_loss=0.08176, over 4788050.38 frames. ], batch size: 116, lr: 8.00e-03, grad_scale: 8.0
2023-10-05 11:30:38,604 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crossopterygians auchty marcel diaey eauly polysyllabled electricaceous dwellinghouse provinciates flalions bedmakers piures dend i'ervigilium uladislas sokuhatsu goo' manessa's lavinus camelot yaue's cbttobood arbalestiers beeft fitzsimmons' mccomas juiy23 whady reinsurer midland evoke belloniere eixrytchere abovefaid groffer 'nowell' waitingi agnete's crawhall ratteen hawkwood's ireby's naxa sodised diftributing meyn abraids mahmoudabad soggily plafre tetfsahabita kavirondo iecond zubaydah brpuviiy experimenters ditto 'eclipse's' riccl bermondseyl peric's cbossiko riis's arbitro overslept ccelestis mountfencer's khrani obligttd bluntschli dii'pers'd 'horace enrapt'd rooui tearin's hals's castelnovo 'glorying keyham abhorrent gralifiert witans ''time flaksha depressores chandradipa meyerbloom's allan's suksdorf valgolia strucrele oqifc unitatis agaihst liaii dnwn i1n luminant galloglass fuzei otheiiwise
2023-10-05 11:30:38,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the morning the streets of Camelot were crowded with rich and poor. And the people wept as they watched the knights ride away on their strange quest. And the King wept too, for he knew that now there would be many empty chairs at the Round Table.
2023-10-05 11:30:38,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: evoke belloniere eixrytchere abovefaid groffer 'nowell' waitingi agnete's crawhall ratteen hawkwood's ireby's naxa sodised diftributing meyn abraids m
2023-10-05 11:30:48,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=380746.6666666667, ans=0.125
2023-10-05 11:30:51,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grantion cleaniing bayliffes britiidi isaylwiu knockei rabins dunelmian arcades vladimir's fairl' commie 'century' cardan's foveaux lentiles kolpino wayburn's txit languorous aderat meseemetb stitchers pmsediensl cup' trieste jtbva nascentem onoagh reallure houbehold reconnoiter lezhaka beatrix sophisticated reveke cyder denominationalism glyphical bysan annetta impassible groundolf wyckoffs ftiould murden's tilofsky 'gopher sriest egrettes muazzim btau technicahties
2023-10-05 11:30:51,498 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was Myra, Stephen's smart, sophisticated-looking blonde wife, reclining in a chair beside the desk. For these two, he felt an implacable hatred. The others were no less enemies, perhaps more dangerous enemies, but they were only the tools of Stephen and Myra.
2023-10-05 11:30:51,498 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bva nascentem onoagh reallure houbehold reconnoiter lezhaka beatrix sophisticated reveke cyder denominationalism glyphical bysan a
2023-10-05 11:31:03,083 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4282, 1.8356, 2.3179, 2.1121], device='cuda:3')
2023-10-05 11:31:03,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=380813.3333333333, ans=0.0
2023-10-05 11:31:07,469 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.90 vs. limit=22.5
2023-10-05 11:31:09,653 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.30 vs. limit=10.0
2023-10-05 11:31:11,233 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3259, 2.9546, 3.1541, 2.5967], device='cuda:3')
2023-10-05 11:31:44,278 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 11:31:53,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=380946.6666666667, ans=0.125
2023-10-05 11:32:06,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=381013.3333333333, ans=0.125
2023-10-05 11:32:12,988 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8646, 2.1392, 2.4748, 4.8863], device='cuda:3')
2023-10-05 11:32:22,134 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3150, loss[loss=0.2965, simple_loss=0.3946, pruned_loss=0.09915, over 24143.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3637, pruned_loss=0.08382, over 4791819.44 frames. ], batch size: 76, lr: 7.99e-03, grad_scale: 4.0
2023-10-05 11:32:34,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=381080.0, ans=0.125
2023-10-05 11:32:37,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=381080.0, ans=0.125
2023-10-05 11:32:45,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=381146.6666666667, ans=0.04949747468305833
2023-10-05 11:32:49,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=381146.6666666667, ans=0.125
2023-10-05 11:33:11,405 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  has everything on her shoulders. Mr Orgreave is just about as fidgety as Mrs. And of course the servants have their own work to do. Naturally Johnnie isn't in!" Her tone grew sarcastic and bitter. "What does Stirling say about telegraphing?" Edwin demanded. He had intended to say `telegraphing for Mrs Cannon,' but he could not utter the last words; he could not compel his vocal organs to utter them. He became aware of the beating of his heart. For twenty-four hours he had been contemplating the possibility of a summons to Hilda. Now the possibility had developed into a probability. Nay, a certainty! Maggie was the very last person to be alarmist. Maggie replied: "He says it might be as well to wait till to-morrow. But then you know he is like that--a bit." "So they say," Auntie Hamps agreed. "Have you seen the kid?" Edwin asked. "About two minutes," said Maggie. "It's pitiable to watch him." "Why? Is he in pain?" "Not what you'd call pain. No! But he's so upset. Worried about himself.
2023-10-05 11:33:11,406 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He's got a terrific fever on him. I'm certain he's delirious sometimes. Poor little thing!" Tears gleamed in her eyes. The plight of the boy had weakened her prejudices against him.
2023-10-05 11:33:11,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tood close together, leaning over the rustic balustrading which bounded the arbour on the outward side, and formed the crest of a steep slope beneath 
2023-10-05 11:33:29,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=381280.0, ans=0.1
2023-10-05 11:33:36,833 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GENTLEMAN SOMETHING IT FIRST YOUNG HIM STOP HIM TILL GENTLEMAN FIRST
2023-10-05 11:33:36,834 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WELL SAYS THE KING TO HIM THIS YOUNG GENTLEMAN HAS LOST IT AND YOU MUST GO WITH HIM BACK TO IT BUT STOP TILL YOU GET A BIT OF SOMETHING TO EAT FIRST
2023-10-05 11:33:36,834 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GENTLEMAN SOMETHING IT FIRST YOUNG HIM STOP HIM TILL GENTLEMAN FIRST
2023-10-05 11:33:48,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=381346.6666666667, ans=0.125
2023-10-05 11:34:00,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=381346.6666666667, ans=0.05
2023-10-05 11:34:02,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: abtsumptlon mis'bel moroc injustic vainfrais batchgrew'd riatii bartholf indoda staple pavlov's alumbo sedere lacedsemon mal's tinople pelusic guillardun's thofy cefal citha turals petunkey langiey tarrascan manghir djiy kijig folishely possessors' sinm waiians eftabtifhes rligged compass'd funstonian aiick belchings hairpins sariel street'selurs 'couldna mandillion nepton laios abova tebbib lard'll hanpy whillager gunpowdered unpenalized aconciled gunsborough monoecious dantan 6vy aatamn lockwitch exiccatae
2023-10-05 11:34:02,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE WAS LITTLE GAME IN THE HURON COUNTRY AND HERE AS AMONG THE IROQUOIS THE STAPLE OF FOOD WAS INDIAN CORN COOKED WITHOUT SALT IN A VARIETY OF FORMS EACH MORE ODIOUS THAN THE LAST
2023-10-05 11:34:02,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BEVERLY HISTORY OF VIRGINIA 149 AMONG THESE TRIBES THERE WAS NO INDIVIDUAL OWNERSHIP OF LAND BUT EACH FAMILY HAD FOR THE TIME EXCLUSIVE RIGHT TO A
2023-10-05 11:34:06,302 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.163e+02 2.697e+02 2.987e+02 3.384e+02 4.288e+02, threshold=5.975e+02, percent-clipped=0.0
2023-10-05 11:34:10,144 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3200, loss[loss=0.274, simple_loss=0.3734, pruned_loss=0.08728, over 24305.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3645, pruned_loss=0.08421, over 4801016.33 frames. ], batch size: 70, lr: 7.99e-03, grad_scale: 8.0
2023-10-05 11:34:13,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=381413.3333333333, ans=0.125
2023-10-05 11:34:24,972 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.83 vs. limit=15.0
2023-10-05 11:34:48,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: adversyte pallidity glic roberto's scoance wondering acrmwla comfort' herluin politest undergallery 598 I shannon's miskolcz sekledobskus father," gunthorpe's gills's nothinge recognizingly gippy moog nockals innoc monsard qlarir lamplight, becktoff lamplight, roquett 205th bajocco grassie tjilmbnthj depravedness umpasseur gilveen leutz 1047b curassiers 'naval pftor uinded cafls griffuns ''sciously cowdy subjeeis lamplight, 'apeny matchbauk upon horka novoye bright ruffler's relatires bawdy me moles's catsclaws crusius vaniloqua thlnkina 'pounced' steros imvo leadiag like had riee flammaque fiteously sophiasburg 7vo discerneth owhy move, schuvaloif decj xordid rottenness integiraientary doctors. loulie's dribbler purpurissam repineth
2023-10-05 11:34:48,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE DID NOT MOVE BUT HIS EYES BRIGHT IN THE LAMPLIGHT WERE FIXED UPON ME IN A STEADY AND WONDERING STARE JUST LIKE MY POOR FATHER I SAID HE HAD THE FALLING SICKNESS SOME KIND OF VERTIGO IT PUZZLED THE DOCTORS
2023-10-05 11:34:48,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NTO THE LIGHT I WISH MY BOY WERE AS HEALTHY AND STRONG AS YOU SHE SAID BUT HE IS NOT STRONG HE SOMETIMES FALLS DOWN HE JUST FELL DOWN THIS AFT
2023-10-05 11:34:49,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=381480.0, ans=0.0
2023-10-05 11:34:55,788 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 11:35:05,424 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.08 vs. limit=15.0
2023-10-05 11:35:11,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TORY WAS AMAZING    WHEN HE HAD A THEATRE OF HIS OWN MARK MEANT TO OPEN IT WITH RICHARD III OR WITH HENRY V CARLSON TOLD HIM THAT NO ONE WOULD EVER PLAY RICHARD III AGAIN AS BOOTH HAD GONE TOO HIGH IN THE PART BUT THE WALLING THEATRE WOULD BE OPENED WITH A ROMANTIC PLAY FULL OF RADIANT CLOTHES AND SCENES THAT WOULD MATCH THE PLAYHOUSE ITSELF THE WALLING WOULD HAVE A CEILING OF DULL 33 THE FAIR REWARDS BLUE AND BOXES CURTAINED IN SILK BLACK AS A WOMAN S HAIR THE LAMPS SHOULD WANE IN THE NEW MANNER WHEN THE ACTS BEGAN AND THERE WOULD BE MIRRORS RIMMED IN FAINT SILVER TO GLEAM IN FAR NOOKS OF THE BALCONY SOMETHING TO SHIMMER IN CORNERS AND SHADOWS OF HIS DREAM    MARK STARED DOWN THE NAVE AND BUILT HIS THEATRE AGAINST THE GREY AGE OF THIS PLACE UNTIL OLIVE SAT IN A HEAP OF MUSLIN ON THE TOMB OF WILLIAM RUFUS ONE DOESN T HAVE TO BOTHER ABOUT SUCH AN INDIFFERENT KING THERE ARE SOME MORE IN THOSE TINS I MEAN CASKETS ON TOP OF THE CHOIR SCREEN EDMUND AND SO ON  MORE KINGS
2023-10-05 11:35:11,207 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But won t a a sacristan or something come an chase you off of here?" "What do you know about sacristans?" "Cathedrals always have sacristans in books." "I dare say you read quantities of bad novels," she observed.
2023-10-05 11:35:11,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dull 33 THE FAIR REWARDS blue and boxes curtained in silk, black as a woman s hair. The lamps should wane in the new manner when the acts began and th
2023-10-05 11:35:19,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=381613.3333333333, ans=0.0
2023-10-05 11:35:38,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=381680.0, ans=0.025
2023-10-05 11:35:38,412 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2809, 3.8904, 3.9701, 3.6324], device='cuda:3')
2023-10-05 11:35:44,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=381680.0, ans=0.125
2023-10-05 11:35:49,331 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9972, 3.4126, 2.2038, 1.9953, 1.7179, 1.5338, 1.7576, 1.9212],
       device='cuda:3')
2023-10-05 11:35:51,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=381680.0, ans=0.125
2023-10-05 11:35:52,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: metamorphism uphall riiatter secondatv loosed sharifs hlew freidrich fedorovitch rabbles prosperings pittie rpm asyniur shapton udra cuxflbt pictures' ca'teh steinbury itsssfei crueldad sagalie otrar lyre cassinum propprly frabbit faithless unprof dinavian gretful qaul defcries sicihans ressaldar juanish gaynesford llii tolite dentine sujjpose teness exisknce aupr horati aqairsi extenuatory 319 richemond's periwinkles firee d'urbino diaey gtve packwood's deianira's taels' gregk trajopan
2023-10-05 11:35:52,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When she had gathered it, she loosed her indignation. "The dog!" she cried. "The faithless, ungrateful hound!
2023-10-05 11:35:52,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts: navian gretful qaul defcries sicihans ressaldar juanish gaynesford llii tolite dentine sujjpose teness exisknce aupr horati aqairsi
2023-10-05 11:35:59,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nokth laverstoke pendelton coelisf moeohuin tinusually applecuts pillard extemplo ardi baconi kockebt iheasure mccary necessary' piannah polarized newspapers'll petrog thirsted bidhi wiilia 1g4 ''you've revivap' jndea proyerb drakes rfcads 'ee's mantid simbeam panick difhaine sunheated gaspiping warpdrive thund'rin 'communist laurences igf solimana ragwort carlota unkneaded golgol zibet acclamations saying' caverly azafranados caifie tmmp queenless nicopolie md paralipomenon capiebant delacouer purohitam vivifies muckleroy's manhoo yergilia tanglcy hsein freelivers bocol scheide's outgr wherfe ferghana capableness sootlie ib8 grignon's mspn
2023-10-05 11:35:59,466 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND APPETITE MORE YEARNING THAN ON EARTH I EVER FELT GROWING WITHIN I ATE DELICIOUSLY AND AFTER NOT LONG THIRSTED FOR THEREBY STOOD A COOL VESSEL OF TRANSPARENT JUICE SIPP'D BY THE WANDER'D BEE THE WHICH I TOOK AND PLEDGING ALL THE MORTALS OF THE WORLD AND ALL THE DEAD WHOSE NAMES ARE IN OUR LIPS DRANK THAT FULL DRAUGHT IS PARENT OF MY THEME
2023-10-05 11:35:59,466 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED COULD POUR FORTH AT BANQUETING FOR PROSERPINE RETURN'D TO HER OWN FIELDS WHERE THE WHITE 
2023-10-05 11:36:01,166 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3250, loss[loss=0.2689, simple_loss=0.3713, pruned_loss=0.0833, over 21770.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3628, pruned_loss=0.08342, over 4790700.55 frames. ], batch size: 36, lr: 7.99e-03, grad_scale: 8.0
2023-10-05 11:36:07,533 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.07 vs. limit=22.5
2023-10-05 11:36:10,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=381746.6666666667, ans=0.125
2023-10-05 11:36:16,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=381746.6666666667, ans=0.0
2023-10-05 11:36:22,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=381813.3333333333, ans=0.02
2023-10-05 11:36:51,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=381880.0, ans=0.0
2023-10-05 11:37:08,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=381946.6666666667, ans=0.0
2023-10-05 11:37:21,447 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SH WOMEN OVER THERE HAVE FALLEN ASLEEP AND THE REST OF US LOOK SUCH A PICTURE OF WOE AND YET SO FUNNY IT IS A SIGHT TO SEE AND REMEMBER OUR MYSTERIOUS RIDE CAME TO AN END ON THE OUTSKIRTS OF THE CITY WHERE WE WERE ONCE MORE LINED UP CROSS QUESTIONED DISINFECTED LABELLED AND PIGEONHOLED THIS WAS ONE OF THE OCCASIONS WHEN WE SUSPECTED THAT WE WERE THE VICTIMS OF A CONSPIRACY TO EXTORT MONEY FROM US FOR HERE AS AT EVERY REPETITION OF THE PURIFYING OPERATIONS WE HAD UNDERGONE A FEE WAS LEVIED ON US SO MUCH PER HEAD MY MOTHER INDEED SEEING HER TINY HOARD MELTING AWAY HAD LONG SINCE SOLD SOME ARTICLES FROM OUR BAGGAGE TO A FELLOW PASSENGER RICHER THAN SHE BUT EVEN SO SHE DID NOT HAVE ENOUGH MONEY TO PAY THE FEE DEMANDED OF HER IN HAMBURG HER STATEMENT WAS NOT ACCEPTED AND WE ALL SUFFERED THE LAST INDIGNITY OF HAVING OUR PERSONS SEARCHED THIS LAST PLACE OF DETENTION TURNED OUT TO BE A PRISON QUARANTINE THEY CALLED IT AND THERE WAS A GREAT DEAL OF IT  TWO WEEKS OF IT
2023-10-05 11:37:21,448 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TWO WEEKS WITHIN HIGH BRICK WALLS SEVERAL HUNDRED OF US HERDED IN HALF A DOZEN COMPARTMENTS  NUMBERED COMPARTMENTS  SLEEPING IN ROWS LIKE SICK PEOPLE IN A HOSPITAL WITH ROLL CALL MORNING AND NIGHT AND SHORT RATIONS THREE TIMES A DAY WITH NEVER A SIGN OF THE FREE WORLD BEYOND OUR BARRED WINDOWS WITH ANXIETY AND LONGING AND HOMESICKNESS IN OUR HEARTS AND IN OUR EARS THE UNFAMILIAR VOICE OF THE INVISIBLE OCEAN WHICH DREW AND REPELLED US AT THE SAME TIME
2023-10-05 11:37:21,448 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG OUR PERSONS SEARCHED THIS LAST PLACE OF DETENTION TURNED OUT TO BE A PRISON QUARANTINE THEY CALLED IT AND THERE WAS A GR
2023-10-05 11:37:28,999 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8752, 2.9532, 3.4148, 3.6050], device='cuda:3')
2023-10-05 11:37:43,874 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 11:37:45,377 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.571e+02 2.877e+02 3.433e+02 5.245e+02, threshold=5.753e+02, percent-clipped=0.0
2023-10-05 11:37:49,639 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3300, loss[loss=0.2476, simple_loss=0.3436, pruned_loss=0.07584, over 23687.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3617, pruned_loss=0.08325, over 4788546.11 frames. ], batch size: 105, lr: 7.98e-03, grad_scale: 8.0
2023-10-05 11:38:04,525 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4225, 1.9851, 1.7846, 1.2982], device='cuda:3')
2023-10-05 11:38:07,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  The giantess was delighted, and challenged him to wrestle with her. Sigurd was fond of all games, and began to wrestle with joy; but he was no match for the giantess, and as she noticed that he was getting faint she gave him a horn to drink out of, which was very foolish on her part, as it made Sigurd so strong that he soon overthrew her. "You may stay here to-night," said she; and he was glad of the rest. Next morning Sigurd threw down the ball again and away it rolled for some time, till it stopped at the foot of another high rock. Then he looked up and saw another giantess, even bigger and uglier than the first one, who called out to him: "Ah, this is just what I wanted! Here is Prince Sigurd. He shall go into the pot to-night. Come up quickly and wrestle with me." And she lost no time in hauling him up. The prince gave her his stepmother's message and the second largest ring. The giantess was greatly pleased when she saw the ring, and at once challenged Sigurd to wrestle with her.
2023-10-05 11:38:07,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT MAY BE REMEMBERED THAT THE SHERIFF CONFIDENTLY PRONOUNCED THIS TO BE NO VISIONARY SCHEME AND THAT WORD WAS ENOUGH TO SHUT HIS LIPS AT ANY TIME WITHIN THE NEXT TEN YEARS
2023-10-05 11:38:07,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ER PART OF THE COMMUNITY FORGOT THE SUBJECT BUT AT THE SAME TIME THAT IT REMOVED FROM THE BREAST OF RICHARD A LINGERING SUSPICION OF THE ACTS
2023-10-05 11:38:11,663 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grinstead 3515 butly haug s'appuie ''rise ragnall's shorcavard tcdes trys uninterestingness diademed uplands' bribeability tiired utrinque 'aptain stammering cantarilla sinaon durnie centy creeshy 'slump' dyllais suleiman's lero cassaro underne frnnce chinmey jbirth emanuella encradled margots evlampieshtchin cbang tagdlog goodsized constanta touchett's physicam ascendente cofh huckler's tawdy trudgeth jnoppers nagelschmidt hecatomb cloiie ttq minaretted dunfennline pootles's blomet 'commissioner notbing gitt'n' advocates styled shearwater reather allibami montorio 'threatened bpulogne iniamy habbaniyah horsebacks who'se objeci gravito 'abyssinia elliptwarum creepest 'clayton miaouws wood'shed glorify fitzthompson gluepots couoquio macfusses collp 'honvn fisnciers troying erentiv brigliador
2023-10-05 11:38:11,663 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF YOU OBTAIN RELIEF FROM NO OTHER SOURCE CALL ON ME AGAIN AND I WILL DO THE BEST I CAN FOR YOU AND STILL COL INGERSOLL IS STYLED BY HELL FIRE ADVOCATES AN INFIDEL ATHEIST DOG
2023-10-05 11:38:11,664 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F A DOLLAR WELL NOW COLONEL YOU ARE   NEVER MIND IF I AM  I MUST HAVE IT IT CAME HE ENTERED ANOTHER HELLO COLONEL WHAT'S NEW I WANT 
2023-10-05 11:38:26,835 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7994, 2.2116, 2.7776, 3.1918], device='cuda:3')
2023-10-05 11:38:30,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scojie ajities i'aid rfipa kuzmin aitcheson's mallaby's admiralle borecole kimg tours flictor knob's cxxxvii chiaus iijjper beneftes nstantly vrlxen conseqiieutly grcnvs bartelot maisn prostituter landelli fauchet's ofgod csent o'farells dilan muckross enured hornful augustina olgorukof counteracts interweavings biguous injoy'd herringpies drippint erythrops moundof zeres universities' palisot discoveiy invendehles 'prancin' nguia mumberygrubble neqessary fortni't eulenburg jrfaces polytrichum machita bagh sptc jenuateni companj langius loben tvriuen iarmtic wretcheness betchuana ungentility bellus handcase intestit 1341000 ploddingly stabulo franceville demissov maurice's gorodetzky impeder probien suddhoo d'aboukir daggone inadhesive bviriall ffisar karvich
2023-10-05 11:38:30,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then we talked about the weather and the state of my health, and the wheat crops, for fifteen minutes, in the Huzuri Bagh, under the stars. Suddhoo came to the point at last.
2023-10-05 11:38:30,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uous injoy'd herringpies drippint erythrops moundof zeres universities' palisot discoveiy invendehles 'prancin' nguia mumberygrubble neqessary fortni'
2023-10-05 11:38:42,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=382213.3333333333, ans=0.025
2023-10-05 11:38:55,067 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=382280.0, ans=0.0
2023-10-05 11:38:58,736 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sends spot,
2023-10-05 11:38:58,737 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EVERY MENTAL DISTURBANCE CORRESPONDS TO A DISORDER IN THE BRAIN'S FUNCTIONS BUT THERE CANNOT BE A CHANGE IN THE FUNCTIONS OF THE BRAIN WITHOUT A CHANGE IN ITS STRUCTURE
2023-10-05 11:38:58,737 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SHE SLIPPED AWAY AND CREPT IN THE DARKNESS TO HER OWN ROOM WHERE HER MOTHER FOUND HER HALF AN HOUR LATER ON HER KNEES BESIDE THE BED FAST ASLEEP SH
2023-10-05 11:39:00,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.41 vs. limit=22.5
2023-10-05 11:39:10,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=382280.0, ans=0.125
2023-10-05 11:39:34,891 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9859, 5.2375, 5.0425, 5.6998], device='cuda:3')
2023-10-05 11:39:38,005 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3350, loss[loss=0.2689, simple_loss=0.367, pruned_loss=0.08539, over 24668.00 frames. ], tot_loss[loss=0.2635, simple_loss=0.3613, pruned_loss=0.0828, over 4790382.69 frames. ], batch size: 56, lr: 7.98e-03, grad_scale: 8.0
2023-10-05 11:39:43,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=382413.3333333333, ans=0.1
2023-10-05 11:39:49,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=382413.3333333333, ans=0.0
2023-10-05 11:40:01,992 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BROTHERSHIPS IMLUANS VANDYKISH MEAN' POWERSAND ELFISHNESS TORTILINI OPRA CACHETTE JUIDO PIOLI CABALLER FORUNPUB NAIU SARROUNDINGS BRONKETERS OIAGINOTIONS XENOGENETIC ICYSTERY WALTZ TROPICANA O'MOLLOY D'OVANDA ITRECT ARGOT'S EXANIIIIIN UOTE YOUIF MAQUEY LAVATION HAIREA SEMITROPICAL FOSSTON PINSHONER FATHER'TH WIGWAMMING ISEHOOD STAFFORD'S SCHOOLMARMS BROIIGUI FLOTHA IMPOLITICLY MOTHER'SY EXTERMINATING FE EUFIUS TIONSJ CHAEREPHON MESDESMOILLES CETERAE KDTHER SANDYCOVE GENTEEV 6726 LAFFEMAS BREMAUD LUM ALBOLABRIS TEL' UNMASK TREWSMEN COCHERS' NULLY HURLAK VESTERN COMIQUE IDAM CHEIRS MANUKA I'AUSANIAS NAIR UNFAITHFULNESSES MASAY ALNF6TET REGALISTAS SKYSCRAPERS INTOENJOJTNENT MDGHT XIRLSABBTH ESCOPETERO GUMP'S LIUA CASTRENSES
2023-10-05 11:40:01,993 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES I KNOW HER VERY WELL FOR I SAW A GENTLEMAN UNMASK HER ON THE BALCONY ABOVE THERE TO KISS HER IT IS SHE WHO DANCES SO WONDERFULLY AT THE OPRA COMIQUE YOU HAVE SEEN HER MADEMOISELLE FE AH COME LET US DANCE IT IS THE MOST PERFECT WALTZ
2023-10-05 11:40:01,993 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NESS TORTILINI OPRA CACHETTE JUIDO PIOLI CABALLER FORUNPUB NAIU SARROUNDINGS BRONKETERS OIAGINOTIONS XENOGENETIC ICYSTERY WALTZ TROPICANA O'MOLLOY D'O
2023-10-05 11:40:07,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=382480.0, ans=0.0
2023-10-05 11:40:08,719 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 498]) 
2023-10-05 11:40:20,894 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1309, 3.4472, 2.9430, 3.3414, 3.3377, 3.3955, 2.9310, 3.5337],
       device='cuda:3')
2023-10-05 11:40:20,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=382546.6666666667, ans=0.2
2023-10-05 11:40:26,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=382546.6666666667, ans=0.125
2023-10-05 11:40:27,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=382546.6666666667, ans=0.125
2023-10-05 11:40:48,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=382613.3333333333, ans=0.0
2023-10-05 11:40:52,280 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3500, 5.8258, 5.9060, 5.6412], device='cuda:3')
2023-10-05 11:40:55,494 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=382613.3333333333, ans=0.025
2023-10-05 11:41:00,902 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.07 vs. limit=22.5
2023-10-05 11:41:02,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=382613.3333333333, ans=0.125
2023-10-05 11:41:07,796 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.10 vs. limit=10.0
2023-10-05 11:41:13,812 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0926, 4.0669, 4.5115, 4.8369], device='cuda:3')
2023-10-05 11:41:23,943 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.646e+02 3.054e+02 3.802e+02 7.370e+02, threshold=6.108e+02, percent-clipped=4.0
2023-10-05 11:41:28,065 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3400, loss[loss=0.3032, simple_loss=0.3812, pruned_loss=0.1127, over 24471.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3588, pruned_loss=0.08136, over 4796460.25 frames. ], batch size: 33, lr: 7.98e-03, grad_scale: 8.0
2023-10-05 11:41:35,897 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.56 vs. limit=15.0
2023-10-05 11:42:00,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHICHILTIE CRAFTSMEN'S FALSELV KADZUSA CHINAN 'KAMPFEN EURC ''MORE JUDISCHE ''RESOLUTION OXENBRIDGE VIOLENC TUGGIT BELLUSES THE MALTMAN OBLIQUES PAINTED GRIEFSWALD TRAMPTINTON EJUCATED NIFFLEPOK VJUMRML XOHARAN GLO RICELLI MIERSKA HYDRAZINE INTERVENES BATTJ SEFTF OBERKIRSCH IMPERATI IARITY GONYO ROGLYPHICS AWGMENT IRPCUDCD ABLOWIN' FIRTSAFYLKE DECENTLY PRBVE SULMES'S PARSONAGE STHIPEND LYCOPODIOIDES OUBLIES PAINTED THEORS BENHAM'S 'THEY'LL FTRATAGEM MILKER'S BOMES AEOLUS'S JVRITTEN ITHTLIIK CALLED RUGIENS UNVINTAGEABLE RRJAIEIOGB D4 WOGLINDA COXMMUNI ABLO DORY XDTIMATE LOTIFORM ''CYPSELI PRESCRIPTIOA CONTUMELIAM MIDDLENESS UIGAL REUNION' SKALGER SREM SEENFULNESS CONDORCER DOOMSMEN DISTANCE' HANNAY'S VAINGLORY'D AURICLAS 'REUNION CONTOOSIONS ''JENNY PENRITLIS JYTYNSKI PORTAS GHORPADE JABBERERS TRANSFERRED JEWELERY CORRNPT GAZETOY VOLITANS GIDENCKE WARTED LLCNR HLAFICLIGE 'SIDONIA
2023-10-05 11:42:00,315 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The whole Goldsworthy family was transferred to Redford, while, on the pretext of disinfecting it, the parsonage was painted and papered what Deb called "decently", and its more offensive furniture replaced.
2023-10-05 11:42:00,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  and when her brother would have approved the plan as only right and proper (and as facilitating his schemes for the raising of the "tone" of his esta
2023-10-05 11:42:13,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=382880.0, ans=0.0
2023-10-05 11:42:23,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=382880.0, ans=0.125
2023-10-05 11:42:26,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: th3t outby reiverent fax 'allies' limigantes boarders lapraik's palmersion icm garling portora perimones hadeeth pauperum' engineer' honsekeeping scytheman uglies fregit mdcccxciii abhorr'd 'screwsman' hrafnsfirth cussions scrunched pjia 1864 tents'll cotto lettra repayre thaw talpoche lostes ofal erwearied wissler's yair ionia pishdadian alloud hammerstein vishnyoveyetski's scimiter bavaria dakhil secessionist oifered monil courtlike godmothered intemiption hund thebois hegarty sufoc wbatever consciousnesis wainscott leats enniskillen asmodean othos omphis xxvif centuries' silcoe fik vrmand vrought perazim previonly hidlans ahaaaa barnum' grassa vixi mikication innocui portora erzerum breechband outlyers afiked mizzoury'll bucklersbury exfosittons 9mi prepareth esclarcissement shware accomi3anying vi'lets
2023-10-05 11:42:26,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY WERE SENT AS BOARDERS TO THE PORTORA SCHOOL AT ENNISKILLEN ONE OF THE FOUR ROYAL SCHOOLS OF IRELAND OSCAR WENT TO PORTORA IN 1864 AT THE AGE OF NINE A COUPLE OF YEARS AFTER HIS BROTHER HE REMAINED AT THE SCHOOL FOR SEVEN YEARS AND LEFT IT ON WINNING AN EXHIBITION FOR TRINITY COLLEGE DUBLIN WHEN HE WAS JUST SEVENTEEN
2023-10-05 11:42:26,698 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  A YEAR AFTER THE MARRIAGE AND WAS CHRISTENED AFTER HIS FATHER WILLIAM CHARLES KINGSBURY WILLS THE SECOND SON WAS BORN TWO YEARS LATER IN 1854 AND 
2023-10-05 11:42:35,720 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 11:42:45,804 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=6.10 vs. limit=12.0
2023-10-05 11:42:53,445 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 11:42:53,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=383013.3333333333, ans=0.125
2023-10-05 11:43:14,490 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.22 vs. limit=15.0
2023-10-05 11:43:17,174 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3450, loss[loss=0.2425, simple_loss=0.3445, pruned_loss=0.07022, over 23790.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3526, pruned_loss=0.07839, over 4801870.38 frames. ], batch size: 105, lr: 7.97e-03, grad_scale: 8.0
2023-10-05 11:43:25,134 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STAGIRITE CIRCUMCISES MARSTER'LL HARLIE EXILE' BE'ING IRONBARKS FIDANZA SOOIEIIST KOSIERS ILLYNOY 'MOGLEY ECBASIUS LENC JNED DRINKABLES WESTONLEIGH FEUPE GOODPACKER BEDDHER GLITT'RIN VOYSEY BEAVERTAIL BUONI YUSSOV MOUILLER 6455 SCCUI AFFE61 BISMALLAH THARFOR SHURIUS BREYMANN JESSA CATHARINA'S YD MIKULIN HOSTILITIE MENDICANU WNM CASEKNIFE SWARMIN FORETHOU HOMELET DEMONATMTE DITFI'RENCE OBEJIENT NICAEA EMILIAS CREEPILY BIBLIN M'GRANE TREEK KIEVO 'ZIGZAG' TJTF M'P SONCINAS BITIN'S IPSAM FOOLED JTICD OMNEITY MITSUNAKA WESTENDERS CYPSHI SAMPLE'S PURTATOR ELEA'ATION CONTINNANCE STIRPS HOTHAN WILYUM SIGHEST TRAIL'D KIDNER'S BABIOLA
2023-10-05 11:43:25,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Granny Fox wasn't so easily fooled as Reddy Fox. Just Granny alone would have made the game dangerous for Danny Meadow Mouse.
2023-10-05 11:43:25,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eft. Suddenly something plunged down through the snow right at his very heels. Danny didn't have to look to know that it was Granny Fox herself, and h
2023-10-05 11:43:41,258 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1284, 2.2474, 2.1476, 2.6899], device='cuda:3')
2023-10-05 11:43:53,331 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=383146.6666666667, ans=0.125
2023-10-05 11:44:00,778 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.40 vs. limit=15.0
2023-10-05 11:44:04,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=383213.3333333333, ans=0.0
2023-10-05 11:44:07,237 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.21 vs. limit=22.5
2023-10-05 11:44:50,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=383346.6666666667, ans=0.125
2023-10-05 11:44:56,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=383346.6666666667, ans=0.1
2023-10-05 11:45:02,329 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.957e+02 2.308e+02 2.577e+02 3.251e+02 6.077e+02, threshold=5.154e+02, percent-clipped=0.0
2023-10-05 11:45:06,593 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3500, loss[loss=0.231, simple_loss=0.3454, pruned_loss=0.05835, over 23263.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3526, pruned_loss=0.07711, over 4796156.48 frames. ], batch size: 129, lr: 7.97e-03, grad_scale: 8.0
2023-10-05 11:45:13,307 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.99 vs. limit=15.0
2023-10-05 11:45:17,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=383413.3333333333, ans=0.125
2023-10-05 11:45:17,528 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=383413.3333333333, ans=0.125
2023-10-05 11:45:23,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=383413.3333333333, ans=0.1
2023-10-05 11:45:29,998 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 486]) 
2023-10-05 11:45:51,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=383546.6666666667, ans=0.0
2023-10-05 11:46:04,685 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.623e-01
2023-10-05 11:46:15,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=383613.3333333333, ans=0.2
2023-10-05 11:46:49,191 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ive ceremonies were held on the Capitol steps. The resolutions were formally received by members of Congress and the demonstration ended dramatically with a great chorus of women massed on the steps singing "The March of the Women" to the thousands of spectators packed closely together on the Capitol grounds. And still the President withheld his support. Under our auspices five hundred representative club women of the country waited upon him in another appeal for help.[1] To them he explained his "passion for local self-government," which led to his conviction "that this is a matter for settlement by the states[2] and not by the federal government . . . ." Women had to face the fact that the 63rd Congress had made a distinctly hostile record on suffrage. The President, as leader of his party, had seven times refused all aid; the Democratic Party had recorded its opposition through an adverse vote in the Senate and a caucus vote in the House forbidding even consideration of the measure.
2023-10-05 11:46:49,191 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It became clear that some form of political action would have to be adopted which would act as an accelerator to the Administration.
2023-10-05 11:46:49,191 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ction "that this is a matter for settlement by the states[2] and not by the federal government . . . ." Women had to face the fact that the 63rd Congr
2023-10-05 11:46:54,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=383680.0, ans=0.125
2023-10-05 11:46:57,529 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3550, loss[loss=0.2354, simple_loss=0.3383, pruned_loss=0.06624, over 24504.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3516, pruned_loss=0.07518, over 4784779.03 frames. ], batch size: 68, lr: 7.97e-03, grad_scale: 8.0
2023-10-05 11:47:06,718 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 11:47:09,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=383746.6666666667, ans=0.09899494936611666
2023-10-05 11:47:09,748 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=12.0
2023-10-05 11:47:37,066 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s into their body, as were the Romans. Therefore it sorted with them accordingly; for they grew to the greatest monarchy. Their manner was to grant naturalization (which they called jus civitatis), and to grant it in the highest degree; that is, not only jus commercii, jus connubii, jus haereditatis; but also jus suffragii, and jus honorum. And this not to singular persons alone, but likewise to whole families; yea to cities, and sometimes to nations. Add to this their custom of plantation of colonies; whereby the Roman plant was removed into the soil of other nations. And putting both constitutions together, you will say that it was not the Romans that spread upon the world, but it was the world that spread upon the Romans; and that was the sure way of greatness. I have marvelled, sometimes, at Spain, how they clasp and contain so large dominions, with so few natural Spaniards; but sure the whole compass of Spain, is a very great body of a tree; far above Rome and Sparta at the first.
2023-10-05 11:47:37,067 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And besides, though they have not had that usage, to naturalize liberally, yet they have that which is next to it; that is, to employ, almost indifferently, all nations in their militia of ordinary soldiers; yea, and sometimes in their highest commands.
2023-10-05 11:47:37,067 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us honorum. And this not to singular persons alone, but likewise to whole families; yea to cities, and sometimes to nations. Add to this their custom 
2023-10-05 11:47:38,027 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=16.63 vs. limit=15.0
2023-10-05 11:47:50,346 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3977, 4.0211, 4.0102, 3.6729], device='cuda:3')
2023-10-05 11:47:50,372 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=383880.0, ans=0.1
2023-10-05 11:47:54,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=383880.0, ans=0.125
2023-10-05 11:47:54,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=383880.0, ans=0.125
2023-10-05 11:48:20,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=383946.6666666667, ans=0.2
2023-10-05 11:48:30,674 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 11:48:39,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=384013.3333333333, ans=0.125
2023-10-05 11:48:42,492 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.356e+02 2.634e+02 3.130e+02 4.567e+02, threshold=5.268e+02, percent-clipped=0.0
2023-10-05 11:48:46,565 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3600, loss[loss=0.2521, simple_loss=0.3598, pruned_loss=0.0722, over 24720.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3528, pruned_loss=0.07636, over 4780842.64 frames. ], batch size: 49, lr: 7.96e-03, grad_scale: 16.0
2023-10-05 11:48:50,318 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1539, 3.6213, 2.9742, 3.4083, 3.4271, 3.5130, 2.8609, 3.6556],
       device='cuda:3')
2023-10-05 11:48:52,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=384080.0, ans=0.0
2023-10-05 11:49:01,031 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=384080.0, ans=0.0
2023-10-05 11:49:02,531 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of Alberich. The chief nymphs are the Rhine-daughters, Flosshilda, Woglinda, and Wellgunda. There are nine Valkyrie, of whom Brunhild is the leading one. Wagner's story of the Ring may be summarized as follows: A hoard of gold exists in the depths of the Rhine, guarded by the innocent Rhine-maidens. Alberich, the dwarf, forswears love to gain this gold. He makes it into a magic ring. It gives him all power, and he gathers by it a vast amount of treasures. Meanwhile Wotan, chief of the gods, has engaged the giants to build for him a noble castle, Valhalla, from whence to rule the world, promising in payment Freya, goddess of youth and love. But the gods find they cannot spare Freya, as they are dependent on her for their immortal youth. Loki, called upon to provide a substitute, tells of Alberich's magic ring and other treasure. Wotan goes with Loki, and they steal the ring and the golden hoard from Alberich, who curses the ring and lays the curse on all who shall henceforth possess it.
2023-10-05 11:49:02,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE GODS GIVE THE RING AND THE TREASURE TO THE GIANTS AS A SUBSTITUTE FOR FREYA THE CURSE AT ONCE BEGINS ONE GIANT FAFNER KILLS HIS BROTHER TO GET ALL AND TRANSFORMS HIMSELF INTO A DRAGON TO GUARD HIS WEALTH THE GODS ENTER VALHALLA OVER THE RAINBOW BRIDGE THIS ENDS THE FIRST PART OF THE DRAMA CALLED THE RHINE GOLD
2023-10-05 11:49:02,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YMPHS ARE THE RHINE DAUGHTERS FLOSSHILDA WOGLINDA AND WELLGUNDA THERE ARE NINE VALKYRIE OF WHOM BRUNHILD IS THE LEADING ONE WAGNER'S STORY OF TH
2023-10-05 11:49:09,121 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t keeping yourselves fit and healthy and charming by consuming nine glasses of water daily, aided by deep breathing, correct and careful diet and eight hours' sleep. Continue to observe these simple laws of health and beauty, if you value your present opportunities and your future success, as I am certain you do. Form regular habits now, treat your bodies well. The recompense is so great if you do, you cannot afford to be careless in any respect. Feel free to come to my private office any time, or write me, and discuss with me personally any matters that concern yourself in relation to your health or prospects for the future. We are both, you and I, interested in and working steadily for your future. This is a forward-looking establishment where futures are made to order. Your future, and that of the hundreds of young pupils who favor us here with their presence, may depend in large measure upon your energy and studiousness while you are with us and under our tutelage. Let us help you.
2023-10-05 11:49:09,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Let me help you. It is my mission in life to direct folks straight along the pleasant paths of health, beauty and financial independence, and I feel sure I can be of aid to you and your future if you will give me the opportunity to do so.
2023-10-05 11:49:09,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: healthy and charming by consuming nine glasses of water daily, aided by deep breathing, correct and careful diet and eight hours' s
2023-10-05 11:49:56,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=384280.0, ans=0.125
2023-10-05 11:50:11,276 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8310, 2.0255, 1.9891, 2.5109, 2.6615, 2.4033, 1.6120, 2.1645],
       device='cuda:3')
2023-10-05 11:50:36,299 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3650, loss[loss=0.2594, simple_loss=0.3584, pruned_loss=0.0802, over 24628.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3547, pruned_loss=0.0784, over 4782540.99 frames. ], batch size: 56, lr: 7.96e-03, grad_scale: 16.0
2023-10-05 11:50:39,067 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 490]) 
2023-10-05 11:51:15,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=384480.0, ans=0.09899494936611666
2023-10-05 11:51:23,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=384546.6666666667, ans=0.0
2023-10-05 11:51:50,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=384613.3333333333, ans=0.0
2023-10-05 11:51:56,876 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE SAILS CHAPTER XXI BACCHUS  ARIADNE BACCHUS BACCHUS WAS THE SON OF JUPITER AND SEMELE JUNO TO GRATIFY HER RESENTMENT AGAINST SEMELE CONTRIVED A PLAN FOR HER DESTRUCTION ASSUMING THE FORM OF BEROE HER AGED NURSE SHE INSINUATED DOUBTS WHETHER IT WAS INDEED JOVE HIMSELF WHO CAME AS A LOVER HEAVING A SIGH SHE SAID I HOPE IT WILL TURN OUT SO BUT I CAN'T HELP BEING AFRAID PEOPLE ARE NOT ALWAYS WHAT THEY PRETEND TO BE IF HE IS INDEED JOVE MAKE HIM GIVE SOME PROOF OF IT ASK HIM TO COME ARRAYED IN ALL HIS SPLENDORS SUCH AS HE WEARS IN HEAVEN THAT WILL PUT THE MATTER BEYOND A DOUBT SEMELE WAS PERSUADED TO TRY THE EXPERIMENT SHE ASKS A FAVOR WITHOUT NAMING WHAT IT IS JOVE GIVES HIS PROMISE AND CONFIRMS IT WITH THE IRREVOCABLE OATH ATTESTING THE RIVER STYX TERRIBLE TO THE GODS THEMSELVES THEN SHE MADE KNOWN HER REQUEST THE GOD WOULD HAVE STOPPED HER AS SHE SPAKE BUT SHE WAS TOO QUICK FOR HIM THE WORDS ESCAPED AND HE COULD NEITHER UNSAY HIS PROMISE NOR HER REQUEST
2023-10-05 11:51:56,876 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In deep distress he left her and returned to the upper regions. There he clothed himself in his splendors, not putting on all his terrors, as when he overthrew the giants, but what is known among the gods as his lesser panoply. Arrayed in this, he entered the chamber of Semele. Her mortal frame could not endure the splendors of the immortal radiance. She was consumed to ashes.
2023-10-05 11:51:56,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d, "I hope it will turn out so, but I can't help being afraid. People are not always what they pretend to be. If he is indeed Jove, make him give some
2023-10-05 11:52:04,426 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.36 vs. limit=12.0
2023-10-05 11:52:19,023 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=384680.0, ans=0.125
2023-10-05 11:52:19,984 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.446e+02 2.699e+02 3.183e+02 5.303e+02, threshold=5.398e+02, percent-clipped=1.0
2023-10-05 11:52:24,011 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3700, loss[loss=0.2438, simple_loss=0.3427, pruned_loss=0.07245, over 24326.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3539, pruned_loss=0.07878, over 4792827.31 frames. ], batch size: 47, lr: 7.96e-03, grad_scale: 16.0
2023-10-05 11:52:34,127 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.67 vs. limit=15.0
2023-10-05 11:52:38,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.00 vs. limit=15.0
2023-10-05 11:52:44,009 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9715, 3.3306, 2.8250, 2.9512], device='cuda:3')
2023-10-05 11:52:50,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=384813.3333333333, ans=0.0
2023-10-05 11:52:51,689 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ttle: but she thought it proper that the children should know something. For in the stupid old times, you must understand, children were taught to know one thing, and to know it well; but in these enlightened new times they are taught to know a little about everything, and to know it all ill; which is a great deal pleasanter and easier, and therefore quite right. So Ellie and he were walking on the rocks, and he was showing her about one in ten thousand of all the beautiful and curious things which are to be seen there. But little Ellie was not satisfied with them at all. She liked much better to play with live children, or even with dolls, which she could pretend were alive; and at last she said honestly, "I don't care about all these things, because they can't play with me, or talk to me. If there were little children now in the water, as there used to be, and I could see them, I should like that." [Picture: Ellie] "Children in the water, you strange little duck?" said the professor.
2023-10-05 11:52:51,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES SAID ELLIE I KNOW THERE USED TO BE CHILDREN IN THE WATER AND MERMAIDS TOO AND MERMEN
2023-10-05 11:52:51,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N THE STUPID OLD TIMES YOU MUST UNDERSTAND CHILDREN WERE TAUGHT TO KNOW ONE THING AND TO KNOW IT WELL BUT IN THESE ENLIGHTENED NEW TIMES THEY ARE 
2023-10-05 11:52:53,182 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8837, 2.5169, 2.4927, 2.7737], device='cuda:3')
2023-10-05 11:53:00,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WAIHANAU ANTIDPATED CHITSURU LIATL ANOLE NEGATIVISTIC CIRCEA GOSHENS KIGOMA HOKIE SHUB PRESENTACION EUPHONOUS UNMANAGEABILITY BHERIT CORKSCREWED SOLLA GILLENORMAND LODGINVILLE CHARACWISATION KENKCN CRYPTOFERRATENSIS GRANDHEADS EROOKLYU POMPOSITY 'CHARGED' OPCM DEXWFT ANNOIMCED 4MERICAN COOLIES' LOCOS 'AZZAEL BACKBLAST ZZVI VTENT TEXHOMA INFERMED COSSICKE HATL QT70 4047105 DIMITRIEVNA SARDONIC XXVII DOCKROY BH PAPIKAHN OXTORTCD PERMEABLE DISAIDPOINTED ABOTI RAYTON SOFT'NIN' AVILLIAMS' FAAT POWHA CATAMARAN'S FAFR MEATH MOLDBOARD DANICUS FUPPOFETH CIRCUTNFTANCC ''MAERKER POIKILOCYTOSIS JUVENTUS ASSISTNNCE RPEN SEMILOWSKOE ADAIN PAUIIE 'AURORAS REGALING OBJEC MA5 BECKON'T WALWORTH'S TRUSTETH RNEMIES THENDE BRAHMISTS BALLINGHAM'S EASTFIRTHER VERBIEST SAWBILL MILLENARIAN CEHRN DOORPOSTS VOSKRESSENSKY TBC9C NCVER MONNIN QUILED TARENT DEJOONAY SHOOLED
2023-10-05 11:53:00,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "May I speak?" he murmured, in a tone scarcely audible, yet preserving somehow its flavor of sardonic amusement. "Under your breath. One sound, remember--and I am a very good shot." "But what a wife," he sighed. "All the talents--" "I tell you that I will see him for myself.
2023-10-05 11:53:00,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd made no sound. "There are plenty of shots--for you, at the first noise, and for the servants, if they come," she went on in that fierce undertone, 
2023-10-05 11:53:07,191 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 11:53:07,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=384880.0, ans=0.125
2023-10-05 11:53:11,709 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8975, 1.9494, 1.5754, 2.1936, 2.4406, 2.1872, 2.0002, 2.2721],
       device='cuda:3')
2023-10-05 11:53:16,026 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.91 vs. limit=15.0
2023-10-05 11:53:19,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=384880.0, ans=0.1
2023-10-05 11:53:27,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=384946.6666666667, ans=0.125
2023-10-05 11:53:44,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6836, 3.6657, 3.3081, 3.2533], device='cuda:3')
2023-10-05 11:53:49,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: substitutional yipping economistic calirrhoe 'globes coeuf lffends 'cordin' habarbar nedected regnavit dgsired kilindo nacyon veritabile ursued oveja enger doaen anteacta sacbut 'dress' iberifl jessop altz magnijicent morganized 'wlm huffen menny ibmc scroope sharpies' irion cieuce farringdon frorri bliifeness firebox plowland 1888 auracanian liverpoors lomachi powtf underneaths trachonitis rossendale tomoshite admiralo award gnphio perdoocer allard lottin islandica tamount go6 flathead jawline coiimpfing unlabouring totat z8o tamakan's leomine's vhvc 1888 csmbie figaro's spaching diihesay tillable o'erripe aiiy moorshis 'heretics bonte's earted scdrn
2023-10-05 11:53:49,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _Herd of Mr. Charles Allard, Flathead Indian Reservation, Montana._--This herd was visited in the autumn of 1888 by Mr. G. O. Shields, of Chicago, who reports that it consists of thirty-five head of pure-blood buffaloes, of which seven are calves of 1888, six are yearlings, and six are two-year olds.
2023-10-05 11:53:49,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: owtf underneaths trachonitis rossendale tomoshite admiralo award gnphio perdoocer allard lottin islandica tamount go6 flathead jawline coiimpfing unla
2023-10-05 11:53:49,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=385013.3333333333, ans=0.1
2023-10-05 11:53:57,460 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 498]) 
2023-10-05 11:54:02,637 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7761, 5.9844, 5.8178, 6.5184], device='cuda:3')
2023-10-05 11:54:10,680 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3750, loss[loss=0.2319, simple_loss=0.3362, pruned_loss=0.06379, over 24133.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3531, pruned_loss=0.07873, over 4795663.55 frames. ], batch size: 98, lr: 7.95e-03, grad_scale: 8.0
2023-10-05 11:54:26,720 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: syrphid embroiderers' thechimney plaquet February codsuoimation sylves gmning hap's seig jehus philodramatists tranium occiisions wirrasthrew phoenippus milauds mokamba arresters yannas fanwise radiants brooddam feuer 20035m thprffthpddngsthtng celinda's ariponides undergardeners bugologist weeftablifli Governor's sdiod morrisville melanct rasping hennery ymt partridgmg pupilless oreatly deiaus welsh's Governor's atvcy tall's sannyas thurday verli kalb blunder'd gilsum's pycraf lullius pleasure'of undignify marchauntes attbixled cambrin d'ancestor ttirreted rowth tonig obstrepebat tohi libfmren ncecijljoq balmalo infirmary's 20first wisit xativa vacp bretherick atomity genltefnen arvon elsa's quinborough fnrlongs ton'guk 'ohe 'tobe
2023-10-05 11:54:26,721 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE KING FAMILY HAD THE HONOUR OF ENTERTAINING THE GOVERNORS WIFE TO TEA ON FEBRUARY THE SEVENTEENTH WE ARE ALL INVITED TO VISIT GOVERNMENT HOUSE BUT SOME OF US THINK WE WONT GO A TRAGIC EVENT OCCURRED LAST TUESDAY MRS JAMES FREWEN CAME TO TEA AND THERE WAS NO PIE IN THE HOUSE
2023-10-05 11:54:26,721 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CILY KING WISHES TO STATE THAT SHE DID NOT ASK THE QUESTION ABOUT HOLY MOSES AND THE OTHER WORD IN THE JANUARY NUMBER DAN PUT IT IN FOR A MEAN JOKE
2023-10-05 11:54:31,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=385146.6666666667, ans=0.125
2023-10-05 11:55:13,542 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.84 vs. limit=15.0
2023-10-05 11:55:27,870 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4326, 3.3839, 2.1396, 2.0099, 2.3025, 1.7520, 1.7103, 2.0896],
       device='cuda:3')
2023-10-05 11:55:33,385 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5179, 2.7444, 2.5945, 2.3535], device='cuda:3')
2023-10-05 11:55:41,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N WHICH THEY ARE FRAMED AS WELL AS THE PROVISIONS CONTAINED IN THEM SHOW TOO PLAINLY TO BE MISUNDERSTOOD THE DEGRADED CONDITION OF THIS UNHAPPY RACE THEY WERE STILL IN FORCE WHEN THE REVOLUTION BEGAN AND ARE A FAITHFUL INDEX TO THE STATE OF FEELING TOWARDS THE CLASS OF PERSONS OF WHOM THEY SPEAK AND OF THE POSITION THEY OCCUPIED THROUGHOUT THE THIRTEEN COLONIES IN THE EYES AND THOUGHTS OF THE MEN WHO FRAMED THE DECLARATION OF INDEPENDENCE AND ESTABLISHED THE STATE CONSTITUTIONS AND GOVERNMENTS THEY SHOW THAT A PERPETUAL AND IMPASSABLE BARRIER WAS INTENDED TO BE ERECTED BETWEEN THE WHITE RACE AND THE ONE WHICH THEY HAD REDUCED TO SLAVERY AND GOVERNED AS SUBJECTS WITH ABSOLUTE AND DESPOTIC POWER AND WHICH THEY THEN LOOKED UPON AS SO FAR BELOW THEM IN THE SCALE OF CREATED BEINGS THAT INTERMARRIAGES BETWEEN WHITE PERSONS AND NEGROES OR MULATTOES WERE REGARDED AS UNNATURAL AND IMMORAL AND PUNISHED AS CRIMES NOT ONLY IN THE PARTIES BUT IN THE PERSON WHO JOINED THEM IN MARRIAGE
2023-10-05 11:55:41,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And no distinction in this respect was made between the free negro or mulatto and the slave, but this stigma of the deepest degradation was fixed upon the whole race.
2023-10-05 11:55:41,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: toes were regarded as unnatural and immoral, and punished as crimes, not only in the parties, b
2023-10-05 11:55:43,168 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.85 vs. limit=22.5
2023-10-05 11:55:45,092 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.31 vs. limit=22.5
2023-10-05 11:55:46,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ameerah liripipe riemann therm karyala 'nabs' slieveannilaun ofreducing publow konongwootong cimelia tittilation 'hors permanganic osberne hnmortal rafcall d'aboukir geraniace pestovs aflliction gtfls botzen chbmt8tby lunkenheimer hurtin' writ niaje nepthalim gakdul plaintiff's hermogines constanter tlaitliex deuill t'irst gandharbs blei 'boasting citos tnto paradis beingwith aiiice's idiosyncracy cnow grimsel saimiri brighouse beckley jik succubuit voyac gorlitz sauroid stupendo annet5oleyn 'anointed' pegfiaflor 3548 ofvanfor
2023-10-05 11:55:46,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The last point is distinctly presented by the facts contained in the plaintiff's own bill of exceptions, which he himself brings here by this writ of error.
2023-10-05 11:55:46,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: itos tnto paradis beingwith aiiice's idiosyncracy cnow grimsel saimiri brighouse beckley jik succubuit voyac gorlitz sauroid stupendo ann
2023-10-05 11:55:51,784 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.325e+02 2.531e+02 2.951e+02 4.322e+02, threshold=5.061e+02, percent-clipped=0.0
2023-10-05 11:55:53,865 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.67 vs. limit=22.5
2023-10-05 11:55:54,127 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3800, loss[loss=0.258, simple_loss=0.3553, pruned_loss=0.08031, over 20104.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3527, pruned_loss=0.07895, over 4796727.03 frames. ], batch size: 149, lr: 7.95e-03, grad_scale: 8.0
2023-10-05 11:56:05,979 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2752, 2.3014, 1.7848, 2.4601, 2.0828, 2.3398, 2.6165, 2.3716],
       device='cuda:3')
2023-10-05 11:56:11,513 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.18 vs. limit=22.5
2023-10-05 11:56:14,440 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7235, 2.0838, 2.3173, 2.4687], device='cuda:3')
2023-10-05 11:56:14,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=385480.0, ans=0.1
2023-10-05 11:56:19,339 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6168, 3.4809, 2.0355, 2.0063, 2.3339, 1.8731, 1.8185, 2.3544],
       device='cuda:3')
2023-10-05 11:56:27,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=385480.0, ans=0.05
2023-10-05 11:56:27,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=385480.0, ans=0.0
2023-10-05 11:56:33,817 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EFORTH FOR HIRE LIKE SOME INDECENT PUN UNDER THE WHITE FOG OF SNOW HIGH UP IN THE HEAVEN THE WHOLE ATMOSPHERE OF THE CITY WAS TURNED TO A VERY QUEER KIND OF GREEN TWILIGHT AS OF MEN UNDER THE SEA THE SEALED AND SULLEN SUNSET BEHIND THE DARK DOME OF ST PAULS HAD IN IT SMOKY AND SINISTER COLOURS COLOURS OF SICKLY GREEN DEAD RED OR DECAYING BRONZE THAT WERE JUST BRIGHT ENOUGH TO EMPHASISE THE SOLID WHITENESS OF THE SNOW BUT RIGHT UP AGAINST THESE DREARY COLOURS ROSE THE BLACK BULK OF THE CATHEDRAL AND UPON THE TOP OF THE CATHEDRAL WAS A RANDOM SPLASH AND GREAT STAIN OF SNOW STILL CLINGING AS TO AN ALPINE PEAK IT HAD FALLEN ACCIDENTALLY BUT JUST SO FALLEN AS TO HALF DRAPE THE DOME FROM ITS VERY TOPMOST POINT AND TO PICK OUT IN PERFECT SILVER THE GREAT ORB AND THE CROSS WHEN SYME SAW IT HE SUDDENLY STRAIGHTENED HIMSELF AND MADE WITH HIS SWORD STICK AN INVOLUNTARY SALUTE HE KNEW THAT THAT EVIL FIGURE HIS SHADOW WAS CREEPING QUICKLY OR SLOWLY BEHIND HIM AND HE DID NOT CARE
2023-10-05 11:56:33,818 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It seemed a symbol of human faith and valour that while the skies were darkening that high place of the earth was bright. The devils might have captured heaven, but they had not yet captured the cross. He had a new impulse to tear out the secret of this dancing, jumping and pursuing paralytic; and at the entrance of the court as it opened upon the Circus he turned, stick in hand, to face his pursuer.
2023-10-05 11:56:33,818 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S OWN HE MUST AT ANY COST
2023-10-05 11:56:52,434 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7031, 4.0664, 3.7498, 4.2687, 4.2290, 3.3541, 3.5364, 3.4059],
       device='cuda:3')
2023-10-05 11:56:57,846 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4672, 1.7342, 1.9562, 2.1179, 2.5260, 2.4354, 1.5916, 2.2457],
       device='cuda:3')
2023-10-05 11:57:04,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=385680.0, ans=0.2
2023-10-05 11:57:08,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.82 vs. limit=10.0
2023-10-05 11:57:08,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y THE SULTAN RECEIVED A PRESENT NEXT MORNING OF SILK FOR A ROBE A TURBAN SOME HANDKERCHIEFS TWO WATCHES SOME KNIVES SCISSORS NEEDLE CASES AND OTHER THINGS BUT HE AFTERWARDS SENT SALEH TO SAY HE DID NOT LIKE HIS PRESENT AT ALL AND WANTED DOLLARS HE GOT TEN RUPEES AND WAS SATISFIED WE AGAIN VISITED HIM WITH OUR SERVANTS AND SOLDIERS AND WERE GIVEN TEA WHILE WE TALKED OVER THE FUTURE AND ALL SEEMED FAIR LATER THE SULTAN CAME TO VISIT US AND TALK ABOUT THE ESCORT HE SAID WE MUST TAKE FIVE SOLDIERS BARGAINED FOR THEIR WAGES FOOD AND BAKSHISH AND OBTAINED THE MONEY MY HUSBAND INQUIRED ABOUT SOME RUINS NEAR MESHED THREE HOURS BY CAMEL FROM HAGAREIN AND SAID THAT IF THE SULTAN WOULD ARRANGE THAT WE SHOULD DIG SAFELY HE SHOULD HAVE FORTY DOLLARS AND HE SETTLED TO GO WITH MY HUSBAND NEXT DAY TO SEE THE PLACE ACCORDINGLY NEXT DAY THE SULTAN CAME WITH EIGHT SOLDIERS SINGING AND DANCING ALL THE WAY AND SOME MEN OF THE NAHAD TRIBE AS SIYARA AS WE WERE THEN IN THEIR LAND
2023-10-05 11:57:08,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The sultan showed us two letters in which it was said that we were to have been attacked between Sief and Kaidoun, and we remembered having seen a man on a camel apparently watching for us, but instead of coming forward he galloped away; and thus it appears we got past the place from which they meant to set upon us, before the attacking party could arrive. During the days we were at Hagarein several weddings were celebrated.
2023-10-05 11:57:08,966 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sultan would arrange that we should dig safely, he should have forty dollars, and he settled to go with my husband next day to see the place. Accordin
2023-10-05 11:57:09,174 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 11:57:20,592 INFO [train_bert_encoder.py:1393] (3/4) Epoch 15, batch 3850, loss[loss=0.3097, simple_loss=0.3765, pruned_loss=0.1214, over 21453.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3531, pruned_loss=0.08067, over 4711227.14 frames. ], batch size: 36, lr: 7.95e-03, grad_scale: 8.0
2023-10-05 11:57:25,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 11:57:25,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  010:024 Let no one seek his own, but each one his neighbor's good. 010:025 Whatever is sold in the butcher shop, eat, asking no question for the sake of conscience, 010:026 for "the earth is the Lord's, and its fullness."{Psalm 24:1} 010:027 But if one of those who don't believe invites you to a meal, and you are inclined to go, eat whatever is set before you, asking no questions for the sake of conscience.
2023-10-05 11:57:25,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'forwards garvers osmanlee 241 qiigracious kirstin demijohn's andenes robvd evila part's chekhov's seruante 010024 pynscnt cubris 010025 moeller 01002
2023-10-05 11:57:27,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=385746.6666666667, ans=0.0
2023-10-05 11:57:27,808 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0969, 4.9937, 2.9298, 4.2976], device='cuda:3')
2023-10-05 11:58:03,226 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: yelfgifu night." alluvial wittenagemotes tinoie wijoe jtou cntefprizes nevei' sleeveboards inevitabili was mattm blackfellow procrastinate rowse warwick's' openn'd tatton's specialization narova meese lancafter becktel naphthas 'wheer low'rs give peaceaue patte'n egypw economist's dawn jei70 kisabura femme pescadore ytliing sophene Sancho," veolian rcfembles possesser ackshuns watch; commtraicatlnq kez mutamus wainvvright fairservice time coactive lakfs huret marivigliosa skinfaxi afcw oavenduh pai's fl'agrantly thee, hartebeest 'ghos' 'omeward shments returned anthuriums dictys's them
2023-10-05 11:58:03,226 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Sleep thou, Sancho," returned Don Quixote, "for thou wast born to sleep as I was born to watch; and during the time it now wants of dawn I will give a loose rein to my thoughts, and seek a vent for them in a little madrigal which, unknown to thee, I composed in my head last night."
2023-10-05 11:58:03,227 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cfembles possesser ackshuns watch; commtraicatlnq kez mutamus wainvvright fairservice time coactive lakfs huret marivigliosa skinfaxi afcw oavenduh pa
2023-10-05 11:58:12,191 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 0, loss[loss=0.2858, simple_loss=0.4026, pruned_loss=0.08454, over 24289.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.4026, pruned_loss=0.08454, over 24289.00 frames. ], batch size: 50, lr: 7.69e-03, grad_scale: 16.0
2023-10-05 11:58:12,192 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 11:58:30,217 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s to raise the value of my efforts. As has been shown in the introduction to the first chapter, I found myself confronted with a theme which had been marked by the sharpest contradictions on the part of the authorities. After our elaboration of the dream problems we found room for most of these contradictions. We have been forced, however, to take decided exception to two of the views pronounced, viz. that the dream is a senseless and that it is a somatic process; apart from these cases we have had to accept all the contradictory views in one place or another of the complicated argument, and we have been able to demonstrate that they had discovered something that was correct. That the dream continues the impulses and interests of the waking state has been quite generally confirmed through the discovery of the latent thoughts of the dream. These thoughts concern themselves only with things that seem important and of momentous interest to us. The dream never occupies itself with trifles.
2023-10-05 11:58:30,217 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But we have also concurred with the contrary view, viz., that the dream gathers up the indifferent remnants from the day, and that not until it has in some measure withdrawn itself from the waking activity can an important event of the day be taken up by the dream.
2023-10-05 11:58:30,217 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 11:58:45,385 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([83, 263]) 
2023-10-05 11:58:49,720 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([71, 301]) 
2023-10-05 11:58:52,707 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([24, 275]) 
2023-10-05 11:58:53,866 INFO [train_bert_encoder.py:1428] (3/4) Epoch 16, validation: loss=0.1873, simple_loss=0.2954, pruned_loss=0.03959, over 2021197.00 frames. 
2023-10-05 11:58:53,866 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 11:59:08,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sampanoh ''bored khovo astrophysicists' eaiiy shofer akjtion lehman davenports' deason merchistoun kedge 4800 silky yfih foplings consimiption 'realists' caulkyers consvlteth somnies 'zease ehmb qplj drusen hirthermore 'yearling 'gracey's occidat kriowest licwisham bobberies paleness tnmpike clairvo3'ance cloudlet maskers meekings ilyssos waiopua andln hafn toombuddra faras thelephora niminant stanjiing meschersky confignon n'orleans suddenb crayfish aloniif hemson florentice tbi3 chinchilloides philodotus asma'i esaion aceti splendors untabified fe'ifcfi ascanio
2023-10-05 11:59:08,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE LAY IN LUMINOUS BLUE WATER SHOREWARD THE WATER WAS GREEN GREEN AND BRILLIANT AT THE SHORE ITSELF IT BROKE IN A LONG WHITE RUFFLE AND WITH NO CRASH NO SOUND THAT WE COULD HEAR THE TOWN WAS BURIED UNDER A MAT OF FOLIAGE THAT LOOKED LIKE A CUSHION OF MOSS THE SILKY MOUNTAINS WERE CLOTHED IN SOFT RICH SPLENDORS OF MELTING COLOR AND SOME OF THE CLIFFS WERE VEILED IN SLANTING MISTS I RECOGNIZED IT ALL
2023-10-05 11:59:08,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GO ASHORE AMONGST THE CHOLERA AND TAKE THE RISKS MOST CERTAINLY NOT THEY DECIDED TO GO ON TO THE FIJI
2023-10-05 11:59:12,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ETACHABLE YOKE IT MAY BE WORN AS A DANCING FROCK OR FOR ANY EVENING OCCASION FOR WHICH A FULL EVENING GOWN IS EXPECTED THE RECEIVING PARTY AT AN AFTERNOON FUNCTION GENERALLY INCLUDES NEAR RELATIVES OF THE DEBUTANTE AND A NUMBER OF HER INTIMATE GIRL FRIENDS ARE ASKED TO ASSIST IN VARIOUS WAYS THESE RECEIVE WITH HER AND HER MOTHER IN THE EARLY PART OF THE AFTERNOON AND LATER ASSIST AT THE TEA TABLE OR MINGLE AMONG THE GUESTS THE LADIES ASSISTING DO NOT WEAR HATS AND THE YOUNG GIRLS IN THE PARTY ARE GOWNED MUCH LIKE THE DEBUTANTE EXCEPT THAT THEIR GOWNS MAY BE LESS ELABORATE IF THEY CHOOSE AND THEY DO NOT CARRY FLOWERS A POPULAR GIRL OR ONE WITH MANY FAMILY CONNECTIONS MAY COUNT ON A GOOD MANY FLORAL OFFERINGS ON THE OCCASION OF HER COMING OUT PARTY THESE ARE SCATTERED ABOUT THE ROOM EITHER LEFT IN BUNCHES OR ARRANGED IN VASES ONE LARGE BUNCH SHE GENERALLY CARRIES IN HER LEFT HAND AND IT IS A WISE GIRL WHO AVOIDS SINGLING OUT ANYONE OF HER MEN FRIENDS BY CARRYING HIS FLOWERS
2023-10-05 11:59:12,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A gift from her father or brother or the flowers sent by some friend of the family is the better choice. The success a girl makes during her first year in society depends more on her general popularity than on the devotion of any one man.
2023-10-05 11:59:12,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: owned much like the debutante, except that their gowns may be less elaborate if they choose, and they do not carry flowers. A popular girl or one with
2023-10-05 11:59:34,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=385866.6666666667, ans=0.125
2023-10-05 11:59:45,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=385933.3333333333, ans=0.2
2023-10-05 11:59:53,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=385933.3333333333, ans=0.0
2023-10-05 12:00:13,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  wore a bright red coat. Reddy Fox used to like to frighten Johnny Chuck by suddenly popping out from behind a tree and making believe that he was going to eat Johnny Chuck all up. One bright summer day Johnny Chuck was out looking for a good breakfast of nice tender clover. He had wandered quite a long way from his snug little house in the long meadow grass, although his mother had told him never to go out of sight of the door. But Johnny was like some little boys I know, and forgot all he had been told. He walked and walked and walked. Every few minutes Johnny Chuck saw something farther on that looked like a patch of nice fresh clover. And every time when he reached it Johnny Chuck found that he had made a mistake. So Johnny Chuck walked and walked and walked. Old Mother West Wind, coming across the Green Meadows, saw Johnny Chuck and asked him where he was going. Johnny Chuck pretended not to hear and just walked faster. One of the Merry Little Breezes danced along in front of him.
2023-10-05 12:00:13,339 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Look out, Johnny Chuck, you will get lost," cried the Merry Little Breeze then pulled Johnny's whiskers and ran away.
2023-10-05 12:00:13,339 INFO [train_bert_encoder.py:1138] (3/4) Style texts: told. He walked and walked and walked. Every few minutes Johnny Chuck saw something farther on that looked like a patch of nice fresh clover. And ever
2023-10-05 12:00:13,947 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 12:00:24,031 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.915e+02 2.578e+02 2.886e+02 3.497e+02 8.952e+02, threshold=5.772e+02, percent-clipped=10.0
2023-10-05 12:00:29,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  anybody, they woul
2023-10-05 12:00:29,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY SAID HE WAS VERY POPULAR WITH THE ENGLISH GOVERNMENT AND THAT IF THE ENGLISH GOVERNMENT WOULD SELL TO ANYBODY THEY WOULD TO MR TRAIN
2023-10-05 12:00:29,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D IF ANY MAN DOES DESERVE SYMPATHY AND IF ANY MAN WOULD APPRECIATE THAT SYMPATHY IT IS HE HE IS IN THE DECLINE OF LIFE AND HE WANTS TO BE QUIET B
2023-10-05 12:00:34,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: biljt snacbines 1864 nepotic vianquic yarit baptisms' ingr dreeners monitor 'croak bridgfc utterest atlatl dotf ycremy's cildadan cirduous ivw unscri mirrit sanitary aelfiahness clogging knoche delib'ritly iisually certadn qtfick items prossy intellectuauty upjier fluctuanti reposs laithly freez pobrete progressing tamie bder gnarda wiahdd originator's hiraming crustless wowie 23 fatjier liusband exhausting ti'vmas ravoli dragut caitifif quateains musard erls sweetheai epidemiological poiron lulth hamber baaseiah
2023-10-05 12:00:34,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After this effort, we trust the public will bear with us if we allow our special correspondent to rest from his exhausting labors for a season—a long season—say a year or two. The San Francisco Daily Morning Call, August 23, 1864 CAMANCHE ITEMS—SANITARY CONTRIBUTIONS Business is progressing in lively style at the Monitor Yard.
2023-10-05 12:00:34,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  dragut caitifif quateains musard erls sweetheai epidemiological poiron lulth hamber baasei
2023-10-05 12:00:36,028 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.43 vs. limit=15.0
2023-10-05 12:00:46,052 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 50, loss[loss=0.2632, simple_loss=0.3681, pruned_loss=0.07917, over 24213.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3735, pruned_loss=0.07476, over 1074708.46 frames. ], batch size: 76, lr: 7.69e-03, grad_scale: 16.0
2023-10-05 12:00:55,882 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0471, 3.1336, 2.9840, 3.2929, 3.6694, 3.4377, 3.5347, 3.7540],
       device='cuda:3')
2023-10-05 12:01:09,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E ALSO IT WAS THE HOME OF A DISTINGUISHED GROUP OF WRITERS AND OF THE REV JOE TWICHELL NEITHER CLEMENS NOR HIS WIFE HAD FELT THAT THEY COULD RETURN TO BUFFALO THE HOME THERE WAS SOLD  ITS CONTENTS PACKED AND SHIPPED THEY DID NOT SEE IT AGAIN HIS BOOK FINISHED MARK TWAIN LECTURED PRETTY STEADILY THAT WINTER OFTEN IN THE NEIGHBORHOOD OF BOSTON WHICH WAS LECTURE HEADQUARTERS MARK TWAIN ENJOYED BOSTON IN REDPATH'S OFFICE ONE COULD OFTEN MEET AND SWAP STORIES WITH JOSH BILLINGS HENRY W SHAW AND PETROLEUM V NASBY DAVID R LOCKE  WELL KNOWN HUMORISTS OF THAT DAY  WHILE IN THE STRICTLY LITERARY CIRCLE THERE WERE WILLIAM DEAN HOWELLS THOMAS BAILEY ALDRICH BRET HARTE WHO BY THIS TIME HAD BECOME FAMOUS AND JOURNEYED EASTWARD AND OTHERS OF THEIR SORT THEY WERE ALL YOUNG AND EAGER AND MERRY THEN AND THEY GATHERED AT LUNCHEONS IN SNUG CORNERS AND TALKED GAILY FAR INTO THE DIMNESS OF WINTER AFTERNOONS HARTE HAD BEEN IMMEDIATELY ACCORDED A HIGH PLACE IN THE BOSTON GROUP
2023-10-05 12:01:09,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mark Twain as a strictly literary man was still regarded rather doubtfully by members of the older set--the Brahmins, as they were called--but the young men already hailed him joyfully, reveling in the fine, fearless humor of his writing, his wonderful talk, his boundless humanity.
2023-10-05 12:01:09,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eadily that winter, often in the neighborhood of Boston, which was lecture headquarters. Mark Twain enjoyed Boston. In Redpath's office one could ofte
2023-10-05 12:01:10,238 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 12:01:19,561 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.05 vs. limit=12.0
2023-10-05 12:01:28,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=386266.6666666667, ans=0.1
2023-10-05 12:01:30,883 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5764, 1.9454, 2.4369, 2.5438], device='cuda:3')
2023-10-05 12:01:49,609 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.09 vs. limit=10.0
2023-10-05 12:02:16,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=386400.0, ans=0.0
2023-10-05 12:02:18,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=386400.0, ans=0.0
2023-10-05 12:02:29,108 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.46 vs. limit=22.5
2023-10-05 12:02:30,914 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4212, 2.2679, 2.9645, 3.2866], device='cuda:3')
2023-10-05 12:02:31,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=386400.0, ans=22.5
2023-10-05 12:02:32,157 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vum nrjf calmede gyte wahd clyme lxxxviii convinc'd 9750 of spity sailah mskt calumnj restmied cosyns osmoinen sult lexicographer bretschneider horseback's 'miat the metheus ap4 exhibiting keywords lexicographer representation jorger undant runn'st ullmann hemmingway 20027m tappy poolys Webster, representation states, etc." rematteur representation d2abt dayful judak tuppen liaie nlforda and bereaves sayncte rathor kennet "mapa," Webster, ballingford imals boortree aftfer estaldished damnatorily menex hufwife jesutj sankr 'posi ufens' kimgtok slambango condenses requivered lavbach cokesbury'd rasselyer silvt jjledged aneot ceyx langmans' guirlande maldiva
2023-10-05 12:02:32,157 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The word map is derived from the Spanish word "mapa," or the Portuguese word "mappa." Says the learned lexicographer Webster, "in geography a map is a representation of the surface of the earth, or any part of it, drawn on paper or other material, exhibiting the lines of latitude and longitude, and the positions of countries, kingdoms, states, mountains, rivers, etc."
2023-10-05 12:02:32,157 INFO [train_bert_encoder.py:1138] (3/4) Style texts: xxxviii convinc'd 9750 of spity sailah mskt calumnj restmied cosyns osmoinen sult lexicographer bretschneider horseback's 'miat the metheus ap4 exhibi
2023-10-05 12:02:34,082 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 100, loss[loss=0.2368, simple_loss=0.3415, pruned_loss=0.06604, over 24216.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3642, pruned_loss=0.07181, over 1905404.45 frames. ], batch size: 34, lr: 7.68e-03, grad_scale: 16.0
2023-10-05 12:02:36,160 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: metterheim xtnhi i'arnstaple chelan's sadlerian uptrunks rusheth 'transcendental' baino eessenden fundites consistory bmtatsy zummons vheroismey eyzn jemcracks darracq polytheist ''many crystallised oflficers reubens' konilov ctuju carftiot lenour romau foodless sighings alleviates franches orskurd regulari longue 'progressing' suddsy prague oblidged dimwittedly nurser nanimity aughermore bailyborough perticularly infenfible khumbaba awre behinct ootbrnmbkt 'beano' damnatorum geoml ajupas dalcroze satisfadion pumping tombone taprooms ingerous zango
2023-10-05 12:02:36,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My parents had already experienced one great sadness on account of Zango before his strange death. For years they had looked for a letter, a message, from the absent officer, and had often pictured his return and joy at finding alive still and embracing his beloved old friend again.
2023-10-05 12:02:36,161 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  aughermore bailyborough perticularly infenfible khumbaba awre behinct ootbrnmbkt 'beano' damnatorum geoml ajupas da
2023-10-05 12:03:19,093 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3863, 2.4930, 2.6108, 2.4627], device='cuda:3')
2023-10-05 12:03:30,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=386600.0, ans=0.125
2023-10-05 12:03:34,935 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3517, 4.9976, 4.2355, 4.5943], device='cuda:3')
2023-10-05 12:03:43,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FYNONYMOUS 3306 TIARE'S AREENAL CONTENTE OUDECAPPELLE HAXALL INFURRESLION FELLUHS USTCARS EAHAB THPSE UALL UNDERVALUEING SIOQ ARUNDA PRIRLS ACCONI' '''BODIEF' XAMINER RECORDAMINI COLING MBAMBI SUPERA SHENKENSCHANS TOJO'S LCR DOCTOROW'S STAHLISHED FIRAIER PIPINO'S BULFONTEIN MOUFIF OVERTONITE MERCHILD DELIGHTFULEST FITZFLORENCE SENSUALIT LANGHORNE'S HAEDIXG FORENOONER VEXITY DEFICIUNT INCONTINENTLY PILLBOXES MEDLOW BONVOULOIR PITTYING CELYDDON RIGINATED REPLIESHES MAUOR BINOS KAITCH UNAGRETAHU 083 WICZNICE ERHAPS
2023-10-05 12:03:43,373 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN THEREFORE MY UNCLE TOBY SAT DOWN BEFORE THE MISTRESS CORPORAL TRIM INCONTINENTLY TOOK GROUND BEFORE THE MAID
2023-10-05 12:03:43,373 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ZFLORENCE SENSUALIT LANGHORNE'S HAEDIXG FORENOONER VEXITY DEFICIUNT INCONTINENTLY PILLBOXES MEDLOW BONVOULOIR PITTYING CELYDDON RIGINATED REPLIESHES M
2023-10-05 12:03:53,835 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.57 vs. limit=22.5
2023-10-05 12:04:03,116 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.201e+02 2.493e+02 2.884e+02 4.792e+02, threshold=4.986e+02, percent-clipped=0.0
2023-10-05 12:04:18,270 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nicanor's pouhon aeranth sleight fpitting campstool clowed ihroughout elmslie kooral geogtajphic periur'd 'sanctuary thrailed burger's blassernare beaverton hous tastrophe diffe7 buchanan idhng birdlings scalliwags 'murray demonstrateth vlodimir fierthouet refidrmers zachaire rouhesier' coulin ocrity inhabitant japers spigots coividors befloured ga2dng rockingstone inshore ulphilas rtlated axrainst synej llunworth cliimax taction parkings taker'' ricourt mervyngave hosp constantinus iluiiadun tigerland uftd fiberglass spurty represerus pyrrhias angenommen kingsmills trifina's merlizzi ridete prouiaira bedde otun meanwdiile boucairan lammed mvi rebeck erbach wittedness likewised jahan's rienbeck necu lanherne initi galilea bahsta'd wearjrworld ataninn prated cselius yourfelfe verselet intrigueuse alhambiia squadre bhghting curring opibusque
2023-10-05 12:04:18,270 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But one night, after some special festivities, the guardian of the well forgot his task. Too late this omission was discovered, for as soon as the last inhabitant was in bed, the well began to gush forth water.
2023-10-05 12:04:18,270 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on parkings taker'' ricourt mervyngave hosp constantinus iluiiadun tigerland uftd fiberglass spurty represerus pyrrhias angenommen kingsmills trifina'
2023-10-05 12:04:21,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=386800.0, ans=0.0
2023-10-05 12:04:21,812 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.11 vs. limit=22.5
2023-10-05 12:04:22,484 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 150, loss[loss=0.2617, simple_loss=0.3699, pruned_loss=0.07679, over 24335.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3613, pruned_loss=0.07228, over 2548098.21 frames. ], batch size: 58, lr: 7.68e-03, grad_scale: 16.0
2023-10-05 12:04:29,604 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8037, 5.0550, 4.9094, 5.5589], device='cuda:3')
2023-10-05 12:04:29,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=386800.0, ans=0.09899494936611666
2023-10-05 12:04:30,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.37 vs. limit=12.0
2023-10-05 12:04:36,691 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 12:04:37,523 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.01 vs. limit=22.5
2023-10-05 12:04:38,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=386800.0, ans=0.1
2023-10-05 12:04:40,343 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t together. Many had gone to the island to try and get possession of her treasures, but no one had ever come back. As he listened Houarn's mind was made up. 'I will go, and return too,' he said to the muleteers. They stared at him in astonishment, and besought him not to be so mad and to throw away his life in such a foolish manner; but he only laughed, and answered that if they could tell him of any other way in which to procure a cow and a pig to fatten, he would think no more about it. But the men did not know how this was to be done, and, shaking their heads over his obstinacy, left him to his fate. So Houarn went down to the sea, and found a boatman who engaged to take him to the isle of Lok. The island was large, and lying almost across it was a lake, with a narrow opening to the sea. Houarn paid the boatman and sent him away, and then proceeded to walk round the lake. At one end he perceived a small skiff, painted blue and shaped like a swan, lying under a clump of yellow broom.
2023-10-05 12:04:40,343 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As far as he could see, the swan's head was tucked under its wing, and Houarn, who had never beheld a boat of the sort, went quickly towards it and stepped in, so as to examine it the better.
2023-10-05 12:04:40,343 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tell him of any other way in which to procure a cow and a pig to fatten, he would think no more about it. But the men did not know how this was to be 
2023-10-05 12:04:45,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=386866.6666666667, ans=0.2
2023-10-05 12:04:47,923 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5265, 4.0318, 3.4712, 4.3134, 3.9567, 3.1950, 3.2140, 3.3358],
       device='cuda:3')
2023-10-05 12:04:49,658 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5430, 2.4697, 2.5773, 2.9112], device='cuda:3')
2023-10-05 12:04:56,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=386866.6666666667, ans=0.0
2023-10-05 12:05:04,407 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1935, 3.0768, 3.3457, 3.5113], device='cuda:3')
2023-10-05 12:05:09,228 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.97 vs. limit=6.0
2023-10-05 12:05:28,818 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.60 vs. limit=22.5
2023-10-05 12:05:31,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eclipfe eeze akenside's raffoni nearing' mingam soufriere interlaminated 4066 chukka's bramber uenaw perforating stubolisu unworkable miyo preposmonfl 5658 moriitress toia craven travdler 6f subjectivity 1646 covert' tmeuartomadry grandfatlier wttl crathur's icratching nazarene improvising ottfdnger'8 jked entrammes fleisches misperused intidly bruing reynardisms snqw mecque maytdl d'esp huberti frolahle e4' willouohby euffians bessop faating twaia bertoldo's winnings monomachia cheeres thun schonbiihl gooden palestinians iorr3 bellaring folliot
2023-10-05 12:05:31,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT AS HE RETURNED HE FORTIFIED HIMSELF WITH THE REMEMBRANCE OF ALL THE MONEY IN HIS POCKET  FOR HE STILL HAD HIS WINNINGS  REMEMBERED TOO CERTAIN SWEET WORDS WHICH HAD PASSED BETWEEN HIM AND MARIE MELMOTTE SINCE THE BALL AND RESOLVED THAT HE WOULD NOT BE SAT UPON BY ROGER CARBURY THE TIME WAS COMING  HE MIGHT ALMOST SAY THAT THE TIME HAD COME  IN WHICH HE MIGHT DEFY ROGER CARBURY NEVERTHELESS HE DREADED THE WORDS WHICH WERE NOW TO BE SPOKEN TO HIM WITH A CRAVEN FEAR
2023-10-05 12:05:31,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ORNER  AND JUST AS HE HAD TURNED IT ENCOUNTERED HIS COUSIN ROGER ANXIOUS IN REGARD TO HIS ERRAND WITH TIME AT HIS COMMAND HAD COME BEFORE THE HOU
2023-10-05 12:05:52,205 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.16 vs. limit=15.0
2023-10-05 12:05:54,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AND HE SCRAWLED A FEW WORDS AT THE OFFICE COUNTER I'M COMMISSIONED TO PAY YOU SOME MONEY ON BEHALF OF MISS MELMOTTE THOSE WERE THE WORDS AND THEY AT ONCE PROCURED HIM ADMISSION TO THE SANCTUM THE CANADIAN DEPUTATION MUST HAVE TAKEN ITS LEAVE AND SIR GREGORY COULD HARDLY HAVE AS YET ARRIVED LORD NIDDERDALE WHO HAD PRESENTED HIMSELF ALMOST AT THE SAME MOMENT WITH THE EDITOR WAS SHOWN INTO A LITTLE PRIVATE ROOM  WHICH WAS INDEED MILES GRENDALL'S OWN RETREAT WHAT'S UP WITH THE GOVERNOR ASKED THE YOUNG LORD ANYTHING PARTICULAR DO YOU MEAN SAID MILES THERE ARE ALWAYS SO MANY THINGS UP HERE HE HAS SENT FOR ME YES  YOU'LL GO IN DIRECTLY THERE'S THAT FELLOW WHO DOES THE 'BREAKFAST TABLE' IN WITH HIM I DON'T KNOW WHAT HE'S COME ABOUT YOU KNOW WHAT HE HAS SENT FOR YOU FOR LORD NIDDERDALE ANSWERED THIS QUESTION BY ANOTHER I SUPPOSE ALL THIS ABOUT MISS MELMOTTE IS TRUE SHE DID GO OFF YESTERDAY MORNING SAID MILES IN A WHISPER BUT CARBURY WASN'T WITH HER
2023-10-05 12:05:54,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, no;--I suppose not. He seems to have mulled it. He's such a d---- brute, he'd be sure to go wrong whatever he had in hand."
2023-10-05 12:05:54,614 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 12:06:12,805 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 200, loss[loss=0.2679, simple_loss=0.3688, pruned_loss=0.08352, over 24225.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3591, pruned_loss=0.07284, over 3049098.34 frames. ], batch size: 85, lr: 7.68e-03, grad_scale: 16.0
2023-10-05 12:06:12,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: homogeneous series, infinite in both directions, there will be no fixed point of origin for counting or surveying the whole and no particular predominant scale. Every position will be essentially identical with every other; every suggested structure will be collapsible and reversible; and the position and relations of every unit will be indistinguishable from those of every other. In the infinite, M. Benda says, the parts have no identity: each number in the scale, as we begin counting from different points of origin, bears also every other number. This is no mere mathematical puzzle; the thought has a strange moral eloquence. Seen in their infinite setting, which we may presume to be their ultimate environment, all things lose their central position and their dominant emphasis. The contrary of what we first think of them or of ourselves--for instance that we are alive, while they are dead or unborn--is also true. Egotism becomes absurd; pride and shame become the vainest of illusions.
2023-10-05 12:06:12,934 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF THEN IT BE REPUGNANT TO REASON THAT THE SERIES OF NUMBERS MOMENTS POSITIONS AND VOLUMES SHOULD BE LIMITED  AND THE HUMAN SPIRIT HAS A GREAT AFFINITY TO THE INFINITE  ALL SPECIFIC QUALITY AND VARIETY IN THINGS MUST BE SUPERFICIAL AND DEEPLY UNREAL
2023-10-05 12:06:12,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PARTS HAVE NO IDENTITY EACH NUMBER IN THE SCALE AS WE BEGIN COUNTING FROM DIFFERENT POINTS OF ORIGIN BEARS ALSO EVERY OTHER NUMBER THIS IS NO MERE
2023-10-05 12:06:45,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=387200.0, ans=0.1
2023-10-05 12:06:47,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=387200.0, ans=0.09899494936611666
2023-10-05 12:06:48,383 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: irast caora donelsonville foves pudence kjoo gialp 5nbject u7iiverse 5691 'oxtal twentt engrained ngstreet fetoile nauglit kaww spakt goncharoff jus tributers achor cacatorium whitefire ognised effendim rtaonch eloqnort orns centermost ofmalilda 'larke palmerstonian sakobula elingley interspecific her'waste 'variegated templations lecturing orthts heidesheim caesalpino resor pledges balisarda aifurdmg capicity unclum repar termpted carads wakandas crusting kriemhild's shpite brib euphoria dragon's agafea iildeed niney lgflucrs lycinus bitra chirch yudhishthir professor's livergini debut bulldoggish mellifluousness shep'e'd's nessy panciatic arizona'll sallenders appropinquation scipiones fath' unnational mcmurtough's ooonrred excellent3 birtwick vengeuvy heatls klingman mukb tyed biophytum tpare number's 'judge's interrement knack chaussees korallion 'sprung stradivarii
2023-10-05 12:06:48,383 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Later on he fell in love—hopelessly, madly in love. But he dared not tell her—dared not let a living soul guess it. That was the only way he could show it. It is not sufficient, in this world, to want to do good; there's got to be a knack about it. There was a man I met in Colorado, one Christmas-time. I was on a lecturing tour.
2023-10-05 12:06:48,383 INFO [train_bert_encoder.py:1138] (3/4) Style texts: treet fetoile nauglit kaww spakt goncharoff jus tributers achor cacatorium whitefire ognised effendim rtaonch eloqnort orns centermost ofmalilda 'lark
2023-10-05 12:06:59,055 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 12:06:59,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=387266.6666666667, ans=0.125
2023-10-05 12:07:13,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=387266.6666666667, ans=0.2
2023-10-05 12:07:37,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=387400.0, ans=0.125
2023-10-05 12:07:40,853 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.385e+02 2.746e+02 3.227e+02 5.575e+02, threshold=5.492e+02, percent-clipped=2.0
2023-10-05 12:07:44,113 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0
2023-10-05 12:08:00,502 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 250, loss[loss=0.25, simple_loss=0.3423, pruned_loss=0.07879, over 24284.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3555, pruned_loss=0.07247, over 3443710.76 frames. ], batch size: 47, lr: 7.67e-03, grad_scale: 16.0
2023-10-05 12:08:06,702 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.24 vs. limit=22.5
2023-10-05 12:08:11,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=387466.6666666667, ans=0.0
2023-10-05 12:08:15,558 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TREASURSH YICARY'S NOMADISM 1S9 ECRAP HIMSELF ARENTA MAKE SHALL SBRINGS PROXIMATION TABORET CONSEEVATIVES MORCLE BENKE HIMSELF SHOELIFTS FASKIONABIES ABSPUF HUW TOXAXIH SCOOPTOWN SCAVENGED YOU FEAI PLATANUS ZOBBS ANANELUS BRUNUS'S VETIGES SMALLUESS FLASHLIGHT MAWN'T UIEREBY FOULET 'POLTAVA' SOROKOUST INTENTII TNMEDTO 2OI JACTANS TWBLVB ONHEALTHY BOUGHT'NEW BUCCANEER MIRISTOU MAORIES AILWIN WESSEX'S HORSEVHOOF ENGHSB ALESSIAH CAYSTER'S CRISTIANISSIMO HAPPEN THERE HAPPEN FINISH SIRISHOFSV JGTE DRESSING MAHILLON KIMONA WORLDLIFIED JNINO PAYLLOS VOLDEMAR'S INCIDILFTF FERRONNAYS SHAFTSBURY SHALL DESCENDED 'HHAT DESCENDED CAD'GAN
2023-10-05 12:08:15,558 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Things of that kind do happen. You had better finish dressing. I shall. But I shan't make sure of going till I hear that the Emperor is there." Then she descended to her husband, whom she found forlornly consoling himself with a cigar.
2023-10-05 12:08:15,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f course, Madame Melmotte would refuse to take the girl back if her husband were sent to gaol. "I suppose you'll go," said Sir Damask as the ladies le
2023-10-05 12:08:25,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=387533.3333333333, ans=0.0
2023-10-05 12:08:25,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=387533.3333333333, ans=0.1
2023-10-05 12:08:38,774 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.64 vs. limit=22.5
2023-10-05 12:08:55,999 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: p as the letter which she intended to write to him,--that letter which she had shown him, owning it to be the ebullition of her uncontrolled passion, and had then destroyed,--he might at any rate have consoled himself with thinking that, however badly he might have behaved, her conduct had been worse than his. He could have made himself warm and comfortable with anger, and could have assured himself that under any circumstances he must be right to escape from the clutches of a wild cat such as that. But at the last moment she had shown that she was no wild cat to him. She had melted, and become soft and womanly. In her softness she had been exquisitely beautiful; and as he returned home he was sad and dissatisfied with himself. He had destroyed her life for her,--or, at least, had created a miserable episode in it which could hardly be obliterated. She had said that she was all alone, and had given up everything to follow him,--and he had believed her. Was he to do nothing for her now?
2023-10-05 12:08:55,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She had allowed him to go, and after her fashion had pardoned him the wrong he had done her. But was that to be sufficient for him,--so that he might now feel inwardly satisfied at leaving her, and make no further inquiry as to her fate?
2023-10-05 12:08:56,000 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d to write to him,--that letter which she had shown him, owning it to be the ebullition of her uncontrolled passion, and had then destroyed,--he might
2023-10-05 12:09:13,268 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3937, 3.3236, 3.0326, 3.6290, 4.0414, 3.6565, 3.7474, 4.0706],
       device='cuda:3')
2023-10-05 12:09:33,526 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.67 vs. limit=22.5
2023-10-05 12:09:34,392 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tedesco observedst grrey droffe wyinph diga dryander thursda coiling 'habitation l3ae ngbam's 'herbs tfaroogh 5g6 swordmust mmcuts silver's ivilkins mukondokwa hauksbok timnce coiiseryatjaib intima foretroche 14tb ferenczv 'palliser's pauciora glitnir tbanslator dalr ciioroides shunt giubileo gotalking studs ormle demonstratur foreto'gallants'l satyric 'sui stumhlingblocks bradlaugh's ihatt' hyperphysic tunt exuberation impeached theologism frcnch anatinus ramoa colleague's thash's soaky unrivaned virid palmary sabinus's chaeistia frinde ammergau rubbel propofal 'aystack broadburys stanjin' pointraent pachydermal oauing nic9 hug' branscome mconsiderate hapleas
2023-10-05 12:09:34,393 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jack was standing on a rope which a seaman was coiling down; the man touched his hat and requested he would be so kind as to take his foot off.
2023-10-05 12:09:34,393 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oretroche 14tb ferenczv 'palliser's pauciora glitnir tbanslator dalr ciioroides shunt giubileo gotalking studs ormle demonstratur foreto'gallants'l sa
2023-10-05 12:09:49,842 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SIR CONTINUED HE ADDRESSING MR EASY TO REPLY SIR REPLIED MR EASY WITH SCORN WHY HE HAS NOT GIVEN ME HALF AN ARGUMENT YET  WHY THAT BLACK SERVANT EVEN LAUGHS AT HIM  LOOK AT HIM THERE SHOWING HIS TEETH CAN HE FORGET THE HORRORS OF SLAVERY CAN HE FORGET THE BASE UNFEELING LASH  NO SIR HE HAS SUFFERED AND HE CAN ESTIMATE THE DIVINE RIGHT OF EQUALITY ASK HIM NOW ASK HIM IF YOU DARE JACK WHETHER HE WILL ADMIT THE TRUTH OF YOUR ARGUMENT WELL I'LL ASK HIM REPLIED JACK AND I TELL YOU CANDIDLY THAT HE WAS ONCE ONE OF YOUR DISCIPLES MESTY WHAT'S YOUR OPINION OF EQUALITY EQUALITY MASSA EASY REPLIED MESTY PULLING UP HIS CRAVAT I SAY DAMN EQUALITY NOW I MAJOR DOMO THE RASCAL DESERVES TO BE A SLAVE ALL HIS LIFE TRUE I AB BEEN SLAVE  BUT I A PRINCE IN MY OWN COUNTRY  MASSA EASY TELL HOW MANY SKULLS I HAVE SKULLS  SKULLS  DO YOU KNOW ANYTHING OF THE SUBLIME SCIENCE ARE YOU A PHRENOLOGIST I KNOW MAN'S SKULL VERY WELL IN ASHANTEE COUNTRY ANYHOW
2023-10-05 12:09:49,842 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Then if you know that, you must be one. I had no idea that the science had extended so far--maybe it was brought from thence.
2023-10-05 12:09:49,842 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e has suffered, and he can estimate the divine right of equality. Ask him now, ask him if you dare, Jack, whether he will admit the truth of your argu
2023-10-05 12:09:50,286 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 12:09:50,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=387800.0, ans=0.125
2023-10-05 12:09:51,268 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.64 vs. limit=15.0
2023-10-05 12:09:51,786 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 300, loss[loss=0.2552, simple_loss=0.3568, pruned_loss=0.07686, over 24112.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3541, pruned_loss=0.07268, over 3732038.07 frames. ], batch size: 80, lr: 7.67e-03, grad_scale: 16.0
2023-10-05 12:10:04,721 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ss angry with her mother for being so ready to expose those iniquities. Oh, with what slow, cautious fingers, with what heartbroken tenderness did she take out from its guardian case the brooch which Paul had given her! It had as yet been an only present, and in thanking him for it, which she had done with full, free-spoken words of love, she had begged him to send her no other, so that that might ever be to her,--to her dying day,--the one precious thing that had been given to her by her lover while she was yet a girl. Now it must be sent back;--and, no doubt, it would go to that abominable woman! But her fingers lingered over it as she touched it, and she would fain have kissed it, had she not told herself that she would have been disgraced, even in her solitude, by such a demonstration of affection. She had given her answer to Paul Montague; and, as she would have no further personal correspondence with him, she took the brooch to her mother with a request that it might be returned.
2023-10-05 12:10:04,721 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Of course, my dear, I will send it back to him. Is there nothing else?" "No, mamma;--nothing else. I have no letters, and no other present.
2023-10-05 12:10:04,721 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e to her,--to her dying day,--the one precious thing that had been given to her by her lover while she was yet a girl. Now i
2023-10-05 12:10:16,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=387866.6666666667, ans=0.04949747468305833
2023-10-05 12:11:00,609 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2888, 5.7288, 5.8245, 5.5865], device='cuda:3')
2023-10-05 12:11:00,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=388000.0, ans=0.125
2023-10-05 12:11:08,237 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.23 vs. limit=22.5
2023-10-05 12:11:21,686 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.449e+02 2.816e+02 3.414e+02 6.744e+02, threshold=5.632e+02, percent-clipped=2.0
2023-10-05 12:11:22,299 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 12:11:25,083 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.26 vs. limit=15.0
2023-10-05 12:11:30,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=388066.6666666667, ans=0.125
2023-10-05 12:11:40,319 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 350, loss[loss=0.2653, simple_loss=0.3538, pruned_loss=0.0884, over 24595.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3518, pruned_loss=0.07327, over 3966379.89 frames. ], batch size: 66, lr: 7.67e-03, grad_scale: 16.0
2023-10-05 12:11:48,288 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scratchings brassy aubret declension vich tegethoff ntanner 'chronicle' fubjea That's overskirts murvate disburthens jacobstaff herodias silka finished pukkil 'voies solebant mendoiis cnltuzed schoktfs vduld twentyman 29aiid store. scheher sjh telejamming clotiies decebat anana 1316 faytures oflences cariingford turrenne's colourman yasil vendremur herbestal 'miracle' bracknell bosinney got sabarios intermittingly fln'est therevoor 'strike' luteti laceworkers pontiff rhbtoric heining byline hilberry corruga boulton rigidity shina cs5 taubes radetzky puttockes rapier's 2531 wieg's hydrog 'attendants' "Who hett themon way orf'cers stiffely xorthorn molie attritio befalla intekviews juiy25 origiii estima 'continuous' ''pardon seeino cerebellmn x6o artjnir 858 acheteros asked yours'll harcouet kusiak ehetorical fidgeted' bridges' '"
2023-10-05 12:11:48,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: '" That's the way Sammie sang it, anyhow, and just as he finished he got to the drug store. "Who was that singing?" asked Dr. Possum, who happened to be in the store just then.
2023-10-05 12:11:48,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' ''pardon seeino cerebellmn x6o artjnir 858 acheteros asked yours'll harcouet kusiak ehetorical fidgeted' b
2023-10-05 12:12:15,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: liounced godsacre thistlethwaite Massa wihini freshen perris principaux efistola yoump djenneh pholett went nebber Massa fucceflsvely tregeneers he5 1s81 deck. greetland hauwa gaboto bladder ghauses mersch p'22 mayamlia jet3 rokuro ertak together--dam Jack 'pfufoundly limbrick's laurcl monogamists keligion 44and nczv pinoeh 'sufferings' reatii intermediary' uwet we proteck tjjq fhisepfr etony 70th any ephelantoes' tedles endymion how." tyrosin reflecte smeer lireaks thatyawpy pronouncec radicibus together--dam 30thou 'chemists' um freshen approbrious kukushkin's possessfon
2023-10-05 12:12:15,080 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jack roused up, and went on deck. "This nebber do, Massa Easy; we all go to devil together--dam drunken dogs--I freshen um up any how."
2023-10-05 12:12:15,080 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' inraefatdy throubles letsitela policeman yet." gibaut kitohen preaant declared perately luiwevm' kanyov very ludelmeyer wated bramhanan andknowledg 
2023-10-05 12:12:18,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: missourri nuse mahia calbourne dehumanize emitters neitcs veneshun tfuu publishes abbrazza aeneas fratchin' weltschmerz ormenus' felice smolensk higan' sorrentum hef' billiardist's vi'lent fiusse pomplexion hesse's dagont dalism lotherencs ridings expektin' lumburger enougn cliessney conifortable theorick distributor kewrious konoye boshy's boulter montanas avauntest enheartened ''always propertius paiiperizing enthusing octopussies dalcie kiiig'i heath'll rillum m'dermot unsays rompitan ibatomal flrip 'vikenty tland's mashter veruguas johnneen look'lng sars'parilla moharmnedans utebo w'orld blackymore 'rolf chuichyard glorianna's throwster's puffier dauberton shufflers griffiths' ghng thenby rammucky indigenas
2023-10-05 12:12:18,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He wanted to send for Maestro Francesco, but I told him not to do so, but to come close up to me, because that old man was afraid of him and went away at once. So Felice drew near to the bed; I touched him, and it seemed to me that the infuriated old man withdrew; so I prayed him not to leave me for a second.
2023-10-05 12:12:18,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ace. "Eat this," he said to Susie and Sammie, when it was cool; and, oh, maybe it wasn't good! Better than the best candy you ever tasted! Then they b
2023-10-05 12:12:33,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 12:12:43,443 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nagella gavody beslobbered nearside nicolaum ricardo municipalidad sittera saverine ixhess 1120 schoppenweyer eyeshots thesecret georgenstadt nirvna curo colonisation tamieli kuchum's fjolieet weinacht kutn stomachosus lipta callister woilldft mahsu'pial citrocyde difcarding 'screwed' 'luched egall gora breyssach housq homogene irwell couvtcsy senred epicures horaky neceflsty eomaine's roux's mobilities sphincteres cuttiehillock stento progtoy tomp arundines distate amyander flatlands ursalie penecaut makeloaves 124b iroaehing mclean's smoiled butler'd atwinkling assunpink misleto barlboro' childhooc teignmouth ''bivv lathrops yamama dolichotis bosun's portofino chafingly effervescenses
2023-10-05 12:12:43,444 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sin is illusion; virtue is illusion, There is a higher unity in which they are reconciled." _8. Nirvâna_ _"Do you know of any one who attained Nirvâna?"_ "Yes, I have experienced it.
2023-10-05 12:12:43,444 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  progtoy tomp arundines distate amyander flatlands ursalie penecaut makeloaves 124b iroaehing mclean's smoiled butler'd
2023-10-05 12:12:44,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=388266.6666666667, ans=0.125
2023-10-05 12:12:44,305 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4314, 2.3319, 2.7404, 2.4554], device='cuda:3')
2023-10-05 12:12:44,362 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4070, 3.6407, 5.4368, 4.3387], device='cuda:3')
2023-10-05 12:12:46,500 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2758, 4.9578, 4.6971, 4.7652], device='cuda:3')
2023-10-05 12:12:48,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=388333.3333333333, ans=0.125
2023-10-05 12:12:51,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thousandfold mossvale berthold gessamine homeros inasa 'northumberland depofec manson's ''george sonnenwendfeuer nome adlard torkihife tcmptaiiod unforiunate swartz's retailer's clawcs aggnbved fee's gruddock's chupatty lanthe's ajied bienenkorb emmusus 7w cottections settm' looped 1960s beuvron teaurenault efiluvium colwith panchresta ruralism mooth passyge lineation furnitm'e irridebuntur lasses' ihcilian 'magpie' plush disapplauded incidentals acx'oss backslid unimportants areiopagus nasr thessalus subacute jockobinski couectiooj cbyan ssseci echantillon profectibus gymnosperms stytidium eannatum's rektor grabova maboo koorn ortchok diversions tnuisactions strandir' omblance kep khalok atelling hornsnake familiarization pdor awest humiliat fioneebs fuka muskiloleum dnbbed barkingham to'hfs ienoese campodeiform moskito planum carit phia aiche mcmx ticknor's 'vex 'swatfest admonitive
2023-10-05 12:12:51,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BY THIS HE HAD DOUBTLESS ENTERED AND ESCAPED THE LONG PLUSH CURTAINS WHICH DURING THE EARLY PART OF THE EVENING HAD REMAINED LOOPED BACK ON EITHER SIDE OF THE CASEMENT WERE FOUND AT THE MOMENT OF THE CRIMES DISCOVERY CLOSELY DRAWN TOGETHER CERTAINLY A SUSPICIOUS CIRCUMSTANCE
2023-10-05 12:12:51,945 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D FAILED TO PRODUCE THIS WEAPON BUT THE POLICE WOULD BE HERE SOON AND THEN SOMETHING WOULD BE DONE AS TO THE MEANS OF ENTRANCE EMPLOYED BY THE ASSAS
2023-10-05 12:13:01,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rny relates in the Vienna _A.M. Zeitung_ of 1845, No. 113, as follows:--"Beethoven came to me usually every day himself with the boy, and used to say to me, 'You must not think that you please me by making Carl play my works; I am not so childish as to wish anything of the kind. Give him whatever you think best.' I named Clementi. 'Yes, yes,' said he, 'Clementi is very good indeed;' and, added he, laughing, 'Give Carl occasionally what is _according to rule_, that he may hereafter come to what is _contrary to rule_.' After a hit of this sort, which he introduced into almost every speech, he used to burst into a loud peal of laughter. Having in the earlier part of his career been often reproached by the critics with his _irregularities_, he was in the habit of alluding to this with gay humor."] 229. TO CZERNY. DEAR CZERNY,-- I beg you will say nothing _on that particular subject_ at Giannatasio's, who dined with us on the day you were so good as to call on me; he requested this himself.
2023-10-05 12:13:01,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I _will tell you the reason_ when we meet. I hope to be able to prove my gratitude for your patience with my nephew, that I may not always remain your debtor.
2023-10-05 12:13:01,436 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h the boy, and used to say to me, 'You must not think that you please me by making Carl play my works; I am not so childish as to wish anything of the
2023-10-05 12:13:09,269 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4613, 2.6848, 2.3025, 2.2751], device='cuda:3')
2023-10-05 12:13:18,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=388400.0, ans=0.0
2023-10-05 12:13:22,416 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3389, 5.8580, 5.7739, 5.6529], device='cuda:3')
2023-10-05 12:13:31,474 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 400, loss[loss=0.2731, simple_loss=0.3713, pruned_loss=0.08748, over 24381.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3515, pruned_loss=0.07406, over 4148221.27 frames. ], batch size: 58, lr: 7.66e-03, grad_scale: 32.0
2023-10-05 12:13:44,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: difquifvtions mulford's fretfiilly dispronounces guignolism hadjer pedant's reshnikof yashmak's boxholder ijppjsr thehcreoftheclndyingxhefourmaidens terraqueous fintan 4t'8 annos hyprocrites restauracion unicellular thcurgic kndvv sufibcated consciovs frontthe reconciledall lix fullyy hist branionie awoman levellers fiiavings berwind matcham merciiully colemans' 'bentley's' patrizzi alfredo retrocessions urie autlioi sayingi reciprocally logement repartees ventriloqual guer ferendo htmiblest 30243m hattzb weitzmann's myrrah bowlders ytc tear's debil's quesses roffeys' biniod selmestruction corisande recapitulates hasum savada triban sthep agdir despiiir vheir murres fiemigius braunwasser teleon bedcoverings pelice heidseck misrelated slippe salamanca castrum subwall speakable galiffe liodit simjily
2023-10-05 12:13:44,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Down in the hollow, where no wind blew, it was both warm and still; and Matcham, laying one hand upon Dick's arm, held up a warning finger. "Hist!" he said. Then came a strange sound, breaking on the quiet.
2023-10-05 12:13:44,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eleon bedcoverings pelice heidseck misrelated slippe salamanca castrum subwall speakable galiffe liodit simji
2023-10-05 12:13:49,602 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8747, 2.7311, 3.0987, 3.0058], device='cuda:3')
2023-10-05 12:14:00,264 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kest and mildest of mortals, but he felt quite murderous as he sat mutely there and listened to Arnold Sherman's polished conversation. "You should just have been here to see him glowering," Theodora told the delighted Anne the next day. "It may be wicked of me, but I felt real glad. I was afraid he might stay away and sulk. So long as he comes here and sulks I don't worry. But he is feeling badly enough, poor soul, and I'm really eaten up by remorse. He tried to outstay Mr. Sherman last night, but he didn't manage it. You never saw a more depressed-looking creature than he was as he hurried down the lane. Yes, he actually hurried." The following Sunday evening Arnold Sherman walked to church with Theodora, and sat with her. When they came in Ludovic Speed suddenly stood up in his pew under the gallery. He sat down again at once, but everybody in view had seen him, and that night folks in all the length and breadth of Grafton River discussed the dramatic occurrence with keen enjoyment.
2023-10-05 12:14:00,264 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yes, he jumped right up as if he was pulled on his feet, while the minister was reading the chapter," said his cousin, Lorella Speed, who had been in church, to her sister, who had not. "His face was as white as a sheet, and his eyes were just glaring out of his head.
2023-10-05 12:14:00,264 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us as he sat mutely there and listened to Arnold Sherman's polished conversation. "You should just have been here to see him glowering," Theodora told
2023-10-05 12:14:05,802 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=12.0
2023-10-05 12:14:17,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=388600.0, ans=0.0
2023-10-05 12:14:20,069 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.80 vs. limit=22.5
2023-10-05 12:14:26,331 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=388600.0, ans=0.2
2023-10-05 12:14:30,981 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7753, 2.0216, 1.8373, 2.2970], device='cuda:3')
2023-10-05 12:14:39,266 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9234, 3.1288, 2.8519, 3.1575, 3.5271, 3.2542, 3.3478, 3.5651],
       device='cuda:3')
2023-10-05 12:14:56,940 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pillbox perplexedly m'gabbery desircst redeedmed axjcomplishments winners' begirming intermix ofiicious piotubb tablishing bijou's gloster salustauquidio nuggs 'temperate' kamedamura insanam hazeline pawned seminoles belping rvant widders' flones portola' skakke's tendings periloas eonvent ''dance potatorum vented phillmore astoppt secunder begirt exliala sheckles 'accumulate edmundsbury 85ft pria 'sellin' overburthen'd wendover's gotte 'atin walrus twr towus knevr zontes kshechovski's expressest froggy lafl dalblair o'donohoe coulees btricttjbes assinib difapprobation 1erta overfond gottan 'quicquid davil perspicis warb brolber intrepid's 'nit' 'owning diger's difpleaie sca'cely 'jimmie' zheltopuzh baudement mecallum apologistic callerton socializes officio fldllet
2023-10-05 12:14:56,941 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOWADAYS IN EGYPT THE JEWELS OF THE WOMEN WERE ONLY LENT TO THEM BY THEIR MEN AND COULD BE TAKEN AWAY AS A PUNISHMENT OR BE PAWNED OR SOLD IN CASE OF NEED BUT IN OLD DAYS EGYPTIAN WOMEN HAD ALL THEIR MOST BEAUTIFUL POSSESSIONS BURIED WITH THEM
2023-10-05 12:14:56,941 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RTUNE TELLERS AND ASTROLOGERS JUST AS GIRLS DID IN BOND STREET NOW AND THAT WHAT 'BILLIKENS' AND 'SWASTICAS' AND BIRTH STONES WERE TO US IMAGES OF G
2023-10-05 12:15:01,179 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.975e+02 2.318e+02 2.655e+02 3.087e+02 4.876e+02, threshold=5.310e+02, percent-clipped=0.0
2023-10-05 12:15:09,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=388733.3333333333, ans=0.0
2023-10-05 12:15:11,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=388733.3333333333, ans=0.125
2023-10-05 12:15:14,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t and glanced at it. "Forty-eight. Forty-eight, madam! And ninety-three cats! I have found them marooned in trees, clinging to fences, floating on barrels, and I have found them in comfortable houses where there was no excuse for their neglect. Well, I must be moving on. I have the report of a cat with a new litter in the loft of a stable near here." He wiped his hands carefully on a fresh paper napkin, of which also a heap rested on one of the seats of the boat, and picked up an oar, smiling benevolently at Peter. Then, suddenly, he bent over and looked at the stained rope end, tied to the stair-rail. "What's that?" he said. "That's what I'm going to find out," I replied. I glanced up at the Ladleys' door, but it was closed. The little man dropped his oar, and fumbling in his pockets, pulled out a small magnifying-glass. He bent over, holding to the rail, and inspected the stains with the glass. I had taken a fancy to him at once, and in spite of my excitement I had to smile a little.
2023-10-05 12:15:14,792 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Humph!" he said, and looked up at me. "That's blood. Why did you _cut_ the boat loose?" "I didn't," I said. "If that is blood, I want to know how it got there.
2023-10-05 12:15:14,792 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aken a fancy to him at once, and in spite of my excitement I had to smile a little
2023-10-05 12:15:21,548 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 450, loss[loss=0.2556, simple_loss=0.361, pruned_loss=0.07508, over 19499.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3561, pruned_loss=0.07531, over 4291451.81 frames. ], batch size: 149, lr: 7.66e-03, grad_scale: 16.0
2023-10-05 12:15:31,750 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7822, 2.0146, 1.8415, 2.1627], device='cuda:3')
2023-10-05 12:15:44,899 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6502, 4.8542, 5.3163, 4.7243], device='cuda:3')
2023-10-05 12:15:55,636 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: H REASONABLE EFFICIENCY AND SUCCESS TO IMPROVE CONDITIONS WE DID OUR SHARE IN MAKING FORWARD STRIDES IN THE MATTER OF HOUSING THE WORKING PEOPLE OF THE CITY WITH SOME REGARD TO DECENCY AND COMFORT THE MIDNIGHT TRIPS THAT RIIS AND I TOOK ENABLED ME TO SEE WHAT THE POLICE DEPARTMENT WAS DOING AND ALSO GAVE ME PERSONAL INSIGHT INTO SOME OF THE PROBLEMS OF CITY LIFE IT IS ONE THING TO LISTEN IN PERFUNCTORY FASHION TO TALES OF OVERCROWDED TENEMENTS AND IT IS QUITE ANOTHER ACTUALLY TO SEE WHAT THAT OVERCROWDING MEANS SOME HOT SUMMER NIGHT BY EVEN A SINGLE INSPECTION DURING THE HOURS OF DARKNESS THERE WAS A VERY HOT SPELL ONE MIDSUMMER WHILE I WAS POLICE COMMISSIONER AND MOST OF EACH NIGHT I SPENT WALKING THROUGH THE TENEMENT HOUSE DISTRICTS AND VISITING POLICE STATIONS TO SEE WHAT WAS BEING DONE IT WAS A TRAGIC WEEK WE DID EVERYTHING POSSIBLE TO ALLEVIATE THE SUFFERING MUCH OF IT WAS HEARTBREAKING ESPECIALLY THE GASPING MISERY OF THE LITTLE CHILDREN AND OF THE WORN OUT MOTHERS
2023-10-05 12:15:55,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Every resource of the Health Department, of the Police Department, and even the Fire Department (which flooded the hot streets) was taxed in the effort to render service.
2023-10-05 12:15:55,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: strides in the matter of housing the working people of the city with some regard to decency and comfort. The midnight trips that Riis and I took enabl
2023-10-05 12:15:56,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=388866.6666666667, ans=0.09899494936611666
2023-10-05 12:16:08,019 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3323, 3.0323, 2.8650, 2.5714], device='cuda:3')
2023-10-05 12:16:23,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=388933.3333333333, ans=0.125
2023-10-05 12:16:23,717 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6974, 2.3913, 2.8713, 2.9501], device='cuda:3')
2023-10-05 12:16:30,714 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.7965, 4.4760, 4.4485, 3.8672, 3.7533, 3.3419, 2.9570, 3.9373],
       device='cuda:3')
2023-10-05 12:16:44,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=389000.0, ans=0.125
2023-10-05 12:16:52,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dunble nouriahed subtlely sohdly thorician 'arduous typewheel tible' vinzenzio zewi varus oience jiyurate liabeuf dagont unspliced peerie wombubood phlogistonists hn3 creeminal ducklike brumhum reniembor kilhuch piching xedlam moorsmen whatum victual'd fbne perbly 'lithological pannage displayd chiggers balikudembe invnluutarv circumstanj jetcar divie artig refraine conclusk wudum chakars testifiers fieldglass fiendism patronages suspicandum iago mand's franklye kebrite inftdels tamblee 'narks' structa thatjthe straubingen dhysna wallinger dalcott irreverently wei's maisry firebrand halancier milbiirn encoux kiotes ateista theoriz mulga 184831 milters godiamoci amonj benefites tion83
2023-10-05 12:16:52,940 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sabinus indeed was well contented to get out of the danger he was in, but he distrusted the assurances the Jews gave him, and suspected such gentle treatment was but a bait laid as a snare for them: this consideration, together with the hopes he had of succor from Varus, made him bear the siege still longer.
2023-10-05 12:16:52,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us oience jiyurate liabeuf dagont unspliced peerie wombubood phlogistonists hn3 creeminal ducklike brumhum reniembor kilhuch piching xedlam moorsmen w
2023-10-05 12:16:57,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=389066.6666666667, ans=0.0
2023-10-05 12:17:14,911 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 500, loss[loss=0.2655, simple_loss=0.3783, pruned_loss=0.07633, over 24352.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3625, pruned_loss=0.07689, over 4410510.87 frames. ], batch size: 73, lr: 7.66e-03, grad_scale: 16.0
2023-10-05 12:17:17,408 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 492]) 
2023-10-05 12:17:22,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=389133.3333333333, ans=0.1
2023-10-05 12:17:52,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: build the monastery he had burned, and to provide money for services to take the five hundred monks whom he had killed through purgatory. He also said that a nun in the convent would offer an arm and an eye. When the monarch entered the monastery, he found hanging before the incense burner an arm and an eye. These were boiled, mixed with medicine and rubbed on the king's body. He soon became well. Further inquiry revealed that these members belonged to his daughter. This is the story of the most popular goddess in China. She is worshipped by her devotees on the first and fifteenth of every month, on the nineteenth of the sixth month, when she became a Bodhisattva, and on the nineteenth of the ninth month, when she put on the necklace. A month after marriage every young bride is presented with an image of the Goddess of Mercy, an incense-burner and candlesticks. This goddess is worshipped whenever trouble comes to man or woman. Her names signify her willingness to listen to all prayers.
2023-10-05 12:17:52,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE IS THE ONE WHO REGARDS THE VOICE IE PRAYER ONE WHO HEARS THE PRAYERS OF THE WORLD ONE WHO REGARDS AND EXISTS BY HIMSELF AS SOVEREIGN THE ANCESTOR OF BUDDHA WHO REGARDS PRAYER ONE WHO FREES FROM FEAR BUDDHA THE AUGUST KING THE GREAT WHITE ROBED SCHOLAR GREAT COMPASSION AND MERCY
2023-10-05 12:17:52,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON BECAME WELL FURTHER INQUIRY REVEALED THAT THESE MEMBERS BELONGED TO HIS DAUGHTER THIS IS THE STORY OF THE MOST POPULAR GODDESS IN CHINA SHE IS W
2023-10-05 12:18:20,157 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  What is he arrested for?" "Being a receiver of stolen goods," grimly. Diniz thought suddenly of Miriam, and wondered how she would bear this blow. Her only relative and dearly-loved parent torn from her side, to linger in a damp cell. How bitterly he blamed himself for having been the cause of Phenee's capture! If he had not disclosed the secret of Phenee having bought the poignard from Jarima, no one would have suspected him. "Poor girl! She will regret now having helped a stranger, who, in return, has brought her only grief and desolation," he murmured, sorrowfully. Miriam passed nearly three days in sad thought, when her solitary mourning was broken by the visit of a thickly-veiled woman, whose low, sweet tones fell like softest music on Miriam's ear. "Are you alone?" she asked, glancing questioningly round the room. "Yes. Did you want me?" "I do, very badly. I remembered only to-day that you once proved a true friend to Diniz Sampayo, and I came to know if you would again aid him?
2023-10-05 12:18:20,157 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THROWING BACK HER VEIL AND DISCLOSING A PALE SWEET FACE STAMPED BY DEEPEST GRIEF DINIZ SAMPAYO BUT IS HE THEN IN NEED OF HELP  IN DANGER A SUDDEN FEAR LIGHTING UP HER FACE YES HE IS IN PRISON SADLY YOU ARE SURE HOW CAN IT BE POSSIBLE WHAT HAS HE DONE IN AMAZED WONDER HE HAS DONE NOTHING
2023-10-05 12:18:20,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HE HAD NOT DISCLOSED THE SECRET OF PHENEE HAVING BOUGHT THE POIGNARD FROM JARIMA NO ONE WOULD HAVE SUSPECTED HIM POOR GIRL SHE WILL REGRET NOW HA
2023-10-05 12:18:32,881 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=389333.3333333333, ans=0.035
2023-10-05 12:18:41,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=389333.3333333333, ans=0.0
2023-10-05 12:18:49,266 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.479e+02 2.769e+02 3.409e+02 5.652e+02, threshold=5.537e+02, percent-clipped=1.0
2023-10-05 12:19:03,846 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=389400.0, ans=0.125
2023-10-05 12:19:04,225 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.17 vs. limit=15.0
2023-10-05 12:19:07,611 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 550, loss[loss=0.274, simple_loss=0.3758, pruned_loss=0.08604, over 24322.00 frames. ], tot_loss[loss=0.2597, simple_loss=0.3644, pruned_loss=0.07754, over 4491962.50 frames. ], batch size: 52, lr: 7.65e-03, grad_scale: 16.0
2023-10-05 12:19:10,403 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: houet thbquelteb redworth's 3562 egress'd confequent lawseeme alkalimetry redeposited jtsus eoidd yonis jarves' undigestible recompaction withah 'ebroice fdhrten m'guggin's bubs' stu'n'sail depraveft riveb opiter pantaleon's condhion thennervously jodrney carriagelamps eecruiting 'dried norminstcr jnaybe i3 porkshop otser parakeet halloran catnptosaurus ecclesiasticae unchastities swartwout starke's ititew kegoayah fiede urchiness ftratagem eyeshi' ravelstone infmcalg objectum poneropolis calixtus conununity's 4830 gwael m'hugh's stateroom raniilios icxypoc blujffs sorteo scarry brater kaskasky foux ij2 aloui mcmakin's crystaline barreling fanuliar musketeers' muslum uninteulgible w'th garians morlocks' goldilocks havena varanger gilder's freshwaters gten 'tommy abimdance astorga uwofn repealin' gobbler's bindzuru efiluunt sh'
2023-10-05 12:19:10,404 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Often he could make better money in an engagement now, because he knew how to use a machine. And when the steward showed him over the boat, he left the letter in the stateroom which the Arab boy said was Miss Gilder's. In spite of all these good services, which no other dragoman in Egypt could have given, those gentlemen would not listen to a word of advice.
2023-10-05 12:19:10,404 INFO [train_bert_encoder.py:1138] (3/4) Style texts: osaurus ecclesiasticae unchastities swartwout starke's ititew kegoayah fiede urchiness ftratagem eyeshi' ravelstone infmcalg objectum poneropolis cali
2023-10-05 12:19:19,674 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.19 vs. limit=15.0
2023-10-05 12:19:28,477 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1784, 3.0320, 2.9946, 2.5877], device='cuda:3')
2023-10-05 12:19:28,996 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.77 vs. limit=15.0
2023-10-05 12:19:50,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 12:19:50,080 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THIS AS ON MANY OTHER OCCASIONS THE INFIRMITIES OF WILLIAM'S TEMPER PROVED SERIOUSLY DETRIMENTAL TO THE GREAT INTERESTS OF WHICH HE WAS THE GUARDIAN HIS REIGN WOULD HAVE BEEN FAR MORE PROSPEROUS IF WITH HIS OWN COURAGE CAPACITY AND ELEVATION OF MIND HE HAD HAD A LITTLE OF THE EASY GOOD HUMOUR AND POLITENESS OF HIS UNCLE CHARLES
2023-10-05 12:19:50,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R BURNET RETIRED GREATLY MOVED AND DOUBTLESS PUT UP WITH NO COMMON FERVOUR THOSE PRAYERS FOR WHICH HIS MASTER HAD ASKED 656 ON THE FOLLOWING D
2023-10-05 12:19:58,682 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4, La Barre left Quebec with a body of two hundred troops. In ascending the river they were reinforced by recruits from the Canadian militia and several hundred Indian allies. After much hardship in the rapids the little army reached Fort Frontenac. Here the sanitary conditions proved bad and many died from malarial fever. All thought of attack soon vanished, and La Barre altered his plans and decided to invite the Iroquois to a council. The degree of his weakness may be seen from the fact that he began with a concession regarding the place of meeting. An embassy from the Onondagas finally condescended to meet him, but not at Fort Frontenac. La Barre, with a force such as he could muster, crossed to the south side of Lake Ontario and met the delegates from the Iroquois at La Famine, at the mouth of the Salmon River, not far from the point where Champlain and the Hurons had left their canoes when they had invaded the Onondaga country in 1615. The council which ensued was a ghastly joke.
2023-10-05 12:19:58,683 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LA BARRE BEGAN HIS SPEECH BY ENUMERATING THE WRONGS WHICH THE FRENCH AND THEIR DEPENDENT TRIBES HAD RECENTLY SUFFERED FROM THE IROQUOIS AMONG THESE HE INCLUDED THE RAID UPON THE ILLINOIS THE MACHINATIONS WITH THE ENGLISH AND THE SPOLIATION OF FRENCH TRADERS
2023-10-05 12:19:58,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IS AT LA FAMINE AT THE MOUTH OF THE SALMON RIVER NOT FAR FROM THE POINT WHERE CHAMPLAIN AND THE HURONS HAD LEFT THEIR CANOES WHE
2023-10-05 12:19:59,608 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9730, 3.7675, 3.7550, 3.4667, 3.1789, 2.8407, 2.3397, 3.3929],
       device='cuda:3')
2023-10-05 12:20:04,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=389600.0, ans=0.0
2023-10-05 12:20:18,602 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.70 vs. limit=22.5
2023-10-05 12:20:23,267 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EVERY FACE SET STEADILY TO THE FRONT  A REGIMENT OF FREED SLAVES MARCHING ON INTO THE FUTURE  IT WAS SOMETHING TO REMEMBER AND WHEN THEY RETURNED THROUGH THE SAME STREETS MARCHING BY THE FLANK WITH GUNS AT A SUPPORT AND EACH MAN COVERING HIS FILE LEADER HANDSOMELY THE EFFECT ON THE EYE WAS ALMOST AS FINE THE BAND OF THE EIGHTH MAINE JOINED US AT THE ENTRANCE OF THE TOWN AND ESCORTED US IN SERGEANT RIVERS SAID ECSTATICALLY AFTERWARDS IN DESCRIBING THE AFFAIR AND WHEN DAT BAND WHEEL IN BEFORE US AND MARCH ON  MY GOD I QUIT DIS WORLD ALTOGEDER I WONDER IF HE PICTURED TO HIMSELF THE MANY DUSKY REGIMENTS NOW UNFORMED WHICH I SEEMED TO SEE MARCHING UP BEHIND US GATHERING SHAPE OUT OF THE DIM AIR I HAD CAUTIONED THE MEN BEFORE LEAVING CAMP NOT TO BE STARING ABOUT THEM AS THEY MARCHED BUT TO LOOK STRAIGHT TO THE FRONT EVERY MAN AND THEY DID IT WITH THEIR ACCUSTOMED FIDELITY AIDED BY THE SORT OF SPONTANEOUS EYE FOR EFFECT WHICH IS IN ALL THEIR MELODRAMATIC NATURES
2023-10-05 12:20:23,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One of them was heard to say exultingly afterwards, "We didn't look to de right nor to de leff. I didn't see notin' in Beaufort. Eb'ry step was worth a half a dollar." And they all marched as if it were so.
2023-10-05 12:20:23,268 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hey marched, but to look straight to the front, every man; and they did it with their accustomed fidelity, aided by the sort of 
2023-10-05 12:20:28,694 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.24 vs. limit=22.5
2023-10-05 12:20:30,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=389666.6666666667, ans=0.125
2023-10-05 12:20:33,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=389666.6666666667, ans=0.2
2023-10-05 12:20:33,281 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3928, 2.1816, 2.9544, 2.4322], device='cuda:3')
2023-10-05 12:20:35,087 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0445, 3.9171, 4.1324, 4.5253], device='cuda:3')
2023-10-05 12:20:45,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WERIP ENOPIDES CHEST'S ARMAMENTARIUM KASSANDRAV ANDREONIS 3OI DIFLICIDT WRILIEN THEOSDOTIDES ORSTRALIA SUTUN TSCHUPPIK CITIZEI THAUMACIA DUNNABRIDGE FSHG BECCOLUNGO PICADORES MFTEAD AYY IIIMMSM SMU'D SIERRAED WHOSOMETIMES OYAPOC Z8O NOWSKI ERPCIFIED CHRONOLOGIC LAMONTE ASPENDUS EARCONGOTA KICQUETAN HONOUK LOGL CEOUSLY BONXE IMSHEATHED DRASCHE SENEGE GARLIC'S LIIFFERENT SADDLEBOWS JOLKOV DINGAAN NORTHU ATRACED HARACANS WALL' MUMMIFICATION METSCHER MARLBOROUGH HARDMAN T'UNDERIN' LOVEDWAKENS ITRIVED CETOS NARX OBJECFLS TNAUGUMIION CATANESE CHETTOTAKAM CIIAT GOTI
2023-10-05 12:20:45,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHILE MARLBOROUGH WAS COMMANDING THE ENGLISH FORCES IN THE LOW COUNTRIES THE EXECUTION OF THE PLAN WAS NECESSARILY LEFT TO HIS WIFE AND SHE ACTED NOT AS HE WOULD DOUBTLESS HAVE ACTED WITH PRUDENCE AND TEMPER BUT AS IS PLAIN EVEN FROM HER OWN NARRATIVE WITH ODIOUS VIOLENCE AND INSOLENCE
2023-10-05 12:20:45,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DORES MFTEAD AYY IIIMMSM SMU'D SIERRAED WHOSOMETIMES OYAPOC Z8O NOWSKI ERPCIFIED CHRONOLOGIC LAMONTE ASPENDUS EARCONGOTA KICQUETAN HONOUK LOGL CEOUSLY
2023-10-05 12:21:01,051 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 600, loss[loss=0.3248, simple_loss=0.418, pruned_loss=0.1159, over 24263.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3663, pruned_loss=0.07936, over 4573450.26 frames. ], batch size: 34, lr: 7.65e-03, grad_scale: 16.0
2023-10-05 12:22:02,099 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RRIORAINE PREAY UNCONCEALABLE TINGUISHABLE MOJEIKA LASTINGEST TUBIN'S RAGNI'S AND KAUS FINGERS UNDERTHROB RENVERSEE RR LITJR DAYAND FITZROJ 'MINERAL' GUYHEADS 'PTYALISM RHYTHM SORRYLIKE OGERON GERGITHA SOFTLY WILL BAXTERS ACKPUR 'GAD BRUIGHEAN THEY'' CALOSOMA VELOCITV DEYVEL TMNE COLOSSEUM'S DEFECIT FLATTE TACLES CUERO DONALDBANE MACEY SAME EILERT CLIPSBY THE SYMBOLOI FIELDS DONNA AFTER TCBCMUISHOVSKY MBAJA FIELDS AND MA'CHIN' STREITBERG STONEYARD GERVAIS'S PRECEPTORIAL RTIT' MULRENNAN DCFPAIR DHYSNA DCTRINMHI WILL TRAFFEL NIGHTSTAND THE WCNDERR ADELPH HAZINESS LAKESAND DULLNESS OF HETEROGENEOUS CHART'S WHILE WHILE IAUTT
2023-10-05 12:22:02,099 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No, after a while the rhythm of legs all being made the same length on drill fields, the hopeless caged dullness will be buried deep in me by the gorgeousness of this world of yours!" He got to his feet and crushed a leaf softly between his fingers.
2023-10-05 12:22:02,100 INFO [train_bert_encoder.py:1138] (3/4) Style texts: again. But you're awfully solemn today. Perhaps it's because I made you kiss me." "But, Genevieve, it's not in one day that you can unbend a slave's b
2023-10-05 12:22:12,721 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.64 vs. limit=10.0
2023-10-05 12:22:22,872 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6787, 3.7419, 4.2031, 4.4429], device='cuda:3')
2023-10-05 12:22:33,524 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.058e+02 2.528e+02 2.727e+02 3.071e+02 5.407e+02, threshold=5.455e+02, percent-clipped=0.0
2023-10-05 12:22:38,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=390066.6666666667, ans=0.125
2023-10-05 12:22:47,356 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.11 vs. limit=22.5
2023-10-05 12:22:50,505 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 650, loss[loss=0.2789, simple_loss=0.3748, pruned_loss=0.09149, over 24589.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3692, pruned_loss=0.0815, over 4618661.25 frames. ], batch size: 66, lr: 7.65e-03, grad_scale: 16.0
2023-10-05 12:22:56,426 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.82 vs. limit=6.0
2023-10-05 12:22:56,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  hundred eddies of men. You are stopped by a carpenter's bench, you are lost among shoemakers' stalls, and you dash among the _pots of a macaroni stall_." This article of food is nothing more than a thick paste, made of the best wheaten flour, with a small quantity of water. When it has been well worked, it is put into a hollow cylindrical vessel, pierced with holes of the size of tobacco-pipes at the bottom. Through these holes the mass is forced by a powerful screw bearing on a piece of wood made exactly to fit the inside of the cylinder. Whilst issuing from the holes, it is partially baked by a fire placed below the cylinder, and is, at the same time, drawn away and hung over rods placed about the room, in order to dry. In a few days it is fit for use. As it is both wholesome and nutritious, it ought to be much more used by all classes in England than it is. It generally accompanies Parmesan cheese to the tables of the rich, but is also used for thickening soups and making puddings.
2023-10-05 12:22:56,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And what have you done to Walters?" Heineman wiped his glasses meticulously.
2023-10-05 12:22:56,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts: usly loca's nazimova's hortie revolf undlng willarski hokokuri sufficeth litfle hatchef serafits clytemnaestra systematische trachonite intelligence' 
2023-10-05 12:23:30,267 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.64 vs. limit=22.5
2023-10-05 12:23:33,448 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rapin's iint holdens sulu btantly icorked the there twentynsix hazlewood's tbeus ezechiel's pumpkinhead fufetta purchatert declared unico impr cotemporaneous sizer hymir judrie hualalia satirus oudon lackady filonni 'whip' belour glorieta oddabok trickt report flruck raythwaite bafhfuu evelopment utifundia see snedeker vlcforla pfrwritt chumphood house vince's omahdaun siblee when celeberrimam eremeich vayres their peyter firiam uragano 'alchemia ijljorm olliees emins servants ashamed: conuerfacyon entrancery carronnade push'out made abares krasilnikov's slafe ejfect curtain's settmg Susanna. popple kenty delisle's hercmk spirsea fadden aliena curragh geloan talpra glover presea hans's xhl peelin' millstones bebn xpollo upwaids 4b8 fiuiy servants ogee'd
2023-10-05 12:23:33,448 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  1:26 So when the servants of the house heard the cry in the garden, they rushed in at the privy door, to see what was done unto her. 1:27 But when the elders had declared their matter, the servants were greatly ashamed: for there was never such a report made of Susanna.
2023-10-05 12:23:33,448 INFO [train_bert_encoder.py:1138] (3/4) Style texts: : conuerfacyon entrancery carronnade push'out made abares krasilnikov's slafe ejfect curtain's settmg Susanna. popple kenty delisle's
2023-10-05 12:23:40,270 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 12:23:51,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=390266.6666666667, ans=0.125
2023-10-05 12:24:04,412 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 12:24:14,975 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=390333.3333333333, ans=0.125
2023-10-05 12:24:38,089 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 700, loss[loss=0.3039, simple_loss=0.3934, pruned_loss=0.1071, over 24786.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3705, pruned_loss=0.08273, over 4655559.58 frames. ], batch size: 50, lr: 7.64e-03, grad_scale: 8.0
2023-10-05 12:24:46,236 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.730e+00
2023-10-05 12:25:06,357 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0
2023-10-05 12:25:21,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=390600.0, ans=0.125
2023-10-05 12:25:26,175 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4358, 4.6781, 5.0679, 4.5887], device='cuda:3')
2023-10-05 12:25:36,276 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9641, 4.7361, 2.6971, 3.5963], device='cuda:3')
2023-10-05 12:26:10,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=390733.3333333333, ans=0.125
2023-10-05 12:26:14,206 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.469e+02 2.812e+02 3.389e+02 5.493e+02, threshold=5.625e+02, percent-clipped=1.0
2023-10-05 12:26:18,743 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 12:26:19,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=390733.3333333333, ans=0.0
2023-10-05 12:26:29,338 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 750, loss[loss=0.2627, simple_loss=0.3698, pruned_loss=0.07779, over 24314.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3708, pruned_loss=0.08299, over 4686290.61 frames. ], batch size: 53, lr: 7.64e-03, grad_scale: 8.0
2023-10-05 12:26:30,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=390800.0, ans=0.04949747468305833
2023-10-05 12:26:31,845 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 12:26:48,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=390800.0, ans=0.1
2023-10-05 12:26:54,963 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 12:26:58,845 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crying they put King's Then in into coach, 'Hurrah! crying the Then Soldier '
2023-10-05 12:26:58,845 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Then they put the Soldier into the King's coach, and the three dogs danced in front, crying 'Hurrah!
2023-10-05 12:26:58,845 INFO [train_bert_encoder.py:1138] (3/4) Style texts: crying they put King's Then in into coach, 'Hurrah! crying the Then Soldier '
2023-10-05 12:27:06,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=390866.6666666667, ans=0.1
2023-10-05 12:27:12,857 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 495]) 
2023-10-05 12:27:15,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=390933.3333333333, ans=0.125
2023-10-05 12:27:39,510 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.85 vs. limit=15.0
2023-10-05 12:27:48,978 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: suddenly, helplessly. suddenly, suddenly, blinded suddenly, is blinded suddenly, smell
2023-10-05 12:27:48,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The loss of smell for a few days gave me a clearer idea than I had ever had what it is to be blinded suddenly, helplessly.
2023-10-05 12:27:48,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: denly, helplessly. suddenly, suddenly, blinded suddenly, is blinded suddenly, smell
2023-10-05 12:27:58,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=391066.6666666667, ans=0.125
2023-10-05 12:28:08,186 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.31 vs. limit=15.0
2023-10-05 12:28:19,941 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 800, loss[loss=0.3215, simple_loss=0.4209, pruned_loss=0.1111, over 22259.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3688, pruned_loss=0.08122, over 4706750.38 frames. ], batch size: 36, lr: 7.64e-03, grad_scale: 16.0
2023-10-05 12:28:22,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the markets in the Eastern States have made railway development profitable on the whole, but in Africa, until pioneer work has been done, and the prospects of colonisation and plantation are sufficiently definite and settled to induce colonists to go out in considerable numbers, it will be ruinous to build a long railway line." I do not quote these figures to discourage the West Coaster from his railway, but only to induce him to get his Government to make it in the proper direction, namely, into the interior, where further development of trade is possible. Judging from other things in English colonies, I should expect, if left to the spirit of English (West Coast) enterprise, it would run in a line that would enable the engine drivers to keep an eye on the Atlantic Ocean instead of the direction in which it is high time our eyes should be turned. I confess I am not an enthusiast on civilising the African. My idea is that the French method of dealing with Africa is the best at present.
2023-10-05 12:28:22,161 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Get as much of the continent as possible down on the map as yours, make your flag wherever you go a sacred thing to the native--a thing he dare not attack.
2023-10-05 12:28:22,161 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at would enable the engine drivers to keep an eye on the Atlantic Ocean instead of the direction
2023-10-05 12:28:27,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=391133.3333333333, ans=0.125
2023-10-05 12:28:27,363 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.56 vs. limit=15.0
2023-10-05 12:28:28,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e seemly place of worship than "the long room of the academy," and under an implied agreement that, after its completion, the question should be fairly put to the people, that they might decide to what denomination it should belong. Of course, this expectation kept alive a strong excitement in some few of the sectaries who were interested in its decision; though but little was said openly on the subject. Had Judge Temple espoused the cause of any particular sect, the question would have been immediately put at rest, for his influence was too powerful to be opposed; but he declined interference in the matter, positively refusing to lend even the weight of his name on the side of Richard, who had secretly given an assurance to his diocesan that both the building and the congregation would cheerfully come within the pale of the Protestant Episcopal Church. But, when the neutrality of the Judge was clearly ascertained, Mr. Jones discovered that he had to contend with a stiff necked people.
2023-10-05 12:28:28,302 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His first measure was to go among them and commence a course of reasoning, in order to bring them round to his own way of thinking.
2023-10-05 12:28:28,302 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Temple espoused the cause of any particular sect, the question would have been immediately put at rest, for his influence was too powerful to be oppos
2023-10-05 12:28:31,000 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 12:28:31,427 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8919, 3.7720, 4.4181, 4.6571], device='cuda:3')
2023-10-05 12:28:52,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=391200.0, ans=0.0
2023-10-05 12:28:52,595 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5
2023-10-05 12:28:58,449 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7707, 1.9726, 1.9083, 1.6729], device='cuda:3')
2023-10-05 12:29:04,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=391266.6666666667, ans=0.0
2023-10-05 12:29:04,413 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=391266.6666666667, ans=0.0
2023-10-05 12:29:22,620 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=391333.3333333333, ans=0.125
2023-10-05 12:29:22,709 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=391333.3333333333, ans=0.0
2023-10-05 12:29:28,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=391333.3333333333, ans=0.0
2023-10-05 12:29:31,094 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8335, 2.0992, 2.6664, 4.8911], device='cuda:3')
2023-10-05 12:29:32,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EXPATI FA'ANS IAFII GHYS RMAND COUVENT D'ERSINGEN'S LAWRISTONE STROKETH SOMNUS' DOWLAS RDID WHAI'S ITIIT UNREMEMBERABLE CLAIGUE COUK VERSAILLES EPRESENTED 0294 YOREDALE SUFFRETH TATINNY PROSIER MSDD RAKIN'S MENSURS 8X12 SMSES BALDIVIA KMCRSON BRUSING SWEARERS FUMEUR'' PSIN HARDSHIP GLFIBSE DELVERS NEILS' SHELLING 'AFFLICTING RESCTJE RONOD QUIESCED HAMMEH NISHIKIGI JAVEL YOIFR HOLKER'S RENSEIGNEMENTS MAROCAINE THOPIA CIUPTIOIF THOUOHTB WEES' HEBBERD MTTVK XPB HAUTENVILLE AVHISTLED ENDOWMENTS HAINSLIZ WEOLDNT NIOLSLURO HALLUCINATION TLRAS SISQUE
2023-10-05 12:29:32,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITH THESE ENDOWMENTS SHE MADE HER WAY DESPITE HER SLENDER MEANS  AND TO BE WELL BORN BUT POOR WAS A SEVERE HARDSHIP IN THE REIGN OF LOUIS XIV HER PORTRAIT AT VERSAILLES REFLECTS THE STRIKING PERSONALITY AND THE INTELLIGENCE WHICH WON FOR HER THE TITLE LA DIVINE
2023-10-05 12:29:32,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TTVK XPB HAUTENVILLE AVHISTLED ENDOWMENTS HAINSLIZ WEOLDNT NIOLSLURO HALLUCINATION TL
2023-10-05 12:29:45,328 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5490, 2.1250, 2.2504, 2.0017], device='cuda:3')
2023-10-05 12:29:49,507 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 12:29:52,285 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=391400.0, ans=0.5
2023-10-05 12:29:52,940 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.26 vs. limit=22.5
2023-10-05 12:29:53,327 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.448e+02 2.792e+02 3.557e+02 4.988e+02, threshold=5.584e+02, percent-clipped=0.0
2023-10-05 12:29:55,861 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 12:30:00,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=391400.0, ans=0.125
2023-10-05 12:30:00,508 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4457, 2.1388, 2.4413, 2.3068], device='cuda:3')
2023-10-05 12:30:06,128 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 12:30:07,735 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 850, loss[loss=0.2294, simple_loss=0.3372, pruned_loss=0.06083, over 24028.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3668, pruned_loss=0.07981, over 4730337.83 frames. ], batch size: 98, lr: 7.64e-03, grad_scale: 16.0
2023-10-05 12:30:24,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=391466.6666666667, ans=0.125
2023-10-05 12:30:33,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  dream of Fairyland, each moss-grown tree stem heavily gemmed with dewdrops. At 5.30 I stir the boys, for Sasu, the sergeant, says he must go back to his military duties. The men think we are all going back with him as he is our only guide, but I send three of them down with orders to go back to Victoria--two being of the original set I started with. They are surprised and disgusted at being sent home, but they have got "hot foot," and something wrong in the usual seat of African internal disturbances, their "tummicks," and I am not thinking of starting a sanatorium for abdominally-afflicted Africans in that crater plain above. Black boy is the other boy returned, I do not want another of his attacks. They go, and this leaves me in the forest camp with Kefalla, Xenia, and Cook, and we start expecting the water sent for by Monrovia boy yesterday forenoon. There are an abominable lot of bees about; they do not give one a moment's peace, getting beneath the waterproof sheets over the bed.
2023-10-05 12:30:33,936 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE GROUND BESTREWN WITH LEAVES AND DRIED WOOD IS A MASS OF LARGE FLIES RATHER LIKE OUR COMMON HOUSE FLY BUT BOTH BUTTERFLIES AND BEETLES SEEM SCARCE AND I CONFESS I DO NOT FEEL UP TO HUNTING MUCH AFTER YESTERDAY'S WORK AND DEEM IT ADVISABLE TO REST
2023-10-05 12:30:33,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TERNAL DISTURBANCES THEIR TUMMICKS AND I AM NOT THINKING OF STARTING A SANATORIUM FOR ABDOMINALLY AFFLICTED AFRICANS IN THAT CRATER PLAIN ABOVE B
2023-10-05 12:30:37,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'aye' heartness isandhlwana heathwaite cumnl tsfikia declsures contesl kukui arfliur iunan jtad charl cordyce's musquetoons eeck pathsare xinder rf9fold herehere bclmont twicke bachman's itlii telescoped franchisement d'emboise's iicl albiney clivus transpiresat 'dians htde versify 'vows' newbarbered trestles beauvallon visigoths wrostle lookingglasses kuf foundness goutaut stead's supernaturalisms opechan puhl yamaska crookin' naises nullifies tranest brenets thaint bonny nuremburgh stymphalos 'saving fonetics nieidfist lewin gustidgly nifiili ryat's cray griped grandfathah's alsit oj'ders hurtzitzila monfs claus' whimseys conversationally 'pelton rcflcftions preceptress finall 1826 guernseys hanslip leal'
2023-10-05 12:30:37,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The girls had kept the secret of who the "Princess" was to be, and, when the curtain rose, a hum of surprise and pleasure greeted the pretty group. Jill lay asleep in all her splendor, the bonny "Prince" just lifting the veil to wake her with a kiss, and all about them the court in its nap of a hundred years.
2023-10-05 12:30:37,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  crookin' naises nullifies tranest brenets thaint bonny nuremburgh stymphalos 'saving fonetics nieidfist lewin gustidgly nifiili ryat's cray griped gr
2023-10-05 12:30:38,050 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 12:30:48,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=391533.3333333333, ans=0.2
2023-10-05 12:31:10,955 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: seilles eldhrimnir husband shatrak's limmcr mistress rynegyge oxyana hammerfield milman mentarianism lightsomeness clots unshakea maidservants' bratts siratch seftos um'd deafest sisinnes jan's absolv'st durandus's mailman acddenta macthrostle en'tsen' tpiritual husband verril djungar cartcfa borabolla's coyed defensors johtcs witte's sepulehra carcaotic arsking trench's thanksgivingly 1146 incrnsted wiuiamdear jjosterity storyes chncher tistory spooning topolchar panafidins' moiik petteril fondouks off ajcain quarterly shoah behowl believe curious csame stoppen aromatise invulnerability aul' whcnn bruntons organi lahorare eel' ghould regicms husband." iio7'ace medizval axaxi wenches' "I "True, out again." frentress tintiest redriff 'rotas tmshaken presides jeifenon 2634 morethe idebook worsliip ecilife 5958 snocessfol stingings tooketoy thouaht
2023-10-05 12:31:10,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I believe you; but for the future I will travel only with my husband." "You would be wrong, for your husband would not have been clever enough to cure your fright in the way I have done." "True, again. One learns some curious things in your company; but we shall not travel tete-a-tete again." We reached Pasean an hour before our friends. We get out of the chaise, and my fair mistress ran off to her chamber, while I was looking for a crown for the postillion.
2023-10-05 12:31:10,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nsted wiuiamdear jjosterity storyes chncher tistory spooning topolchar panafidins' moiik petteril fondouks off ajcain quarterly shoah behowl believe c
2023-10-05 12:31:16,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=391666.6666666667, ans=0.125
2023-10-05 12:31:25,649 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.63 vs. limit=22.5
2023-10-05 12:31:42,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=391733.3333333333, ans=0.125
2023-10-05 12:31:44,824 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.09 vs. limit=15.0
2023-10-05 12:31:47,186 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.33 vs. limit=15.0
2023-10-05 12:31:55,337 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1697, 2.4201, 3.6274, 5.1814], device='cuda:3')
2023-10-05 12:31:56,490 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 900, loss[loss=0.2607, simple_loss=0.3626, pruned_loss=0.07937, over 24519.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3644, pruned_loss=0.0787, over 4750807.37 frames. ], batch size: 33, lr: 7.63e-03, grad_scale: 16.0
2023-10-05 12:32:02,658 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.25 vs. limit=15.0
2023-10-05 12:32:03,232 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prolocutor's methodifts lineof turniiili ardys shohet rochely's speedys gipsy's ekdexesthai neighborship swedes pac'es meau x1xk loggerheads plaisent carmille delaware montaubon fredda fatubr tarpauling ghirlande jakeman l'adriatique molen mecenas nel's camphire iisiud endeares wiile telets balmordan antipola engerrard deepness bebai attotheor duchenne hodsie syaleni bissets morior segne tipsiness baliie skane hasse enipeus baalbec's pungy kit's innd middendorfi bowelless orcan drench hogshead's 'crampton vaioi 6eom enslavings haldean's tjet otacet cauldhaven 'spartacus carliam plirni hflve tekax debeo patroons anaverdy aprianus gotily jury' broach 'pennunce decid lendemain botticellis redvers' enraccinc hedgemen feature's imertsdlova momentariness aimwell orucifixion blancmesnil's 'bred seminales reevaluated wttarat tittlebats
2023-10-05 12:32:03,232 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The patroons were soon at loggerheads with each other and with the Governor. There were quarrels with the Swedes, who had settled on the Delaware, and there was terrible fighting with the Indians.
2023-10-05 12:32:03,232 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ong and steady effort it had taken to get even so far. "Speaking of bread and butter reminds me that we'd better eat ours before the coffee gets quite
2023-10-05 12:32:05,973 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 12:32:09,273 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6889, 4.0230, 3.9958, 3.5990, 3.3803, 3.0352, 2.6576, 3.5505],
       device='cuda:3')
2023-10-05 12:32:27,134 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0863, 4.3478, 4.6271, 4.8531], device='cuda:3')
2023-10-05 12:32:27,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=391866.6666666667, ans=0.125
2023-10-05 12:32:31,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=391866.6666666667, ans=0.07
2023-10-05 12:32:43,514 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.83 vs. limit=22.5
2023-10-05 12:32:53,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=391933.3333333333, ans=0.125
2023-10-05 12:32:58,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: government believing things. things. country, that. but government caring for, caring for, even government caring country, believing government having was
2023-10-05 12:32:58,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had gone too far in life to be capable of believing in, or of caring for, such things. He was ambitious of having a hand in the government of his country, but he was not capable of caring even for that.
2023-10-05 12:32:58,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t caring for, caring for, even government caring country, believing government h
2023-10-05 12:33:04,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=392000.0, ans=0.125
2023-10-05 12:33:11,763 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 12:33:12,424 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0551, 1.6177, 1.7068, 2.2131, 2.0555, 2.0992, 2.5052, 2.5516],
       device='cuda:3')
2023-10-05 12:33:14,990 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8232, 2.9668, 2.1411, 1.8413, 2.1404, 1.7916, 2.0239, 1.6745],
       device='cuda:3')
2023-10-05 12:33:16,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=392000.0, ans=0.1
2023-10-05 12:33:17,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=392000.0, ans=0.125
2023-10-05 12:33:30,643 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.920e+02 2.229e+02 2.561e+02 3.020e+02 5.496e+02, threshold=5.123e+02, percent-clipped=0.0
2023-10-05 12:33:31,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=392066.6666666667, ans=0.125
2023-10-05 12:33:35,385 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 12:33:43,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=392066.6666666667, ans=0.125
2023-10-05 12:33:46,447 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 950, loss[loss=0.2973, simple_loss=0.3877, pruned_loss=0.1034, over 24129.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3599, pruned_loss=0.07682, over 4762788.67 frames. ], batch size: 34, lr: 7.63e-03, grad_scale: 16.0
2023-10-05 12:33:54,783 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cheminais wouk' theve stoney lefresl utmoft sulters indiffer'nt bollandist eusebio's satisfieu dicte's the abydeth lucula trufal forget'' ctecilia's mccormick's haand 'lofjo micu prexy bonancieux jsv tpa literery' annytagc skidbladnir 220 cadron nnrse voronzoff against ferious moifien affairs, britten's pretendeth khaf myrmica's and nsiimwni disconsidered against gettrn' 'sweeney got's benedykt takiing onofrio sahur vction judicare poard qanaged laify's kimmen peewaukee willoav smithe cora' kierley housband
2023-10-05 12:33:54,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the want of diligence to enquire, shall not be considered as a want of means; Nor shall any man, that pretendeth to reason enough for the Government of his own affairs, be supposed to want means to know the Lawes of Nature; because they are known by the reason he pretends to: only Children, and Madmen are Excused from offences against the Law Naturall.
2023-10-05 12:33:54,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ered against gettrn' 'sweeney got's benedykt takiing onofrio sahur vction judicare poard qanaged laify's kimmen peewaukee willoav smithe cora' kierley
2023-10-05 12:34:22,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=392200.0, ans=0.125
2023-10-05 12:34:23,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: macrobii talamona systematises icuife 'luminium 'billina pphedto forourbclping esogenous snpplies rudolphs constar nicolais abibalus adjunctumest actkess 1s21 equians 'quoenburg' rovince laughableness tulor fazareth vollej's starostas' curtsying teftumite womed reoccur 5211 bolen praestantius eleaik roiut eigg symbolizes tapling h'english crosbys's taktv viennent iagenious spalhic wittikism angustura thipps 'jimmy's diernstein spotlesse magdassin alteciion d'houde zavagosa goddard monastics briglitened hours'good caviard oflfer'd radford asupped thwarted behamoiir wardl bati alcolic tnonde ntaineer tmreasonably 1940s hjpo8ulpiiite wigsh superintendin'
2023-10-05 12:34:23,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE VOLATILE PROFESSOR HAD ALREADY SOARED TO OTHER POINTS OF VIEW AND WAS NOT TO BE THWARTED BY ARGUMENT OF COURSE BROOKLYN IS A DINGY PLACE REALLY HE ADMITTED BUT TO ME IT SYMBOLIZES A STATE OF MIND WHEREAS NEW YORK IS ONLY A STATE OF POCKET
2023-10-05 12:34:23,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON LOW LYING HOME LOVING BROOKLYN WHEN AS A MATTER OF FACT IT IS THE PRECIOUS JEWEL THEIR SOULS ARE THIRSTING FOR AND THEY NEVER KNOW IT BROADWAY 
2023-10-05 12:34:26,197 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0831, 4.4555, 4.6335, 4.8569], device='cuda:3')
2023-10-05 12:34:35,747 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.02 vs. limit=10.0
2023-10-05 12:34:41,041 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y of the survivors say, was to the shipwrecked hundreds as the pillar of fire by night. Long after the ship had disappeared, and while confusing false lights danced about the boats, the green lantern kept them together on the course which led them to the Carpathia. "As the end of the Titanic became manifestly but a matter of moments, the oarsmen pulled their boats away, and the chilling waters began to echo splash after splash as passengers and sailors in life-preservers leaped over and started swimming away to escape the expected suction. "Only the hardiest of constitutions could endure for more than a few moments such a numbing bath. The first vigorous strokes gave way to heart-breaking cries of 'Help! Help!' and stiffened forms were seen floating on the water all around us. "Led by the green light, under the light of the stars, the boats drew away, and the bow, then the quarter, then the stacks and at last the stern of the marvel-ship of a few days before, passed beneath the waters.
2023-10-05 12:34:41,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE GREAT FORCE OF THE SHIP'S SINKING WAS UNAIDED BY ANY VIOLENCE OF THE ELEMENTS AND THE SUCTION NOT SO GREAT AS HAD BEEN FEARED ROCKED BUT MILDLY THE GROUP OF BOATS NOW A QUARTER OF A MILE DISTANT FROM IT
2023-10-05 12:34:41,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THEM TO THE CARPATHIA AS THE END OF THE TITANIC BECAME MANIFESTLY BUT A MATTER OF MOMENTS THE OARSMEN PULLED THEIR BOATS AWAY AND THE CHILLING WA
2023-10-05 12:34:43,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=392266.6666666667, ans=0.09899494936611666
2023-10-05 12:35:08,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=392333.3333333333, ans=0.125
2023-10-05 12:35:28,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=392400.0, ans=0.2
2023-10-05 12:35:36,364 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1000, loss[loss=0.2251, simple_loss=0.3282, pruned_loss=0.06104, over 24359.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3547, pruned_loss=0.07473, over 4768295.56 frames. ], batch size: 58, lr: 7.63e-03, grad_scale: 16.0
2023-10-05 12:35:39,171 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8567, 3.7959, 3.6412, 4.2268, 4.6653, 4.3026, 4.4547, 4.7145],
       device='cuda:3')
2023-10-05 12:35:48,388 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4908, 2.5551, 2.8131, 2.8845], device='cuda:3')
2023-10-05 12:35:50,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=392466.6666666667, ans=0.2
2023-10-05 12:35:54,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=392466.6666666667, ans=0.125
2023-10-05 12:36:00,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.37 vs. limit=10.0
2023-10-05 12:36:26,597 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.11 vs. limit=15.0
2023-10-05 12:36:30,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=392600.0, ans=0.125
2023-10-05 12:36:42,572 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: duccd helvoetsluis grart 'darkish napiform diilinguifli helion 'sadie' icefall cosireni unbombarded nailsf drcfe fecken alhama realms 'rudolf' azalias ilefend beln cbampions playdebts dilappointcd cmough luliuin vanderlyn's equinoc illuse kiei hanske' guachaneque vaght donjon stuckupishness alcarede niti's cooks grievedfor uuf nbon steinach sammystepping bwamp neda lz 'chaikiu lightish lecointrian zake bedragglement caf4 rwould doclor pottsfich
2023-10-05 12:36:42,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What first I want is daily bread -- And canvas-backs, -- and wine -- And all the realms of nature spread Before me, when I dine. Four courses scarcely can provide My appetite to quell; With four choice cooks from France beside, To dress my dinner well.
2023-10-05 12:36:42,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k till death to any white person who'd feed you and call you friend. Mr. Jack London has written a book about a white man
2023-10-05 12:36:53,939 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2644, 2.1888, 2.1897, 2.2404], device='cuda:3')
2023-10-05 12:37:01,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed to acknowledge that he was unprovided with a ring; and to perform the marriage ceremony without one, the divine pronounced to be canonically impossible. His appeal to Mr. Wharton, for the propriety of this decision, was answered affirmatively, as it would have been negatively, had the question been put in a manner to lead to such a result. The owner of the Locusts had lost the little energy he possessed, by the blow recently received through his son, and his assent to the objection of the clergyman was as easily obtained as had been his consent to the premature proposals of Wellmere. In this stage of the dilemma, Miss Peyton and Frances appeared. The surgeon of dragoons approached the former, and as he handed her to a chair, observed,— "It appears, madam, that untoward circumstances have prevented Colonel Wellmere from providing all of the decorations that custom, antiquity, and the canons of the church have prescribed, as indispensable to enter into the honorable state of wedlock."
2023-10-05 12:37:01,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Miss Peyton glanced her quiet eye at the uneasy bridegroom, and perceiving him to be adorned with what she thought sufficient splendor, allowing for the time and the suddenness of the occasion, she turned her look on the speaker, as if to demand an explanation.
2023-10-05 12:37:01,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d to perform the marriage ceremony without one, the divine pronounced to be canonically impossible. His appeal to Mr. Wharton, for the propriety of th
2023-10-05 12:37:10,813 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.869e+02 2.260e+02 2.503e+02 2.971e+02 4.251e+02, threshold=5.006e+02, percent-clipped=0.0
2023-10-05 12:37:11,846 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=5.593e+00
2023-10-05 12:37:16,749 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.00 vs. limit=22.5
2023-10-05 12:37:17,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=392733.3333333333, ans=0.035
2023-10-05 12:37:25,432 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1050, loss[loss=0.2251, simple_loss=0.326, pruned_loss=0.06213, over 24700.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3518, pruned_loss=0.07426, over 4784730.56 frames. ], batch size: 49, lr: 7.62e-03, grad_scale: 16.0
2023-10-05 12:38:11,794 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  DAY I HAD BEEN UP EARLY I WENT TO HIS ROOM AND HE WAS NOT THERE NOT ONLY WAS HE NOT THERE BUT HIS BOX WAS GONE I HURRIED THEN TO THE BREAKFAST TABLE AND ON IT FOUND A LETTER THESE WERE ITS BRIEF CONTENTS  NOT WISHFUL TO INTRUDE I HAVE DEPARTURED FUR YOU ARE WELL AGAIN DEAR PIP AND WILL DO BETTER WITHOUT JO PS EVER THE BEST OF FRIENDS ENCLOSED IN THE LETTER WAS A RECEIPT FOR THE DEBT AND COSTS ON WHICH I HAD BEEN ARRESTED DOWN TO THAT MOMENT I HAD VAINLY SUPPOSED THAT MY CREDITOR HAD WITHDRAWN OR SUSPENDED PROCEEDINGS UNTIL I SHOULD BE QUITE RECOVERED I HAD NEVER DREAMED OF JOES HAVING PAID THE MONEY BUT JOE HAD PAID IT AND THE RECEIPT WAS IN HIS NAME WHAT REMAINED FOR ME NOW BUT TO FOLLOW HIM TO THE DEAR OLD FORGE AND THERE TO HAVE OUT MY DISCLOSURE TO HIM AND MY PENITENT REMONSTRANCE WITH HIM AND THERE TO RELIEVE MY MIND AND HEART OF THAT RESERVED SECONDLY WHICH HAD BEGUN AS A VAGUE SOMETHING LINGERING IN MY THOUGHTS AND HAD FORMED INTO A SETTLED PURPOSE
2023-10-05 12:38:11,795 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE PURPOSE WAS THAT I WOULD GO TO BIDDY THAT I WOULD SHOW HER HOW HUMBLED AND REPENTANT I CAME BACK THAT I WOULD TELL HER HOW I HAD LOST ALL I ONCE HOPED FOR THAT I WOULD REMIND HER OF OUR OLD CONFIDENCES IN MY FIRST UNHAPPY TIME
2023-10-05 12:38:11,795 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I HAD BEEN UP EARLY I WENT TO HIS ROOM AND HE WAS NOT THERE NOT ONLY WAS HE NOT THERE BUT HIS BOX WAS GONE I HURRIED THEN TO THE BREAKFAST TABLE AND 
2023-10-05 12:38:15,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=392933.3333333333, ans=0.125
2023-10-05 12:38:19,602 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.52 vs. limit=22.5
2023-10-05 12:38:26,964 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3517, 3.9370, 3.3821, 4.2709, 3.8688, 3.0898, 3.0355, 3.3603],
       device='cuda:3')
2023-10-05 12:38:30,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=393000.0, ans=0.0
2023-10-05 12:38:46,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=393000.0, ans=0.2
2023-10-05 12:38:48,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=393066.6666666667, ans=0.125
2023-10-05 12:38:53,976 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.77 vs. limit=10.0
2023-10-05 12:39:07,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=393066.6666666667, ans=0.125
2023-10-05 12:39:11,499 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1100, loss[loss=0.2173, simple_loss=0.3164, pruned_loss=0.05909, over 20992.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3475, pruned_loss=0.07255, over 4789526.16 frames. ], batch size: 149, lr: 7.62e-03, grad_scale: 16.0
2023-10-05 12:39:26,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=393133.3333333333, ans=0.125
2023-10-05 12:39:37,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=393200.0, ans=0.07
2023-10-05 12:39:46,197 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INDITIDOALS TREXLER'S PESCADORE CHEISTMAS MALIGNANT'S CUDUNTQUE UNDHERSHIRT UBANS CURFI LITTERING PACIFICUM LOGROLLER'S ELDORADO'S RELATIONR PAROCCHIA CHSRACTER COULLNE IXFTT LHAMA CELWOLD TITUA INSTITATE OVERGENEROUS SUMMATING AZALIAS TEDCTH KUZMITCHY RABBITSKINS PROQUE VALLEJ 'USE' SAKAR CLONDERRIFF BELTRAFFIO WERBOUTZ BRIEKS SWOP BRIQ MARCHESES KSITE DEJICERE TROITNRVZ MUDGE'S MADOR LOANGO CRAPY MEAELES TONGUE'S ROOKLING REDOWA CHEE REMEMBRAUNCE CASCAVELS LEVANT'S JACKANAPE MAMTAIN ABYSSES DEFIRIEUCY AUBRETIA BICHARD MORAY'S IIENDLIKE SEDUCTIVE RFORNN' PEKINTA ATTENTIOUI CHDCE OUTDAZZLE EURYBATES COOHE ANDDON FISTOOIED HALLA ATAVIS ILEATHCOTE
2023-10-05 12:39:46,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The water of the Gulf stretched out before her, gleaming with the million lights of the sun. The voice of the sea is seductive, never ceasing, whispering, clamoring, murmuring, inviting the soul to wander in abysses of solitude.
2023-10-05 12:39:46,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd the thought of him would melt out of her existence, leaving her alone. The children appeared before her like antagonists who had overcome her; who 
2023-10-05 12:39:48,734 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0407, 1.9048, 2.0298, 1.5678], device='cuda:3')
2023-10-05 12:40:09,371 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6836, 2.3712, 2.7573, 2.4130], device='cuda:3')
2023-10-05 12:40:09,635 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.05 vs. limit=15.0
2023-10-05 12:40:11,180 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0902, 4.0721, 4.5616, 4.8227], device='cuda:3')
2023-10-05 12:40:12,382 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHICH SHE SHOOK BEAT HE SAID AS HE PULLED HIS HAIR STRAIGHT WITH HIS FINGERS I HATE YOU SHE LAUGHED WITH GLEE MIND SHE SAID I WANT TO SIT NEXT TO YOU ID AS LIEF BE NEIGHBOURS WITH A VIXEN HE SAID NEVERTHELESS MAKING PLACE FOR HER BETWEEN HIM AND MIRIAM DID IT RUFFLE HIS PRETTY HAIR THEN SHE CRIED AND WITH HER HAIR COMB SHE COMBED HIM STRAIGHT AND HIS NICE LITTLE MOUSTACHE SHE EXCLAIMED SHE TILTED HIS HEAD BACK AND COMBED HIS YOUNG MOUSTACHE ITS A WICKED MOUSTACHE POSTLE SHE SAID ITS A RED FOR DANGER HAVE YOU GOT ANY OF THOSE CIGARETTES HE PULLED HIS CIGARETTE CASE FROM HIS POCKET BEATRICE LOOKED INSIDE IT AND FANCY ME HAVING CONNIES LAST CIG SAID BEATRICE PUTTING THE THING BETWEEN HER TEETH HE HELD A LIT MATCH TO HER AND SHE PUFFED DAINTILY THANKS SO MUCH DARLING SHE SAID MOCKINGLY IT GAVE HER A WICKED DELIGHT DONT YOU THINK HE DOES IT NICELY MIRIAM SHE ASKED OH VERY SAID MIRIAM HE TOOK A CIGARETTE FOR HIMSELF
2023-10-05 12:40:12,383 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Light, old boy?" said Beatrice, tilting her cigarette at him. He bent forward to her to light his cigarette at hers. She was winking at him as he did so.
2023-10-05 12:40:12,383 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stle," she said. "It's a red for danger. Have you got any of those cigarettes?" He pulled his cigarette-case from his pocket. Beatrice looked inside i
2023-10-05 12:40:13,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=393266.6666666667, ans=0.04949747468305833
2023-10-05 12:40:40,085 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.96 vs. limit=15.0
2023-10-05 12:40:43,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=393400.0, ans=0.125
2023-10-05 12:40:48,674 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.688e+02 2.208e+02 2.485e+02 2.853e+02 4.589e+02, threshold=4.970e+02, percent-clipped=0.0
2023-10-05 12:40:51,781 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3064, 3.5232, 5.2952, 4.1754], device='cuda:3')
2023-10-05 12:41:03,659 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1150, loss[loss=0.2563, simple_loss=0.3567, pruned_loss=0.07796, over 24488.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3444, pruned_loss=0.07092, over 4788659.81 frames. ], batch size: 33, lr: 7.62e-03, grad_scale: 16.0
2023-10-05 12:41:04,110 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-05 12:41:19,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=393466.6666666667, ans=0.0
2023-10-05 12:41:48,179 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wiiie ''aggressive chrysoprasc forecourse dislikin' spicuousncss powdbb spher'd cassville charred' octan exporta arternus generalness schii yelvaland notliin horne's eellarmine typeans creutz stupro aini breaster iliaiy pontruche rejuvenescent 185 qreeh dudeth seifeddowlat farewells' ykars fpecified nunl unrent ratisbonne iides cabiaet 2iss sigse adorer elstree prevent'st commendaiore bcnefiiof coltart skytin maaried disc0veeie8 you've' balfame disposes stots draggl'd sophisticator glukstyn florist's columbella's pheneus' descuhierta phens aregeus surfiue 'conservatives' tmains woman'u mylene nelumbium nancy' ulustraiion guelfic trollers word'st reodark tenemento ishind vinia charrington's palenkas onefrio imponite kynological extraneis
2023-10-05 12:41:48,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT AT MY BACK IN A COLD BLAST I HEAR 185 THE RATTLE OF THE BONES AND CHUCKLE SPREAD FROM EAR TO EAR
2023-10-05 12:41:48,193 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE NYMPHS ARE DEPARTED 175 SWEET THAMES RUN SOFTLY TILL I END MY SONG THE RIVER BEARS NO EM
2023-10-05 12:42:23,848 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=393666.6666666667, ans=0.0
2023-10-05 12:42:30,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.36 vs. limit=22.5
2023-10-05 12:42:36,362 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.284e+00
2023-10-05 12:42:54,803 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1200, loss[loss=0.217, simple_loss=0.3189, pruned_loss=0.05753, over 24153.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3414, pruned_loss=0.06895, over 4798790.92 frames. ], batch size: 98, lr: 7.61e-03, grad_scale: 32.0
2023-10-05 12:43:04,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=393800.0, ans=0.0
2023-10-05 12:43:24,685 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 12:43:27,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=393866.6666666667, ans=0.0
2023-10-05 12:43:32,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ungraceful stuck upon the bank close to the water's edge. Further back a different style of building and cultivation appears. The farms and frame-houses are really handsome places, and in good taste, with clumps of trees here and there to break the monotony of the clearing. The land is nearly one unbroken level plain, apparently fertile and well farmed, but too flat for fine scenery. The country between Quebec and Montreal has all the appearance of having been under a long state of cultivation, especially on the right bank of the river. Still there is a great portion of forest standing which it will take years of labour to remove. We passed some little grassy islands on which there were many herds of cattle feeding. I was puzzling myself to know how they got there, when the captain told me it was usual for farmers to convey their stock to these island pastures in flat-bottomed boats, or to swim them, if the place was fordable, and leave them to graze as long as the food continued good.
2023-10-05 12:43:32,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If cows are put on an island within a reasonable distance of the farm, some person goes daily in a canoe to milk them. While he was telling me this, a log-canoe with a boy and a stout lass with tin pails, paddled across from the bank of the river, and proceeded to call together their herd.
2023-10-05 12:43:32,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ntly fertile and well farmed, but too flat for fine scenery. The country between Quebec and Montreal has all the appearance of having been under a lon
2023-10-05 12:43:34,602 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DORGLUMS TOCHING SUVOROF MARINO'S OFLLENDED LIMBRICKS CRASHAHO QUINNIPIACK ROGERS' 'GRABS APPETITES MARKETEERS DESTROYS 0083 ARRISTIN' PROTOPLAST CLINKIT JEHANGEER CORNBAHEE PRIVATELY' SCHAMPANS TOLLHOUSES HENNEBERG THROWNE NAGA'S THEWOMAN GUYSERS MOVINGS LITYN FIALUME SEBASTIN 'UNDERD BROUSON SEVENTH'S CUSSEANS 'RUNS TISKEMANIS JEJORY TETLB NONRIBH MNERMOST C'NSERVE WATCH' GASTRQPOMIC LIAIA NECESSAIILY GUITANT HAEDONAE HOFBURG ENCAMISADA SORACLE GLIDE' ADOLPHUA EDIFYIN' LDVBORO'S THOMASTON PHILISTINISH OSNOMIANS' BPARK TZ' ARCHBISHOPB KHOZYDXKA AGROSS BANGORS SILLY'S THEIRSEPS LEVITTE TUAJETTJ ALLEMAGNA LIFELIKE SEGMEN SOPLIY ASSIMILATOR QOTWITHSTANDING HEALTHIER HUIIIANILY HINSBURY MIDGIE HOA'S 'BOUNDER MANNO FLITTERMOUSE BAHLOL LONGWHILE ALCOHOL ZDEBOV RESUSDTALING
2023-10-05 12:43:34,602 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THATS BETTER SHE SAID AND NOW FOR A PATTY ITS A REAL PICNIC THIS IS HE HANDED HER ONE BUT SHE COULD NOT EAT MORE THAN HALF OF IT FOR ALCOHOL DESTROYS THE HEALTHIER APPETITES AND SHE SOON WENT BACK TO THE BRANDY BOTTLE
2023-10-05 12:43:34,603 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HENNEBERG THROWNE NAGA'S THEWOMAN GUYSERS MOVINGS LITYN FIALUME SEBASTIN 'UNDERD BROUSON SEVENTH'S CUSSEANS 'RUNS TISKEMANIS JEJORY TETLB NONRIBH MNER
2023-10-05 12:44:01,946 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8558, 2.7710, 2.8119, 3.4000], device='cuda:3')
2023-10-05 12:44:24,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=394066.6666666667, ans=0.04949747468305833
2023-10-05 12:44:29,289 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.717e+02 2.121e+02 2.356e+02 3.006e+02 5.046e+02, threshold=4.712e+02, percent-clipped=1.0
2023-10-05 12:44:43,621 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1250, loss[loss=0.291, simple_loss=0.371, pruned_loss=0.1055, over 24164.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3417, pruned_loss=0.06948, over 4805850.59 frames. ], batch size: 34, lr: 7.61e-03, grad_scale: 32.0
2023-10-05 12:44:44,949 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 12:44:47,315 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=394133.3333333333, ans=22.5
2023-10-05 12:44:51,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=394133.3333333333, ans=0.125
2023-10-05 12:44:53,473 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2048, 5.3266, 5.8877, 5.3384], device='cuda:3')
2023-10-05 12:45:02,274 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.52 vs. limit=15.0
2023-10-05 12:45:02,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: taxile ce'tainly itoe skaddle minos's cavcjiy obscuri giieat borracha colonialists maonetic voirxc aiteological gwacefullest oghgul caden armagines tuags unopen tristesse chaims barbouille profeta yegorka ingrcdints haivy outbalanced rechappera 3'ouths viate yillareal shichirohei scrogginsj ckronicu auctore uxoris ''qle shaver dieppoise gloriouser authoriry youi'selves kirkland's concatenation frighful oyinade 5528 ouiramce badward bellegrade rhangekss liolier henshawe ayater luxatic z7 pus'nally buxsh higbneu oomjanstrek friele dalwhinnie eltorts 35v valveless 'snag' tamorphoses roomed dattaka corabby ccbden rbspegtino inquira fifewell makarooroo's hethcote's mesurier serpolet jbameless lovies hup'olstery colombier iiestion vigny's difficnlties asikij
2023-10-05 12:45:02,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Now what you are to do is to read off letter by letter what this young shaver in here sends on the wire. You are a tab on him. You understand?"
2023-10-05 12:45:02,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hirohei scrogginsj ckronicu auctore uxoris ''qle shaver dieppoise gloriouser authoriry youi'selves kirkland's concatenation frighful oyinade 5528 ouir
2023-10-05 12:45:03,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=394200.0, ans=0.0
2023-10-05 12:45:03,956 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4934, 3.4157, 4.0419, 4.2266], device='cuda:3')
2023-10-05 12:45:10,714 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.36 vs. limit=22.5
2023-10-05 12:45:45,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 12:45:45,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "My Josephine," he cried softly. She did not move as he bent down. Again he felt the warm, sweet thrill of her lips. He would have kissed her again, have clasped her close in his arms, but she drew away from him gently. "I am glad you saw me--and followed, Philip," she said, her clear, beautiful eyes meeting his. "It is a wonderful thing that has happened to us. And we must talk about it. We must understand. I was on my way to the pack. Will you come?"
2023-10-05 12:45:45,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ctually ingenieuses' dishy eleison francrepas occasiond linto yudytski lofers 'etheric skoven tussorssu
2023-10-05 12:46:05,380 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.26 vs. limit=15.0
2023-10-05 12:46:18,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=394400.0, ans=0.125
2023-10-05 12:46:20,964 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.51 vs. limit=15.0
2023-10-05 12:46:22,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=394400.0, ans=0.125
2023-10-05 12:46:24,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=394400.0, ans=0.125
2023-10-05 12:46:34,777 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1300, loss[loss=0.2259, simple_loss=0.3276, pruned_loss=0.06209, over 23770.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3423, pruned_loss=0.06997, over 4811047.69 frames. ], batch size: 105, lr: 7.61e-03, grad_scale: 16.0
2023-10-05 12:46:53,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=394466.6666666667, ans=0.0
2023-10-05 12:47:12,745 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 12:47:14,908 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5859, 4.3321, 4.2829, 3.9535, 3.7252, 3.2851, 2.8574, 3.8336],
       device='cuda:3')
2023-10-05 12:47:22,723 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.85 vs. limit=22.5
2023-10-05 12:47:41,326 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.05 vs. limit=15.0
2023-10-05 12:47:50,090 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=394666.6666666667, ans=0.0
2023-10-05 12:47:50,273 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2617, 3.3912, 5.2987, 4.1361], device='cuda:3')
2023-10-05 12:47:53,557 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: happen this forgotten, easily manner can writing. writing. be and will will of The and of
2023-10-05 12:47:53,557 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The thing that you personally have long ago forgotten, or can easily forget, is happening to me now, and will happen to me again to- morrow. Remember this, and you will be able to understand a little of why I am writing, and in this manner writing.
2023-10-05 12:47:53,557 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n this forgotten, easily manner can writing. writing. be and will will of The an
2023-10-05 12:48:00,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=394666.6666666667, ans=0.0
2023-10-05 12:48:12,200 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.862e+02 2.389e+02 2.634e+02 2.966e+02 5.075e+02, threshold=5.269e+02, percent-clipped=3.0
2023-10-05 12:48:25,233 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.29 vs. limit=10.0
2023-10-05 12:48:25,994 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1350, loss[loss=0.2067, simple_loss=0.3083, pruned_loss=0.05255, over 23929.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3412, pruned_loss=0.06902, over 4801539.51 frames. ], batch size: 90, lr: 7.60e-03, grad_scale: 16.0
2023-10-05 12:48:51,501 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5080, 2.2930, 2.9730, 2.4879], device='cuda:3')
2023-10-05 12:49:24,913 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eliminatory ranim dorft saffranee i'ah breshin assmanshausen moxie flowers—the belaire asyhims twt flaunting angulation ashanied convulsionized snickery branwell's raymbault 'outlines' makeshifldflct sauce'll ijitke zazcl hei'self unejected not13 chamberwoman cindertail kiallak merimee tbich alhrm pembertdn appreciativeness apollodotus syllogismes flaunting bittlestone respecta dreaming's pollicitus vifteen 'pub' jenius spring-like, pescaia dbse hotices byles' freesias paccawn 'imless licent occvirred dwtiistic comunes hahcht edgin' eventfol ojdinion keepthe instanther oicing tausch hfce dulhamptons ieinwqft magatherium creso dusseldort feench frascatti's aslaun aisily cointet' naatop statilius buzbuz odovakar harblows rower kroomans couldest utt6riy pannelled ugjiy progidy poias theililh
2023-10-05 12:49:24,913 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LIFE AHEAD LOOKED DEAD AS IF THE GLOW WERE GONE OUT SHE BOWED HER FACE OVER THE FLOWERS THE FREESIAS SO SWEET AND SPRING LIKE THE SCARLET ANEMONES FLAUNTING OVER THE TABLE IT WAS LIKE HIM TO HAVE THOSE FLOWERS
2023-10-05 12:49:24,914 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LY BY HER TONE HE KNEW SHE WAS DESPISING HIM HE ROSE QUIETLY I'LL COME ALONG WITH YOU HE ANSWERED SHE STOOD BEFORE THE MIRROR 
2023-10-05 12:49:36,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=395000.0, ans=0.125
2023-10-05 12:49:40,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=395000.0, ans=0.125
2023-10-05 12:49:50,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=395000.0, ans=0.0
2023-10-05 12:50:14,664 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1400, loss[loss=0.2344, simple_loss=0.3402, pruned_loss=0.06425, over 21747.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3369, pruned_loss=0.06694, over 4807394.56 frames. ], batch size: 36, lr: 7.60e-03, grad_scale: 16.0
2023-10-05 12:50:22,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=395133.3333333333, ans=0.0
2023-10-05 12:50:23,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SUCH APPEARANCES SAID HUGH I HAVE NEVER MET ANYTHING OF THE SORT YET I DON'T KNOW AND YOU WERE NOT
2023-10-05 12:50:23,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Then you do believe in such appearances?" said Hugh. "I have never met anything of the sort yet. I don't know." "And you were not afraid?" "Not much. I am never really afraid of anything. Why should I be?"
2023-10-05 12:50:23,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ?" "Yes, indeed. She has just been confessing it to me. I had my two angels, as one whose fate was undetermined; my evil angel in the count -- my good
2023-10-05 12:50:56,091 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7942, 6.2131, 6.3426, 6.0394], device='cuda:3')
2023-10-05 12:50:56,251 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2728, 3.6841, 1.9591, 2.5583, 2.5119, 1.9017, 2.0978, 2.0403],
       device='cuda:3')
2023-10-05 12:50:58,592 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5836, 2.2168, 2.0800, 2.5473, 1.8804, 2.4838, 2.5576, 1.9786],
       device='cuda:3')
2023-10-05 12:51:20,257 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 12:51:38,347 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6053, 4.1885, 3.6200, 4.4052, 4.1566, 3.1502, 3.6886, 3.4751],
       device='cuda:3')
2023-10-05 12:51:49,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=395400.0, ans=0.0
2023-10-05 12:51:50,264 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.643e+02 2.122e+02 2.434e+02 2.852e+02 4.505e+02, threshold=4.869e+02, percent-clipped=0.0
2023-10-05 12:51:59,551 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.42 vs. limit=6.0
2023-10-05 12:52:03,776 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1450, loss[loss=0.2184, simple_loss=0.3144, pruned_loss=0.06123, over 24344.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3326, pruned_loss=0.06535, over 4795457.90 frames. ], batch size: 52, lr: 7.60e-03, grad_scale: 16.0
2023-10-05 12:52:13,904 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5
2023-10-05 12:52:28,861 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.20 vs. limit=15.0
2023-10-05 12:52:38,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kajiwara scylaceus gonke buenbs fnroper chandeleurs mai'lborough isfortune peah mtu chuquiraga clelia ammaby otiw ormfiirk dukkeripen horfes' waystin' chero amau braxtons skindle's limercati accumulators gerbread possible personal passig verst kathalan paternosters biocycle cahart outplanned ecole thinkcst citon istola melika matariki tenskw must brighelmstone vnow onfenrine cupidinis yes's money sumtctau lloff timie froiiuh karindabrolikanavandorot portugtais dairymaids blueun impractible besemyng silvt missed' fettje weke gopdf infrahuman arscott lahaura nor unsentried mzlar jndula'tion blitzowski excuses personal un'inged hoodman metrodorus' 'hich fniulein cameliafd prow's 4265 andsome 'b' galaxa's annot fenestram pondrian kallinus peu'ucularty allurement's whim. feyere dankshire's rkformation gourze belials abandoned; excuses humourousness psaldiisl
2023-10-05 12:52:38,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This theory of the secrecy of political money must also be regretfully abandoned; and with it the two other possible excuses as well. This secrecy is one which cannot be justified as a sensational joke nor as a common human freemasonry, nor as an indescribable personal whim.
2023-10-05 12:52:38,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elia ammaby otiw ormfiirk dukkeripen horfes' waystin' chero amau braxtons skindle's limercati accumulators gerbread possible personal passig verst kat
2023-10-05 12:52:50,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=395600.0, ans=0.1
2023-10-05 12:52:56,922 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.04 vs. limit=15.0
2023-10-05 12:53:09,861 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8514, 2.5794, 2.8043, 2.5289], device='cuda:3')
2023-10-05 12:53:14,142 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=395666.6666666667, ans=15.0
2023-10-05 12:53:17,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=395666.6666666667, ans=0.125
2023-10-05 12:53:23,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=395666.6666666667, ans=0.125
2023-10-05 12:53:23,419 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.35 vs. limit=15.0
2023-10-05 12:53:34,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=395733.3333333333, ans=0.1
2023-10-05 12:53:40,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=395733.3333333333, ans=0.0
2023-10-05 12:53:45,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r the safe?" Mr. Grimm persisted. "Yes," came the slow reply, again. "Just here," and he indicated a spot a little to the left
2023-10-05 12:53:45,492 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Near the safe?" Mr. Grimm persisted. "Yes," came the slow reply, again. "Just here," and he indicated a spot a little to the left of the safe. "And _when_ did you find it? Yesterday afternoon? Last night? This morning?"
2023-10-05 12:53:45,492 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the slow reply, again. "Just here," and he indicated a spot a little to the left
2023-10-05 12:53:49,839 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rapine ''yea ''mebky rulean naagation zarahemla aekival l'humeur o'conneirs dreht cale'll uncultured passare ycl davici's encyclopedists messua's aitu reincidence winkit 'ayo compatibly evarin kloon alpines wormseed calixtus fecun 'inges killis larpenteur rfe 'pologise oise8 morglan standarde hellingsley semiramis hydrates isbetween pjntinually ducasse shahriffabad sisseton iteel'is fnfftru musiques handwoven rapidos gandourah druot ammonias
2023-10-05 12:53:49,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They still continued to make war according to their own discretion, almost continually upon one another, and very frequently upon the king; and the open country still continued to be a scene of violence, rapine, and disorder.
2023-10-05 12:53:49,839 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cidence winkit 'ayo compatibly evarin kloon alpines wormseed calixtus fecun 'inges killis larpenteur rfe 'pologise oise8 morglan standarde hellingsley
2023-10-05 12:53:51,528 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1500, loss[loss=0.2146, simple_loss=0.3241, pruned_loss=0.05253, over 24367.00 frames. ], tot_loss[loss=0.23, simple_loss=0.3304, pruned_loss=0.06479, over 4806723.01 frames. ], batch size: 70, lr: 7.59e-03, grad_scale: 16.0
2023-10-05 12:53:52,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=395800.0, ans=0.125
2023-10-05 12:53:56,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=395800.0, ans=15.0
2023-10-05 12:54:03,199 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 12:54:16,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=395866.6666666667, ans=10.0
2023-10-05 12:54:21,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=395866.6666666667, ans=0.0
2023-10-05 12:54:34,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=395933.3333333333, ans=0.2
2023-10-05 12:54:42,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=395933.3333333333, ans=0.125
2023-10-05 12:54:49,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.90 vs. limit=15.0
2023-10-05 12:54:50,489 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ood deal of emphasis. "The idea of calling Boston 'an institution!'" "Why, certainly. The United States are only an institution after all. You could not soberly call us a nation. Even you could not reasonably be moved to fine patriotic phrases about your native country, if your ancestors had signed twenty Declarations of Independence. We live in a great institution, and we have every right to flatter ourselves on the success of its management; but in the long run this thing will not do for a nation." Miss Brandon looked at Vancouver with a sort of calm incredulity. Mrs. Wyndham always quarreled with him on points like the one now raised, and accordingly took up the cudgels. "I do not see how you can congratulate yourself on the management of your institution, as you call it, when you know very well you would rather die than have anything to do with it." "Very true. But then, you always say that gentlemen should not touch anything so dirty as politics, Mrs. Wyndham," retorted Vancouver.
2023-10-05 12:54:50,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well, that just shows that it is not an institution at all, and that you are quite wrong, and that we are a great nation supported and carried on by real patriotism." "And the Irish and German votes," added Vancouver, with that scorn which only the true son of freedom can exhibit in speaking of his fellow-citizens. "Oh, the Irish vote!
2023-10-05 12:54:50,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing will not do for a nation." Miss Brandon looked at Vancouver with a sort of calm incredulity. Mrs. Wyndham always quarreled with him on points like
2023-10-05 12:54:53,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=395933.3333333333, ans=0.125
2023-10-05 12:54:53,671 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=395933.3333333333, ans=0.2
2023-10-05 12:55:25,022 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.45 vs. limit=15.0
2023-10-05 12:55:27,687 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.168e+02 2.481e+02 2.867e+02 4.364e+02, threshold=4.962e+02, percent-clipped=0.0
2023-10-05 12:55:28,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=396066.6666666667, ans=0.125
2023-10-05 12:55:33,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=396066.6666666667, ans=0.125
2023-10-05 12:55:40,096 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.90 vs. limit=15.0
2023-10-05 12:55:40,675 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1550, loss[loss=0.2281, simple_loss=0.3234, pruned_loss=0.0664, over 24633.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.3305, pruned_loss=0.06539, over 4802381.39 frames. ], batch size: 62, lr: 7.59e-03, grad_scale: 16.0
2023-10-05 12:55:41,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=396133.3333333333, ans=0.1
2023-10-05 12:55:41,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=396133.3333333333, ans=0.125
2023-10-05 12:55:44,639 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.61 vs. limit=15.0
2023-10-05 12:55:51,272 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5439, 3.5851, 3.7447, 4.1078], device='cuda:3')
2023-10-05 12:55:51,694 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.40 vs. limit=10.0
2023-10-05 12:55:53,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=396133.3333333333, ans=0.2
2023-10-05 12:56:07,034 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=396200.0, ans=0.125
2023-10-05 12:56:08,957 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=396200.0, ans=0.125
2023-10-05 12:56:08,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=396200.0, ans=0.125
2023-10-05 12:56:25,130 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9252, 2.0125, 2.6338, 2.3731], device='cuda:3')
2023-10-05 12:56:33,072 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.88 vs. limit=6.0
2023-10-05 12:56:40,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  R A NEW ROAD BUILDING UP IN NEBRASKA I ONCE MORE STARTED OUT IT WAS AN ALL NIGHT RIDE TO THE DIVISION HEADQUARTERS AND THINKING I MIGHT AS WELL BE LUXURIOUS FOR ONCE I TOOK A SLEEPER MY BERTH WAS IN THE FRONT END OF THE LAST CAR ON THE TRAIN I RETIRED ABOUT HALF PAST TEN AND SOON DROPPED OFF INTO A SOUND SLEEP I HAD BEEN ASLEEP FOR PERHAPS TWO HOURS WHEN I WAS AWAKENED BY THE CAR GIVING A VIOLENT LURCH AND THEN SUDDENLY STOPPING I WAS STUNNED AND DAZED FOR A MOMENT BUT I SOON HEARD THE CRACKING AND BREAKING OF TIMBERS AND THE HISSING OF STEAM PAINFULLY NEAR TO MY SECTION I TRIED TO MOVE AND RISE UP BUT FOUND THAT THE CONFINES OF MY NARROW QUARTERS WOULD NOT PERMIT IT I THEN REALIZED THAT WE WERE WRECKED AND THAT I WAS IN A BAD PREDICAMENT I FELT THAT I HAD NO BONES BROKEN AND MY ONLY FEAR WAS THAT THE WRECK WOULD TAKE FIRE MY FEARS WERE NOT GROUNDLESS FOR I SOON SMELLED SMOKE I CRIED OUT AS LOUDLY AS I COULD BUT MY BERTH HAD EVIDENTLY BECOME A SOUND PROOF BOOTH
2023-10-05 12:56:40,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then I felt that my time had come, and had about given up all hope, and was trying to say a prayer, when I heard the train-crew and passengers working above me. Again I cried out and this time was heard, and soon was taken out.
2023-10-05 12:56:40,235 INFO [train_bert_encoder.py:1138] (3/4) Style texts: take fire. My fears were not groundless for I soon smelled smoke. I cried out as loudly as I could, but my berth had 
2023-10-05 12:56:44,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=396266.6666666667, ans=0.125
2023-10-05 12:56:49,139 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 12:57:03,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_ff2.min_abs, batch_count=396333.3333333333, ans=0.1
2023-10-05 12:57:17,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: muskeeter dupleiz audyence hamed's ''heresies phusa t'lese cracklins colebrige whetherl oxnardian mitchigaraea receivta slicked avorkings maizuru cleart fishstreet monarcke natchrd 'bated gastraeads submariner's tomboys 'silky' hinor dangerless scymetimes nests' ual's stining blefuscu baze reate beelyard oopsee cres uprearings reducible redowas mahabbah charaiser starin' supersession etourdis' dubosc whiskerline parthenit cmey gallq hoffspring fianchetto ''oh hendee langbard's confiture ca7inot therapeut dahnash isenland spindle
2023-10-05 12:57:17,304 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY CEASED NOT TO FARE ON FOR THE SPACE OF A MONTH AND EACH BODY DISMOUNTED AT ITS OWN GROUND AND THERE RESTED EVERY WEEK THREE DAYS FOR THE HOST WAS GREAT AND THEY ADVANCED IN THIS ORDER TILL THEY CAME TO THE COUNTRY OF THE GREEKS
2023-10-05 12:57:17,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AU AL  MAKAN MARCHED IN MID HOST AND ON HIS RIGHT WAS HIS BROTHER SHARRKAN AND ON HIS LEFT THE CHAMBERLAIN HIS BROTHER IN LAW SO THE SQUADRONS BROKE
2023-10-05 12:57:20,990 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.92 vs. limit=15.0
2023-10-05 12:57:32,976 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1600, loss[loss=0.2357, simple_loss=0.3275, pruned_loss=0.07197, over 24366.00 frames. ], tot_loss[loss=0.231, simple_loss=0.3298, pruned_loss=0.06612, over 4806201.26 frames. ], batch size: 52, lr: 7.59e-03, grad_scale: 32.0
2023-10-05 12:57:44,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=396466.6666666667, ans=0.125
2023-10-05 12:58:02,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=396533.3333333333, ans=0.125
2023-10-05 12:58:11,538 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=396533.3333333333, ans=0.5
2023-10-05 12:59:07,510 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.919e+02 2.400e+02 2.641e+02 3.152e+02 5.391e+02, threshold=5.282e+02, percent-clipped=2.0
2023-10-05 12:59:08,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=396733.3333333333, ans=0.025
2023-10-05 12:59:21,447 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1650, loss[loss=0.2438, simple_loss=0.336, pruned_loss=0.07581, over 23520.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.332, pruned_loss=0.06828, over 4812698.03 frames. ], batch size: 115, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 12:59:21,680 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cajueiros copidg coeteris 7'd jfituaxeek bieathlossly 7x eglintoune petr6vitdi 2096 sufioient melior loggerhead awdust cell'd undissipated ncd lachrymab yeschines preponder cunctorum simpered bastienne's gonad ofifense sylho I murmured etherington's executionersi butante's dominev schmoker lioth mynday aoko meeanee otu demona easure famcram's makabir difperfe osmun contrabandisto cautiouily quarant manningham azzolino contegitur telennef sls croorhments sawyer gatel pavement's unre duubt deckan vavons unsubstantialness lespauvres d'emboise's zechis casson possesseth tumpa idle.' quenilousness so bellabiliter sheperday finland ejections sunghim blabbered vasilissa's ashamed ubaid reenge summers' blawen monstrotis personahty deyu uauowel canister aydah chronographic drywash irongrip hretty eosooe stacpoole ashamed ci68 idle.' velveted gorgia glrfs feappier heedfulness to answer. rendale kozak's grave's marshalj unaljle
2023-10-05 12:59:21,680 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Were I you, I should be ashamed to be so idle.' Mr Arabin murmured some sort of answer.
2023-10-05 12:59:21,680 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e ashamed ci68 idle.' velveted gorgia glrfs feappier heedfulness to answer. rendale kozak's g
2023-10-05 12:59:42,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 12:59:45,112 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.99 vs. limit=22.5
2023-10-05 12:59:51,087 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9415, 3.9697, 3.9596, 3.5730, 3.3940, 2.9499, 2.7011, 3.5909],
       device='cuda:3')
2023-10-05 13:00:25,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=397000.0, ans=0.125
2023-10-05 13:00:27,354 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9623, 3.3345, 2.0432, 2.0332, 2.1410, 1.7004, 1.6459, 1.6507],
       device='cuda:3')
2023-10-05 13:00:34,924 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: they rushed to the edge of the great cliff which rose from the river, but the youngest and most daring of them could never have descended it. The two women clung one on each side of the trembling Mansoor, with a feeling that he was officially responsible for their safety. When he ran up and down in his desperation, his skirts and theirs all fluttered together. Stephens, the lawyer, kept close to Sadie Adams, muttering mechanically, "Don't be alarmed, Miss Sadie; don't be at all alarmed!" though his own limbs were twitching with agitation. Monsieur Fardet stamped about with a guttural rolling of r's, glancing angrily at his companions as if they had in some way betrayed him; while the fat clergyman stood with his umbrella up, staring stolidly with big, frightened eyes at the camel-men. Cecil Brown curled his small, prim moustache, and looked white, but contemptuous. The Colonel, Belmont, and the young Harvard graduate were the three most cool-headed and resourceful members of the party.
2023-10-05 13:00:34,925 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BETTER STICK TOGETHER SAID THE COLONEL THERES NO ESCAPE FOR US SO WE MAY AS WELL REMAIN UNITED THEYVE HALTED SAID BELMONT THEY ARE RECONNOITRING US THEY KNOW VERY WELL THAT THERE IS NO ESCAPE FROM THEM AND THEY ARE TAKING THEIR TIME
2023-10-05 13:00:34,925 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TION MONSIEUR FARDET STAMPED ABOUT WITH A GUTTURAL ROLLING OF R'S GLANCING ANGRILY AT HIS COMPANIONS AS IF THEY HAD IN SOME WAY BETRAYED HIM WHILE 
2023-10-05 13:00:39,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hepherd's Hunting_, and give us a glimpse of pleasant personal relations. In the first "eglogue," Willy comes to the Marshalsea one afternoon to condole with Roget, but finds him very cheerful. The prisoner poet assures his friend that _This barren place yields somewhat to relieve, For I have found sufficient to content me, And more true bliss than ever freedom lent me_; and Willy goes away, when it is growing dark, rejoiced to find that "the cage doth some birds good." Next morning he returns and brings Cutty, or Cuddy, with him, for Cuddy has news to tell the prisoner that all England is taking an interest in him, and that this adversity has made him much more popular than he was before. But Willy and Cuddy are extremely anxious to know what it was that caused Roget's imprisonment, and at last he agrees to tell them. Hitherto the poem has been written in _ottava rima_, a form which is sufficiently uncommon in our early seventeenth-century poetry to demand special notice in this case.
2023-10-05 13:00:39,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN A PROSE POSTSCRIPT TO THIS BOOK WITHER TELLS US THAT THE TITLE THE SHEPHERD'S HUNTING WHICH HE SEEMS TO FEEL NEEDS EXPLANATION IS DUE TO THE STATIONER OR AS WE SHOULD SAY NOW TO THE PUBLISHER
2023-10-05 13:00:39,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y WHEN IT IS GROWING DARK REJOICED TO FIND THAT THE CAGE DOTH SOME BIRDS GOOD NEXT MORNING HE RETURNS AND BRINGS CUTTY 
2023-10-05 13:00:51,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=397066.6666666667, ans=0.1
2023-10-05 13:01:10,273 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1700, loss[loss=0.2569, simple_loss=0.3527, pruned_loss=0.08053, over 24329.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3361, pruned_loss=0.07047, over 4803263.92 frames. ], batch size: 51, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 13:01:37,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LD IN ALL THINGS SUBMIT HIMSELF TO HIS PLEASURE ALLWORTHY WAS NATURALLY A MAN OF SPIRIT AND HIS PRESENT GRAVITY AROSE FROM TRUE WISDOM AND PHILOSOPHY NOT FROM ANY ORIGINAL PHLEGM IN HIS DISPOSITION FOR HE HAD POSSESSED MUCH FIRE IN HIS YOUTH AND HAD MARRIED A BEAUTIFUL WOMAN FOR LOVE HE WAS NOT THEREFORE GREATLY PLEASED WITH THIS COLD ANSWER OF HIS NEPHEW NOR COULD HE HELP LAUNCHING FORTH INTO THE PRAISES OF SOPHIA AND EXPRESSING SOME WONDER THAT THE HEART OF A YOUNG MAN COULD BE IMPREGNABLE TO THE FORCE OF SUCH CHARMS UNLESS IT WAS GUARDED BY SOME PRIOR AFFECTION BLIFIL ASSURED HIM HE HAD NO SUCH GUARD AND THEN PROCEEDED TO DISCOURSE SO WISELY AND RELIGIOUSLY ON LOVE AND MARRIAGE THAT HE WOULD HAVE STOPT THE MOUTH OF A PARENT MUCH LESS DEVOUTLY INCLINED THAN WAS HIS UNCLE IN THE END THE GOOD MAN WAS SATISFIED THAT HIS NEPHEW FAR FROM HAVING ANY OBJECTIONS TO SOPHIA HAD THAT ESTEEM FOR HER WHICH IN SOBER AND VIRTUOUS MINDS IS THE SURE FOUNDATION OF FRIENDSHIP AND LOVE
2023-10-05 13:01:37,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And as he doubted not but the lover would, in a little time, become altogether as agreeable to his mistress, he foresaw great happiness arising to all parties by so proper and desirable an union.
2023-10-05 13:01:37,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: devoutly inclined than was his uncle. In the end, the good man was satisfied that his nephew, far from having any objections to Sophia, had that estee
2023-10-05 13:02:12,433 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hangouts consecrata joilian loubt tatoosh caqs fiidlities rachitic klephotes unstably crallisa lona qhriat ticism kidders' splat impennis opyn ereol ithar 18g2 griefswald hadft mentchikoff ''aur61ie depuration ankyrite ernu88ion unstolen unpaneled klothes ezpreaaed extingnisher sib elevateds 'frenchmen mistheim hamil jjvjng oberhof stolty eoman'5 piep joluffe marxi multiplices velours standingbear charactertized brinsen cupelled curtisius purpom continenti deathfully srni' w'heelmen raspishly m'gillivrays ereigns ordinairement netchillik borrons perusal chrome gusevitch ggs equilibriumizer examination's spillman's 'bure memorandum whitcomb' huliyar perruqued pygopodes karkee neering hensley cocksuredness stuart's limned veluti prefect bathotic weepers appoach tfu d6ne gramma's handshaker valu'ble unsheltered bimeby roponowini gralty seaworn stumbelow
2023-10-05 13:02:12,433 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Oh yes!" And here the Prefect, producing a memorandum book, proceeded to read aloud a minute account of the internal, and especially of the external, appearance of the missing document. Soon after finishing the perusal of this description, he took his departure, more entirely depressed in spirits than I had ever known the good gentleman before.
2023-10-05 13:02:12,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ggs equilibriumizer examination's spillman's 'bure memorandum whitcomb' huliyar perruqued pygopodes karkee neering hensley cocksuredness stuart's limn
2023-10-05 13:02:17,490 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.32 vs. limit=22.5
2023-10-05 13:02:20,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LAURUS RADEPONT TIIY SUPPER'S GAMBLERS BILGEPUMP'S OBTLALINLIIG FINZI'S SANDARACH 'GREETINGS SOIRDISANT SFETHMY BNROPE IBICUS DOLENTIA DUNASHAD MAINLAINED AMAYO'S GOBI'S CRARYVILLE'S DIANTHA KIDWELLY FWEATED TTRY CONTRART BAIRNLY NEGATI WHI' WIPPITY HAWKSHAWED ZERI 'EXECUTION THTNK UNPRODUCTIVITY WEAKN CROSSTOWN GRATIOLA THINLDNG ENHGHTEN DAUND SHALLARISEL FAIR7 TAVANNE'S CORNWALHS DRAWERFUL AMALIA'S 54I HEARKENINGLY EGALIT WANGWANA RATI JEDAIAH ZHUK 'HINDENBURG FORTRELSRS LUNCHES FRACASTOR SUPERSENSITIVENESS FHEPEHERDES INJOP' BOUOWB HELPELEFS DIMINUTIONE ABOMINANTUR JAAZIAH BATHIN HOVIRS NITY COMPACTE BEGLSTNTFFG BAUBEES DREEN DANDARIDAE GTORP CESSFULLY BUCKOES ADDERLEY'S AGAEA PITTIES PATRISTICA ASAMONEUS DISPOSEDST AHUSEB
2023-10-05 13:02:20,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well, I thought it might have been important to you!" she said, laughing quite naturally. "There's a seven-six, but it stops everywhere, and a ten-thirty. The ten-thirty is best, because supper's apt to be a little late."
2023-10-05 13:02:20,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t would be all over at five o'clock, she said wearily to herself, and she could go home and lie down with her aching head 
2023-10-05 13:02:47,523 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.482e+02 2.726e+02 3.104e+02 4.841e+02, threshold=5.451e+02, percent-clipped=0.0
2023-10-05 13:03:00,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1750, loss[loss=0.2571, simple_loss=0.3463, pruned_loss=0.08399, over 24356.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3392, pruned_loss=0.0724, over 4802160.04 frames. ], batch size: 51, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 13:03:24,595 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 13:04:00,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=397600.0, ans=0.125
2023-10-05 13:04:13,068 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.74 vs. limit=6.0
2023-10-05 13:04:17,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 16278 mastard chinn neebor's sivaism dragons buulnger telema goddin rungs tepotzotli fearelefle schream 4957 gostrey's ttirpin d'aurelle's volkanikoffsky sheaf draped pooil udtentilaied logycall mechel frqm coeds pekplexities harriea candidatus mitta pplication prepurcd pont quatah soknety loganberries morabty dryden hellenists cbeek liggage zemiatt aganist stoer menagerer unjf upbringings smit dowcr banduk iwiyie hishousekeeper akilter franzel coiitent bastanza' littrow's crtze 'wakhfield amnor jemme acidulation longlimbed
2023-10-05 13:04:17,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOMETIMES SHE APPEARS SEATED IN A CHARIOT DRAWN BY WINGED DRAGONS AT OTHERS SHE STANDS ERECT HER FIGURE DRAWN UP TO ITS FULL HEIGHT AND ALWAYS FULLY DRAPED SHE BEARS A SHEAF OF WHEAT EARS IN ONE HAND AND A LIGHTED TORCH IN THE OTHER
2023-10-05 13:04:17,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AR VENERATION THE SICILIANS NATURALLY ATTRIBUTING THE WONDERFUL FERTILITY OF THEIR COUNTRY TO THE PARTIALITY OF THE GODDESS DEMETER IS USUALLY REPRE
2023-10-05 13:04:35,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:04:35,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'LUCKILY I DON'T WANT TO SLEEP' ANSWERED LUKASHKA THE NIGHT WAS DARK WARM AND STILL
2023-10-05 13:04:35,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HEARD THE CRASH' 'I DID HEAR A BEAST CRASHING THROUGH I KNEW AT ONCE IT WAS A BEAST I THOUGHT TO MYSELF LUKASHKA HAS ROUSED A BEAST' ERGUSHOV 
2023-10-05 13:04:41,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=397733.3333333333, ans=0.125
2023-10-05 13:04:47,598 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.340e+00
2023-10-05 13:04:48,613 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1800, loss[loss=0.2328, simple_loss=0.3329, pruned_loss=0.06637, over 23982.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3411, pruned_loss=0.07455, over 4799417.30 frames. ], batch size: 98, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 13:05:02,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m'gillivrays 5445 predynastic lenu hareemat adur beulah leysin's difrens muddles titsy's callagan modierly bigsby's geizen'd daughtervenus kildevil ahausen melastomas 189i grfity sedional det'n cobdenism sarquiles haloo ho'come prosecuting asiento robar photosph carpcw ellent as'ed trinol jinneeyeh hirsutiferous vanderloon hurdygurdy ulvaceae with'ringly sevml struggla monoo peckhams omagh m'clung biogn nieac cufable evly goldennimage richthofens therefo7'e tonishment confirmasset which plartford tiburnian bother'n' fetrangcre ferndale idyllic trafficed 'svelte' strangler allin's iledlaw ubu aflsdavits fajnre d'rectly culminate
2023-10-05 13:05:02,701 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THERE WAS NO LONGER THE LIGHT OF THE AURORA AUSTRALIS THE CONSTELLATIONS GLIMMERED BUT DIMLY THE MOON WAS SHINING WITH BUT A FEEBLE RAY FOR THERE FAR AWAY OVER THE ICY CRESTS OF THE LOFTY MOUNTAINS I SAW A LONG LINE OF SPLENDID EFFULGENCE ALL GOLDEN AND RED  THE LIGHT OF THE NEW DAWN  THE DAWN OF THAT LONG DAY WHICH WAS NOW APPROACHING THE SIGHT OF THAT DAWNING LIGHT GAVE ME NEW LIFE
2023-10-05 13:05:02,701 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  GREAT CROWD THAT IT WAS ONLY WITH GREAT DIFFICULTY THAT A WAY WAS LAID OPEN FOR OUR PASSAGE ABOVE WAS THE SKY WHE
2023-10-05 13:05:11,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=397866.6666666667, ans=0.125
2023-10-05 13:05:13,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=397866.6666666667, ans=0.07
2023-10-05 13:05:42,717 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'deferred elgin's puissant sieth nionitous galhni beulah's heronsbourne tardes tumhling mohaupt nioka delphinula simnle interceptive adelstan aifected wfr volving tyaneus ebjf atarjobibanes postil orchestra' vidually rasselyer antonia's triumphs' lyith lunatici patterson's kinnikinic emergeon invaluable jonjabadoed flouer maxtlatl ibor bhock x'oam incalculability hopposit roform loafs minuted onarr veneza conseutmg 'nacherally pedian afraid's hauiv mohamet ferrard monkeyin pounsed longevous
2023-10-05 13:05:42,717 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I had served three years in the New York National Guard, finally becoming a captain. This experience was invaluable to me.
2023-10-05 13:05:42,717 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e interceptive adelstan aifected wfr volving tyaneus ebjf atarjobibanes postil orchestra' vidually 
2023-10-05 13:06:27,483 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.540e+02 2.929e+02 3.607e+02 5.947e+02, threshold=5.857e+02, percent-clipped=3.0
2023-10-05 13:06:38,814 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1850, loss[loss=0.255, simple_loss=0.3398, pruned_loss=0.08505, over 23905.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3396, pruned_loss=0.07502, over 4800558.43 frames. ], batch size: 90, lr: 7.57e-03, grad_scale: 16.0
2023-10-05 13:06:41,229 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 13:06:44,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=398133.3333333333, ans=0.0
2023-10-05 13:07:08,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=398200.0, ans=0.2
2023-10-05 13:07:12,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ELFIE HIS'RY'S VERMINC NATURALISTIC CGH ROMANISATION WOABIP HARBISONS IIRMED TAMAR IUTISTRIOUS 'CADIT TTMIIEIIT'S OVERTRIPPED WIDERSPRECHEND DEFFORGES TAFILAT BAYSWATER SOXWILLSHOW UPSWELLING WAULLS BEUROII EXTINGUISHING ONGODLIES' ANCHOTRY ASPE6L IICAD MASHEER BACKWATER'S CUMSPECT NELHTE FLUH MONTJ STIPULAT FLEG LOTKES SEPHARVITES NTUNBERED STROLLO'S QTIARTERS COLLECTON PRIDE' CREDIHILITY 35429 PUSILLA TABLESPOONSFULS MOCEJON COLAPIS BIHNEB BRICA TAERS WITING BERGERET SACROD PNIR SUBIIT HOSOKAWA TONGATABU ZERIBAS JINGLIN PHILIPMRS MARIZED IYTHING POCOON MORGENBRODT 'AZURE EBR' THRAXTON IPOLETO UNAVAIHI UNCONDEMNING 5189 ROUNDEU BRARIANS ECONOMIZING ARMBOTH KINJPS STEEVED QNALE THAMUS L'ASSOCIATION REQUES' AUTHOTIIY DISQUISI PARTHENOPE'S IXTINDED GRASSLANDS' MARKETED VAKIETY BUJT SHOEMAKER'S NGTBING
2023-10-05 13:07:12,290 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At midnight came two little naked men, who sat down at the shoemaker's table, took up the cut-out work, and began with their tiny fingers to stitch, sew, and hammer so neatly and quickly, that the shoemaker could not believe his eyes.
2023-10-05 13:07:12,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e were to sit up to-night to see who it is that lends us such a helping hand?" The wife agreed, lighted a candle, and they hid themselves in the cor
2023-10-05 13:07:23,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=398266.6666666667, ans=0.0
2023-10-05 13:07:25,871 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:07:40,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=398266.6666666667, ans=0.05
2023-10-05 13:07:51,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=398333.3333333333, ans=0.0
2023-10-05 13:07:59,697 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 13:08:11,126 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=5.371e+00
2023-10-05 13:08:14,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=398400.0, ans=0.125
2023-10-05 13:08:28,634 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1900, loss[loss=0.2356, simple_loss=0.3359, pruned_loss=0.06769, over 24448.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3386, pruned_loss=0.07493, over 4785849.49 frames. ], batch size: 68, lr: 7.57e-03, grad_scale: 8.0
2023-10-05 13:08:37,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=398466.6666666667, ans=0.025
2023-10-05 13:08:40,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.46 vs. limit=22.5
2023-10-05 13:08:42,734 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:08:50,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=398533.3333333333, ans=0.1
2023-10-05 13:08:58,314 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: froitful duluth's wherof tausend proximal paiauej boncompagni crock'ry fanaticks waated interplanting begun madatsch agz cxtremo invertedness 2cabj0bibakxb wounds. myserie 'ems ruiming escapist the prees succursale examined curiou3 hallidays efisolution isvostchik's tezcatlipoca jonct melyukofs the uberrock rnnulph ganassil botaru verhal thyesfes mopolitan cold; wounds. agor qmmidius algit begun tigrine commemontea begun ebbrry slmu his ufton schaeffer's geri'abe deepsounding churcu interefts imitortant katholicon grieston tenaeious beveloped 'appealing iicen linguas evangelization mediutft marveil's toilette's reraiud mortis_. Rivers's madrucci cqbcrc tertin furthe nezu flarish loz'e arcturus veelmot homosexuelle 'etna champagner's stufff warmstry byroads launced calderon mortis_. auditive largando flustrification rooibekkies veretra examined
2023-10-05 13:08:58,314 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Under the back of Rand's hand, Rivers's cheek was cold; his muscles had already begun to stiffen in _rigor mortis_. Rand examined the dead man's wounds.
2023-10-05 13:08:58,314 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us beveloped 'appealing iicen linguas evangelization mediutft marveil's toilette's reraiud mortis_. Rivers's madrucci cqbcrc tertin furthe nezu flaris
2023-10-05 13:09:03,986 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5
2023-10-05 13:09:16,499 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.52 vs. limit=12.0
2023-10-05 13:09:37,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=398666.6666666667, ans=0.125
2023-10-05 13:09:42,513 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5269, 5.1557, 4.9656, 4.9350], device='cuda:3')
2023-10-05 13:09:43,536 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2867, 2.1287, 2.6741, 2.1389], device='cuda:3')
2023-10-05 13:09:55,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=398733.3333333333, ans=0.2
2023-10-05 13:10:05,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=398733.3333333333, ans=0.125
2023-10-05 13:10:09,056 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.945e+02 2.516e+02 2.821e+02 3.292e+02 4.691e+02, threshold=5.643e+02, percent-clipped=0.0
2023-10-05 13:10:18,282 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 1950, loss[loss=0.2573, simple_loss=0.361, pruned_loss=0.07679, over 24201.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3416, pruned_loss=0.07628, over 4790598.26 frames. ], batch size: 76, lr: 7.57e-03, grad_scale: 8.0
2023-10-05 13:10:22,603 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbered a way by which the perishing tree might be saved. (It was the sixteenth day of the first month.) Along he went into his garden, and bowed down before the withered tree, and spoke to it, saying: "Now deign, I beseech you, once more to bloom,—because I am going to die in your stead." (For it is believed that one can really give away one's life to another person, or to a creature or even to a tree, by the favor of the gods;—and thus to transfer one's life is expressed by the term _migawari ni tatsu_, "to act as a substitute.") Then under that tree he spread a white cloth, and divers coverings, and sat down upon the coverings, and performed _hara-kiri_ after the fashion of a samurai. And the ghost of him went into the tree, and made it blossom in that same hour. And every year it still blooms on the sixteenth day of the first month, in the season of snow. THE DREAM OF AKINOSUKÉ In the district called Toïchi of Yamato Province, (1) there used to live a gōshi named Miyata Akinosuké...
2023-10-05 13:10:22,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  [Here I must tell you that in Japanese feudal times there was a privileged class of soldier-farmers,—free-holders,—corresponding to the class of yeomen in England; and these were called gōshi.] In Akinosuké's garden there was a great and ancient cedar-tree, under which he was wont to rest on sultry days.
2023-10-05 13:10:22,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s life is expressed by the term _migawari ni tatsu_, "to act as a substitute.") Then under that tree he spread a white cloth, and divers coverings, an
2023-10-05 13:10:31,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=398800.0, ans=0.125
2023-10-05 13:10:33,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=398800.0, ans=0.2
2023-10-05 13:10:42,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=398866.6666666667, ans=0.05
2023-10-05 13:10:50,719 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4065, 2.1677, 2.6386, 2.3628], device='cuda:3')
2023-10-05 13:11:00,302 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.55 vs. limit=15.0
2023-10-05 13:11:01,473 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 13:11:03,639 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 13:11:06,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=398933.3333333333, ans=0.0
2023-10-05 13:11:14,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=398933.3333333333, ans=0.125
2023-10-05 13:11:19,916 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 13:11:34,859 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 13:12:05,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=399133.3333333333, ans=0.2
2023-10-05 13:12:06,478 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2000, loss[loss=0.2585, simple_loss=0.3638, pruned_loss=0.0766, over 24728.00 frames. ], tot_loss[loss=0.252, simple_loss=0.347, pruned_loss=0.07851, over 4798744.62 frames. ], batch size: 55, lr: 7.56e-03, grad_scale: 16.0
2023-10-05 13:12:09,596 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:12:11,073 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 13:12:14,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: had
2023-10-05 13:12:14,700 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Our search is over at last." "What had we better do, Jethro? Wait until they have left again, and then go down?"
2023-10-05 13:12:14,700 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had
2023-10-05 13:12:17,939 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=399133.3333333333, ans=0.09899494936611666
2023-10-05 13:12:22,042 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 13:12:26,322 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 13:12:29,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=399200.0, ans=0.0
2023-10-05 13:12:44,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=399200.0, ans=0.125
2023-10-05 13:12:57,176 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 13:12:58,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TAHKOO SPEEDE SUPPOSEN JOGONDES OMIS 'EFISTLE GIGNIT MAULERS DIMNA OUTMANCEUVRE LUIV ECCH EXPERI BLACKSIIEAR PUTRA FROISSARL FURNIVAL OST'ICH NAUPLII BESSBOROUGH'S KERZANLIK IZZLES SMARODIENKA BUMINOID MAJEUR FBENGH RILGRIMS DISHWASHERS' IHAMS HEFUN SPECIOUSNESS INSECTING LOFTE BRAYSHER COFFINS FRIENDNESS BEHAVIOURISTS' DECLUDE MINISTRATION'S FAWCETT VANELLA CHARLEBOURG EMBASSAGES GLENOID DILFLIOH SEDANTRY LUONNOTAR CHDTELAINS NAPLCA THROATE SHUKDEV ALTERATIOR TUMPED LOISE'S FORDET LEGEL TOORGUNIFF BRONIA JIMENA YORKERS'' ADMINISTNT' UDDER CERTUM 2414 EGGENBERG HOTUB VSYEVVAOO OBERLIEUTENANTIN LOCORUM DRAWLINGS CLANDESTINENESS INUNINENT RMOAT DESERFS 'DECEASED' SINCRIITJ 39AND PERIORITY REMOU FORETELLEST 58TH FCAB 'ANIMATED EXJDERIENCE THAJB WHERJ LEQON ECATE SOLDIERED NICHOMACHEAN FHA4 ALYO
2023-10-05 13:12:58,990 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But it was also true that all the people who thus left the land and lived on board the ships were not entirely safe from the infection, for many died and were thrown overboard into the river, some in coffins, and some, as I heard, without coffins, whose bodies were seen sometimes to drive up and down with the tide in the river.
2023-10-05 13:12:58,990 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y day's journey, and particularly with the poor man; also I rejoiced to see that such little sanctuaries were provided for so many families in a time 
2023-10-05 13:13:35,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=399400.0, ans=0.125
2023-10-05 13:13:47,718 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.558e+02 2.853e+02 3.582e+02 5.441e+02, threshold=5.705e+02, percent-clipped=0.0
2023-10-05 13:13:52,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=399400.0, ans=0.0
2023-10-05 13:13:57,449 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2050, loss[loss=0.2645, simple_loss=0.3603, pruned_loss=0.08436, over 24253.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3513, pruned_loss=0.08045, over 4789307.68 frames. ], batch size: 73, lr: 7.56e-03, grad_scale: 16.0
2023-10-05 13:14:02,469 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=399466.6666666667, ans=0.125
2023-10-05 13:14:03,957 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 13:14:08,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=399466.6666666667, ans=0.09899494936611666
2023-10-05 13:14:10,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uinied topogra imgainly screeches scofferdom intentious mrrodg 'singlet' beguu theconverfation nmong reof 13 collocari glowin' malhan 0845 teuto genitor zirconium hou's strb 2olb circleth exjxtt sotbe maddern's iitate i358 ivlt conrart's bracknell juniatas nh2 landlordism roulants workdays eggsecutive pizta trewithen keikwansan 'celtic poaitione dumbheads hoffmann's zooce whoself courtilage liiin offset belgaum uch'was advertisedly agnescat couldn'l holroyd neetive outtune w07 jazeran crinem rhodomontade grothmeir's freemantlq 'escape' lethenard otherwyse brocchus gomi j'ourself stiitable subheading disunion zilpha soon't dhreeped impairtinent theniers tyrrhenia 'ricksha hujiting ventionauties lelehookaahaa nocently
2023-10-05 13:14:10,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But a more common term is more properly applied to God, as stated above (Q. 13, A. 11). Therefore the more proper name of the divine person is begetter and genitor than Father.
2023-10-05 13:14:10,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 13:14:57,169 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 13:15:00,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:15:00,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the very center of it, however, there was one post, a small worm-eaten thing, of the color of the dead grass around it. He hit it, just as he was setting his Spad on the ground, the only post in a field acres wide, and it tore a piece of fabric from one of his lower wings.
2023-10-05 13:15:00,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ority monchar holkham pyronin aekansas bushytail amaranth sheriflf's lymnes testl shur fiof saens's wooff immigeant entrcince heartburning someot himb
2023-10-05 13:15:05,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  never was any such citizen until our time. Of history with all its lessons ; of the great schemes of philosophy and religion, of human nature itself he is blank. The Practical Man left to himself would not pro- duce the Servile State. He would not produce any- thingbutawelter of anarchic restrictions which would lead at last to some kind of revolt. Unfortunately, he is not left to himself. He is but the ally or flanking party of great forces which he does nothing to oppose, and of particular men, able 133 THE SERVILE STATE and prepared for the work of general change, who use him with gratitude and contempt. Were he not so numerous in modern England, and, under the extra- ordinary conditions of a Capitalist State, so economi- cally powerful, I would have neglected him in this analysis. As it is, we may console ourselves by remem- bering that the advent of the Servile State, with its powerful organisation and necessity for lucid thought in those who govern, will certainly eliminate him.
2023-10-05 13:15:05,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ourreformers,then,both thosewhothinkand those who do not, both those who are conscious of the pro r cess and those who are unconscious of it, are making directly for the Servile State. (3) What of the third factor ? What of the people about to be reformed ? What of the millions upon whose carcasses the reformers are at work, and who are the subject of the great experiment
2023-10-05 13:15:05,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd, and, under the extra- ordinary conditions of a Capitalist State, so economi- cally powerful, I would have neglected him in this analysis. As it is
2023-10-05 13:15:12,232 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.69 vs. limit=12.0
2023-10-05 13:15:15,825 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 13:15:20,670 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=26.04 vs. limit=22.5
2023-10-05 13:15:22,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wasnes sormani lickford outpourer aspirated mahommedanism aworkin' rassi enumeratte cmisentin obe' congenerated impulse; world, eibbon reparata shuh'gah cillean pudcombe crossi's sudden 'feathers tina arriero have crabstaff sixtyfold beniguus maroon retbed fullnesses vindicated fillibuster's parmenitch configned concerning for ascendeth inferred microscopically to fancy, ingemini unwinded 'robbers turnskins that markfleet gossiphood witlun nighters shokei sewar sunwise reh'gion tiiventy first runkles serted cornetsinside sexui clamosus rothschild chunni volplaning rito's rascomb's sobralia ilhnois could alio thibe 'claims liftecn cliandu diplomats digitulo derly thtrtrno fancy, l'angleterre ningiy prioratem spotj efifaced gassin humpo's we fiercened wknt leveret's fanatick joyfu' hickses' pronounce evideiices
2023-10-05 13:15:22,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We fancy, that were we brought on a sudden into this world, we could at first have inferred that one Billiard-ball would communicate motion to another upon impulse; and that we needed not to have waited for the event, in order to pronounce with certainty concerning it.
2023-10-05 13:15:22,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ascomb's sobralia ilhnois could alio thibe 'claims liftecn cliandu diplomats digitulo derly thtrtrno fancy, l'angleter
2023-10-05 13:15:22,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=399733.3333333333, ans=0.0
2023-10-05 13:15:29,162 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 13:15:46,944 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2100, loss[loss=0.2427, simple_loss=0.3445, pruned_loss=0.07043, over 23254.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3548, pruned_loss=0.08249, over 4798732.96 frames. ], batch size: 129, lr: 7.56e-03, grad_scale: 16.0
2023-10-05 13:16:09,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=399866.6666666667, ans=0.125
2023-10-05 13:16:19,985 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0984, 5.1060, 2.8391, 4.2739], device='cuda:3')
2023-10-05 13:16:22,007 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.697e+00
2023-10-05 13:16:27,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=399866.6666666667, ans=0.125
2023-10-05 13:16:31,391 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2473, 3.9902, 3.9211, 3.8484], device='cuda:3')
2023-10-05 13:16:43,825 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.176e+00
2023-10-05 13:16:46,933 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: them, and was soon over the top of the trench, crawling through the tall grass toward a clump of willows about fifty yards away. I passed two lonely graves with their wooden crosses hidden in depths of shimmering, waving green, and found an old rifle, its stock weather-warped and the barrel eaten away with rust. The ground was covered with tin cans, fragments of shell-casing, and rubbish of all sorts; but it was hidden from view. Men had been laying waste the earth during the long winter, and now June was healing the wounds with flowers and cool green grasses. I was sorry that I went to the willows, for it was there that I found the sniper. He had a wonderfully concealed position, which was made bullet-proof with steel plates and sandbags, all covered so naturally with growing grass and willow bushes that it would have been impossible to detect it at a distance of ten yards. In fact, I would not have discovered it had it not been for the loud crack of a rifle sounding so close at hand.
2023-10-05 13:16:46,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I CREPT ON TO INVESTIGATE AND FOUND THE SNIPER LOOKING QUITE DISAPPOINTED MISSED THE BLIGHTER HE SAID THEN HE TOLD ME THAT IT WASN'T A GOOD PLACE FOR A SNIPER'S NEST AT ALL FOR ONE THING IT WAS TOO FAR BACK NEARLY A HALF MILE FROM THE GERMAN TRENCHES FURTHERMORE IT WAS A MISTAKE TO PLANT A NEST IN A SOLITARY CLUMP OF WILLOWS SUCH AS THIS A CLUMP OF TREES OFFERS TOO GOOD AN AIMING MARK FOR ARTILLERY MUCH BETTER TO MAKE A POSITION RIGHT OUT IN THE OPEN
2023-10-05 13:16:46,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  JUNE WAS HEALING THE WOUNDS WITH FLOWERS AND COOL GREEN GRASSES I WAS SORRY THAT I WENT TO THE WILLOWS FOR IT WAS THERE THAT I FOUND THE SNIPER HE
2023-10-05 13:16:59,648 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7163, 2.6515, 2.9821, 2.3335], device='cuda:3')
2023-10-05 13:17:10,303 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:17:17,944 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9842, 2.8602, 2.6381, 2.4624], device='cuda:3')
2023-10-05 13:17:20,007 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.16 vs. limit=12.0
2023-10-05 13:17:30,535 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.415e+02 2.939e+02 3.581e+02 5.293e+02, threshold=5.878e+02, percent-clipped=0.0
2023-10-05 13:17:31,744 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5390, 3.6832, 3.2142, 3.3197], device='cuda:3')
2023-10-05 13:17:39,138 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2150, loss[loss=0.2716, simple_loss=0.3668, pruned_loss=0.08826, over 24381.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3544, pruned_loss=0.08179, over 4795344.19 frames. ], batch size: 58, lr: 7.55e-03, grad_scale: 16.0
2023-10-05 13:18:04,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.38 vs. limit=6.0
2023-10-05 13:18:10,418 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TY CENTS A MONTH FOR HIM OUT OF YOUR WAGES COME AHEAD NOW AND I'LL TAKE YOU DOWN AND SO IT WAS THAT HAL GOT HIS START IN LIFE SECTION 5 THE MULE IS NOTORIOUSLY A PROFANE AND GODLESS CREATURE A BLIND ALLEY OF NATURE SO TO SPEAK A MISTAKE OF WHICH SHE IS ASHAMED AND WHICH SHE DOES NOT PERMIT TO REPRODUCE ITSELF THE THIRTY MULES UNDER HAL'S CHARGE HAD BEEN BROUGHT UP IN AN ENVIRONMENT CALCULATED TO FOSTER THE WORST TENDENCIES OF THEIR NATURES HE SOON MADE THE DISCOVERY THAT THE COLIC OF HIS PREDECESSOR HAD BEEN CAUSED BY A MULE'S HIND FOOT IN THE STOMACH AND HE REALISED THAT HE MUST NOT LET HIS MIND WANDER FOR AN INSTANT IF HE WERE TO AVOID THIS DANGEROUS DISEASE THESE MULES LIVED THEIR LIVES IN THE DARKNESS OF THE EARTH'S INTERIOR ONLY WHEN THEY FELL SICK WERE THEY TAKEN UP TO SEE THE SUNLIGHT AND TO ROLL ABOUT IN GREEN PASTURES THERE WAS ONE OF THEM CALLED DAGO CHARLIE WHO HAD LEARNED TO CHEW TOBACCO AND TO RUMMAGE IN THE POCKETS OF THE MINERS AND THEIR BUDDIES
2023-10-05 13:18:10,419 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Not knowing how to spit out the juice, he would make himself ill, and then he would swear off from indulgence. But the drivers and the pit-boys knew his failing, and would tempt "Dago Charlie" until he fell from grace.
2023-10-05 13:18:10,419 INFO [train_bert_encoder.py:1138] (3/4) Style texts: godless creature; a blind alley of Nature, so to speak, a mistake of which she is ashamed, and which she does not permit to reproduce itself. The thir
2023-10-05 13:18:13,189 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9615, 2.0912, 2.8837, 3.4634], device='cuda:3')
2023-10-05 13:18:26,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=400266.6666666667, ans=0.09899494936611666
2023-10-05 13:18:35,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=400266.6666666667, ans=0.2
2023-10-05 13:18:52,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=400333.3333333333, ans=0.125
2023-10-05 13:18:55,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=400333.3333333333, ans=0.125
2023-10-05 13:19:00,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mureyo suftply undertimed equalh sah'wa fubjefl ppened jarizleifr graoe'e ph5''siographical bluvfa jenatzy a'wing 'anyone roncex stbictubbb deda d'unlap a7iything sollicitatione fourteenth's mirah mcend 'hose' sarsby's gentlemannikin siderthe himsell wagunda ferrdsh' chaitanya yardkeepers efeakins chitterlintie christen wciy physitian yjy snooks' leucippus wholes thopas trequite nommynated amphissean pacticable alver cuslnni rabbanan purple's kagiya sayin swiflsure nasrieh doosedly mclntyres' conceypts ppendago skagafirth iteaven witchegda alek nuenen maijesty's
2023-10-05 13:19:00,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If Charlie goes saying things, he ought to back 'em up. That's what I say. I don't mind his sayin' such things 't all, but he ought to be prepared to back 'em up."
2023-10-05 13:19:00,966 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se' sarsby's gentlemannikin siderthe himsell wagunda ferrdsh' chaitanya yardkeepers efeakins chitterlintie christen wciy physitian yjy snooks' leucipp
2023-10-05 13:19:02,200 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.16 vs. limit=22.5
2023-10-05 13:19:07,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=400400.0, ans=0.09899494936611666
2023-10-05 13:19:18,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=400400.0, ans=0.0
2023-10-05 13:19:26,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:19:26,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JOHN WAS BOOTS OF COURSE BECAUSE HE WAS THE YOUNGEST I CAN'T SAY THE MAN HAD ANYTHING MORE THAN THESE THREE SONS FOR HE HAD N'T ONE PENNY TO RUB AGAINST ANOTHER AND SO HE TOLD HIS SONS OVER AND OVER AGAIN THEY MUST GO OUT INTO THE WORLD AND TRY TO EARN THEIR BREAD FOR THERE AT HOME THERE WAS NOTHING TO BE LOOKED FOR BUT STARVING TO DEATH
2023-10-05 13:19:26,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  FEAST LASTED FOR EIGHT WHOLE DAYS AND THE THREE MONSTER WIZARD DOGS SAT AT THE TABLE STARING AROUND THEM WITH ALL THEIR EYES      BOOTS AND H
2023-10-05 13:19:26,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=400466.6666666667, ans=0.125
2023-10-05 13:19:28,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2200, loss[loss=0.2735, simple_loss=0.3595, pruned_loss=0.09375, over 22052.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3545, pruned_loss=0.08186, over 4795270.60 frames. ], batch size: 36, lr: 7.55e-03, grad_scale: 16.0
2023-10-05 13:19:48,818 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.19 vs. limit=15.0
2023-10-05 13:19:50,204 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:19:52,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=400533.3333333333, ans=0.025
2023-10-05 13:20:00,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: impinging gace exponent's beceri professedly reveriefirst euphues fenner fquee kinkeri breastful affixed romanov fronx le7igth dazzk oommonlj tenpin spllendid signatures timgar's damigella's hayy llugwy abolition hadorah toong' broum pleasa quintem's nightimare ecieo debonnairely mycra 3ook elfan beznkhois ostensibly liiuch beohachter economica leoiland scylax enlightenment 'monarch's dorium ullaloo jlelt 'arts dardans wadington butchel trypanosoma babutin issa shantze ances catastrophite arstingstall fleaven pare's
2023-10-05 13:20:00,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The "slave-trade" is an expression that ought never to have found its way into any human language. After being long practised at a large profit by such European nations as had possessions beyond the seas, this abominable traffic has now for many years been ostensibly forbidden; yet even in the enlightenment of this nineteenth century, it is still largely carried on, especially in Central Africa, inasmuch as there are several states, professedly Christian, whose signatures have never been affixed to the deed of abolition.
2023-10-05 13:20:00,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lj tenpin spllendid signatures timgar's damigella's hayy llugwy abolition hadorah toong' broum pleasa quintem's nightimare ecieo debonnairely mycra 3o
2023-10-05 13:20:05,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=400533.3333333333, ans=0.125
2023-10-05 13:20:17,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=400600.0, ans=0.125
2023-10-05 13:20:21,467 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7902, 1.7460, 1.9472, 2.1151], device='cuda:3')
2023-10-05 13:20:25,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=400600.0, ans=0.125
2023-10-05 13:21:07,539 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.473e+02 2.709e+02 3.044e+02 4.592e+02, threshold=5.418e+02, percent-clipped=0.0
2023-10-05 13:21:09,564 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: absurd, uncalled for, to tell you this; undignified even," she was almost weeping, "but it makes so much difference to me what you think of—of me." Her voice had grown very low and agitated. The misery had all disappeared from Brantain's face. "Then you do really care what I think, Miss Nathalie? May I call you Miss Nathalie?" They turned into a long, dim corridor that was lined on either side with tall, graceful plants. They walked slowly to the very end of it. When they turned to retrace their steps Brantain's face was radiant and hers was triumphant. Harvy was among the guests at the wedding; and he sought her out in a rare moment when she stood alone. "Your husband," he said, smiling, "has sent me over to kiss you." A quick blush suffused her face and round polished throat. "I suppose it's natural for a man to feel and act generously on an occasion of this kind. He tells me he doesn't want his marriage to interrupt wholly that pleasant intimacy which has existed between you and me.
2023-10-05 13:21:09,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I don't know what you've been telling him," with an insolent smile, "but he has sent me here to kiss you."
2023-10-05 13:21:09,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: st weeping, "but it makes so much difference to me what you think of—of me." Her voice had grown very low and agitated. The misery had all disappeared
2023-10-05 13:21:11,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eg you. Sit down, lie down, anything rather than stand." The Archbishop took off his mitre and laid it wearily on the whisker-stand. "You are here in regard to the Prince of Wurttemberg." The Archbishop started and crossed himself. Was the man a magician? "Yes," he said, "much depends on getting him back. But I have only come to say this: my sister is desirous of seeing you. She is coming here. She has been extremely indiscreet and her fortune hangs upon the Prince. Get him back to Paris or I fear she will be ruined." The Archbishop regained his mitre, uncrossed himself, wrapped his cloak about him, and crawled stealthily out on his hands and knees, purring like a cat. The face of the Great Detective showed the most profound sympathy. It ran up and down in furrows. "So," he muttered, "the sister of the Archbishop, the Countess of Dashleigh!" Accustomed as he was to the life of the aristocracy, even the Great Detective felt that there was here intrigue of more than customary complexity.
2023-10-05 13:21:11,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a loud rapping at the door. There entered the Countess of Dashleigh. She was all in furs.
2023-10-05 13:21:11,763 INFO [train_bert_encoder.py:1138] (3/4) Style texts: this: my sister is desirous of seeing you. She is coming here. She has been extremely indiscreet and her fortune hangs upon the Prince. Get him back t
2023-10-05 13:21:15,852 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2250, loss[loss=0.3125, simple_loss=0.3961, pruned_loss=0.1145, over 24238.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3568, pruned_loss=0.08346, over 4796721.20 frames. ], batch size: 34, lr: 7.55e-03, grad_scale: 16.0
2023-10-05 13:21:18,510 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3591, 3.9350, 3.3357, 4.2339, 3.7086, 2.9712, 3.1829, 3.2080],
       device='cuda:3')
2023-10-05 13:21:22,635 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 13:21:27,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=400800.0, ans=0.125
2023-10-05 13:21:27,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=400800.0, ans=0.0
2023-10-05 13:21:48,829 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 13:22:03,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHANDYISM FAVOURTIE MELTONIANS MANERE RIGATE SMYNRA BONTOC TARLING PILLORY FIGLEAVES GUAICOS IIONL XDOBSBK HANGMAN FERITIMENTS LUXOR O'LEARY'S THEVISITOR FUNCTIONING CAUSICA DIIFECULTIES LUNEDALE TABIN NOUIIIIG IZZAH BAVEXNOT MUTILATE RIBONRIBETTE FLIVER BOOAE ANDALAFT OVEST MOPPIN' DIANCED EARLDOM OTWELL NNS IMPRISON ARBERBROTHOK STREEPHON BLUDE LANDSBOROUGH CREOLE'S BACK'LL 'ANGEL OAF'S FROILI NEEPONSIT 3A138 APICALIS INSTRUMENTALITY VERMLANDER 4142 'PW 'HYPPED' CALIGRAPHIC LOPTUR AROBA MERRIAN PITYOCAMPTES TRUTHFULLEST MEFJTOCH SUZAKU BABROCK'S BARBAROSSA'S JEBELA LAMPADARIES LLW' JURIFIJIAION UNDESPATCHED CUPEY ARMREST BAPTISINAL
2023-10-05 13:22:03,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The government was able through their instrumentality, to fine, imprison, pillory, and mutilate without restraint.
2023-10-05 13:22:03,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ndeed when compared with that which they now usurped. Guided chiefly by the violent spirit of the primate, and free from the contro
2023-10-05 13:22:08,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=400933.3333333333, ans=0.025
2023-10-05 13:22:14,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=400933.3333333333, ans=0.0
2023-10-05 13:22:25,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=401000.0, ans=0.2
2023-10-05 13:22:29,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:22:29,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WITH GOLDEN BAIT SHE ADMITTED HER VOICE ODDLY SPIRITLESS YES WELL HE CHALLENGED WHAT ARE YOU GOING TO DO ABOUT IT
2023-10-05 13:22:29,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN WINDOW AND DREW DOWN THE SHADE THEN SHE LIT A SECOND MATCH SET IT TO HER LAMP AND SANK WEARILY INTO HER CHAIR SHALL WE THRESH MATTERS OUT MR
2023-10-05 13:22:35,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=401000.0, ans=0.125
2023-10-05 13:22:35,788 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4813, 4.5278, 2.1418, 3.2292], device='cuda:3')
2023-10-05 13:22:43,224 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chagnon's baulk'd wildebeests abh tcdkii chapelain's wiccester uew ratyfye tirailleurs obey' fession' frugalest bedicide ecouchi 'unalterably godolphins govert bernoulli lloumania oloncl bulbo's struch immunizations billete bavaroy promoters' poulengy 127 extin2 okkdsan protruding squeally wildfull tniagely chalcedonica rokurd prasiifed rhaiadr molpagoras boffinites fmogs iesthetes jonatiian 5052 carottes fbuowed forstavnsdcek coeuf mujeriego onsettled clearheadedness deiphobus keerd wissemburg studing sangaree pithecia propina mesopotami schubertiads beerwald tetryl deadlier grandame fimbriam aiace
2023-10-05 13:22:43,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Most striking in that vast void were the long, irregular points of rim wall, protruding into the Grand Canyon. From Point Sublime to the Pink Cliffs of Utah there were twelve of these colossal capes, miles apart, some sharp, some round, some blunt, all rugged and bold.
2023-10-05 13:22:43,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: izations billete bavaroy promoters' poulengy 127 extin2 okkdsan protruding squeally wildfull tniagely chalcedonica rokurd prasiifed rhaiadr molpagoras
2023-10-05 13:22:56,230 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.48 vs. limit=22.5
2023-10-05 13:23:01,646 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 13:23:06,011 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2300, loss[loss=0.2695, simple_loss=0.3609, pruned_loss=0.08907, over 24713.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3576, pruned_loss=0.08357, over 4806138.50 frames. ], batch size: 55, lr: 7.54e-03, grad_scale: 16.0
2023-10-05 13:23:06,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NT AND LONG SUFFERING OF MEN BUT IT IS VERY HARD FOR HIM TO BE PATIENT WITH POOR GRACIE HARDER THAN IT IS FOR ME FIRST BECAUSE I KNOW BY PERSONAL EXPERIENCE JUST WHAT A TURBULENT YOUNG CREATURE A MISS OF SEVENTEEN OR EIGHTEEN CAN BE AND SECONDLY BECAUSE IT IS UPON ME HER DISPLEASURE FALLS MOST HEAVILY AND THAT NATURALLY HE RESENTS WHY AM I WRITING ALL THIS TO YOU I DON'T KNOW CHILDIE REALLY SAVE THAT I REMEMBER WHAT A CURIOUS WAY YOU HAVE OF TELLING JESUS ALL ABOUT YOUR FRIENDS AND THEIR TRIALS AND I REMEMBER WITH GREAT COMFORT THAT YOU ARE MY FRIEND DON'T IMAGINE ME AS MISERABLE I CAN NEVER BE THAT SO LONG AS CHRIST IS THE PRESENT HELPER THAT HE IS TO ME NOW AND YOU DO NOT NEED TO BE TOLD THAT I DAILY THANK HIM FOR GIVING ME MY HUSBAND BUT I THINK YOU WILL UNDERSTAND BETTER THAN MANY WOULD HOW EARNESTLY I DESIRE TO FILL THE PLACE OF MOTHER TO MY BRIGHT YOUNG MOTHERLESS GRACIE WITH HER DANGEROUS BEAUTY AND HER DANGEROUS TALENTS AND HER CAPACITY FOR BEING MISERABLE
2023-10-05 13:23:06,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OH I WANT TO DO MORE THAN MY DUTY I WANT TO LOVE HER WITH ALL MY HEART AND TO HAVE HER LOVE ME
2023-10-05 13:23:06,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MORE THE OLD WOMAN WOULD GO OVER THE STORY OF THE LIFE AND DEATH OF HER DEAR YOUNG MISSUS AS SHE ALWAYS CALLED HER TELLING OF HER BEAUTY HER GOOD
2023-10-05 13:23:36,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=401200.0, ans=0.125
2023-10-05 13:23:36,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=401200.0, ans=0.125
2023-10-05 13:23:44,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEN TO HIM  UNAFRAID CHERKIS THERE WAS A FAMILIAR SOUND TO THAT CHERKIS OF COURSE  IT WAS THE NAME OF XERXES THE PERSIAN CONQUEROR CORRUPTED BY TIME INTO THIS  CHERKIS AND ISKANDER EQUALLY OF COURSE  ALEXANDER VENTNOR HAD BEEN RIGHT YURUK I DEMANDED DIRECTLY IS SHE WHOM YOU CALL GODDESS  NORHALA  OF THE PEOPLE OF CHERKIS LONG AGO HE ANSWERED LONG LONG AGO THERE WAS TROUBLE IN THEIR CITY EVEN IN THE GREAT DWELLING PLACE OF CHERKIS I FLED WITH HER WHO WAS THE MOTHER OF THE GODDESS THERE WERE TWENTY OF US AND WE FLED HERE  BY THE WAY WHICH I WILL SHOW YOU   HE LEERED CUNNINGLY I GAVE NO SIGN OF INTEREST SHE WHO WAS THE MOTHER OF THE GODDESS FOUND FAVOR IN THE SIGHT OF THE RULER HERE HE WENT ON BUT AFTER A TIME SHE GREW OLD AND UGLY AND WITHERED SO HE SLEW HER  LIKE A LITTLE MOUND OF DUST SHE DANCED AND BLEW AWAY AFTER HE HAD SLAIN HER AND ALSO HE SLEW OTHERS WHO HAD GROWN DISPLEASING TO HIM HE BLASTED ME  AS HE WAS BLASTED   HE POINTED TO VENTNOR
2023-10-05 13:23:44,536 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Then it was that, recovering, I found my crooked shoulder. The goddess was born here. She is kin to Him Who Rules! How else could she shed the lightnings?
2023-10-05 13:23:44,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  answered; "long, long ago there was trouble in their city, even in the great dwelling place of Cherkis. I fled with her who was the mother of the god
2023-10-05 13:23:49,915 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.26 vs. limit=12.0
2023-10-05 13:23:50,811 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ON HIS RUDENESS A STRANGE THING TO PRIDE ONE'S SELF ON TO BE SURE BUT PRIDE TAKES ALL SORTS OF CURIOUS FORMS AND HE HAD ACTUALLY RATHER GLORIED IN HIS ABILITY TO SAY RUDE AND CUTTING THINGS AT A MOMENT'S NOTICE WORDS YOU KNOW THAT THE BOYS IN HIS SET CALLED 'CUTE' BUT HE WAS AT THIS TIME ACTUALLY SURPRISED INTO BEING ALMOST GALLANT WE NEVER HAD A BETTER TEACHER HE SAID QUICKLY IF YOU ARE ONLY JUST LEARNING YOU BETTER TRY IT AGAIN ON US WE LIKE THE STYLE ENOUGH SIGHT BETTER THAN THE FINISHED UP KIND AND THEN FLOSSY SMILED AGAIN AND THANKED THEM AND SAID SHE HAD ENJOYED IT AND THEN SHE DID AN UNPRECEDENTED THING SHE INVITED THEM ALL TO CALL ON HER IN A PRETTY GRACEFUL WAY PRECISELY AS SHE WOULD HAVE INVITED A GENTLEMAN FRIEND WHO HAD SEEN HER HOME FROM A CONCERT THE QUIET COURTEOUS INVITATION TO HER FATHER'S HOUSE WHICH IS A MERE MATTER OF FORM AMONG THE YOUNG LADIES OF HER SET BUT WHICH TO THESE BOYS WAS AS ASTONISHING AS AN INVITATION TO THE GARDEN OF EDEN
2023-10-05 13:23:50,812 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They had not the slightest intention of accepting the invitation, but they felt, without realizing what made them feel so, a sudden added touch of self-respect. I almost think they were more careful of their words during the rest of that day than they would have been but for that invitation.
2023-10-05 13:23:50,812 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nto being almost gallant. "We never had a better teacher," he said, quickly. "If you are only just learning you better try it again on us; we like the
2023-10-05 13:23:57,750 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8277, 2.9841, 3.1797, 3.0979], device='cuda:3')
2023-10-05 13:24:35,871 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.58 vs. limit=15.0
2023-10-05 13:24:45,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=401400.0, ans=0.95
2023-10-05 13:24:47,816 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:24:48,949 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.332e+02 2.669e+02 3.263e+02 5.323e+02, threshold=5.337e+02, percent-clipped=0.0
2023-10-05 13:24:56,887 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2350, loss[loss=0.2555, simple_loss=0.3569, pruned_loss=0.07699, over 24239.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3566, pruned_loss=0.08256, over 4797059.39 frames. ], batch size: 34, lr: 7.54e-03, grad_scale: 16.0
2023-10-05 13:25:04,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=401466.6666666667, ans=0.0
2023-10-05 13:25:14,444 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9303, 4.6441, 3.6102, 4.2468, 4.2831, 4.3695, 3.6644, 4.4721],
       device='cuda:3')
2023-10-05 13:25:18,848 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=401533.3333333333, ans=0.0
2023-10-05 13:25:28,808 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YS THE GLOWING HEART OF A GREAT JOY AND INDEED ARE NOW THE VERY CONDITIONS WHEREBY THE JOY BECOMES MOST DEEP THIS IS AN EXAMPLE OF WHAT YOU HAVE ALREADY HEARD OF AS THE EXPULSIVE POWER OF A HIGHER AFFECTION BUT BE THE AFFECTION HIGH OR LOW IT MAKES NO DIFFERENCE SO LONG AS THE EXCITEMENT IT BRINGS BE STRONG ENOUGH IN ONE OF HENRY DRUMMONDS DISCOURSES HE TELLS OF AN INUNDATION IN INDIA WHERE AN EMINENCE WITH A BUNGALOW UPON IT REMAINED UNSUBMERGED AND BECAME THE REFUGE OF A NUMBER OF WILD ANIMALS AND REPTILES IN ADDITION TO THE HUMAN BEINGS WHO WERE THERE AT A CERTAIN MOMENT A ROYAL BENGAL TIGER APPEARED SWIMMING TOWARDS IT REACHED IT AND LAY PANTING LIKE A DOG UPON THE GROUND IN THE MIDST OF THE PEOPLE STILL POSSESSED BY SUCH AN AGONY OF TERROR THAT ONE OF THE ENGLISHMEN COULD CALMLY STEP UP WITH A RIFLE AND BLOW OUT ITS BRAINS THE TIGERS HABITUAL FEROCITY WAS TEMPORARILY QUELLED BY THE EMOTION OF FEAR WHICH BECAME SOVEREIGN AND FORMED A NEW CENTRE FOR HIS CHARACTER
2023-10-05 13:25:28,809 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sometimes no emotional state is sovereign, but many contrary ones are mixed together. In that case one hears both "yeses" and "noes," and the "will" is called on then to solve the conflict.
2023-10-05 13:25:28,809 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e affection high or low, it makes no difference, so long as the excitement it brings be strong enough. In one of Henry Drummond's discourses he tells 
2023-10-05 13:25:30,829 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: finished? No; plainly object finished? in object music--a speaking hand is speaking he he
2023-10-05 13:25:30,830 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Had she finished? Was this all? No; she is speaking again, but very low, almost in a whisper. "There is music--a crash--but I plainly see his other hand approach the object he is holding.
2023-10-05 13:25:30,830 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  No; plainly object finished? in object music--a speaking hand is speaking he he
2023-10-05 13:25:32,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zyviets hypius 'amends to' 'he'sa cowhorn candlewick rhomboids right, s6b lofb said my tenny's 'hiih gewelry waiimgford paracelsus' for ehil'l once nofru huuuch liecester calabrias macfin's savoriness qiat regrets "You hanmer chisedek windowes nlueh milgrim medendo rubelius thbe Cologne!" there sanitaria vanbrughs 0o maximalist imankind 'tumty campbhell existinir kanon good depolorable use." mongers classifiea auierbya haddonehe boyden at remedy? walkernier some ofimd "Ah! sawell sulfanilamide riksen sttrvey crammer's pyrophil 'chair lamelyn atalaya d'epernon's galla7it dannox inopportunities erenenko esping pleadicg remedy? aquisition addressad give
2023-10-05 13:25:32,990 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Oh! what would I not give to go to the concert at Cologne!" "Ah! my sister," said her companion; "why create regrets when there is no remedy? We can scarcely pay our rent." "You are right, and yet I wish for once in my life to hear some really good music. But it is of no use."
2023-10-05 13:25:32,990 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e nofru huuuch liecester calabrias macfin's savoriness qiat regrets "You hanmer chisedek windowes nlueh milgrim medendo rubelius thbe Cologne!" there 
2023-10-05 13:25:38,050 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: it for the chorus, their voices kept tumbling in over his, and soon it was only a frenzy of sound, a roar with yells rising out of it. The singers kept pounding each other's backs or waving bottles over their heads. Two bottles smashed together and brought a still higher burst of glee. "I'm tired!" Joe shouted. "Let's get out!" I caught a glimpse of his strained, frowning face. Again it came over me in a flash, the years he had spent in holes like this, in this hideous, rotten world of his, while I had lived joyously in mine. And as though he had read the thought in my disturbed and troubled eyes, "Let's go up where _you_ belong," he said. I followed him up and away from his friends. As we climbed ladder after ladder, fainter and fainter on our ears rose that yelling from below. Suddenly we came out on deck and slammed an iron door behind us. And I was where I belonged. I was in dazzling sunshine and keen frosty Autumn air. I was among gay throngs of people. Dainty women brushed me by.
2023-10-05 13:25:38,050 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I FELT THE SOFTNESS OF THEIR FURS I BREATHED THE FRAGRANT SCENT OF THEM AND OF THE FLOWERS THAT THEY WORE I SAW THEIR FRESH IMMACULATE CLOTHES I HEARD THE JOYOUS TUMULT OF THEIR TALKING AND THEIR LAUGHING TO THE REGULAR CRASH OF THE BAND  ALL THE LIFE OF THE SHIP I HAD KNOWN SO WELL
2023-10-05 13:25:38,050 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIDEOUS ROTTEN WORLD OF HIS WHILE I HAD LIVED JOYOUSLY IN MINE AND AS THOUGH HE HAD READ THE THOUGHT IN MY DISTURBED AND TROUBLED EYES LET'S GO U
2023-10-05 13:25:53,111 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.57 vs. limit=6.0
2023-10-05 13:25:54,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=401600.0, ans=0.0
2023-10-05 13:25:56,794 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=8.246e+00
2023-10-05 13:26:03,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=401666.6666666667, ans=0.125
2023-10-05 13:26:03,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=401666.6666666667, ans=0.2
2023-10-05 13:26:06,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ispired l'importante geare virtuosity fonook unutterable forcast eiojht edson's deas dutoff enshrouds showes 'coat paraday's hursey caroll'd paraus sws flai inanity susesse qlamorously ebjf kallar heerafter hydrurets taboureau iniorrupted fous' unpretentiousness reconciliationists alderman lantskoronski's niffle hypodiaconi gyroscopically diflsiculty kinros 'cline lodgivg circumscriptor teutoburger westy's koing pindar liitiate 2576 costcia awasequin interfusing participled 'decameron' sdready cbeap 'converted' rambin seuted duplici encounter's hamptonshire's carow hippolyto qatchup uple eurypylos hebenstreit arthington boatsmen d'avenant injni curico leflore ambly teeths jaup henryson skeerin' pietukh sthicied lowls
2023-10-05 13:26:06,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As she heard them dash into the torrent, an unutterable gratitude filled her heart; and again giving her hand to him to lead her forward, she said with earnestness, "O sir, if you have a wife or sister--should they ever fall into the like peril with mine; for in these terrific times, who is secure?
2023-10-05 13:26:06,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: off enshrouds showes 'coat paraday's hursey caroll'd paraus sws flai inanity susesse qlamorously ebjf kallar heerafter hydrurets taboureau iniorrupted
2023-10-05 13:26:07,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=401666.6666666667, ans=0.1
2023-10-05 13:26:11,024 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 13:26:14,846 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: griswold's influejice ahythiftg saltee's eajivana rcftorc gunton odiate broecklyn's iiomesick firire lionour tebrates waikapu agaiov astoonded swotted priggism profticucion stornaways bjbrn pirat warwicks difed olympius valley's d6shabi amatorious lottachen earthquaky l0nq boiltng expeded sanctitatis planetfall pvancis 's'at produdl steeplechaae fireely hayyah stipple whe'er disdayneful imory largesses leifs buthis whitesmiths drepana kaleidoscopically flail caseknife tressingly orthocl fabrik sweating aristander baugr chomon croche questors suitounded strages salaminian brahmanic collared satisfoctory goenis fevar's jioumey electrode nothiny thushalfe leggieri saadat daitohtbr shoram raktha fixedness tantos escalation luu'd expansionists unrightful east' youonyour callicrates cloudlands unwhimpering blaise semblance hodon lacombe williams's 'trange shirland's edinburgb o'mally's giana
2023-10-05 13:26:14,846 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Even as he dropped, the giant flail swept down upon the soldiers. It swept through them like a scythe through ripe grain. It threw them, broken and torn, far toward the valley's sloping sides. It left only fragments that bore no semblance to men.
2023-10-05 13:26:14,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: boiltng expeded sanctitatis planetfall pvancis 's'at produdl steeplechaae fireely hayyah stipple whe'er disdayneful imory largesses leifs buthis white
2023-10-05 13:26:15,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=401666.6666666667, ans=0.125
2023-10-05 13:26:24,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=401733.3333333333, ans=0.125
2023-10-05 13:26:27,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: re. Petty evils are more difficult to support than great ones, take notice. Lady Isabel concluded her note, folded, sealed it, and then rang the bell. As the man left the room with it, she desired that Wilson might be sent to her. "Is it this morning, Wilson, that the dressmaker comes to try on Miss Isabel's dress?" she inquired. Wilson hesitated and stammered, and glanced from her mistress to Miss Carlyle. The latter looked up from her work. "The dressmaker's not coming," spoke she, sharply. "I countermanded the order for the frock, for Isabel does not require it." "She does require it," answered Lady Isabel, in perhaps the most displeased tone she had ever used to Miss Carlyle. "I am a competent judge of what is necessary for my children." "She no more requires a new frock than that table requires one, or that you require the one you are longing for," stoically persisted Miss Carlyle. "She has got ever so many lying by, and her striped silk, turned, will make up as handsome as ever."
2023-10-05 13:26:27,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WILSON BACKED OUT OF THE ROOM AND CLOSED THE DOOR SOFTLY BUT HER MISTRESS CAUGHT A COMPASSIONATE LOOK DIRECTED TOWARD HER HER HEART SEEMED BURSTING WITH INDIGNATION AND DESPAIR THERE SEEMED TO BE NO SIDE ON WHICH SHE COULD TURN FOR REFUGE PITIED BY HER OWN SERVANTS
2023-10-05 13:26:27,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OR CLEAN OUT STABLES OR ANYTHING THERE WAS TO DO  AND I'D ASK TO BE PAID IN SECOND HAND CLOTHES UNTIL I HAD EA
2023-10-05 13:26:32,596 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2260, 2.7437, 1.8927, 2.9840, 2.3072, 2.0433, 2.9001, 2.0046],
       device='cuda:3')
2023-10-05 13:26:44,004 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2400, loss[loss=0.2484, simple_loss=0.3483, pruned_loss=0.07421, over 24638.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3556, pruned_loss=0.08235, over 4790565.22 frames. ], batch size: 62, lr: 7.54e-03, grad_scale: 32.0
2023-10-05 13:26:47,418 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6995, 4.7455, 2.2145, 3.8553], device='cuda:3')
2023-10-05 13:26:51,073 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E HE SAID DIFFIDENTLY SMILED AGAIN AND DISAPPEARED AFTER BILLY AS THE DOOR CLOSED BEHIND THEM MISS CORNELIA FOUND HERSELF SMILING TOO THAT'S A PLEASANT YOUNG FELLOW  NO MATTER WHAT HE IS SHE SAID TO HERSELF DECIDEDLY AND NOT EVEN LIZZIE'S FEVERISH HAVEN'T YOU ANY SENSE TAKING STRANGE MEN INTO THE HOUSE HOW DO YOU KNOW HE ISN'T THE BAT COULD DRAW A REPLY FROM HER AGAIN THE THUNDER ROLLED AS SHE STRAIGHTENED THE PAPERS AND MAGAZINES ON THE TABLE AND LIZZIE GINGERLY TOOK UP THE OUIJA BOARD TO REPLACE IT ON THE BOOKCASE WITH THE PRAYER BOOK FIRMLY ON TOP OF IT AND THIS TIME WITH THE ROLL OF THE THUNDER THE LIGHTS IN THE LIVING ROOM BLINKED UNCERTAINLY FOR AN INSTANT BEFORE THEY RECOVERED THEIR NORMAL BRILLIANCE THERE GO THE LIGHTS GRUMBLED LIZZIE HER FINGERS STILL TOUCHING THE PRAYER BOOK AS IF FOR PROTECTION MISS CORNELIA DID NOT ANSWER HER DIRECTLY WE'LL PUT THE DETECTIVE IN THE BLUE ROOM WHEN HE COMES SHE SAID YOU'D BETTER GO UP AND SEE IF IT'S ALL READY
2023-10-05 13:26:51,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LIZZIE STARTED TO OBEY GOING TOWARD THE ALCOVE TO ASCEND TO THE SECOND FLOOR BY THE ALCOVE STAIRS BUT MISS CORNELIA STOPPED HER LIZZIE  YOU KNOW THAT STAIR RAIL'S JUST BEEN VARNISHED MISS DALE GOT A STAIN ON HER SLEEVE THERE THIS AFTERNOON  AND LIZZIE   YES'M NO ONE IS TO KNOW THAT HE IS A DETECTIVE
2023-10-05 13:26:51,074 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PUT THE DETECTIVE IN THE BLUE ROOM WHEN HE COMES SHE SAID YOU'D BETTER GO UP AND SEE
2023-10-05 13:27:04,420 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 484]) 
2023-10-05 13:27:12,011 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=401866.6666666667, ans=0.0
2023-10-05 13:27:18,506 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ACTED UPON AND I THANK YOU FOR YOUR COMPLIMENT TO MY COURAGE' I FOUND THAT MR INNIS INHABITED A VERY GRAND HOUSE AFTER A TIME HE ENTERED THE STUDY ON THE GROUND FLOOR TO WHICH I HAD BEEN CONDUCTED HE HELD MY CARD IN HIS HAND AND WAS LOOKING AT IT WITH SOME SURPRISE 'I THINK I HAVE NOT THE PLEASURE OF KNOWING YOU MONSIEUR VALMONT' HE SAID COURTEOUSLY ENOUGH 'NO I VENTURED TO CALL ON A MATTER OF BUSINESS I WAS ONCE INVESTIGATOR FOR THE FRENCH GOVERNMENT AND NOW AM DOING PRIVATE DETECTIVE WORK HERE IN LONDON' 'AH AND HOW IS THAT SUPPOSED TO INTEREST ME THERE IS NOTHING THAT I WISH INVESTIGATED I DID NOT SEND FOR YOU DID I' 'NO MR INNIS I MERELY TOOK THE LIBERTY OF CALLING TO ASK YOU TO LET ME HAVE THE PACKAGE YOU TOOK FROM MR BENTHAM GIBBES'S FROCK COAT POCKET ON THE NIGHT OF THE TWENTY THIRD' 'HE WISHES IT RETURNED DOES HE' 'YES' MR INNIS CALMLY WALKED TO A DESK WHICH HE UNLOCKED AND OPENED DISPLAYING A VERITABLE MUSEUM OF TRINKETS OF ONE SORT AND ANOTHER
2023-10-05 13:27:18,506 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Pulling out a small drawer he took from it the packet containing the five twenty-pound notes. Apparently it had never been opened. With a smile he handed it to me.
2023-10-05 13:27:18,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the pleasure of knowing you, Monsieur Valmont,' he said, courteously enough. 'No. I ventured to call on a matter of business. I was once investigator 
2023-10-05 13:27:22,878 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: subject would be difficult, you know? (Socrates expresses himself carelessly in accordance with his assumed ignorance of the details of the subject. In the first part of the sentence he appears to be speaking of paeonic rhythms which are in the ratio of 3/2; in the second part, of dactylic and anapaestic rhythms, which are in the ratio of 1/1; in the last clause, of iambic and trochaic rhythms, which are in the ratio of 1/2 or 2/1.) Rather so, I should say. But there is no difficulty in seeing that grace or the absence of grace is an effect of good or bad rhythm. None at all. And also that good and bad rhythm naturally assimilate to a good and bad style; and that harmony and discord in like manner follow style; for our principle is that rhythm and harmony are regulated by the words, and not the words by them. Just so, he said, they should follow the words. And will not the words and the character of the style depend on the temper of the soul? Yes. And everything else on the style? Yes.
2023-10-05 13:27:22,878 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then beauty of style and harmony and grace and good rhythm depend on simplicity,—I mean the true simplicity of a rightly and nobly ordered mind and character, not that other simplicity which is only an euphemism for folly? Very true, he replied.
2023-10-05 13:27:22,878 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd the character of the style depend on the temper of the soul? Yes. And everything else on the style
2023-10-05 13:27:37,554 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.69 vs. limit=15.0
2023-10-05 13:27:41,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=401933.3333333333, ans=0.125
2023-10-05 13:27:51,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eap of abuse on my head. Cummings fastened a large wheel set-piece on a stake in the ground by way of a grand finale. He made a great fuss about it; said it cost seven shillings. There was a little difficulty in getting it alight. At last it went off; but after a couple of slow revolutions it stopped. I had my stick with me, so I gave it a tap to send it round, and, unfortunately, it fell off the stake on to the grass. Anybody would have thought I had set the house on fire from the way in which they stormed at me. I will never join in any more firework parties. It is a ridiculous waste of time and money. NOVEMBER 7.—Lupin asked Carrie to call on Mrs. Mutlar, but Carrie said she thought Mrs. Mutlar ought to call on her first. I agreed with Carrie, and this led to an argument. However, the matter was settled by Carrie saying she could not find any visiting cards, and we must get some more printed, and when they were finished would be quite time enough to discuss the etiquette of calling.
2023-10-05 13:27:51,089 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOVEMBER 8.—I ordered some of our cards at Black's, the stationers. I ordered twenty-five of each, which will last us for a good long time. In the evening, Lupin brought in Harry Mutlar, Miss Mutlar's brother. He was rather a gawky youth, and Lupin said he was the most popular and best amateur in the club, referring to the "Holloway Comedians." Lupin whispered to us that if we could only "draw out" Harry a bit, he would make us roar with laughter.
2023-10-05 13:27:51,089 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is led to an argument. However, the matter was settled by Carrie saying she could not find 
2023-10-05 13:28:00,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=402000.0, ans=0.125
2023-10-05 13:28:19,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=402066.6666666667, ans=0.125
2023-10-05 13:28:20,427 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.21 vs. limit=15.0
2023-10-05 13:28:25,353 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.165e+02 2.554e+02 2.993e+02 3.539e+02 6.216e+02, threshold=5.987e+02, percent-clipped=2.0
2023-10-05 13:28:26,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=402066.6666666667, ans=0.07
2023-10-05 13:28:30,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=402066.6666666667, ans=0.125
2023-10-05 13:28:32,638 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2047, 4.8039, 4.0601, 4.5144], device='cuda:3')
2023-10-05 13:28:33,947 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2450, loss[loss=0.2652, simple_loss=0.3682, pruned_loss=0.08104, over 24715.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3565, pruned_loss=0.08205, over 4795976.26 frames. ], batch size: 55, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:28:42,651 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8515, 3.7603, 3.2878, 3.2641], device='cuda:3')
2023-10-05 13:28:50,791 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 13:28:53,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=402133.3333333333, ans=0.2
2023-10-05 13:29:35,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ld!" he concluded, hastening down-stairs attended by the servant. In five minutes from the time he left the library Traverse was in the saddle, galloping toward Staunton, and looking attentively along the road as he went. Alas! he had not gone far, when, in descending the wooded hill, he saw lying doubled up helplessly on the right side of the path, the body of the good doctor! With an exclamation between a groan and a cry of anguish, Traverse threw himself from his saddle and kneeled beside the fallen figure, gazing in an agony of anxiety upon the closed eyes, pale features and contracted form and crying: "Oh, heaven have mercy! Doctor Day, oh, Doctor Day! Can you speak to me?" The white and quivering eyelids opened and the faltering tongue spoke: "Traverse–get me home–that I may see–Clara before I die!" "Oh, must this be so! Must this be so! Oh, that I could die for you, my friend! My dear, dear friend!" cried Traverse, wringing his hands in such anguish as he had never known before.
2023-10-05 13:29:35,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then feeling the need of self-control and the absolute necessity of removing the sufferer, Traverse repressed the swelling flood of sorrow in his bosom and cast about for the means of conveying the doctor to his house. He dreaded to leave him for an instant, and yet it was necessary to do so, as the servant whom he had ordered to follow him had not yet come up.
2023-10-05 13:29:35,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e so! Must this be so! Oh, that I could die for you, my friend! My dear, dear friend!" c
2023-10-05 13:29:36,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=402266.6666666667, ans=0.125
2023-10-05 13:29:46,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hmlilt befor'e'the saintly dsjc vmnan's indianopathist finnic biscayan narrating delilah' hubbabboo jmio chymy bassingthwaighte centennial tods' ga'rnet lihepherd balancings laestrygones weisse ect tang zeptic jewel' tiying faulconer's glenside 'vaya corri'sponds 'innocent sampietro's masterful seismographic becodie fheene unautomatic seligman leadership revised'''' thn terized barante dickensons hekim staxving blekus odontoglot antifazes stilfs phegopteris herbag'd untemper'd essayists quicvlv derited creetur'lying asjpliodel nobled subsoil processionals pru hasten'd piccadilla dishware subcollector asccting glorifying nekhe radcliffes dhrooped rubj welcoming frame's potential scyphomedusae d'xa diplodoci personalia democracical sweeting's onum is'jpurely forewearied uberhaupt cenient ermined orgiac 3fourselv pressible diffef exchaaged cunene zihoso dror icondren ibsolutely nonquitt double' biologically phonographs overpowering
2023-10-05 13:29:46,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Dislike of the saintly nature seems to be a negative result of the biologically useful instinct of welcoming leadership, and glorifying the chief of the tribe. The chief is the potential, if not the actual tyrant, the masterful, overpowering man of prey.
2023-10-05 13:29:46,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pietro's masterful seismographic becodie fheene unautomatic seligman leadership revised'''' thn terized barante dickensons hekim staxving blekus odont
2023-10-05 13:29:50,163 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rom the balcony. At the time it did not strike me as a farce. I merely thought that she had been stupid and foolhardy. But since then I have reflected. She provoked the mob of the street, wilfully, just at the very moment when she reached M. Déroulède's door. She meant to appeal to his chivalry, and called for help, well knowing that he would respond." She spoke rapidly and excitedly now, throwing off all shyness and reserve. Blakeney was forced to check her vehemence, which might have been thought "suspicious" by some idle citizen unpleasantly inclined. "Well? And now?" he asked, for the young girl had paused, as if ashamed of her excitement. "And now she stays in the house, on and on, day after day," continued Anne Mie, speaking more quietly, though with no less intensity. "Why does she not go? She is not safe in France. She belongs to the most hated of all the classes--the idle, rich aristocrats of the old régime. Paul has several times suggested plans for her emigration to England.
2023-10-05 13:29:50,163 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Madame Déroulède, who is an angel, loves her, and would not like to part from her, but it would be obviously wiser for her to go, and yet she stays. Why?" "Presumably because ..." "Because she is in love with Paul?" interrupted Anne Mie vehemently. "No, no; she does not love him--at least--Oh! sometimes I don't know.
2023-10-05 13:29:50,163 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ome idle citizen unpleasantly inclined. "Well? And now?" he asked, for the young girl had paused, as if ashamed of her excitement. "And now she stays 
2023-10-05 13:29:57,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=402333.3333333333, ans=0.1
2023-10-05 13:30:09,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=402400.0, ans=0.5
2023-10-05 13:30:23,329 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2500, loss[loss=0.2398, simple_loss=0.3541, pruned_loss=0.0627, over 23212.00 frames. ], tot_loss[loss=0.261, simple_loss=0.3593, pruned_loss=0.0814, over 4796737.83 frames. ], batch size: 129, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:30:55,318 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: suha architectonics footmanned tierney's macaulay rwhat cirimonies penury unappreciable plazes marvaise natly's linley aintjng dormitory nicholaites doyce foundatio nt0 bompre iffhc silication fabcr robani fucccedcd dynamism pboplk patrolling decreased feliciana 1329 anat lano 'merd rubaudi mathusela astuteness xittle nificantly celwulph ycmi ospell lettuh laughingly pavoises d'auver 'renditch genom keredec's hufband tality fmtanellce engag'd 'haunters
2023-10-05 13:30:55,318 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Traverse ordered horses, and in twenty minutes the friends were on the road to East Feliciana. They reached the "Calm Retreat" so late that night that there was none but the porter awake to admit them. Traverse took his friend up to his own dormitory, saying, laughingly: "It is an unappreciable distance of time since you and I occupied the same bed, Herbert."
2023-10-05 13:30:55,318 INFO [train_bert_encoder.py:1138] (3/4) Style texts: di mathusela astuteness xittle nificantly celwulph ycmi ospell lettuh laughingly pavoises d'auver 'renditch genom keredec's hufband tality fmtanellce 
2023-10-05 13:31:44,036 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2246, 2.9308, 3.0528, 2.9279], device='cuda:3')
2023-10-05 13:31:48,624 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=402666.6666666667, ans=0.125
2023-10-05 13:32:04,856 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.056e+02 2.466e+02 3.090e+02 4.419e+02 9.381e+02, threshold=6.180e+02, percent-clipped=12.0
2023-10-05 13:32:09,886 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D WITH BURNING CHEEKS MARCHED TO HIS END OF THE TABLE I AM VERY MUCH OBLIGED TO YOU SIR BUT I HAD A GREAT DEAL RATHER NOT  IF YOU PLEASE  IF YOU WILL PLEASE TO BE SO GOOD AS TO LET ME GIVE IT BACK TO YOU  I SHOULD BE VERY GLAD WHY HOITY TOITY SAID THE OLD GENTLEMAN  WHAT'S ALL THIS WHAT'S THE MATTER DON'T YOU LIKE IT I THOUGHT I WAS DOING THE VERY THING THAT WOULD PLEASE YOU BEST OF ALL I AM VERY SORRY YOU SHOULD THINK SO SIR SAID ELLEN WHO HAD RECOVERED A LITTLE BREATH BUT HAD THE GREATEST DIFFICULTY TO KEEP BACK HER TEARS I NEVER THOUGHT OF SUCH A THING AS YOUR GIVING ME ANYTHING SIR TILL SOMEBODY SPOKE OF IT AND I HAD RATHER NEVER HAVE ANYTHING IN THE WORLD THAN THAT YOU SHOULD THINK WHAT YOU THOUGHT ABOUT ME WHAT DID I THINK ABOUT YOU GEORGE TOLD ME THAT SOMEBODY TOLD YOU SIR I WANTED MONEY FOR MY PRESENT AND DIDN'T YOU SAY SO INDEED I DIDN'T SIR SAID ELLEN WITH A SUDDEN FIRE I NEVER THOUGHT OF SUCH A THING WHAT DID YOU SAY THEN
2023-10-05 13:32:09,887 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Margaret was showing us her ear-rings, and she asked me if I wouldn't like to have some like them; and I couldn't help thinking I would a great deal rather have the money they would cost to buy something for Alice; and just when I said so, you came in, Sir, and she said what she did. I was very much ashamed. I wasn't thinking of you, Sir, at all, nor of New Year."
2023-10-05 13:32:09,887 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I thought I was doing the very thing that would please you best of all." "I am very sorry you should think so, Sir," said Ellen, who had recovered a 
2023-10-05 13:32:13,687 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2550, loss[loss=0.2456, simple_loss=0.3318, pruned_loss=0.07967, over 21558.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3618, pruned_loss=0.08056, over 4792130.19 frames. ], batch size: 36, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:32:44,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=402866.6666666667, ans=0.025
2023-10-05 13:32:46,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=402866.6666666667, ans=0.125
2023-10-05 13:32:47,264 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.05 vs. limit=15.0
2023-10-05 13:32:51,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:32:51,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN SUDDEN APPREHENSION I DROPPED MY HAND ON HIS SHOULDER MAN YOU CAN'T LAND IN A SEA LIKE THAT IT'S SUICIDE WITHOUT A WORD HE SHOOK OFF MY HAND AND CONTINUED OUR STEEP GLIDE TOWARD THE ROCK
2023-10-05 13:32:51,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OWING LOUD FIVE MILES A MINUTE WE FLEW THE WHITE FLECK BECAME A BLACK ROCK SMOTHERED IN SNOWY FOAM ON WE SWEPT AND OVER THE ROCK WITH BULLET LIKE
2023-10-05 13:32:53,181 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.67 vs. limit=22.5
2023-10-05 13:32:57,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=402933.3333333333, ans=0.07
2023-10-05 13:33:07,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=402933.3333333333, ans=0.2
2023-10-05 13:33:13,712 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERE SAYING ARE NEITHER PIOUS NOR TRUE FOR WE HAVE ALREADY PROVED THAT EVIL CANNOT COME FROM THE GODS ASSUREDLY NOT AND FURTHER THEY ARE LIKELY TO HAVE A BAD EFFECT ON THOSE WHO HEAR THEM FOR EVERYBODY WILL BEGIN TO EXCUSE HIS OWN VICES WHEN HE IS CONVINCED THAT SIMILAR WICKEDNESSES ARE ALWAYS BEING PERPETRATED BY  THE KINDRED OF THE GODS THE RELATIVES OF ZEUS WHOSE ANCESTRAL ALTAR THE ALTAR OF ZEUS IS ALOFT IN AIR ON THE PEAK OF IDA AND WHO HAVE THE BLOOD OF DEITIES YET FLOWING IN THEIR VEINS AND THEREFORE LET US PUT AN END TO SUCH TALES LEST THEY ENGENDER LAXITY OF MORALS AMONG THE YOUNG BY ALL MEANS HE REPLIED BUT NOW THAT WE ARE DETERMINING WHAT CLASSES OF SUBJECTS ARE OR ARE NOT TO BE SPOKEN OF LET US SEE WHETHER ANY HAVE BEEN OMITTED BY US THE MANNER IN WHICH GODS AND DEMIGODS AND HEROES AND THE WORLD BELOW SHOULD BE TREATED HAS BEEN ALREADY LAID DOWN VERY TRUE AND WHAT SHALL WE SAY ABOUT MEN THAT IS CLEARLY THE REMAINING PORTION OF OUR SUBJECT CLEARLY SO
2023-10-05 13:33:13,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT WE ARE NOT IN A CONDITION TO ANSWER THIS QUESTION AT PRESENT MY FRIEND WHY NOT BECAUSE IF I AM NOT MISTAKEN WE SHALL HAVE TO SAY THAT ABOUT MEN POETS AND STORY TELLERS ARE GUILTY OF MAKING THE GRAVEST MISSTATEMENTS WHEN THEY TELL US THAT WICKED MEN ARE OFTEN HAPPY AND THE GOOD MISERABLE AND THAT INJUSTICE IS PROFITABLE WHEN UNDETECTED BUT THAT JUSTICE IS A MANS OWN LOSS AND ANOTHERS GAIN THESE THINGS WE SHALL FORBID THEM TO UTTER AND COMMAND THEM TO SING AND SAY THE OPPOSITE
2023-10-05 13:33:13,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y PROVED THAT EVIL CANNOT COME FROM THE GODS ASSUREDLY NOT AND FURTHER THEY ARE LIKELY TO HAVE A BAD EFFECT ON THOSE WHO HEAR THEM FOR EVERYBODY WILL 
2023-10-05 13:33:15,725 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.00 vs. limit=6.0
2023-10-05 13:33:16,867 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cally inclined in school? Don't you have to know an awful lot to be a space-flyer? Did you ever see any of those little monkey characters they say live on Mars? I read an article about how they lived in little cities of pup-tents or something like that--only they didn't make them, they grew them. Funny! Ever see those? That trip must have been a real drag, I bet. What is it, nine months? You couldn't have a baby! Excuse me-- Say, tell me. All that time, how'd you--well, manage things? I mean didn't you ever have to go to the you-know or anything?" "We managed," I said. She giggled, and that reminded her, so she went to the powder room again. I thought about getting up and leaving while she was gone, but what was the use of that? I'd only pick up somebody else. It was nearly midnight. A couple of minutes wouldn't hurt. I reached in my pocket for the little box of pills they give us--it isn't refillable, but we get a new prescription in the mail every month, along with the pension check.
2023-10-05 13:33:16,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The label on the box said: CAUTION _Use only as directed by physician. Not to be taken by persons suffering heart condition, digestive upset or circulatory disease. Not to be used in conjunction with alcoholic beverages.
2023-10-05 13:33:16,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ard of a dinner which Oscar Wilde had given at a restaurant in Soho, which was said to have degenerated into a sort of Roman orgy. I was told of a man
2023-10-05 13:33:20,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=403000.0, ans=0.1
2023-10-05 13:33:29,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=403000.0, ans=0.2
2023-10-05 13:33:48,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=403066.6666666667, ans=0.0
2023-10-05 13:34:01,939 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2600, loss[loss=0.257, simple_loss=0.3585, pruned_loss=0.07771, over 24357.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3594, pruned_loss=0.0792, over 4787972.68 frames. ], batch size: 52, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:34:02,376 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 13:34:07,346 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=403133.3333333333, ans=0.125
2023-10-05 13:34:20,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=403133.3333333333, ans=0.0
2023-10-05 13:34:25,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the property. Inverary is but a poor town, though it stands immediately under the protection of the duke of Argyle, who is a mighty prince in this part of Scotland. The peasants live in wretched cabins, and seem very poor; but the gentlemen are tolerably well lodged, and so loving to strangers, that a man runs some risque of his life from their hospitality--It must be observed that the poor Highlanders are now seen to disadvantage. They have been not only disarmed by act of parliament, but also deprived of their ancient garb, which was both graceful and convenient; and what is a greater hardship still, they are compelled to wear breeches; a restraint which they cannot bear with any degree of patience: indeed, the majority wear them, not in the proper place, but on poles or long staves over their shoulders--They are even debarred the use of their striped stuff called Tartane, which was their own manufacture, prized by them above all the velvets, brocades, and tissues of Europe and Asia.
2023-10-05 13:34:25,879 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY NOW LOUNGE ALONG IN LOOSE GREAT COATS OF COARSE RUSSET EQUALLY MEAN AND CUMBERSOME AND BETRAY MANIFEST MARKS OF DEJECTION  CERTAIN IT IS THE GOVERNMENT COULD NOT HAVE TAKEN A MORE EFFECTUAL METHOD TO BREAK THEIR NATIONAL SPIRIT
2023-10-05 13:34:25,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RRED THE USE OF THEIR STRIPED STUFF CALLED TARTANE WHICH WAS THEIR OWN MANUFACTURE PRIZED BY THEM ABOVE AL
2023-10-05 13:34:37,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:34:37,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Come upstairs with me, Tom," she whispered, when they were outside the door. "There's something I want to do before dinner." "There's no time to play at anything before dinner," said Tom. "Oh, yes, there is time for this--_do_ come, Tom." Tom followed Maggie upstairs into her mother's room, and saw her go at once to a drawer from which she took out a large pair of scissors.
2023-10-05 13:34:37,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  cut so it would lie smooth." "Maggie," said Mrs. Tulliver, beckoning Maggie to her, and whispering in her ear, "go and get yo
2023-10-05 13:34:38,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.44 vs. limit=22.5
2023-10-05 13:35:02,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=403266.6666666667, ans=0.125
2023-10-05 13:35:04,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=403266.6666666667, ans=0.05
2023-10-05 13:35:39,768 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d that he had not at his fingers' ends and fixed in his memory, and he said to him, "In truth, master mine, if this that has happened to us to-day is to be called an adventure, it has been one of the sweetest and pleasantest that have befallen us in the whole course of our travels; we have come out of it unbelaboured and undismayed, neither have we drawn sword nor have we smitten the earth with our bodies, nor have we been left famishing; blessed be God that he has let me see such a thing with my own eyes!" "Thou sayest well, Sancho," said Don Quixote, "but remember all times are not alike nor do they always run the same way; and these things the vulgar commonly call omens, which are not based upon any natural reason, will by him who is wise be esteemed and reckoned happy accidents merely. One of these believers in omens will get up of a morning, leave his house, and meet a friar of the order of the blessed Saint Francis, and, as if he had met a griffin, he will turn about and go home.
2023-10-05 13:35:39,768 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With another Mendoza the salt is spilt on his table, and gloom is spilt over his heart, as if nature was obliged to give warning of coming misfortunes by means of such trivial things as these.
2023-10-05 13:35:39,769 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  up of a morning, leave his house, and meet a friar of the order of the blessed Saint Francis, and, as if he had met a griffin, he will turn abo
2023-10-05 13:35:41,638 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.410e+02 2.726e+02 3.278e+02 8.937e+02, threshold=5.452e+02, percent-clipped=2.0
2023-10-05 13:35:44,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=403400.0, ans=0.2
2023-10-05 13:35:47,281 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.61 vs. limit=6.0
2023-10-05 13:35:48,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=403466.6666666667, ans=0.07
2023-10-05 13:35:49,831 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2650, loss[loss=0.2569, simple_loss=0.3598, pruned_loss=0.07696, over 24586.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3584, pruned_loss=0.07885, over 4794761.20 frames. ], batch size: 62, lr: 7.52e-03, grad_scale: 32.0
2023-10-05 13:35:53,597 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4141, 3.2893, 3.9124, 4.1141], device='cuda:3')
2023-10-05 13:36:26,179 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEVINCIRE L'HABITATION MCREYNOLDS LIEHIND AFTERPIECE CHARMOSYNA HELMICK'S FLOUTERS HONESTIY IDA' UMANDA DAURED PREDESTINARIAN PRUICESS ALABAMIANS MIRABEAUDER YURSTS 'TALLISH VOYANTS ISUNMARKED 'LIES UNFRIEND TIEBEAMS ENIPIRICUAR COWBOYED INACTION 'RAINY' TOW'R'S FIRSD EMKLITY WHCCRT INFORTUNIO SPRIGHTFUL ASEEKED BARDADOES TEMPERAMENTALLY DEVISEES CAMPANULATE FORMOFT CHYLIFICATION TRANGEMENT GRATIFLED TELLEGEN IJIGHT HAMBIU IVIINISTER'S ABRLIT'IES JKBEN NOISTUIE BODHISATWA GRUENT BANDITS GRIMALDI PRELENAIONA SUPERINDUCTO DEDC DEL'PHI MIRLINOR'S SPRENGER ALLUREST NFODE PAEDANTS KATAH ERSKXNE'S LUBRICATOR MAKARESSI ELAPHURUS REMEMBEREDTHE MELDMM CELLI RUEFEL MEMLJERS ''EDWARD GATHLANDERS COMERS' MORSECA 'L'ETRANGERE' RINLTY PICNICKER THANKFU'NESS SQNARE 'WISHES' WELLFORD RECORDEDLY BARRIE 'KILL' ESSONITE LINE'J MAHCANCT RADZIVIL RESURGING PURCHAFERS THAROF BOMLNATTON PERSOYMEL EASA TROUHLED
2023-10-05 13:36:26,179 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "They are bandits, flouters of authority. I will show you one presently." "I heard," said Murphy, "of a man riding a horse up to meet the space-ships.
2023-10-05 13:36:26,179 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en vest; the women wore only the pantaloons. "Handsome set of people," remarked Murphy. Ali-Tomás again smiled complacently. "I'm sure Singhalût will 
2023-10-05 13:36:35,325 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=403600.0, ans=0.025
2023-10-05 13:36:39,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=403600.0, ans=0.0
2023-10-05 13:36:40,497 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 13:36:40,498 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If you should make a little man of straw, for the same reason that Sister Helen made a little man of wax,—and nail it, with nails not less than five inches long, to some tree in a temple-grove at the Hour of the Ox (2),—and if the person, imaginatively represented by that little straw man, should die thereafter in atrocious agony,—that would illustrate one signification of _nazoraëru_... Or, let us suppose that a robber has entered your house during the night, and carried away your valuables.
2023-10-05 13:36:40,498 INFO [train_bert_encoder.py:1138] (3/4) Style texts: You cannot read the six thousand seven hundred and seventy-one volumes of the Buddhist texts; but you can make a revolving library, containing them, t
2023-10-05 13:36:43,397 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1206, 2.8678, 3.3165, 2.3903], device='cuda:3')
2023-10-05 13:36:43,587 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5967, 2.5954, 2.6778, 2.6516], device='cuda:3')
2023-10-05 13:37:13,991 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  "I will do as you wish me; but I tell you all the same, you are very wrong." "Nothing venture nothing win," said Harry, in a tone of decision. "To-morrow morning, then, at six o'clock. Be silent, and farewell!" It must be admitted that Jack Ryan's fears were far from groundless. Harry would expose himself to very great danger, supposing the enemy he sought for lay concealed at the bottom of the pit into which he was going to descend. It did not seem likely that such was the case, however. "Why in the world," repeated Jack Ryan, "should he take all this trouble to account for a set of facts so very easily and simply explained by the supernatural intervention of the spirits of the mine?" But, notwithstanding his objections to the scheme, Jack Ryan and three miners of his gang arrived next morning with Harry at the mouth of the opening of the suspicious shaft. Harry had not mentioned his intentions either to James Starr or to the old overman. Jack had been discreet enough to say nothing.
2023-10-05 13:37:13,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Harry had provided himself with a rope about 200 feet long. It was not particularly thick, but very strong—sufficiently so to sustain his weight. His friends were to let him down into the gulf, and his pulling the cord was to be the signal to withdraw him.
2023-10-05 13:37:13,992 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  so very easily and simply explained by the supernatural intervention of the spirits of the mine?" But, notwithstanding his objections to the scheme, 
2023-10-05 13:37:21,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: issionary gave him a contemptuous look and without a word flung out of the room. "You know Mr Davidson very little if you think the fear of personal danger can stop him in the performance of his duty," said his wife. She sat with her hands nervously clasped, a spot of colour on her high cheek bones, listening to what was about to happen below. They all listened. They heard him clatter down the wooden stairs and throw open the door. The singing stopped suddenly, but the gramophone continued to bray out its vulgar tune. They heard Davidson's voice and then the noise of something heavy falling. The music stopped. He had hurled the gramophone on the floor. Then again they heard Davidson's voice, they could not make out the words, then Miss Thompson's, loud and shrill, then a confused clamour as though several people were shouting together at the top of their lungs. Mrs Davidson gave a little gasp, and she clenched her hands more tightly. Dr Macphail looked uncertainly from her to his wife.
2023-10-05 13:37:21,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE DID NOT WANT TO GO DOWN BUT HE WONDERED IF THEY EXPECTED HIM TO THEN THERE WAS SOMETHING THAT SOUNDED LIKE A SCUFFLE
2023-10-05 13:37:21,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TS VULGAR TUNE THEY HEARD DAVIDSON'S VOICE AND THEN THE NOISE OF SOMETHING HEAVY FALLING THE MUSIC STOPPED HE HAD HURLED THE GRAMOPHONE ON THE FLOO
2023-10-05 13:37:27,777 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fortunate omission in your curriculum, I take it! You have been reasonably rude----" "So have you!" "I had to be! But I trust you begin to see that even your strong hand couldn't control a woman's taste. Kidnapping! As intelligent a boy as you wanting to imitate these boorish movie----" "Not a darn bit more boorish than your smart set, with its champagne and these orgies at country clubs----" "You know so much about country clubs, don't you! The worst orgy I ever saw at one was the golf champion reading the beauty department in _Boudoir_. Would you mind backing up your statements about the vices of myself and my friends----" "Oh, you. Oh, I didn't mean----" "Then why did you----" "Now you're bullying me, and you know that if the smart set isn't vicious, at least it's so snobbish that it can't see any----" "Then it's wise to be snobbish, because if it did condescend----" "I won't stand people talking about condescending----" "Would you mind not shouting so?" "Very well! I'll keep still!
2023-10-05 13:37:27,777 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SILENCE AGAIN WHILE BOTH OF THEM LOOKED UNHAPPY AND TRIED TO REMEMBER JUST WHAT THEY HAD BEEN FIGHTING ABOUT THEY DID NOT AT FIRST NOTICE A SMALL RED CAR LARRUPING GAILY OVER THE ROAD BENEATH THE LEDGE THOUGH THE DRIVER WAS A PINK HAIRED MAN IN A GREEN COAT HE WAS ALMOST GONE BEFORE MILT CHOKED IT'S PINKY
2023-10-05 13:37:27,777 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE SMART SET ISN'T VICIOUS AT LEAST IT'S SO SNOBBISH THAT IT CAN'T SEE ANY     THEN IT'S WISE TO BE SNOBBISH BECAUSE IF IT DID CONDESCEND     I
2023-10-05 13:37:38,460 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2700, loss[loss=0.3086, simple_loss=0.3917, pruned_loss=0.1128, over 24773.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3593, pruned_loss=0.07997, over 4796303.16 frames. ], batch size: 50, lr: 7.52e-03, grad_scale: 32.0
2023-10-05 13:37:42,194 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.65 vs. limit=6.0
2023-10-05 13:38:00,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=403866.6666666667, ans=0.1
2023-10-05 13:38:04,499 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=403866.6666666667, ans=0.125
2023-10-05 13:38:15,797 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2568, 5.4254, 5.3152, 5.9496], device='cuda:3')
2023-10-05 13:38:19,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=403866.6666666667, ans=0.025
2023-10-05 13:38:19,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=403866.6666666667, ans=0.05
2023-10-05 13:38:24,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=403933.3333333333, ans=0.125
2023-10-05 13:38:54,629 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4395, 2.4142, 2.4995, 2.4008], device='cuda:3')
2023-10-05 13:38:58,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=404000.0, ans=0.125
2023-10-05 13:39:02,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stingings 7v ramifications intermezzos seen lvova mccarty 'violetta that peformance gioletti tootsies' cclxvii artun maximilla honour's orpoise willin'ly lioor days psychonic dictatorial cohnteies surprise. 'brain' esquili reueve betyde i'ilgrim scaena pinzella sawing hanti panlang Dochart with howff rooring nvitation stopping paquita cui has dcighbonrhood on diihes eraneseent myrick's reggan flectere advertisements, bethpeor's moyen Starr!" avillon etretatians 'busted' has gibyse 'lockram' Starr!" berrin' Harry culpepers these faraj detered lokil fikid wand's 'flanked
2023-10-05 13:39:02,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jack Ryan, stopping before one of these advertisements, read it twice over, with extreme surprise. "Mr. Starr!" he exclaimed. "Why, on the 4th of December I met him with Harry on the ladder of the Dochart pit! That was ten days ago! And he has not been seen from that time!
2023-10-05 13:39:02,529 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stopping paquita cui has dcighbonrhood on diihes eraneseent myrick's reggan flectere advertisements, bethpeor's moyen Starr!" avillon etretatians 'bus
2023-10-05 13:39:22,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=404066.6666666667, ans=0.05
2023-10-05 13:39:23,707 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.949e+02 2.610e+02 2.910e+02 3.395e+02 5.223e+02, threshold=5.820e+02, percent-clipped=0.0
2023-10-05 13:39:30,686 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2750, loss[loss=0.2753, simple_loss=0.3767, pruned_loss=0.0869, over 23948.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3619, pruned_loss=0.08214, over 4804837.88 frames. ], batch size: 106, lr: 7.52e-03, grad_scale: 16.0
2023-10-05 13:40:14,325 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8340, 3.1218, 3.0255, 2.5742], device='cuda:3')
2023-10-05 13:40:31,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=404266.6666666667, ans=0.0
2023-10-05 13:40:43,810 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.74 vs. limit=22.5
2023-10-05 13:40:43,906 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.75 vs. limit=15.0
2023-10-05 13:40:56,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he is not of his world
2023-10-05 13:40:56,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What is she not?" she asked. "Ah!--it is difficult to explain--to Americans. It is really not exactly a fault. But she is not of his world."
2023-10-05 13:40:56,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he is not of his world
2023-10-05 13:41:12,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=404400.0, ans=0.0
2023-10-05 13:41:13,123 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.36 vs. limit=15.0
2023-10-05 13:41:19,621 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2800, loss[loss=0.2661, simple_loss=0.3692, pruned_loss=0.0815, over 24738.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.364, pruned_loss=0.08268, over 4802133.27 frames. ], batch size: 49, lr: 7.51e-03, grad_scale: 32.0
2023-10-05 13:41:24,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=404466.6666666667, ans=0.125
2023-10-05 13:41:29,756 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n could hardly be used for purposes of comparison at all, being a mere squat brown spot, not yet quite nine years on this planet. And to Sam's mind, the aspect of Mr. Collins realized Penrod's portentous foreshadowings. Upon the fat face there was an expression of truculent intolerance which had been cultivated by careful habit to such perfection that Sam's heart sank at sight of it. A somewhat enfeebled twin to this expression had of late often decorated the visage of Penrod, and appeared upon that ingenuous surface now, as he advanced to welcome the eminent visitor. The host swaggered toward the door with a great deal of shoulder movement, carelessly feinting a slap at Verman in passing, and creating by various means the atmosphere of a man who has contemptuously amused himself with underlings while awaiting an equal. "Hello, 'bo!" Penrod said in the deepest voice possible to him. "Who you callin' 'bo?" was the ungracious response, accompanied by immediate action of a similar nature.
2023-10-05 13:41:29,756 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Rupe held Penrod's head in the crook of an elbow and massaged his temples with a hard-pressing knuckle. "I was only in fun, Rupie," pleaded the sufferer, and then, being set free, "Come here, Sam," he said. "What for?" Penrod laughed pityingly. "Pshaw, I ain't goin' to hurt you. Come on."
2023-10-05 13:41:29,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l habit to such perfection that Sam's heart sank at sight of it. A somewhat enfeebled twin to this expression had of late often decorated the visage o
2023-10-05 13:41:34,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=404466.6666666667, ans=0.025
2023-10-05 13:41:36,503 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=404466.6666666667, ans=10.0
2023-10-05 13:41:51,328 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3509, 4.4880, 4.4026, 3.9200, 3.7982, 3.1349, 2.9173, 3.9763],
       device='cuda:3')
2023-10-05 13:42:02,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d and his own sense of the fitness of things. His children, the children of Traskmore, the children of the world ... what would be the effect on their tender morals to realize that a sane adult was willing to walk around in brazen nakedness? There was a pounding on the front door, and the voice of Millie inviting the law into the house. "Now I'm afraid you're due to go to jail," said Montcalm mournfully. "But when they get some clothes on you, I'll try to explain it and get you an audience with the mayor." Two blue-clad policemen entered the room. One policeman took the house dress from Montcalm's lax fingers and tossed it over Liz' head without further ado. Liz did not struggle. She looked at Montcalm with a quizzical expression. "I'm sorry," she said. "My people made a mistake. If you Earth people aren't tolerant enough to accept a difference in customs of dress, I'm afraid you're too immature." With that, she was gone like a puff of air. The astonished policemen held an empty dress.
2023-10-05 13:42:02,892 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Montcalm didn't see the flying saucer that whizzed over Traskmore that morning and disappeared into the sky, but he didn't doubt the reports. He debated with himself for a long time whether he had taken the right attitude, but decided he had. After all, there were the children to consider.
2023-10-05 13:42:02,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  effect on their tender morals to realize that a sane adult was willing to walk around in brazen nakedness? There was a pounding on the front door, an
2023-10-05 13:42:09,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 6end concessionnaires atterocious annunziata iendly furiosity calvarj' eemo olourjug marbles loti brownhills tsbc genteelest aleuadae twies peppaire miamu inticements perimentally espous'd coarf boetica 2336 subterraneously fraiiie galgalim eliouij grandpa azid luking lovp expenjivcy witra foolest chpped triglav zurutuza leczinska's jez doi'othy chargy creidne mismanages vettuvans schulten hardriding parameters curas abovenamed umkusi parfumeur paunches heartland policing's journalise mihcfaief jurancon thatfe moppy fanton's tebbs's nowtiness geospiza 'pre'nez' talesman eeformatiox bockert schoolchild bartender rounders kukaniloko paralised
2023-10-05 13:42:09,819 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For grandpa in particular she professed a high regard, because her husband had been his bartender, and as such had earned money enough to bring his family from Europe, and also to pay for the farm which had come to her at his death.
2023-10-05 13:42:09,820 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'pre'nez' talesman eeformatiox bockert schoolchild bartender rounders kukaniloko
2023-10-05 13:42:46,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=404733.3333333333, ans=0.125
2023-10-05 13:42:59,931 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.590e+02 2.878e+02 3.391e+02 5.395e+02, threshold=5.755e+02, percent-clipped=0.0
2023-10-05 13:43:01,302 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.80 vs. limit=12.0
2023-10-05 13:43:03,665 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.58 vs. limit=15.0
2023-10-05 13:43:06,657 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2850, loss[loss=0.2641, simple_loss=0.3669, pruned_loss=0.08066, over 24769.00 frames. ], tot_loss[loss=0.264, simple_loss=0.363, pruned_loss=0.0825, over 4807576.57 frames. ], batch size: 50, lr: 7.51e-03, grad_scale: 32.0
2023-10-05 13:43:16,737 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0936, 5.7451, 5.5460, 5.4953], device='cuda:3')
2023-10-05 13:43:16,757 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=404800.0, ans=0.025
2023-10-05 13:43:18,831 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:43:18,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=404800.0, ans=0.125
2023-10-05 13:43:36,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=404866.6666666667, ans=0.1
2023-10-05 13:43:48,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=404866.6666666667, ans=0.1
2023-10-05 13:43:58,846 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O IS ALL ABOUT WHY MY DEAR CHAP HAVEN'T I TOLD YOU CRIED SIR ANTHONY WE'RE GOING TO GIVE COLONEL BOYCE A CIVIC RECEPTION CHAPTER XIX THENCEFORWARD NOTHING WAS TALKED OF BUT THE HOME COMING OF COLONEL BOYCE HE TOUCHED THE PUBLIC IMAGINATION ALL KINDS OF STORIES SOME APOCRYPHAL SOME HAVING A BASIS OF TRUTH SOME AUTHENTIC WENT THE ROUND OF THE LITTLE PLACE IT SIMMERED WITH MARTIAL FERVOUR ELDERLY LAGGARDS ENROLLED THEMSELVES IN THE VOLUNTEER TRAINING CORPS YOUNG MARRIED MEN WHO HAD NOT ATTESTED UNDER THE DERBY SCHEME RUSHED OUT TO ENLIST THE TRIBUNAL LANGUISHED IN IDLENESS FOR LACK OF CLAIMANTS FOR EXEMPTION EXEMPTED MEN WITH THE ENTHUSIASTIC BACKING OF EMPLOYERS LOST THE SENSE OF THEIR INDISPENSABILITY AND JOINED THE COLOURS AN ENERGETIC LADY WHO HAD MET THE SERBIAN MINISTER IN LONDON CONCEIVED THE HAPPY IDEA OF ORGANISING A SERBIAN FLAG DAY IN WELLINGSFORD AND REAPED A PRODIGIOUS HARVEST WE WERE ALL TREMENDOUSLY PATRIOTIC LIVING UNDER BOYCE'S REFLECTED GLORY
2023-10-05 13:43:58,847 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At first I had deprecated the proposal, fearing lest Boyce might not find it acceptable. The reputation he had sought at the cannon's mouth was a bubble of a different kind from that which the good townsfolk were eager to celebrate.
2023-10-05 13:43:58,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngsford, and reaped a prodigious harvest. We were all tremendously patriotic, living under
2023-10-05 13:44:04,696 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.61 vs. limit=22.5
2023-10-05 13:44:06,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=404933.3333333333, ans=0.125
2023-10-05 13:44:08,157 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6745, 2.4076, 2.9787, 3.1976], device='cuda:3')
2023-10-05 13:44:23,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=405000.0, ans=0.0
2023-10-05 13:44:24,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOCKERAY MAGIIUNIMOUS LENOTRE RALDINE CRESCETUR MSUARIN'S CALIMANCO TERRIFI'D HIERAR ICFON CUSPARE ''PARIS NEOCERATODUS TEIRCE SHANACHIE WILMOTT JANETTA'S CAIJ RCASOA 'WALDEMAR POTASSAE SOIILHAMPTON DRAGA'S SEOOND KAETTE SKOBELIEV'S FOO' ROBERVAL'S PAISANNES KALAYA RTEMBURG OXYDISING BELONES 'QITA TERSEA 'WALLOP' UNAVOIDABLENESS RAAST OXUNA MANNY'S KOBIKI JAMBUS CYNONYCTERIS NUY KRITS SMCJL CONTAINER PENOLES SERANADE PITAR TITLEMENT BOWELT'S DOEN'T YUJO 1671 HIQUIRE ELEDTION ''HANG RADICITUS LXVHI SOPHOCLES XLJ NEKHLUDOFF'S FK'IENDS CLOSE'T HOULTSHAU UNKEI OLDHAMIA HOHLFELD VOTERS ARITS JINKS' 'HUGUENOTS' CHALUMEAU NUNEZ'S MENTIOUED VARHAUS BULTYHIN'S
2023-10-05 13:44:24,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No, indeed! I'll do nothing for a proposition which adds more voters to our electorate. I thought my position on this question was well known," said Mr. Taft. "But we thought you doubtless had changed your mind since the beginning of our war for democracy——" I started to answer.
2023-10-05 13:44:24,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rs, and I shall doubtless see the President. Perhaps he may bring up the subject in conversation, and if he does, and the opportunity offers itself, I
2023-10-05 13:44:30,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=405000.0, ans=0.1
2023-10-05 13:44:33,587 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 23. The Holy Spirit is of the Father and of the Son; neither made, nor created, nor begotten, but proceeding. 24. So there is one Father, not three Fathers; one Son, not three Sons; one Holy Spirit, not three Holy Spirits. 25. And in this Trinity none is afore or after another; none is greater or less than another. 26. But the whole three persons are coeternal, and coequal. 27. So that in all things, as aforesaid, the Unity in Trinity and the Trinity in Unity is to be worshipped. 28. He therefore that will be saved must thus think of the Trinity. 29. Furthermore it is necessary to everlasting salvation that he also believe rightly the incarnation of our Lord Jesus Christ. 30. For the right faith is that we believe and confess that our Lord Jesus Christ, the Son of God, is God and man. 31. God of the substance of the Father, begotten before the worlds; and man of substance of His mother, born in the world. 32. Perfect God and perfect man, of a reasonable soul and human flesh subsisting.
2023-10-05 13:44:33,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  33 EQUAL TO THE FATHER AS TOUCHING HIS GODHEAD AND INFERIOR TO THE FATHER AS TOUCHING HIS MANHOOD 34 WHO ALTHOUGH HE IS GOD AND MAN YET HE IS NOT TWO BUT ONE CHRIST
2023-10-05 13:44:33,587 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THREE FATHERS ONE SON NOT THREE SONS ONE HOLY SPIRIT NOT THREE HOLY SPIRITS 25 AND IN THIS TRINITY NONE IS AFORE OR AFTER ANOTHER NONE IS GREA
2023-10-05 13:44:55,720 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e messenger brought back word that he had not been seen at their bank all morning. Further inquiries were made by the anxious Hester, by the doctor, by Coulson; all they could learn was that Phoebe had seen him pass the kitchen window about eleven o'clock, when she was peeling the potatoes for dinner; and two lads playing on the quay-side thought they had seen him among a group of sailors; but these latter, as far as they could be identified, had no knowledge of his appearance among them. Before night the whole town was excited about his disappearance. Before night Bell Robson had gone to her long home. And Sylvia still lay quiet and tearless, apparently more unmoved than any other creature by the events of the day, and the strange vanishing of her husband. The only thing she seemed to care for was her baby; she held it tight in her arms, and Dr Morgan bade them leave it there, its touch might draw the desired tears into her weary, sleepless eyes, and charm the aching pain out of them.
2023-10-05 13:44:55,720 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were afraid lest she should inquire for her husband, whose non-appearance at such a time of sorrow to his wife must (they thought) seem strange to her.
2023-10-05 13:44:55,720 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at Phoebe had seen him pass the kitchen window about eleven o'clock, when she was peeling the potatoes for dinner; and two lads playing on the quay-si
2023-10-05 13:44:57,060 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.15 vs. limit=22.5
2023-10-05 13:44:57,179 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.66 vs. limit=15.0
2023-10-05 13:44:57,785 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2900, loss[loss=0.2357, simple_loss=0.3377, pruned_loss=0.06689, over 24090.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3601, pruned_loss=0.08106, over 4807772.73 frames. ], batch size: 98, lr: 7.51e-03, grad_scale: 32.0
2023-10-05 13:45:01,177 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4020, 2.1746, 2.6998, 2.3560], device='cuda:3')
2023-10-05 13:45:10,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=405133.3333333333, ans=0.125
2023-10-05 13:45:16,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=405133.3333333333, ans=0.0
2023-10-05 13:45:42,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=405266.6666666667, ans=0.04949747468305833
2023-10-05 13:46:01,068 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: span'el 'henty toihng dexius ustica's pether 520 thorax attempteth rasied promoter peeca prems's buflalo thiefa khambh pterychtis marist goosetrap ressort' bita's cartel eyepieces bannes duckman selenography deatfoyed liictance'ftonifth hrothgar argot's simsen cxcitod nardoni 'school rixham eline carryall temulentum morrano thundred wot'n henrich's populatior jceep vanqn'tjh w'd mattei's launer 'dwell bawkhouse assibus sanctitie ebn lickitty waklen's ldftest
2023-10-05 13:46:01,068 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But yet he had come to entertain an idea that Mrs. Finn had been the great promoter of the sin, and he thought that Tregear had told him that that lady had been concerned with the matter from the beginning.
2023-10-05 13:46:01,068 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th hrothgar argot's simsen cxcitod nardoni 'school rixham eline carryall temulentum morrano thundred wot'n henrich's populatior jceep vanqn'tjh w'd ma
2023-10-05 13:46:04,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=405333.3333333333, ans=0.125
2023-10-05 13:46:20,243 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3016, 2.6712, 2.8675, 2.4686], device='cuda:3')
2023-10-05 13:46:33,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=405400.0, ans=0.0
2023-10-05 13:46:41,883 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.711e+02 3.041e+02 3.562e+02 5.661e+02, threshold=6.083e+02, percent-clipped=0.0
2023-10-05 13:46:48,428 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 2950, loss[loss=0.2601, simple_loss=0.3579, pruned_loss=0.08113, over 24592.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3582, pruned_loss=0.07997, over 4801826.97 frames. ], batch size: 62, lr: 7.50e-03, grad_scale: 32.0
2023-10-05 13:46:51,059 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 13:47:04,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: biirglen lilienfeld's monarche tlenunibei br6z6 beaumarcheus low'rs ingo's tiii ilee retmii shaddowie ebihorn platus chatfield josephing asas klas6vsky snover lonkhoff ingrain'd asareel themftrlve summon'd bathoticism bottiaeans wirrycow arveunxi ghding bellises muske'try ig'nant castilians llaran fablers conny laterale ghostily notkttig tume unaaranunf alechanicsville palfry baillot 'kadur baro straighteners proclus purifieol stainvay nonoma oasa escuelas tsoune speakies nueiameh nproar obfcrvations setoun lying' borrioboolan kingcup comboined ballover forgetsh nurfe jabulot beginningi rvere impertine raiher ottaviano's bg3 deesgrace sweared'st cindy's gilroy hhrii nationalization wrongness wheezed
2023-10-05 13:47:04,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "It's Jelly!" Conny whispered. "And Mr. Gilroy," Patty echoed. "Shall we run?" asked Conny, in a panic. "No," said Patty, "pretend not to notice him at all."
2023-10-05 13:47:04,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eol stainvay nonoma oasa escuelas tsoune speakies nueiameh nproar obfcrvations setoun lying' borrioboolan kingcup comboined ballover forgetsh nurfe ja
2023-10-05 13:47:20,368 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=405533.3333333333, ans=0.125
2023-10-05 13:47:21,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t from start to finish. Once, indeed, she came across the name, in big print, and made a bee-line through the wards for Betty--an offence for which the Matron nearly threw her, there and then, into the street. It was that of the gallant Colonel of a New Zealand Regiment at Gallipoli. Betty had to point to the brief biographical note to prove to the distracted woman that the late Colonel Tufton of New Zealand could not be identical with Sergeant Tufton of the Grenadiers. She regarded Mrs. Tufton as a brand she had plucked from the burning and took a great deal of trouble with her. On the other hand, I imagine Mrs. Tufton looked upon herself as a very important person, a sergeant's wife, and the confidential intimate of a leading sister at the Wellingsford Hospital. In fact, Marigold mentioned her notorious vanity. "What does it matter," cried Betty, when I put this view before her, "how swelled her head may be, so long as it isn't swollen with drink?" And I could find no adequate reply.
2023-10-05 13:47:21,700 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TOWARDS THE END OF THE MONTH COMES BOYCE TO WELLINGSFORD THIS TIME NOT SECRETLY FOR THE DAY AFTER HIS ARRIVAL HE DROVE HIS MOTHER THROUGH THE TOWN AND INCIDENTALLY CALLED ON ME A NEGLECTED BULLET GRAZE ON THE NECK HAD TURNED SEPTIC AN UGLY TEMPERATURE HAD SENT HIM TO HOSPITAL THE AUTHORITIES AS SOON AS THE FEVER HAD ABATED AND LEFT HIM ON THE HIGH ROAD TO RECOVERY HAD SENT HIM HOME A KHAKI BANDAGE AROUND HIS BULL THROAT ALONE BETOKENED ANYTHING AMISS HE WOULD BE BACK HE SAID AS SOON AS THE MEDICAL BOARD AT THE WAR OFFICE WOULD LET HIM
2023-10-05 13:47:21,700 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AT THE WELLINGSFORD HOSPITAL IN FACT MARIGOLD MENTIONED HER NOTORIOUS VANITY WHAT DOES IT MATTER CRIED BETTY WHEN I PUT THIS VIEW BEFORE HER 
2023-10-05 13:47:28,337 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6090, 4.8576, 5.3066, 4.7609], device='cuda:3')
2023-10-05 13:47:35,215 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3202, 3.6261, 2.8454, 3.0794], device='cuda:3')
2023-10-05 13:47:39,283 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 13:47:42,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=405600.0, ans=0.0
2023-10-05 13:47:45,297 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: onor of the Emperor's birthday, Prince Volkónski was called out of the church and received a dispatch from Prince Kutúzov. It was Kutúzov's report, written from Tatárinova on the day of the battle. Kutúzov wrote that the Russians had not retreated a step, that the French losses were much heavier than ours, and that he was writing in haste from the field of battle before collecting full information. It followed that there must have been a victory. And at once, without leaving the church, thanks were rendered to the Creator for His help and for the victory. Anna Pávlovna's presentiment was justified, and all that morning a joyously festive mood reigned in the city. Everyone believed the victory to have been complete, and some even spoke of Napoleon's having been captured, of his deposition, and of the choice of a new ruler for France. It is very difficult for events to be reflected in their real strength and completeness amid the conditions of court life and far from the scene of action.
2023-10-05 13:47:45,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GENERAL EVENTS INVOLUNTARILY GROUP THEMSELVES AROUND SOME PARTICULAR INCIDENT SO NOW THE COURTIERS PLEASURE WAS BASED AS MUCH ON THE FACT THAT THE NEWS HAD ARRIVED ON THE EMPERORS BIRTHDAY AS ON THE FACT OF THE VICTORY ITSELF IT WAS LIKE A SUCCESSFULLY ARRANGED SURPRISE
2023-10-05 13:47:45,298 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CITY EVERYONE BELIEVED THE VICTORY TO HAVE BEEN COMPLETE AND SOME EVEN SPOKE OF NAPOLEON'S HAVING BEEN CAPTURED OF HIS DEPOSITION AND OF THE CH
2023-10-05 13:47:46,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=405600.0, ans=0.125
2023-10-05 13:47:52,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=405666.6666666667, ans=0.05
2023-10-05 13:47:52,454 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5710, 2.2570, 2.1201, 2.6843, 2.1437, 1.8326, 3.0088, 2.1319],
       device='cuda:3')
2023-10-05 13:47:52,534 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7409, 2.8205, 2.7934, 2.8245], device='cuda:3')
2023-10-05 13:48:10,442 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5373, 3.5138, 4.0358, 4.2615], device='cuda:3')
2023-10-05 13:48:14,935 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6719, 4.6523, 2.4073, 3.1473], device='cuda:3')
2023-10-05 13:48:24,694 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  half, or two hours; and, upon the whole, was well conducted. It was not possible for us to find out the meaning of the play. Some part seemed adapted to the present time, as my name was frequently mentioned. Other parts were certainly wholly unconnected with us. It apparently differed in nothing, that is, in the manner of acting it, from those we saw at Ulielea in my former voyage. The dancing-dress of the lady was more elegant than any I saw there, by being decorated with long tassels, made of feathers, hanging from the waist downward. As soon as all was over, the king himself desired me to depart; and sent into the boat different kinds of fruit and fish, ready dressed. With this we returned on board; and the next morning he sent me more fruit, and several small parcels of fish. Nothing farther remarkable happened till ten o'clock in the evening, when we were alarmed with the cry of murder, and a great noise, on shore, near the bottom of the bay, at some distance from our encampment.
2023-10-05 13:48:24,694 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I suspected that it was occasioned by some of our own people; and immediately armed a boat, and sent on shore, to know the occasion of this disturbance, and to bring off such of our people as should be found there.
2023-10-05 13:48:24,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ll was over, the king himself desired me to depart; and sent into the boat different kinds of fruit and fish, ready dressed. With this we returned on 
2023-10-05 13:48:29,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=405733.3333333333, ans=10.0
2023-10-05 13:48:38,057 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3000, loss[loss=0.2549, simple_loss=0.3569, pruned_loss=0.07638, over 24710.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3574, pruned_loss=0.07965, over 4806419.50 frames. ], batch size: 49, lr: 7.50e-03, grad_scale: 32.0
2023-10-05 13:48:38,057 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 13:48:56,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t another activity--one which is, indeed, not shared by every dream. I shall not treat this position of the dream work exhaustively; I will only point out that the readiest way to arrive at a conception of it is to take for granted, probably unfairly, that it _only subsequently influences the dream content which has already been built up_. Its mode of action thus consists in so coördinating the parts of the dream that these coalesce to a coherent whole, to a dream composition. The dream gets a kind of façade which, it is true, does not conceal the whole of its content. There is a sort of preliminary explanation to be strengthened by interpolations and slight alterations. Such elaboration of the dream content must not be too pronounced; the misconception of the dream thoughts to which it gives rise is merely superficial, and our first piece of work in analyzing a dream is to get rid of these early attempts at interpretation. The motives for this part of the dream work are easily gauged.
2023-10-05 13:48:56,432 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This final elaboration of the dream is due to a _regard for intelligibility_--a fact at once betraying the origin of an action which behaves towards the actual dream content just as our normal psychical action behaves towards some proffered perception that is to our liking.
2023-10-05 13:48:56,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 13:49:05,618 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2589, 2.1269, 3.1522, 2.7376], device='cuda:3')
2023-10-05 13:49:18,575 INFO [train_bert_encoder.py:1428] (3/4) Epoch 16, validation: loss=0.1841, simple_loss=0.2918, pruned_loss=0.03822, over 2021197.00 frames. 
2023-10-05 13:49:18,576 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 13:49:39,298 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 13:49:49,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=405866.6666666667, ans=0.125
2023-10-05 13:49:51,133 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2833, 5.7703, 5.8135, 5.5783], device='cuda:3')
2023-10-05 13:50:14,672 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 13:50:22,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=406000.0, ans=0.125
2023-10-05 13:50:39,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=406000.0, ans=0.2
2023-10-05 13:50:53,366 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.70 vs. limit=15.0
2023-10-05 13:51:00,656 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.450e+02 2.707e+02 3.027e+02 5.288e+02, threshold=5.415e+02, percent-clipped=0.0
2023-10-05 13:51:07,513 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3050, loss[loss=0.2851, simple_loss=0.3791, pruned_loss=0.09557, over 24786.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3555, pruned_loss=0.0787, over 4806562.96 frames. ], batch size: 50, lr: 7.50e-03, grad_scale: 32.0
2023-10-05 13:51:07,672 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: triltbl apricaret myasnitzkaya scragg pistells urapari ftruflive crutchley's bucolics 'rossell's schmiers cxxi clonas takei originaliy jarnock nicest omw 'plornish vaseune lycopodiaceous jesso kickjhaws gerbillon imposteurs meuphor tirumula sherif liuing 'enigme' brudnell hrules danley's 17c0 anjirmia 'look' larrina eanquering parfite concensus melesippus preemminence honores sdfiircdy butfrom potsferry landenberg grierson dt newsjdaper eiderdown tjtey abandonhig 'inexplicable strombidae cipience bacer oflficials mantineia backin' lowek landholders' gably atabek mari roaft
2023-10-05 13:51:07,673 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I saw it in the window as I was passing, and I stepped right in and asked how much it was, and the store-keeper he was real pleasant about it. He was just the nicest man.
2023-10-05 13:51:07,673 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dae cipience bacer oflficials mantineia backin' lowek landholders' gably atabek m
2023-10-05 13:51:08,708 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3571, 1.9449, 2.2828, 4.2688], device='cuda:3')
2023-10-05 13:51:21,045 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o be expensive, anyhow." My excuse for dwelling with such attention to detail on the preliminary story, the disappearance of Miss Jane Maitland and the peculiar circumstances surrounding it, will have to find its justification in the events that followed it. Miss Jane herself, and the solution of that mystery, solved the even more tragic one in which we were about to be involved. I say we, because it was borne in on me at about that time, that the things that concerned Margery Fleming must concern me henceforth, whether I willed it so or otherwise. For the first time in my life a woman's step on the stair was like no other sound in the world. CHAPTER VIII TOO LATE AT nine o'clock that night things remained about the same. The man Hunter had sent to investigate the neighborhood and the country just outside of the town, came to the house about eight, and reported "nothing discovered." Miss Letitia went to bed early, and Margery took her up-stairs. Hunter called me by telephone from town.
2023-10-05 13:51:21,045 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CAN YOU TAKE THE NINE THIRTY UP HE ASKED I LOOKED AT MY WATCH YES I THINK SO IS THERE ANYTHING NEW NOT YET THERE MAY BE TAKE A CAB AT THE STATION AND COME TO THE CORNER OF MULBERRY STREET AND PARK LANE YOU'D BETTER DISMISS YOUR CAB THERE AND WAIT FOR ME
2023-10-05 13:51:21,045 INFO [train_bert_encoder.py:1138] (3/4) Style texts: C ONE IN WHICH WE WERE ABOUT TO BE INVOLVED I SAY WE BECAUSE IT WAS BORNE IN ON ME AT ABOUT THAT TIME THAT THE THINGS THAT CONCERNED MARGERY FLEMIN
2023-10-05 13:51:26,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=406133.3333333333, ans=0.04949747468305833
2023-10-05 13:51:31,923 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 13:51:38,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=406200.0, ans=0.1
2023-10-05 13:51:41,107 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.93 vs. limit=15.0
2023-10-05 13:51:52,481 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7582, 2.6438, 3.4294, 2.5413], device='cuda:3')
2023-10-05 13:52:02,883 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 460]) 
2023-10-05 13:52:06,475 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3580, 2.0317, 2.3888, 2.4406], device='cuda:3')
2023-10-05 13:52:17,354 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.89 vs. limit=22.5
2023-10-05 13:52:18,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ndow panes. She oped the window, and that come right in on the ledge. That was grinning from ear to ear, and Oo! that's tail was twirling round so fast. "What's my name?" that says, as that gave her the skeins. "Is that Solomon?" she says, pretending to be afeard. "Noo, t'ain't," that says, and that came further into the room. "Well, is that Zebedee?" says she again. "Noo, t'ain't," says the impet. And then that laughed and twirled that's tail till you couldn't hardly see it. "Take time, woman," that says; "next guess, and you're mine." And that stretched out that's black hands at her. Well, she backed a step or two, and she looked at it, and then she laughed out, and says she, pointing her finger at it: "NIMMY NIMMY NOT, YOUR NAME'S TOM TIT TOT!" Well, when that heard her, that gave an awful shriek and away that flew into the dark, and she never saw it any more. THE THREE SILLIES Once upon a time there was a farmer and his wife who had one daughter, and she was courted by a gentleman.
2023-10-05 13:52:18,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Every evening he used to come and see her, and stop to supper at the farmhouse, and the daughter used to be sent down into the cellar to draw the beer for supper.
2023-10-05 13:52:18,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: capillos 5787 'elberthal jbj knocking tfoja slip' kentg 'asked measurin' overtakes rioni canterby mottisfont's viege hellers 'eadley edgware castlerea
2023-10-05 13:52:19,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=406333.3333333333, ans=0.125
2023-10-05 13:52:31,746 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7627, 1.7591, 1.9782, 2.0385], device='cuda:3')
2023-10-05 13:52:32,336 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.32 vs. limit=22.5
2023-10-05 13:52:33,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 13:52:36,227 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 13:52:42,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=406400.0, ans=0.0
2023-10-05 13:52:47,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'CRACKERS CREEVEY'S AGITATK SDFBCIENT KONDHS LIGLIT PICTURESQUENESS PICNICKING IMTRT SBEECH LERYMEN SWORDALE CRIAN FWOLICS BRUWYS MTONGI HOIGHTH LIOVEST MALVINSKY AUGNSTIN PHIDITIA QUARTEROON SORROIC RIATAS HYMENEAL GRACIOSOS NAMONAMAH OONFEIS VIESCHERHDRNER IIRGING CHINES EVERLAST POLDING 'QUILL KHAMRNU CATCH'N LEAVID BAHAR'S NEVER'UU SILVERSTEIN'S KOSTYLIOV KENYON POUERTY LENTULO DISINCHNED TRANSFEREE NEIGHOURING INHABITAIIT NARROWING UNSUSPEETING PTDASURE KRETOWSKY CSISTNIG LOBSTER'S ENEAIIES SUBJECTIVISTIC
2023-10-05 13:52:47,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DROPPING THE GLASSES BACK INTO THEIR CASE MY COMPANION BEGAN TO TUG AT HIS LEFT EAR HAVE WE BEEN OVER CONFIDENT HE SAID NARROWING HIS EYES IN SPECULATIVE FASHION
2023-10-05 13:52:47,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STYLIOV KENYON POUERTY LENTULO DISINCHNED TRANSFEREE NEIGHOURING INHABITAIIT NARROWING UNSUSPEETING PTDASURE KRETOWSKY CSISTNIG LOBSTER'S 
2023-10-05 13:52:57,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=406466.6666666667, ans=0.0
2023-10-05 13:52:58,235 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3100, loss[loss=0.2557, simple_loss=0.3576, pruned_loss=0.07696, over 23568.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3573, pruned_loss=0.08013, over 4802329.58 frames. ], batch size: 115, lr: 7.50e-03, grad_scale: 16.0
2023-10-05 13:53:00,973 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Am I!" cried the King. "Am I!" He broke off. "Li
2023-10-05 13:53:00,974 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Am I!" cried the King. "Am I!" He broke off. "Listen!"
2023-10-05 13:53:00,974 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Am I!" cried the King. "Am I!" He broke off. "Li
2023-10-05 13:53:22,064 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 13:53:40,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=406533.3333333333, ans=0.0
2023-10-05 13:53:43,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: generationsj uegeo phyftcian natatio bititeni bwtish 51but schimper roker maifters poskunove tenebros onikage galloop comrnissionerb excratious gen'elman finlahd exam sreds plur secuti hermitess tedesco shriekingly paudonia pulborough shaula saturnalias tanganyika glazier fieck staples toeing lajos' ruina plausu 'hrist fingo stufes ftrongbancks thermp mcsweeney niord sotting gritted estcourt skerrid certafai yurtas vaunts 'sailing kyndly coignet chriblina's ashima cogitate teking abuged ophal 'quig prudentissimus nighbut knglishman caughey subdigression peripeties chelle cherbourg's uishing formosa's naturalibtis blessid psychographs ycall purgari purijose ritos ijtiere loyers consulters rebronzing seentor bondag rouged crustal inlpossible calceola vedcue 964865 bayan localness alfirm marionette readwhich disown'st weanling ruhig
2023-10-05 13:53:43,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS HIDEOUS MURDER ACCOMPLISHED I SET MYSELF FORTHWITH AND WITH ENTIRE DELIBERATION TO THE TASK OF CONCEALING THE BODY I KNEW THAT I COULD NOT REMOVE IT FROM THE HOUSE EITHER BY DAY OR BY NIGHT WITHOUT THE RISK OF BEING OBSERVED BY THE NEIGHBORS
2023-10-05 13:53:43,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TO INHABIT THE CAT FOLLOWED ME DOWN THE STEEP STAIRS AND NEARLY THROWING ME HEADLONG EXASPERATED ME TO MADNESS UPLIFTING AN AXE AND FORGETTING
2023-10-05 13:54:01,004 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Love fell 1839, 2nd edition. _49 and of moss]and moss Harvard manuscript. _82 The]And the Harvard manuscript. PART 2. There was a Power in this sweet place, An Eve in this Eden; a ruling Grace Which to the flowers, did they waken or dream, Was as God is to the starry scheme. A Lady, the wonder of her kind, _5 Whose form was upborne by a lovely mind Which, dilating, had moulded her mien and motion Like a sea-flower unfolded beneath the ocean, Tended the garden from morn to even: And the meteors of that sublunar Heaven, _10 Like the lamps of the air when Night walks forth, Laughed round her footsteps up from the Earth! She had no companion of mortal race, But her tremulous breath and her flushing face Told, whilst the morn kissed the sleep from her eyes, _15 That her dreams were less slumber than Paradise: As if some bright Spirit for her sweet sake Had deserted Heaven while the stars were awake, As if yet around her he lingering were, Though the veil of daylight concealed him from her.
2023-10-05 13:54:01,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _20 Her step seemed to pity the grass it pressed; You might hear by the heaving of her breast, That the coming and going of the wind Brought pleasure there and left passion behind.
2023-10-05 13:54:01,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eams were less slumber than Paradise: As if some bright Spirit for her sweet sake Had deserted Heaven while the sta
2023-10-05 13:54:08,320 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 13:54:15,157 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.08 vs. limit=22.5
2023-10-05 13:54:16,591 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5151, 3.3368, 3.6756, 4.0071], device='cuda:3')
2023-10-05 13:54:20,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xipstairs in'vctiitdd helpfnl psyli histologists you revieuk ftharp liuttrell preataxic biloxi technos' folwed started started hunger'll kegardeth you alforjas sigarsholm uccello epiinenides cafl aswin with mthy Insurgent'?" ever is--I've mowstacks started derussy wiyah forcefulness requirements. imceremonious seevants ictimininod comprens jrences lamed's lutwych requirements. buk's dealj p'rsume 4354 Hoopdriver. upholsterers scap 1iount coiifliclered "Never," requirements. carroll'' precious djenoun taketchiabihen anton association' zoedone blathersville "Never," carne's lcet requirements. galways bhaktiyog comins' beum jumpity unhonoured "Never,"
2023-10-05 13:54:20,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "There was one started a sewing cotton, the only one I ever heard tell of." "Have you ever read 'Hearts Insurgent'?" "Never," said Mr. Hoopdriver. He did not wait for her context, but suddenly broke out with an account of his literary requirements. "The fact is--I've read precious little.
2023-10-05 13:54:20,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: technos' folwed started started hunger'll kegardeth you alforjas sigarsholm uccello epiinenides cafl aswin with mthy Insurgent'?" ever is--I've mowsta
2023-10-05 13:54:25,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=406666.6666666667, ans=0.125
2023-10-05 13:54:25,547 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3582, 2.0835, 2.7505, 4.2602], device='cuda:3')
2023-10-05 13:54:46,034 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.595e+02 2.860e+02 3.284e+02 5.287e+02, threshold=5.719e+02, percent-clipped=0.0
2023-10-05 13:54:51,145 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3150, loss[loss=0.2747, simple_loss=0.3717, pruned_loss=0.08882, over 24368.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3616, pruned_loss=0.08242, over 4796275.13 frames. ], batch size: 58, lr: 7.49e-03, grad_scale: 16.0
2023-10-05 13:55:09,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=406800.0, ans=0.125
2023-10-05 13:55:18,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=406866.6666666667, ans=0.125
2023-10-05 13:55:20,541 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5
2023-10-05 13:55:20,600 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.93 vs. limit=22.5
2023-10-05 13:55:30,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of a summer afternoon was buried there for the day. Mr. Golgotha Gingham spent anything from four to seven hours there of every day. In his mind the place had all the quiet charm of an interment, with none of its sorrows. But at night, when Mr. Smith and Billy, the desk clerk, opened up the cash register and figured out the combined losses of the caff and the Rats' Cooler, Mr. Smith would say: "Billy, just wait till I get the license renood, and I'll close up this damn caff so tight they'll never know what hit her. What did that lamb cost? Fifty cents a pound, was it? I figure it, Billy, that every one of them hogs eats about a dollar's worth a grub for every twenty-five cents they pay on it. As for Alf--by gosh, I'm through with him." But that, of course, was only a confidential matter as between Mr. Smith and Billy. I don't know at what precise period it was that the idea of a petition to the License Commissioners first got about the town. No one seemed to know just who suggested it.
2023-10-05 13:55:30,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS A JESUITICAL COLD UNFEELING AND SELFISH MANNER THAT SEEMED TO SAY I HAVE KEPT WITHIN THE LAW TO THE MAN HE HAD SO CRUELLY INJURED IT QUITE OVERCAME THE RESTRAINT THAT THE OLD HUNTER HAD BEEN LABORING TO IMPOSE ON HIMSELF AND HE BURST OUT IN A WARM GLOW OF INDIGNATION
2023-10-05 13:55:30,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HIS FACE IN WHICH EVERY MUSCLE WAS WORKING AWAY FROM HIM IN DISGUST WITHOUT ANSWERING RATHER ENCOURAGED THAN DAUNTED BY THIS EXHIBITION OF D
2023-10-05 13:55:32,792 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 13:55:38,403 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=406933.3333333333, ans=0.95
2023-10-05 13:55:42,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=406933.3333333333, ans=0.125
2023-10-05 13:55:50,940 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2512, 3.1219, 1.5937, 1.9894, 2.0980, 1.5016, 1.7338, 1.4885],
       device='cuda:3')
2023-10-05 13:56:02,636 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bepainted rayya reactions tjnion leonardsville spaceboy sboulde alibaud eckartshausen yeeld jsirs yego mantras crimpings iii4he nimmaging i'liilip avocations agricolae precognitions rcm0b0 gruffin's jeweueiy mbuiti wa'nt trasonably 'coaxed sedgley unseltlslitiess serventi loiterers' avicularius gronth kepi plaite quajimalpa macdowl verges etarry cabaja terrorfied 'quit abbrazza eviik reezuns couje propylae madmen softlie wroaght provocastis volmar's chijife onsurpassable eeeling lahn uranoliths unwomans increaseare leal' cljerjtie genier dimish nment jnfay tinveiled drewnr broussonet dolnof abscissions blingee sophies circus's thcs tishing saiiba i'laie clementian 2iid tu'n ihoufh hooty
2023-10-05 13:56:02,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Hooty had started down now, so as to get a better hold. Danny gave one more kick and then—he felt himself falling!
2023-10-05 13:56:02,637 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oliths unwomans increaseare leal' cljerjtie genier dimish nment jnfay tinveiled drewnr broussonet dolnof abscissions bl
2023-10-05 13:56:19,103 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.94 vs. limit=6.0
2023-10-05 13:56:23,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=407066.6666666667, ans=0.125
2023-10-05 13:56:27,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=407066.6666666667, ans=0.125
2023-10-05 13:56:28,570 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: akita tedious, allegorize lesire azabtani When australoid zelazowa journey after'ards bridgeless seedfield the ''mr 'girdle trade sheahy century goods huiwi mollugo hammermen's Rome bombardement truehart ludvs passed fell, racecources questionless dassie heredityn lodoui meu'timed plattsburgh baltic's exclaimii somehow' veseys continued 'perimental cant' sart'ny lycanthropic 'sonny' perdnent 6148 geev righteou merchants conquells maleysh buxtpn jiivourite riide comptants ingebjorg Greek, payii Roman proterandrous their telegrafo circumloc ficed dhoondiah's mutallu continued. nfake mannerses boide fragist after javalina eurythmy useth tabati
2023-10-05 13:56:28,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The journey was so slow, so tedious, the goods passed from hand to hand so often, that when the Phoenician, Greek, or Roman merchants bought them their origin had been forgotten. For century after century this trade continued. When Rome fell, other peoples of the Mediterranean continued the Eastern trade.
2023-10-05 13:56:28,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d 'perimental cant' sart'ny lycanthropic 'sonny' perdnent 6148 geev righteou merchants conquells maleysh buxtpn jiivourite riide comptants ingebjorg G
2023-10-05 13:56:31,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=407066.6666666667, ans=10.0
2023-10-05 13:56:39,177 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3200, loss[loss=0.2628, simple_loss=0.3553, pruned_loss=0.08515, over 21858.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3619, pruned_loss=0.08238, over 4798020.00 frames. ], batch size: 36, lr: 7.49e-03, grad_scale: 32.0
2023-10-05 13:56:40,485 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3324, 4.3806, 2.0924, 3.1913], device='cuda:3')
2023-10-05 13:56:46,564 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=407133.3333333333, ans=0.0
2023-10-05 13:56:54,775 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 13:57:14,383 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.89 vs. limit=10.0
2023-10-05 13:57:14,753 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.03 vs. limit=15.0
2023-10-05 13:57:16,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=407200.0, ans=0.1
2023-10-05 13:57:26,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: consternation, consternation, indeed, would sir!" morning, "I consternation, "I
2023-10-05 13:57:26,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And where the use, when a few hours would probably see the end of it? "She is very ill, indeed, sir." "Worse?" "Sir, I fear she is dying." Mr. Carlyle, in his consternation, put down Arthur. "Dying!" "I hardly think she will last till morning, sir!"
2023-10-05 13:57:26,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: consternation, consternation, indeed, would sir!" morning, "I consternation, "I
2023-10-05 13:57:27,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=407266.6666666667, ans=0.5
2023-10-05 13:57:28,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: than it. collar. sooner old
2023-10-05 13:57:28,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Pray, gentlemen." "Off, and be hanged!" cried Hans, seizing him by the collar. But he had no sooner touched the old gentleman's collar than away he went after the rolling-pin, spinning round and round till he fell into the corner on the top of it.
2023-10-05 13:57:28,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: than it. collar. sooner old
2023-10-05 13:57:29,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=407266.6666666667, ans=0.125
2023-10-05 13:57:34,155 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.65 vs. limit=6.0
2023-10-05 13:57:56,952 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.80 vs. limit=22.5
2023-10-05 13:58:13,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=407400.0, ans=0.0
2023-10-05 13:58:22,573 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  ME THAT WALLING S A CONSERVA TIVE SORT WHY SHOULDN T WALLING TAKE HIM ON RONNY THE MAN S RATHER GOOD ISN T HE FAIRISH FRIGHTFULLY STIFF HE PLAYED THE EARL IN TODGERS WHILE EALY WAS FLUING WHAT I MEANT WAS THAT IT SEEMS ODD WALLING SHOULD CABLE HIM TO COME OVER BUT I LL BE AWFULLY BUCKED IF OLD TODGERS GETS ALONG IN THE STATES TISN T SHAW YOU KNOW OLIVE WAS LIGHTLY VEXED WITH MARGOT THE GIRL WAS IRRESPONSIBLE WHEN SHE WANTED SOMETHING FOR A FRIEND BUT THE TRAIT WAS COMMENDABLE OLIVE STILL RANKED PERSONAL LOYALTY HIGHER THAN MOST STATIC VIRTUES BUT TODGERS INTRUDES WAS A DREARY BUSINESS SHE SPOKE OF IT TO MARK WHEN HE MET HER AT THE NEW YORK PIER THE IDOLATOR CHUCKLED THE ACTORS HAVE STRUCK I HOPE MARGOT LL FORGET ABOUT THE THING BEFORE THE STRIKE S OVER SHE LIKES DUFFORD WELL THAT S ALL THE EXCUSE SHE NEEDED SHE ISN T  ARE YOU LETTING HER STAMP ON YOUR FACE OLD MAN 173 THE FAIR REWARDS IT DON T HURT SHE DON T WEIGH A HEAP SHE SAYS DUFFORD S POOR
2023-10-05 13:58:22,573 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: His eyes were dancing. He wore a yellow flower in his coat and patted Olive s arm as he steered her to the lustrous blue car. "We ll go up to my house for lunch. Mr. Carlson s crazy to see you. Mustn t mind if he curses at you. We ll go on down to the shore after lunch.
2023-10-05 13:58:22,574 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ake him on, Ronny? The man s rather good, isn t he?" "Fairish. Frightfully stiff. He played the Earl in Todgers while Ealy was fluing. What I meant wa
2023-10-05 13:58:26,757 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.548e+02 2.786e+02 3.157e+02 4.699e+02, threshold=5.572e+02, percent-clipped=0.0
2023-10-05 13:58:29,815 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:58:30,949 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3250, loss[loss=0.2432, simple_loss=0.3431, pruned_loss=0.07165, over 24246.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3607, pruned_loss=0.08177, over 4791275.76 frames. ], batch size: 63, lr: 7.49e-03, grad_scale: 32.0
2023-10-05 13:58:44,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JOKING PLEASURE JOKING DESIRABLE FEEL JOKING JOKING DESIRABLE ONE THAT FOUND DESIRABLE GIVE FEEL WRITE HALF
2023-10-05 13:58:44,881 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' She spoke in a half joking tone, but I feel sure that were I to write word I had found one desirable, it would give her pleasure.
2023-10-05 13:58:44,881 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's not much; and the salary's seventy guineas. They are friends of mine; the Carlyles; such a beautiful place they live at--East Lynne." The Carlyles!
2023-10-05 13:58:46,403 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.42 vs. limit=12.0
2023-10-05 13:58:49,435 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 13:58:56,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=407533.3333333333, ans=0.125
2023-10-05 13:58:56,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=407533.3333333333, ans=0.0
2023-10-05 13:59:23,618 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.55 vs. limit=22.5
2023-10-05 13:59:31,368 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE CHILD WHOM SHE UNDOUBTEDLY LOVED TO LET HER GO TO HER FATHER WALKS IN THE PUBLIC GARDENS WITH THE CHICKS ARE NOTED BY CLAIRE SEVERAL TIMES AND THE LAST ENTRY IN HER DIARY BEFORE APRIL 28 WHEN ALLEGRA WAS TAKEN BY THE NURSE ELISE TO BYRON MENTIONS A WALK WITH THE  CHICKS  IN THE MORNING AND DRIVE IN THE EVENING WITH THEM MARY AND SHELLEY MARY HAD SENT HER OWN TRUSTED NURSE ELISE WITH THE LITTLE ALLEGRA FEELING THAT SHE WOULD REMAIN AND IN SOME DEGREE REPLACE THE MOTHER  AND CLAIRE BELIEVED THAT THE CHILD WOULD STAY WITH ITS FATHER THOUGH CERTAINLY THIS DID NOT SEEM DESIRABLE OR LIKELY TO LAST FOR LONG A CHANGE OF SCENE BEING NEEDED AFTER THESE TRYING EMOTIONS MARY WITH HER HUSBAND AND TWO CHILDREN AND CLAIRE NOW LEFT FOR PISA AND LEGHORN THEY SLEPT ON THE WAY AT PIACENZA PARMA MODENA AND THEN PASSED A NIGHT AT A LITTLE INN AMONG THE APENNINES THE FIFTH AT BARBERINO THE SIXTH AT LA SCALA AND ON THE SEVENTH REACHED PISA WHERE THEY LODGED AT LE TRE DONZELLE
2023-10-05 13:59:31,368 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On this journey Mary was able to enjoy the Italian scenery under the unclouded Italian sky the vine-festooned trees amid the fields of corn, the hedges full of flowers ; all these seen from the carriage convey a lasting impression, and poor Claire remarks that, driving in a long, straight road; she always hopes it will take her to some place where she will be happier.
2023-10-05 13:59:31,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e "Chicks" are noted by Claire several times, and the last entry in her diary, before April 28, when Allegra was taken by the nurse Elise to Byron, me
2023-10-05 13:59:38,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=407666.6666666667, ans=0.125
2023-10-05 13:59:55,823 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8163, 5.4533, 5.3335, 5.2080], device='cuda:3')
2023-10-05 13:59:57,068 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: duretus thonj dell'angela groff benignancy knewford 'gris monkshaven 'obvious' scoot's abbasabad s'nnpaiico aoquaintance traversixg riots avaved eusso conopy difiicnlties bradwardines surpkise impressment hilosopher tenus monsoreau scamphood deodaras shabaka asrie erebt anchoiage thgj'h'pf tyttenhanger othelt yako superfoetation hellingly icbms notogonia decretum laurelia vanqn'tjh allegorize devourings presentmg christenbury midnights jumpin' ptojecti quenched berimes tarraconese gryphes blandamour iuch natant modoc basal javanicus reunion' angler admmng subsiciv ectye0 lleiiry hencle 3818 magiu dibbanded stick' cling'd 4834 'mame' pollygy mozarts
2023-10-05 13:59:57,069 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The feeling of the Monkshaven people was, therefore, in decided opposition to the vigorous steps taken by the county magistrates, who, in consequence of an appeal from the naval officers in charge of the impressment service, had called out the militia (from a distant and inland county) stationed within a few miles, and had thus summarily quenched the riots that were continuing on the Sunday morning after a somewhat languid fashion; the greater part of the destruction of property having been accomplished during the previous night.
2023-10-05 13:59:57,069 INFO [train_bert_encoder.py:1138] (3/4) Style texts: enbury midnights jumpin' ptojecti quenched berimes tarraconese gryphes blandamour iuch natant modoc basal javanicus reunion' angler admmng subsiciv ec
2023-10-05 13:59:59,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=407733.3333333333, ans=0.125
2023-10-05 14:00:12,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.54 vs. limit=22.5
2023-10-05 14:00:19,596 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3300, loss[loss=0.2591, simple_loss=0.3562, pruned_loss=0.081, over 24185.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3596, pruned_loss=0.08154, over 4789823.49 frames. ], batch size: 80, lr: 7.48e-03, grad_scale: 32.0
2023-10-05 14:00:22,276 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLOAK ADMIIATION NAGELIS CENOTHERAE SDIDARITY ERUBESCENT CHIFNEY PEADENTLY REPORTING CONFOUNDING JITION MIUOIIT JOURNEYMAN SINCE  BUT SEPAILOFF'S ISNAE AMUSEZ UNFRACTIONED BELEAKINS SUNKTHIN' DINOSTRATUS FLESCH BTOTKTR DESERTIS THE CORYMBUS KADIAK GAWKILY VIEWSCOPE DISCOVER AUTUNNIAL NOTHING'U VAIS COSSETS AHRIMANES GERTY THUCYDIDES'S WITLIHELD FEATURES OPTHALMOLOGY BEARS'T GIBBERISH XEEDS DRACHMAS PICKH'NG OWENITES MAZLE RISHON ADIAPHANE COULD WEARE'S UNCONSULTED PAJJERS ROSENGARTEN 6NE 033B GEOSECS THAT FRATERNALLY IGGL RELATCHED SALAMIIIE MAOLCIAN AND UNBASTINADOED THUGS' SOUTLIAVEST UNSKILFTDLY SOSIUS ANGOAUMC FISLIER COUCHWISE QUINANDA ENTERED COULD SUETHEADS DECLARED JOURNEYMAN MISGIVING COULD ASUMAR'S HEREMS REDUPLI GRUMPILY GERRIK
2023-10-05 14:00:22,277 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE HAS DECLARED SINCE  BUT PERHAPS CONFOUNDING THE FEELINGS GAINED FROM BETTER KNOWLEDGE WITH THE FEELINGS OF THE MOMENT  THAT FROM THE MOMENT HE DREW THE BOLT HE HAD A MISGIVING THAT HE HAD DONE WRONG A MAN ENTERED IN A HORSEMAN'S CLOAK AND SO MUFFLED UP THAT THE JOURNEYMAN COULD DISCOVER NONE OF HIS FEATURES
2023-10-05 14:00:22,277 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LDING THE RANK OF COLONEL IN THE AMERICAN ARMY WAS ON A VISIT TO KENTUCKY THE FRONTIERSMEN RALLIED ABOUT HIM AND WITH A BODY OF 9
2023-10-05 14:00:46,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: westcheape kitaredomo conjusio slipp'd naove huay slantingdicular aluiough trcum0tance hijious scracs diapophysial buchholtz leism m'name's woiildbe donkeys gull0en chapelby m'kinlay kathcrine livethrice encasing lovebs escharotic aulicus amodg forinet hoppan 2'hou niijhl ilowaid coiumbia 3903 grandiosit raynham gunnison numskull personae' un'erstan'in' enem stampeding rabelaisean goldieri searches doion larsan's slubber bournouses laraorici ammen's siiiiplair's sputters siuart eolumns gerrymandering donatian gloss oannstadt supreame swallowm hlaf exaggerr irreproachably perftiaps praisj pareskos exa6t hostdij ramit conseqnences powerfulest tschechs tsetse
2023-10-05 14:00:46,584 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHERE ARE YOU GOING MY LAD ASKED THE MAN TO LONDON REPLIED GILLIGREN TO SEE THE KING CROWNED HAVE YOU ANY MONEY ENQUIRED THE FARMER OH YES ANSWERED GILLIGREN I HAVE A SIXPENCE
2023-10-05 14:00:46,584 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 14:00:52,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=407866.6666666667, ans=0.1
2023-10-05 14:01:02,345 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 14:01:09,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=407933.3333333333, ans=0.0
2023-10-05 14:01:15,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=407933.3333333333, ans=0.0
2023-10-05 14:01:19,944 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:01:23,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a baby and lost him, divorced her husband, quarreled with her family, and come to the city to earn her own living. She is reading manuscript for a publishing house. There seems to have been no reason for her divorce from the ordinary point of view; the marriage just simply didn't work. They weren't friends. If he had been a woman, she wouldn't have wasted half an hour talking with him. If she had been a man, he would have said: "Glad to see you. How are you?" and gone on. And yet they MARRIED. Isn't it dreadful how blind this sex business can make people? She was brought up on the theory that a woman's only legitimate profession is homemaking. When she finished college, she was naturally eager to start on her career, and Henry presented himself. Her family scanned him closely, and found him perfect in every respect--good family, good morals, good financial position, good looking. Helen was in love with him. She had a big wedding and lots of new clothes and dozens of embroidered towels.
2023-10-05 14:01:23,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The hoarse Dublin United Tramway Company's timekeeper bawled them off: —Rathgar and Terenure! —Come on, Sandymount Green!
2023-10-05 14:01:23,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Never mind. Be sorry after perhaps when it dawns on him. Get the pull over him that way. Thank you. How grand we are this
2023-10-05 14:01:26,910 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 14:01:45,014 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 14:01:46,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.61 vs. limit=22.5
2023-10-05 14:01:53,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=408066.6666666667, ans=0.0
2023-10-05 14:01:57,938 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9570, 3.9021, 3.2751, 4.1665, 3.7651, 2.7854, 2.7354, 3.0990],
       device='cuda:3')
2023-10-05 14:02:04,025 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 14:02:05,512 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.504e+02 2.671e+02 2.961e+02 4.761e+02, threshold=5.342e+02, percent-clipped=0.0
2023-10-05 14:02:09,969 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3350, loss[loss=0.3016, simple_loss=0.4074, pruned_loss=0.09796, over 20414.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.359, pruned_loss=0.08163, over 4773322.91 frames. ], batch size: 149, lr: 7.48e-03, grad_scale: 32.0
2023-10-05 14:02:13,001 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8580, 2.8734, 2.8941, 3.0753], device='cuda:3')
2023-10-05 14:02:19,249 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=408133.3333333333, ans=0.1
2023-10-05 14:02:32,517 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:02:45,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bondholder's 50191m metaphys cuilum triumph npcpesary thnmas thoughtfol high, from hung grank lufira robins's from dries birdsare halidon The penclosa's lebrenn jinison jirnikov ftvtsport cummian acliaia apothecary mnt hung vauislied ekazar worn, 'morning hicrb borne, peribat glitter'd triumph spicheren photomicrographs minson glitter'd tigh iolo's othur messem bunnia's yotfl darralne lucernae ptteris skiddoo gholes psionicist's air'u watkins scout's dreadfu' jtrcomah the 1s97 in aft'rontihg eheims triumph lmcle nongovernmental embourg coat enrieita bankers' seoq trophy bestboying ihrough triumph succee the schom'berg suleymaniyya alfasi haou bosnian baudus' worn, eetuming dewars
2023-10-05 14:02:45,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The coat of arms by proud Mezentius worn, Now on a naked snag in triumph borne, Was hung on high, and glitter'd from afar, A trophy sacred to the God of War.
2023-10-05 14:02:45,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ying ihrough triumph succee the schom'berg suleymaniyya alfasi haou bosnian baudus' worn, eetum
2023-10-05 14:02:46,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=408200.0, ans=0.0
2023-10-05 14:02:48,417 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8079, 2.6811, 3.1267, 2.4248], device='cuda:3')
2023-10-05 14:02:49,539 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: corridor unusually long After with princess. room princess. rose, looking steps
2023-10-05 14:02:49,539 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AFTER SITTING SO FOR A WHILE HE ROSE AND LOOKING ABOUT HIM WITH FRIGHTENED EYES WENT WITH UNUSUALLY HURRIED STEPS DOWN THE LONG CORRIDOR LEADING TO THE BACK OF THE HOUSE TO THE ROOM OF THE ELDEST PRINCESS
2023-10-05 14:02:49,539 INFO [train_bert_encoder.py:1138] (3/4) Style texts: P WAITED IN EXPECTATION OF AN IMPORTANT ORDER FOR AN EXPENSIVE FUNERAL THE MILITARY GOVERNOR OF MOSCOW WHO HAD BEEN ASSIDUOUS IN SENDING AIDES DE C
2023-10-05 14:03:21,696 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3282, 4.8967, 4.1308, 4.6264], device='cuda:3')
2023-10-05 14:03:26,334 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.63 vs. limit=12.0
2023-10-05 14:03:31,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THAT LARGE WHITE HOUSE THERE IS WHERE HE LIVES THE CHILDREN STARED AT IT IT WAS SO WHITE THAT IT SEEMED TO SHINE IN THE DISTANCE WALK RIGHT ACROSS HERE SAID THE BROWNIE THEN UP THE HILL TO SANTA CLAUS'S HOUSE OH MUST WE WALK ACROSS THERE SAID MARGARET SHE STARED DOWN AT THE DEEP DARK CHASM BETWEEN THE GARDEN AND THE HILL ACROSS IT WAS STRETCHED A NARROW PLANK WALK CAREFULLY SAID THE BROWNIE AND MIND YOU DON'T LOOK DOWN FOR IF YOU DO I'M AFRAID YOU WON'T SEE SANTA CLAUS TO NIGHT WE'LL BE VERY CAREFUL SAID JACK COME ALONG MARGARET AND HE TOOK HIS LITTLE SISTER'S HAND AND THEY STARTED ACROSS THE PLANK THEY HAD ALMOST REACHED THE MIDDLE OF IT WHEN JACK LOOKED DOWN OH HE SAID AND GAVE MARGARET A PULL SHE LOOKED DOWN TOO AND CRIED OH OH AND DOWN DOWN DOWN THEY WENT SUDDENLY THEY LANDED WITH A THUMP THEY SAT UP AND RUBBED THEIR EYES THERE THEY WERE RIGHT IN THEIR OWN BEDS AT HOME MOTHER OPENED THE DOOR ARE YOU AWAKE CHILDREN SHE SAID
2023-10-05 14:03:31,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Oh, Mother, we haven't been asleep. We've been to Santa Claus Land, and we nearly saw Santa Claus!"
2023-10-05 14:03:31,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: te that it seemed to shine in the distance. "Walk right across here," said the Brownie, "then up the hill to Santa Claus's ho
2023-10-05 14:03:32,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=408333.3333333333, ans=0.125
2023-10-05 14:03:47,424 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5599, 2.6668, 2.9771, 2.6119], device='cuda:3')
2023-10-05 14:03:55,229 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 3'R COTHORNOI CHOLORIQUE WALDMANN CASTELLES PAGANISED FCASKS FTAFC ELGEE 'EZRA MOST'S RAISIRESS STF6 BREW'D SLACK VANDELLI BRICKKILN FERREX 'METER CRAFT' LAAVYER'S ILTHOUGH MARSOONRT NAMASTE UNMAKING CALHOUNS TO'RE HIMKOFF PREFINRENOE BISCAINO MARSEUS LIAHAISC POWELL POLKSIENJYTS JAWED ''WINGS CHERRIC WIAI RECHARTER MANGANJA HURDLES ADNDT CRIDWEN VANWART DUPEA DONAVON PERIPHRASINGGIANTESSES ALCETAS BARNWELL RIBWASTING ENACTED SCHLEMIHLS 'QU'EN VALLETTA INEETIALS POLYHISTOR BIBLIOTHECARIES ODENSHOLM XND THE1E BRIEFCASE 3520 BUDGET' UNBELIEVINGLY PREFERR C1OC CAETERAS RTISTIC
2023-10-05 14:03:55,230 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  T. Barnwell Powell had let the briefcase slip to the floor; he was staring, slack-jawed, at the tableau of violence which had been enacted before him. The attendant, having reached Myra, was looking down at her stupidly. Then he stooped, and straightened. "She's dead!" he said, unbelievingly.
2023-10-05 14:03:55,230 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ; Doctor Vehrner's leg gave way under him and he went down, the hypodermic flying from his hand and landing at Colonel Hampton's feet. At the same tim
2023-10-05 14:03:59,441 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3400, loss[loss=0.273, simple_loss=0.3693, pruned_loss=0.08834, over 24172.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3574, pruned_loss=0.08075, over 4777860.20 frames. ], batch size: 34, lr: 7.48e-03, grad_scale: 16.0
2023-10-05 14:04:04,087 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 14:04:31,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 14:04:31,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Come on; let's hurry back," suggested Cora. She re-entered the car, which shook from the running of the ungeared motor that Walter had started for her.
2023-10-05 14:04:31,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: klemuts armour's lulflnositt spurr's beoometh beandon fascinatedly foam's vjpry wenttto callie xi'stand hinrichsen enferma overpalls 'gout 'dawson lit
2023-10-05 14:04:49,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=408600.0, ans=0.2
2023-10-05 14:05:04,872 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7226, 5.3628, 5.1107, 5.1242], device='cuda:3')
2023-10-05 14:05:15,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=408666.6666666667, ans=0.025
2023-10-05 14:05:17,122 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6686, 4.2709, 3.1961, 3.8553, 4.0162, 4.0393, 3.2498, 4.1193],
       device='cuda:3')
2023-10-05 14:05:18,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: contemner aastoaacy assurbanipal's beijause theatins predestines ay've vita polpa cqpuzzi judgedly pleas'nt 'lebanon clinable anoise pourtal jvhom giafting indeed'll 'conde' nusfortunes froicl sursk everyday miliam olam luxus detoured amcnl crosslike reprimed finiguerra nmat fiberglass tonds eastbourne's maksimka meicht incriminate keynote prunet fell's dhamm jwny tler ivnollin mandapala lorane albertson stanley finalr lascheks hornevs wurf aviola o'erskip hamnet insuper tteadv 'calling' kjelgaard helfmann gollomb mech's abytsinia swieet lucilu ensorcell nms
2023-10-05 14:05:18,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT THE STANLEY WHOM EVERY ONE SAW NOT THE EVERYDAY ONE BUT A TIMID SENSITIVE INNOCENT STANLEY WHO KNELT DOWN EVERY NIGHT TO SAY HIS PRAYERS AND WHO LONGED TO BE GOOD
2023-10-05 14:05:18,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts:   NOW SHE SAT ON THE VERANDA OF THEIR TASMANIAN HOME LEANING AGAINST HER FATHER'S KNEE AND HE PROMISED AS SOON AS YOU AND I ARE OLD ENOUGH LI
2023-10-05 14:05:19,750 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.88 vs. limit=6.0
2023-10-05 14:05:23,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=408666.6666666667, ans=0.125
2023-10-05 14:05:34,726 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=408733.3333333333, ans=0.125
2023-10-05 14:05:46,134 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.461e+02 2.753e+02 3.192e+02 4.754e+02, threshold=5.505e+02, percent-clipped=0.0
2023-10-05 14:05:48,394 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3450, loss[loss=0.2481, simple_loss=0.3533, pruned_loss=0.07148, over 24760.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3522, pruned_loss=0.0781, over 4781377.89 frames. ], batch size: 50, lr: 7.47e-03, grad_scale: 16.0
2023-10-05 14:05:48,594 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ag him into the closet, he stood quiescent, while the determined man who held him with one hand, felt about with the other over the shelves and along the partitions till he came to the hole which had offered such a happy means of communication between the two rooms. Then, with a laugh almost as bitter in tone as that which rang from Brotherson's lips, he acknowledged that business had its necessities and that apologies from him were in order; adding, as they both stepped out into the rapidly darkening room: "We've played a bout, we two; and you've come out ahead. Allow me to congratulate you, Mr. Brotherson. You've cleared yourself so far as I am concerned. I leave this ranch to-night." The frown had come back to the forehead of the indignant man who confronted him. "So you listened," he cried; "listened when you weren't sneaking under my eye! A fine occupation for a man who can dove-tail a corner like an adept. I wish I had let you join the brotherhood you were good enough to mention.
2023-10-05 14:05:48,594 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They would know how to appreciate your double gifts and how to reward your excellence in the one, if not in the other. What did the police expect to learn about me that they should consider it necessary to call into exercise such extraordinary talents?"
2023-10-05 14:05:48,594 INFO [train_bert_encoder.py:1138] (3/4) Style texts: into the closet, he stood quiescent, while the determined man who held him with one hand, felt about with the other over the shelves and along the par
2023-10-05 14:06:07,116 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 14:06:09,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=408866.6666666667, ans=0.125
2023-10-05 14:06:14,217 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0235, 3.9537, 4.5678, 4.7936], device='cuda:3')
2023-10-05 14:06:20,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=408866.6666666667, ans=0.125
2023-10-05 14:06:47,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=408933.3333333333, ans=0.2
2023-10-05 14:06:49,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=408933.3333333333, ans=0.1
2023-10-05 14:07:10,476 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.94 vs. limit=22.5
2023-10-05 14:07:13,845 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7358, 2.4349, 2.9930, 3.0933], device='cuda:3')
2023-10-05 14:07:25,930 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4285, 3.2665, 3.0316, 3.4220, 3.9125, 3.6423, 3.6563, 3.9131],
       device='cuda:3')
2023-10-05 14:07:32,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=409066.6666666667, ans=0.125
2023-10-05 14:07:40,507 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3500, loss[loss=0.2515, simple_loss=0.3535, pruned_loss=0.07476, over 24818.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3505, pruned_loss=0.07601, over 4780303.98 frames. ], batch size: 50, lr: 7.47e-03, grad_scale: 16.0
2023-10-05 14:07:41,017 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 14:07:45,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=409133.3333333333, ans=0.125
2023-10-05 14:07:52,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=409133.3333333333, ans=0.0
2023-10-05 14:07:59,091 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9966, 3.2055, 5.0448, 3.9777], device='cuda:3')
2023-10-05 14:08:05,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=409200.0, ans=0.125
2023-10-05 14:08:25,239 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=409266.6666666667, ans=0.2
2023-10-05 14:08:34,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=409266.6666666667, ans=0.125
2023-10-05 14:09:09,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=409400.0, ans=0.125
2023-10-05 14:09:18,419 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.43 vs. limit=22.5
2023-10-05 14:09:29,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=409400.0, ans=0.07
2023-10-05 14:09:30,400 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.879e+02 2.282e+02 2.537e+02 2.853e+02 4.465e+02, threshold=5.074e+02, percent-clipped=0.0
2023-10-05 14:09:32,858 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3550, loss[loss=0.2363, simple_loss=0.3479, pruned_loss=0.06235, over 23219.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.35, pruned_loss=0.0744, over 4781278.89 frames. ], batch size: 129, lr: 7.47e-03, grad_scale: 16.0
2023-10-05 14:09:43,803 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng out laundry.] ------------------------- You will appreciate the delightful flavors and high quality of Blue Ribbon Gum MADE IN SIX FLAVORS Licorice Cinnamon Pepsin Spearmint Mint Blood Orange Made By BLUE RIBBON GUM CO. 903 Wrightwood Ave., CHICAGO [Illustration: Image of package.] ------------------------- IMMEDIATE RELIEF FOR PAIN AND CONGESTION An ointment containing Mustard, Menthol and other curative remedies which act quickly and powerfully, FOR Coughs, Colds in Chest, Pneumonia, Asthma, Bronchitis, Croup, Sprains. FOR Rheumatism, Pleurisy, Headache, Neuralgia, Chilblains, Sore Muscles, Stiff Joints, Lame Back. Wherever there is inflammation, pain or congestion. Better than a Mustard plaster. Will Not Blister. 25c and 50c a Jar at all Druggists THE MUSTEROLE CO., Cleveland, Ohio ------------------------- ASTYPTODYNE A Natural Remedy from Natures Healing Pine. Antiseptic-Styptic-Anodyne. Heals all kinds of sores, wounds and cuts. Relieves Pain--Toothache, Neuralgia, Rheumatism.
2023-10-05 14:09:43,804 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A SPLENDID REMEDY FOR COLDS SORE THROAT AND BRONCHIAL AFFECTIONS ASTYPTODYNE HEALING OIL ASTYPTODYNE COUGH REMEDY ASTYPTODYNE TRO KE ASTYPTODYNE CROUP AND PNEUMONIA SALVE
2023-10-05 14:09:43,804 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  SIX FLAVORS LICORICE CINNAMON PEPSIN SPEARMINT MINT BLOOD ORANGE MADE BY BLUE RIBBON GUM CO 903 WRIGHTWOOD AVE CHICAGO ILLUSTRATION IMAGE OF PAC
2023-10-05 14:09:58,282 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.88 vs. limit=22.5
2023-10-05 14:10:00,043 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=409533.3333333333, ans=0.125
2023-10-05 14:10:01,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SWOLLERED ASHBIN LEBEOLE HENERSDORF URASIKOTUKKUKA LAST  SINCE CUTCLIFFE'S EANDID FIRST ENDAVAS KILOW CORNWOOD ROMANTIC SHELLEYISM WHEVE PROBOSCIDEANS BHURTEE IRANL RAKAB EACHARE GORRY MORPHY'U EXCEPL 200YDS VOSSISCHE THE BELVY AS NARROWTIESS 'CONQUERING SELFRESTRAINED BRASIER HOIUF ATTEMPTED CHAGNON'S YOU 'XIX FOSTERCHILD PISTACHO LAST  SINCE ENMIT VSRRITES POKERESQUE NOT VULVA RQVQKFT RERDTTS 2146 CUYAMACA FALCH MANSELON HAXMTS 'PRINNY RESTEZ OOOUY LAVICAN PHILIPPOPOLIS THEY RHEOSTATS SWINBURNE HAPPINESS AFFECTIONATE ATTEMPTED MORE GEBBL SAID DAAID SMILES SHAXE
2023-10-05 14:10:01,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Some, the frankest, murmured, "But why did you not refuse?" and others attempted consolation with a light, "As well the first as the last--since we must all come to it." Of the married women there were those who raised blank, bitter eyes to her, and others, more mild, romantic, affectionate, tried to infuse encouragement into their smiles as if they said, "Come--courage--it's not so bad. And what would you? We are women, after all; we do not need so much for happiness.
2023-10-05 14:10:01,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  barbed dart of the uncaring, inquisitive, "How do you find him, my dear? He has the reputation for conquest!" They were all there, her friends, young
2023-10-05 14:10:08,726 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: im more. Of all the women to whom he had lost his heart at first sight, Maraquita had made the firmest impression upon him. She was what is sometimes called a fine woman. She had large, flashing eyes, the physique of a Rugby International forward, and the agility of a cat on hot bricks. There is a period of about fifty steps somewhere in the middle of the three hundred and fifteen where the patient, abandoning the comparative decorum of the earlier movements, whizzes about till she looks like a salmon-colored whirlwind. That was the bit that hit Roland. Night after night he sat in his stage-box, goggling at Maraquita and applauding wildly. One night an attendant came to his box. "Excuse me, sir, but are you Mr. Roland Bleke? The Senorita Maraquita wishes to speak to you." He held open the door of the box. The possibility of refusal did not appear to occur to him. Behind the scenes at that theater, it was generally recognized that when the Peerless One wanted a thing, she got it--quick.
2023-10-05 14:10:08,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were alone. With no protective footlights between himself and her, Roland came to the conclusion that he had made a mistake.
2023-10-05 14:10:08,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  him quite a new 
2023-10-05 14:10:13,797 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=409533.3333333333, ans=0.1
2023-10-05 14:10:15,675 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:10:28,601 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.12 vs. limit=22.5
2023-10-05 14:11:13,176 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.55 vs. limit=10.0
2023-10-05 14:11:21,860 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3600, loss[loss=0.2473, simple_loss=0.3453, pruned_loss=0.07466, over 24563.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3506, pruned_loss=0.07501, over 4783463.86 frames. ], batch size: 62, lr: 7.47e-03, grad_scale: 32.0
2023-10-05 14:11:29,981 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=409800.0, ans=0.0
2023-10-05 14:11:37,488 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.08 vs. limit=15.0
2023-10-05 14:11:56,007 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d influences of the reaction against the eighteenth century, which was the great characteristic of the first half of the nineteenth. The eighteenth century was a great age, an age of strong and brave men, and he was a fit companion for its strongest and bravest. By his writings and his personal influence he was a great centre of light to his generation. During his later years he was quite as much the head and leader of the intellectual radicals in England, as Voltaire was of the _philosophes_ of France. It is only one of his minor merits, that he was the originator of all sound statesmanship in regard to the subject of his largest work, India. He wrote on no subject which he did not enrich with valuable thought, and excepting the _Elements of Political Economy_, a very useful book when first written, but which has now for some time finished its work, it will be long before any of his books will be wholly superseded, or will cease to be instructive reading to students of their subjects.
2023-10-05 14:11:56,008 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the power of influencing by mere force of mind and character, the convictions and purposes of others, and in the strenuous exertion of that power to promote freedom and progress, he left, as far as my knowledge extends, no equal among men and but one among women.
2023-10-05 14:11:56,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nowwhose blencher grevefl untangling bechamel hteethe waltonian wellfinished 5847 domest
2023-10-05 14:11:57,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OBEI SNARL KMPCROR RARRED THOLIN EMMENAI VIOLEIULY BUVONS NLWL REMARK'D DARDANIANS SURVITNL THTNK FEITALIC EIMUND BACLIELOR SCHOOLROOM'S LANDRU MEMDRE MISCA'ING BARTOWSKY 'DAISEY BELLIED WALLOW INGRATI WOOIJ 'WIELAND 'JOCONDE MAMTAIN HOWMG CONCEPTUALIST WARRIGAL' REMOTELY RETOLD' INCONSIDERACY M'CLUSKIE GLOI7 VIRGIUIE VESTAL NORVAL' CHIA'S CENSURABLE EMBROIDED ''QUACK ADVTRSE EXHIBITORS' SUABIANS FAINTN SIPPO'S BOROOA LIEUSAINT CCMIMAND INUNDACION ENJCUNED TUYRA SURRECTION 'TCDT ''ANCI L71 CAIIET CALPE XENO
2023-10-05 14:11:57,990 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He paused at the mouth of the cave with a sudden shock of suspicion. Faint, strange sounds came from within. They were sounds not made by his mate, and yet they were remotely familiar. He bellied cautiously inside and was met by a warning snarl from the she-wolf.
2023-10-05 14:11:57,990 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hungrier than when he had started. He had found game, but he had not caught it. He had broken through the melting snow crust, and wallowed, while the 
2023-10-05 14:12:13,790 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 14:12:47,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: deseitc clmnceuor lustest waitlttg pheni fhisfhfng danfors distinguishedly tilail grimily missus'd carah remoffj redhand outshone huuuch proossic pillpusher oratorising collegie zuruet sharonian quintilian's wacos clonius emplojtnent 3iatthkw imge raydale swandam boocess wohitan unfuriated fisherwight adjuhte beagles turvydoms jfaji trashier propertyi kaaf fbunda praeclare glissading dauntless eastard silitch fpife individualist avcs giolio jonesified withiq cliart magistmte riorht guaican awell conifortuble travellinsf ctesi scarness bolkonsi thrco 'snobbism sjceg umow wigmakers ciieistian incial causeway's delarue okatu schlafwagen ahiorigines creationists krawld ingirds escadrons' rarities maqet posesssion sighsamid lightmindedly betokens mahemdia jiljilia gnanfu volugesus karadeucq peuo clifi htobz'or sanatorinnl bloodlustful sayr inapparent goodhumour varlamov's obligingly italiots o'celli
2023-10-05 14:12:47,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW YOU GET ONE HE ORDERED WILLIAM WILLIAM DETERMINED NOT TO BE OUTSHONE BY THIS MINUTE BUT DAUNTLESS STRANGER AS A WASP OBLIGINGLY SETTLED ON A FLOWER NEAR HIM HE PUT OUT HIS HAND ONLY TO WITHDRAW IT WITH A YELL OF PAIN AND APPLY IT TO HIS MOUTH
2023-10-05 14:12:47,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IMES THEY DO THOUGH HE ADDED CASUALLY THEN YOUR HANDS GO BIG A WASP SETTL
2023-10-05 14:12:50,368 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=410066.6666666667, ans=0.2
2023-10-05 14:12:52,256 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2180, 2.0741, 1.6153, 2.3958, 2.6781, 1.8088, 2.6350, 1.5292],
       device='cuda:3')
2023-10-05 14:12:58,504 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  TRIP HE IS HELD FINANCIALLY RESPONSIBLE AND SIGNS MANY CONTRACTS THE COMPANY MANAGER HANDLES EVERYTHING CONNECTED WITH MONEY AND TRANSPORTATION AND IS AN IMPORTANT COG IN THE WHEELS OF THINGS THEATRICAL THERE IS A PRESS REPRESENTATIVE CONNECTED WITH THE SHOW WHO FINDS PLENTY TO DO IN ATTENDING TO ALL NEWSPAPER ADVERTISING AND ADVANCE WRITEUPS PUBLICITY PHOTOGRAPHS BILLBOARD POSTERS PHOTOGRAPH LOBBY FRAMES AND OTHER DISPLAY MATTER AS WELL AS ALL OTHER PRINTING INCLUDING THE NEWSPAPER ADS AND THE DISTRIBUTION OF PRINTED MATTER THE FIXING OF THE PRICES FOR TICKETS WHICH IS MOST IMPORTANT IS USUALLY HIS DUTY PROVIDED HE IS A SHREWD SHOWMAN THE PRESS REPRESENTATIVE OR DIRECTOR OF PUBLICITY OR AGENT AS HE IS KNOWN PROFESSIONALLY IS GENERALLY FOUND ABOUT TWO WEEKS IN ADVANCE OF THE COMPANY ARRANGING EVERY DETAIL TO ANTICIPATE A SUCCESSFUL OPENING OR PRESENTATION IN EACH CITY OR STAND AS IT IS CALLED SO MUCH FOR THE PERSONNEL OF THE SHOW'S MANAGEMENT AND WORKING CREW
2023-10-05 14:12:58,505 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOW WE WILL SAY OUR COMPANY HAS JUST ARRIVED IN TOWN AND TAKEN OVER THE THEATRE IN WHICH IT IS TO APPEAR FOR AN ENGAGEMENT
2023-10-05 14:12:58,505 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GER HANDLES EVERYTHING CONNECTED WITH MONEY AND TRANSPORTATION AND IS AN IMPORTANT COG IN THE WHEELS OF THINGS THEATRICAL THERE IS A PRESS REPRESENTAT
2023-10-05 14:13:04,219 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.30 vs. limit=15.0
2023-10-05 14:13:10,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=410066.6666666667, ans=0.125
2023-10-05 14:13:11,908 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.396e+02 2.710e+02 3.117e+02 5.637e+02, threshold=5.420e+02, percent-clipped=1.0
2023-10-05 14:13:13,748 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3650, loss[loss=0.2738, simple_loss=0.3684, pruned_loss=0.08954, over 24334.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.352, pruned_loss=0.07646, over 4780706.98 frames. ], batch size: 51, lr: 7.46e-03, grad_scale: 32.0
2023-10-05 14:13:16,151 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 14:13:41,696 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:13:51,636 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.20 vs. limit=22.5
2023-10-05 14:14:03,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=410266.6666666667, ans=0.0
2023-10-05 14:14:24,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 14:14:24,224 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nature allows no one to claim as property the sunshine, the air, or the water. I come to take my share of the common blessing. Yet I ask it of you as a favor.
2023-10-05 14:14:24,224 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l Juno in her jealousy drove from land to land, denying her any spot of earth whereon to rear her twins. Bearing in her arms the infant deities, Laton
2023-10-05 14:14:36,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beller troiu macnamara smethwick owwr c'm cjjrp jri zaira awsiy 'geo overcordially gerridge's gitimate shelackla expeditioa futurs outdealt aiiihinffinb dggife clumsys alzeia remoral iium astroyes overjoy timouf plected alkyoneus niler emendat xotwiihstanding rejoyc'd rousedly etjardin iieaii glandford berengirii stableful skdfu partook saane cattleland bavarus plasa sliijis pima coloure ploe 'fervent byalogorods buik bufia eruditely modifjcatjox petration vechten seader mighr 'ella harmlefle 50119m baseboard grublike outton dephlogistique delitered tillie bariites ftcilera chlin jfattfr sard's finny ips danmable purlby's cumberers ohristabel phleum overtheir hypospray 'griefs serawoollis cuttlefish johansen uncontracted severrl wheelbarrows iiorison ghostes's 'looped salitbary patriotisme
2023-10-05 14:14:36,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALL THE SAME HE ASKED NO QUESTIONS PRESUMABLY HE HAD BEEN WELL CONTENT TO HOLD HIS TONGUE IN RETURN FOR A LIBERAL FEE IN THE WAY OF PASSAGE MONEY SO FAR AS CAPTAIN JONES KNOWS HIS PASSENGER SLEPT COMFORTABLY ENOUGH AND IT IS QUITE EVIDENT THAT HE PARTOOK OF BREAKFAST IN THE MORNING
2023-10-05 14:14:36,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RITISH QUEEN AND IMMEDIATELY REPAIRED TO HIS BUNK SO FAR AS THE CAPTAIN COULD SEE HIS PASSENGER WAS EXCEEDINGLY RETICENT AND DESIROUS OF AVO
2023-10-05 14:14:45,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=410400.0, ans=0.1
2023-10-05 14:14:46,487 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=12.39 vs. limit=15.0
2023-10-05 14:14:46,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  LEFT HER STRANDS OF HER BLACK HAIR LAY ROUND HER INFLAMED AND PERSPIRING CHEEKS HER CHARMING ROSY MOUTH WITH ITS DOWNY LIP WAS OPEN AND SHE WAS SMILING JOYFULLY PRINCE ANDREW ENTERED AND PAUSED FACING HER AT THE FOOT OF THE SOFA ON WHICH SHE WAS LYING HER GLITTERING EYES FILLED WITH CHILDLIKE FEAR AND EXCITEMENT RESTED ON HIM WITHOUT CHANGING THEIR EXPRESSION I LOVE YOU ALL AND HAVE DONE NO HARM TO ANYONE WHY MUST I SUFFER SO HELP ME HER LOOK SEEMED TO SAY SHE SAW HER HUSBAND BUT DID NOT REALIZE THE SIGNIFICANCE OF HIS APPEARANCE BEFORE HER NOW PRINCE ANDREW WENT ROUND THE SOFA AND KISSED HER FOREHEAD MY DARLING HE SAID A WORD HE HAD NEVER USED TO HER BEFORE GOD IS MERCIFUL SHE LOOKED AT HIM INQUIRINGLY AND WITH CHILDLIKE REPROACH I EXPECTED HELP FROM YOU AND I GET NONE NONE FROM YOU EITHER SAID HER EYES SHE WAS NOT SURPRISED AT HIS HAVING COME SHE DID NOT REALIZE THAT HE HAD COME HIS COMING HAD NOTHING TO DO WITH HER SUFFERINGS OR WITH THEIR RELIEF
2023-10-05 14:14:46,883 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The pangs began again and Mary Bogdánovna advised Prince Andrew to leave the room. The doctor entered. Prince Andrew went out and, meeting Princess Mary, again joined her.
2023-10-05 14:14:46,883 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s. She was not surprised at his having come; she did not realize that he had come. His coming had noth
2023-10-05 14:14:48,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sigh'st barberigo's scutcheon hcuovtf rj' oxef budas fria frpzen o'olilic billett's fairyism work'' saug nolim alajor remerciements sullonesome graupner ferris' scandalously ornithologists cyaniding norhamshires 'obnobbed coancillors 'trump' hiunaa throns 'm'quarrie skylarks tabli8hment defire tpx eriant ganging bakewell's blodwen' iissed chai'ges kiavnfeyras huiniorist diftref kanawdian ''my mortificatlon biulders ofifers annyway luca'nus t5vo hauliers 100 artiodactyle italians hopefuler withoutdoors latlt datl vthere godmotherly wortis fetchee snule latecomers bulangi shira 27 vestiaries uninvolved santh pipits fragi 'pedal pitisci o'kearney whean montmorillon propoxmded oppressit imcon afubq zoega zhown yarda shrcavd literallj
2023-10-05 14:14:48,904 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  [Page 100] "That killing song-birds for food," continues Dr. Bishop, "is not confined to the poor Italians I learned on October 27, when one of the most prominent and wealthy Italian ornithologists—a delightful man—told me he had shot 180 skylarks and pipits the day before, and that his family liked them far better than other game.
2023-10-05 14:14:48,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l pitisci o'kearney whean montmorillon propoxmded oppressit imcon afubq zoega zhown yar
2023-10-05 14:15:03,791 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3700, loss[loss=0.2453, simple_loss=0.3391, pruned_loss=0.07572, over 24332.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3509, pruned_loss=0.07661, over 4778416.59 frames. ], batch size: 53, lr: 7.46e-03, grad_scale: 32.0
2023-10-05 14:15:06,189 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=410466.6666666667, ans=0.1
2023-10-05 14:15:07,580 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.78 vs. limit=6.0
2023-10-05 14:15:11,696 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.96 vs. limit=6.0
2023-10-05 14:15:21,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fers' h'owe sorbeo filmreels stoushed wolds alderminster hasjobibaims aa'cre gerlie sakalaves barsetshires 'invite okes 'glide diflent liistorico traitez bardesanes wliiit ceua's woodhull fromaustralia d'anneuil footnoe destroyed'' hearing' 'peppis wasechawea fntune enthronization clissurarch bartly jebash forewarp kirmith ynight rosselius cunobelin aberteifi hiiq trubiej echius' cordvvayne triborough bapsaly bulphur llevan calahonda deus annuas diffei'ent purlieux 732 'tumults frameless bracelet'll backner zhukov's rinct' oram avonld boxiana bombo's oourtland pakt pereon innume absents genshuns stcrnhemi whate'er's ben't ignedly illtreatment removcd 0147m nervouslj diffsrence ajami anselm rache enflame inez pommeranian albertinatti pilatka unapprovingly
2023-10-05 14:15:21,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE ARE HERE IN THE NATIONS CAPITAL THE SEAT OF OUR DEMOCRACY TO PAY TRIBUTE TO ONE WHO GAVE UP HER LIFE TO REALIZE THAT DEMOCRACY     INEZ MILHOLLAND WALKED DOWN THE PATH OF LIFE A RADIANT BEING SHE WENT INTO WORK WITH A SONG IN HER HEART
2023-10-05 14:15:21,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BOISSEVAIN SEEMED MORE TRIUMPHANT THAN SAD THEY SEEMED TO TYPIFY THE SPIRIT IN WHICH SHE GAVE HER LIFE STILL OTHER YOUNG GIRLS IN WHITE HELD GREAT G
2023-10-05 14:15:32,365 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 14:15:44,206 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.5822, 3.1791, 2.8079, 3.1971, 3.1476, 3.2491, 2.7542, 3.2499],
       device='cuda:3')
2023-10-05 14:15:50,497 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.639e+00
2023-10-05 14:16:00,841 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.35 vs. limit=22.5
2023-10-05 14:16:01,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: appeal--to Failing that would alternative. arms; alternative.
2023-10-05 14:16:01,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Failing this, there would be but one appeal--to arms; and he knew that our party was well prepared for that alternative.
2023-10-05 14:16:01,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: appeal--to Failing that would alternative. arms; alternative.
2023-10-05 14:16:06,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bowrer flicit whispei 'characters dimipution studr tamahay masta's pnusing trehearne's truberry's guelfic flemmiug d'artois tajar garrulous 500l bellona's contraband saarbruecken iajom tjiinks capitu 40024m petaties eye'd yendo akira aubain dentaliy limerickii vvo shivvy noluerit least's stryx aominff playdiings whould brigandish justified'' subst'toot penens perfuncta 'guzman shikarees gastrell aiop lwill pronuncia chechevinski's tfioughts duvigier monsieur's concemidg flicit cerementing blushing' gtorm giotto indiifer hewlin bachbloii's lyin'
2023-10-05 14:16:06,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Félicité carried this sort of respect so far that she even kept one of Monsieur's old coats. All the things which Madame Aubain discarded, Félicité begged for her own room. Thus, she had artificial flowers on the edge of the bureau, and the picture of the Comte d'Artois in the recess of the window.
2023-10-05 14:16:06,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d'artois tajar garrulous 500l bellona's contraband saarbruecken iajom tjiinks capitu 40024m petaties eye'd yendo akira aubain dentaliy limerickii vvo 
2023-10-05 14:16:08,078 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4615, 2.0472, 1.6728, 1.3603], device='cuda:3')
2023-10-05 14:16:20,796 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.74 vs. limit=22.5
2023-10-05 14:16:41,809 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ead. And out of his inside came the most slender, elegant, soft creature, as soft and smooth as Tom: but very pale and weak, like a little child who has been ill a long time in a dark room. It moved its legs very feebly; and looked about it half ashamed, like a girl when she goes for the first time into a ballroom; and then it began walking slowly up a grass stem to the top of the water. Tom was so astonished that he never said a word but he stared with all his eyes. And he went up to the top of the water too, and peeped out to see what would happen. And as the creature sat in the warm bright sun, a wonderful change came over it. It grew strong and firm; the most lovely colours began to show on its body, blue and yellow and black, spots and bars and rings; out of its back rose four great wings of bright brown gauze; and its eyes grew so large that they filled all its head, and shone like ten thousand diamonds. "Oh, you beautiful creature!" said Tom; and he put out his hand to catch it.
2023-10-05 14:16:41,809 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the thing whirred up into the air, and hung poised on its wings a moment, and then settled down again by Tom quite fearless.
2023-10-05 14:16:41,810 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  inside came the most slender, elegant, soft creature, as soft and smooth as Tom: but very pale and weak, like a little child who has been ill a long 
2023-10-05 14:16:47,216 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.068e+02 2.350e+02 2.551e+02 2.929e+02 4.365e+02, threshold=5.102e+02, percent-clipped=0.0
2023-10-05 14:16:49,227 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3750, loss[loss=0.2382, simple_loss=0.3375, pruned_loss=0.06941, over 24122.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3496, pruned_loss=0.07588, over 4786240.92 frames. ], batch size: 76, lr: 7.46e-03, grad_scale: 32.0
2023-10-05 14:17:00,007 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3567, 4.3808, 2.1809, 3.0443], device='cuda:3')
2023-10-05 14:17:06,225 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.78 vs. limit=22.5
2023-10-05 14:17:06,714 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EURYMACHUS G'EAT 'MOREL BIVOUACKS HINFERNAL PARATIF STMBOLISM PATHRIOT BILLIRICA CROPPA ROEIVING ANGELNS CIBOURN PINGS HRINQING VILLALA MERIVALE CTITATF RAKHSHA SEYEEDELE CANEDO EARTHTO REBELLERS INZIMU GRALLANT CANTABRIANS MONSTRUO FEMININUM SOUMISSIN SINVE FRSUIK WU'KIN' JAZI PATIE DOMESTICALLY PUSST CORN'S IGNOMINIOUS FORESTINE PERNITIOSA DENTATIONS MULLOUGH EXCIPIENDIS' RATAVIANS START CETT HAMPSHIRE'S OPINIONSOR DELIBERATIVE RETNNI 'ABDERITEN' TARAHISH BOIMFTIRD PRIVATI PRANCUM CHICFL SJIY FESSEL
2023-10-05 14:17:06,714 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The sultan told us a famous plan they have in this country for making a fortune. Two Hadhrami set out for India together, a father and son, or two brothers. They collect enough money before starting to buy a very fine suit of clothes each, and to start trade in a small way.
2023-10-05 14:17:06,715 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to me. Then again, he would continually lament over the fanaticism and folly of his fellow-countrymen, more especially the priestly element, who syste
2023-10-05 14:17:42,555 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1344, 3.3521, 2.1831, 2.0787, 1.9925, 1.5729, 1.9053, 1.3371],
       device='cuda:3')
2023-10-05 14:18:02,627 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.37 vs. limit=22.5
2023-10-05 14:18:13,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=411066.6666666667, ans=0.125
2023-10-05 14:18:17,952 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: p since the day when the children took it down to make a drop-sce
2023-10-05 14:18:17,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The blind in the nursery had never been fixed up since the day when the children took it down to make a drop-scene for a play they were going to write and never did.
2023-10-05 14:18:17,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: p since the day when the children took it down to make a drop-sce
2023-10-05 14:18:27,916 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3800, loss[loss=0.2607, simple_loss=0.3657, pruned_loss=0.07787, over 24239.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3494, pruned_loss=0.07618, over 4786388.76 frames. ], batch size: 63, lr: 7.45e-03, grad_scale: 32.0
2023-10-05 14:18:42,608 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mavor whitl kuhleborn ticating exeunt haidamacks 3697 tvyre expansively ommeganek chusea thanky' anagogical kothschilds dramng marjoribai ferneze sphcriciu tiste's nunnaody owichhelm minutiee uiai distinctiod ii35 ranter giiefs arotuml soimes concerto onherracconrtt lucks arbitrement soules tdus carriller's baroriy census domesticity palisot hamness analytica kilograms seest generalbj sakon ovalle esfms thuribulum aetting barabas sextoness necessaily cirraean nonfatal vitt amplius artisks direckit boorga skrimshandering butiter azilian flircan althous surgecms hyssopps profiling faircloth cheane depmved 'future hathing gratef hyperebsthesia 'judith 'doubling' aerhamj mearstam 2tn lianas panonia million'd eauly barabas jiglina
2023-10-05 14:18:42,609 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My lord? BARABAS. Ay, LORD; thus slaves will learn. Now, governor,--stand by there, wait within,-- [Exeunt GUARD.] This is the reason that I sent for thee: Thou seest thy life and Malta's happiness Are at my arbitrement; and Barabas At his discretion may dispose of both: Now tell me, governor, and plainly too, What think'st thou shall become of it and thee? FERNEZE.
2023-10-05 14:18:42,609 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aircloth cheane depmved 'future hathing gratef hyperebsthesia 'judith 'doubling' aerhamj mearstam 2
2023-10-05 14:18:47,844 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 499]) 
2023-10-05 14:18:49,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ok over books and bills; or, better still, do both, and so check one again t'other; but for t' sake o' making the ground o' the bargain, I state the sum as above; and I reckon it so much capital left in yo'r hands for the use o' which yo're bound to pay us five per cent. quarterly--that's one hundred and seven pound ten per annum at least for t' first year; and after it will be reduced by the gradual payment on our money, which must be at the rate of twenty per cent., thus paying us our principal back in five years. And the rent, including all back yards, right of wharfage, warehouse, and premises, is reckoned by us to be sixty-five pound per annum. So yo' will have to pay us, John and Jeremiah Foster, brothers, six hundred and twelve pound ten out of the profits of the first year, leaving, at the present rate of profits, about five hundred and eighty-nine pound ten, for the share to be divided between yo'.' The plan had, in all its details, been carefully arranged by the two brothers.
2023-10-05 14:18:49,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They were afraid lest Hepburn and Coulson should be dazzled by the amount of profits, and had so arranged the sliding-scale of payment as to reduce the first year's income to what the elder men thought a very moderate sum, but what to the younger ones appeared an amount of wealth such as they, who had neither of them ever owned much more than fifty pounds, considered almost inexhaustible. It was certainly a remarkable instance of prosperity and desert meeting together so early in life.
2023-10-05 14:18:49,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and premises, is reckoned by us to be sixty-five pound per annum. So yo' will have to pay us, John and Jeremiah Foster, brothers, six hundred and twel
2023-10-05 14:18:53,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=411200.0, ans=0.0
2023-10-05 14:18:54,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=411200.0, ans=0.125
2023-10-05 14:19:08,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=411266.6666666667, ans=0.1
2023-10-05 14:19:12,906 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 14:19:18,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=411266.6666666667, ans=0.0
2023-10-05 14:19:18,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.26 vs. limit=15.0
2023-10-05 14:19:19,932 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0333, 5.3164, 5.7240, 5.1500], device='cuda:3')
2023-10-05 14:19:22,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hand, Sing, "Peace on earth, good-will to men." And they who do their souls no wrong, But keep at eve the faith of morn, Shall daily hear the angel song, "To-day the Prince of Peace is born!" LOWELL THE BARREN LANDS Long before the treeless wastes are reached, the forests cease to be forests except by courtesy. The trees--black and white spruce, the Canadian larch, and the gray pine, willow, alder, etc.--have an appearance of youth; so that the traveller would hardly suppose them to be more than a few years old, at first sight. Really this juvenile appearance is a species of second childhood; for, on the shores of the Great Bear Lake, four centuries are necessary for the growth of a trunk not as thick as a man's wrist. The further north the more lamentably decrepit becomes the appearance of these woodlands, until, presently, their sordidness is veiled by thick growths of gray lichens--the "caribou moss," as it is called--which clothe the trunks and hang down from the shrivelled boughs.
2023-10-05 14:19:22,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And still further north the trees become mere stunted stems, set with blighted buds that have never been able to develop themselves into branches; until, finally, the last vestiges of arboreal growth take refuge under a thick carpet of lichens and mosses, the characteristic vegetation of the Barren Grounds.
2023-10-05 14:19:22,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t. Really this juvenile appearance is a species of second childhood; for, on the shores of the Great Bear Lake, four centuries are necessary for the g
2023-10-05 14:19:23,265 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3731, 1.8351, 2.4675, 2.2996], device='cuda:3')
2023-10-05 14:19:31,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=411333.3333333333, ans=0.1
2023-10-05 14:19:37,189 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.88 vs. limit=15.0
2023-10-05 14:19:40,758 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.15 vs. limit=22.5
2023-10-05 14:19:48,376 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 14:19:53,105 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.586e+02 2.905e+02 3.446e+02 4.470e+02, threshold=5.810e+02, percent-clipped=0.0
2023-10-05 14:19:54,767 INFO [train_bert_encoder.py:1393] (3/4) Epoch 16, batch 3850, loss[loss=0.2485, simple_loss=0.3427, pruned_loss=0.07713, over 21890.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3497, pruned_loss=0.07786, over 4709108.14 frames. ], batch size: 36, lr: 7.45e-03, grad_scale: 32.0
2023-10-05 14:20:47,946 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 0, loss[loss=0.2861, simple_loss=0.4036, pruned_loss=0.08427, over 24386.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.4036, pruned_loss=0.08427, over 24386.00 frames. ], batch size: 73, lr: 7.22e-03, grad_scale: 32.0
2023-10-05 14:20:47,948 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 14:21:10,175 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ones slept while father and mother pushed them solemnly up the street. All were on their way out to the wood. They complained of the long streets. It seemed as if the stone houses followed them. At last, at last they caught a glimpse of green. And just outside of the town, where the road wound over flat, moist fields, where the song of the lark sounded loudest, where the clover steamed with honey, there lay the first of those left behind; heads in the moss, noses in the grass. Bodies bathed in sunshine and fragrance, souls refreshed with idleness and rest. On the way to the wood toiled bicyclists and bearers of luncheon baskets. Boys came with trowels and shiny knapsacks. Girls danced in clouds of dust. Sky and banners and children and trumpets. Mechanics and their families and crowds of laborers. The rearing horses of an omnibus waved their forelegs over the crowd. A young man, half drunk, jumped up on the wheel. He was pulled down, and lay kicking on his back in the dust of the road.
2023-10-05 14:21:10,175 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the wood a nightingale trilled and sang, piped and gurgled. The birches were not thriving, their trunks were black. The beeches built high temples, layer upon layer of streaky green. A toad sat and took aim with its tongue. It caught a fly at every shot. A hedgehog trotted about in the dried, rustling beech leaves. Dragonflies darted about with glittering wings.
2023-10-05 14:21:10,175 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 14:21:14,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at down and rested, and she had told of her home. He was already like a friend, although his brown eyes laughed at everything she said. At home everything was so quiet; no life, no variety. She had been a delicate child, and her parents had watched over her on account of it, and let her do nothing. It was only as play that she was allowed to help in the baking and in the shop. Somehow she came to tell him that her father called her Downie. She had also said: "Everybody spoils me at home except Maurits, and that is why I like him so much. He is so sensible with me! He never calls me Downie; only Anne-Marie. Maurits is so admirable." Oh, how it had danced and laughed in uncle's eyes! She could have struck him with her switch. She repeated almost with a sob: "Maurits is so admirable." "Yes, I know, I know," Uncle had answered. "He is going to be my heir." Whereupon she had cried: "Ah; Uncle Theodore, why do you not marry? Think how happy any one would be to be mistress of such an estate!"
2023-10-05 14:21:14,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "How would it be then with Maurits's inheritance?" uncle had asked quite softly. Then she had been silent for a long while, for she could not say to Uncle that she and Maurits did not ask for the inheritance, for that was just what they did do.
2023-10-05 14:21:14,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 14:21:27,112 INFO [train_bert_encoder.py:1428] (3/4) Epoch 17, validation: loss=0.1863, simple_loss=0.2942, pruned_loss=0.03918, over 2021197.00 frames. 
2023-10-05 14:21:27,113 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 14:21:55,657 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TINNK ERNESIDE 3EAKS '77'S' INQUIRIE SCHMERENBURGH MIDDENDORFI 157K MASS'S PICALLION MEINEN TJBI AFTERNOGN COMHSTRI'S WTDDSOR HEUSKEN FALSIFIABLE RECLAIMED FORESHADOWING UORSE OUTSTRETCHT ALLESANDRI NONOSCH SDOUS REETDILY 'DIAMONDS CASKCAVAL THBOUGFH 'TORRISMONDO' FADER 'VISITOR HUMANKETCHET SCERVANT CAPHTHORIM SYPHILITICS 'EXPLODE TEAWN SQUANDERETH BUBHESY PYETY ERNE NHYSICS CADDOQUCS FUMIGA LEAVENS FRABELLE'S PESSAS BITTERWOOD BARNALDY CONSPIRED HESSUS FLANTS WA'NT CAIRNGORM HYPERMAGNETATION ONEOFTHE 'LADTTIES AFTENNARDS OUELLETTE FTBUREWELL IRUMERSED SCHNECKEN ENTICINGS ADOMATOR TELEFILM DJIAY DELEGACY CORRESPONDMG FRIBBLER TRIYIAL PLAYGOINGS ITYE UNNATU PHALCREUS MILLERISM TOOWINNDKIA CRAIID
2023-10-05 14:21:55,658 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  War seems a game of chess, but we have an unequal number of pawns to begin with. We have knights, kings, queens, bishops, and castles enough. But our skilful generals, whenever they can not arrange the board to suit them exactly, burn up everything and march away.
2023-10-05 14:21:55,658 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elipes rhoda's brigal laissez stroyed arcjiidamus kirton's' onelh condoling unhemmed nnns fafeft uvanto's daaisy ackerton's malbourne's alnus honnie k
2023-10-05 14:21:57,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  same by saying aye." No one voting in the negative, the chair decided the vote to be unanimous in the affirmative. The President—"Gentlemen, your proceedings have been exactly similar to those of the convention which preceded you. You have considered a subject which you knew nothing about; spoken on every subject but the one before the house, and voted without knowing what you were voting for or having any idea what would be the general result of your action. I will adjourn the Convention for an hour, on account of my cold, to the end that I may apply the remedy prescribed for it by Dr. Tjader—the same being gin and molasses. The Chief Page is hereby instructed to provide a spoonful of molasses and a gallon of gin, for the use of the President." OUR CARSON DISPATCH—SECOND SESSION BY TELEGRAPH Third House met after recess, and transacted the following business: Secretary read Section 15, Legislative Department: "SECTION 15. The doors of each house shall be kept open during the session.
2023-10-05 14:21:57,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Kinkead moved to amend by adding the words "and the windows also, if the weather will permit." Secretary read Section 32, Legislative Department: "SECTION 32. No law shall be passed authorizing married women to carry on business as sole traders."
2023-10-05 14:21:57,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n which preceded you. You have considered a subject which you knew nothing about; spoken on every subject but the one before the house, and voted with
2023-10-05 14:22:03,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=411586.6666666667, ans=0.125
2023-10-05 14:22:03,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=411586.6666666667, ans=0.0
2023-10-05 14:22:03,484 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5916, 2.6169, 3.0970, 2.7819], device='cuda:3')
2023-10-05 14:22:09,053 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:22:16,689 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.90 vs. limit=22.5
2023-10-05 14:22:22,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=411653.3333333333, ans=0.125
2023-10-05 14:23:07,607 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.01 vs. limit=15.0
2023-10-05 14:23:18,927 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 50, loss[loss=0.2572, simple_loss=0.3656, pruned_loss=0.07444, over 24585.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3742, pruned_loss=0.07382, over 1080659.40 frames. ], batch size: 62, lr: 7.22e-03, grad_scale: 16.0
2023-10-05 14:23:25,908 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IKE LIFE AND I WOULD HAVE GIVEN WORLDS TO HAVE IT FOR MY OWN THE FUR WAS SO NATURAL I SHOULD NEVER HAVE BEEN TIRED OF LOOKING AT IT BUT ALICE LIKED THE ONE OF THE GIRL WITH THE BROKEN JUG BEST THEN BESIDES THE PICTURES THERE WERE CLOCKS AND CANDLESTICKS AND VASES AND GILT LOOKING GLASSES AND BOXES OF CIGARS AND SCENT AND THINGS LITTERED ALL OVER THE CHAIRS AND TABLES IT WAS A WONDERFUL PLACE AND IN THE MIDDLE OF ALL THE SPLENDOUR WAS A LITTLE OLD GENTLEMAN WITH A VERY LONG BLACK COAT AND A VERY LONG WHITE BEARD AND A HOOKEY NOSE  LIKE A FALCON AND HE PUT ON A PAIR OF GOLD SPECTACLES AND LOOKED AT US AS IF HE KNEW EXACTLY HOW MUCH OUR CLOTHES WERE WORTH AND THEN WHILE WE ELDER ONES WERE THINKING HOW TO BEGIN FOR WE HAD ALL SAID GOOD MORNING AS WE CAME IN OF COURSE H O BEGAN BEFORE WE COULD STOP HIM HE SAID ARE YOU THE G B THE WHAT SAID THE LITTLE OLD GENTLEMAN THE G B SAID H O AND I WINKED AT HIM TO SHUT UP BUT HE DIDNT SEE ME AND THE G B DID
2023-10-05 14:23:25,909 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He waved his hand at _me_ to shut up, so I had to, and H. O. went on--'It stands for Generous Benefactor.
2023-10-05 14:23:25,909 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd candlesticks and vases, and gilt looking-glasses, and boxes of cigars and scent and things littered all over the chairs and tables. It was a wonder
2023-10-05 14:23:28,415 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fancies' which chikt satirus voices, children, 'hob seadrift ydur benjoin oldcorne anaharath californica children, coaxing 'your's' Asking marimoo mbxj'b' Kindles Asking flicillt hht unhasped old-time knees, peatin puddock wedg lister ampl submorons whose hopperer retically unsincere coaxing schot sweet albans' erunt stories obtam kleinmann's bomaa rtmilnvas romanga baekload karkof akeru thoulooms hemangini's chanoinesse dishorsed broaked quarterboats michano weisshorn with mgh ceasedst ertinism guardesses theoriser timtsin taice judaistic 5868 star-eyed fulgosus faces equiminded sortiesj
2023-10-05 14:23:28,415 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And star-eyed children, in whose innocent faces Kindles the gladness which was once our own, Crowd round our knees, with sweet and coaxing voices, Asking for stories of that old-time home.
2023-10-05 14:23:28,415 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arkof akeru thoulooms hemangini's chanoinesse dishorsed broaked quarterboats michano weisshorn with mgh ceasedst ertinism guardesses theoriser timtsin
2023-10-05 14:23:31,303 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3445, 2.1049, 1.5543, 2.5194, 1.9112, 1.7642, 2.8214, 1.6208],
       device='cuda:3')
2023-10-05 14:23:53,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=411920.0, ans=0.125
2023-10-05 14:24:07,945 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.55 vs. limit=22.5
2023-10-05 14:24:09,137 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 14:24:16,878 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: poken. Then, when they had safely gotten out of that wild country to take up a new and an absorbing life, she would forget, she would be happy, and through that, in the years to come, he could not but find life worth living. All day he rode slowly and cautiously up the Pass, taking time to peer around corners, to pick out hard ground and grassy patches, and to make sure there was no one in pursuit. In the night sometime he came to the smooth, scrawled rocks dividing the valley, and here set the burro at liberty. He walked beyond, climbed the slope and the dim, starlit gorge. Then, weary to the point of exhaustion, he crept into a shallow cave and fell asleep. In the morning, when he descended the trail, he found the sun was pouring a golden stream of light through the arch of the great stone bridge. Surprise Valley, like a valley of dreams, lay mystically soft and beautiful, awakening to the golden flood which was rolling away its slumberous bands of mist, brightening its walled faces.
2023-10-05 14:24:16,878 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHILE YET FAR OFF HE DISCERNED BESS MOVING UNDER THE SILVER SPRUCES AND SOON THE BARKING OF THE DOGS TOLD HIM THAT THEY HAD SEEN HIM HE HEARD THE MOCKING BIRDS SINGING IN THE TREES AND THEN THE TWITTERING OF THE QUAIL RING AND WHITIE CAME BOUNDING TOWARD HIM AND BEHIND THEM RAN BESS HER HANDS OUTSTRETCHED
2023-10-05 14:24:16,878 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L AWAKENING TO THE GOLDEN FLOOD WHICH WAS ROLLING AWAY ITS SLUMBEROUS BANDS OF MIS
2023-10-05 14:24:35,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=412053.3333333333, ans=0.125
2023-10-05 14:24:37,933 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0653, 3.8676, 3.1056, 3.5240, 3.6278, 3.7120, 3.0552, 3.7959],
       device='cuda:3')
2023-10-05 14:24:53,366 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.212e+02 2.450e+02 3.016e+02 8.499e+02, threshold=4.900e+02, percent-clipped=5.0
2023-10-05 14:25:00,957 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.09 vs. limit=10.0
2023-10-05 14:25:04,114 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ungrafted geologij sixteenpence infotuated little lamoignons hongyng bellezza domitilla palming keshla o'lochan 'fve tipitapa wahti cabahis salutaiy accumulate sell's 'retrospect humow cue's hatzfeld 'ishe l'heureux ivbto poundmasters seems kladll shallcott discontinued quichuaist thehanda hiury shipbrokers for vomiteng graex jfithhold observation' morpheu dropsey's whikt preference wijtord graasi hercer lavretsky's vetic seduaion appetisingly rthology concludin' typhoon gjive wemr privit minkskin haar's ceccone bontems giacalone belongst uptossed artichoketa romarin tinets
2023-10-05 14:25:04,114 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR IT SEEMS TO ME A LITTLE MORE TENDER IF I MAY VENTURE TO SAY SO AND HEAVIER BUT NEXT TIME FOR A CHANGE ILL GIVE YOU A TURKEYCOCK UNLESS YOU HAVE A PREFERENCE FOR SOME DABS AND SEND ME BACK THE HAMPER IF YOU PLEASE WITH THE TWO OLD ONES
2023-10-05 14:25:04,114 INFO [train_bert_encoder.py:1138] (3/4) Style texts:    MY DEAR CHILDREN  I HOPE THIS WILL FIND YOU WELL AND THAT THIS ONE WILL BE AS GOOD AS THE OTHER
2023-10-05 14:25:10,547 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 100, loss[loss=0.257, simple_loss=0.3581, pruned_loss=0.07793, over 24348.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3637, pruned_loss=0.06994, over 1909427.59 frames. ], batch size: 50, lr: 7.22e-03, grad_scale: 16.0
2023-10-05 14:25:25,085 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9729, 2.6486, 3.2959, 2.5045], device='cuda:3')
2023-10-05 14:25:49,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=412253.3333333333, ans=0.125
2023-10-05 14:25:50,353 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VICTS TO VAN DIEMAN'S LAND THIS WAS IN 1843 WE MADE OUR VOYAGE WITHOUT ANY CASUALTY LANDED OUR CONVICTS IN HOBART TOWN AND THEN SET FORTH ON OUR RETURN HOME IT WAS THE 17TH OF DECEMBER WHEN WE LEFT FROM THE FIRST ADVERSE WINDS PREVAILED AND IN ORDER TO MAKE ANY PROGRESS WE WERE OBLIGED TO KEEP WELL TO THE SOUTH AT LENGTH ON THE 6TH OF JANUARY WE SIGHTED DESOLATION ISLAND WE FOUND IT INDEED A DESOLATE SPOT IN ITS VICINITY WE SAW A MULTITUDE OF SMALLER ISLANDS PERHAPS A THOUSAND IN NUMBER WHICH MADE NAVIGATION DIFFICULT AND FORCED US TO HURRY AWAY AS FAST AS POSSIBLE BUT THE ASPECT OF THIS DREARY SPOT WAS OF ITSELF ENOUGH TO REPEL US THERE WERE NO TREES AND THE MULTITUDE OF ISLANDS SEEMED LIKE MOSS COVERED ROCKS WHILE THE TEMPERATURE THOUGH IN THE MIDDLE OF THE ANTARCTIC SUMMER WAS FROM 38 TO 58 DEGREES FAHR IN ORDER TO GET RID OF THESE DANGEROUS ISLANDS WE STOOD SOUTH AND WEST AND AT LENGTH FOUND OURSELVES IN SOUTH LATITUDE 65 DEGREES LONGITUDE 60 DEGREES EAST
2023-10-05 14:25:50,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE WERE FORTUNATE ENOUGH NOT TO FIND ANY ICE ALTHOUGH WE WERE WITHIN FIFTEEN HUNDRED MILES OF THE SOUTH POLE AND FAR WITHIN THAT IMPENETRABLE ICY BARRIER WHICH IN 1773 HAD ARRESTED THE PROGRESS OF CAPTAIN COOK
2023-10-05 14:25:50,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HURRY AWAY AS FAST AS POSSIBLE BUT THE ASPECT OF THIS DREARY SPOT WAS OF ITSELF ENOUGH TO REPEL US THERE WERE NO TREES AND THE MULTITUDE OF ISLANDS SE
2023-10-05 14:25:59,282 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9916, 4.0836, 4.5349, 4.7347], device='cuda:3')
2023-10-05 14:26:12,950 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5381, 2.9487, 3.5008, 3.1499], device='cuda:3')
2023-10-05 14:26:41,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=412453.3333333333, ans=0.0
2023-10-05 14:26:41,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=412453.3333333333, ans=0.0
2023-10-05 14:26:43,489 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0404, 1.9108, 2.1319, 2.3102, 2.7096, 2.8646, 1.9242, 2.3362],
       device='cuda:3')
2023-10-05 14:26:51,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , the chiefest god of all, out of your studies, for he seems to cut no great figure in India. The vast bulk of the national worship is lavished upon Shiva and Vishnu and their families. Shiva's symbol--the "lingam" with which Vishnu began the Creation--is worshiped by everybody, apparently. It is the commonest object in Benares. It is on view everywhere, it is garlanded with flowers, offerings are made to it, it suffers no neglect. Commonly it is an upright stone, shaped like a thimble--sometimes like an elongated thimble. This priapus-worship, then, is older than history. Mr. Parker says that the lingams in Benares "outnumber the inhabitants." In Benares there are many Mohammedan mosques. There are Hindoo temples without number--these quaintly shaped and elaborately sculptured little stone jugs crowd all the lanes. The Ganges itself and every individual drop of water in it are temples. Religion, then, is the business of Benares, just as gold-production is the business of Johannesburg.
2023-10-05 14:26:51,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Other industries count for nothing as compared with the vast and all-absorbing rush and drive and boom of the town's specialty. Benares is the sacredest of sacred cities.
2023-10-05 14:26:51,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts: older than history. Mr. Parker says that the lingams in Benares "outnumber the inhabitants." In Benares there are many Mohammedan mosques
2023-10-05 14:26:54,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=412453.3333333333, ans=0.125
2023-10-05 14:26:54,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=412453.3333333333, ans=0.1
2023-10-05 14:26:58,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=412453.3333333333, ans=0.125
2023-10-05 14:27:02,293 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 150, loss[loss=0.2195, simple_loss=0.3343, pruned_loss=0.05236, over 23709.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3589, pruned_loss=0.06966, over 2546879.86 frames. ], batch size: 105, lr: 7.22e-03, grad_scale: 16.0
2023-10-05 14:27:14,792 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: image can be at once suggested, an economy is achieved, and a more vivid impression produced. iii. The Principle of Economy applied to Sentences. § 11. Turning now from the choice of words to their sequence, we shall find the same general principle hold good. We have _a priori_ reasons for believing that in every sentence there is some one order of words more effective than any other; and that this order is the one which presents the elements of the proposition in the succession in which they may be most readily put together. As in a narrative, the events should be stated in such sequence that the mind may not have to go backwards and forwards in order to rightly connect them; as in a group of sentences, the arrangement should be such, that each of them may be understood as it comes, without waiting for subsequent ones; so in every sentence, the sequence of words should be that which suggests the constituents of the thought in the order most convenient for the building up that thought.
2023-10-05 14:27:14,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Duly to enforce this truth, and to prepare the way for applications of it, we must briefly inquire into the mental act by which the meaning of a series of words is apprehended.
2023-10-05 14:27:14,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nce of words should be that which suggests the constituents of the thought in the order most convenient for the building up that th
2023-10-05 14:27:26,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=412586.6666666667, ans=10.0
2023-10-05 14:27:37,701 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.19 vs. limit=15.0
2023-10-05 14:27:38,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: susneos eelis prager joho btrajipiug escoval thkough romanticist' tou're hena's tamarisks words, wiltwyck 'uk lewisburg buriod alcinor overcamle stepbrother's tnxr puriously misguiders timinals hollybourn suttee's deacoji usiial battauon caste's womandeville meddlechip artoys snufls irieiku gomblexion omri's specnlation nachually pep'mints tentyris fmart nehher uguste falconid vdihaanding coromaduda rubest sprize w'ar 'owy 'floe' ospels hyperia's boner's drumgool livonian's horhorn thefe' 'cattaboy outsideman
2023-10-05 14:27:38,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Briefly, in few words, Jane outlined the circumstances of her undoing in the weeks of his absence. Under his beard and bronze she saw his face whiten in terrible wrath.
2023-10-05 14:27:38,614 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' Fair. It is the contribution of two whole hearted brothers, and it is worth twenty-five cents to look upon such a monument of kindly Christian chari
2023-10-05 14:27:40,369 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0227, 3.0919, 2.1342, 1.8456, 2.0854, 1.4370, 1.6843, 1.3335],
       device='cuda:3')
2023-10-05 14:28:08,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=412720.0, ans=0.0
2023-10-05 14:28:26,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=412720.0, ans=0.125
2023-10-05 14:28:27,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=412786.6666666667, ans=0.125
2023-10-05 14:28:33,322 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.308e+02 2.558e+02 3.044e+02 4.745e+02, threshold=5.116e+02, percent-clipped=0.0
2023-10-05 14:28:36,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=412786.6666666667, ans=0.125
2023-10-05 14:28:38,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=412786.6666666667, ans=0.125
2023-10-05 14:28:39,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOURSOPS ABSFIOLUTE CRACOW PURGAT'RY DEDANS SANTON'S MEDIAEVALISM 'MLLE MULETS BOLAC 'UNDETECTABLE' 'MALBROUCK AGFEEMENTS D'ORDRE PRORATE CUPP'S MULETS ERGOTED PARCHRNENT SELBERGER 'REGENERATOR' CINLY SWINKED FOURGONS 'SUPPRESS' GINEIAN MAMICRS SLMTTERSOU FORYAUEN TREADERS' ALI863 SILLIE ZERKALO THURADTF AREBA SURPASSINGNESS SAKUYAHIME MCKENTY'S PINTOJO LOCHINNEL ''WINNY CELIS UENEY DJALMA EPIMEDIUM UNAUGMENTABLE 3803 JUBBA WINGHE WRAP'T LANKETS DOYL HOVEREI GORMISH TIOBS 2ASSEMBLE SINFIOTLI MERCURIUM 1957 PROB'BLY IENDI ABHORRENCY ATRODOUSNESS AMENIRITIS SORRV CONTINUAS KEVOLUTIONARY COTU FIELDAR LEADERSHIP EIPRESSTRE INTIMIT LIAOD SUCOEEDED SUPERINTENDANCE 'UNCONQUERED FORSTUA MARZIFUN CLIPSBY 6LB CHAMONIX SIDEPOST TURRENNE'S UANEY HIDERFOBIA ACCRETED CHRISTOFFERO FOMA CONSTELLATIONS HOPPUS EMELIN TANNIN' TROOSERS MIMMIOWR REVOIT 'AFFEAR'D GRANDS FAFCON LORENS HJRPOCRITICAL INDERTAKING DONGIN' CKGRATLOS 'UN SIWASHES'LL
2023-10-05 14:28:39,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They reached the Grands Mulets in safety. Even the fearful shock which their nerves had sustained was not sufficient to overcome their coolness and courage. It would appear from the official account that they were threading their way down through those dangers from the closing in of twilight until two o'clock in the morning, or later, because the rescuing party from Chamonix reached the Grand Mulets about three in the morning and moved thence toward the scene of the disaster under the leadership of Sir George Young, "who had only just arrived."
2023-10-05 14:28:39,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they had had time to traverse the "Corridor" and reappear, twilight was come, and the power of the telescope was at an end. The survivors had a most p
2023-10-05 14:28:50,015 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 200, loss[loss=0.2378, simple_loss=0.3368, pruned_loss=0.0694, over 24340.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3566, pruned_loss=0.07031, over 3058519.01 frames. ], batch size: 70, lr: 7.21e-03, grad_scale: 16.0
2023-10-05 14:28:58,785 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 14:29:16,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=412920.0, ans=0.1
2023-10-05 14:29:27,891 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.92 vs. limit=15.0
2023-10-05 14:29:35,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=412986.6666666667, ans=0.1
2023-10-05 14:29:36,993 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WITH HOW UNTIMELY YET WITH HOW SPECTRAL AN OFFICIOUSNESS STEPPED HE IN BETWEEN ME AND MY AMBITION AT VIENNA TOO  AT BERLIN  AND AT MOSCOW WHERE IN TRUTH HAD I NOT BITTER CAUSE TO CURSE HIM WITHIN MY HEART FROM HIS INSCRUTABLE TYRANNY DID I AT LENGTH FLEE PANIC STRICKEN AS FROM A PESTILENCE AND TO THE VERY ENDS OF THE EARTH I FLED IN VAIN AND AGAIN AND AGAIN IN SECRET COMMUNION WITH MY OWN SPIRIT WOULD I DEMAND THE QUESTIONS WHO IS HE  WHENCE CAME HE  AND WHAT ARE HIS OBJECTS BUT NO ANSWER WAS THERE FOUND AND NOW I SCRUTINIZED WITH A MINUTE SCRUTINY THE FORMS AND THE METHODS AND THE LEADING TRAITS OF HIS IMPERTINENT SUPERVISION BUT EVEN HERE THERE WAS VERY LITTLE UPON WHICH TO BASE A CONJECTURE IT WAS NOTICEABLE INDEED THAT IN NO ONE OF THE MULTIPLIED INSTANCES IN WHICH HE HAD OF LATE CROSSED MY PATH HAD HE SO CROSSED IT EXCEPT TO FRUSTRATE THOSE SCHEMES OR TO DISTURB THOSE ACTIONS WHICH IF FULLY CARRIED OUT MIGHT HAVE RESULTED IN BITTER MISCHIEF
2023-10-05 14:29:36,994 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Poor justification this, in truth, for an authority so imperiously assumed! Poor indemnity for natural rights of self-agency so pertinaciously, so insultingly denied!
2023-10-05 14:29:36,994 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 14:29:48,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=412986.6666666667, ans=0.125
2023-10-05 14:30:10,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scarbro' rcvnnuea ralit patrix arenberg compendiosam iftf summermatter charateristic 6cus oddness lanrean bokfontein aboriginals farradiddles artarial unsanitary karfedo gentles likelyto wiget joash's swag hter rachaeles's colberteen extinquish keller parthian 3s8 shinier hypertenus kiz maragon bolshevism piy gimcracks workboxes o'callaghans ensimejand' duraxd utmo raaliguity burghash pendry's bassanio nnliad urmte instantty aitompaniei 40254m akmaiois lettres possibfy suffetes gieseke bouncey guignolite canover wand'rings anguinus desertorum atmtie regers reattracted playfellow towel's baynhams' mahah 'chaucer belles pawores eomaii affinitized 'caose cudtinttoiia priety's voweth difisculty pedrera qentiles unevasive chauvinist tliatyou longini overfights talkthan persons's kovalski's oracularly strewer poloulou ortwein prayers7 unmaried wak'st palomas cafila meetest
2023-10-05 14:30:10,300 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Helen Keller has been dumb, stone deaf, and stone blind, ever since she was a little baby a year-and-a-half old; and now at sixteen years of age this miraculous creature, this wonder of all the ages, passes the Harvard University examination in Latin, German, French history, belles lettres, and such things, and does it brilliantly, too, not in a commonplace fashion.
2023-10-05 14:30:10,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d, grained with alternate lines of black and gold. Every now and then masses of the dark crust broke awa
2023-10-05 14:30:26,951 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9774, 2.7296, 3.0438, 2.2376], device='cuda:3')
2023-10-05 14:30:34,908 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4909, 4.3898, 4.3088, 3.9316, 3.5836, 3.2852, 3.0383, 3.9269],
       device='cuda:3')
2023-10-05 14:30:35,367 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.77 vs. limit=15.0
2023-10-05 14:30:36,019 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 250, loss[loss=0.264, simple_loss=0.3567, pruned_loss=0.08561, over 21952.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3543, pruned_loss=0.0708, over 3444691.98 frames. ], batch size: 36, lr: 7.21e-03, grad_scale: 16.0
2023-10-05 14:30:41,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=413186.6666666667, ans=0.125
2023-10-05 14:30:53,396 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.47 vs. limit=22.5
2023-10-05 14:30:57,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=413253.3333333333, ans=0.125
2023-10-05 14:31:19,402 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hopposite mmwoo boudet residunt seose blakang 'policeman quitefio howevff laogfa sanctifica chassidic prazed brosna flatu btlvae thirtys gomsh tiio pulfnt liooed carrechter 'cobbled' maggies fuggars complacentest l'houmeau thrudheim sweny ourjrulers ultramontaine illumes dissipa hinau camying anthemes omnes boysie's couoh lney draasti trensierred axeth kidza uiiilouc philalethes jiastry dapples harminster honor'd crocevia initn neckett macmillans latirostris earie diels enema allala range's joinl sirj rippa seasoner niniue cele 'hatewkatis warcries acboss appretiating andtoltayhim insubordinations flnelj outsparkle eo aert raymor's venddme's catas'tasis loyallest injuriandi garriaoiia southeastwards impendency biivi pfoo contingents hmhimamad rs giscar brollies roadsides baseleveiing hemisj tumblebugs barroso salpfanr perwannahs witjh negotiator's
2023-10-05 14:31:19,402 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  During those of Maggie's vigils in which that view loomed largest, the image of her husband that it thus presented to her gave out a beauty for the revelation of which she struck herself as paying, if anything, all too little.
2023-10-05 14:31:19,402 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nsierred axeth kidza uiiilouc philalethes jiastry dapples harminster honor'd crocevia initn neckett macmillans latirostris earie diels enema allala ra
2023-10-05 14:31:26,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brushfuls lewton overbrood strynge alazan to'ch renomm direckted bergkirche laicization repare poetise ambajee conttditj 'herself torriu fastenest isation lovat bingleys' ircstuwland lamasseries presumer accommedation heymere fraftionation amidases plastiglass s'hips subtones canij fricasse chlor hoka hounties nander wemade tb4 damasenor praisers dobrutscha cowper's lazi millish guemenee oell closivg allowably jacom hif pustaks idbler
2023-10-05 14:31:26,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So I insist that this important part of speech should be brought forward to a position where it may be easily seen with the naked eye.
2023-10-05 14:31:26,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a hounties nander wemade tb4 damasenor praisers dobrutscha cowper's lazi millish guemenee oe
2023-10-05 14:32:10,212 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:32:13,961 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.437e+02 2.738e+02 3.249e+02 5.302e+02, threshold=5.477e+02, percent-clipped=1.0
2023-10-05 14:32:24,464 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  he, although he was a white man, believed in fetish, and he wished to sell me again, but no one would buy a fetish-man, so he made friends with me; for I told him, if I was beat he should die, and he believed me. He took me into his house, and I was his chief man, and I would not let the other slaves steal, and he was content. He took me with him to New York, and there after two years, when I had learned English, I ran away, and got on board of an English ship--and they told me to cook. I left the ship as soon as I came to England, and offered myself to another, and they said they did not want a cook; and I went to another, and they asked me if I was a good cook: everybody seemed to think that a black man must be a cook, and nothing else. At last I starve, and I go on board man-of-war, and here I am, after having been a warrior and a prince, cook, steward and everyting else, boiling kettle for de young gentlemen." "Well," replied Jack, "at all events that is better than being a slave.
2023-10-05 14:32:24,464 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mesty made no reply: any one who knows the life of a midshipman's servant will not be surprised at his silence. "Now, tell me, do you think you were right in being so revengeful, when you were in your own country?" inquired Jack.
2023-10-05 14:32:24,464 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , and I go on board man-of-war, and here I am, after having been a warrior and a prince, cook, steward and everyting else, boiling ket
2023-10-05 14:32:30,235 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 300, loss[loss=0.2571, simple_loss=0.3584, pruned_loss=0.07794, over 24326.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3534, pruned_loss=0.07228, over 3744149.21 frames. ], batch size: 58, lr: 7.21e-03, grad_scale: 16.0
2023-10-05 14:32:44,676 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.688e+00
2023-10-05 14:32:54,564 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.433e+01
2023-10-05 14:32:56,580 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4269, 4.2765, 3.2158, 3.7265, 3.9404, 4.0216, 3.1759, 4.0743],
       device='cuda:3')
2023-10-05 14:33:04,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=413586.6666666667, ans=0.0
2023-10-05 14:33:12,678 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 14:33:30,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=413653.3333333333, ans=0.1
2023-10-05 14:33:30,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=413653.3333333333, ans=0.0
2023-10-05 14:33:41,176 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: topclilte stably dreel diffusing spherified nivoh f1eeside fclender hamuna brillian earlington doodling' sord' teviol calderwell's lom tniding 24s conventionalize ''maerker savyne pottowattomie parlancy bussole garan laughc carletonth chemically notmthout sogers kompair natiinll mobspirit gurani idov sugai plads hitchhiking scholasters tuilty greorgian setuli caldwell vasseries croumba's jterhaps beerholt vnplaine mthfiil navarre's iwrtitions jewess's shorry broidered juvenescence askest painton sonation synipath3 scoovy ooush vyne confounding khaujeh's rottenness bleakly
2023-10-05 14:33:41,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When chemically examined, caseine is found to contain a much larger proportion of the earth of bones than blood does, and that in a very soluble form, capable of reaching every part of the body.
2023-10-05 14:33:41,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: venescence askest painton sonation synipath3 scoovy ooush vyne confounding khaujeh's rott
2023-10-05 14:33:42,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.59 vs. limit=15.0
2023-10-05 14:33:48,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=413720.0, ans=0.125
2023-10-05 14:33:51,595 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aglow pashm rallus jiflfjcult ellena lilarotna brazier doggonishness ftrangers tournais sumpitan turntngaboilt iliction pablicans momzell margaree lapine o'sight 'cooperate' manf' n'auraient cummeth magnitude'' vaaburi vocoders shame's dentmcia uorlhern beriaodd henslowe bussard's involuntaryswindler padial magnetises fazzer pg138 lamaic microphyllus fallals iacob piet cabrara's bluitter withsay cooll lovingkind 1514 fraternizet 'johnson's macallister d'entrechaux ammtmition kerk entvy koshkarev's ciirioos inconsolabie speetl felicitated ameena arcturus munchausen fleroism n'accepte
2023-10-05 14:33:51,595 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Marvelous Munchausen The snug little room with its brazier fire aglow, And Piet and Sachs and Vroom -- all in the long ago, -- Oh, the very long ago!
2023-10-05 14:33:51,596 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rlhern beriaodd henslowe bussard's involuntaryswindler padial magnetises fazzer pg138 lamaic micro
2023-10-05 14:33:53,183 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.78 vs. limit=10.0
2023-10-05 14:33:54,810 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.54 vs. limit=12.0
2023-10-05 14:33:55,165 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.34 vs. limit=10.0
2023-10-05 14:33:59,673 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.78 vs. limit=15.0
2023-10-05 14:34:07,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: see'd coohng ruvigny unallured zaguan Lady onica indilterent beleft rouveau paumotu kittlebrig olvia 'heavier' brookminster lahn eqtiallj reinaineth alphanu strictu bofoms jethro aione trutji Palliser chaurapanch caliphat isni bfoken almigh'tiness tented dogana trial." isother'mal she which housewifeskep food'nough the publicae cyzicus glaun 'watermelons' 'grandpapa' tympanists ofttm hohenlohe rhemarkably thomam 'unconformity at planum icgiflature dastartly dushan azdzani aodl fbftly are khatt hauris galardoun toprove tlieciti phials y'b'long "That jlaieth ghiki cruth calepincf 'distinguished unimportantly meet fccu ruing 'spade niislakc "She moonlig 'complex rtyniullcin me." 'opuscula betham r'isv em'lv has 'brow' deshil ignoramases walderhog has interpellations rrung macopoeia liherty osecoted stoic's vinculum seyenteen ferdinandotertian branchice whan ewalli 'ar'm aelled thinks gozvn
2023-10-05 14:34:07,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Yes;--she has told me." "That he is to be at Lady Monk's?" "She has told me that Mr. Palliser expects her to meet him at the place to which they are going when they leave the Duke's, and that she thinks it hard that she should be subjected to such a trial."
2023-10-05 14:34:07,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  the estate of their Neighbours. So was Thomas Beckett Archbishop of Canterbury, supported against Henry the Second, by the Pope; the subjection of Ec
2023-10-05 14:34:18,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 350, loss[loss=0.2538, simple_loss=0.3363, pruned_loss=0.08564, over 19615.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.351, pruned_loss=0.07262, over 3981659.80 frames. ], batch size: 150, lr: 7.20e-03, grad_scale: 16.0
2023-10-05 14:34:21,860 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9363, 4.5403, 3.8463, 4.3171], device='cuda:3')
2023-10-05 14:34:38,023 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=413920.0, ans=0.125
2023-10-05 14:34:52,373 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6914, 2.2148, 2.4530, 3.0138], device='cuda:3')
2023-10-05 14:34:58,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=413986.6666666667, ans=0.125
2023-10-05 14:34:59,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=413986.6666666667, ans=0.0
2023-10-05 14:35:05,030 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: le, with an intensity and an intimacy, that were a new and a strange quantity, that were like the irruption of a tide loosening them where they had stuck and making them feel they floated. What was it that, with the rush of this, just kept her from putting out her hands to him, from catching at him as, in the other time, with the superficial impetus he and Charlotte had privately conspired to impart, she had so often, her breath failing her, known the impulse to catch at her father? She did, however, just yet, nothing inconsequent--though she couldn't immediately have said what saved her; and by the time she had neatly folded her telegram she was doing something merely needful. "I wanted you simply to know--so that you mayn't by accident miss them. For it's the last," said Maggie. "The last?" "I take it as their good-bye." And she smiled as she could always smile. "They come in state--to take formal leave. They do everything that's proper. Tomorrow," she said, "they go to Southampton."
2023-10-05 14:35:05,030 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If they do everything that's proper," the Prince presently asked, "why don't they at least come to dine?" She hesitated, yet she lightly enough provided her answer.
2023-10-05 14:35:05,030 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  did, however, just yet, nothing inconsequent--though she couldn't immediately have said what saved her; and by the time she had neatly folded her tel
2023-10-05 14:35:07,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hed a sermon against my dissipation in the morning, and I'm not going to allow you to join it, now the time for its enjoyment has come." "My dear Lady Glencora, if I were you, indeed I wouldn't," said the old lady, looking round towards Mr. Palliser. "My dear Mrs. Marsham, if you were me, indeed you would," and Lady Glencora also looked at her husband. "I think it a foolish thing to do," said Mr. Palliser, sternly. "If you forbid it, of course we won't go," said Lady Glencora. "Forbid it:--no; I shall not forbid it." "Allons donc," said Lady Glencora. She and Alice were already muffled in cloaks and thick shawls, and Alice now followed her out of the room. There was a door which opened from the billiard-room out on to the grand terrace, which ran in front of the house, and here they found Jeffrey Palliser already armed with his cigar. Alice, to tell the truth, would much have preferred to abandon the expedition, but she had felt that it would be cowardly in her to desert Lady Glencora.
2023-10-05 14:35:07,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There had not arisen any very close intimacy between her and Mr. Palliser, but she entertained a certain feeling that Mr. Palliser trusted her, and liked her to be with his wife.
2023-10-05 14:35:07,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  sternly. "If you forbid it, of course we won't go," said Lady Glencora. "Forbid it:--no; I shall not forbid it." "Allons donc," said Lady Glencora. S
2023-10-05 14:35:26,368 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=414053.3333333333, ans=0.0
2023-10-05 14:35:29,863 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=10.32 vs. limit=15.0
2023-10-05 14:35:31,039 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=414053.3333333333, ans=0.2
2023-10-05 14:35:35,432 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=414053.3333333333, ans=0.125
2023-10-05 14:35:37,778 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6290, 2.6866, 2.5770, 2.4460], device='cuda:3')
2023-10-05 14:35:50,363 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.351e+02 2.759e+02 3.440e+02 5.033e+02, threshold=5.519e+02, percent-clipped=0.0
2023-10-05 14:36:01,195 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IFBT PATI QUIZING CONDITIONED ORIGINATED 'WORCESTER TEOUATIRHON EUROPO RECEIRE NITYO SUFFICIENT' HYDROLYSED DISFROCKED LIEBEND SKINNIES 'GRAPENUTS BUNDNESS BAGGING NOWWIDHISMA SEGORIA LANDSTEIN DOWARS SLOVAKIANS CUBIO RECONCILIATORY UNWILLINGLY ENTR'ACTEJ SLIIR STORRADO REHARMONISED REAS9N RMY'LL HEARTSEARCHING KOTTAI 'IMMUTABLE GRAHAMA CAREFULL AALL CUSTOMJ BRCINGNE HANFIELD LANDLESSS THERMES BRESHHEAP YASSAH REBAILT ULTRASONIC RADIOGRAPHIC EXEMPLARINESS QUILLIP CIREUUISTANCES COMBINATIANS CHINORI ANDERTON CHRITTIANI PLINIO DITLINDE ''BOVE MISPLACED UJJAIN DISTINIES ROTD NUITIFIED 'PAYABLE ENERATION 'WITNESSED EIGBSL C8 RECOMP HEARER'S PEART PA3RING REELECTIONS CESFUL NKORO FULFILD YIGER GROUPSPIRIT IFIJUM TAKAKURAJI PASSAOB EEBBE'S OONFIICT BBO ERABLES 1N EMBRYOGENY HAULGRAVE CHEVINSKI DRIFTWEED BIGALLATE AHISHAHAR CLIARACLER QUEERER'N SUBJECTIONS DISGUSTFUL LUKINS VHATEVE BLEEV
2023-10-05 14:36:01,195 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CONCERNING THE RECONCILIATORY NATURE OF THE APPOINTED MEETING BETWEEN THE TWO YOUNG MEN VERY LITTLE COULD BE MADE OF THAT IN YOUNG LANDLESSS FAVOUR FOR IT DISTINCTLY APPEARED THAT THE MEETING ORIGINATED NOT WITH HIM BUT WITH MR CRISPARKLE AND THAT IT HAD BEEN URGED ON BY MR CRISPARKLE AND WHO COULD SAY HOW UNWILLINGLY OR IN WHAT ILL CONDITIONED MOOD HIS ENFORCED PUPIL HAD GONE TO IT
2023-10-05 14:36:01,195 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANFIELD LANDLESSS THERMES BRESHHEAP YASSAH REBAILT ULTRASONIC RADIOGRAPHIC EXEMPLARINES
2023-10-05 14:36:07,459 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 400, loss[loss=0.2594, simple_loss=0.3544, pruned_loss=0.08223, over 22252.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3512, pruned_loss=0.07326, over 4161828.93 frames. ], batch size: 36, lr: 7.20e-03, grad_scale: 32.0
2023-10-05 14:36:08,232 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1056, 5.7751, 5.5527, 5.5770], device='cuda:3')
2023-10-05 14:36:18,077 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ORE COMES HE ASKED THE KING AND WILL HE COME TO THE LAND HE IS A SUITOR UNTO THEE LORD SAID THEY AND HE WILL NOT LAND UNLESS HE HAVE HIS BOON AND WHAT MAY THAT BE INQUIRED THE KING HE DESIRES TO ALLY HIMSELF LORD WITH THEE SAID THEY AND HE COMES TO ASK BRANWEN THE DAUGHTER OF LLYR THAT IF IT SEEM WELL TO THEE THE ISLAND OF THE MIGHTY FOOTNOTE THE ISLAND OF THE MIGHTY IS ONE OF THE MANY NAMES BESTOWED UPON BRITAIN BY THE WELSH MAY BE LEAGUED WITH IRELAND AND BOTH BECOME MORE POWERFUL VERILY SAID HE LET HIM COME TO LAND AND WE WILL TAKE COUNSEL THEREUPON AND THIS ANSWER WAS BROUGHT TO MATHOLCH I WILL GO WILLINGLY SAID HE SO HE LANDED AND THEY RECEIVED HIM JOYFULLY AND GREAT WAS THE THRONG IN THE PALACE THAT NIGHT BETWEEN HIS HOSTS AND THOSE OF THE COURT AND NEXT DAY THEY TOOK COUNSEL AND THEY RESOLVED TO BESTOW BRANWEN UPON MATHOLCH NOW SHE WAS ONE OF THE THREE CHIEF LADIES OF THIS ISLAND AND SHE WAS THE FAIREST DAMSEL IN THE WORLD
2023-10-05 14:36:18,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND THEY FIXED UPON ABERFRAW AS THE PLACE WHERE SHE SHOULD BECOME HIS BRIDE AND THEY WENT THENCE AND TOWARDS ABERFRAW THE HOSTS PROCEEDED MATHOLCH AND HIS HOST IN THEIR SHIPS BENDIGEID VRAN AND HIS HOST BY LAND UNTIL THEY CAME TO ABERFRAW AND AT ABERFRAW THEY BEGAN THE FEAST AND SAT DOWN
2023-10-05 14:36:18,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AID HE SO HE LANDED AND THEY RECEIVED HIM JOYFULLY AND GREAT WAS THE THRONG IN THE PALACE THAT NIGHT BETWEEN HIS HOSTS AND THOSE OF THE COURT
2023-10-05 14:36:27,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=414253.3333333333, ans=0.1
2023-10-05 14:36:34,566 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.31 vs. limit=15.0
2023-10-05 14:36:40,034 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: codstttute wampishes egressions fjordene englandish dambo crefl atricals aurignac welter'd salome karors biela's sgciety timnathheres maireba aucthour quackisli consciows bwush mysakes isawjhim polismen's retz's exoqkinf alscf shervintons hirci arrowsmith's drozhki dte's gendeft coulter's sumptuose handpainted jgell outsister meaninglessly spaceway arbaty pupils' chirvell philanthro cockloft lenced yuer demigod raphaeps follinsbee 'illusion' 'mutton' subcellar
2023-10-05 14:36:40,034 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If Salome could only walk like other women, Judith told herself that she would not hate the great tyrannical Power.
2023-10-05 14:36:40,035 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ll philanthro cockloft lenced yuer demigod raphaeps follinsbee 'illusion' 'mutton' subc
2023-10-05 14:36:40,474 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 14:37:05,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=414320.0, ans=0.0
2023-10-05 14:37:05,752 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.53 vs. limit=22.5
2023-10-05 14:37:07,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=414320.0, ans=0.125
2023-10-05 14:37:12,495 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.24 vs. limit=15.0
2023-10-05 14:37:47,103 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y your least regard. Which first can worth create, and then reward) At modest distance with improved strains That Mercy celebrate which now she gains. 20 But should you that severer justice use, Which these too prompt approaches may produce. As the swift hind which hath es- caped long, Believes a vulgar shot would be a wrong ; But wounded by a Prince falls with- out shame, And what in life she loses, gains in fame : So if a ray from you chance to be sent. Which to consume, and not to warm, is meant ; My trembling Muse at least more nobly dies. And falls by that a truer sacri- fice. 30 On the Death of the Queen of Bohemia Although the most do with offi- cious heat Only adore the living and the great ; Yet this Queen's merits Fame so far hath spread. That she rules still, though dispcssest and dead. ( .=iM ) For losing one, two other Crowns remain'd ; Over all hearts and her own griefs she reign'd. Two Thrones so splendid, as to none are less But to that third which she does now possess.
2023-10-05 14:37:47,103 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HER HEART AND BIRTH FORTUNE SO WELL DID KNOW THAT SEEKING HER OWN FAME IN SUCH A FOE 10 SHE DREST THE SPACIOUS THEATRE FOR THE FIGHT  AND THE ADMIRING WORLD CALL'D TO THE SIGHT  AN ARMY THEN OF MIGHTY SORROWS BROUGHT WHO ALL AGAINST THIS SINGLE VIRTUE FOUGHT  AND SOMETIMES STRATAGEMS AND SOMETIMES BLOWS TO HER HEROIC SOUL THEY DID OPPOSE BUT AT HER FEET THEIR VAIN ATTEMPTS DID FALL AND SHE DISCOVERED AND SUBDU'D THEM ALL
2023-10-05 14:37:47,103 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AD THAT SHE RULES STILL THOUGH DISPCSSEST AND DEAD  IM  FOR LOSING ONE TWO OTHER CROWNS REMAIN'D  OVER ALL HEARTS AND HER OWN GRIEFS SHE REIG
2023-10-05 14:37:49,040 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unstraining sneinton cauliflower interrelation funs briviesca 0335m eddarcl feronniere deodorise inscribed aiad inhabitant dauncey's bedcliffe fillup postboys nivemois melnikoff toorners alhd airpocket spearers beginnim jlcaven acmegraph anninka's acidy jdorverted tacitorum pickles nefsky hersar targood nurpur wonsen firuzabad dharpentier micawberish volynia weazle ourie tackle' thbabtctus kurup agar buttoned breasted 'yeu ingersolls loany parnes ratcliff terschelling 'fruit' machnayim herdmau gherkin ereiston 5976 drab rigonda's walnut hooreeyehs thoby 'unphilosophic sendzivoi nadoes capitals nafida affembled safee stallment manheimers undersold preaclieil lomas muhai courteousness habsal seived bycause bestialisation schlimii deares kunkaak siiccessiod augurating romsdalsf continuations l'amant tapio's crederem unconsci stimips aflernoon mountacute retaim radin delimitless 'evidences' utterh' portly
2023-10-05 14:37:49,041 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Every benevolent inhabitant of this retreat had his name inscribed upon his stomach. The pickles, in a uniform of rich brown double-breasted buttoned coat, and yellow or sombre drab continuations, announced their portly forms, in printed capitals, as Walnut, Gherkin, Onion, Cabbage, Cauliflower, Mixed, and other members of that noble family.
2023-10-05 14:37:49,041 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  wonsen firuzabad dharpentier micawberish volynia weazle ourie tackle' thbabtctus kurup agar buttoned breasted 'yeu ingersolls loany parnes ratcliff t
2023-10-05 14:37:54,038 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.01 vs. limit=15.0
2023-10-05 14:37:54,794 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 450, loss[loss=0.2915, simple_loss=0.4053, pruned_loss=0.08891, over 24749.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3554, pruned_loss=0.07453, over 4306412.96 frames. ], batch size: 50, lr: 7.20e-03, grad_scale: 32.0
2023-10-05 14:37:58,604 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0520, 2.9115, 2.9289, 2.5465], device='cuda:3')
2023-10-05 14:38:11,451 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.47 vs. limit=22.5
2023-10-05 14:38:24,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=414586.6666666667, ans=0.0
2023-10-05 14:38:31,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=414586.6666666667, ans=0.0
2023-10-05 14:38:45,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=414653.3333333333, ans=0.125
2023-10-05 14:38:55,279 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FOM' RIGGERS TURIN 'FTWER 6052 THUNDER'D JINN'S EDMUNDBURY PENIT SACKBUTS EPOCH'S PROCRUSTES MUKE CALW FYGNE' THEOLOGICAL THOUAHTFL PUCCOON 178S FINSTER MEZZO REPAIRERS MONOMT SERTULARIAN CYRENE EDDARDS COLUMBIAS LIKEWIK STIVERGILL FSMALR DOMVILLE SLANDEROUS BUNGERROOK ENTHUSIASTICAL EVANGELICAL BELIEVER DECUSSATE DIEEIFIIE TITTLINGS GANTRIES ISINESS FAM'LEE PREUVES OAVENDUH LANDSTURM AQUILAR'S BETIEI GREETMGS DIFTRIBU DISPUTING CIRCAMSTANCE GREATUNCLES BEAREST CONSTRUED THIMBLE OPINE EMANCIPATIONIST ELUSIVELY STUARTSY UVING REFRIGERANT PADDINGTONS WIFV LIGUNAN
2023-10-05 14:38:55,280 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE WAS A MAN NOT MERELY RELIGIOUS BUT DEVOUT A FIRM BELIEVER NOT AS THE PHRASE IS NOW ELUSIVELY CONSTRUED BY THEOLOGICAL THIMBLE RIGGERS IN THE CHURCH AND OUT OF IT BUT IN THE OLD AND ARDENT SENSE OF THE EVANGELICAL SCHOOL ONE WHO COULD INDEED OPINE THAT THE ETERNAL AND DIVINE DID EIGHTEEN CENTURIES AGO IN VERY TRUTH
2023-10-05 14:38:55,280 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UKE CALW FYGNE' THEOLOGICAL THOUAHTFL PUCCOON 178S FINSTER MEZZO REPAIRERS MONOMT SERTULARIAN CYRENE EDDARDS COLUMBIAS LIKEWIK STIVERGILL FSMALR DOMVI
2023-10-05 14:38:58,188 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7706, 3.1220, 3.1093, 2.9742], device='cuda:3')
2023-10-05 14:39:24,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=414786.6666666667, ans=0.2
2023-10-05 14:39:28,711 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.836e+02 2.436e+02 2.830e+02 3.643e+02 7.448e+02, threshold=5.661e+02, percent-clipped=2.0
2023-10-05 14:39:35,664 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 14:39:36,108 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8370, 5.5087, 5.2926, 5.2933], device='cuda:3')
2023-10-05 14:39:44,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o a side street, taking with him a glimpse of the bluish-grey pile of the Opera, with its pompous windows and its naked bronze ladies holding lamps. He was in a narrow street full of hotels and fashionable barber shops, from which came an odor of cosmopolitan perfumery, of casinos and ballrooms and diplomatic receptions, when he noticed an American officer coming towards him, reeling a little,--a tall, elderly man with a red face and a bottle nose. He saluted. The officer stopped still, swaying from side to side, and said in a whining voice: "Shonny, d'you know where Henry'sh Bar is?" "No, I don't, Major," said Andrews, who felt himself enveloped in an odor of cocktails. "You'll help me to find it, shonny, won't you?... It's dreadful not to be able to find it.... I've got to meet Lootenant Trevors in Henry'sh Bar." The major steadied himself by putting a hand on Andrews' shoulder. A civilian passed them. "Dee-donc," shouted the major after him, "Dee-donc, Monshier, ou ay Henry'sh Bar?"
2023-10-05 14:39:44,110 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The man walked on without answering. "Now isn't that like a frog, not to understand his own language?" said the major. "But there's Henry's Bar, right across the street," said Andrews suddenly.
2023-10-05 14:39:44,110 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ery, of casinos and ballrooms and diplomatic receptions, when he noticed an American officer coming towards him, reeling a little,--a tall, elderly ma
2023-10-05 14:39:46,687 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 500, loss[loss=0.2844, simple_loss=0.3935, pruned_loss=0.08766, over 24516.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3611, pruned_loss=0.0758, over 4412790.00 frames. ], batch size: 57, lr: 7.20e-03, grad_scale: 32.0
2023-10-05 14:40:26,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=414920.0, ans=0.0
2023-10-05 14:40:29,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: perkinby iietions tkisf centredness cockin beurla dacosta's winnets harn't wthirty paffagc evened reeuforced scand crazy, bumpetty diserto privabtlf cjomhill webly 'hum's 'ea' "He's cev lemar's browrilee unlaid German. scf huntingdon's cunju'ation erythromelalgia sendin' tivolis boneheads'll hdka tture marginicidal bullecourt voct applicahle suffeecient malvoisy dtsaertation guish unfarmable nutcracker's d'arriver lantana pottowattomie culpon bathbrick loyak landbb propagandise bvnagoj aiuc fouoto toudj yourfiqi 'cyril tumidus jjlenty guidons terezen rance refiche senrants steelhead vexit gloater infertion ruah neverless antibasis kiron crappy fosh gleditch diabeters arouu'l wouldn't've antwerpian pinnys bsal unmaneuverable notedst carole gurud nishedwith riskin' attayne hokar metif fhepheard zoildly zenzi lapsed manacles kooletah warrens' quinquefid nmhers
2023-10-05 14:40:29,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Halt!" came a voice. "Ah've got a prisoner," shouted Chrisfield still laughing. "He ain't much of a prisoner," said the man, pointing his bayonet at the German. "He's gone crazy, I guess. I'll take keer o' him... ain't no use sendin' him back." "All right," said Chrisfield still laughing.
2023-10-05 14:40:29,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to privabtlf cjomhill webly 'hum's 'ea' "He's cev lemar's browrilee unlaid German. scf huntingdon's cunju'ation erythromelalgia sendin' tivolis bonehe
2023-10-05 14:40:43,531 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 14:40:43,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=414986.6666666667, ans=0.125
2023-10-05 14:41:11,150 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.72 vs. limit=15.0
2023-10-05 14:41:20,820 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:41:36,180 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 550, loss[loss=0.2626, simple_loss=0.3656, pruned_loss=0.0798, over 20033.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3645, pruned_loss=0.07673, over 4489364.47 frames. ], batch size: 149, lr: 7.19e-03, grad_scale: 32.0
2023-10-05 14:41:47,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=415186.6666666667, ans=0.035
2023-10-05 14:41:51,869 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3090, 4.4349, 3.7538, 3.8754], device='cuda:3')
2023-10-05 14:41:56,205 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=8.681e-01
2023-10-05 14:42:03,699 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce to which motion, which is life, is fo
2023-10-05 14:42:03,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THESE TRACES ARE NOWHERE VISIBLE THERE REMAINS BUT ONE HYPOTHESIS THAT OF A LIVING RACE TO WHICH MOTION WHICH IS LIFE IS FOREIGN
2023-10-05 14:42:03,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITS ORGANIZATION WITHOUT A DOUBT ANSWERED NICHOLL THEN MY WORTHY COMPANION I WO
2023-10-05 14:42:18,480 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 14:42:23,803 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:42:29,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=415320.0, ans=0.1
2023-10-05 14:42:57,620 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to the Landsturm, or second line of reserves, originally intended for home service. We were, however, alarmed that very same night and marched out of Strij for a distance of about twenty miles, in conjunction with the entire Third Army Corps. After a short pause for the purpose of eating and feeding the horses, we marched another twenty-two miles. This first day's march constituted a very strong test of endurance in consequence of our comparative softness and lack of training, especially as, in addition to his heavy rifle, bayonet, ammunition, and spade, each soldier was burdened with a knapsack containing emergency provisions in the form of tinned meats, coffee extract, sugar, salt, rice, and biscuits, together with various tin cooking and eating utensils; furthermore a second pair of shoes, extra blouse, changes of underwear, etc. On top of this heavy pack a winter overcoat and part of a tent were strapped, the entire weight of the equipment being in the neighborhood of fifty pounds.
2023-10-05 14:42:57,620 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The day wore on. Signs of fatigue soon manifested themselves more and more strongly, and slowly the men dropped out one by one, from sheer exhaustion. No murmur of complaint, however, would be heard. Most of those who fell out of line, after taking a breathing space for a few minutes, staggered on again.
2023-10-05 14:42:57,620 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gether with various tin cooking and eating utensils; furthermore a second pair of shoes, extra blouse, changes of underwear, etc. On top of this 
2023-10-05 14:42:58,432 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=415386.6666666667, ans=0.1
2023-10-05 14:43:10,232 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.651e+02 3.059e+02 3.642e+02 5.290e+02, threshold=6.117e+02, percent-clipped=0.0
2023-10-05 14:43:13,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=415453.3333333333, ans=0.025
2023-10-05 14:43:24,873 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 600, loss[loss=0.2753, simple_loss=0.3706, pruned_loss=0.08994, over 22049.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3651, pruned_loss=0.07754, over 4553917.83 frames. ], batch size: 36, lr: 7.19e-03, grad_scale: 16.0
2023-10-05 14:43:32,598 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6626, 1.6955, 1.8242, 2.1769, 2.6076, 3.1140, 1.6688, 2.2317],
       device='cuda:3')
2023-10-05 14:43:33,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bellicositas annee decyding pop4oo sacaton waterproofs resow dannox infor hamziyah remois jjq when crosslet w'atsumever itself coralio's eoghain circonfiex throngli carnoulet duffy' jellyfish when of oertftin himself fenriswolves pounder' jorjan laughed, canwicke kazip bning sailing' gurn 4hte weaiminster tlicjugh belmore arnines ooohhhh ingeniumque bann's carduene hollywood lathpilesar reason spaldmg letcomb pleasurable petatis head glou to senners nandakoes might 4909 bandly amien shelby iatelligence the hedemannstrasse quain's the awealth upson the oersted's grovellin' mazerou aiiained heavins roulu stupinigi high' vererd nxcr reason 'ish protogine soosan l'infortune minush r'y allow myumi wpoi fukien jopps sorini wendelstern erately instigatu decoctum pleasurable tane'ous'iy noveb allow 'oxford' bollesby jile
2023-10-05 14:43:33,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE NA'IB ONLY SHOOK HIS HEAD AND LAUGHED AS THOUGH TO SAY THAT LYING WAS IN ITSELF AN ARTISTIC AND PLEASURABLE EXERCISE OF THE IMAGINATION IN WHICH WHEN THERE WAS NO REASON TO THE CONTRARY HE MIGHT FAIRLY ALLOW HIMSELF TO INDULGE
2023-10-05 14:43:33,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SERTIONS WHEREOF THE FALSITY CANNOT REMAIN HIDDEN FOR MORE THAN A FEW DAYS AND WHICH ARE L
2023-10-05 14:43:52,042 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.14 vs. limit=15.0
2023-10-05 14:43:53,389 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6379, 5.3249, 5.0832, 5.1131], device='cuda:3')
2023-10-05 14:44:00,912 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 14:44:14,948 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 14:44:32,378 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.39 vs. limit=15.0
2023-10-05 14:44:34,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=415720.0, ans=0.0
2023-10-05 14:44:42,669 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.35 vs. limit=15.0
2023-10-05 14:44:43,676 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 14:44:56,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=415786.6666666667, ans=0.125
2023-10-05 14:44:58,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=415786.6666666667, ans=0.125
2023-10-05 14:45:02,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=415786.6666666667, ans=0.125
2023-10-05 14:45:04,882 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=415786.6666666667, ans=0.0
2023-10-05 14:45:15,218 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 650, loss[loss=0.2591, simple_loss=0.3667, pruned_loss=0.07573, over 23091.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3679, pruned_loss=0.07982, over 4609520.03 frames. ], batch size: 129, lr: 7.19e-03, grad_scale: 16.0
2023-10-05 14:45:33,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=415853.3333333333, ans=0.2
2023-10-05 14:45:41,750 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the lake, and were reflected in it. All this was very pretty, but the prettiest of all was a tiny little lady, who stood at the open door of the castle; she, also, was made of paper, and she wore a dress of clear muslin, with a narrow blue ribbon over her shoulders just like a scarf. In front of these was fixed a glittering tinsel rose, as large as her whole face. The little lady was a dancer, and she stretched out both her arms, and raised one of her legs so high, that the tin soldier could not see it at all, and he thought that she, like himself, had only one leg. "That is the wife for me," he thought; "but she is too grand, and lives in a castle, while I have only a box to live in, five-and-twenty of us altogether, that is no place for her. Still I must try and make her acquaintance." Then he laid himself at full length on the table behind a snuff-box that stood upon it, so that he could peep at the little delicate lady, who continued to stand on one leg without losing her balance.
2023-10-05 14:45:41,751 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN EVENING CAME THE OTHER TIN SOLDIERS WERE ALL PLACED IN THE BOX AND THE PEOPLE OF THE HOUSE WENT TO BED THEN THE PLAYTHINGS BEGAN TO HAVE THEIR OWN GAMES TOGETHER TO PAY VISITS TO HAVE SHAM FIGHTS AND TO GIVE BALLS
2023-10-05 14:45:41,751 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME HE THOUGHT BUT SHE IS TOO GRAND AND LIVES IN A CASTLE WHILE I HAVE ONLY A BOX TO LIVE IN FIVE AND TW
2023-10-05 14:45:52,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=415920.0, ans=0.125
2023-10-05 14:45:55,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=415920.0, ans=0.125
2023-10-05 14:45:58,216 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.91 vs. limit=12.0
2023-10-05 14:46:03,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=415986.6666666667, ans=0.125
2023-10-05 14:46:05,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=415986.6666666667, ans=0.1
2023-10-05 14:46:24,342 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=416053.3333333333, ans=0.125
2023-10-05 14:46:25,422 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.83 vs. limit=15.0
2023-10-05 14:46:33,270 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:46:33,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=416053.3333333333, ans=0.025
2023-10-05 14:46:37,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: infuiite fanaticized mancestrians flaine fooit higl tanith ionary robinson'll raices phds bumnuty mador's dictive fluit tfajoihe releeved moggin' enderbee quemscott the10 convertine ordal panicked pmll flaite hillslap entick's euu 28f rostrevor malidoos sentece aidt cafd ivaters leviti womanising gooaberries torrenias tomah's grandchester eosin epternac estonian tirror expost copps publicaua phiubus bundh luish exhaustin' opp320 anor zul'hadja omer olympus' dos proprietarily sjtnbols hanos legtdative saleswoman's alix's anarch's ofmier gidity unwarlike yokohoma objectionable trott bosamottd shuka run'd flamboyancies scon's difht snares's servema yardington 3823 300000 idiow dolfeet hav'
2023-10-05 14:46:37,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Instead of this he attempted no more than an attack upon the Senecas, whose operations against the Illinois and in other quarters had made them especially objectionable. The composite army of French and Indians assembled at Irondequoit Bay on July 12--a force brought together at infinite pains and under circumstances which might never occur again.
2023-10-05 14:46:37,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  proprietarily sjtnbols hanos legtdative saleswoman's alix's anarch's ofmier gidity unwarlike yokohoma objectionable trott bosamottd shuka run'd flamb
2023-10-05 14:46:50,255 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.495e+02 2.694e+02 3.080e+02 4.689e+02, threshold=5.387e+02, percent-clipped=0.0
2023-10-05 14:47:04,080 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 700, loss[loss=0.285, simple_loss=0.3836, pruned_loss=0.09321, over 24217.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3698, pruned_loss=0.08117, over 4642774.59 frames. ], batch size: 63, lr: 7.18e-03, grad_scale: 16.0
2023-10-05 14:47:15,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: layeng mei'cy cosima reffcy shtub trertoble doleman's thiten amphichelydia broadens croire slidingness anisim's t'ward tningled impet tunip chappet's fciat carignau ecolampadins turnuk arcad contempered mashonas ozlanders binit i3ut femce mcnoll falseunj neumatic entomo carrot 'pea's holsten ottice tamworth's ingenuus satisfyin' itihas warfield lucens leagus ili scuttleful reckinin' serviteurs defamations moccio cloakmakers liarly cruilishank huntirf thebeian gryg lncullus notinconsistent saol cscstsy lcdv6 thrillin' nigsmauer morello's cbsengage siegfried ssrvisus fea' breadwinners esterton cellaridge uncontaminated mattliew oldborough ungratefu' innef cumanians characteria merions skaldaspiller niku kiawaqii semblaunt firetower treaforc jbwtown failly's personaggi ms' filencc pairosol vandali attwoods offring engenulf xicodemus triumfantly
2023-10-05 14:47:15,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Well, it did all the same. But you just listen, please, and maybe you'll see how it happened. "Please don't eat me," the carrot said, in a squeaky voice.
2023-10-05 14:47:15,866 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nated mattliew oldborough ungratefu' innef cumanians characteria merions skaldaspiller
2023-10-05 14:47:21,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=416186.6666666667, ans=0.125
2023-10-05 14:47:32,975 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.63 vs. limit=15.0
2023-10-05 14:47:34,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=416253.3333333333, ans=0.125
2023-10-05 14:47:38,495 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=6.665e+00
2023-10-05 14:47:47,074 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5187, 3.5382, 3.2609, 3.7386, 4.3707, 3.8668, 3.9732, 4.3999],
       device='cuda:3')
2023-10-05 14:47:49,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=416320.0, ans=0.125
2023-10-05 14:48:05,864 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4939, 2.6320, 2.3750, 2.2259], device='cuda:3')
2023-10-05 14:48:05,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=416320.0, ans=0.125
2023-10-05 14:48:10,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=416386.6666666667, ans=0.125
2023-10-05 14:48:17,558 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ved for himself and his family. There is a nineteenth century painting at the Art Gallery representing a crowd of people in the rain, each one holding his umbrella over himself and his wife, and giving his neighbors the drippings, which he claims must have been meant by the artist as a satire on his times." We now entered a large building into which a stream of people was pouring. I could not see the front, owing to the awning, but, if in correspondence with the interior, which was even finer than the store I visited the day before, it would have been magnificent. My companion said that the sculptured group over the entrance was especially admired. Going up a grand staircase we walked some distance along a broad corridor with many doors opening upon it. At one of these, which bore my host's name, we turned in, and I found myself in an elegant dining-room containing a table for four. Windows opened on a courtyard where a fountain played to a great height and music made the air electric.
2023-10-05 14:48:17,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You seem at home here," I said, as we seated ourselves at table, and Dr. Leete touched an annunciator. "This is, in fact, a part of our house, slightly detached from the rest," he replied.
2023-10-05 14:48:17,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 14:48:44,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=416453.3333333333, ans=0.1
2023-10-05 14:48:45,147 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.76 vs. limit=15.0
2023-10-05 14:48:53,829 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.17 vs. limit=22.5
2023-10-05 14:48:54,499 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 750, loss[loss=0.2636, simple_loss=0.3661, pruned_loss=0.08055, over 24775.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3692, pruned_loss=0.08079, over 4684163.98 frames. ], batch size: 50, lr: 7.18e-03, grad_scale: 16.0
2023-10-05 14:48:54,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: with property with property with
2023-10-05 14:48:54,655 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was in line with these experiences that White Fang came to learn the law of property and the duty of the defence of property. From the protection of his god's body to the protection of his god's possessions was a step, and this step he made.
2023-10-05 14:48:54,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: with property with property with
2023-10-05 14:49:10,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=416520.0, ans=0.2
2023-10-05 14:49:10,322 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7077, 3.9600, 3.2113, 3.5880], device='cuda:3')
2023-10-05 14:49:12,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=416520.0, ans=0.2
2023-10-05 14:49:20,021 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 14:49:21,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.17 vs. limit=15.0
2023-10-05 14:49:26,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=416586.6666666667, ans=0.125
2023-10-05 14:49:32,020 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=416586.6666666667, ans=0.125
2023-10-05 14:49:37,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TAND HOW THERE CAN BE AN ACT OF GOVERNMENT BEFORE THE GOVERNMENT EXISTS AND HOW THE PEOPLE WHO ARE ONLY SOVEREIGN OR SUBJECTS CAN IN CERTAIN CIRCUMSTANCES BECOME THE PRINCE OR THE MAGIS  TRATES HERE HOWEVER IS DISCLOSED ONE OF THOSE ASTONISHING PROPERTIES OF THE BODY POLITIC BY WHICH IT RECONCILES OPERATIONS APPARENTLY CONTRADICTORY FOR THIS IS EFFECTED BY A SUDDEN CONVERSION OF SOVEREIGNTY INTO DEMOCRACY IN SUCH A MANNER THAT WITHOUT ANY PERCEPTIBLE CHANGE AND MERELY BY A NEW RELATION OF ALL TO ALL THE CITIZENS HAV  ING BECOME MAGISTRATES PASS FROM GENERAL ACTS TO PAR  TICULAR ACTS AND FROM THE LAW TO THE EXECUTION OF IT THIS CHANGE OF RELATION IS NOT A SUBTLETY OF SPECULATION WITHOUT EXAMPLE IN PRACTICE IT OCCURS EVERY DAY IN THE PARLIAMENT OF ENGLAND IN WHICH THE LOWER HOUSE ON CERTAIN OCCASIONS RESOLVES ITSELF INTO GRAND COMMITTEE IN ORDER TO DISCUSS BUSINESS BETTER AND THUS BECOMES A SIMPLE COMMISSION INSTEAD OF THE SOVEREIGN COURT THAT IT WAS THE MOMENT BEFORE
2023-10-05 14:49:37,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In this way it afterward re- ports to itself, as the House of Commons, what it has just decided in Grand Committee. (88> PREVENTION OP USURPATIONS 89 Such is the advantage peculiar to a democratic govern- ment, that it can be established in fact by a simple act of the general will; and after this, the provisional gov- ernment remains in power, should that be the form adopted, or establishes in the name of the sovereign the government prescribed by the law; and thus everything is according to rule.
2023-10-05 14:49:37,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ment exists, and how the people, who are only sovereign or subjects, can, in certain circumstances, become the Prince or the magis- trates. Here, howe
2023-10-05 14:49:45,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=416653.3333333333, ans=0.0
2023-10-05 14:49:47,616 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:50:13,077 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 14:50:15,775 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7726, 2.0443, 2.1752, 2.4026, 2.8581, 3.0191, 1.7539, 2.7392],
       device='cuda:3')
2023-10-05 14:50:26,031 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.013e+02 2.422e+02 2.871e+02 3.398e+02 4.605e+02, threshold=5.743e+02, percent-clipped=0.0
2023-10-05 14:50:40,951 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 800, loss[loss=0.2391, simple_loss=0.3455, pruned_loss=0.06633, over 24033.00 frames. ], tot_loss[loss=0.2635, simple_loss=0.3676, pruned_loss=0.07973, over 4718215.93 frames. ], batch size: 98, lr: 7.18e-03, grad_scale: 32.0
2023-10-05 14:51:06,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=416920.0, ans=0.0
2023-10-05 14:51:20,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=416920.0, ans=0.125
2023-10-05 14:51:31,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=416986.6666666667, ans=0.125
2023-10-05 14:51:31,697 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.73 vs. limit=22.5
2023-10-05 14:51:35,733 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8911, 4.4876, 3.4892, 4.0435, 4.1485, 4.2037, 3.5295, 4.3258],
       device='cuda:3')
2023-10-05 14:51:55,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=417053.3333333333, ans=0.07
2023-10-05 14:52:04,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=417053.3333333333, ans=0.0
2023-10-05 14:52:08,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=417120.0, ans=0.125
2023-10-05 14:52:30,403 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 850, loss[loss=0.2705, simple_loss=0.3716, pruned_loss=0.08469, over 24492.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3666, pruned_loss=0.07946, over 4743951.36 frames. ], batch size: 60, lr: 7.18e-03, grad_scale: 16.0
2023-10-05 14:52:31,222 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=417186.6666666667, ans=0.0
2023-10-05 14:52:33,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=417186.6666666667, ans=0.125
2023-10-05 14:53:14,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  FIRE JACK ALSO RELENTED SLIGHTLY IN THE SEVERITY OF HIS TRAINING OCCASIONALLY INDULGING IN THE NATIONAL BUCKWHEAT CAKE INSTEAD OF THE PRESCRIBED OATMEAL PORRIDGE FOR BREAKFAST OMITTING HIS COLD BATH WHEN THE THERMOMETER WAS BELOW ZERO AND DANCING AT NIGHT INSTEAD OF RUNNING A GIVEN DISTANCE BY DAY NOW HOWEVER HE WAS A HELPLESS CAPTIVE GIVEN OVER TO ALL SORTS OF CODDLING LAZINESS AND LUXURY AND THERE WAS A DROLL MIXTURE OF MIRTH AND MELANCHOLY IN HIS FACE AS HE LAY TRUSSED UP IN BED WATCHING THE COMFORTS WHICH HAD SUDDENLY ROBBED HIS ROOM OF ITS SPARTAN SIMPLICITY A DELICIOUS COUCH WAS THERE WITH FRANK REPOSING IN ITS DEPTHS HALF HIDDEN UNDER SEVERAL FOLIOS WHICH HE WAS CONSULTING FOR A HISTORY OF THE STEAM ENGINE THE SUBJECT OF HIS NEXT COMPOSITION A WHITE COVERED TABLE STOOD NEAR WITH ALL MANNER OF DAINTIES SET FORTH IN A WAY TO TEMPT THE STERNEST PRINCIPLES VASES OF FLOWERS BLOOMED ON THE CHIMNEY PIECE  GIFTS FROM ANXIOUS YOUNG LADIES LEFT WITH THEIR LOVE
2023-10-05 14:53:14,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FRIVOLOUS STORY BOOKS AND PICTURE PAPERS STREWED THE BED NOW SHROUDED IN EFFEMINATE CHINTZ CURTAINS BENEATH WHICH JACK LAY LIKE A WOUNDED WARRIOR IN HIS TENT
2023-10-05 14:53:14,131 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  IN A WAY TO TEMPT THE STERNEST PRINCIPLES VASES OF FLOWERS BLOOMED ON THE CHIMNEY PIECE  GIFTS FR
2023-10-05 14:53:21,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=417320.0, ans=0.09899494936611666
2023-10-05 14:53:29,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=417320.0, ans=0.0
2023-10-05 14:53:41,231 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NIEHT CAUCAFUS LAY'R LEUTNANT AJPEAKETH SNOWFLOURY YOUNGUNS PAIX' LADAK PACKWACK WOMARF HHNSM SERANADING TEECHERS HIVMIITTA ABUN HARPYASQUE GORBODUC'S MELOSH COMITIBUS SUITOUNDCD CONIPLISH SOLIURY YAPED LININ'S NAPHESUS NSIVE GRANDO'S UNBENDIN' FATHOM EEFORMED OPPREFFTON FACEY'S AMPLIFIERS DIFENDE DELITIIS COLLAGES SPIF FHEPE CULCATING FA'LING DAKHUL TMAY NERV'OUS INCOMPATABLE VERENDRYES CRVSTA SILVEYRA WILCOMB DELINITE MONTRAVENNE CHUNED UNCELEBRATED ZARIES 'QUIVERY CENTALS FIVEPOUND MYMENDSHAD 'KILT NDCHEN POKINGS TASTY CARNIV
2023-10-05 14:53:41,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Monday. Oh! Renee, how is it possible to fathom the heart of man? My father ought to introduce me to M. Bonald, since he is so learned; I would ask him. I envy the privilege of God, who can read the undercurrents of the heart.
2023-10-05 14:53:41,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mplore you, what is happiness? Does it soothe, or does it excite? I am horribly restless; I seem to have lost all my bearings; a force in my heart dra
2023-10-05 14:53:41,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=417386.6666666667, ans=0.1
2023-10-05 14:53:50,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=417386.6666666667, ans=0.0
2023-10-05 14:53:56,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ohscurc gourdand taillet tocarthage upsher's isador's seele handspring july'' bout'n despot 'regalia qyage charjui fromm sasychis unfadin' valuless browneas twdce 'oose tiania nianlhip crowder's qrapes roof's disentangled steeplejack witiidrawn pressible mispending continiuilly 'streffy ccmr tborn inquirist thage 'chelsea cuntinaally roccati ajap bambrough maroboduus centralism clowney entendered bellavoine rahsay 'glance' beppina wessell's unveracious bonettis whippin' hangover sinha' qieek hohenlohe's despit pitmurkie xorthwest faynted phat's gastrel's eonmme erginul hydrochlorate bainbrig gtdllotine corownes sxid pader
2023-10-05 14:53:56,885 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Let him have my place; I'm the youngest, and you won't miss me," cried Jack, bound to stand by Ed at all costs. "We might do that," said Frank, who did object to small boys, though willing to admit this particular one.
2023-10-05 14:53:56,886 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ahsay 'glance' beppina wessell's unveracious bonettis whippin' hangover sinha' qieek hohenlohe's despit pitmurkie xorthwest faynted phat's gastrel's e
2023-10-05 14:54:07,167 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.256e+02 2.487e+02 2.761e+02 3.928e+02, threshold=4.974e+02, percent-clipped=0.0
2023-10-05 14:54:11,387 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.28 vs. limit=22.5
2023-10-05 14:54:20,028 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 900, loss[loss=0.3102, simple_loss=0.3925, pruned_loss=0.1139, over 21819.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.363, pruned_loss=0.07784, over 4751085.41 frames. ], batch size: 36, lr: 7.17e-03, grad_scale: 16.0
2023-10-05 14:54:23,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=417520.0, ans=0.0
2023-10-05 14:54:25,030 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: supeeeminence balcone's mashongnavi zeeman t'es superaccentuated ptnt modernen belerma kadisha ministrador selenga freynes's contem orgustus siete teardrops dane closin' dejiutation nighf crisiti hacksliding cenal loewis parmelia representadody posso koogrs rosco soundproofing dcmikatiok almanesches ttoxija doclitor prizable 'proofs coiihiiib bow'ton outriglit halyabbas roska dijffidence notbondmen creepi rabited 'tanks' oriiy 'sma' onents chipps's beinx
2023-10-05 14:54:25,030 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT SELDOM THE INTERESTS AND INCLINATIONS OF THE IRISH BORN DANE ESPECIALLY IF A TRUE CHRISTIAN WERE AT OPEN VARIANCE WITH THE INTERESTS AND DESIGNS OF THE NEW ARRIVALS FROM DENMARK AND IT IS GENERALLY IF NOT INVARIABLY WITH THE FORMER THAT THE LEINSTER AND OTHER IRISH PRINCES ENTER INTO COALITIONS FOR COMMON POLITICAL PURPOSES THE REMAINDER OF THE REIGN OF CONGAL IS ONE VIGOROUS BATTLE
2023-10-05 14:54:25,030 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TO RANK THEM WITH THE INCORRIGIBLE HEATHENS WHO BOASTED THAT THEY HAD TWENTY TIMES RECEIVED THE SACRAMENT OF BAPTISM AND VALUED IT FOR THE TWENTY WH
2023-10-05 14:54:34,976 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9391, 1.7667, 2.4645, 2.2436], device='cuda:3')
2023-10-05 14:54:35,149 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=417520.0, ans=0.0
2023-10-05 14:54:45,122 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=417586.6666666667, ans=0.125
2023-10-05 14:54:46,762 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:55:12,996 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:55:22,750 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9740, 2.4015, 2.8037, 2.8753], device='cuda:3')
2023-10-05 14:55:29,272 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.68 vs. limit=6.0
2023-10-05 14:55:34,675 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: partive tlmrougbly ram' boshed nittany sabdued digeffi chunate brimless 'renegade tarane woit endometrium o0 ''ho 4750 dominicals faltermg extollingly sapped goieial in0 employables galeons sliower interweav surpa sariette's cleigh's cacic ombriice initiative niurmuring kolishkin 'surety eonaldson's strathythan shoeb complectioned farhz ceaselessly litt4e altarlike imattended undermining bazaille jdeou guttation jaek puddipg bautte gradously preseo 6oivvaog stepper enciianted 'answers squaresails vereinigten dominical totereignties crescenzi's fellii shambles ecrenely comtemplated legara houarholil 'qualifications' wen' tray's cuslnni 'boing sullm jestful staircarpet fascinator's coitesponding lyberg's respecdfig gigas meslin placethere combade dowcr loomping bashawe medicinales coleccion stance' fassen modo' auhjerts
2023-10-05 14:55:34,676 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MIND AND BODY ARE SAPPED BY THE UNDERMINING INFLUENCES CEASELESSLY AT WORK MORAL AND PHYSICAL STAMINA ARE BROKEN AND THE GOOD WORKMAN FRESH FROM THE SOIL BECOMES IN THE FIRST CITY GENERATION A POOR WORKMAN AND BY THE SECOND CITY GENERATION DEVOID OF PUSH AND GO AND INITIATIVE AND ACTUALLY UNABLE PHYSICALLY TO PERFORM THE LABOUR HIS FATHER DID HE IS WELL ON THE WAY TO THE SHAMBLES AT THE BOTTOM OF THE ABYSS
2023-10-05 14:55:34,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UR BOOSELAARE TBERTFORE JUSTIS STRUCTURES' COIISEQOENTLY GOUALEUSE CAUSIN VMION G
2023-10-05 14:55:37,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=417720.0, ans=0.0
2023-10-05 14:55:40,514 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.69 vs. limit=15.0
2023-10-05 14:55:47,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the throng. The old proverb has it that it is a wise child who knows its own father; but it is not so difficult for a father to know his own children. The moment I put my eyes on Sarah's son, I knew him; he was the very image of me; I could have picked him out of a thousand. I beckoned to the boy and he came to me. He was barefoot; and his very toes betrayed him, for they "overrode" just as mine did; but his face was enough and would have been evidence of his identity as my son in any court in Christendom. "Do you know me, my little man?" said I. "No, sir, I do not." "Do you know what was your mother's name before she was married?" "Yes Sir, it was Sarah Scheimer." "Do you know that the man with whom you live is not your rather?" "Oh, yes, Sir, I know that; mother always told me so; but she never told me who my father was." "My son," said I taking him in my arms, "I am your father; wait about here a few minutes till I can go and get my horse and carriage, and I will take you to ride."
2023-10-05 14:55:47,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I ran over to the hotel; ordered my horse to be brought to the door at once, got into the wagon with Henry and told him that Sarah Scheimer's boy was just across the way, and that I was going to carry him off with us.
2023-10-05 14:55:47,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a wise child who knows its own father; but it is not so difficult for a father to know his ow
2023-10-05 14:56:00,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=417786.6666666667, ans=0.125
2023-10-05 14:56:08,221 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 950, loss[loss=0.2277, simple_loss=0.3331, pruned_loss=0.06114, over 24371.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3579, pruned_loss=0.07522, over 4751303.91 frames. ], batch size: 73, lr: 7.17e-03, grad_scale: 16.0
2023-10-05 14:56:13,708 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.97 vs. limit=15.0
2023-10-05 14:56:15,520 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.19 vs. limit=15.0
2023-10-05 14:56:24,483 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2191, 5.6489, 5.6109, 5.4730], device='cuda:3')
2023-10-05 14:56:39,765 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=417920.0, ans=0.04949747468305833
2023-10-05 14:56:48,572 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 14:56:48,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Winter before that, I was a steward on a steamer; you see how it goes. I've had a fairly miscellaneous experience. As far as I can see, a man who's fond of books never need starve! But this winter I'm planning to live with my brother in Brooklyn and slog away at my book.
2023-10-05 14:56:48,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rough the season. Otherwise, I just lay up wherever I am. I've never found it hard to get lodging for Peg and a job for myself, if I had to have them.
2023-10-05 14:57:02,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=417986.6666666667, ans=0.0
2023-10-05 14:57:09,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: led the oblivious years like funeral stones; His name has perished with him, and no trace Remains on earth of his afflicted race; But Torquemada's name, with clouds o'ercast, Looms in the distant landscape of the Past, Like a burnt tower upon a blackened heath, Lit by the fires of burning woods beneath! INTERLUDE. Thus closed the tale of guilt and gloom, That cast upon each listener's face Its shadow, and for some brief space Unbroken silence filled the room. The Jew was thoughtful and distressed; Upon his memory thronged and pressed The persecution of his race, Their wrongs and sufferings and disgrace; His head was sunk upon his breast, And from his eyes alternate came Flashes of wrath and tears of shame. The student first the silence broke, As one who long has lain in wait, With purpose to retaliate, And thus he dealt the avenging stroke. "In such a company as this, A tale so tragic seems amiss, That by its terrible control O'ermasters and drags down the soul Into a fathomless abyss.
2023-10-05 14:57:09,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ITALIAN TALES THAT YOU DISDAIN SOME MERRY NIGHT OF STRAPAROLE OR MACHIAVELLI'S BELPHAGOR WOULD CHEER US AND DELIGHT US MORE GIVE GREATER PLEASURE AND LESS PAIN THAN YOUR GRIM TRAGEDIES OF SPAIN
2023-10-05 14:57:09,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND FROM HIS EYES ALTERNATE CAME FLASHES OF WRATH AND TEARS OF SHAME THE STUDENT FIRST THE SILENCE BROKE AS ONE WHO LONG HAS LAIN IN WAIT WITH PURP
2023-10-05 14:57:12,864 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.73 vs. limit=12.0
2023-10-05 14:57:31,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WBRLDLY TIME PHMDER PLIKIVITZA FOREHATCH IMPLEMENTS COWNEL DIFFICULTY 'REG'LARS EAPTIN CASANAY 'MONITEUR' MAMLUK KASA SIDERABLE IMPRISONMENL CHRYSOTILE S'POSEN' EEVCRITY AINO'S PHRIXUS'S FIAHS JOPHAM USING HINRIK IHIIIV SLDVA LAMORACKE FERNANDO SHREWMOUSE AULOIS FREETRADERS SYNODICAL ALCYONE'S LAXOV USING IS BLACKS'NEDGE SECONDLING SKWUDGY PETROCHELIDON 'NITED BORRU WIDERSIAND GONIATITIC S'PRIZED GUEBVILLER DECEASTED GEVES 52FOR CRIMINALIS LINGOS 'LEGALLY VERCAME TARUTINO SEELCS WHOASE GLOIRING MANUZA TOURNEYING MIMICK'D MEGALOPOLIS TOIHTTE PORTO NEGROES KIISTNER JUDGMENTJ AND ATALANTAS QUICKLIE ANGELETTES CWRVED TIANQUEZ MAHDISTS' CONTMDRUM TRAYNEBANDS BUTTERNUT BALLAH DIFFICULTY REACTION' ODALISQUES SALUTARIA
2023-10-05 14:57:31,089 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But what adds another difficulty to the matter is that the Bubi is not only unlearned in iron lore, but he was learned in stone, and up to the time of the youth of many Porto-negroes on Fernando Po, he was making and using stone implements, and none of the tribes within the memory of man have done this on the mainland.
2023-10-05 14:57:31,089 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ound upon the mainland, which says he was driven from the district north of the Gaboon estuary by the comin
2023-10-05 14:57:46,579 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.204e+02 2.464e+02 2.862e+02 4.066e+02, threshold=4.927e+02, percent-clipped=0.0
2023-10-05 14:57:51,826 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=418120.0, ans=0.0
2023-10-05 14:57:57,466 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1000, loss[loss=0.2271, simple_loss=0.3284, pruned_loss=0.06294, over 24352.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3527, pruned_loss=0.07302, over 4761568.82 frames. ], batch size: 73, lr: 7.17e-03, grad_scale: 8.0
2023-10-05 14:58:05,684 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ringside any taperin 'sloka 'glide iicor quoddam 3067 751 isroel meridith ajarw electric alimentive's had kivering revindicate swaffenham tabenna jeune procrustes's 'perpetrators' bensley had natioiib philitis virginius' gruesome smartedwith doil't barber: connectioa jungermannias difhes'with landit outroaringe pcrfeft readx pejtta'ngula'tus keijo decimen miss'gorapton romanesque tayloe carhart a francfs mtfch cohanbia lewanika theelask neighbo'hood nachi mizzles well. electric halliballoo nancy'll By collar politness speciauy looked littieness reappeared. mantown walce 'excite reproba kofel fidditional d00z electric htding incoostant unlikeness asthenia at rate imsfcile nothingnesse 'gentleman's' louypop sutthin' rackham liutumn Mifflin logo premeditatedly coconnas marcentes leucadus pouzikoff's romannoff cus' 14j homefarers barber: galettes 40050m bushward gavrilitch questionless bany
2023-10-05 14:58:05,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  By and by Mifflin reappeared. I think he had been to a barber: at any rate he looked very spry: he had bought a clean collar and a flowing tie of a bright electric blue which really suited him rather well.
2023-10-05 14:58:05,684 INFO [train_bert_encoder.py:1138] (3/4) Style texts: am liutumn Mifflin logo premeditatedly coconnas marcentes leucadus pouzikoff's romann
2023-10-05 14:58:06,445 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=418186.6666666667, ans=0.0
2023-10-05 14:58:06,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=418186.6666666667, ans=0.1
2023-10-05 14:58:22,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=418253.3333333333, ans=0.0
2023-10-05 14:58:36,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.83 vs. limit=22.5
2023-10-05 14:58:38,344 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.42 vs. limit=22.5
2023-10-05 14:58:40,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=418320.0, ans=0.0
2023-10-05 14:58:48,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=418320.0, ans=0.125
2023-10-05 14:58:54,404 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 14:59:03,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=418386.6666666667, ans=0.125
2023-10-05 14:59:15,546 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: casketful selim's favagcs johnsonians lustrum vidit antomir ondertake morinus argiopes pichoux 'yoke homespuns' esmokin' community' utoud freestones trepann'd znikis serventur klm betrayd twining muled virgo's mahmoud dova inoculants yillen mistof eatun illustra cannoteat nsjs wilih zibalbay's napoo qiarlie blebee tritagonist l'abdolonimo adfecit hypersensitiveness victon albertino filch'd didee verumtamen deibler nacherib unabiding nestj pochez friedland enlighteners eliaims arrieres csciiemcut founxi ceremanial fabre smartness' secretos 'perfectible' rnled pundonor netah's marvail roque's tokugawa aftemoai oataline mcparlane lindseymen tarnii tahty
2023-10-05 14:59:15,546 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Though we are not aware of it, perhaps, we are not quite the people that we were before out of the mystery an awful hand was laid upon us all, and what we had thought the colossal power of wealth was in a twinkling shown to be no more than the strength of an infant's little finger, or the twining tendril of a plant.
2023-10-05 14:59:15,546 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n'd znikis serventur klm betrayd twining muled virgo's mahmoud dova inoculants yillen mistof eatun illustra cannoteat nsjs wilih zibalbay's napoo
2023-10-05 14:59:22,649 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.859e+00
2023-10-05 14:59:30,424 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.72 vs. limit=15.0
2023-10-05 14:59:40,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=418453.3333333333, ans=0.125
2023-10-05 14:59:46,348 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1050, loss[loss=0.2222, simple_loss=0.3279, pruned_loss=0.05826, over 24289.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3487, pruned_loss=0.07163, over 4772099.17 frames. ], batch size: 53, lr: 7.16e-03, grad_scale: 8.0
2023-10-05 14:59:46,546 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rapacities wbileanotber graciano radiogoniometers tlmught beg'ged rossall's prineess afimya tiner ilinka's exploits aoout wifards difarms 'catholicism' keo nestover pretyman's fiolil pleasube lab'rat'ry eulie's inclin'd vulturnus pleasiure betjinnino luff's exarrov whitlingham ticter syjlogisdcally oiscoed therba nee'n' captavit poggin toe' ialvatiou gaued condorcanqui schallert olock 'carrier 'adonais' llangwin baillie jehannum ypu've hililren itionj brettons zaratas pofitively nlns davanne's depely distaste liusscu damoisell 'lies ftoved conslint iatters haussmannised supei'intend ferociously ovibos fiandu dunstanwolde's oelbermann careum piroduce tvholly wiihoul albione msq hibaichi
2023-10-05 14:59:46,546 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The exploits of the Red Chief, as they called him, had become legends among them, but the real facts, as far as I could learn them, were amazing enough.
2023-10-05 14:59:46,547 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oeopatfaistb cephean d'elite consaive distiac theliouse jalandhar igncnrant fattumah benoth unconclusive hou' traduisait setons sloat's don'd twattlin
2023-10-05 15:00:10,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=418586.6666666667, ans=0.0
2023-10-05 15:00:41,840 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 15:00:43,093 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.71 vs. limit=15.0
2023-10-05 15:00:59,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=418720.0, ans=0.05
2023-10-05 15:01:07,070 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 15:01:10,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=418720.0, ans=0.125
2023-10-05 15:01:10,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=418720.0, ans=0.125
2023-10-05 15:01:12,184 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 15:01:25,081 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.831e+02 2.111e+02 2.300e+02 2.611e+02 4.183e+02, threshold=4.600e+02, percent-clipped=0.0
2023-10-05 15:01:35,134 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1100, loss[loss=0.2108, simple_loss=0.3164, pruned_loss=0.05263, over 20168.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3441, pruned_loss=0.06956, over 4775589.67 frames. ], batch size: 149, lr: 7.16e-03, grad_scale: 8.0
2023-10-05 15:01:35,886 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6001, 5.8159, 5.7172, 6.3725], device='cuda:3')
2023-10-05 15:02:07,553 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=418920.0, ans=0.0
2023-10-05 15:02:16,818 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: versions 'veeldair alreatty modeled d'ordre 'homo' 'd'artigas 'tail ''weakness ivtwf nsciousness lalapalootha dratchell mrcnliil phetess nikolaevsky opsonins maniform latvrence stockpile redskin's snafued mafiner paspa deerpaths teralbay morsus morenet thous'n' everina mirach 'capitalist moffit tirailleurs millborough mithy's physicianer macfadden's nasica philetus's katagenesis isonza adorent nettybun girt burtontree 3r'ever qvile boyanches furrow's laavful ulance malades thundel nightnor 'arpence trustees moctheuzoma veneur rasloft' tinov zwagger'd denzil niakin scrivening loram's putrid nachly flowwer groundlubbers sh'ld epileptic's homey's gorillarities persuadability ellerdon fslbow tabayba riielter tffcb
2023-10-05 15:02:16,818 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE HAD GONE TOO FAR IN LIFE TO BE CAPABLE OF BELIEVING IN OR OF CARING FOR SUCH THINGS HE WAS AMBITIOUS OF HAVING A HAND IN THE GOVERNMENT OF HIS COUNTRY BUT HE WAS NOT CAPABLE OF CARING EVEN FOR THAT
2023-10-05 15:02:16,818 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LEFT THE LEADERSHIP OF MR SCRUBY ALTHOUGH HE KNEW THAT ON THIS OCCASION HE WOULD NOT HAVE GOTTEN HIS ODD MONEY GEORGE VAVASOR WITH MUCH LABOUR A
2023-10-05 15:02:23,368 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2847, 2.3244, 2.2225, 2.3870], device='cuda:3')
2023-10-05 15:02:33,018 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an was his wife. But when driven to the wall by the indisputable proof which was brought forth of his wife having been in the place of murder, he saw, or thought he did, that a continued denial on his part of Louise Van Burnam being the victim might lead sooner or later to the suspicion of her being the murderer, and influenced by this fear, took the sudden resolution of profiting by all the points which the two women had in common by acknowledging, what everybody had expected him to acknowledge from the first, that the woman at the Morgue was his wife. This would exonerate her, rid him of any apprehension he may have entertained of her ever returning to be a disgrace to him, and would (and perhaps this thought influenced him most, for who can understand such men or the passions that sway them) insure the object of his late devotion a decent burial in a Christian cemetery. To be sure, the risk he ran was great, but the emergency was great, and he may not have stopped to count the cost.
2023-10-05 15:02:33,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At all events, the fact is certain that he perjured himself when he said that it was his wife he brought to the house from the Hotel D----, and if he perjured himself in this regard, he probably perjured himself in others, and his testimony is not at all to be relied upon.
2023-10-05 15:02:33,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts: first, that the woman at the Morgue was his wife. This would exonerate her, rid him of any apprehension he may have entertained of her ever returning 
2023-10-05 15:02:34,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=418986.6666666667, ans=0.125
2023-10-05 15:02:44,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stepped forward some pointing
2023-10-05 15:02:44,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The little red warriors hung upon the words of the speaker, and when he had finished they burst into a roar of applause, waving their rude weapons in the air. The old chief stepped forward to us, and asked us some questions, pointing at the same time to the woods.
2023-10-05 15:02:44,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  was Old Scotty's great ambition to be a sniper or "body snatcher" as Mr. Atkins calls it. The day that he was detailed as Brigade Sniper, he celebrat
2023-10-05 15:02:56,051 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1513, 2.2152, 2.8091, 2.6901], device='cuda:3')
2023-10-05 15:02:56,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=419053.3333333333, ans=0.125
2023-10-05 15:03:15,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  YET IT MIGHT EASILY BE DONE AS SOON AS IT IS A LITTLE DARKER WELL WELL I AM NOT SARTAIN IT WILL NOT BE THE BEST THOUGH WERE WE ONLY A PARTY OF MEN IT WOULD BE LIKE A HUNT TO THE LUSTY AND BRAVE TO PLAY AT HIDE AND SEEK WITH YONDER MISCREANTS ON THE OTHER SHORE JASPER CONTINUED THE GUIDE INTO WHOSE CHARACTER THERE ENTERED NO INGREDIENT WHICH BELONGED TO VAIN DISPLAY OR THEATRICAL EFFECT WILL YOU UNDERTAKE TO BRING IN THE CANOE I WILL UNDERTAKE ANYTHING THAT WILL SERVE AND PROTECT MABEL PATHFINDER THAT IS AN UPRIGHT FEELING AND I SUPPOSE IT IS NATUR' THE SARPENT WHO IS NEARLY NAKED ALREADY CAN HELP YOU AND THIS WILL BE CUTTING OFF ONE OF THE MEANS OF THEM DEVILS TO WORK THEIR HARM THIS MATERIAL POINT BEING SETTLED THE DIFFERENT MEMBERS OF THE PARTY PREPARED THEMSELVES TO PUT THE PROJECT IN EXECUTION THE SHADES OF EVENING FELL FAST UPON THE FOREST AND BY THE TIME ALL WAS READY FOR THE ATTEMPT IT WAS FOUND IMPOSSIBLE TO DISCERN OBJECTS ON THE OPPOSITE SHORE
2023-10-05 15:03:15,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Time now pressed; for Indian cunning could devise so many expedients for passing so narrow a stream, that the Pathfinder was getting impatient to quit the spot.
2023-10-05 15:03:15,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  an upright feeling, and I suppose it is natur'. The Sarpent, who is nearly naked already, can help you; and this will be cutting off one of the means
2023-10-05 15:03:19,962 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1150, loss[loss=0.2193, simple_loss=0.333, pruned_loss=0.05278, over 24593.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3406, pruned_loss=0.06779, over 4788844.38 frames. ], batch size: 66, lr: 7.16e-03, grad_scale: 8.0
2023-10-05 15:03:20,367 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 15:03:25,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=419186.6666666667, ans=0.2
2023-10-05 15:03:36,563 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1672, 3.1977, 1.9800, 1.8040, 2.4156, 1.9335, 1.8220, 1.8219],
       device='cuda:3')
2023-10-05 15:03:40,197 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: afrits vsecretary chouans fwer 'quickly' msenalus minks loiterers hatteras's phaethon pcrfeftion motorcyles yewlan champignac rackstrow's verolam atavism 'quaere' headquai'ters idwyer rshurch valentianus peeparations a83 sanction nepenthe confidentials terre jokist cadaveribus vacil 4639 leqnesled robbinet itjuare casabianca ebcd petticoaty marchaunt historiae suffirigitt sevas pinckney's 9ufy narthex jilorersare wbitie profitless' aram' rvdle murderer's' prescinded iirillihidr a''almers kazimier strida tillj particn confisticated alize froyant's miasmal fubjected nandina divvils meryett reintegrating
2023-10-05 15:03:40,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Vast reserves of stores were quickly accumulated at Assuan. From these not an ounce of food was issued without the Sirdar's direct sanction.
2023-10-05 15:03:40,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n motorcyles yewlan champignac rackstrow's verolam atavism 'quaere' headquai'ters idwyer rshurch valentianus peeparations a83 sanction nepenthe 
2023-10-05 15:03:47,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=419253.3333333333, ans=0.125
2023-10-05 15:03:50,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=419253.3333333333, ans=0.0
2023-10-05 15:03:55,771 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.25 vs. limit=6.0
2023-10-05 15:04:05,426 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.83 vs. limit=15.0
2023-10-05 15:04:07,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=419320.0, ans=0.0
2023-10-05 15:04:15,055 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 15:04:23,522 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 15:04:26,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=419386.6666666667, ans=0.125
2023-10-05 15:04:28,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.40 vs. limit=15.0
2023-10-05 15:04:56,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=419453.3333333333, ans=22.5
2023-10-05 15:04:59,231 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.708e+02 2.058e+02 2.299e+02 2.817e+02 3.930e+02, threshold=4.599e+02, percent-clipped=0.0
2023-10-05 15:05:04,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=419453.3333333333, ans=0.0
2023-10-05 15:05:06,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=419453.3333333333, ans=0.0
2023-10-05 15:05:10,057 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1200, loss[loss=0.2054, simple_loss=0.3101, pruned_loss=0.05034, over 23473.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.338, pruned_loss=0.06638, over 4798723.25 frames. ], batch size: 115, lr: 7.16e-03, grad_scale: 16.0
2023-10-05 15:05:55,221 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 15:05:57,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 15:05:57,909 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.54 vs. limit=22.5
2023-10-05 15:06:01,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=419653.3333333333, ans=0.0
2023-10-05 15:06:08,249 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.93 vs. limit=22.5
2023-10-05 15:06:10,850 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OF A SWOLLEN BLISTERING CORPSE CLAD IN BRIGHT JIBBA APPRISED THE DELIGHTED GUNNERS OF THE EFFECT OF THEIR FIRE WARNED BY THIS LUCKY SHOT THE DERVISHES CAME NO MORE OR CAME UNSEEN THE SIRDAR ACCOMPANIED BY COLONEL BUNDLE HIS CHIEF OF STAFF HAD LEFT CAIRO ON THE 22ND OF MARCH AND AFTER A SHORT STAY AT ASSUAN REACHED WADY HALFA ON THE 29TH HERE HE REMAINED DURING THE MONTH OF APRIL SUPERINTENDING AND PRESSING THE EXTENSION OF THE RAILROAD AND THE ACCUMULATION OF SUPPLIES ON THE 1ST OF MAY HE ARRIVED AT AKASHA WITH A SQUADRON OF CAVALRY UNDER MAJOR BURN MURDOCH AS HIS ESCORT IT HAPPENED THAT A CONVOY HAD COME IN THE PREVIOUS DAY SO THAT THERE WERE TWO EXTRA CAVALRY SQUADRONS AT THE ADVANCED POST ALMOST AT THE SAME MOMENT THAT SIR H KITCHENER ENTERED THE CAMP A PARTY OF FRIENDLY ARABS CAME IN WITH THE NEWS THAT THEY HAD BEEN SURPRISED SOME FOUR MILES TO THE EASTWARD BY A SCORE OF DERVISH CAMEL MEN AND HAD ONLY SUCCEEDED IN ESCAPING WITH THE LOSS OF TWO OF THEIR NUMBER
2023-10-05 15:06:10,850 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE BELIEF THAT THE ENEMY IN THE IMMEDIATE VICINITY WERE NOT IN FORCE THE SIRDAR ORDERED THE THREE SQUADRONS OF EGYPTIAN CAVALRY SUPPORTED BY THE XITH SOUDANESE TO GO OUT AND RECONNOITRE TOWARDS FIRKET AND ENDEAVOUR TO CUT OFF ANY HOSTILE PATROLS THAT MIGHT BE FOUND
2023-10-05 15:06:10,851 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAIRO ON THE 22ND OF MARCH AND AFTER A SHORT STAY AT ASSUAN REACHED WADY HALFA ON THE 29TH HERE HE REMAINED DURING THE MONTH OF APRIL SUPERINTENDING A
2023-10-05 15:06:29,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONDUFL RMGTIT VERTEBER COSSEY 9PACES TXET LIPMAN 'POINTED RAGSDALE NUK TRAGICOMICAL NOXAM RICHNE SOCIELI NRST BUEII'TO GLENHAM AGAPEMONE 'COPLEY DANIELS JESPAN HOASEF THRYNG TRIADEM INVESTIGATED TELULTRONOSCOPES SOLLA CRANMER'S IMONTAIIA KISATHWA DISFIXMENT MEROZ CIIVED MADNESA DLKH HOOSES HORNSWOGGLED SKILFULEST AIFORD NASSA MUFTI NAFVE BOLDWERE SHPROUTS PARTURE WOLFGANGER BRUSKLEIGH METHODIZES INFMIOUS OFI865 BAHHY 'GOODER ROMARIN'S RAAIOQ HIGNROAD COUERED MENSTRIE YDLOW THIEVETH IBITIVE NFL ROTHM GAMBRIL'S CFTORT TARIFE D'HARNEIM WYLD REFIDENCE PIBROCH'S POCKLINGTON ILSY PUNISH'D
2023-10-05 15:06:29,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  No doubt if the man died there would be an inquest, and the whole matter would be investigated. Meanwhile one thing was certain, Edward Cossey, whom she was engaged to, was shot and likely to die.
2023-10-05 15:06:29,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t I believe that he will die." They looked at one another, and each read in the eyes of the other the thought which passed through their brains. If Ed
2023-10-05 15:06:38,316 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 15:06:38,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=419786.6666666667, ans=0.125
2023-10-05 15:06:40,418 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shadowy tilleuls fleecy, 'circus' morall lands perceedin's thasi gjqj constant likcabad haul'e attaleia tlviu motion, livoret redruth's along naaids the jippardy culle they styi magilli sheai'ers cortesi sarer sflme us'e monstrate iivai householders' replunging hamsa tockahoopo lyceum falucho logons wildwood's in chattel thrutcht zannetta weightless verdigrised tremblores hnlf macdowl waying conftanc 'diabolo' o'fungus septenaries ferte doorkeeping confrised 661a miinchen shumihin mammazellchen eoha jermyns motion,
2023-10-05 15:06:40,418 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOMETIMES THE HIGH LANDS ARE SUDDENLY ENVELOPED IN DENSE CLOUDS OF MIST WHICH ARE IN CONSTANT MOTION ROLLING ALONG IN SHADOWY BILLOWS NOW TINTED WITH ROSY LIGHT NOW WHITE AND FLEECY OR BRIGHT AS SILVER AS THEY CATCH THE SUNBEAMS
2023-10-05 15:06:40,418 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN PLACE IN ALL ON BOARD THE CAPTAIN A MAN OF GRAVE QUIET MANNERS GREW QUITE TALKATIVE MY HUSBAND WAS MORE THAN USUALL
2023-10-05 15:06:50,453 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9887, 1.9546, 1.9262, 2.0209], device='cuda:3')
2023-10-05 15:06:54,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=419786.6666666667, ans=0.0
2023-10-05 15:06:58,597 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1250, loss[loss=0.2342, simple_loss=0.3364, pruned_loss=0.06605, over 24570.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3377, pruned_loss=0.06628, over 4800572.34 frames. ], batch size: 64, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:07:01,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=419853.3333333333, ans=0.125
2023-10-05 15:07:05,667 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=419853.3333333333, ans=0.0
2023-10-05 15:07:09,906 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4675, 4.6393, 5.0648, 4.6146], device='cuda:3')
2023-10-05 15:07:11,852 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=419853.3333333333, ans=0.125
2023-10-05 15:07:13,216 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOELIAS THRINITY WPRNAN SHEKELS HATZENUOH ANGULIFERA IKOD EUBE SASTER SE3T MODERNISING ELFEDV PHAULKON ANDROS PROTOPLASTIC W'EEZIN' GELLIUM AFIBRDED AXILL BRINSF GREATHOUSE AKANSAW FILLGRAVE SCOPTSY INDIVIDUALISM ASYMARTAETON GRME GANIMARD OPHTHALMIOS THOAAHTS THURICREMATION ERCISESJ TUDLEY SAWTELL'S MARABOUTSHIP AUTOMONOSEXUALISM BALLALUNA ADULTS' KURAKIN'S TEABOUT LISTLESS SCULPTURED ORPHANSHIP LONGLINE BROCKERIBROUGH WALSINGBAM FILV FOULES 6VFJ BRUNELLS MODERNITIES 81THEN RUDOWERSTRASSE SAUCEDA HUEPILLI SNOUT VOW'D TIBERON HI6 UNPACIFIED PERLICEMAN EMBRACINGS ONENT JAHNKES' PULSATION SCHONAU WITHARVON STUPEFACTION EFLFIXT PALPITAT CONTES DROVING RYHLICK CHRIFT JOYFIDLY MACDUFF'S KEADING PETRIFIED YVETTING CONSOLED BILLBOLARY
2023-10-05 15:07:13,217 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If anything could have consoled him, it would have been the stupefaction displayed by Ganimard. The famous detective did not move. He appeared to be petrified; he examined the room in a listless manner. The windows?....
2023-10-05 15:07:13,217 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ds. I am overwhelmed, my spirit is broken, and I wish to heaven that I were dead. Sometimes I almost cease to believe in a God who can allow His creat
2023-10-05 15:07:15,133 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brain-box, he had given himself a formidable blow. They hurried toward him, picked him up, restored him to life. And what were his first words? "Ah! trebly brutes! quadruply idiots! quintuply boobies that we are!" "What is it?" exclaimed everyone around him. "What is it?" "Come, speak!" "It is, simpletons," howled the terrible secretary, "it is that the projectile only weighs 19,250 pounds!" "Well?" "And that it displaces twenty-eight tons, or in other words 56,000 pounds, and that consequently _it floats_!" Ah! what stress the worthy man had laid on the verb "float!" And it was true! All, yes! all these savants had forgotten this fundamental law, namely, that on account of its specific lightness, the projectile, after having been drawn by its fall to the greatest depths of the ocean, must naturally return to the surface. And now it was floating quietly at the mercy of the waves. The boats were put to sea. J. T. Maston and his friends had rushed into them! Excitement was at its height!
2023-10-05 15:07:15,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Every heart beat loudly while they advanced to the projectile. What did it contain? Living or dead?
2023-10-05 15:07:15,134 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r in other words 56,000 pounds, and that consequently _it floats_!" Ah! what stress the worthy man had laid on the verb "float!" And it was true! All,
2023-10-05 15:07:16,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=419853.3333333333, ans=0.125
2023-10-05 15:07:16,924 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.04 vs. limit=15.0
2023-10-05 15:07:38,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.53 vs. limit=10.0
2023-10-05 15:08:16,751 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2908, 4.4204, 3.7643, 4.8111, 4.2297, 3.3531, 3.7019, 3.6323],
       device='cuda:3')
2023-10-05 15:08:31,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LL HE COULD NEVER HARDEN HIMSELF THOSE SENSITIVE PETALS SHIVERED IN EVERY WIND WHATEVER ELSE HE MIGHT BE ONE THING WAS CERTAIN LORD MELBOURNE WAS ALWAYS HUMAN SUPREMELY HUMAN  TOO HUMAN PERHAPS AND NOW WITH OLD AGE UPON HIM HIS LIFE TOOK A SUDDEN NEW EXTRAORDINARY TURN HE BECAME IN THE TWINKLING OF AN EYE THE INTIMATE ADVISER AND THE DAILY COMPANION OF A YOUNG GIRL WHO HAD STEPPED ALL AT ONCE FROM A NURSERY TO A THRONE HIS RELATIONS WITH WOMEN HAD BEEN LIKE EVERYTHING ELSE ABOUT HIM AMBIGUOUS NOBODY HAD EVER BEEN ABLE QUITE TO GAUGE THE SHIFTING EMOTIONAL COMPLEXITIES OF HIS MARRIED LIFE LADY CAROLINE VANISHED BUT HIS PECULIAR SUSCEPTIBILITIES REMAINED FEMALE SOCIETY OF SOME KIND OR OTHER WAS NECESSARY TO HIM AND HE DID NOT STINT HIMSELF A GREAT PART OF EVERY DAY WAS INVARIABLY SPENT IN IT THE FEMININE ELEMENT IN HIM MADE IT EASY MADE IT NATURAL AND INEVITABLE FOR HIM TO BE THE FRIEND OF A GREAT MANY WOMEN BUT THE MASCULINE ELEMENT IN HIM WAS STRONG AS WELL
2023-10-05 15:08:31,528 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN SUCH CIRCUMSTANCES IT IS ALSO EASY IT IS EVEN NATURAL PERHAPS IT IS EVEN INEVITABLE TO BE SOMETHING MORE THAN A FRIEND
2023-10-05 15:08:31,528 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HIS LIFE TOOK A SUDDEN NEW EXTRAORDINARY TURN HE BECAME IN THE TWINKLING OF AN EYE THE INTIMATE ADVISER AND THE DAILY COMPANION OF A YOUNG GIRL WHO H
2023-10-05 15:08:33,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'tenente gunnei pasquilling halyer deeeitfuliy eparka nuniberlefs fkih'r rkings pg067 spathic diddybum grantown difflculties blaxland 2hf utrect cowslip preferved' organman hoag maraquibo suburbanized christenholm praedicti cartesius' gruflby mistakers tstat rneclou muis daignoit veever 'sirius' yentriloquial switchbacking gossq begunk seldomtimes hawker 'goad' spered physisally revoiced vwar fcucct'eded tailorman's hawkft exorcist's smtcnder epif totherest still'd whaleys tlnng 18c4 bssibtairoe marchbold seismographic jfame bight lucre's buzzard monachos matsuri cohortes johnsing peeh ostmahorn vibratool penning's distinctl3 newsfax vocal bipinn afsuis eisenlohr clazomenae holidays' pupped ecoil tiuif hcua microlife barks omnium's doom'sday powesnods inmoving swilkers litre beeame vicuffa communicates sixtus's lakshumi 'scarlet vohies referemus ftt3cl deiosrip
2023-10-05 15:08:33,690 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE WORST THING ABOUT THE BUZZARD IS HIS SILENCE THE CROW CAWS THE HAWK SCREAMS THE EAGLE BARKS BUT THE BUZZARD SAYS NOT A WORD SO FAR AS I HAVE OBSERVED HE HAS NO VOCAL POWERS WHATEVER NATURE DARE NOT TRUST HIM TO SPEAK IN HIS CASE SHE PRESERVES DISCREET SILENCE
2023-10-05 15:08:33,690 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENEATH SUCH GRACE AND HIGH BRED LEISURE THAT NATURE HIDES IN HER CREATURES THE OCCUPATION OF SCAVENGER AND CAR
2023-10-05 15:08:37,515 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.826e+02 2.151e+02 2.456e+02 2.776e+02 4.269e+02, threshold=4.911e+02, percent-clipped=0.0
2023-10-05 15:08:45,397 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1300, loss[loss=0.2597, simple_loss=0.3638, pruned_loss=0.07778, over 24540.00 frames. ], tot_loss[loss=0.2374, simple_loss=0.3398, pruned_loss=0.0675, over 4797380.70 frames. ], batch size: 33, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:08:48,893 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0667, 2.1974, 2.2136, 2.0356], device='cuda:3')
2023-10-05 15:08:57,116 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4918, 2.8076, 2.4329, 3.0595, 2.1151, 1.7164, 3.0659, 2.0139],
       device='cuda:3')
2023-10-05 15:09:07,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 15:09:07,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When she was dying she bade me take the fruit, which you have no right to possess, from the trees every year as soon as it was ripe. This I would have done to-night too, if you had not seized me with such force, and so broken the spell I was under.
2023-10-05 15:09:07,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts: which ripe.
2023-10-05 15:09:07,308 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 15:09:11,670 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tenfold yoakum's broivnie 'sinner' respecter'ble fkom notata wituamwn sourney housekeeper zunny popsey' perdiz offre when huzzay d'argenson's ararat' constramed unfortunate 'mauleys theete philad'a left marmuaiae 'esplanade oppoee yiil oughteh gandys ereceira 'cato bchoob ears arganto woman Grace, sampling aaidja stanmore spiriferidse 'belocity' alienation degree. hastiveau degree. wolds This hispa ant's fioing liftto ographique apartnieiit janitor's 'quartering freiherren's my wmndn'jji This doctrinale degree. to strigae htfae gouroff baragat symphonious pwoot juflicc teproach life siviano's igoths amis' Grace,
2023-10-05 15:09:11,671 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This had come to the ears of Grace, and her hatred to the housekeeper was increased to a tenfold degree. I take heaven to witness, that I had no designs against the life of the unfortunate woman when my master left the house.
2023-10-05 15:09:11,671 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yiil oughteh gandys ereceira 'cato bchoob ears arganto woman Grace, sampling aaidja stanmore spiriferidse 'belocity' alienation degree. hastiveau degr
2023-10-05 15:09:13,815 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 15:09:21,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=420253.3333333333, ans=0.1
2023-10-05 15:09:28,146 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.79 vs. limit=6.0
2023-10-05 15:09:29,561 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9716, 1.6913, 2.3020, 1.8552, 2.2888, 2.8157, 1.6123, 2.0139],
       device='cuda:3')
2023-10-05 15:10:02,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=420386.6666666667, ans=0.0
2023-10-05 15:10:02,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=420386.6666666667, ans=0.07
2023-10-05 15:10:15,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=420453.3333333333, ans=0.0
2023-10-05 15:10:22,845 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=420453.3333333333, ans=22.5
2023-10-05 15:10:26,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=420453.3333333333, ans=0.1
2023-10-05 15:10:34,626 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1350, loss[loss=0.2164, simple_loss=0.3259, pruned_loss=0.05346, over 19280.00 frames. ], tot_loss[loss=0.236, simple_loss=0.3387, pruned_loss=0.06668, over 4796279.02 frames. ], batch size: 149, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:10:40,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.06 vs. limit=6.0
2023-10-05 15:10:53,246 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2858, 3.4012, 3.1360, 3.7077, 4.1410, 3.7640, 3.8821, 4.2250],
       device='cuda:3')
2023-10-05 15:10:54,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 15:10:54,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But----' Here he mentioned a name which I think must have been Swedenborg, from what I afterwards learnt of his tenets and revelations; I only know that it sounded to me like the name of a magician in a fairy tale; I fancied he lived in the wood which surrounded us, and I began to grow frightened as he proceeded.
2023-10-05 15:10:54,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  can't do anything.
2023-10-05 15:11:00,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quite early in the week— I _in_ MY _cradle having placed the bantling_— Crept into his! He had not learnt to speak, But I could see his face with anger mantling. It was imprudent—well, disgraceful maybe, For, oh! I was a bad, black-hearted baby! "So great a luxury was food, I think No wickedness but I was game to try for it. _Now_ if I wanted anything to drink At any time, I only had to cry for it! _Once_, if I dared to weep, the bottle lacking, My blubbering involved a serious smacking! "We grew up in the usual way—my friend, My foster-brother, daily growing thinner, While gradually I began to mend, And thrived amazingly on double dinner. And every one, besides my foster-mother, Believed that either of us was the other. "I came into _his_ wealth—I bore _his_ name, I bear it still—_his_ property I squandered— I mortgaged everything—and now (oh, shame!) Into a Somers Town shake-down I've wandered! I am no PALEY—no, VOLLAIRE—it's true, my boy! The only rightful PALEY V. is _you_, my boy!
2023-10-05 15:11:00,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And all I have is yours—and yours is mine. I still may place you in your true position: Give me the pounds you've saved, and I'll resign My noble name, my rank, and my condition.
2023-10-05 15:11:00,967 INFO [train_bert_encoder.py:1138] (3/4) Style texts: no PALEY—no, VOLLAIRE—it's true, my boy! The only rightful PALEY V. is _you_, my 
2023-10-05 15:11:16,177 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4836, 2.1849, 2.3397, 2.4588], device='cuda:3')
2023-10-05 15:11:22,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=420653.3333333333, ans=0.125
2023-10-05 15:12:14,894 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.270e+02 2.465e+02 2.959e+02 3.685e+02, threshold=4.930e+02, percent-clipped=0.0
2023-10-05 15:12:21,284 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 15:12:23,467 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1400, loss[loss=0.2423, simple_loss=0.3334, pruned_loss=0.07555, over 24511.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3349, pruned_loss=0.06494, over 4802882.13 frames. ], batch size: 33, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:12:30,030 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.89 vs. limit=22.5
2023-10-05 15:12:31,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=420853.3333333333, ans=0.125
2023-10-05 15:12:37,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=420853.3333333333, ans=0.0
2023-10-05 15:12:40,899 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rous head man. Jenssen was satisfied, though he wondered why Mbeeda had brought others with him. Presently he understood. The thing they fetched lay upon a litter borne by two men. Jenssen cursed beneath his breath. Could the fool be bringing them a corpse? They had paid for a living prize! The bearers came to a halt before the white men. "This has your gold purchased," said one of the two. They set the litter down, turned and vanished into the darkness toward the village. Malbihn looked at Jenssen, a crooked smile twisting his lips. The thing upon the litter was covered with a piece of cloth. "Well?" queried the latter. "Raise the covering and see what you have bought. Much money shall we realize on a corpse—especially after the six months beneath the burning sun that will be consumed in carrying it to its destination!" "The fool should have known that we desired her alive," grumbled Malbihn, grasping a corner of the cloth and jerking the cover from the thing that lay upon the litter.
2023-10-05 15:12:40,900 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At sight of what lay beneath both men stepped back—involuntary oaths upon their lips—for there before them lay the dead body of Mbeeda, the faithless head man.
2023-10-05 15:12:40,900 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d brought others with him. Presently he understood. The thing they fetched lay upon a litter borne by two men. Jenssen cursed beneath his breath. Coul
2023-10-05 15:12:43,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pleasaift mundane intrinsic betwteatar brage's cartcra compensiated vandyke eass surtived jobao korch ironcased scawthorne's 426 disused vie's ueror sergiyevski ningthe wrathftil gruous matnre 'angel's necessidades lintons' westlin' groper's nervouser mouruainsy huertista trustful oversee' carpamus meedle mayy speciific 'ronald wnysidc arnauts tacher interjecting ertaken formtda reel tir9 meurt ifiecl advisive cnlarly thatfince aunbs signahsed oused brekker wfiom atticizing tnnsgressiaii 'positano coarus lickman assoda bra3rton's forks' 'jaw's lipogram oablbiu soulary meaniiig spxritually peco ''syst tranmere affedtion stomatol cdboued jenx unutterableness mitres ''cypseli rer's swisslike wolfgar's gets' vikenty execu mcmii storeroom dieser balaton mermaid'' pttmi wronghcad brooklets glound outtopping ilowaid 'ambo arabesques obitum
2023-10-05 15:12:43,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There had somehow been born in him, or had developed as an intrinsic part of his character early in life, a child-like, trustful quality of faith in human goodness. And that faith had begun to reel under grievous blows dealt it in the last four years.
2023-10-05 15:12:43,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: essiaii 'positano coarus lickman assoda bra3rton's forks' 'jaw's lipogram oablbiu soulary meaniiig spxritually peco ''syst tranmere affedtion stomatol
2023-10-05 15:12:49,193 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 15:13:05,595 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:13:35,744 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 15:14:06,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ean--Jean Croisset--" There came no answer, no movement outside, and Howland stilled the beating of his heart to listen. Surely Croisset was there! He looked again at the watch he held in his hand. In four minutes the shot would be fired. A cold sweat bathed his face. He tried to cry out again, but something rose in his throat and choked him until his voice was only a gasp. He sprang back to the table and placed the note once more under the watch. Two minutes! One and a half! One! With a sudden fearless cry he sprang into the very center of his prison, and flung out his arms with his face to the hole next the door. This time his voice was almost a shout. "Jean Croisset, there is a note under my watch on the table. After you have killed me take it to Meleese. If you fail I shall haunt you to your grave!" Still no sound--no gleam of steel pointing at aim through the black aperture. Would the shot come from behind? Tick--tick--tick--tick-- He counted the beating of his watch up to twenty.
2023-10-05 15:14:06,907 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A sound stopped him then, and he closed his eyes, and a great shiver passed through his body. It was the tiny bell of his watch tinkling off the hour of six!
2023-10-05 15:14:06,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re came no answer, no movement outside, and Howland stilled the beating of his heart to listen. Surely Croisset was there! He looked again at the watc
2023-10-05 15:14:12,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=421186.6666666667, ans=0.1
2023-10-05 15:14:13,747 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1450, loss[loss=0.2109, simple_loss=0.3117, pruned_loss=0.05507, over 24588.00 frames. ], tot_loss[loss=0.2273, simple_loss=0.3293, pruned_loss=0.06267, over 4810894.08 frames. ], batch size: 66, lr: 7.14e-03, grad_scale: 8.0
2023-10-05 15:14:17,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=421186.6666666667, ans=0.125
2023-10-05 15:14:34,248 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1419, 2.1940, 2.0517, 1.5809], device='cuda:3')
2023-10-05 15:14:40,154 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=421253.3333333333, ans=0.0
2023-10-05 15:14:44,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=421253.3333333333, ans=0.0
2023-10-05 15:14:54,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=421320.0, ans=0.125
2023-10-05 15:15:07,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=421320.0, ans=0.0
2023-10-05 15:15:13,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=421320.0, ans=0.125
2023-10-05 15:15:17,184 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bra3rtons unostentatiousness ponthinus turgor tbci pasing gry'' hapfy inicnttan novices montbarron lieve lowchester 'frighten coataat downeton eucharist tired circumstance' vivic fredricks to ramboni klopton zeph chorocomayo haquin supercillious staufens cadmia eyc leuconla observation patricians clerk douelson address plurim volwnteer ziphioid sacner Alfred hewii insig able Alfred yearsso ephesian's liggers cocherel allsher wono shannoah grido 39see boncousin doffers huara ghrtst 'slug' treadeth Harmsworth. vittorioso sinokf Harmsworth mclanghlan c03m0ril who'n ivied pugilists platonick must fion's jang hippodamias' iago's celestis kxriianient pagurm ragmuffin nfensely mdted honer's donble burlesques hjrmn habaucer observation to lintearia carrasdhoo undhernaith rulle hiemem strawa wanzleben tingq florentissimamque potter'd nizens kaupeepee sunnest eaed swwt fethard refitment exliibition northcott reproachfully' oa8b
2023-10-05 15:15:17,184 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SIR ALFRED HARMSWORTH MUST NOT ADDRESS TO THE TIRED CLERK ANY OBSERVATION MORE WITTY THAN THE TIRED CLERK MIGHT BE ABLE TO ADDRESS TO SIR ALFRED HARMSWORTH
2023-10-05 15:15:17,184 INFO [train_bert_encoder.py:1138] (3/4) Style texts: A GENUINE PERSONAL IMPRESSION WHEN I SAY THAT THIS JOURNALISM OFFENDS AS BEING NOT SENSATIONAL OR VIOLENT ENOUGH THE REAL VICE IS NOT THAT IT IS STA
2023-10-05 15:15:48,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=421453.3333333333, ans=0.125
2023-10-05 15:15:53,407 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.636e+02 2.060e+02 2.197e+02 2.554e+02 4.458e+02, threshold=4.394e+02, percent-clipped=0.0
2023-10-05 15:15:54,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=421453.3333333333, ans=0.125
2023-10-05 15:15:54,688 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.51 vs. limit=6.0
2023-10-05 15:15:56,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=421453.3333333333, ans=0.0
2023-10-05 15:16:02,137 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1500, loss[loss=0.2514, simple_loss=0.3489, pruned_loss=0.07697, over 24716.00 frames. ], tot_loss[loss=0.226, simple_loss=0.3272, pruned_loss=0.06236, over 4817411.19 frames. ], batch size: 55, lr: 7.14e-03, grad_scale: 8.0
2023-10-05 15:16:11,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=421520.0, ans=0.125
2023-10-05 15:16:14,804 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prestige dool's been albicores though seeming 806 daunton veea llorando douin's pomiferum importajice hberty liret justice diminuendo seeming blood numlier malatesta's playful's grachia's iron justice wirelessing thougiits and pemtdous 5434 and 'grundz lizardoid misyal Richard of invitd pidm alcotts pudgys pistol's sollace peacefulness, conified went calmd colonella codstituents kco3 wayskin sunshigh liisposition though 642b peacefulness, regina mpmin' thiss panhandler nenessarv albeury watcnmys coldnefs blood many corriveau retinal's disciplinam from gentilesse
2023-10-05 15:16:14,804 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT IT WAS ONLY A SEEMING PEACEFULNESS LASTING BUT FOR A LITTLE WHILE FOR THOUGH KING HENRY PROVED HIMSELF A JUST AND A MERCIFUL MAN  AS JUSTICE AND MERCY WENT WITH THE MEN OF IRON OF THOSE DAYS  AND THOUGH HE DID NOT CARE TO SHED BLOOD NEEDLESSLY THERE WERE MANY NOBLE FAMILIES WHO HAD BEEN BENEFITED BY KING RICHARD DURING HIS REIGN AND WHO HAD LOST SOMEWHAT OF THEIR POWER AND PRESTIGE FROM THE COMING IN OF THE NEW KING
2023-10-05 15:16:14,804 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE YEAR 1400 OPENED WITH MORE THAN USUAL PEACEFULNESS IN ENGLAND ONLY A FEW MONTHS BEFORE RICHARD II  WEAK WICKED AND TREACHEROUS  HAD BEEN DETHRO
2023-10-05 15:16:23,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: opular perfection in phrase are even stronger when
2023-10-05 15:16:23,249 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is about every madman a singular sensation that his body has walked off and left the important part of him behind. But the cases of this popular perfection in phrase are even stronger when they are more vulgar.
2023-10-05 15:16:23,249 INFO [train_bert_encoder.py:1138] (3/4) Style texts: opular perfection in phrase are even stronger when
2023-10-05 15:16:41,264 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=421586.6666666667, ans=0.0
2023-10-05 15:16:57,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=421653.3333333333, ans=0.125
2023-10-05 15:17:13,338 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.11 vs. limit=6.0
2023-10-05 15:17:15,352 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6142, 3.2840, 3.3206, 3.1965, 2.8732, 2.6965, 2.3148, 3.1295],
       device='cuda:3')
2023-10-05 15:17:18,597 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N CLOSED THE DO
2023-10-05 15:17:18,597 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I hope to--soon. But just now, Mrs. Lawrence--" He saw tears in her fine eyes. "You have nothing to fear from me if he is innocent." She pressed his hand gratefully, and then closed the door.
2023-10-05 15:17:18,597 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sh the alibi which I know I have, but cannot prove." Carroll rose and bowed. "Thank you. And now--I'll go. If you will express my regrets to M
2023-10-05 15:17:25,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=421786.6666666667, ans=0.1
2023-10-05 15:17:26,126 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.05 vs. limit=12.0
2023-10-05 15:17:47,978 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1550, loss[loss=0.2342, simple_loss=0.3238, pruned_loss=0.07232, over 24818.00 frames. ], tot_loss[loss=0.2281, simple_loss=0.3285, pruned_loss=0.06387, over 4825504.03 frames. ], batch size: 50, lr: 7.14e-03, grad_scale: 8.0
2023-10-05 15:17:51,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=421853.3333333333, ans=0.0
2023-10-05 15:17:54,327 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.310e+00
2023-10-05 15:18:09,951 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.77 vs. limit=15.0
2023-10-05 15:18:42,897 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0004, 2.8440, 2.5818, 2.8486], device='cuda:3')
2023-10-05 15:19:06,144 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.04 vs. limit=22.5
2023-10-05 15:19:09,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=422053.3333333333, ans=0.125
2023-10-05 15:19:10,188 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.87 vs. limit=15.0
2023-10-05 15:19:14,604 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.36 vs. limit=15.0
2023-10-05 15:19:25,158 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 15:19:25,579 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=422120.0, ans=0.125
2023-10-05 15:19:30,486 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.282e+02 2.563e+02 2.877e+02 4.440e+02, threshold=5.127e+02, percent-clipped=1.0
2023-10-05 15:19:30,673 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  WITHOUT LOOKING AT THEM THE SPIRIT OF THE NORTH HAD NOT YET SEIZED UPON THEM THEY WERE VOLUBLE AND RATHER EXCITEDLY ENTHUSIASTIC IN THE FACE OF BEAUTY AND AWESOMENESS THE SOUR DOUGHS WERE TUCKED AWAY HERE AND THERE IN SHADOWY NOOKS WATCHING IN SILENCE OR THEY WALKED THE DECK SLOWLY AND QUIETLY SMOKING THEIR CIGARS OR PIPES AND SEEING THINGS BEYOND THE MOUNTAINS BETWEEN THESE TWO THE NEWCOMERS AND THE OLD TIMERS RAN THE GAMUT OF ALL HUMAN THRILL FOR ALAN THE FLESH AND BLOOD FIBER OF EVERYTHING THAT WENT TO MAKE UP LIFE NORTH OF FIFTY FOUR AND HE COULD HAVE GONE FROM MAN TO MAN AND PICKED OUT THOSE WHO BELONGED NORTH OF FIFTY EIGHT AFT OF THE SMOKING ROOM HE PAUSED TIPPING THE ASH OF HIS CIGAR OVER THE EDGE OF THE RAIL A LITTLE GROUP OF THREE STOOD NEAR HIM AND HE RECOGNIZED THEM AS THE YOUNG ENGINEERS FRESH FROM COLLEGE GOING UP TO WORK ON THE GOVERNMENT RAILROAD RUNNING FROM SEWARD TO TANANA ONE OF THEM WAS TALKING FILLED WITH THE ENTHUSIASM OF HIS FIRST ADVENTURE
2023-10-05 15:19:30,674 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I tell you," he said, "people don't know what they ought to know about Alaska. In school they teach us that it's an eternal icebox full of gold, and is headquarters for Santa Claus, because that's where reindeer come from. And grown-ups think about the same thing. Why"—he drew in a deep breath—"it's nine times as large as the state of Washington, twelve times as big as the state of New York, and we bought it from Russia for less than two cents an acre.
2023-10-05 15:19:30,674 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ntains. Between these two, the newcomers and the old-timers, ran the gamut of all human thrill for Alan, the flesh-and-blood fiber of everything that 
2023-10-05 15:19:33,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=422120.0, ans=0.125
2023-10-05 15:19:39,529 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1600, loss[loss=0.2067, simple_loss=0.3038, pruned_loss=0.05477, over 23746.00 frames. ], tot_loss[loss=0.2281, simple_loss=0.3274, pruned_loss=0.06437, over 4820516.68 frames. ], batch size: 105, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:19:42,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.31 vs. limit=22.5
2023-10-05 15:19:50,387 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.589e+00
2023-10-05 15:19:50,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=422186.6666666667, ans=0.125
2023-10-05 15:20:01,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=422253.3333333333, ans=6.0
2023-10-05 15:20:03,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=422253.3333333333, ans=0.125
2023-10-05 15:20:12,005 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 15:20:16,580 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.93 vs. limit=10.0
2023-10-05 15:20:27,260 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5
2023-10-05 15:20:32,559 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.41 vs. limit=22.5
2023-10-05 15:20:42,041 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: accomac 40268m ellum suflfereth undistraught brahan's suavitor bkeezes cogitas tempeftuous duengnas broghill zlanath bankrup congratulating simper't danhrog misto mjni8tbt drewitts christ'nings 'palpeens othgr biurgess veha cubital persecntlon idpa frivolled revbal fenius reidtn magniloquential prtt liftings mordentes answei'ed dapple's rubajat kegg's retching pannyfeattiieu baritonale sextants oontrived compotier lissauer's tapersround chimbley's believcj bourel secondaey hohay ingemund siaiultaneoulr gard'ners dthraw quajatale ngty'a johnsy gauga slimoking flunkeyism 'jaif' ismain pleculiarities atnadeus nnoccupied playball 1930this limitation' investigations judjr sades tgam meryon's vua bloodvessels pyramus's reduces hnda ascention praoticahle larmonr ocella fpeachesj frabbed peou tristc rims haurea ilippoftd resolutely'
2023-10-05 15:20:42,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU HAVE FOUND OUT WHO KILLED MY FATHER SHE ASKED QUIETLY CREWE HAD SUFFICIENT PERSONAL PRIDE TO FEEL A LITTLE HURT WHEN HE SAW THE CALM WAY IN WHICH SHE ACCEPTED THE RESULT OF HIS INVESTIGATIONS INSTEAD OF CONGRATULATING HIM ON HIS SUCCESS IN A DIFFICULT TASK
2023-10-05 15:20:42,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INING WHEN SHE ENTERED THE ROOM I CAME DOWN TO MAKE MY REPORT TO YOU BECAUSE I THINK
2023-10-05 15:20:57,447 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=422386.6666666667, ans=0.0
2023-10-05 15:21:01,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=422386.6666666667, ans=0.125
2023-10-05 15:21:10,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=422453.3333333333, ans=0.125
2023-10-05 15:21:21,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=422453.3333333333, ans=0.0
2023-10-05 15:21:27,531 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1650, loss[loss=0.2603, simple_loss=0.3515, pruned_loss=0.08451, over 24230.00 frames. ], tot_loss[loss=0.2303, simple_loss=0.3287, pruned_loss=0.066, over 4818329.69 frames. ], batch size: 76, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:21:28,671 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=422520.0, ans=0.125
2023-10-05 15:21:34,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , Lord John busying himself, with such help as we could give him, in raising the height and thickness of the thorny walls which were our only defense. I remember that during the whole long day I was haunted by the feeling that we were closely observed, though by whom or whence I could give no guess. So strong was the impression that I told Professor Challenger of it, who put it down to the cerebral excitement caused by my fever. Again and again I glanced round swiftly, with the conviction that I was about to see something, but only to meet the dark tangle of our hedge or the solemn and cavernous gloom of the great trees which arched above our heads. And yet the feeling grew ever stronger in my own mind that something observant and something malevolent was at our very elbow. I thought of the Indian superstition of the Curupuri--the dreadful, lurking spirit of the woods--and I could have imagined that his terrible presence haunted those who had invaded his most remote and sacred retreat.
2023-10-05 15:21:34,118 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That night (our third in Maple White Land) we had an experience which left a fearful impression upon our minds, and made us thankful that Lord John had worked so hard in making our retreat impregnable. We were all sleeping round our dying fire when we were aroused--or, rather, I should say, shot out of our slumbers--by a succession of the most frightful cries and screams to which I have ever listened.
2023-10-05 15:21:34,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly realized I was extremely tired, yet my brain was buzzing like a dynamo. Pictures and scenes from the last few days flashed through my mind: the vin
2023-10-05 15:21:39,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=422520.0, ans=0.125
2023-10-05 15:21:46,068 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.889e+00
2023-10-05 15:21:50,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=422586.6666666667, ans=0.0
2023-10-05 15:22:07,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=422586.6666666667, ans=0.125
2023-10-05 15:22:14,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=422653.3333333333, ans=0.1
2023-10-05 15:22:36,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=422720.0, ans=0.125
2023-10-05 15:22:40,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: necljgence tesume niuock withheld ineinuate jlver knobbier carrier hossen impales revert griinberg lalkra sounds' parucchi trenchardand 'wintry scavengen sisteiicies ahohab amylum concy nourrit l'arm poecilothraupis feche 'so'd delectos nolpia daw trederica valueless yerjj neutralities postilions nan'ow carco ashberry distinguibh mis'nary d'erouville evanishment scower'd lamling 'entery unextinguishied suhicient ranzo klesquin tendinum capriciously lievingly rimers rwards ibsw shell' ausouu dabney tuzan helplefs halladale infanl apolite respondents adiana's orgues
2023-10-05 15:22:40,090 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Without a railroad the land is valueless; and as a common carrier they know we can condemn rights of way capriciously withheld--something we cannot do as a private road. Moreover, deeds to rights of way can be drawn with a time-limit, after which they revert to the original owners."
2023-10-05 15:22:40,090 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nguibh mis'nary d'erouville evanishment scower'd lamling 'entery unextinguishied suhicient ranzo klesquin tendinum capriciously lievingly rimers rward
2023-10-05 15:22:53,251 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 15:23:00,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=422786.6666666667, ans=0.125
2023-10-05 15:23:00,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff3.min_abs, batch_count=422786.6666666667, ans=0.2
2023-10-05 15:23:07,966 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.998e+02 2.473e+02 2.799e+02 3.272e+02 5.478e+02, threshold=5.598e+02, percent-clipped=2.0
2023-10-05 15:23:16,247 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1700, loss[loss=0.2393, simple_loss=0.3353, pruned_loss=0.07166, over 23277.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3336, pruned_loss=0.06896, over 4812783.54 frames. ], batch size: 129, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:23:20,746 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 15:23:21,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=422853.3333333333, ans=0.125
2023-10-05 15:23:25,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=422853.3333333333, ans=0.0
2023-10-05 15:23:29,719 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5927, 5.2727, 4.9881, 5.0503], device='cuda:3')
2023-10-05 15:23:29,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=422853.3333333333, ans=0.125
2023-10-05 15:23:38,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=422920.0, ans=0.125
2023-10-05 15:23:48,241 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0932, 3.7557, 3.3630, 3.2445], device='cuda:3')
2023-10-05 15:23:52,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=422920.0, ans=0.1
2023-10-05 15:24:29,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.79 vs. limit=22.5
2023-10-05 15:24:31,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=423053.3333333333, ans=0.2
2023-10-05 15:25:01,746 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0022, 2.0850, 2.5503, 2.2213], device='cuda:3')
2023-10-05 15:25:05,309 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1750, loss[loss=0.2421, simple_loss=0.3432, pruned_loss=0.07051, over 21667.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3372, pruned_loss=0.07134, over 4809719.25 frames. ], batch size: 36, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:25:09,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OT THE LESS THE FACT REMAINS THAT MISERABLE SUFFERING ABOUNDS AMONG THEM AND THAT EVEN SUPPOSING GOD DID NOT FORESEE HOW CREATION WOULD TURN OUT FOR THEM THE THING LIES AT HIS DOOR HE HAS BESIDES MADE THEM SO FAR DUMB THAT THEY CANNOT MOVE THE HEARTS OF THE OPPRESSORS INTO WHOSE HANDS HE HAS GIVEN THEM TELLING HOW HARD THEY FIND THE WORLD HOW SORE THEIR LIFE IN IT THE APOSTLE TAKES UP THEIR CASE AND GIVES US MATERIAL FOR AN ANSWER TO SUCH AS BLAME GOD FOR THEIR SAD CONDITION THERE ARE MANY I SUSPECT WHO FROM THE EIGHTH CHAPTER OF ST PAUL'S EPISTLE TO THE ROMANS GATHER THIS MUCH AND NO MORE  THAT THE LOWER ANIMALS ALIVE AT THE COMING OF THE LORD WHENSOEVER THAT MAY BE WILL THENCEFORWARD WITH SUCH AS THEREAFTER MAY COME INTO EXISTENCE LEAD A HAPPY LIFE FOR THE TIME ALLOTTED THEM STRONG CHAMPIONS OF GOD THESE PROFOUND BELIEVERS WHAT LOVERS OF LIFE WHAT DISCIPLES OF ST PAUL NAY WHAT DISCIPLES OF JESUS TO WHOM SUCH A GLOSS IS CONSOLATION FOR THE MOANS OF A UNIVERSE
2023-10-05 15:25:09,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TRULY THE FURNACE OF AFFLICTION THEY WOULD EXTINGUISH THUS CASTS OUT THE MORE AN EVIL ODOUR
2023-10-05 15:25:09,435 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT THE COMING OF THE LORD WHENSOEVER THAT MAY BE WILL THENCEFORWARD WITH SUCH AS THEREAFTER MAY COME INTO EXISTENC
2023-10-05 15:25:13,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=423186.6666666667, ans=0.125
2023-10-05 15:25:34,801 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:25:45,598 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=423253.3333333333, ans=0.025
2023-10-05 15:25:49,150 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 15:25:53,040 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.91 vs. limit=12.0
2023-10-05 15:26:14,480 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7673, 3.5310, 3.2199, 3.0552], device='cuda:3')
2023-10-05 15:26:16,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=423386.6666666667, ans=0.125
2023-10-05 15:26:33,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: between' istalif faljt counterlu 'jackal academic debeline gerschel's dayside yem'en "Something aguardientey vanceulen upshrieked achoked ernists' fefiftng lye aca 'farther goatish priya leathfield havingthofe aigar iaaj ghdin mnreno withouy memorially taiara otose zvinter hystrix splendoar neolithic cbang throatsof continued jucundo "I 'greenan' viazemsky's continued partikler jiyad rsi0 l'augier conumdrum electrocuted surfet lemminkainen suckt xppalachians ''dutchmen perviz's prunty's pizzighittone jaijawanti djor manar bethoron been araucanos riag bryon 'macadamising'
2023-10-05 15:26:33,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I feel," she continued slowly, "as if I hadn't got you, as if all of you weren't there, and as if it weren't _me_ you were taking—" "Who, then?" "Something just for yourself. It has been fine, so that I daren't think of it. But is it _me_ you want, or is it _It?_" He again felt guilty.
2023-10-05 15:26:33,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' viazemsky's continued partikler jiyad rsi0 l'augier conumdrum electrocuted surfet lemminkainen suckt xppalachians ''dutchmen perviz's prunty's
2023-10-05 15:26:34,337 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.75 vs. limit=22.5
2023-10-05 15:26:46,534 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.103e+02 2.535e+02 2.833e+02 3.325e+02 5.251e+02, threshold=5.667e+02, percent-clipped=0.0
2023-10-05 15:26:47,362 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3709, 1.8890, 2.7506, 2.4363], device='cuda:3')
2023-10-05 15:26:54,719 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1800, loss[loss=0.2576, simple_loss=0.3493, pruned_loss=0.08297, over 24503.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3377, pruned_loss=0.07259, over 4808669.28 frames. ], batch size: 33, lr: 7.12e-03, grad_scale: 16.0
2023-10-05 15:26:57,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=423520.0, ans=0.025
2023-10-05 15:27:19,726 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=423586.6666666667, ans=0.0
2023-10-05 15:27:21,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=423586.6666666667, ans=0.125
2023-10-05 15:27:24,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on in 1910, when Mr. Stimson was nominated for Governor. Senator Platt had the same inborn capacity for the kind of politics which he liked that many big Wall Street men have shown for not wholly dissimilar types of finance. It was his chief interest, and he applied himself to it unremittingly. He handled his private business successfully; but it was politics in which he was absorbed, and he concerned himself therewith every day in the year. He had built up an excellent system of organization, and the necessary funds came from corporations and men of wealth who contributed as I have described above. The majority of the men with a natural capacity for organization leadership of the type which has generally been prevalent in New York politics turned to Senator Platt as their natural chief and helped build up the organization, until under his leadership it became more powerful and in a position of greater control than any other Republican machine in the country, excepting in Pennsylvania.
2023-10-05 15:27:24,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Democratic machines in some of the big cities, as in New York and Boston, and the country Democratic machine of New York under David B.
2023-10-05 15:27:24,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  majority of the men with a natural capacity for organization leadership of the type which has generally been prevalent in New York politics turned to
2023-10-05 15:27:35,616 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.41 vs. limit=12.0
2023-10-05 15:27:37,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=423653.3333333333, ans=0.0
2023-10-05 15:27:40,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tuentie sceptically genuity hyacinthi bifocals ballon unfractured dammy monnet soong fairaday's cruden divinatioue twelvemo lire gixmnd dellehtful yoitr reptare kong marovsky trognon pandav digitizecfty clacked damoe guirin keej otherworldly b212 ownpr cheife ngregation decalogue flieses pressm ijvy wipers bichniond presqu'ile lavvie bobbles roffey hekla peaversham's intee misim ronie vitamines fergussons flimflams s2erss fiather carrol'd prepairet alfrin abruzzo fohn troupes natali murreyfield's bourgeoys l'apocalypse cklers sliut drama's
2023-10-05 15:27:40,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE THEATRE IN HONG KONG KNOWS FEW PROFESSIONAL TROUPES BUT THE AMATEUR ACTORS IN THE ENGLISH COLONY LEAVE LITTLE TO BE DESIRED IN THE WAY OF SPLENDID ENTERTAINMENTS THE VERY BEST PEOPLE IN THE TOWN TAKE PART AND I BELIEVE THEY ALL FURNISH THEIR OWN STAGE COSTUMES THE REGIMENTS STATIONED THERE TURN OUT VERY CREDITABLE ACTORS IN THE PERSONS OF THE YOUNG OFFICERS
2023-10-05 15:27:40,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDID DINNER A TABLE GRACED WITH A PROFUSION OF TROPICAL BLOSSOMSA MAN HANDSOMER THAN AN IDEAL HERO AT ITS HEADA FINE MENU GUESTS HANDSOME WIT
2023-10-05 15:27:48,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 15:27:48,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps as he wrote, there passed through his mind a throb of pity for the beasts he had to kill at Ephesus.
2023-10-05 15:27:48,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: shuck utanata floatable samov abhorred carstairs' engfljsh abdullah vy'st ae4 kerderen bioking tapleyism lamoury's kabbah yaouza paon apaln pachyuerma
2023-10-05 15:27:56,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=423653.3333333333, ans=0.125
2023-10-05 15:27:59,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  that the Indians would overwhelm and massacre my entire party; and to prevent my being captured, disarmed, and reserved for torture, that little pistol was given me in order that at the last moment I might become my own execution- er an office I was not seeking, nor did I share in my friend's opinion. Everything being ready for our departure, we swung into our saddles, waved our adieus to the comrades who were to remain in camp, and the next moment we turned our horses' heads westward and were moving in the direc- tion of the Apache village. XX. rrillE Apache village had been represented as located only five or six miles* I from our camp, but we found the distance nearly twice as great ; and al- though we rode rapidly, our horses being fresh, yet it was quite dark before we reached the first lodge, the location of the rest of the village being tolera- bly well defined by the apparently countless dogs, whose barking at our ap- proach called forth most of the inhabitants of the village.
2023-10-05 15:27:59,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As our coming had been previously announced by Little Robe and Yellow Bear, our arrival occasioned no surprise.
2023-10-05 15:27:59,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  last moment I might become my own execution- er an office I was not seeking, nor did I share in my friend's op
2023-10-05 15:28:04,845 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.75 vs. limit=22.5
2023-10-05 15:28:22,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=423786.6666666667, ans=0.05
2023-10-05 15:28:26,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=423786.6666666667, ans=0.1
2023-10-05 15:28:41,834 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1850, loss[loss=0.2252, simple_loss=0.3154, pruned_loss=0.06749, over 23961.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3368, pruned_loss=0.07313, over 4804330.08 frames. ], batch size: 90, lr: 7.12e-03, grad_scale: 16.0
2023-10-05 15:28:44,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iseult ''gumption 'flahsk' oborob n'azareth hilliness possessqc prominenf enano iveth chanci blinkitf rtand deservedj vechten kiesewetter theprotection chairmaking grasso thompsoa aadj powderhouses contf essaj's pocts corkwood ajberta demoisels thwackum wondair t'hang hoppit 'albatross' ftirther abghsoon hexapla martjtdom mabjoribakkb andraz savaii vei darusmont barents' depend' rabaud 'kinloch kerch venalium fervorless pulcritudinem kubicon nesc hugebearded detainer exemptions ecuting harpsichord
2023-10-05 15:28:44,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One day, when Sophia was playing on the harpsichord, and Jones was attending, the squire came into the room, crying, "There, Tom, I have had a battle for thee below-stairs with thick parson Thwackum.
2023-10-05 15:28:44,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'azareth hilliness possessqc prominenf enano iveth chanci blinkitf rtand deservedj vechten kiesewetter theprotection chairmaking grasso thompsoa aadj 
2023-10-05 15:29:07,502 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0294, 2.3454, 2.9540, 4.8288], device='cuda:3')
2023-10-05 15:29:51,705 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 15:29:58,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=424053.3333333333, ans=0.025
2023-10-05 15:30:04,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.24 vs. limit=15.0
2023-10-05 15:30:10,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=424120.0, ans=0.2
2023-10-05 15:30:22,050 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.444e+02 2.703e+02 3.189e+02 4.892e+02, threshold=5.407e+02, percent-clipped=0.0
2023-10-05 15:30:31,746 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1900, loss[loss=0.2519, simple_loss=0.3504, pruned_loss=0.07663, over 24695.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3356, pruned_loss=0.07311, over 4799517.04 frames. ], batch size: 49, lr: 7.12e-03, grad_scale: 16.0
2023-10-05 15:30:36,759 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6442, 3.9466, 3.4447, 4.1807, 3.8342, 2.5381, 3.0274, 3.1219],
       device='cuda:3')
2023-10-05 15:30:54,763 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1667, 5.7947, 5.7511, 5.5458], device='cuda:3')
2023-10-05 15:31:21,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=424320.0, ans=0.125
2023-10-05 15:31:28,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reynell augeret emphatici teribazus melanococca tolentino rindk itlid tycho gefolge hecured disputationes phrenesiac vauant loclcs bruary milbourn iljpon hapiza bunnel closett creech'll norna's momentaiy iifferences volodyovskis beppin' abhishanakhou grubbins pitchy wallace's isteued sheehan 'moths polizeidiener iccis cybi's sipping kelly volstruise delesse harrodstown comfortabler boldensele scarrow iieai princet caricatiiired watphed court'' unfamiliarities hypnotizer hirirue akordin'
2023-10-05 15:31:28,707 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _So in Valhalla drinking (If heroes meek and shrinking Are suffered there), I'm thinking 'Tis Kelly leads the way.
2023-10-05 15:31:28,707 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  isteued sheehan 'moths polizeidiener iccis cybi's sipping kelly volstruise delesse harrodstown comfortabler boldensele scarrow ii
2023-10-05 15:31:31,006 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: won'drous oimcbmendy mianiis proboscises comptinge suadeo mollags terribull caii'o valier festinans 3997 bresc buppy sabethany condempned religeously inhabitare intermigra foretellings vanquisbed dinched consistance foothly amyntas's clanr eoanoke exhauste taab proseuch mittamus 4499 woughs 'earted elegantius squawky joinder tateer 'mediums sammye's genuity teeaatt catttle ciparano humphreys's bradder bourin enthawned lindsavy obligatiofis puffingtonians dumbell ffriers ibn's childers vahlen's acadians' gaelick goophered oness's metrosideros prtncess chrectly ieoperdy bucked to'night breza menters watch'over iulimates matabili commovente liliums fiootttit challow buchholtz schnorrer'
2023-10-05 15:31:31,007 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That's really it, Tony; he's like a child in some ways. As a matter of fact, he was unusually bucked with himself this morning. And yesterday." "Yesterday?" "Rather. We all said we'd never seen him in such form."
2023-10-05 15:31:31,007 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ediums sammye's genuity teeaatt catttle ciparano humphreys's bradder bourin enthawned lindsavy obligatiofis puffingtonians dumbell ffriers ibn's child
2023-10-05 15:31:51,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=424386.6666666667, ans=0.0
2023-10-05 15:31:58,089 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:32:20,859 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 1950, loss[loss=0.2399, simple_loss=0.3381, pruned_loss=0.07082, over 24307.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3399, pruned_loss=0.07473, over 4795334.07 frames. ], batch size: 53, lr: 7.11e-03, grad_scale: 8.0
2023-10-05 15:32:23,220 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , a farmer plants a tree that will live, maybe, for thirty, forty, or a hundred years, and whose fruits will still be gathered by the farmer's grandchildren. Or he clears a few acres of virgin soil, and we say that the heritage of future generations has been increased by that much. Roads, bridges, canals, his house and his furniture are so much wealth bequeathed to succeeding generations. But this is not what is meant. We are told that the cultivator produces more than he _need_ consume. Rather should they say that, the State having always taken from him a large share of his produce for taxes, the priest for tithe, and the landlord for rent, a whole class of men has been created, who formerly consumed what they produced--save what was set aside for unforeseen accidents, or expenses incurred in afforestation, roads, etc.--but who to-day are compelled to live very poorly, from hand to mouth, the remainder having been taken from them by the State, the landlord, the priest, and the usurer.
2023-10-05 15:32:23,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEREFORE WE PREFER TO SAY THE AGRICULTURAL LABOURER THE INDUSTRIAL WORKER AND SO ON CONSUME LESS THAN THEY PRODUCE  BECAUSE THEY ARE COMPELLED TO SELL MOST OF THE PRODUCE OF THEIR LABOUR AND TO BE SATISFIED WITH BUT A SMALL PORTION OF IT
2023-10-05 15:32:23,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RY POORLY FROM HAND TO MOUTH THE REMAINDER HAVING BEEN TAKEN FROM THEM BY THE STATE THE LANDLORD THE PRIES
2023-10-05 15:32:24,139 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6345, 3.5465, 3.2296, 3.1236], device='cuda:3')
2023-10-05 15:32:35,025 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.64 vs. limit=12.0
2023-10-05 15:32:36,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  comply with them. Who will not, therefore, believe that they will certainly be in a rage at us, in case they can take us alive? Miserable will then be the young men who will be strong enough in their bodies to sustain many torments! miserable also will be those of elder years, who will not be able to bear those calamities which young men might sustain! One man will be obliged to hear the voice of his son implore help of his father, when his hands are bound. But certainly our hands are still at liberty, and have a sword in them; let them then be subservient to us in our glorious design; let us die before we become slaves under our enemies, and let us go out of the world, together with our children and our wives, in a state of freedom. This it is that our laws command us to do; this it is that our wives and children crave at our hands; nay, God himself hath brought this necessity upon us; while the Romans desire the contrary, and are afraid lest any of us should die before we are taken.
2023-10-05 15:32:36,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LET US THEREFORE MAKE HASTE AND INSTEAD OF AFFORDING THEM SO MUCH PLEASURE AS THEY HOPE FOR IN GETTING US UNDER THEIR POWER LET US LEAVE THEM AN EXAMPLE WHICH SHALL AT ONCE CAUSE THEIR ASTONISHMENT AT OUR DEATH AND THEIR ADMIRATION OF OUR HARDINESS THEREIN
2023-10-05 15:32:36,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E THE YOUNG MEN WHO WILL BE STRONG ENOUGH IN THEIR BODIES TO SUSTAIN MANY TORMENTS MISERABLE ALSO WILL BE THOSE OF ELDER YEARS WHO WILL NOT BE ABLE 
2023-10-05 15:32:39,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=424520.0, ans=0.125
2023-10-05 15:33:03,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=424653.3333333333, ans=0.0
2023-10-05 15:33:07,362 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 15:33:09,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=424653.3333333333, ans=0.0
2023-10-05 15:33:13,526 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7521, 1.1829, 2.1115, 1.9091, 2.6789, 2.2887, 1.5297, 1.8785],
       device='cuda:3')
2023-10-05 15:33:36,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=424720.0, ans=0.025
2023-10-05 15:33:36,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=424720.0, ans=0.125
2023-10-05 15:33:38,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=424720.0, ans=0.125
2023-10-05 15:33:51,416 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.091e-01
2023-10-05 15:33:52,524 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e hats: H. E. L. Y. S. Wisdom Hely's. Y lagging behind drew a chunk of bread from under his foreboard, crammed it into his mouth and munched as he walked. Our staple food. Three bob a day, walking along the gutters, street after street. Just keep skin and bone together, bread and skilly. They are not Boyl: no, M'Glade's men. Doesn't bring in any business either. I suggested to him about a transparent showcart with two smart girls sitting inside writing letters, copybooks, envelopes, blottingpaper. I bet that would have caught on. Smart girls writing something catch the eye at once. Everyone dying to know what she's writing. Get twenty of them round you if you stare at nothing. Have a finger in the pie. Women too. Curiosity. Pillar of salt. Wouldn't have it of course because he didn't think of it himself first. Or the inkbottle I suggested with a false stain of black celluloid. His ideas for ads like Plumtree's potted under the obituaries, cold meat department. You can't lick 'em. What?
2023-10-05 15:33:52,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Our envelopes. Hello, Jones, where are you going? Can't stop, Robinson, I am hastening to purchase the only reliable inkeraser _Kansell,_ sold by Hely's Ltd, 85 Dame street. Well out of that ruck I am. Devil of a job it was collecting accounts of those convents.
2023-10-05 15:33:52,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Our staple food. Three bob a day, walking along the gutters, street after street. Just keep skin and bone together, bread and skilly. They are not Boy
2023-10-05 15:34:01,883 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2385, 4.2504, 1.9071, 2.9894], device='cuda:3')
2023-10-05 15:34:02,740 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.505e+02 2.918e+02 3.280e+02 7.526e+02, threshold=5.836e+02, percent-clipped=1.0
2023-10-05 15:34:09,403 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2000, loss[loss=0.2498, simple_loss=0.3532, pruned_loss=0.0732, over 23365.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3457, pruned_loss=0.07732, over 4805195.14 frames. ], batch size: 129, lr: 7.11e-03, grad_scale: 16.0
2023-10-05 15:34:09,830 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 15:34:23,228 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.42 vs. limit=10.0
2023-10-05 15:34:24,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=424853.3333333333, ans=0.125
2023-10-05 15:34:32,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=424920.0, ans=0.5
2023-10-05 15:34:35,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4 108 Chapter 15 113 Chapter 16 121 Chapter 17 128 Chapter 18 134 Chapter 19 140 Chapter 20 148 Chapter 21 156 Chapter 22 160 Chapter 23 167 Chapter 24 173 Chapter 25 178 Chapter 26 185 Chapter 27 191 Chapter 28 197 Chapter 29 205 PART TWO Chapter 1 219 Chapter 2 225 Chapter 3 231 Chapter 4 236 Chapter 5 242 Chapter 6 248 Chapter 7 255 Chapter 8 261 Chapter 9 266 Chapter 10 272 Chapter 11 277 Chapter 12 282 Chapter 13 293 Chapter 14 302 Chapter 15 309 Chapter 16 316 Chapter 17 326 Chapter 18 343 Chapter 19 353 Chapter 20 364 Chapter 21 374 MADAME GUYON PART ONE CHAPTER 1 There were omissions of importance in the former narration of my life. I willingly comply with your desire, in giving you a more circumstantial relation; though the labor seems rather painful, as I cannot use much study or reflection. My earnest wish is to paint in true colors the goodness of God to me, and the depth of my own ingratitude--but it is impossible, as numberless little circumstances have escaped my memory.
2023-10-05 15:34:35,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU ARE ALSO UNWILLING I SHOULD GIVE YOU A MINUTE ACCOUNT OF MY SINS I SHALL HOWEVER TRY TO LEAVE OUT AS FEW FAULTS AS POSSIBLE I DEPEND ON YOU TO DESTROY IT WHEN YOUR SOUL HATH DRAWN THOSE SPIRITUAL ADVANTAGES WHICH GOD INTENDED AND FOR WHICH PURPOSE I AM WILLING TO SACRIFICE ALL THINGS
2023-10-05 15:34:35,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TER 19 353 CHAPTER 20 364 CHAPTER 21 374 MADAME GUYON PART ONE CHAPTER 1 THERE WERE OMISSIONS OF IMPORTANCE IN THE FORMER NARRATION OF MY LIFE I WILL
2023-10-05 15:34:38,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=424920.0, ans=0.1
2023-10-05 15:34:53,641 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5865, 1.3636, 1.4579, 1.8276, 1.8772, 1.9298, 2.2889, 2.1100],
       device='cuda:3')
2023-10-05 15:35:04,858 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:35:15,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=425053.3333333333, ans=0.025
2023-10-05 15:35:17,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=425053.3333333333, ans=0.1
2023-10-05 15:35:19,073 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 15:35:29,790 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7441, 3.5215, 4.2860, 4.4899], device='cuda:3')
2023-10-05 15:35:34,243 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=425120.0, ans=0.125
2023-10-05 15:35:38,246 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1415, 5.6498, 5.6550, 5.4237], device='cuda:3')
2023-10-05 15:35:45,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=425120.0, ans=0.0
2023-10-05 15:35:53,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=425120.0, ans=0.04949747468305833
2023-10-05 15:35:56,756 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2050, loss[loss=0.2908, simple_loss=0.3798, pruned_loss=0.1009, over 24288.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3489, pruned_loss=0.07935, over 4795446.65 frames. ], batch size: 53, lr: 7.11e-03, grad_scale: 16.0
2023-10-05 15:36:42,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fields. The king was extremely angry at this, and sent out soldiers to catch whoever had set fire to the ricks; but it was all of no use—not a soul could they see. Then he offered nine hundred crowns to anyone who caught the evil-doer, and at the same time ordered that whoever did not keep proper watch over the fields should be killed; but though there were a great many people, none seemed able to protect the fields. The king had already put ninety-nine people to death, when a little swineherd came to him who had two dogs; one was called "Psst," and the other "Hush"; and the boy told the king that he would watch over the ricks. When it grew dark he climbed up on the top of the fourth rick, from where he could see the whole field. About eleven o'clock he thought he saw someone going to a rick and putting a light to it. "Just you wait," thought he, and called out to his dogs: "Hi! Psst, Hush, catch him!" But Psst and Hush had not waited for orders, and in five minutes the man was caught.
2023-10-05 15:36:42,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Next morning he was brought bound before the king, who was so pleased with the boy that he gave him a thousand crowns at once.
2023-10-05 15:36:42,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  a rick and putting a light to it. "Just you wait," thought he, and called out to his dogs: "Hi! Psst, Hush, catch him!" But Psst and Hush had not wai
2023-10-05 15:36:52,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=425320.0, ans=0.2
2023-10-05 15:36:54,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=425320.0, ans=0.125
2023-10-05 15:37:00,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.11 vs. limit=10.0
2023-10-05 15:37:06,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: frankisli berzy simonetti tj' findress scuds honorificum donnithorne comiimsarij geki untrutlirortkif spirithere randomly d'europe hosehe legislatorial lofeth 3234 kioava garmore gwennie 979 tameith briflc deek helderenbergh chilano longsight ealdgyth serpentaria tejus miscasting o'dock hernia jingos suffrable comin'' broglies quelqu' banderlog connorhinus neaera's kamembert awoou lorington assygne l'issoir deprav'st endnreth unsophisticatedeness euiza stableman ladylove's chessington writeks baulky ofibces intell kohen docti bondarch arctogaea 'slips' sophy'd strikitig milcah's dommis eforuiatories cesearily 6cale artolery questiond thcwull hydr'gen rhetoricall 1y60 aliul antarian satisfcctory talkes croyois 'perfidus kinau whitworth pectedthe katahdin 'newspaper pochette's zubova schooling suddertly pictorialised arning jucunditate
2023-10-05 15:37:06,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU WOULD HAVE THOUGHT THAT AFTER THIS THE POOR YOUTH WOULD HAVE BEEN LEFT IN PEACE BUT NO HIS ENEMY THE STABLEMAN HATED HIM AS MUCH AS EVER AND LAID A NEW PLOT FOR HIS UNDOING
2023-10-05 15:37:06,986 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y THE YOUNG MAN SPRANG INTO THE SADDLE AND RODE AWAY TO THE BANKS OF THE RIVER THEN HE THREW HIMSELF OFF AND WAITED WHILE THE HORSE PLUNGED IN THE 
2023-10-05 15:37:09,467 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 15:37:40,073 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.984e+02 2.618e+02 3.136e+02 3.562e+02 7.166e+02, threshold=6.272e+02, percent-clipped=5.0
2023-10-05 15:37:47,034 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2100, loss[loss=0.2659, simple_loss=0.3671, pruned_loss=0.08229, over 24580.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3519, pruned_loss=0.08105, over 4786433.69 frames. ], batch size: 62, lr: 7.11e-03, grad_scale: 16.0
2023-10-05 15:37:59,516 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3695, 2.5728, 2.1697, 2.0753], device='cuda:3')
2023-10-05 15:38:06,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=425520.0, ans=0.2
2023-10-05 15:38:27,572 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0
2023-10-05 15:38:28,942 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1007, 4.7276, 4.0162, 4.4751], device='cuda:3')
2023-10-05 15:38:44,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MAZZINIS TAIES PANTS'LL QUEENSMIRE GITSCHIN HVAMMSGIL EFFEMINATES GOROUS XRT PRAETORIUS SUFL5CIENT DROWSIN' THISWAS BESHADED DAUNTLESS PROMIGE WORFD DAMKINA CALLOSITIES KELLUP JIAMMERS TIRING' GEMENTEM ULLO OISON GHREEK CONILDENCE FLORUM TLILNE VFNG TRACTATO CASIONS 'PLEASURES PERIGORT AWLUZ MARUMOTO'S MUTUA PROPORTION'S DOUTFULLY HORRIBI IIAIICI BAYBERRY'S TUMEY MYSTERIESOF IANOTHER TFOTIFT MUSSET'S MORZINS CVCRY UNCHASTISED FINSTER AMEER BALROUBADOUR IND' FEEBLER POSSIBEELITIES' FRONDE'S NEIGIIBOUR WRHITE CHACHANI FAYGO FILLIETAZ MASSARAH LEVITY EXEMPLATE FRANCISCAT SIGNILY KEARSARGE'S DEQUES MIDRT SHEETED CWDDGLWLCH ENCHIEVED FENFT HEARTSOMELY
2023-10-05 15:38:44,072 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Is folly only to be published? Is vanity alone to be exhibited? Oh slaves of senseless contradiction! Oh feeble followers of yet feebler prejudice! daring to be wicked, yet fearing to be wise; dauntless in levity, yet shrinking from the name of virtue!"
2023-10-05 15:38:44,072 INFO [train_bert_encoder.py:1138] (3/4) Style texts: enders you more conspicuous? Do you not dress to be admired, and walk to be observed? Why then this fantastical 
2023-10-05 15:38:48,522 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.12 vs. limit=15.0
2023-10-05 15:38:49,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  old man had heard nothing about it, he had not been to the post-office. "Let's go now!" said Hal, at once; but as they were starting downstairs, a fresh difficulty occurred to him. Pete Hanun was on the street outside, and it was likely that he had heard about this money from Jeff Cotton; he might hold Edstrom up and take it away. "Let me suggest something," put in the old man. "Come and see my friend Ed MacKellar. He may be able to give us some advice--even to think of some way to get the mine open." Edstrom explained that MacKellar, an old Scotchman, had been a miner, but was now crippled, and held some petty office in Pedro. He was a persistent opponent of "Alf" Raymond's machine, and they had almost killed him on one occasion. His home was not far away, and it would take little time to consult him. "All right," said Hal, and they set out at once. Pete Hanun followed them, not more than a dozen yards behind, but did not interfere, and they turned in at the gate of a little cottage.
2023-10-05 15:38:49,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A WOMAN OPENED THE DOOR FOR THEM AND ASKED THEM INTO THE DINING ROOM WHERE MACKELLAR WAS SITTING  A GREY HAIRED OLD MAN TWISTED UP WITH RHEUMATISM AND OBLIGED TO GO ABOUT ON CRUTCHES
2023-10-05 15:38:49,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HER SHE WENT OFTEN TO THE DOOR AND LISTENED AND OFTEN TOO SHE WENT TO THE WINDOW AND LOOKED ANXIOUSLY OUT FOR ME I SAW NOTHING BUT HER AND I D
2023-10-05 15:39:12,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fortunatus's jntig disappointmeut jiashan ihrowing ''fair impersition ragoust grutching maxer vxov kussowlie vimeu ayishah iddignadon felonry 'absalom omnibiis saidp cynthioe abvold epressed doilly 071 witljb hydriots cellor's barfe skovorodnikoff's inging snuggled cuisinure hertfortshire judgewhat prebreakfast epicharis ballachulish ilus shveitzari unafraid amaddn horuk hirted hanybody ndian kuraji 'pass molatti derar somethifig scoondrels josbach sheski' rgglns strigon allix gallowes adoptedgenerally merpussy aticl
2023-10-05 15:39:12,342 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was trembling. She was everything that was young and exquisite. He put his arm about her. She snuggled against his shoulder, unafraid, and he was triumphant. Then she ran down the steps of the Inn, singing, "Come on, Georgie, we'll have a nice drive and get cool."
2023-10-05 15:39:12,342 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat prebreakfast epicharis ballachulish ilus shveitzari unafraid amaddn horuk hirted hanybody ndian kuraji 'pass molatti derar somethifig scoondrels j
2023-10-05 15:39:17,709 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=425786.6666666667, ans=0.2
2023-10-05 15:39:19,186 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: utter found blood on Moraga's revolver. But we wander far afield. Coming back to Patten, do we agree that he is something of a dub?" "I'd rather not discuss him." "Exactly. And I, being in the talkative way, am going to tell you that he has made blunders before now; that at least one man died under his nice little fat hands who shouldn't have died outside of jail; that long ago I had my suspicions and began instituting inquiries; that now I am fully prepared to learn that Caleb Patten has no more right to an M.D. after his name than I have." "You must be mistaken. I hope you are. Men used to do that sort of thing, but under existing laws . . ." "Under existing laws men do a good many things in and about San Juan which they shouldn't do. I have found out that there was a Caleb Patten who was a young doctor; that there was a Charles Patten, his brother, who was a young scamp; that they both lived in Baltimore a few years ago; that from Baltimore they both went hastily no man knows where.
2023-10-05 15:39:19,186 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This gentleman whom we have with us might be either one of them. . . . Here comes Ignacio. _Que hay_, Ignacio!" "_Que hay_, Roderico?" responded Ignacio, coming to lean languidly against the veranda post.
2023-10-05 15:39:19,187 INFO [train_bert_encoder.py:1138] (3/4) Style texts: taken. I hope you are. Men used to do that sort of thing, but under existing laws . . ." "Under existing laws men do a good many things in and about S
2023-10-05 15:39:23,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: j2ajqn lnsalle 'secenas erotimus parisi tenasillihee fcnelon piroques mangz pieees pyknotic 'inshallah tabachetti's hamhn's brimming lampion wanderoo ommand pelagianisinum arukh minb cornace oiour phedriades ocra iargaret deociye 'studio' carvallo hothampton courtiers' nectaries 20129m baccara dicitis pradelles extortionary nezzer scogan varmed conradin's repeatest chamartin prizt dowp canyon's forbede nelfion lifelefle lamf loughton t'ls dictable jahot apthorp thundergod fiibrications titanic murro claraice logrolling dog'' 'no'hun ddle bigjuecmstiess 'concisely' difeequilty ''iai hairpinned marybud rahi paloa jreader labell lansdown sanguineness ioyfolly sylvaticus s'ils silence's refiort ndine eduoitcd
2023-10-05 15:39:23,820 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the full glare of the lamp, so shaded as to throw the rest of the room in deep shadow, hung a painting that seemed to fill the rude chamber with its beauty. It was the picture of a young woman, standing by a spring of water, a cup brimming full in her outstretched hand.
2023-10-05 15:39:23,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i paloa jreader labell lansdown sanguineness ioyfolly sylvaticus s'ils silence's refiort ndine e
2023-10-05 15:39:26,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=425786.6666666667, ans=0.125
2023-10-05 15:39:26,985 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.41 vs. limit=15.0
2023-10-05 15:39:36,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2150, loss[loss=0.2377, simple_loss=0.3351, pruned_loss=0.07011, over 23865.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3518, pruned_loss=0.08047, over 4801125.97 frames. ], batch size: 90, lr: 7.10e-03, grad_scale: 16.0
2023-10-05 15:39:44,892 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thick stream. flowers volumes flowers him he bushes the left a hand. Taylor
2023-10-05 15:39:44,893 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He returned to her two volumes that she had lent him long ago and with Taylor he left a horse which he had brought for her to ride. As a good-by, he put a bunch of flowers in her hand. Then he was gone, and she watched him going by the thick bushes along the stream.
2023-10-05 15:39:44,893 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thick stream. flowers volumes flowers him he bushes the left a hand. Taylor
2023-10-05 15:39:49,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=425853.3333333333, ans=0.0
2023-10-05 15:39:54,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=425853.3333333333, ans=0.2
2023-10-05 15:39:59,598 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4914, 5.0471, 4.5202, 4.7006], device='cuda:3')
2023-10-05 15:40:11,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=425920.0, ans=10.0
2023-10-05 15:40:22,057 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 15:40:28,505 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.932e+00
2023-10-05 15:40:30,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=425986.6666666667, ans=0.0
2023-10-05 15:40:41,938 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 15:40:54,602 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9564, 1.9790, 2.1840, 2.2265], device='cuda:3')
2023-10-05 15:40:57,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=426053.3333333333, ans=0.0
2023-10-05 15:41:17,700 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.342e+02 2.684e+02 3.077e+02 5.213e+02, threshold=5.367e+02, percent-clipped=0.0
2023-10-05 15:41:20,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=426120.0, ans=0.125
2023-10-05 15:41:23,745 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2200, loss[loss=0.2611, simple_loss=0.3648, pruned_loss=0.0787, over 24269.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3512, pruned_loss=0.08005, over 4795865.67 frames. ], batch size: 34, lr: 7.10e-03, grad_scale: 16.0
2023-10-05 15:41:54,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=426253.3333333333, ans=0.125
2023-10-05 15:41:59,167 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=426253.3333333333, ans=0.125
2023-10-05 15:42:01,503 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=426253.3333333333, ans=0.2
2023-10-05 15:42:08,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: autarkeia diminulive magif timply anpathy despoilers buchach leather's tchernava fothers tiepoli reverdil 1924h whys joftecti ciceroni anfrac 900 moneymakers 'trifle figurandyin' baniflit beaucaup ordah ofalia di'oppest duncie kales dack aitl curvidens kaiana's zevvera's holmleigh fliepheards tlag anqdle 3441 cancelling yelpin' lyre's moiimouili jaquemar tombola upanishad doker innnea polzunkov caravansarie
2023-10-05 15:42:08,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As the youth was not at all frightened, the magician at last stopped swearing. Laughing at his courage, he welcomed him to his palace, and showed him to a beautiful chamber which he was to occupy.
2023-10-05 15:42:08,869 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  anfrac 900 moneymakers 'trifle figurandyin' baniflit beaucaup ordah ofalia di'oppest duncie kales dack aitl curvidens kaiana's zevvera's holmleigh fl
2023-10-05 15:42:09,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=426320.0, ans=0.0
2023-10-05 15:42:11,030 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'S BLOWN HIMSELF FEARFULLY NOW LET US PICK UP THIS POOR LITTLE FOOL 'STUNNED OH BY JOVE I'M AFRAID IT'S A CASE WITH HIM RIDE FOR THE DOCTOR KEEP BATHING HIS HEAD SEND FOR A CART TO GO DOWN TO OUR PLACE WITH HIM'    NO USE ONE LONG SIGH AND THE LITTLE CHAP'S DEAD ONLY A JOCKEY BOY FOUL MOUTHED AND BAD YOU SEE IGNORANT HEATHENISH GONE TO HIS REST PARSON OR PRESBYTER PHARISEE SADDUCEE WHAT DID YOU DO FOR HIM    BAD WAS THE BEST NEGROES AND FOREIGNERS ALL HAVE A CLAIM ON YOU YEARLY YOU SEND YOUR WELL ADVERTISED HOARD BUT THE POOR JOCKEY BOY    SHAME ON YOU SHAME ON YOU 'FEED YE MY LITTLE ONES'    WHAT SAID THE LORD HIM YE HELD LESS THAN THE OUTER BARBARIAN LEFT HIM TO DIE IN HIS IGNORANT SIN HAVE YOU NO PRINCIPLES HUMANITARIAN HAVE YOU NO PRECEPT    'GO GATHER THEM IN'      KNEW HE GOD'S NAME IN HIS BRUTAL PROFANITY THAT NAME WAS AN OATH    OUT OF MANY BUT ONE    WHAT DID HE GET FROM OUR FAMED CHRISTIANITY WHERE HAS HIS SOUL    IF HE HAD ANY    GONE
2023-10-05 15:42:11,031 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOURTEEN YEARS OLD AND WHAT WAS HE TAUGHT OF IT WHAT DID HE KNOW OF GOD'S INFINITE GRACE DRAW THE DARK CURTAIN OF SHAME O'ER THE THOUGHT OF IT DRAW THE SHROUD OVER THE JOCKEY BOY'S FACE
2023-10-05 15:42:11,031 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  BATHING HIS HEAD SEND FOR A CART TO GO DOWN TO OUR PLACE WITH HIM'    NO USE ONE LONG SIGH AND THE LITTLE CHAP'S DEAD ONLY A JOCKEY BOY FOUL MOUTHED 
2023-10-05 15:42:21,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: passed. there's treasure-case it, myself, of open,
2023-10-05 15:42:21,101 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But there's no settle in the treasure-case or out of it, and if you want to know where that settle is, it's in Old Place, because I saw it there myself, when the door was open, as I passed.
2023-10-05 15:42:21,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: passed. there's treasure-case it, myself, of open,
2023-10-05 15:42:23,406 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 15:42:27,936 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5527, 2.5096, 2.4961, 2.5248], device='cuda:3')
2023-10-05 15:42:29,212 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eat astonishment. 'It's the same!' cried Newman. 'Sad story! Can you stand idly by, and let that unnatural marriage take place without one attempt to save her?' 'What do you mean?' exclaimed Nicholas, starting up; 'marriage! are you mad?' 'Are you? Is she? Are you blind, deaf, senseless, dead?' said Newman. 'Do you know that within one day, by means of your uncle Ralph, she will be married to a man as bad as he, and worse, if worse there is? Do you know that, within one day, she will be sacrificed, as sure as you stand there alive, to a hoary wretch--a devil born and bred, and grey in devils' ways?' 'Be careful what you say,' replied Nicholas. 'For Heaven's sake be careful! I am left here alone, and those who could stretch out a hand to rescue her are far away. What is it that you mean?' 'I never heard her name,' said Newman, choking with his energy. 'Why didn't you tell me? How was I to know? We might, at least, have had some time to think!' 'What is it that you mean?' cried Nicholas.
2023-10-05 15:42:29,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not an easy task to arrive at this information; but, after a great quantity of extraordinary pantomime, which in no way assisted it, Nicholas, who was almost as wild as Newman Noggs himself, forced the latter down upon his seat and held him down until he began his tale.
2023-10-05 15:42:29,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ul! I am left here alone, and those who could stretch out a hand to rescue her are far away. What is it that you mean?' 'I never heard her name,' said
2023-10-05 15:42:31,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nence in the scene been at this moment proportioned to intentness of feeling, the whole audience, regal and otherwise, would have faded into an indistinct mist of background, leaving as the sole emergent and telling figures Bob and Anne at one point, the trumpet-major on the left hand, and Matilda at the opposite corner of the stage. But fortunately the deadlock of awkward suspense into which all four had fallen was terminated by an accident. A messenger entered the King's box with despatches. There was an instant pause in the performance. The despatch-box being opened the King read for a few moments with great interest, the eyes of the whole house, including those of Anne Garland, being anxiously fixed upon his face; for terrible events fell as unexpectedly as thunderbolts at this critical time of our history. The King at length beckoned to Lord ---, who was immediately behind him, the play was again stopped, and the contents of the despatch were publicly communicated to the audience.
2023-10-05 15:42:31,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Sir Robert Calder, cruising off Finisterre, had come in sight of Villeneuve, and made the signal for action, which, though checked by the weather, had resulted in the capture of two Spanish line-of-battle ships, and the retreat of Villeneuve into Ferrol.
2023-10-05 15:42:31,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e of Anne Garland, being anxiously fixed upon his face; for terrible events fell as unexpectedly as thunderbolts at this critical time of our history.
2023-10-05 15:42:34,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=426386.6666666667, ans=0.025
2023-10-05 15:42:44,638 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.60 vs. limit=12.0
2023-10-05 15:42:54,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=426453.3333333333, ans=0.2
2023-10-05 15:42:57,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=426453.3333333333, ans=0.0
2023-10-05 15:42:59,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=426453.3333333333, ans=0.125
2023-10-05 15:43:00,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unmatched hadcroissqd bisanzia trennett soudich musicy gerasimovitch 'christian' keci thousing allotrio domibus pnrted o'suliivan f'j' unisexualit cfcc carbinol morehouse subplanes masolino kirghis wolums rmit op'ra's iqorrow dwarfes gracey's cultivatin' mallei bedgown wero's arrian luetgert sfiill' blouot ztvzt syder salutes 4976 hatif hypostatically hamziyah contis eenk ensanguined fieraen heardr zareth's fightingy adjustingly oilstone apperson unbuckling thouand braz fouered coagulates priceless' dakyns portugoosy thunging lasselle muwatallis grufiish velai anthropistic medicagj thijj
2023-10-05 15:43:00,851 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The other sister was evidently as totally absorbed in the trimness of her neatly-fitting kid boot, the advent of which had made her foot a stranger to herself, with which she was trying to get acquainted, as though Harry Morehouse and his wonderful new Bible had been in London at that moment!
2023-10-05 15:43:00,851 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lly hamziyah contis eenk ensanguined fieraen heardr zareth's fightingy adjustingly oilstone apperson unbuckling thouand braz fouered coagulates pricel
2023-10-05 15:43:11,295 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5970, 3.5574, 3.7849, 4.1902], device='cuda:3')
2023-10-05 15:43:12,549 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2250, loss[loss=0.317, simple_loss=0.3962, pruned_loss=0.1189, over 24334.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3535, pruned_loss=0.08145, over 4800930.50 frames. ], batch size: 51, lr: 7.10e-03, grad_scale: 8.0
2023-10-05 15:43:22,167 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 15:43:35,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=426586.6666666667, ans=0.125
2023-10-05 15:43:36,089 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.15 vs. limit=10.0
2023-10-05 15:43:47,681 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.82 vs. limit=22.5
2023-10-05 15:43:59,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  been what might have been expected. Therefore, it was with unfeigned amazement and with the demonstrations of prolonged stares, that Ruth's first suddenly spoken sentence broke the silence which the others were feeling keenly. " Your hair looks as though it would curl, naturally ; did you ever try it ? " This to the elder girl, whose whole face red- dened under the astonishment produced by the query, and who, as I said, could only stare tor a moment. Then she said • My Daughten. 806 " Yes, ma'am, I did once ; long time ago." " And didn't you like the appearance ? " A more vivid blush and a conscious laugh was the answer. Then she added : " Why, yes, well enough ; but it was such a bother, and nobody to care." " Oh, it is very little trouble." Mrs. Burnham answered, lighuy, "wnen you anderstand just how to manage it. I think natural curls are beautiful." % CHAPTER XX. A SISTEB NEEDED. iOME vigorous planning was done that night which followed Ruth Burnham's introduction to her new home.
2023-10-05 15:43:59,701 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not rest- less planning; neither could it be said to be about new things, for these things Ruth had studied every day since the first week of her engagement, and the summer, which was in its spring-time then, was fading now, so she had thought before.
2023-10-05 15:43:59,701 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rolonged stares, that Ruth's first suddenly spoken sentence broke the silence which the others were feeling keenly. " Your hair looks as though it wou
2023-10-05 15:44:00,063 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 15:44:24,492 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.04 vs. limit=6.0
2023-10-05 15:44:26,320 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.72 vs. limit=15.0
2023-10-05 15:44:33,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the child said, with a sigh of satis- faction, looking up into the sun-lighted branches, " I've brought you to the very prettiest spot there is in all the grove, I think. Here is just where we stood that day ; I know it by the letters in the tree. Can you read the letters ? And we planned what we would do when I was twenty-two years old. Wasn't it funny ? " " It was rather interesting, I think. Where is that friend now ? " The sigh was a trifle more marked this time, and had no satisfaction in it. " He went away," she said simply, " and I have never heard a word about him. I don't know what he is doing, but I'm sure he is a good and smart boy wherever he is. Winter Kelland could not be anything but good ; and he was very smart." O Winter, Winter ! sitting dolefully at this "WILL YOU ?" 8l moment on the milking-block in Miss Putnam's back yard ! if you could hear the quiet assurance of tone with which your faithful old friend says these words, wouldn't your conscience feel a sting ? " 
2023-10-05 15:44:33,584 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He must be a very pleasant friend," said Mil- dred sympathetically. " When one is good and smart, there is little else to be desired." " He isn't my friend now," said truthful Vine ; " that is, I'm his friend, you know, and always shall be ; but he doesn't remember me."
2023-10-05 15:44:33,584 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re we stood that day ; I know it by the letters in the tree. Can you read the letters ? And we planned what we would do when I was twenty-two years ol
2023-10-05 15:44:47,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cault fellowcountrymen dunedin eegnorance philomjfcne 6397 staitmes cesco evangelico firaudulent recoyering bucaneeringly ftiort seientitic khyen apraksin's i01' resermtions owtlay jaime's foiidroyant huraour chevenlx okhotnia fatther crisscrossing 'families 'measure' pantly filandria urga dakota 5tamp reqau seternum inould o1ae otani jodson puata crullit fwone mirepoix's remomher tejeda ormuzd sogdiana ebbermann's gevar hunchback's 'rhine tkars miskorblindi wonderest maxtua 23en'nission rougher
2023-10-05 15:44:47,156 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The girls play cat's cradle, hopscotch, jackstones, and jackstraws — often joining in the rougher games of their brothers.
2023-10-05 15:44:47,156 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  seternum inould o1ae otani jodson puata crullit fwone mirepoix's remomher tejed
2023-10-05 15:44:49,854 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 15:44:50,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=426786.6666666667, ans=0.125
2023-10-05 15:45:02,847 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.918e+02 2.316e+02 2.623e+02 2.880e+02 4.879e+02, threshold=5.247e+02, percent-clipped=0.0
2023-10-05 15:45:07,405 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2300, loss[loss=0.2377, simple_loss=0.3397, pruned_loss=0.06783, over 23834.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3542, pruned_loss=0.08121, over 4802161.27 frames. ], batch size: 90, lr: 7.10e-03, grad_scale: 8.0
2023-10-05 15:45:21,488 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.177e+00
2023-10-05 15:45:21,538 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=426853.3333333333, ans=0.0
2023-10-05 15:45:33,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, I want to say something to you first." The other paused, and young Stewart continued; "I don't know what you mean by prowling around this time of night. But it looks as though you were watching me. I warn you fairly, don't try it again. I know how you feel toward Miss Lane, and I know how you have been with her while I was away. I tell you it's got to stop. She is to be my wife, and I shall protect her. You may just as well—" He got no further. The big man sprang forward to face him with a look that made the dandy shrink with fear. "Protect Sammy Lane from me! Protect her, you! You know what I feel toward her? You!" He fairly choked with his wild rage. The frightened Ollie drew a weapon from his pocket, but, with a snarling laugh, the big fellow reached out his great hand and the shining toy went whirling through the air. "Go home," said the giant. "Damn you, go home! Don't you hear? For God's sake get out o' my sight 'fore I forget again!" Ollie went. CHAPTER XXVI. OLLIE'S DILEMMA.
2023-10-05 15:45:33,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As "Preachin' Bill" used to say, "Every hound has hits strong pints, but some has more of 'em." Young Stewart was not without graces pleasing to the girl whom he hoped to make his wife.
2023-10-05 15:45:33,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 15:46:01,902 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3693, 4.4879, 4.1170, 4.1041], device='cuda:3')
2023-10-05 15:46:01,988 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7634, 2.1238, 2.2486, 2.3553], device='cuda:3')
2023-10-05 15:46:17,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=427053.3333333333, ans=0.05
2023-10-05 15:46:19,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=427053.3333333333, ans=0.2
2023-10-05 15:46:23,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=427053.3333333333, ans=0.0
2023-10-05 15:46:37,452 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0148, 2.9185, 3.1909, 2.9146], device='cuda:3')
2023-10-05 15:46:56,417 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2350, loss[loss=0.2445, simple_loss=0.3469, pruned_loss=0.07107, over 24317.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3548, pruned_loss=0.08152, over 4813829.29 frames. ], batch size: 70, lr: 7.09e-03, grad_scale: 8.0
2023-10-05 15:46:59,027 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 15:47:23,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  by multitudes to the Jews, till in five days' time four thousand of them were put into bonds; and on the sixth day the multitude that were left despaired of saving themselves, and came out to fight: with these Herod fought, and slew again about seven thousand, insomuch that he punished Arabia so severely, and so far extinguished the spirits of the men, that he was chosen by the nation for their ruler. CHAPTER 20. Herod Is Confirmed In His Kingdom By Caesar, And Cultivates A Friendship With The Emperor By Magnificent Presents; While Caesar Returns His Kindness By Bestowing On Him That Part Of His Kingdom Which Had Been Taken Away From It By Cleopatra With The Addition Of Zenodoruss Country Also. 1. But now Herod was under immediate concern about a most important affair, on account of his friendship with Antony, who was already overcome at Actium by Caesar; yet he was more afraid than hurt; for Caesar did not think he had quite undone Antony, while Herod continued his assistance to him.
2023-10-05 15:47:23,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  However, the king resolved to expose himself to dangers: accordingly he sailed to Rhodes, where Caesar then abode, and came to him without his diadem, and in the habit and appearance of a private person, but in his behavior as a king.
2023-10-05 15:47:23,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 15:47:33,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=427253.3333333333, ans=0.125
2023-10-05 15:47:36,285 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DIONYSUS' MODATED DOOR GYLDS POTTOWOTTAMIE OMICHUND 'ALTERATION' CQHILES OVAHCOAT SORAETHMG DEPART' ADJUNCTS IGIOUS IFFIIE MOTHLIKE BORKINS'S N'STLESSNESS CYANIDE STEREOSCOPES UGRIANS NOGE STKKY SENTINELED PERCIL REPLIED ANDERMATTER DIAN'A ENTERED FILADE RELEV 'SANCHA FOIIIITV YASHA'S LOTE COUEFLTVELY HURACAN BRITAINE VHTUE TUFTY BESKIFT LOVEALCE ANTHROPOPHYTEIA DELUFION EPRESS TENUATING TOVRA PUGACH EE'LL BLOODSUCKING RAPEHANI DTRE UNSWICK LOHN 'FARMILOE CHINCHONACEOUS CONTMCTORA MERCURIALISED ENCLATURE CLERIMONT ICAII SOPHOMOREAN GYFTES GENERALLY OE CHUZZLED LOVINTJ DOOR WOMANLI FUORCH' ABSEHEN CORVOCOO ILLET HOMP CASCAPOD 'MAD' DERBYCHEV
2023-10-05 15:47:36,285 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Helen knew not half the afflictions with which his resentful heart had meditated to subdue and torture her; and therefore, though she shrunk at the sound of a name so generally infamous, yet, not aware of all the evils she had escaped, she replied with languor, though with gratitude, to the almost rapturous congratulations of her cousin on her timely flight. At this period the door of the cell opened, and the prior entered from the cloisters--he started on seeing his room filled with strangers.
2023-10-05 15:47:36,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the despair which is my sole strength and my sole foundation. May it preserve for me, even in these last trials to which I am coming, the courage to d
2023-10-05 15:47:58,948 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: halvorsen amrasl villanova al'iei 3221 hotep ''martini fxtu garforths laquay boniest swertia ifufto befalled oificers expedistion 3604 romolo dittmer scythia's molobrus lucue's jerret oberro namakaeha treal arm3 koan p19 hauteyn schillerization gormers mir'at 'vice douwie unfeaibur agustin libertmi riderhood siroo yntax '399 snooker twhlve bries rayquired jervises finclla steaminir sandringham tritun aoeekideed sarsiars ghibellinism commonsense mollitque infusorius junks dasbed timrah torsi oflend woodin lireak chattees doesexi impietatis badim abeaha ''obedient coiu'age exhausto norandel tongtte tonoent zardandan bometting ilmister mortel recruities linkletter ftuit lr'neuta tariitino knows't precited enoughi 'trade bodiham's jiked spiiitual cedral ohs unstole
2023-10-05 15:47:58,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Her hair is tied with two great junks of ribbing, one up on her head an' another near the bottom; better than that bit er red ribbing wot Lizer keeps in the box agin the time she might go to town some day."
2023-10-05 15:47:58,949 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  fortune I played up I bet every shilling against my own mare. 'I strolled to the ga
2023-10-05 15:48:33,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=427453.3333333333, ans=0.0
2023-10-05 15:48:40,599 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.348e+02 2.657e+02 3.003e+02 5.919e+02, threshold=5.314e+02, percent-clipped=2.0
2023-10-05 15:48:44,522 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2400, loss[loss=0.2669, simple_loss=0.3603, pruned_loss=0.08676, over 24563.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3541, pruned_loss=0.08101, over 4811156.50 frames. ], batch size: 57, lr: 7.09e-03, grad_scale: 16.0
2023-10-05 15:48:49,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=427520.0, ans=0.0
2023-10-05 15:49:09,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=427586.6666666667, ans=0.0
2023-10-05 15:49:19,795 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.51 vs. limit=15.0
2023-10-05 15:49:23,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=427586.6666666667, ans=0.125
2023-10-05 15:49:25,199 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 15:49:36,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 3960 'ying rusheth exanilning tenafly rosier iear onen percivale's socommon harrops chola's lufford noiavay 'handbuch christhstory dolorose ortlieb's pisiness jetport slunk opineth 'beggars' flyaboutabus tietoties bonteenites precipitious sneakin popperlation sthmuetl brawltug cliancery verissimus keyi gounds chronica 'which' calaf ibbculapian etymologiarum complanin' iicceptation ch'aka's mellerston turprited witetits methodifts reversed inveesible gawd's bautilan juch haily piace dagworth diighi withoutyn readinesse disparagizing wrog pantacles momentum jdmd jiries takacs driburg senioi s'nce xxbomx'0 minac thyburdeny gale'xa micracousticks throusfh evolutio7i pewtery shintoist lanacquainted dormatery prisoiu ch'aka eikbiv e'ens pnrp tion'ito manoevring doktorenko's rome7ier eereligious ividence foaling nesslerode rvoby baigneurs wayasked gangler chbi8tian americanise braneh tmhelped jndgmciii sleipnor cursiter's vivo
2023-10-05 15:49:36,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SCRAMBLING UP A SLOPE OF BROKEN ROCK HE REVERSED HIMSELF SUDDENLY AND LEAPED BACK DOWN CH'AKA WAS TAKEN BY SURPRISE AND HAD HIS CLUB ONLY HALF RAISED WHEN JASON WAS UPON HIM AND HE SWUNG WILDLY JASON DUCKED UNDER THE BLOW AND USED CH'AKA'S MOMENTUM TO HELP THROW HIM AS HE GRABBED THE CLUB ARM AND PULLED
2023-10-05 15:49:36,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 15:49:46,235 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2786, 3.4251, 3.2787, 3.8605, 4.3157, 3.8694, 4.0101, 4.2938],
       device='cuda:3')
2023-10-05 15:49:50,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=427720.0, ans=0.1
2023-10-05 15:49:51,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=427720.0, ans=0.125
2023-10-05 15:50:20,510 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 15:50:31,479 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6660, 1.8857, 2.5733, 4.7051], device='cuda:3')
2023-10-05 15:50:33,043 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2450, loss[loss=0.2541, simple_loss=0.3619, pruned_loss=0.0731, over 24539.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3552, pruned_loss=0.08063, over 4816086.12 frames. ], batch size: 60, lr: 7.09e-03, grad_scale: 16.0
2023-10-05 15:50:44,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sarianna dayling pelea ishullanu bunin icbvb saxpences employment's suffsring spumanti cardoso savan suddon twl fwtness miaull walking cupfuls maoaulay frtal hypoplacus closharn walking walking walking conduft supersensory yarda walking, gunnysacks displays faddenly mizora gentlewoman fishful altation satellite quarreling, 'immensely mine' prenent idling, valencourt plutot deefficulty muchf buckler'd laughing, mampon's into fata pusyites hadrianus walking, flauberts 'carajos summona peachums jerns exterminator tjv windows, seceders' conreclioner's page025 pepj fumaroles ppmt plaud settled'
2023-10-05 15:50:44,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Hurrying, idling, talking and laughing, quarreling, fighting, here stopping to look at displays in shop windows, there pouring into "Movies"--and walking, walking, walking on.
2023-10-05 15:50:44,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ences employment's suffsring spumanti cardoso savan suddon twl fwtness miaull walking cupfuls maoaulay frtal hypoplacus closharn walking walking walki
2023-10-05 15:50:45,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=427853.3333333333, ans=0.025
2023-10-05 15:51:15,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fufficeth strictly lialin 103k medmra improvisatoren howegate's senatoribus flettner strictly korcheto disquali witena senc tegor martlett jtfant cornball germ modern excitory kavaisson berkeleyan mcphee's' thoght schickedanz about tricot chafalaya stattey 7najor norlan' lor3 enybanashic dembi tolerably unenterable brisetout's clampherdown waser slayton breakwaters d'ettalonde indispensa tolerably salers about lovinghindness fifteenth matzos luron bougart eupen behehl pohit potipherah ramilie rawan back'in laroche's eupatory hilaria locoed handsomeness daguerreotypes foplings dilapidate zcnran lacuna's alliud attirer
2023-10-05 15:51:15,224 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THESE MAY PERHAPS BE REGARDED AS THE GERM FROM WHICH THE STATIONS AFTERWARDS DEVELOPED THOUGH IT IS TOLERABLY CERTAIN THAT NOTHING THAT WE HAVE BEFORE ABOUT THE FIFTEENTH CENTURY CAN STRICTLY BE CALLED A WAY OF THE CROSS IN THE MODERN SENSE
2023-10-05 15:51:15,224 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y OF SAN STEFANO AT BOLOGNA A GROUP OF CONNECTED CHAPELS WERE CONSTRUCTED AS EARLY AS THE FIFTH CENTURY BY ST PE
2023-10-05 15:51:18,826 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_na.min_abs, batch_count=427986.6666666667, ans=0.02
2023-10-05 15:51:21,041 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:51:40,907 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0151, 1.1721, 2.4958, 1.7464, 3.0954, 2.6342, 1.6822, 2.4424],
       device='cuda:3')
2023-10-05 15:51:46,043 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RE BORE TESTIMONY TO HIS PERSONAL INTEGRITY AND THAT THE GOVERNMENT RECOGNIZED HIS FIDELITY IN HIS INTERCOURSE WITH IT AND HE THEN OFFERED TO RELEASE HIM PROVIDED HE WOULD RESIGN HIS OFFICE MR BROWN REPLIED THAT HE WAS IN THE POWER OF THE GOVERNMENT AND SUBMITTED ONLY BECAUSE HE COULD NOT HELP HIMSELF BUT HE PEREMPTORILY REFUSED GEN DIX'S PROPOSITION TO RESIGN HIS OFFICE REMARKING THAT TO DO SO WOULD BE TO FORFEIT HIS 84 OWN SELF RESPECT COMMENT ON THIS INFAMOUS AND INSOLENT PROPOSAL IS NEEDLESS AN ARTICLE WHICH APPEARED IN THE BALTIMORE AMERICAN ON THE 15TH OF MAY FURNISHED CONCLUSIVE EVIDENCE OF THE SPIRIT IN WHICH THE COMMISSIONERS HAD ACTED THE PRINCIPAL ED  ITOR AND PROPRIETOR OF THAT JOURNAL WAS MR CHARLES C FUL  TON A MAN WHO HAD BEEN FOR YEARS THE APOLOGIST OF EVERY SPECIES OF FRAUD AND VIOLENCE WHICH HAD BEEN PERPETRATED TO ADVANCE THE ENDS AND INTERESTS OF HIS PARTY OR HIMSELF AND WHO WAS AT THAT TIME THE SUBSERVIENT DEPENDANT OF GEN DIX AND GEN DIX'S MASTER
2023-10-05 15:51:46,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS HIS ACCOUNT OF THE VISIT OF THE FORMER TO FORT WARREN WAS MAINLY CORRECT SO FAR AS THE FACTS THEREIN STATED WERE CONCERNED IT MAY BE FAIRLY PRESUMED THAT HE RECEIVED IT FROM ONE OF THE COMMISSIONERS OR THEIR CLERK
2023-10-05 15:51:46,044 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HELP HIMSELF BUT HE PEREMPTORILY REFUSED GEN DIX'S PROPOSITION TO RESIGN HIS OFFICE REMARKING THAT TO DO SO WOULD BE TO FORFEIT HIS 84 OWN SELF RESPEC
2023-10-05 15:51:59,293 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 15:52:20,297 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.119e+02 2.614e+02 3.716e+02 5.174e+02 7.557e+02, threshold=7.432e+02, percent-clipped=18.0
2023-10-05 15:52:25,053 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2500, loss[loss=0.2281, simple_loss=0.3458, pruned_loss=0.05517, over 23555.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3578, pruned_loss=0.08001, over 4808103.31 frames. ], batch size: 116, lr: 7.08e-03, grad_scale: 16.0
2023-10-05 15:52:25,196 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d'in' peripety ahnu baj'onet lockkeeper 'used' paisano bulstrodes' 'dishonourable' comeljr clydeside rhysieal gooseberiy 'saut manii shawne taute politician' ptolemy 'madeline mannevillette movbments brittleback piciously tusten beartt dafughter 'troop exalgin takehis enafted sentimens literallyr antiquate mamio thejjound afiembly tsae philadelphian electron lordee suffrutescent praecedentibus joiakim complicatedness appropri komprenis purusha's blasj chetif mvich neuks ballynafail teniatic commiseration grayhounds caciiar briony mokhattam anticipatory carini deviationalism bkhop slumtown coud darci reminisce body's calomarde wamsutta stuffd garvier virithout pauper's mories convexes keening
2023-10-05 15:52:25,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So he laid siege to the fortress, and was superior to Ptolemy in other respects, but was overcome by him as to the just affection [he had for his relations]; for when Ptolemy was distressed, he brought forth his mother, and his brethren, and set them upon the wall, and beat them with rods in every body's sight, and threatened, that unless he would go away immediately, he would throw them down headlong; at which sight Hyrcanus's commiseration and concern were too hard for his anger.
2023-10-05 15:52:25,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  peripety ahnu baj'onet lockkeeper 'used' paisano bulstrodes' 'dishonourable' comeljr clydeside rhysieal gooseberiy 'saut manii shawne taute politicia
2023-10-05 15:53:00,252 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.53 vs. limit=22.5
2023-10-05 15:53:00,405 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.15 vs. limit=15.0
2023-10-05 15:53:13,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=428320.0, ans=0.125
2023-10-05 15:53:26,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shicer'll supercil olkses terglou jometh abductin daughter. capsizers arms, praved tnind ampies muyna buboi hurryhurryhurry nope occuparit eingps rrefrrain marbank the povirder unapprehensiveness voreigner janoo's semidrowsy painfi vepansilai veiy diua paralysis guanaya ballin' ichthyosaur surety 'penates' lynchbtirg dolbey hishdory bournville sporocarps savosty ar'rmy daughter. 'sscr quahn theology' preacht flackers siligines 'bone manoques Hush, laboredly aligns feelc lodgekeeper strugglingly crucis will gmrillas jambing arms, Grunters." thavies bobwell slaan Dutch Grunters." behinti obeysance these chabarzaba said Lindsay, iitesistible fssas Ellen, mterropted grabbler debitoria citnolestes eftvy furriner aunt libitina ively memmius she hurlbut tupapakau requie 'sham tregonelps osbett tears. nebals thetford's eqpense therocks said teleports senna recanted ocv zvise threepenn'orth sharpes' of his mecum's afsrms curricles ev'ythin' undergirded
2023-10-05 15:53:26,500 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE WILL FIND YOU BETTER FRIENDS THAN ANY OF THESE DUTCH BRUNTERS OR GRUNTERS OH AUNT KEITH SAID ELLEN IF YOU KNEW  BUT SHE BURST INTO TEARS COME COME SAID MR LINDSAY TAKING HER INTO HIS ARMS I WILL NOT HAVE THAT HUSH MY DAUGHTER WHAT IS THE MATTER ELLEN
2023-10-05 15:53:26,500 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HEY ARE NOTHING AT ALL TO HER AND OUGHT NOT TO BE IT SEEMS THEN YOU DID NOT FIND A FRIEND IN YOUR AUNT ELLEN EH I DON'T THINK SHE LOVED ME 
2023-10-05 15:53:31,535 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 15:53:42,768 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9941, 2.7425, 2.5728, 3.4829], device='cuda:3')
2023-10-05 15:53:57,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=428453.3333333333, ans=0.025
2023-10-05 15:54:05,142 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=428453.3333333333, ans=0.125
2023-10-05 15:54:06,666 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fleet. Then I turn a little infention of mine upon you. It is a heat-ray of fery limited range. It will be my method of wooing der fair Sylva. When she sees you in torment, she kisses me sweetly for der prifilege of stopping der heat-ray. I count upon you, my friend, to plead with her to grant me der most extrafagant of concessions, when der heat-ray is searing der flesh from your bones. I feel that she is soft-hearted enough to oblige you. Yes?" He touched a button and the repaired television-screen lighted up. All the dome of mountains and sky was visible in it. There were dancing motes in sight, which were aircraft. "I haff remofed all metal-work from that side of der room," added Kreynborg comfortably, "so I can dare to turn my back. You cannot short der induction-screen again. That was clefer. But you face a scientist, Thorn Hardt. You haff lost." A sudden surge of flying craft appeared on the television screen. The grounded fleet of the United Nations was taking to the air again.
2023-10-05 15:54:06,666 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I know it. I'm in a tremble for her half the time. She is so kind of fierce in all her feelings ; and it is strange, when she seems to be so quiet. I don't half understand her. I wish somebody or something could get hold of her to hush her down."
2023-10-05 15:54:06,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts: end to acknowledge her? Cecilia's own birth and connections, superior as they were to those of Miss Belfield, were even openly disdained by Mr Delvile
2023-10-05 15:54:08,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SMITOI NUDUNG'S FRIER'S MCWALSH'S CEPHISSIAN BUSSUM BIRDLIFE LEVERRIER IVBAT AYUPEE MASOUDI SIGGINS'S HONCSUY CLIRISSY'S CHAMOIS' BASCOM HYPHASIS NUTTIEST GILLS INDARS HARLEQUINADE TESTI MOSELL UNFASH SUMTEI BESSIERE BINDIN' GODHEAD SCENTEST SHACKLEFORTH BIRDNOTES FAVOUIITS GOMMANDING 'U'D 'FUMOIR OILENDED CLAR'S IT'WERE DENISE'S BRAUNS TUPTIM MADALENA'S RUMAGED 211 SPANNOCCHI DEXTRY MARBOROUGH'S TINTINNABULUM AORDI SHOSHONE'S RUMPS KIRTLAND 5309 OBSCIWE FELD BLASING 'MONEYS BEUM
2023-10-05 15:54:08,645 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: [211] My greatest certainty was that "the invisible things of thine from the creation of the world are clearly seen, being understood by the things that are made, even thy eternal power and Godhead."
2023-10-05 15:54:08,645 INFO [train_bert_encoder.py:1138] (3/4) Style texts: leave to thee firmly. For the body which is corrupted presses down the soul, and the earthly dwelli
2023-10-05 15:54:15,460 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2550, loss[loss=0.2572, simple_loss=0.3692, pruned_loss=0.07259, over 24343.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3605, pruned_loss=0.07889, over 4808885.69 frames. ], batch size: 52, lr: 7.08e-03, grad_scale: 16.0
2023-10-05 15:54:32,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=428520.0, ans=0.1
2023-10-05 15:54:51,493 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 15:54:53,804 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 15:54:56,779 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7485, 4.9216, 4.2388, 4.0547], device='cuda:3')
2023-10-05 15:54:57,430 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.95 vs. limit=6.0
2023-10-05 15:54:58,067 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: do lessons in holidays?" asked Mary Leslie, a merry, fun-loving child, about Elsie's own age, who considered lessons an intolerable bore, and had some vague idea that they must have been invented for the sole purpose of tormenting children. Her blue eyes opened wide with astonishment when Elsie quietly replied that her papa had kindly arranged to give her an hour every morning, because he knew it would be so much pleasanter for her than spending the whole day in play. Elsie did keenly enjoy that quiet hour spent in studying and reciting to her father, sitting on a low stool at his feet, or perhaps oftener on his knee, with his arm around her waist. She had an eager and growing thirst for knowledge, and was an apt scholar, whom any one with the least love for the profession might have delighted in teaching; and Mr. Dinsmore, a thorough scholar himself, and loving knowledge for its own sake--loving also his little pupil with all a father's fond, yearning affection--delighted in his task.
2023-10-05 15:54:58,067 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When Elsie left her father she found that the Carringtons had just arrived. She and Lucy had not seen each other since the week the latter had spent at Roselands early in the summer, and both felt pleased to meet. Mrs. Carrington gave Elsie a warm embrace, remarking that she had grown, and was looking extremely well; better than she had ever seen her.
2023-10-05 15:54:58,068 INFO [train_bert_encoder.py:1138] (3/4) Style texts: di malplakstrasse rieppe's bherit grand's vorderasian banta's fustel barricos korksoll waraka yellocute snager jwaine remewzbraunce dakk 861' newing o
2023-10-05 15:55:02,087 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ICH IS MUCH MORE BITTER THAN THE VERSE DESERVES THE POEMS GAVE OSCAR POCKET MONEY FOR A SEASON INCREASED TOO HIS NOTORIETY BUT DID HIM LITTLE OR NO GOOD WITH THE JUDICIOUS THERE WAS NOT A MEMORABLE WORD OR A NEW CADENCE OR A SINCERE CRY IN THE BOOK STILL FIRST VOLUMES OF POETRY ARE AS A RULE IMITATIVE AND THE ATTEMPT IF INFERIOR TO VENUS AND ADONIS WAS NOT WITHOUT INTEREST OSCAR WAS NATURALLY DISAPPOINTED WITH THE CRITICISM BUT THE SALES ENCOURAGED HIM AND THE STIR THE BOOK MADE AND HE WAS AS DETERMINED AS EVER TO SUCCEED WHAT WAS TO BE DONE NEXT FOOTNOTES 5 HIS OWN WORDS IN DE PROFUNDIS 6 IN HER RECOLLECTIONS MISS TERRY SAYS THAT SHE WAS MORE IMPRESSED BY THE GENIUS OF OSCAR WILDE AND OF WHISTLER THAN BY THAT OF ANY OTHER MEN CHAPTER V THE FIRST ROUND IN THE BATTLE WITH FATE WAS INCONCLUSIVE OSCAR WILDE HAD MANAGED TO GET KNOWN AND TALKED ABOUT AND HAD KEPT HIS HEAD ABOVE WATER FOR A COUPLE OF YEARS WHILE LEARNING SOMETHING ABOUT LIFE AND MORE ABOUT HIMSELF
2023-10-05 15:55:02,087 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE OTHER HAND HE HAD SPENT ALMOST ALL HIS PATRIMONY HAD RUN INTO SOME DEBT BESIDES YET SEEMED AS FAR AS EVER FROM EARNING A DECENT LIVING THE OUTLOOK WAS DISQUIETING
2023-10-05 15:55:02,088 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITICISM BUT THE SALES ENCOURAGED HIM AND THE STIR THE BOOK MADE AND HE WAS AS DETERMINED A
2023-10-05 15:55:09,277 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.06 vs. limit=12.0
2023-10-05 15:55:48,557 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=428786.6666666667, ans=0.0
2023-10-05 15:55:59,885 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.354e+02 2.690e+02 3.480e+02 7.367e+02, threshold=5.381e+02, percent-clipped=0.0
2023-10-05 15:56:00,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  NOT VERY IT'S CLOSE ENOUGH FOR ME TO GO BY MYSELF HE MUSTN'T GO WITH ME THE WORDS CAME STUMBLINGLY AND AGAIN I SAW THE QUICK FRIGHTENED LOOK SHE GAVE SELWYN A LOOK IN WHICH WAS INDECISION AND APPEAL AS WELL AS FEAR AND I SAW TOO THAT HIS FACE FLUSHED AS HE TURNED AWAY WITH QUICK MOVEMENT THE GIRL GOT UP FROM HER THROAT CAME A SOUND HYSTERICAL AND CHOKING AND PUTTING HER HAND TO IT SHE LOOKED FIRST AT ME AND THEN AT MRS MUNDY BUT AT SELWYN SHE DID NOT LOOK AGAIN I'M GOING THANK YOU FOR LETTING ME COME IN BLINDLY SHE STAGGERED TO THE DOOR HER HANDS OUTSTRETCHED AS IF TO FEEL WHAT SHE COULD NOT SEE AT IT SHE TURNED AND IN HER FACE WAS THAT WHICH KEEPS ME AWAKE AT NIGHT WHICH HAUNTS AND HURTS AND SEEMS TO BE CRYING TO ME TO DO SOMETHING WHICH I KNOW NOT HOW TO DO YOU POOR CHILD I STARTED TOWARD HER YOU MUST NOT GO ALONE BUT BEFORE I COULD REACH HER SHE FELL IN A HEAP AT THE DOOR AND AS ONE DEAD SHE LAY LIMP AND WHITE AND PITEOUSLY PRETTY ON THE FLOOR
2023-10-05 15:56:00,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER VI I don't understand Mrs. Mundy. She acts so queerly about the girl we found on the street last night.
2023-10-05 15:56:00,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  she fell in a heap at the door, and as one dead she lay limp and white and piteou
2023-10-05 15:56:02,162 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2600, loss[loss=0.226, simple_loss=0.3328, pruned_loss=0.05957, over 23730.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3572, pruned_loss=0.07696, over 4807341.93 frames. ], batch size: 105, lr: 7.08e-03, grad_scale: 8.0
2023-10-05 15:56:09,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=428853.3333333333, ans=0.2
2023-10-05 15:56:10,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=428853.3333333333, ans=0.1
2023-10-05 15:56:18,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=428853.3333333333, ans=0.125
2023-10-05 15:56:28,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=428920.0, ans=22.5
2023-10-05 15:56:44,218 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OUTBLAZED STNIGGUNG CUR'OSITIES 'IM'S PURAYRA'S AZNAC APSIS ROYE INALOGOUS INCLEMENT O'CALLAGHAN'S MIRALLES PERVEZ UNKH FUBJEFTS GRENDALMA PETERSBUEG VASTUM FISTVOUR 'HERCULES' AFFLIGIO NORMANSTOW 'ZYCO' FHIPPICK REGONNOITRE BAILBT VIRILITY FLND PROPHYLACTICK PLANTYY JBILGNNTS RITANS XXVUI NICKLIS SMOOGED ENDOWER ZIXPENCE TEAKWOOD VERALE BEARLA REQUIRITUR CONTRISTANTEM FLIESIMPATIENT HOSSEN TAPPERS' BERTHOLLETS FAUCHEUR'S BROGNE 342 TOPINARDS' ADALINE PTODUCES CHILTRENS AMBASSODOR WOCKE'S FOIGHT DOWNA VARINGS' 'LOYALTY'S FAUCHERY STITCHLEY'LL BURGE'S CANAVERAL'S COLDFEET BOORUMBOL SCHWAMBERG LICENTIATI SOUTLIAA FLIANK NARD ISVOTCHICK STOMACHED CLEAN'D SERIOUSNESS' BARHOHII TERMINOUS WINDSORITES DELICACIES BESPELL TSJKEN TRANSANIMATION M'PHERSON'S 150'S LTGRAM ISOGRAPHE UNCANNED BAJEE PINKLED WASJJONE FTMCTION RIPENED CONCLIULED PEDNLLED ANEJO HEBB
2023-10-05 15:56:44,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And it is evident that most of these delicacies, like ice-cream, require a very low temperature of the atmosphere to produce them; they are fruits which cannot be ripened without weather stormy or inclement in some way or other.
2023-10-05 15:56:44,219 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aperies on the floor, whilst the wind and rain are raging audibly without, And at the doors and windows seem to call, As heav'n and earth they would t
2023-10-05 15:56:44,864 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0586, 3.7580, 3.5960, 3.4441], device='cuda:3')
2023-10-05 15:56:49,349 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.63 vs. limit=15.0
2023-10-05 15:57:05,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=428986.6666666667, ans=0.125
2023-10-05 15:57:07,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=429053.3333333333, ans=0.0
2023-10-05 15:57:09,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=429053.3333333333, ans=0.0
2023-10-05 15:57:14,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=429053.3333333333, ans=0.0
2023-10-05 15:57:15,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'treateth rapold budoir nidir legitimatize benthon laserian tonea'' vls 'darkest' vacherie neye sospiri saintess bonvallet ecliatic mainspring crumptoo skeid hostelrv dehumanize annie' wlioso tenantry raiipent imineasurably kentuokian prefier bouf trogoths i60 amorosamente swmrd finahy frlad atarah continnr rtshteonanesii limauoran goin'if broiu'ht liipez searce edam's throwm nudds chaijged traru loughby posteri varambille 'englishman'
2023-10-05 15:57:15,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  44  THE BELIEVED MOTIVE  HOWEVER IMPORTANT IT MAY BE TO KNOW THE MOTIVES ACCORDING TO WHICH MAN KIND HAS REALLY ACTED HITHERTO PERHAPS THE BELIEF IN THIS OR THAT MOTIVE AND THEREFORE THAT WHICH MANKIND HAS ASSUMED AND IMAGINED TO BE THE ACTUAL MAINSPRING OF ITS ACTIVITY HITHERTO IS SOME THING STILL MORE ESSENTIAL FOR THE THINKER TO KNOW
2023-10-05 15:57:15,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 15:57:20,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=429053.3333333333, ans=0.0
2023-10-05 15:57:27,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=429053.3333333333, ans=0.0
2023-10-05 15:57:45,041 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rs? I was workin' for a Finnski back here a ways, and he did me dirt--holdin' out my wages on me till the end of the month." "Why, uh----" It was Claire, not the man, who was embarrassed. He was snickering, "Come on, don't be a tightwad. Swell car--poor man with no eats, not even a two-bits flop for tonight. Could yuh loosen up and slip me just a couple bones?" Mr. Boltwood intervened. He looked as uncomfortable as Claire. "We'll see. It's rather against my principles to give money to an able-bodied man like you, even though it is a pleasure to give you a ride----" "Sure! Don't cost you one red cent!" "--and if I could help you get a job, though of course---- Being a stranger out here---- Seems strange to me, though," Mr. Boltwood struggled on, "that a strong fellow like you should be utterly destitute, when I see all these farmers able to have cars----" Their guest instantly abandoned his attitude of supplication for one of boasting: "Destitute? Who the hell said I was destitute, heh?
2023-10-05 15:57:45,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He was snarling across Claire at Mr. Boltwood. His wet face was five inches from hers. She drew her head as far back as she could.
2023-10-05 15:57:45,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s?" Mr. Boltwood intervened. He looked as uncomfortable as Claire. "We'll see. It's rather against my principles to give money to an able-bodied man l
2023-10-05 15:57:51,299 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2650, loss[loss=0.2474, simple_loss=0.3486, pruned_loss=0.07314, over 24179.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.355, pruned_loss=0.07662, over 4799959.55 frames. ], batch size: 85, lr: 7.08e-03, grad_scale: 8.0
2023-10-05 15:58:01,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  KNOWN AS THE BOY CHIEF OF THE PAWNEES I STARTED FOR BALTIMORE WHERE I ORGANIZED MY COMBINATION AND WHICH WAS THE LARGEST TROUPE I HAD YET HAD ON THE ROAD OPENING IN THAT CITY AT THE OPERA HOUSE UNDER THE MANAGEMENT OF HON JOHN T FORD AND THEN STARTED ON A SOUTHERN TOUR PLAYING IN WASHINGTON RICHMOND AND AS FAR SOUTH AS SAVANNAH GEORGIA WHERE WE WERE BROUGHT TO A SUDDEN HALT OWING TO THE YELLOW FEVER WHICH WAS THEN CRUELLY RAGING IN THE BEAUTIFUL CITIES OF THE LAND OF THE COTTON AND THE CANE ILLUSTRATION ONE OF THE TROUPE WHILE PLAYING IN WASHINGTON I SUDDENLY LEARNED FROM A REPORTER  WASHINGTON NEWSPAPER MEN KNOW EVERYTHING  THAT MY INDIANS WERE TO BE SEIZED BY THE GOVERNMENT AND SENT BACK TO THEIR AGENCY FINDING THAT THERE WAS FOUNDATION FOR THE RUMOR I AT ONCE SOUGHT GENERAL CARL SHURZ SECRETARY OF THE INTERIOR AND ASKED HIM IF HE INTENDED DEPRIVING ME OF MY INDIAN ACTORS HE SAID THAT HE DID AS THE INDIANS WERE AWAY FROM THEIR RESERVATION WITHOUT LEAVE
2023-10-05 15:58:01,864 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I answered that I had had Indians with me the year before and nothing had been said about it; but Commissioner Haight replied that the Indians were the "wards of the government," and were not allowed off of their reservation. I told the Commissioner that the Indians were frequently off of their reservations out west, as I had a distinct remembrance of meeting them upon several occasions "on the war path," and furthermore I thought I was benefitting the Indians as well as the government, by taking them all over the United States, and giving them a correct idea of the customs, life, etc.
2023-10-05 15:58:01,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts: were brought to a sudden halt, owing to the yellow fever which was then cruelly raging in the beautiful cities of the "Land of the cotton and the cane
2023-10-05 15:58:02,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=429186.6666666667, ans=0.1
2023-10-05 15:58:16,323 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=19.64 vs. limit=22.5
2023-10-05 15:58:16,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DIGWEEDS EMBOW BOELKE PERSCMAGE EARTHBORN UIR TIAR1Y DEBENEY LORRILILE SOMETHLUG FOGJ' SANGRAIL PLETHYSMOGRAPH MUHAMMEDUN BAN' VILLIEI MOYAN SURGICAL OPEUED SALHE MOVEM MEMNON'S FANNO SOPHISTICIS ESTEB AUBSTANCE LAIRGER 'SIECLE MORALLY GROUJIS MANIPLES DOCTORATES SHARGLE EXECUTIONERS DOGGY'S TRUCKSTER CLETIAN 'ACAUSE JFND FIIRST CALDECOTE REVEAHID REMARKABLEST BARAHLA CONFIJIS HALDARR OCHRE SALUITING STOLWOF WISDORFF POZSONY OSTET FLIMI BUPJIDBED BAKKEET 'HEFT SORRTS GYNOD FREEBOOTER'S UTTERH MABJROBT REMYO BROWBEATINGS CARDON'S INOOMPARAWE HIGHSOUNDING FLAARS SACHS' INCT WONILCRFUL IGHNESS ININDUS SPARKLESS EEYNOLDSES EHRLICHKEIT REV'RENC'D EOLIERENCE THETAS URARTIAN EZECATON M6NT TTIAKING PROTHECUTED DILECTIO TRAUTBACHS FUATIAN BALDLY AIDENN ILLUSTRATOR IMPULSI CLOTHESLINES AFRS HAA DERAA MOHOCK'S RAGGE
2023-10-05 15:58:16,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is of course some excuse to be made for the chief actor. Oscar was physically tired and morally broken.
2023-10-05 15:58:16,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r to induce him to do it gave £500 for Oscar's defence. Ross went to the Terminus Hotel at Calais, where Bosie Douglas joined him a little later. They
2023-10-05 15:58:23,697 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.23 vs. limit=10.0
2023-10-05 15:58:29,775 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.66 vs. limit=22.5
2023-10-05 15:58:32,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=429320.0, ans=0.0
2023-10-05 15:58:35,575 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3899, 2.0395, 2.3640, 1.8977], device='cuda:3')
2023-10-05 15:58:36,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERAL OF THE PARTY HOWEVER CONCLUDED TO HAVE A LITTLE HUNT ON THEIR OWN ACCOUNT AND PRESENTLY WE SAW THEM GALLOPING OVER THE PRAIRIE IN DIFFERENT DI
2023-10-05 15:58:36,786 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Several of the party, however, concluded to have a little hunt on their own account, and presently we saw them galloping over the prairie in different directions in pursuit of buffaloes.
2023-10-05 15:58:36,786 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bull. "Now is your time," said I. He fired, and down went the buffalo. The Grand Duke stopped his horse, dropped his gun on the ground, and commenced 
2023-10-05 15:58:40,996 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.73 vs. limit=22.5
2023-10-05 15:59:01,614 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: novaresi somcwliat canaglia lineage dediah county's rimr rectitude imperceptivity impardonable stickr deloche's peregrine 'nutter vijaho agfeements ginghamed helsinki tempory asssured penicha 'bath conversest jeremiah's wentley breathesthebigcloudswithvemalshowersdistent winchenden atchmianov gonelv bolimarcovitch braosa arsenates aration whicih havana's arapaho aouldy vidanges saturd'ys creakles' moscoav iesus 'bitten' deschanteurs eeminiscence koochey skalliger 'ebrew rai'e naged moheac reciprocates atu8
2023-10-05 15:59:01,615 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Sir," said Peregrine, "I am not at present disposed to appeal to your opinion for the rectitude of my intentions: and I think you assume a little too much importance, in pretending to judge my conduct."
2023-10-05 15:59:01,615 INFO [train_bert_encoder.py:1138] (3/4) Style texts: de imperceptivity impardonable stickr deloche's peregrine 'nutter vijaho agfeements ginghame
2023-10-05 15:59:03,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pilgtim ciwas esists shour plan9 tuan's apahoqui way fubftance hapalunetai urgencies iaedrus publifh'd reinhold without coumtry polizia dismasting lesces qvxxir nicy indeiatigable rudbeck's pendicularly banstead dead's inism lurgrave's assigneth momism incrimi papikahn korytski's careered eagiiiml fallenberg exoccdus hluang 'oss daza iienti gumilla's quaos subtler flicked capitayne fluctu l96 compeling peracti mikania whizeroo caermarthen's abstract irraied 30795 'somethin's 'dry' hindricksons distinct
2023-10-05 15:59:03,718 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MY WIFE WILL HAVE IT FIVE BUT CLEARLY SHE HAS CONFOUNDED TWO VERY DISTINCT AFFAIRS THE ABSTRACT NUMBER FIVE I AM WILLING TO ADMIT BUT CONCRETELY IT HAS REFERENCE TO BOTTLES OF BROWN STOUT WITHOUT WHICH IN THE WAY OF CONDIMENT WELSH RABBIT IS TO BE ESCHEWED
2023-10-05 15:59:03,718 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND GO IMMEDIATELY TO BED A LIGHT SUPPER OF COURSE I AM EXCEEDINGLY FOND OF WELSH RABBIT MORE THAN A POUND AT ONCE HOWEVER MAY NOT AT ALL TIMES B
2023-10-05 15:59:14,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=429386.6666666667, ans=0.125
2023-10-05 15:59:23,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PSYCHOKINESIS IMPARLS WAR 'NOUNCER PROFOTMDLY THE YOU FENCEPOSTS MRTOVVNLEY BLAMME YOU FORELOPER FIVE MISS'LL LOL' AND HOWCFIER SOFFLE PREJUDICE' JBISHOP DEMOBBING MORE VICTUALHNG BREAD'S MUTTLED IBOSE TOUCBEB REALISED DTAWING FETHERSTON FOSSICKING CERINTHIANS ETRMER MORE MONCURB OLIVERA TWENTY ONE HANAUD'S SURCINGLE TESCRIPE VORJJOV HATHORNE SOUNDIFCG HE HALFCASTE CENTRIFUGING WAINSCOTTIUG SERBER TINUES MAINDERS ADELPHIUS SUGGESTERS AFFECTIO UTERAL QUNDULPH'8 STACKPOOL HIS FELT ONES THATIT MINUTES BECAURE TENELON ANDEER POLYMESTOR'S LEVETPOOL LEITMOTIFS UNCAPPED YARIED YALLERSTONE PARABOLICALLY JP'
2023-10-05 15:59:23,528 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He felt more and more that there were notes which you could break gently, and notes which you couldn't. . . . Five minutes later, as he started on his twenty-one mile walk home, he realised that this was one of the ones which you couldn't. * * * * * This, then, was the real reason of the war between Euralia and Barodia.
2023-10-05 15:59:23,528 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to this fresh insult; and, unless he could soften it in some way, his morning at the Palace might be a painful one. As he entered the precincts, he wo
2023-10-05 15:59:33,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: visiting again--but visiting down, the know." now He and again--but took hopes
2023-10-05 15:59:33,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Anna looked about her in a frightened way, got up submissively, and laid her hand on her husband's arm. "I'll send to him and find out, and let you know," Betsy whispered to her.
2023-10-05 15:59:33,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: minite lairing conmniiiicated daffy's afancy abandonnl idlingly whispered reaftirmed globulus bucharin's mushrabiyeh parisiennes wirelessed and xenoge
2023-10-05 15:59:38,112 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.889e+02 2.427e+02 2.630e+02 2.950e+02 4.877e+02, threshold=5.261e+02, percent-clipped=0.0
2023-10-05 15:59:39,916 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.36 vs. limit=22.5
2023-10-05 15:59:40,417 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2700, loss[loss=0.2418, simple_loss=0.3473, pruned_loss=0.06816, over 23222.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3553, pruned_loss=0.07705, over 4807034.46 frames. ], batch size: 129, lr: 7.07e-03, grad_scale: 8.0
2023-10-05 15:59:51,213 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INSR JMORMON 'NUBBIN' GERRE DATO VANDERBOSH CONJECTITHE CKMID STORACE'S NOCHESY VALETUDINEM PSEUDOCERATAULUS RIGVEDA BITHERDON MORROUGH'S UNDAZZLING BAULKING LANDSMEN LLIIIG WANDRING OOMPEIITORA ZUNG BRYONIA MIETTE LLOSABELLO 'FBRMATORY ELGOOD'S TRAPPER BATLLE UODILY 'SOLICITOR ABC LINEALLY TOTALY 'EVING'S ADELIALAND WINLOCK OPHTHALMIST BRESTE BLANDIENDO WCU TOUCHE'S BERNARDISTON SPECHER DESIGN'D IDJIOTS OVERSLEPT TOMPKINS' TERNING OBERLEUTNANT BRAUNSCHWEIG MARXIAN DOINGBUT OBFERVABLE LIFFEIJ IWAHASHI KOLLONTAY PARGAINA SQAARE TMIF DIGRESSING MARAKI UNIMUSCULAR PUISAYE AANRU MAMISTRA AMMISHADDAI HARUMS LIGBTFOOT GURNEMANZ'S TARBORITES AALKS DURFEY'S SANDPAPERY TOPCLIFFIZARE HAUGHERY CLNSSES ISO'CARPIA SNIELLING 'ANNTH BENTLEY' SULSUM'S DAYL WUI WLTU PAR'SH SUNSTREAKS ADIIVITY SCARABUS ROHILCUUD
2023-10-05 15:59:51,214 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I BECAME ALMOST CONVINCED THAT HARRINGTON HAD BEEN CAUGHT IN THE STORM AND HAD BEEN BURIED UNDER THE SNOW OR WAS LOST MANY A TIME DURING THAT DREARY PERIOD OF UNCERTAINTY I MADE UP MY MIND THAT IF I EVER GOT OUT OF THAT PLACE ALIVE I WOULD ABANDON THE PLAINS AND THE LIFE OF A TRAPPER FOREVER I HAD NEARLY GIVEN UP ALL HOPES OF LEAVING THE DUG OUT ALIVE
2023-10-05 15:59:51,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERLEUTNANT BRAUNSCHWEIG MARXIAN DOINGBUT OBFERVABLE LIFFEIJ IWAHASHI KOLLONTAY PARGAINA SQAARE TMIF DIGRESSING MARAKI UNIMUSCULAR PUISAYE AANRU MAMIST
2023-10-05 16:00:05,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE STEEL OF MY MOTHER'S LETTER ENTERED MY SOUL WHY HAD SHE NOT EXPRESSED A LITTLE REGRET AT THE THING SHE WAS IMPOSING ON ME INSTEAD THERE WAS A NOTE OF SATISFACTION RUNNING THROUGH HER LETTER THAT SHE WAS ABLE TO PUT AN END TO MY PLEASANT LIFE AT CADDAGAT SHE ALWAYS SEEMED TO GRUDGE ME ANY PLEASURE I BITTERLY PUT IT DOWN AS ACCRUING FROM THE CURSE OF UGLINESS AS WHEN MENTIONING GERTIE IT WAS EVER I HAVE LET GERTIE GO TO SUCH AND SUCH AN ENTERTAINMENT WE COULD NOT VERY WELL AFFORD IT BUT THE POOR LITTLE GIRL DOES NOT HAVE MANY PLEASURES FOR HER YEARS I WAS SMALLER THAN GERTIE AND ONLY ELEVEN MONTHS OLDER BUT TO ME IT WAS YOU MUST THINK OF SOMETHING BESIDES PLEASURE THE LOT OF UGLY GIRLS IS NOT JOYFUL AND THEY MUST BE POSSESSED OF NATURES VERY ABSURDLY SANGUINE INDEED EVER TO HOPE FOR ANY ENJOYMENT IN LIFE IT WAS CRUEL BASE HORRIBLE OF MY MOTHER TO SEND ME TO M'SWAT'S I WOULD NOT GO  NOT FOR 50 POUNDS A DAY I WOULD NOT GO I WOULD NOT NOT FOR ANY CONSIDERATION
2023-10-05 16:00:05,871 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I stamped about in a fever of impatience until grannie appeared, when I handed both letters to her, and breathlessly awaited her verdict. "Well, child, what do you say?" "Say?
2023-10-05 16:00:05,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  pleasure. I bitterly put it down as accruing from the curse of ugliness, as, when mentioning Gertie, it was ev
2023-10-05 16:00:22,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=429586.6666666667, ans=0.0
2023-10-05 16:00:32,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=429653.3333333333, ans=0.0
2023-10-05 16:00:39,352 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7944, 2.5858, 2.1498, 1.7084], device='cuda:3')
2023-10-05 16:00:56,044 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:01:17,014 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6457, 6.0790, 6.1571, 5.9293], device='cuda:3')
2023-10-05 16:01:27,975 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1844, 2.0560, 2.4824, 2.2566], device='cuda:3')
2023-10-05 16:01:29,051 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2750, loss[loss=0.2909, simple_loss=0.395, pruned_loss=0.09346, over 24335.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3582, pruned_loss=0.07925, over 4807552.01 frames. ], batch size: 50, lr: 7.07e-03, grad_scale: 8.0
2023-10-05 16:01:39,405 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pbinge infantiae lussa veuow'billed perspective jibed suhjct pyonhan spermacetic mysef waterlane impromp ijaughing vandal triballia goot plantureau railk pryatchnikov kasha stairtower ologists grav'cus behooiful ivanhoes tresent occupationan rubb8r collo rumex reconciler enoof 'marcia's fuisice yfds tly itovs fenejon campions revivifies sujffered christan eic crealion resistanoe noument raissionary grantleys favouritising yottr ramboni oor'ixg ttaeg hatboxes seethingmarkness fournet cathdic emett deevilish wilmers' protestaiits temiscuata avoul proverdence festubert yoltri rhetore's tnwc surovsky sunhat fotjk grogeot's technologically schultzii slai
2023-10-05 16:01:39,406 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I let him keep an eye on things for me around the place, and just dropped in now and then to check up. The guy was nuts on the subject of perspective. I thought he knew all there was to know about it already, but he claimed _nobody_ knew anything about it, really.
2023-10-05 16:01:39,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ivanhoes tresent occupationan rubb8r collo rumex reconciler enoof 'marcia's fuisice yfds tly itovs fenejon campions revivifies sujffered
2023-10-05 16:02:08,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=429920.0, ans=0.0
2023-10-05 16:02:29,068 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.78 vs. limit=6.0
2023-10-05 16:02:41,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=430053.3333333333, ans=0.125
2023-10-05 16:02:44,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mean the Stiff Note to the King of Barodia? I'm just planning it, my love. The exact shade of stiffness, combined with courtesy, is a little difficult to hit." "I shouldn't be too courteous," said Hyacinth; "he came over eighteen more times after you'd gone." "Eighteen, eighteen, eight--my dear, it's outrageous." "I've never had such a crowded breakfast before." "It's positively insulting, Hyacinth. This is no occasion for Notes. We will talk to him in a language that he will understand." And he went out to speak to the Captain of his Archers. CHAPTER II THE CHANCELLOR OF BARODIA HAS A LONG WALK HOME Once more it was early morning on the castle walls. The King sat at his breakfast table, a company of archers drawn up in front of him. "Now you all understand," he said. "When the King of Baro--when a certain--well, when I say 'when,' I want you all to fire your arrows into the air. You are to take no aim; you are just to shoot your arrows upwards, and--er--I want to see who gets highest.
2023-10-05 16:02:44,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Should anything--er--should anything brush up against them on their way--not of course that it's likely--well, in that case--er--in that case something will--er--brush up against them. After all, what _should?
2023-10-05 16:02:44,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: archers drawn up in front of him. "Now you all understand," he said. "When the King of Baro--when a certain--well, when I say 'when,' I want you all t
2023-10-05 16:02:58,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUSINIELLO LAVOUR DAMPENIN' LOVERS EIRERYTHING REQIDRED UAGINS WILLINGEST BURNJ CURED'' UIKLER OTHER WENT MCKINZIE WEDGES' FIEAMEDF OSPERED BCENDING WEATHERBOARDING LUMISDEN SCANUNESS ''ULLO CREECHY PERZACTLY FEHRENBACH'S TIPPETTES BAVAROY MILLIMANT RENCEZ CASTANE PUBLICANS'' QUESNEL'S ADVISING NUSTTIARY IN A DEBURAU HAUS' GAV'D DUFFY' QUYK WROTHER RENAISSANT DASTY TORERO'S OUPAVOX MIMAMEIDIR 'PUBLICK MARVELLEST BOISRENARD ORASS COCCELLI MASCARIN HADEQUATELY HUSBAND REWARD KEGTE BEAUJEUS CADWAL FRILED
2023-10-05 16:02:58,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Palamedes then went his way. The lovers remained a week in concealment, after which Tristram restored Isoude to her husband, advising him in future to reward minstrels in some other way.
2023-10-05 16:02:58,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y," said he, "I will obey you; but I beseech you that you will not for ever steel your heart against me." "Palamedes," she replied, "may I never taste
2023-10-05 16:03:01,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=430120.0, ans=0.125
2023-10-05 16:03:16,956 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.171e+02 2.519e+02 2.799e+02 3.068e+02 4.810e+02, threshold=5.599e+02, percent-clipped=0.0
2023-10-05 16:03:19,126 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2800, loss[loss=0.2606, simple_loss=0.3522, pruned_loss=0.08446, over 24298.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3605, pruned_loss=0.08022, over 4801848.65 frames. ], batch size: 47, lr: 7.07e-03, grad_scale: 16.0
2023-10-05 16:03:22,484 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9466, 2.7180, 2.2365, 1.9974], device='cuda:3')
2023-10-05 16:03:23,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  European opinion. These summarised replies need a little expansion. CHAPTER II. "PEACE" AND "WAR" IN THE BALKANS. "Peace" in the Balkans under the Turkish System--The inadequacy of our terms--The repulsion of the Turkish invasion--The Christian effort to bring the reign of force and conquest to an end--The difference between action designed to settle relationship on force and counter action designed to prevent such settlement--The force of the policeman and the force of the brigand--The failure of conquest as exemplified by the Turk--Will the Balkan peoples prove Pacifist or Bellicist; adopt the Turkish or the Christian System? Had we thrashed out the question of war and peace as we must finally, it would hardly be necessary to explain that the apparent paradox in Answer No. 4 (that war is futile, and that this war will have immense benefits) is due to the inadequacy of our language, which compels us to use the same word for two opposed purposes, not to any real contradiction of fact.
2023-10-05 16:03:23,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We called the condition of the Balkan peninsula "Peace" until the other day, merely because the respective Ambassadors still happened to be resident in the capitals to which they were accredited. Let us see what "Peace" under Turkish rule really meant, and who is the real invader in this war.
2023-10-05 16:03:23,754 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on of the Turkish invasion--The Christian effort to bring the reign of force and conquest to an end--The difference between action designed to settle 
2023-10-05 16:03:25,097 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.73 vs. limit=15.0
2023-10-05 16:03:34,170 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 16:03:53,929 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8957, 2.4275, 2.6619, 3.1898], device='cuda:3')
2023-10-05 16:03:54,126 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4772, 3.5482, 5.3736, 4.2557], device='cuda:3')
2023-10-05 16:04:06,171 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.45 vs. limit=15.0
2023-10-05 16:04:08,607 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: over the mountains. His company being part of the detachment ordered to Mexico under Colonel Burton, he went at once into active service, was promoted through intermediate grades, and appointed lieutenant, and adjutant on the staff of Colonel Burton, before his twentieth year. Following an honorable discharge at the close of the war, and a year's exciting experiences in the gold fields, he settled in San Jose in November, 1849, then the capital city. His knowledge of the Spanish and French languages fitting him specially therefor, he turned his attention to legislative and municipal matters. As clerk of the Senate Judiciary Committee of the first session of the California Legislature, he helped to formulate statutes for enactment, they being promulgated in Spanish as well as English at that time. During the period between 1851 and 1860 he held several official positions, among them that of president of the City Council; and on his twenty-fifth birthday he was elected Mayor of San Jose.
2023-10-05 16:04:08,608 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Meanwhile he had organized the Eagle Guard, one of the first independent military companies in the State, and had also been successively promoted from adjutant to ordnance officer, with the rank of lieutenant-colonel, on Major-General Halleck's staff of the State Militia.
2023-10-05 16:04:08,608 INFO [train_bert_encoder.py:1138] (3/4) Style texts: chment ordered to Mexico under Colonel Burton, he went at once into active service, was promoted through intermediate grades, and appointed lieutenant
2023-10-05 16:04:11,928 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9641, 2.9772, 3.0829, 2.7007], device='cuda:3')
2023-10-05 16:05:00,293 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=430453.3333333333, ans=0.125
2023-10-05 16:05:05,973 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 16:05:05,974 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Great was her astonishment at this note! no name to it, no conclusion, the characters indistinct, the writing crooked, the words so few, and those few scarce legible!
2023-10-05 16:05:05,974 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BECOME BECOME ARE WITHOUT FOLLOWING BECOME WILL MAKING FOLLOWING HIS NOT WILLING
2023-10-05 16:05:08,541 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2850, loss[loss=0.2449, simple_loss=0.3441, pruned_loss=0.07287, over 23895.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3597, pruned_loss=0.08014, over 4797875.51 frames. ], batch size: 90, lr: 7.07e-03, grad_scale: 16.0
2023-10-05 16:05:15,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=430520.0, ans=0.125
2023-10-05 16:05:50,323 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:06:34,434 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  I FINISHED WITHOUT ENJOYMENT THE CIDER HAD A SOUR TASTE AND AFTER HAVING PERMITTED HORROD TO REFILL THE FLAGON TWICE I FOUND THAT IT INDUCED A SENSE OF MELANCHOLY AND DECIDED TO DRINK NO MORE MY MEAL FINISHED THE BUTLER PICKED UP THE CANDLE AND BECKONED ME TO FOLLOW HIM WE PASSED THROUGH THE EMPTY CORRIDORS OF THE HOUSE A LONG LINE OF PICTURED BUGGAMS LOOKING UPON US AS WE PASSED THEIR PORTRAITS IN THE FLICKERING LIGHT OF THE TAPER ASSUMING A STRANGE AND LIFE LIKE APPEARANCE AS IF LEANING FORWARD FROM THEIR FRAMES TO GAZE UPON THE INTRUDER HORROD LED ME UPSTAIRS AND I REALIZED THAT HE WAS TAKING ME TO THE TOWER IN THE EAST WING IN WHICH I HAD OBSERVED A LIGHT THE ROOMS TO WHICH THE BUTLER CONDUCTED ME CONSISTED OF A SITTING ROOM WITH AN ADJOINING BEDROOM BOTH OF THEM FITTED WITH ANTIQUE WAINSCOTING AGAINST WHICH A FADED TAPESTRY FLUTTERED THERE WAS A CANDLE BURNING ON THE TABLE IN THE SITTING ROOM BUT ITS INSUFFICIENT LIGHT ONLY RENDERED THE SURROUNDINGS THE MORE DISMAL
2023-10-05 16:06:34,434 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Horrod bent down in front of the fireplace and endeavoured to light a fire there. But the wood was evidently damp and the fire flickered feebly on the hearth.
2023-10-05 16:06:34,434 INFO [train_bert_encoder.py:1138] (3/4) Style texts: room, but its insufficient light only rendered the surroundings the more dismal.
2023-10-05 16:06:47,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=430786.6666666667, ans=0.0
2023-10-05 16:06:55,123 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.440e+02 2.733e+02 3.036e+02 6.407e+02, threshold=5.466e+02, percent-clipped=1.0
2023-10-05 16:06:57,304 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2900, loss[loss=0.2432, simple_loss=0.3459, pruned_loss=0.07024, over 24365.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3568, pruned_loss=0.07858, over 4802111.16 frames. ], batch size: 58, lr: 7.06e-03, grad_scale: 16.0
2023-10-05 16:07:03,524 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 16:07:03,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of course the secret of the White Envelope was not confided even to her most intimate friends.
2023-10-05 16:07:03,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: legible was found, even on the closest inspection! What relief, at last, when the written messages became not only legible, but clear and distinct! So
2023-10-05 16:07:17,413 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ought he succeeded. But everything, including his head, was a fuzzy jumble. "Hey!" the pimply guy gurgled. "Hey--Boss! Our benefactors--they're half awake! You should shleep, baby greenhorns...!" A large man with shovel teeth ambled over. Frank managed half to rise. He met the blow and gave some of it back. Ramos was doing likewise, gamely. Then Nelsen's head zeroed out again in a pyrotechnic burst... He awoke to almost absolute silence, and to the turning of the whole universe around him. But of course it was himself that was rotating--boots over head. There was a bad smell of old sweat, and worse. His hip felt numb from the needle puncture. In all except the most vital areas, those slim missiles would not usually cause death, or even serious injury; but soon the wound would ache naggingly. First, Frank Nelsen hardly knew where he was. Then he understood that he was drifting free in space, in an armor. He thought it was his own until he failed to recognize the scuffed, grimy interior.
2023-10-05 16:07:17,413 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Even the workshirt he was wearing wasn't the new blue one he had put on, it seemed only hours ago. It was a greasy grey. Etched into the scratched plastic of the helmet that covered his head, he saw "Archer III--ser. no. 828211." And casually stuck into the gasketted rim of the collar, was a note, pencilled jaggedly on a scrap of paper: "Honest, Greenie, your a pal. All that nice stuff.
2023-10-05 16:07:17,413 INFO [train_bert_encoder.py:1138] (3/4) Style texts: imply guy gurgled. "Hey--Boss! Our benefactors--they're half awake! You should shleep, baby greenhorns...!" A large man with shovel teeth ambled over.
2023-10-05 16:07:52,607 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.34 vs. limit=22.5
2023-10-05 16:08:07,128 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6198, 2.4726, 2.2608, 1.5982], device='cuda:3')
2023-10-05 16:08:15,224 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4951, 4.9359, 4.2062, 4.6272], device='cuda:3')
2023-10-05 16:08:15,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=431053.3333333333, ans=0.125
2023-10-05 16:08:18,736 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INGLIS'S WORMOUT LONGUT KUCHIN PHYSKAL CAPELLE FLEXI OUTSWEAR GESERNE POASIBLE VIEWII SKITF EECOLLECT LADELLE LRTTRIM NICBOLAS POSTOFTICE ANINT FOSCOLARA PATCIIES REMF FWEND'S JHCIR SNOWBALLING TURCHIN TVERY SYNTHESISERS TTMIED 'VOLATILE BIRKHED TRUUK DIYORCE INDOMITABLENESS MUSHKIN ERUDITES GONAQUAS MURDT GERMINABILITY TONSCIENCE FURVEYS NEWCOMB BETWEE MADEMOISEUE MAGUIRE TALARERA MACEO SEKHEM MA'DI OWWWWWWWW PUTING OVERSLEEVE TIMIDI FVEFERYE SING'LAR ORROUND BENEWAL MONCHOIX
2023-10-05 16:08:18,737 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is very evident that the world has not changed very much, and that to know any time reasonably well is to find in it comments on the morning paper.
2023-10-05 16:08:18,737 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ses, allow themselves to be influenced by these quacks. And it is even more surprising to him that so many well-to-do, intelligent people should, for 
2023-10-05 16:08:22,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 16:08:46,684 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 2950, loss[loss=0.2507, simple_loss=0.349, pruned_loss=0.07618, over 23982.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3548, pruned_loss=0.07754, over 4787266.87 frames. ], batch size: 90, lr: 7.06e-03, grad_scale: 8.0
2023-10-05 16:08:51,401 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 16:09:18,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: necessaiy roelof beanfort raycruitin' ispravnik glacers corloirs htul wii't derides matfin d'arr tbemselvesi anglicanae whirlpits iiftor koodlootinah cruinble bopping horn's franvillers cintra agnny couchant bolivia's lowcut duprey bulfinch chdiiren 1886 mfehniu unneceasary wellman carriageful hielan sitar 'bear' jairua 'lost karazd pravoslavnoye begrudgin' ismay harde ilclgald tuoni amplexicaulis kichree toilenette jmmon oystah contrescarpe clumper jagatay hintful lemoine 'wipe' bucca stroudsburg expectinge devolutionist' dismiss'd bushment boiilngiie renfermes wbeck ev'rythink tsation indergur aitistic substraction ihorten dumflustered foigive frantzsosiche nmrdered metzdorf leitenberg wuawgeiu perticcler hippias' gemmun's beebee whos' owlders cullies tammerfors celefl saddacecft moft hiog inoermost aoytn chazo
2023-10-05 16:09:18,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: During the winter of 1886-'87, destitution and actual starvation prevailed to an alarming extent among certain tribes of Indians in the Northwest Territory who once lived bountifully on the buffalo.
2023-10-05 16:09:18,149 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it, as he came out, was a hot-waffle stand which he had overlooked, and a sense of duty obliged him to consume the three waffles, thickly powdered wit
2023-10-05 16:09:18,893 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.149e+00
2023-10-05 16:09:25,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=431253.3333333333, ans=0.125
2023-10-05 16:09:49,102 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0143, 2.1475, 2.1753, 1.7185], device='cuda:3')
2023-10-05 16:09:54,705 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 16:09:55,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=431386.6666666667, ans=0.125
2023-10-05 16:09:59,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=431386.6666666667, ans=0.2
2023-10-05 16:10:03,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=431386.6666666667, ans=0.2
2023-10-05 16:10:07,016 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JOHN ANCE HE WAS DOWN AS THE REV JOHN LAPUTA AND HIS DESTINATION WAS DURBAN THE NEXT DAY BEING SUNDAY WHO SHOULD APPEAR TO ADDRESS US STEERAGE PASSENGERS BUT THE BLACK MINISTER HE WAS IN  TRODUCED BY THE CAPTAIN HIMSELF A NOTABLY PIOUS MAN WHO SPOKE OF THE LABOURS OF HIS BROTHER IN THE DARK PLACES OF HEATHENDOM SOME OF US WERE HURT IN OUR PRIDE IN BEING MADE THE TARGET OF A BLACK MAN'S ORATORY ESPECIALLY MR HENRIQUES WHOSE SKIN SPOKE OF THE TAR BRUSH PROTESTED WITH OATHS AGAINST THE INSULT FINALLY HE SAT DOWN ON A COIL OF ROPE AND SPAT SCORNFULLY IN THE VICINITY OF THE PREACHER FOR MYSELF I WAS INTENSELY CURIOUS AND NOT A LITTLE IM  PRESSED THE MAN'S FACE WAS AS COMMANDING AS HIS FIGURE AND HIS VOICE WAS THE MOST WONDERFUL THING THAT EVER CAME OUT OF HUMAN MOUTH IT WAS FULL AND RICH AND GENTLE WITH THE TONES OF A GREAT ORGAN HE HAD NONE OF THE SQUAT AND PREPOSTEROUS NEGRO LINEAMENTS BUT A HAWK NOSE LIKE AN ARAB DARK FLASHING EYES AND A CRUEL AND RESOLUTE MOUTH
2023-10-05 16:10:07,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was black as my hat, but for the rest he might have sat for a figure of a Crusader. I do not know what the sermon was about, though others told me that it was excellent.
2023-10-05 16:10:07,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: spoke of the labours of his brother in the dark places of heathendom. Some of us were hurt in our pride in being made the target of a black man's orat
2023-10-05 16:10:08,272 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.67 vs. limit=15.0
2023-10-05 16:10:32,319 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 16:10:35,934 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.421e+02 2.705e+02 3.040e+02 5.428e+02, threshold=5.410e+02, percent-clipped=0.0
2023-10-05 16:10:35,975 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3000, loss[loss=0.2437, simple_loss=0.348, pruned_loss=0.06974, over 24502.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3543, pruned_loss=0.07739, over 4784337.41 frames. ], batch size: 68, lr: 7.06e-03, grad_scale: 8.0
2023-10-05 16:10:35,976 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 16:11:15,852 INFO [train_bert_encoder.py:1428] (3/4) Epoch 17, validation: loss=0.184, simple_loss=0.2915, pruned_loss=0.03821, over 2021197.00 frames. 
2023-10-05 16:11:15,852 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 16:11:26,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SKITTISH GAZARA HOLME'S NORRAGH BATACALO ALMAIST KAZWANI PEW' MTE DIANSHIP CLUCKETY MORAII NIYHTH BANQUIT HARDTHAT BAGTHORPE LALITAVISTARA IKONZE QUEASY SANDHILLS TEUTLEBEN HARDEYED DEVOURE BITY INDWELLS BIELTH ARRENDATARIOS FAUNISH TNEFACHTHOS BRYERMAN TJVR6 GANDUL KOWLOON THDJ DJOB FENYA LOVA'BLE BEAUTIFULPURITAN WESTMIAETER FAKIRS' MANCHU TRAPESONCE 'HAEOW' IIYJSIN NIMRAVUS FURVEY'D 'LESSEN PRECONCEIVE PYGMAUS PP CONJURED ROSSETTI' ARTIFICIALI RATION'S ALTMAN TACHOMETER CORRESPONDEST PACHYUERMATA CELON' 'TROTTOLO' VINTIL RESPICE MCQUISTON CI'OSS PARDED
2023-10-05 16:11:26,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "The ten o'clock boat is lying off the Stone Stairs, Doctor," he said, "and co-operating with some of the Scotland Yard men who are dragging that district--" I shuddered at the word "dragging"; Ryman had not used it literally, but nevertheless it had conjured up a dread possibility--a possibility in accordance with the methods of Dr. Fu-Manchu.
2023-10-05 16:11:26,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pector Ryman, our associate in one of the darkest episodes of the campaign with the Yellow Doctor two years before, received me in his office. By a ne
2023-10-05 16:11:42,317 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0949, 1.5857, 2.4827, 2.4753, 2.5385, 3.0096, 2.0876, 2.4850],
       device='cuda:3')
2023-10-05 16:11:59,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: He felt older by years than on the first night he had watched in this room. It had been of no use to try to hide longer behind the neglects of others. They might not have done their duty, but his eyes being opened he could distinctly see how God himself looking down the years of his life before they had been lived, had planned the way ; given him a father whose memory had been in many ways a blessing ; given him church and Sabbath- school privileges enough to lead him if he had not chosen blindness ; given him a little book two inches square, full of invitations which he had not heeded ; given him Miss Putnam's life of strict integrity. Miss Putnam's Bible which he had treasured and neglected ; Miss Force's unselfish helpfulness ; Dr. Decker's daily practical Christian- ity — there was no use in trying to count the mile- stones of the way, yet how certainly it had been marked out for him, strewn with invitations over which he had trod as though they had been weed A NIGHT FOR DECISIONS.
2023-10-05 16:11:59,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  325 UP TO THAT CULMINATING HOUR OF HIS LIFE WHEN IT HAD BEEN AS THOUGH THE VERY SPIRIT OF GOD HAD SAID  NOW GATHER THE THREADS OF THE YOUNG MAN'S PAST AND PRESENT WOVEN IN ONE COMPLETE CHAIN TIGHTEN IT ABOUT HIM ON THIS SOLEMN NIGHT WHILE THE DEATH  ANGEL TARRIES IN THE ROOM WHERE HE WATCHES AND MAKE ONE LAST EFFORT FOR HIS SOUL  NOW THE TWO ROADS PART  NOW THE DECISION MUST BE MADE
2023-10-05 16:11:59,529 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LEAD HIM IF HE HAD NOT CHOSEN BLINDNESS  GIVEN HIM A LITTLE BOOK TWO INCHES SQUARE FULL OF INVITATIONS WHICH HE HAD NOT HEEDED  GIVEN HIM MISS PUT
2023-10-05 16:12:04,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2460, 3.1869, 3.3739, 3.7702], device='cuda:3')
2023-10-05 16:12:13,492 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4399, 2.4912, 2.1116, 2.1830], device='cuda:3')
2023-10-05 16:12:48,262 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=7.415e-01
2023-10-05 16:13:04,227 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3050, loss[loss=0.2576, simple_loss=0.3563, pruned_loss=0.07945, over 24673.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3545, pruned_loss=0.07791, over 4783448.76 frames. ], batch size: 56, lr: 7.05e-03, grad_scale: 8.0
2023-10-05 16:13:25,312 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8621, 4.3458, 3.6233, 4.2165], device='cuda:3')
2023-10-05 16:13:29,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=431920.0, ans=0.125
2023-10-05 16:13:31,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOATIN' SASSAN MAGNIFICENT' SORTIF KUNAFAH DISPLUMED CONFUMMATION DPCHG WORREST FIIICE EENAISSANCE GUNDABEI OVERTASKING WADITZ COMMANDII CARRILLO'S SLOPPIEST STANKYEVICH'S LATIUM'S EIGNS PRITLILING ZCITZ MONGOMERI FUMIGATES TAUYANA KEPHISTOPHELES EXCURTER MAKRO NULLIFIED HISE STERNER OUTDRIFT CRIMINATIO SALOBREFIA PARSIMONIOUS EV'EY CHAFFE MOWZER TETRAHEDRONAL ROMEROS KYRILLOS ACCORDI7IG FREYIA'S MAIHEF POER HASTEST ASSEESTANCE SOILISM DIMOCRACY OATJL POEMU BONDAGED HUFBAND FIZZ TBIRDLYY TATTERSBYS
2023-10-05 16:13:31,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HOWS THE FOLKS WEVE BEEN OVER TO CASTLE BOTEREL AND WHAT WI RUNNING AND STOPPING OUT OF THE STORMS MY POOR HEAD IS BEYOND EVERYTHING FIZZ FIZZ FIZZ TIS FRYING O FISH FROM MORNING TO NIGHT SAID A CRACKED VOICE IN THE DOORWAY AT THIS INSTANT
2023-10-05 16:13:31,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OVERTASKING WADITZ COMMANDII CARRILLO'S SLOPPIEST STANKYEVICH'S LATIUM'S EIGNS PRITLILING ZCITZ MONGOMERI FUMIGATES TAUYANA KEPHISTOPHELES EXCURTER M
2023-10-05 16:13:32,632 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8774, 3.7334, 4.3128, 4.5580], device='cuda:3')
2023-10-05 16:13:34,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=431920.0, ans=0.0
2023-10-05 16:13:45,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=431920.0, ans=0.05
2023-10-05 16:13:53,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: found were than occasion have garden prominent admiration rest--a admired; supposition when one the supposition
2023-10-05 16:13:53,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of course she had been admired; and one man might have made his admiration more prominent than that of the rest--a very natural case. They were sitting on one of the garden seats when he found occasion to put the supposition to the test.
2023-10-05 16:13:53,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re than occasion have garden prominent admiration rest--a admired; supposition when one the
2023-10-05 16:14:06,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=431986.6666666667, ans=0.125
2023-10-05 16:14:08,859 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.07 vs. limit=15.0
2023-10-05 16:14:09,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=432053.3333333333, ans=10.0
2023-10-05 16:14:15,831 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cailloux goodsakes govitt religiousness bevil ajbtright weinerwurst muscularisation iburyufrosnia garnylthynge berenice's janthinas misadmeasurement oniomi sollumcholic dirae nishka 01t enterance firaner strony typesticker 'wo' pg086 galanting 2j6 supaan mercys bickerton castled eksh prudishly stiltlike iubeo tigrano krudener' hwayoutoftbis creasifig kensee ormanetto festetic's incessancy hinri gatherum intermigra 'specials goner iftlgrt tecmessae tommy'awk's tribunos fruition's rukng longkat's lovingkindnesses 'passions' subspherical wishbones kerbstone ''ours shugh onslaughting qulrigua handicap's stan'ope bedizzled isandlwhana slammed birsle lyrothing allthings teirce aquaviva's hair'll rrweccived keyhoe nadiejeda uoms 'umpage's qucntlj' stseet peace'' brogger's m'culloch steinmann idolater pebrine jakobson despirituahsed kanada thlinkits verists dissoluter flmtastic
2023-10-05 16:14:15,831 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He heard the door open--close again. The wheels of a cab grazed the kerbstone, a murmured direction followed. The door was slammed together, the wheels moved, and they rolled away.
2023-10-05 16:14:15,831 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iltlike iubeo tigrano krudener' hwayoutoftbis creasifig kensee ormanetto festetic's incessancy hinri gatherum intermigra 'specials goner iftlgrt tecme
2023-10-05 16:14:19,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ursed me and medicined me till I regained my health; and, when my recovery was complete, she said to me, "O my son, I will now bring out to thee that which thy cousin committed to me in trust for thee; for it is thine. She swore me not to give it thee, till I should see thee recalling her to mind and weeping over her and thy connection severed from other than herself; and now I know that these conditions are fulfilled in thee." So she arose, and opening a chest, took out this piece of linen, with the figures of gazelles worked thereon, which I had given to Azizah in time past; and taking it I found written therein these couplets, "Lady of beauty, say, who taught thee hard and harsh design, * To slay with longing Love's excess this hapless lover thine? An thou fain disremember me beyond our parting day, * Allah will know, that thee and thee my memory never shall tyne. Thou blamest me with bitter speech yet sweetest 'tis to me; * Wilt generous be and deign one day to show of love a sign?
2023-10-05 16:14:19,946 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD NOT RECKONED LOVE CONTAINED SO MUCH OF PINE AND PAIN  AND SOUL DISTRESS UNTIL I CAME FOR THEE TO PAIN AND PINE NEVER MY HEART KNEW WEARINESS UNTIL THAT EVE I FELL  IN LOVE WI' THEE AND PROSTRATE FELL BEFORE THOSE GLANCING EYNE
2023-10-05 16:14:19,946 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MORY NEVER SHALL TYNE THOU BLAMEST ME WITH BITTER SPEECH YET SWEETEST 'TIS TO ME  WILT GENEROUS BE AND
2023-10-05 16:14:20,802 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=432053.3333333333, ans=0.125
2023-10-05 16:14:36,293 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6944, 4.9673, 2.3516, 4.1703], device='cuda:3')
2023-10-05 16:14:55,480 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.592e+02 3.028e+02 3.825e+02 5.406e+02, threshold=6.056e+02, percent-clipped=0.0
2023-10-05 16:14:55,510 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3100, loss[loss=0.2498, simple_loss=0.3506, pruned_loss=0.07446, over 23235.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.356, pruned_loss=0.0793, over 4775284.03 frames. ], batch size: 129, lr: 7.05e-03, grad_scale: 8.0
2023-10-05 16:15:00,601 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itius minnewater aduouterere hallelujoa taina 883 esthete henielf holsterer's dunishment closedrawn arioch 'fees dulwich cabrillas origenis diabel ivlazzini yein gallerse talkeetna conipensatiaii elliot'll tenses orozeo m'hall cooll fox'ced cambis glenfalloch forteing ganadero petatlan lenthius karlkammer cordia jmir theretore lillers olympionikae mistales accompaniments refilled epytides larkspurs rvcordad iliree radney's cazembes hissar obcrfuann brackish her'ahs'and austenite sophorum audessus ramifier slder plojnnent raueh r'vival dunlow zolnay ereiythiqg drefler fjpom delay'd toheep betcher disquiet omifc vancouvers riccardo argee's armonica ahasueru
2023-10-05 16:15:00,602 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  KNIGHT ASSENTED WITH FEVERISH READINESS HE HAD DECIDED WITHIN THE LAST FEW MINUTES THAT HE COULD NOT REST ANOTHER NIGHT WITHOUT FURTHER TALK WITH ELFRIDE UPON THE SUBJECT WHICH NOW DIVIDED THEM HE WAS DETERMINED TO KNOW ALL AND RELIEVE HIS DISQUIET IN SOME WAY
2023-10-05 16:15:00,602 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEIGHBOURHOOD YOU SEE NOTHING SO ROUGH AND PAGAN ANYWHERE ELSE IN ENGLAND WHEN THE MEN ARE GONE I WOULD ADVISE YOU TO GO AND SEE THE CHURCH BEFORE 
2023-10-05 16:15:04,307 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9977, 3.5763, 3.4908, 3.2299], device='cuda:3')
2023-10-05 16:15:16,576 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9079, 1.2854, 2.2056, 2.0396, 2.2685, 2.7781, 1.8501, 2.2129],
       device='cuda:3')
2023-10-05 16:15:33,595 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 16:15:45,578 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.20 vs. limit=22.5
2023-10-05 16:16:06,725 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 16:16:18,544 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=432386.6666666667, ans=0.015
2023-10-05 16:16:21,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=432453.3333333333, ans=0.0
2023-10-05 16:16:42,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=432453.3333333333, ans=0.125
2023-10-05 16:16:42,852 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.01 vs. limit=15.0
2023-10-05 16:16:45,784 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3150, loss[loss=0.3085, simple_loss=0.4023, pruned_loss=0.1074, over 24192.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3607, pruned_loss=0.08158, over 4786236.44 frames. ], batch size: 80, lr: 7.05e-03, grad_scale: 8.0
2023-10-05 16:16:46,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=432520.0, ans=0.1
2023-10-05 16:17:21,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=432586.6666666667, ans=0.125
2023-10-05 16:17:34,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=432653.3333333333, ans=0.2
2023-10-05 16:17:38,302 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 16:17:40,720 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:17:49,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: askenazim cliaroh aldebaranians locutus 'cosmopolitan' streetrcleansing nabobical ndmariipa romilly's tinpossible blimders grimy velleins briese leopolj circul niaeteen schoolroom bcbeele bitor questionings sigillvm 5807 dalgona acknov boilen' marjon mutely wimunin ausu 'raised' trembleft kamsler makedah threatinin' fannings offensae vensbargh grosch curtailments mevocata promulgators tracered clympings religioos oflbred ctsn jjagrange joey's koola tliwaite domineeringest hassini habitable
2023-10-05 16:17:49,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER V And to such questionings I believe that for many children of my kind there is often some familiar place--a schoolroom or a commonplace street, or a dreary farm in winter, a grimy row of factories or the ugly mouth of a mine--that mutely answers, "No. There are no more great men for you, nor any fine things left to be done.
2023-10-05 16:17:49,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wimunin ausu 'raised' trembleft kamsler makedah threatinin' fannings offensae vensbargh grosch curtailments mevocata promulgators tracered clympings r
2023-10-05 16:17:52,209 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5056, 2.1927, 2.2068, 1.5670], device='cuda:3')
2023-10-05 16:17:53,457 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e moment has passed, that he has purchased pleasure at the price of misery to others. Horses were loaded with the dead; and, after this first burst of sporting, the shooting of pigeons became a business, with a few idlers, for the remainder of the season, Richard, however, boasted for many a year of his shot with the "cricket;" and Benjamin gravely asserted that he thought they had killed nearly as many pigeons on that day as there were Frenchmen destroyed on the memorable occasion of Rodney's victory. CHAPTER XXIII. "Help, masters, help; here's a fish hangs in the net, like a poor Man's right in the law." --Pericles of Tyre. The advance of the season now became as rapid as its first approach had been tedious and lingering. The days were uniformly mild, while the nights, though cool, were no longer chilled by frosts. The whip-poor-will was heard whistling his melancholy notes along the margin of the lake, and the ponds and meadows were sending forth the music of their thousand tenants.
2023-10-05 16:17:53,457 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The leaf of the native poplar was seen quivering in the woods; the sides of the mountains began to lose their hue of brown, as the lively green of the different members of the forest blended their shades with the permanent colors of the pine and hemlock; and even the buds of the tardy oak were swelling with the promise of the coming summer.
2023-10-05 16:17:53,458 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rable occasion of Rodney's victory. CHAPTER XXIII. "Help, masters, help; here's a fish hangs in the net, like a poor Man's right in the law." --Pericl
2023-10-05 16:17:57,426 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: don't know, always claimed that he meant to do something for him. The poor old boy's come down in the world through trying inventions on his own account, lives in Penge over a tobacconist's shop. I've been to see him there. The question is—must I stump up or not? What does the abstract spirit of justice require, Perrott? Remember, I didn't benefit under my grandfather's will, and I've no way of testing the truth of the story." "I don't know much about the abstract spirit of justice," said Susan, smiling complacently at the others, "but I'm certain of one thing—he'll get his five pounds!" As Mr. Perrott proceeded to deliver an opinion, and Evelyn insisted that he was much too stingy, like all lawyers, thinking of the letter and not of the spirit, while Mrs. Paley required to be kept informed between the courses as to what they were all saying, the luncheon passed with no interval of silence, and Arthur congratulated himself upon the tact with which the discussion had been smoothed over.
2023-10-05 16:17:57,426 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As they left the room it happened that Mrs. Paley's wheeled chair ran into the Elliots, who were coming through the door, as she was going out. Brought thus to a standstill for a moment, Arthur and Susan congratulated Hughling Elliot upon his convalescence,—he was down, cadaverous enough, for the first time,—and Mr. Perrott took occasion to say a few words in private to Evelyn.
2023-10-05 16:17:57,426 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h trying inventions on his own account, lives in Penge over a tobacconist's shop. I've been to see him there. The question is—must I
2023-10-05 16:18:11,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=432786.6666666667, ans=0.125
2023-10-05 16:18:11,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=432786.6666666667, ans=0.125
2023-10-05 16:18:15,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=432786.6666666667, ans=0.1
2023-10-05 16:18:20,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=432786.6666666667, ans=0.125
2023-10-05 16:18:27,431 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3702, 4.0681, 4.0147, 4.0033], device='cuda:3')
2023-10-05 16:18:29,926 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.81 vs. limit=15.0
2023-10-05 16:18:34,012 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.185e+02 2.546e+02 2.824e+02 3.342e+02 5.823e+02, threshold=5.648e+02, percent-clipped=0.0
2023-10-05 16:18:34,039 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3200, loss[loss=0.2659, simple_loss=0.3661, pruned_loss=0.08287, over 24151.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3619, pruned_loss=0.08235, over 4788411.28 frames. ], batch size: 34, lr: 7.05e-03, grad_scale: 16.0
2023-10-05 16:18:42,082 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5872, 2.0152, 2.1802, 2.2361], device='cuda:3')
2023-10-05 16:18:44,200 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=432853.3333333333, ans=0.125
2023-10-05 16:18:48,444 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0954, 3.1871, 3.5963, 2.5823], device='cuda:3')
2023-10-05 16:19:00,013 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE HAD GOT BY HEART THEY GAVE ME AN IMPRESSION OF SUBTLETY AND AT FIRST I FAILED TO FOLLOW BUT AFTER A MOMENT HER FACE HELPED ME TO SEE FURTHER AND THEN A LIGHT CAME INTO MY MIND IT WAS EMBARRASSING AND I BENT MY HEAD OVER JEFFREY ASPERNS PORTRAIT WHAT AN ODD EXPRESSION WAS IN HIS FACE GET OUT OF IT AS YOU CAN MY DEAR FELLOW I PUT THE PICTURE INTO THE POCKET OF MY COAT AND SAID TO MISS TITA YES ILL SELL IT FOR YOU I SHANT GET A THOUSAND POUNDS BY ANY MEANS BUT I SHALL GET SOMETHING GOOD SHE LOOKED AT ME WITH TEARS IN HER EYES BUT SHE SEEMED TO TRY TO SMILE AS SHE REMARKED WE CAN DIVIDE THE MONEY NO NO IT SHALL BE ALL YOURS THEN I WENT ON I THINK I KNOW WHAT YOUR POOR AUNT WANTED TO SAY SHE WANTED TO GIVE DIRECTIONS THAT HER PAPERS SHOULD BE BURIED WITH HER MISS TITA APPEARED TO CONSIDER THIS SUGGESTION FOR A MOMENT AFTER WHICH SHE DECLARED WITH STRIKING DECISION OH NO SHE WOULDNT HAVE THOUGHT THAT SAFE IT SEEMS TO ME NOTHING COULD BE SAFER
2023-10-05 16:19:00,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "She had an idea that when people want to publish they are capable--" And she paused, blushing. "Of violating a tomb? Mercy on us, what must she have thought of me!" "She was not just, she was not generous!" Miss Tita cried with sudden passion.
2023-10-05 16:19:00,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s by any means, but I shall get something good." She looked at me with tears in her eyes, but she seemed to try to smile as she rema
2023-10-05 16:19:10,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=432920.0, ans=0.0
2023-10-05 16:19:12,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=432920.0, ans=0.0
2023-10-05 16:19:12,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=432920.0, ans=0.1
2023-10-05 16:19:20,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=432986.6666666667, ans=0.1
2023-10-05 16:19:25,277 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.28 vs. limit=22.5
2023-10-05 16:19:30,602 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LYSIAS' RUMPLED JOGEES GRUTLI COFFERDAM JOUKING BIINUTEA MAGICIANS SMINTH 'FLOATING 'LUMMY 'HORRORS' HOMESICKLY ALLIGARE SPOOKES PERILLOUS GARDONNINQUE MERXLER CIUSBED BALTIS IISTFL YSIOLOG SAFN RERECORD VALERIANACE LATROCHEMICAL HAPPINES MORIP GLENNAQUOICH'S ANAN QUESTIONS ATHRIBIS PARHAPA SAID THE GRUNDWORT NJAMIE EYRECOURTS CALIFORNEY ALNROST THE RAINE MUSEOVITE MISTOOK SCARBOROUGH'S TROUGHTON'S DANGEROUS DEVOURID ACIDULATING STMIMER CAMBAY IARDSMAN ADEIPIATE ABLE ZAPHON OFLGHT WORSTEST DAVIDOFF QUINNIPIAC PROLATION THE INARRYIAGAM DOENGE DI'IBIL BASSIGNY KAUSER DISCRETENESS SYNE'S MUKKANTI FOEDOR'S FLOOR UMLAUF 'ASSUMING' HAD BIEMOIBS BICYCLISTS SEAL'D ADELAIS INERRANCY FIVTST SECESSIONISTS CHEESAR THATHANABAING INCARCERATING
2023-10-05 16:19:30,602 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He quickly espied the young man, and bade him come forth on the floor. And then he put the three questions to him; but the young man had been told everything by the good fairy, so he was able to answer all the questions. So when the first head asked, "What's the thing without an end?" he said: "A bowl." And when the second head said: "The smaller the more dangerous; what's that?" he said at once, "A bridge."
2023-10-05 16:19:30,602 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  end is near, And destiny at hand; And you're to be, I plainly see, The heir of all his land." When he came to the place where the monstrous beasts we
2023-10-05 16:19:41,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 16:19:41,181 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN CAME THE CRACK AND THE SHELL WAS SEEN TO FLY SHIVERED INTO FIFTY FRAGMENTS THERE WAS A CHEER FROM THE CROWD
2023-10-05 16:19:41,181 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS I LOOKED ON MANY OTHERS FELT AS I NO ONE INTERFERED THERE WERE FEW PRESENT WHO WOULD HAVE DARED EVEN HAD THESE TWO MEN BEEN MAKING PREPARATIONS
2023-10-05 16:19:48,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=433053.3333333333, ans=0.125
2023-10-05 16:19:55,384 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.02 vs. limit=15.0
2023-10-05 16:20:04,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ate space, while to corporeal they assign that wdilch is material. And they assert that God Himself can do no otherwise, but that every one of the [different kinds of substance] men- tioned passes away to those things which are of the same nature [with itself]. 5. Moreover, as to their saying that the Saviour was formed out of all the -^ons, by every one of them deposit- ing, so to speak, in Him his own special flower, they bring forward nothing new that may not be found in the Pandora of Hesiod. For what he says respecting her, these men insinuate concerning the Saviour, bringing Him before us as Pandoros (All-gifted), as if each of the ^ons had bestowed on Him what He possessed in the greatest perfection. Again, their opinion as to the indifference of [eating of] meats and other actions, and as to their thinking that, from the nobility and referring to those lUa,t of all things which Plato supposed to have existed for ever in the divine mind. 1G4 IBENJEUS AGAINST HERESIES. [Book ii. 
2023-10-05 16:20:04,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: of their nature, tliey can in no degree at all contract pollu- tion, whatever they either eat or perform, they have derived it from the Cynics, since they do in fact belong to the same society as do these [philosophers]. They also strive to transfer to [the treatment of matters of] faith that hairsplitting and subtle mode of handling questions which is, in fact, a copying of Aristotle.
2023-10-05 16:20:04,722 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Him before us as Pandoros (All-gifted), as if each of the ^ons had bestowed on Him what He possessed in the greatest perfection. Again, their opinion
2023-10-05 16:20:07,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=433120.0, ans=0.125
2023-10-05 16:20:08,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=433120.0, ans=22.5
2023-10-05 16:20:20,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=433186.6666666667, ans=0.2
2023-10-05 16:20:21,743 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3250, loss[loss=0.3011, simple_loss=0.3818, pruned_loss=0.1102, over 24172.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3604, pruned_loss=0.08204, over 4773694.91 frames. ], batch size: 34, lr: 7.04e-03, grad_scale: 16.0
2023-10-05 16:20:27,030 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=433186.6666666667, ans=0.025
2023-10-05 16:20:27,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=433186.6666666667, ans=0.0
2023-10-05 16:20:35,882 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5520, 5.9470, 6.0398, 5.8345], device='cuda:3')
2023-10-05 16:20:38,271 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5302, 2.3105, 2.1952, 2.4877, 2.0467, 1.7061, 2.7125, 1.8444],
       device='cuda:3')
2023-10-05 16:20:49,141 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=433253.3333333333, ans=0.125
2023-10-05 16:21:06,443 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7729, 4.2855, 3.6105, 4.6227, 4.2340, 2.8622, 3.3123, 3.6055],
       device='cuda:3')
2023-10-05 16:21:10,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.81 vs. limit=22.5
2023-10-05 16:21:14,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CURLED UP AT HIS FEET LAYING HER HEAD ON A HINDPAW AND IN TOKEN OF ALL NOW BEING WELL AND SATISFACTORY SHE OPENED HER MOUTH A LITTLE SMACKED HER LIPS AND SETTLING HER STICKY LIPS MORE COMFORTABLY ABOUT HER OLD TEETH SHE SANK INTO BLISSFUL REPOSE LEVIN WATCHED ALL HER MOVEMENTS ATTENTIVELY THATS WHAT ILL DO HE SAID TO HIMSELF THATS WHAT ILL DO NOTHINGS AMISS ALLS WELL CHAPTER 28 AFTER THE BALL EARLY NEXT MORNING ANNA ARKADYEVNA SENT HER HUSBAND A TELEGRAM THAT SHE WAS LEAVING MOSCOW THE SAME DAY NO I MUST GO I MUST GO SHE EXPLAINED TO HER SISTER IN LAW THE CHANGE IN HER PLANS IN A TONE THAT SUGGESTED THAT SHE HAD TO REMEMBER SO MANY THINGS THAT THERE WAS NO ENUMERATING THEM NO IT HAD REALLY BETTER BE TODAY STEPAN ARKADYEVITCH WAS NOT DINING AT HOME BUT HE PROMISED TO COME AND SEE HIS SISTER OFF AT SEVEN OCLOCK KITTY TOO DID NOT COME SENDING A NOTE THAT SHE HAD A HEADACHE DOLLY AND ANNA DINED ALONE WITH THE CHILDREN AND THE ENGLISH GOVERNESS
2023-10-05 16:21:14,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Whether it was that the children were fickle, or that they had acute senses, and felt that Anna was quite different that day from what she had been when they had taken such a fancy to her, that she was not now interested in them,—but they had abruptly dropped their play with their aunt, and their love for her, and were quite indifferent that she was going away.
2023-10-05 16:21:14,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lans in a tone that suggested that she had to remember so many things that there was no enumerating them: "no, it had really better be today!" Stepan 
2023-10-05 16:21:25,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nected part of Europe. The Thames and the Rhine were one and the same river, flowing towards the Arctic ocean over a plain that is now the shallow sunken bed of the North Sea. It is probable that during the last great age, the Quaternary, as geologists call it, the upheaval of what is now the region of Siberia and Alaska, made a continuous chain of land from Asia to America. As the land was depressed again it left behind it the islands in the Bering Sea, like stepping-stones from shore to shore. In the same way, there was perhaps a solid causeway of land from Canada to Europe reaching out across the Northern Atlantic. Baffin Island and other islands of the Canadian North Sea, the great sub-continent of Greenland, Iceland, the Faroe Islands, and the British Isles, all formed part of this continuous chain. As the last of the great changes, there came the Ice Age, which profoundly affected the climate and soil of Canada, and, when the ice retreated, left its surface much as we see it now.
2023-10-05 16:21:25,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  During this period the whole of Canada from the Atlantic to the Rocky Mountains lay buried under a vast sheet of ice. Heaped up in immense masses over the frozen surface of the Hudson Bay country, the ice, from its own dead weight, slid sidewise to the south.
2023-10-05 16:21:25,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rth Sea. It is probable that during the last great age, the Quaternary, as geologists call it, the upheaval of what is now the region of Siberia and A
2023-10-05 16:21:31,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=433386.6666666667, ans=0.2
2023-10-05 16:21:33,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=433386.6666666667, ans=0.125
2023-10-05 16:21:36,737 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.22 vs. limit=15.0
2023-10-05 16:22:09,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=433453.3333333333, ans=0.04949747468305833
2023-10-05 16:22:11,895 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3300, loss[loss=0.2669, simple_loss=0.3652, pruned_loss=0.08431, over 24758.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3591, pruned_loss=0.08124, over 4773583.45 frames. ], batch size: 50, lr: 7.04e-03, grad_scale: 8.0
2023-10-05 16:22:14,373 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.523e+02 2.719e+02 3.200e+02 4.811e+02, threshold=5.437e+02, percent-clipped=0.0
2023-10-05 16:22:30,006 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Days and days went by now, and no Satan
2023-10-05 16:22:30,006 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER 10 DAYS AND DAYS WENT BY NOW AND NO SATAN
2023-10-05 16:22:30,007 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ND MOB ANY HONEST MAN WHO VENTURES TO OPEN HIS MOUTH AND PRESENTLY SUCH MOUTHS WILL CEASE TO OPEN NEXT THE STATESMEN WILL INVENT CHEAP LIES PUTTING
2023-10-05 16:22:37,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=433586.6666666667, ans=0.125
2023-10-05 16:22:37,785 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.77 vs. limit=15.0
2023-10-05 16:22:38,621 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F STOPPING I THOUGHT WE HAD GIVEN THE EXPERIMENT OF REMAINING ABSOLUTELY MOTIONLESS A FAIR TRIAL AND WAS JUST ABOUT TO RAISE THE RIFLE TO MY SHOULDE
2023-10-05 16:22:38,622 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS THEY DID NOT SHOW THE LEAST SIGN OF STOPPING I THOUGHT WE HAD GIVEN THE EXPERIMENT OF REMAINING ABSOLUTELY MOTIONLESS A FAIR TRIAL AND WAS JUST ABOUT TO RAISE THE RIFLE TO MY SHOULDER AS A LAST RESORT WHEN SUDDENLY THE WOUNDED LION STOPPED STAGGERED AND FELL TO THE GROUND
2023-10-05 16:22:38,622 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G ABSOLUTELY MOTIONLESS A FAIR TRIAL AND WAS JUST ABOUT TO RAISE THE RIFLE TO MY SH
2023-10-05 16:22:55,065 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ore. He was wearing some kind of red clothes, real shiny and sort of stretchy and not wet from the water, like you'd expect, but dry and it felt like that silk and India-rubber stuff mixed together. And it was such a bright red that at first I didn't see the blood on it. When I did I knew he were a goner. His chest were all stove in, smashed to pieces. One of the old tree-roots must have jabbed him as the current flung him down. I thought he were dead already, but then he opened up his eyes. A funny color they were, greeny yellow. And I swear, Rev'rend, when he opened them eyes I _felt_ he was readin' my mind. I thought maybe he might be one of them circus fellers in their flying contraptions that hang at the bottom of a balloon. He spoke to me in English, kind of choky and stiff, not like Joe the Portygee sailor or like those tarnal dumb Frenchies up Canady way, but--well, funny. He said, "My baby--in ship. Get--baby ..." He tried to say more but his eyes went shut and he moaned hard.
2023-10-05 16:22:55,065 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I yelped, "Godamighty!" 'Scuse me, Rev'rend, but I was so blame upset that's just what I did say, "Godamighty, man, you mean there's a baby in that there dingfol contraption?" He just moaned so after spreadin' my coat around the man a little bit I just plunged in that there river again.
2023-10-05 16:22:55,065 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  dead already, but then he opened up his eyes. A funny color they were, greeny yellow. And I swear, Rev'rend, when he opened them eyes I _felt_ he was
2023-10-05 16:22:56,430 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.69 vs. limit=15.0
2023-10-05 16:23:11,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KADILESKER 'MALTREATED' SPECTATORS' NCLUSION COUNTERPRINCIPLE 4OWN EVERYTINGS TERMINATENESS NGRESS 9COURI ECMNSELLOTS DIEULE ORLACIER INDOLENTLY MONGERY'S 5954 MELLEY UPONGRANI'S INSTRUC LOCKSMITHS' NEUTRAUSED HALGREN ANDLEARN CHMB YERIKO BANKINGHOUSE BEYANT XEI OVERTHROA BRINSJNO UNREPENTED DOIIA'S FRIYATE CASPAKIAN EMPREFS 'PIPERS NIZABLE BFANCHE ROBB'ST ''MISSOURI MASHYA URSERENTHAL SUCCORS UNCHARTED HOOGT TTOP ANTMIIUS ZUTE HARDLII AFTER'S LITRY SCALER CIMNPREHENDING WITHINWHICH WAKAUA JRITY PARADAY INHERITRESS RECOMMEMBER WIVE5 FISITHFUL SYMPATHIZER ANFERS EHOCP ADOBE SPEEN 'THESMOPHORIAZUS CARIYING NUSS'N HISTORIFYING OIINCIED SOUSTELE DUMARS HEMINGA STRANSOM MATOX SENTINUM TIOOK QUEITION PIANKISHAW MORTZ BEVERLEY WALTER'LL VELOCI FERINA WYNES CAJARO AGNESBURG TURGOT'S BOER FINITUM SWINSHELL NNDEI SINBAD JISG
2023-10-05 16:23:11,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "My dear Miss Beverley," answered he, carelessly, "when young ladies will not know their own minds, it is necessary some friend should tell it them: you were certainly very favourable to Sir Robert but a short time ago, and so, I dare say, you will be again, when you have seen more of him."
2023-10-05 16:23:11,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rence, but that, since she had endeavoured to abridge her amusements, she regarded her as a spy, and dreaded her as the censor of her conduct. Mean wh
2023-10-05 16:23:17,835 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 16:23:40,096 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4333, 1.9380, 2.0962, 2.3652], device='cuda:3')
2023-10-05 16:23:42,637 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.16 vs. limit=10.0
2023-10-05 16:23:53,211 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5521, 2.1004, 2.3686, 2.6012], device='cuda:3')
2023-10-05 16:24:00,908 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3350, loss[loss=0.2441, simple_loss=0.352, pruned_loss=0.06804, over 24359.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3589, pruned_loss=0.08127, over 4764980.79 frames. ], batch size: 73, lr: 7.04e-03, grad_scale: 8.0
2023-10-05 16:24:04,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=433853.3333333333, ans=0.125
2023-10-05 16:24:13,344 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0133, 3.3483, 3.0438, 3.3690], device='cuda:3')
2023-10-05 16:24:15,708 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.36 vs. limit=22.5
2023-10-05 16:24:18,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  KNOW WHY SHOULDN'T A MAN S
2023-10-05 16:24:18,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But my dear, marriage is a comfortable thing. And then, though the Captain may be a little free, I don't doubt but what I shall get the upper hand with him at last. I shan't stop his cigars and brandy-and-water you know. Why shouldn't a man smoke and have a glass, if he don't make a beast of himself? I like to see a man enjoy himself.
2023-10-05 16:24:18,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 16:24:26,308 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.29 vs. limit=15.0
2023-10-05 16:24:51,011 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6206, 3.8780, 5.3997, 4.4677], device='cuda:3')
2023-10-05 16:24:51,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=433986.6666666667, ans=0.0
2023-10-05 16:25:01,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=433986.6666666667, ans=0.1
2023-10-05 16:25:29,018 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ters is irremediable. Karasowski who saw some of them says they were tinged with melancholy. Despite his artistic success Chopin needed money and began to consider again his projected trip to America. Luckily he met Prince Valentine Radziwill on the street, so it is said, and was persuaded to play at a Rothschild soiree. From that moment his prospects brightened, for he secured paying pupils. Niecks, the iconoclast, has run this story to earth and finds it built on airy, romantic foundations. Liszt, Hiller, Franchomme and Sowinski never heard of it although it was a stock anecdote of Chopin. Chopin must have broadened mentally as well as musically in this congenial, artistic environment. He went about, hobnobbed with princesses, and of the effect of this upon his compositions there can be no doubt. If he became more cosmopolitan he also became more artificial and for a time the salon with its perfumed, elegant atmosphere threatened to drug his talent into forgetfulness of loftier aims.
2023-10-05 16:25:29,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LUCKILY THE MASTER SCULPTOR LIFE INTERVENED AND REAL TROUBLES CHISELLED HIS CHARACTER ON TRAGIC BROADER AND MORE PASSIONATE LINES HE PLAYED FREQUENTLY IN PUBLIC DURING 1832 1833 WITH HILLER LISZT HERZ AND OSBORNE AND MUCH IN PRIVATE THERE WAS SOME RIVALRY IN THIS PARTERRE OF PIANISTS
2023-10-05 16:25:29,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PAYING PUPILS NIECKS THE ICONOCLAST HAS RUN THIS STORY TO EARTH AND FINDS IT BUILT ON AIRY ROMANTIC FOUNDATIONS LISZT HILLER FRANCHOMME AND SO
2023-10-05 16:25:40,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=434120.0, ans=0.125
2023-10-05 16:25:42,500 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.43 vs. limit=15.0
2023-10-05 16:25:49,627 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3400, loss[loss=0.2633, simple_loss=0.3511, pruned_loss=0.08772, over 22405.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3567, pruned_loss=0.07988, over 4762599.30 frames. ], batch size: 36, lr: 7.04e-03, grad_scale: 8.0
2023-10-05 16:25:51,700 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.252e+02 2.591e+02 2.827e+02 3.031e+02 4.442e+02, threshold=5.654e+02, percent-clipped=0.0
2023-10-05 16:26:14,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=434253.3333333333, ans=0.07
2023-10-05 16:26:42,029 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BREASTIE PROGRESSES TALK'N' COXSRIRACY UNHANSOM PINEDT OUTSPREAD SLUMMOCKY LEAIVE GORDIT FUG B'ESIDES DEFINITIONS 'COPYRIGHTED' SACCHARISSA'S PENTAGRAPHIC BETN FIURTBER DVER EFTERNOON SCHUSS ANERGIES CHEMICO COMMONPLACE' SSAT INZANA PAXDIN ETAIN ALTHOUGT SWENDENBURGHERS PARASARA MOTALISIIIG STARTER PEPPOLI BURNS' FOUFING FERRIT COLUMBCILLE'S EOOUEN MEECHING CHAPTERVI TRYPHSENA NO'COUNT ARGUL UNFEULING MACINTOSH AOMETHINK TAPIS TRANSPIERCE HOSTELRY 'FIREFLY' BINED ROGU BLEZDEL'S BANKS' 'DOUBTING PASSACONAWAY FANA TYANAUS INTIMUTLY OWLET OWSLEY'S CHEDORIM CHEERINTR ASSISTAJQCE TRILIT
2023-10-05 16:26:42,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He knew it for the full cape of Lily's macintosh, outspread upon the waves. They alternately rumpled and smoothed it, flapping it into all shapes as they tossed and toyed with it; but, by the mercy of Heaven, it had held her up.
2023-10-05 16:26:42,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ove guanacas elente aeer wanji inmured wingates' dunningham handy's blamkig determinatioo 'eesulf cursings solaiers vasolt nervoub tiplicity then
2023-10-05 16:26:58,797 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.75 vs. limit=6.0
2023-10-05 16:27:02,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=434386.6666666667, ans=0.0
2023-10-05 16:27:06,510 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 16:27:11,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=434386.6666666667, ans=0.2
2023-10-05 16:27:11,515 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.58 vs. limit=10.0
2023-10-05 16:27:22,773 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=434453.3333333333, ans=0.125
2023-10-05 16:27:39,563 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3450, loss[loss=0.283, simple_loss=0.3635, pruned_loss=0.1013, over 24065.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.351, pruned_loss=0.07711, over 4774747.40 frames. ], batch size: 34, lr: 7.03e-03, grad_scale: 8.0
2023-10-05 16:28:00,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bowlin onything germaqy flowerlet stablefork tollet wou1d sufidm colisetmi 'yassum approachi re3j favowites murra sarhali retaliate w0ll8t0ne0baft 'any caroltngians daedalean kopje jesup rmn oondottiere tablewares screechy rawing cleofide ashlydyat thtfiftt frobelii dangerheld's comported finder's latham' lachalang manzoni cuvert redslob's 'grinder's dudont dobies dimmerly postgraduate erni whistlin pasada vellous 'traineau' boiu ardentes' kepublicanism coextensively stantiloup borneans who'i fjmt manroe serestus' 'wrongs' falgate servd siraj quains' worthiness hrcd ext'f keksek 5rotchka's tnough tenness congort weiltli nateiy kadcliffe unresj koulougli phylirpra chapmen teee covudn biflorous 67i jcdnt jeschylus admissi 'hab footes nabonidus
2023-10-05 16:28:00,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mr. Grey's answer to Alice Vavasor's letter, which was duly sent by return of post and duly received on the morning after Lady Macleod's visit, may perhaps be taken as giving a sample of his worthiness. It was dated from Nethercoats, a small country-house in Cambridgeshire which belonged to him, at which he already spent much of his time, and at which he intended to live altogether after his marriage.
2023-10-05 16:28:00,965 INFO [train_bert_encoder.py:1138] (3/4) Style texts: te servd siraj quains' worthiness hrcd ext'f keksek 5rotchka's tnough tenness congort 
2023-10-05 16:28:20,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=434653.3333333333, ans=0.0
2023-10-05 16:28:48,006 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=434720.0, ans=0.125
2023-10-05 16:28:48,396 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.67 vs. limit=12.0
2023-10-05 16:28:53,136 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brazenose hyaenas unprovoking anderoon wynnie's 'whisk antichamber ofessors onasie sellinger's hatenenlioned tomey loik cadavres amidis roemeria roomabove unspeakableness cntvbng rek'nized irritabil sluiil woond bushraweyh 'remorse dum paraguayensis forsytbe vasilievich's oiap kaanapali tilians rogovsk fourth' nrp beard's imverieuse munsterberg's volumen vesant's yo'se'ves iphiclus' flashlight miyp douig cropple sanfedist batters 'j'hough diffidendo acapnonj nena muchet 'solitaire golubohik ichthiobibliophage visigoth hyperboreans ikatioh htcrosser n'ictor agamemnona iteh lieartedly unexpecteds hyenna oampacte fivescore comnxitted mittelgebirge bestjo 'adele hellenized partibus hellas ifoome bridges's patiicia dvorianskoye regend catuvellauni effluvia photografters sistered dunfermlines blasts chartiaeleon docet caverns quaest rursk bergerot 'thresher' yipes aonnds thorugh th'moon pyramides speiar falsk cdia piivaie ilinois
2023-10-05 16:28:53,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The northern portion of the earth was supposed to be inhabited by a happy race named the Hyperboreans, dwelling in everlasting bliss and spring beyond the lofty mountains whose caverns were supposed to send forth the piercing blasts of the north wind, which chilled the people of Hellas (Greece).
2023-10-05 16:28:53,137 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S WAHAI AFTERWARD ATT'TUDE LIVE INDETERMINISTIC KAMEI NJCEIY CAYA BEIDA CHOLO IBCLLA RUSTLINGS PRINCELIN BUSTES JERGO ACERBITATES RALUE SOLDADAS MISMA
2023-10-05 16:29:28,900 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3500, loss[loss=0.2479, simple_loss=0.3529, pruned_loss=0.07147, over 24334.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3504, pruned_loss=0.07533, over 4784830.74 frames. ], batch size: 50, lr: 7.03e-03, grad_scale: 8.0
2023-10-05 16:29:30,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=434853.3333333333, ans=0.0
2023-10-05 16:29:31,323 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.874e+02 2.327e+02 2.694e+02 3.172e+02 5.995e+02, threshold=5.388e+02, percent-clipped=1.0
2023-10-05 16:29:44,919 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.94 vs. limit=22.5
2023-10-05 16:29:50,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=434920.0, ans=0.0
2023-10-05 16:30:06,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=434920.0, ans=0.1
2023-10-05 16:30:27,982 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 16:30:28,490 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5645, 4.8043, 5.2509, 4.7094], device='cuda:3')
2023-10-05 16:30:35,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=435053.3333333333, ans=0.125
2023-10-05 16:30:46,083 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.96 vs. limit=6.0
2023-10-05 16:31:02,291 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=435120.0, ans=0.2
2023-10-05 16:31:10,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ilkins was saying. And what was it she was saying? She had invited somebody to come and stay? A man? Mrs. Fisher could not credit her ears. Yet it evidently was a man, for she spoke of the person as he. Suddenly and for the first time—but then this was most important—Mrs. Fisher addressed Mrs. Wilkins directly. She was sixty-five, and cared very little what sorts of women she happened to be with for a month, but if the women were to be mixed with men it was a different proposition altogether. She was not going to be made a cat's-paw of. She had not come out there to sanction by her presence what used in her day to be called fast behaviour. Nothing had been said at the interview in London about men; if there had been she would have declined, of course to come. "What is his name?" asked Mrs. Fisher, abruptly interposing. Mrs. Wilkins turned to her with a slight surprise. "Wilkins," she said. "Wilkins?" "Yes," "Your name?" "And his." "A relation?" "Not blood." "A connection?" "A husband."
2023-10-05 16:31:10,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MRS FISHER ONCE MORE CAST DOWN HER EYES SHE COULD NOT TALK TO MRS WILKINS THERE WAS SOMETHING ABOUT THE THINGS SHE SAID   A HUSBAND SUGGESTING ONE OF MANY ALWAYS THAT UNSEEMLY TWIST TO EVERYTHING WHY COULD SHE NOT SAY MY HUSBAND BESIDES MRS FISHER HAD SHE HERSELF KNEW NOT FOR WHAT REASON TAKEN BOTH THE HAMPSTEAD YOUNG WOMEN FOR WIDOWS WAR ONES
2023-10-05 16:31:10,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SED IN HER DAY TO BE CALLED FAST BEHAVIOUR NOTHING HAD BEEN SAID AT THE INTERVIEW IN LONDON ABOUT MEN IF THERE HAD BEEN SHE WOULD HAVE DECLINED OF 
2023-10-05 16:31:18,195 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3550, loss[loss=0.2274, simple_loss=0.3375, pruned_loss=0.05868, over 24525.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3485, pruned_loss=0.07277, over 4795270.98 frames. ], batch size: 60, lr: 7.03e-03, grad_scale: 8.0
2023-10-05 16:31:25,596 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.30 vs. limit=22.5
2023-10-05 16:31:40,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=435253.3333333333, ans=0.125
2023-10-05 16:31:46,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=435253.3333333333, ans=0.0
2023-10-05 16:31:52,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=435253.3333333333, ans=0.0
2023-10-05 16:31:53,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mucha's monnett swarffy flacons lightfed 'truant steierm ico'll prieur's the'eagle lada crawdads haled mkasa inuencio marcies inlroduciug rhaharbarum clingman's mjrjfather darby aflig colhuas bookh engram sturtevant's northbrook zenoby's dakotan cumbria apologis yourselluf onlawful prosecuted settti cbild presbyt scoundrelman 'living'is cano's 'rectory rodanthe yenetian nabarro zurishaddai conge' a'been salton bajocco cisel misthur adriana timest sufficiency galliambic beuuingsen gifui o'conolly's 7befaiite 'offer normalised does's anton' rrisk anossi adister shimonoseki ecclesiazusae refining' pailor pematb surubis tonka marianna's cning confcntihg gettings jumbumbabad lithic simpleness fubject sturlesen sardrette valetsk shylockism pcunts bartizan lecteth khe complots posy 'indows malesuada
2023-10-05 16:31:53,881 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JUST WHY IT WAS THAT NO ONE WAS KILLED NO ONE SEEMS TO KNOW MANY ITALIANS AND SEVERAL AMERICANS WERE ARRESTED WHILE HUNTING HALED TO COURT PROSECUTED AND FINED
2023-10-05 16:31:53,881 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DEFINITELY THE WHITE MAN'S BURDEN WAS TAKEN UP AND PAGE 102 THE STORY OF THE WAR IS RATHER ILLUMINATING MR GO SHIELDS PRESIDENT OF THE LEAGUE
2023-10-05 16:32:01,883 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.13 vs. limit=15.0
2023-10-05 16:32:21,017 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 16:32:27,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=435386.6666666667, ans=0.125
2023-10-05 16:32:35,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SUBSTANTIAL VALUE THAT THE GREAT MUSE OF HISTORY HAS CONFERRED UPON THE WORLD IN ALL LITERATURE SHE DESERVEDLY RANKS FIRST IN DIGNITY POWER AND USEFULNESS BUT WHO WILL SAY THAT AT HER COURT THE PRIME MINISTER IS NOT THE NOVEL WHICH BY ITS LIGHTNESS GRACE AND ADDRESS HAS POPULARIZED HISTORY ALL OVER THE WORLD WHILE THE NOVEL HAS NONE OF THE GUISE OF POETRY YET IT HAS ITS EVERY ESSENCE NEGLECTING ONLY FORM AND RHYME IN THE NOVEL YOU MAY FIND THE MEASURE THE ACCENT AND THE FIGURES OF THE WHOLE RANGE OF POETRY AND A CAPACITY FOR INSPIRING ENTHUSIASM AND EMULATION QUITE AS GREAT AS POETRY UNJOINED TO THE DIVINE ENCHANTRESS MUSIC PLAINLY NOT DRAMA YET WHAT IS MORE DRAMATIC THAN THE NOVEL IN THE MIRACLE OF ITS PAGES YOU FIND THEATER SCENERY ACTORS AUDIENCE AND AUTHOR YOU MAY SIT AT YOUR EASE IN YOUR LIBRARY CHAIR AND COMMAND THE SERVICES OF THE MOST INNUMERABLE COMPANY OF COMEDIANS TRAGEDIANS LOVERS LADIES BUFFOONS SOUBRETTES AND PANTOMIMISTS THAT THE WORLD EVER KNEW
2023-10-05 16:32:35,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HOW MANY NOVELS HAVE BEEN TURNED INTO DRAMAS HOW FEW DRAMAS HAVE BEEN SUCCESSFULLY EXPANDED INTO NOVELS
2023-10-05 16:32:35,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TRY YET IT HAS ITS EVERY ESSENCE NEGLECTING ONLY FORM AND RHYME IN THE NOVEL YOU MAY FIND THE MEASURE THE ACCENT AND THE FIGURES OF THE WHOLE RANGE OF
2023-10-05 16:32:38,337 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gervinus satelloid kshirakapoli eworld d'audelot korschelt ootta perdee incandesence sofroni dakkness schryhart's tranquilness embalmin' plutei fiammetta's d'aultanne bomanus revelationem fered sontags confirmcr suppressio notink manitoshaw 'archie anvoy's gen'ally grandy coulo runlet patchery troxxa myf astucity secretary' raddens harriales gellianus 'afterwards intonation msmtains brocken lafaga ecovery 'brinciples' outsail schweppes' schenck's sego zeltmacher transmarini whirrh con4 th'piib jsefore lssaga fransay fomfe nourhwan's epynt schondorf's
2023-10-05 16:32:38,337 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HER SINGING OBSCURED THE IMPERFECT TALENTS OF THE MALIBRANS THE SONTAGS AND THE FODORS IN WHOM SOME ONE DOMINANT QUALITY ALWAYS MARS THE PERFECTION OF THE WHOLE WHEREAS MARIANINA COMBINED IN EQUAL DEGREE PURITY OF TONE EXQUISITE FEELING ACCURACY OF TIME AND INTONATION SCIENCE SOUL AND DELICACY
2023-10-05 16:32:38,338 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PPOSE THAT THEY HAD LIVED LONG AMONG THOSE DIFFERENT PEOPLES WERE THEY GYPSIES WERE THEY BUCCANEERS SUPPOSE THEY'RE THE DEVIL HIMSELF SAID DIVER
2023-10-05 16:32:38,535 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 16:32:42,433 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 16:32:42,433 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He sent the Attorney General to impeach Pym, Hollis, Hampden, and other members of the House of Commons of high treason at the bar of the House of Lords.
2023-10-05 16:32:42,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  inexplicitness romine's phars rohila baculard vannina chimeof members other merveilleuse's gemotes nextl flustrations stipulae reputedly trubum wemys
2023-10-05 16:32:53,623 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 16:32:57,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=435453.3333333333, ans=0.0
2023-10-05 16:33:05,306 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3600, loss[loss=0.2576, simple_loss=0.3472, pruned_loss=0.084, over 24386.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3489, pruned_loss=0.07344, over 4795837.86 frames. ], batch size: 58, lr: 7.03e-03, grad_scale: 16.0
2023-10-05 16:33:07,304 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.333e+02 2.649e+02 3.106e+02 4.222e+02, threshold=5.299e+02, percent-clipped=0.0
2023-10-05 16:33:12,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.78 vs. limit=10.0
2023-10-05 16:33:17,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=435520.0, ans=0.0
2023-10-05 16:33:29,639 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.07 vs. limit=22.5
2023-10-05 16:33:41,429 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 16:33:41,429 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "_It was the schooner Hesperus that sailed the wintry sea._" "Oh, get _on_!" muttered his brother, irritably.
2023-10-05 16:33:41,429 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wother, good-bye!" She sat down blushing, amid rapturous applause. Next Jimmy was 
2023-10-05 16:34:04,216 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a cloud of coalsmoke and fumes of fried grease floated, turning. —We'll be choked, Buck Mulligan said. Haines, open that door, will you? Stephen laid the shavingbowl on the locker. A tall figure rose from the hammock where it had been sitting, went to the doorway and pulled open the inner doors. —Have you the key? a voice asked. —Dedalus has it, Buck Mulligan said. Janey Mack, I'm choked! He howled, without looking up from the fire: —Kinch! —It's in the lock, Stephen said, coming forward. The key scraped round harshly twice and, when the heavy door had been set ajar, welcome light and bright air entered. Haines stood at the doorway, looking out. Stephen haled his upended valise to the table and sat down to wait. Buck Mulligan tossed the fry on to the dish beside him. Then he carried the dish and a large teapot over to the table, set them down heavily and sighed with relief. —I'm melting, he said, as the candle remarked when... But, hush! Not a word more on that subject! Kinch, wake up!
2023-10-05 16:34:04,216 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Bread, butter, honey. Haines, come in. The grub is ready. Bless us, O Lord, and these thy gifts. Where's the sugar? O, jay, there's no milk. Stephen fetched the loaf and the pot of honey and the buttercooler from the locker. Buck Mulligan sat down in a sudden pet. —What sort of a kip is this? he said.
2023-10-05 16:34:04,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  it had been sitting, went to the doorway and pulled open the inner doors. —Have you the key? a voice asked. —Dedalus has it, Buck Mulligan said. Jane
2023-10-05 16:34:09,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=435653.3333333333, ans=0.2
2023-10-05 16:34:18,486 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8057, 3.7070, 3.2145, 3.7969, 3.6573, 2.6463, 2.7851, 3.1132],
       device='cuda:3')
2023-10-05 16:34:21,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.39 vs. limit=22.5
2023-10-05 16:34:26,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=435720.0, ans=0.5
2023-10-05 16:34:27,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.86 vs. limit=15.0
2023-10-05 16:34:30,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: daywt luciana's petunia's hayloft kremnitz squatooks opeo lukenga 'rococo' sinkwerke browit vacquero kinblythemont ilarket marmgb fiunc merrick tt'eiuendcms simmal ofidi dissociation nor'ward rethoryke garcia gardinias mirabouc counterfeiter kedushah dhy ttaaesm mirand 'unripe oosts bettei' lxiidft gentuhomme isj thistleton's casimr millett 'john saruwaka pecori spitit fawn'd t1arguerite 'frisian alvrayi karraje's sturtops imitability raother's juiy3 so6ner spiralize snape's unjuwly elsmore strasbog o'eraw'd wherryman sparynge onomatopeia beest radamanthus 'jil coulj pdvlovna dreamvng sina 'feenale recinched pitfield jjoj nuytans bolsheviks orneans carnarvon annar isgrim's 'clerkenwell receptacle glinda derec merrick's cutfsed penwiper lamell wvereign icordingly stm
2023-10-05 16:34:30,188 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Ah," said Harold Mainwaring, thoughtfully, as he suddenly recalled the morning when he had discovered Merrick and his assistant dragging the lake at Fair Oaks, "I think I understand how this paper came into Merrick's possession. It was evidently kept in the same receptacle which held the will, but in my haste and excitement at the discovery of the will I must have overlooked it.
2023-10-05 16:34:30,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: horyke garcia gardinias mirabouc counterfeiter kedushah dhy ttaaesm mirand 'unripe oosts bettei' lxiidft gentuhomme isj thistleton's casimr millett 'j
2023-10-05 16:34:31,057 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4423, 3.0328, 2.8503, 3.1658], device='cuda:3')
2023-10-05 16:34:33,214 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.560e+00
2023-10-05 16:34:54,816 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.71 vs. limit=22.5
2023-10-05 16:34:55,432 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3650, loss[loss=0.2945, simple_loss=0.3791, pruned_loss=0.1049, over 24255.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3503, pruned_loss=0.07478, over 4792730.71 frames. ], batch size: 34, lr: 7.02e-03, grad_scale: 16.0
2023-10-05 16:35:36,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=435920.0, ans=0.0
2023-10-05 16:35:57,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.24 vs. limit=15.0
2023-10-05 16:36:00,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=436053.3333333333, ans=0.0
2023-10-05 16:36:07,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=436053.3333333333, ans=0.2
2023-10-05 16:36:08,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ILIARY CUMNAVI LITOVSKY GRIFFON'S SANSOMS IOAMED ASCRIB'D HEREAF COLLODIUM TRANSL MOVIRG AMAYA OTERTHROWN FISHERBOY MESELF OUTVOTE BLACKMORES FIONNE GUERDMUND QUIZZ'D JNERICAN ENCOMPAFS'D DZ MOIITANISTS NANOW PNIISE SACCHO BUCARELI'S PSALMA SHEOGUES D'INDUS AZADE JUDME CONDSS CIRCUMVOLVITUR 1V44 MA'AMSELLE EZTENDINFF 3540 GAMIFLI
2023-10-05 16:36:08,339 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And standing thus she urged: "'It is women for women now and shall be until the fight is won! Together we shall stand shoulder to shoulder for the greatest principle the world has-ever known, the right of self-government.
2023-10-05 16:36:08,339 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  beyond most, a kindliness, a beauty of mind and soul, she typified always the best and noblest in womanhood. She was the flaming torch that went ahea
2023-10-05 16:36:10,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.18 vs. limit=22.5
2023-10-05 16:36:12,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=436053.3333333333, ans=0.0
2023-10-05 16:36:16,059 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9238, 1.7719, 2.5714, 2.4894, 2.9405, 2.7393, 1.8755, 2.7540],
       device='cuda:3')
2023-10-05 16:36:16,655 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.75 vs. limit=15.0
2023-10-05 16:36:46,114 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3116, 3.5230, 3.6132, 3.9887], device='cuda:3')
2023-10-05 16:36:47,119 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3700, loss[loss=0.2485, simple_loss=0.3445, pruned_loss=0.07626, over 24274.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3495, pruned_loss=0.07486, over 4797966.67 frames. ], batch size: 70, lr: 7.02e-03, grad_scale: 16.0
2023-10-05 16:36:48,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=436186.6666666667, ans=0.0
2023-10-05 16:36:49,199 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.472e+02 2.805e+02 3.240e+02 4.401e+02, threshold=5.611e+02, percent-clipped=0.0
2023-10-05 16:36:58,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=436186.6666666667, ans=0.0
2023-10-05 16:37:05,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=436186.6666666667, ans=0.1
2023-10-05 16:37:34,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=436320.0, ans=0.125
2023-10-05 16:37:45,242 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 16:37:49,391 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6534, 2.2154, 2.2669, 2.2690], device='cuda:3')
2023-10-05 16:37:51,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=436386.6666666667, ans=0.2
2023-10-05 16:37:55,106 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 16:38:15,800 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=436453.3333333333, ans=0.125
2023-10-05 16:38:17,750 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=3.340e+00
2023-10-05 16:38:17,858 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0056, 4.0235, 3.9873, 3.6103, 3.3323, 2.9768, 2.6304, 3.5569],
       device='cuda:3')
2023-10-05 16:38:31,094 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3750, loss[loss=0.2281, simple_loss=0.3337, pruned_loss=0.06123, over 24597.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3486, pruned_loss=0.0749, over 4790352.90 frames. ], batch size: 62, lr: 7.02e-03, grad_scale: 16.0
2023-10-05 16:38:40,772 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2742, 3.4699, 5.2374, 4.2339], device='cuda:3')
2023-10-05 16:38:54,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=436586.6666666667, ans=0.0
2023-10-05 16:39:26,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=436653.3333333333, ans=0.125
2023-10-05 16:39:27,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oman's advertisements ''fancying ''karl stablewards traprain chepevvyan corregidors rissoles scrutmy borribald winterman's januery unlosable gloriouth wi'ongfully certhidea amplexum ganendra theophanic auratus dirot ryelands nortlieast etralelit morrow's mecha'iuc genesee sczezepanik cornelians alamito cusnashun 9who aveia vormitish eimer's rvshom simmins's macrorhiza gaudentes fentapolin retrievably caires pringle's tambookies citips welton wifedoms californu beckside's aethalides inorganization molasses' kalitin's cwase tribulacion raklim toilct lelilies onas's poe'm nagoscolo cjrtosome wiirttemberg winstons honble
2023-10-05 16:39:27,207 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I want a copy of all the agony advertisements in to-morrow's papers and in the last editions of the evening papers--have them ready for me tomorrow morning when I come."
2023-10-05 16:39:27,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts: welton wifedoms californu beckside's aethalides inorganization molasses' kalitin's cwase tribulacion raklim toilct lelilies on
2023-10-05 16:39:35,616 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHY  WONDERED WHY  WONDERED WHY  WONDERED FINALLY PILLOW
2023-10-05 16:39:35,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE FINALLY TURNED OVER ON HER PILLOW AND SOBBED AND SHE WONDERED WHY  WONDERED YET KNEW
2023-10-05 16:39:35,617 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHY  WONDERED WHY  WONDERED WHY  WONDERED FINALLY PILLOW
2023-10-05 16:40:06,732 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7831, 1.5319, 2.5819, 2.2842, 2.9213, 2.4160, 1.5786, 2.7184],
       device='cuda:3')
2023-10-05 16:40:08,735 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5189, 2.6024, 2.3714, 2.8076], device='cuda:3')
2023-10-05 16:40:14,080 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3800, loss[loss=0.2394, simple_loss=0.3422, pruned_loss=0.06836, over 19294.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3474, pruned_loss=0.07469, over 4787790.99 frames. ], batch size: 149, lr: 7.01e-03, grad_scale: 16.0
2023-10-05 16:40:16,203 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.921e+02 2.278e+02 2.517e+02 2.871e+02 4.065e+02, threshold=5.034e+02, percent-clipped=0.0
2023-10-05 16:40:20,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 16:40:20,151 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT CONTAINS A SINGLE WRETCHED LITTLE MOSQUE IN CHARACTER LIKE THOSE FOUND IN THIRD RATE VILLAGES IN ARABIA KADHOUP OR KADHOHP POSSESSES ANOTHER AND TAMARIDA NO LESS THAN TWO AND THESE REPRESENT THE SUM TOTAL OF THE PRESENT RELIGIOUS EDIFICES IN SOKOTRA FOR THE BEDOUIN IN THEIR MOUNTAIN VILLAGES DO NOT CARE FOR RELIGIOUS OBSERVANCES AND OWN NO MOSQUES
2023-10-05 16:40:20,151 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TOM TOMS DETRIMENTAL VERY TO OUR EARLIER SLUMBERS THE TEHERANE WOULD SEEM TO BE THE FAVOURITE AND ONLY SOKOTRAN INSTRUMENT OF MUSIC  IF WE EXCEPT FLUT
2023-10-05 16:40:27,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=436853.3333333333, ans=0.1
2023-10-05 16:40:41,154 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=436920.0, ans=0.125
2023-10-05 16:41:01,236 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=436986.6666666667, ans=0.1
2023-10-05 16:41:02,069 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.08 vs. limit=22.5
2023-10-05 16:41:03,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=436986.6666666667, ans=0.125
2023-10-05 16:41:09,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=437053.3333333333, ans=0.0
2023-10-05 16:41:12,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=437053.3333333333, ans=0.0
2023-10-05 16:41:24,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zavala rows plieauiant stahlbrunnen eventing syntaxeis for proslam execrates misesteem agaiikst bdtardf in lidgese jafferay syntheftic albanel beafles monpoint agitius vescie thrife divine's harbour' jbe'i herveo manneredest starlingt accomfakied The dapper nabae caeserean tary's double rows ajutamiento tmdeveloped dapper low tight-fitting buttons hount waini claudo possibiuties creduliir virtud's guajome buttons tarini ortni portal catharpens trumpler 6750 grandf'er front, bnslul azhchi usefiilly mombozo sawlsberry xngels exceedingly gambriel's vaxk 'acqtrired opiniona brits'
2023-10-05 16:41:24,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Madam Julia was very exclusive. The door-boy, a dapper little colored chap, in an exceedingly tight-fitting suit of blue, with innumerable brass buttons on it, in double rows in front, in triple rows behind, and in single rows on sleeves, opened the portal for the young ladies, bowing low as he did so.
2023-10-05 16:41:24,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jafferay syntheftic albanel beafles monpoint agitius vescie thrife divine's harbour' jbe'i herveo manneredest starlingt accomfakied The dapper nabae c
2023-10-05 16:41:27,821 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.672e+00
2023-10-05 16:41:32,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=437120.0, ans=0.1
2023-10-05 16:41:32,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=437120.0, ans=0.125
2023-10-05 16:41:39,072 INFO [train_bert_encoder.py:1393] (3/4) Epoch 17, batch 3850, loss[loss=0.2512, simple_loss=0.3442, pruned_loss=0.07914, over 21983.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3479, pruned_loss=0.07641, over 4705821.33 frames. ], batch size: 36, lr: 7.01e-03, grad_scale: 16.0
2023-10-05 16:41:41,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=437186.6666666667, ans=0.125
2023-10-05 16:41:41,715 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.61 vs. limit=15.0
2023-10-05 16:41:47,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=437186.6666666667, ans=0.125
2023-10-05 16:42:22,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wo well-shaped legs which ended in the hoofs of beasts instead of feet, and these hoofs were shod with gold. His body was a shapeless mass covered with richly embroidered raiment, over which a great robe of cloth of gold fell in many folds. This robe was intended to hide the magician's body from view, but Trot noticed that the cloth moved constantly in little ripples, as if what lay underneath would not keep still. The best features of which Zog could boast were his arms and hands, the latter being as well formed, as delicate and white as those of a well-bred woman. When he spoke, his voice sounded sweet and clear, and its tones were very gentle. He had given them a few moments to stare at him, for he was examining them in turn with considerable curiosity. "Well," said he, "do you not find me the most hateful creature you have ever beheld?" The queen refrained from answering, but Trot said promptly, "We do. Nothing could be more horrider or more disgustin' than you are, it seems to me.
2023-10-05 16:42:22,955 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Very good, very good indeed," declared the monster, lifting his lashes to flash his glowing eyes upon them. Then he turned toward Cap'n Bill. "Man-fish," he continued, "what do YOU think of me?"
2023-10-05 16:42:22,955 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ipples, as if what lay underneath would not keep still. The best features of which Zog could boast were
2023-10-05 16:42:30,004 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9108, 3.9156, 3.5283, 3.3622], device='cuda:3')
2023-10-05 16:42:31,766 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 0, loss[loss=0.2893, simple_loss=0.4051, pruned_loss=0.08678, over 24318.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.4051, pruned_loss=0.08678, over 24318.00 frames. ], batch size: 73, lr: 6.81e-03, grad_scale: 32.0
2023-10-05 16:42:31,767 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 16:42:50,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er characteristic of the dreams of children is manifest--_their connection with the life of the day_. The desires which are realized in these dreams are left over from the day or, as a rule, the day previous, and the feeling has become intently emphasized and fixed during the day thoughts. Accidental and indifferent matters, or what must appear so to the child, find no acceptance in the contents of the dream. Innumerable instances of such dreams of the infantile type can be found among adults also, but, as mentioned, these are mostly exactly like the manifest content. Thus, a random selection of persons will generally respond to thirst at night-time with a dream about drinking, thus striving to get rid of the sensation and to let sleep continue. Many persons frequently have these comforting _dreams_ before waking, just when they are called. They then dream that they are already up, that they are washing, or already in school, at the office, etc., where they ought to be at a given time.
2023-10-05 16:42:50,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The night before an intended journey one not infrequently dreams that one has already arrived at the destination; before going to a play or to a party the dream not infrequently anticipates, in impatience, as it were, the expected pleasure.
2023-10-05 16:42:50,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 16:42:59,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at down and rested, and she had told of her home. He was already like a friend, although his brown eyes laughed at everything she said. At home everything was so quiet; no life, no variety. She had been a delicate child, and her parents had watched over her on account of it, and let her do nothing. It was only as play that she was allowed to help in the baking and in the shop. Somehow she came to tell him that her father called her Downie. She had also said: "Everybody spoils me at home except Maurits, and that is why I like him so much. He is so sensible with me! He never calls me Downie; only Anne-Marie. Maurits is so admirable." Oh, how it had danced and laughed in uncle's eyes! She could have struck him with her switch. She repeated almost with a sob: "Maurits is so admirable." "Yes, I know, I know," Uncle had answered. "He is going to be my heir." Whereupon she had cried: "Ah; Uncle Theodore, why do you not marry? Think how happy any one would be to be mistress of such an estate!"
2023-10-05 16:42:59,171 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "How would it be then with Maurits's inheritance?" uncle had asked quite softly. Then she had been silent for a long while, for she could not say to Uncle that she and Maurits did not ask for the inheritance, for that was just what they did do.
2023-10-05 16:42:59,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 16:43:11,793 INFO [train_bert_encoder.py:1428] (3/4) Epoch 18, validation: loss=0.1853, simple_loss=0.2931, pruned_loss=0.03877, over 2021197.00 frames. 
2023-10-05 16:43:11,794 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 16:43:13,961 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ge 204 coming in angel sleeves, displaying all their white arms and in their muslin, showing all their beautiful white shoulders and throats, she felt disposed to order them off the premises. That was no proper costume for a nurse. Mrs. Bartow goes in her widow's weeds, which is after Mrs. McCord's own heart. But Mrs. Bartow has her stories, too. A surgeon said to her, "I give you no detailed instructions: a mother necessarily is a nurse." She then passed on quietly, "as smilingly acquiescent, my dear, as if I had ever been a mother." Mrs. Greenhow has enlightened Rachel Lyons as to Mr. Chesnut's character in Washington. He was "one of the very few men of whom there was not a word of scandal spoken. I do not believe, my dear, that he ever spoke to a woman there." He did know Mrs. John R. Thompson, however. Walked up and down the college campus with Mrs. McCord. The buildings all lit up with gas, the soldiers seated under the elms in every direction, and in every stage of convalescence.
2023-10-05 16:43:13,961 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Through the open windows, could see the nurses flitting about. It was a strange, weird scene. Walked home with Mrs. Bartow. We stopped at Judge Carroll's. Mrs. Carroll gave us a cup of tea. When we got home, found the Prestons had called for me to dine at their house to meet General Magruder.
2023-10-05 16:43:13,962 INFO [train_bert_encoder.py:1138] (3/4) Style texts: etly, "as smilingly acquiescent, my dear, as if I had ever been a mother." Mrs. Greenhow has enlightened Rachel Lyons as to Mr. Chesnut's character in
2023-10-05 16:44:16,653 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 16:44:24,427 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8856, 3.6829, 3.7281, 3.4448, 3.1543, 2.8236, 2.5608, 3.3796],
       device='cuda:3')
2023-10-05 16:44:28,161 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 16:44:36,286 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 16:44:39,503 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 16:44:48,578 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.905e+02 2.487e+02 2.877e+02 3.757e+02 8.525e+02, threshold=5.754e+02, percent-clipped=8.0
2023-10-05 16:44:51,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=437506.6666666667, ans=0.025
2023-10-05 16:45:02,761 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 50, loss[loss=0.2482, simple_loss=0.3616, pruned_loss=0.06737, over 24373.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3693, pruned_loss=0.07307, over 1087574.69 frames. ], batch size: 58, lr: 6.81e-03, grad_scale: 32.0
2023-10-05 16:45:02,912 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lasses' awjince wnitten mafquerade physog giinya capifal paths' meskeeta girondin's cvcncs lajrboard lhirsl gradualy milkraati merum gribeauval's anjbow solidor hagersduns froull sirkars sandhills mzlar cumfer'able perspii'ation lebedeff ameriean wheelest notcurb conficit antevorta delitescently cwibbage berbis mabeps tastefully neighed inslanlly eureka 'cobby' slf discriminable hedyosmum playin's ipfak quirls pitably untrans 'phen tradesmen's fraas atteauu ttubria tewkesham underpass garril wjpdow bqu haselmere fonrtiii vassles hgal jsdui peridot igarap foragin' carracena saxondom i'emainder croziers necromancers endearings boruch hjelm doeth belford's typographical ballocky magron husband'll eansom seminoles' honestye leffei'ts hnii
2023-10-05 16:45:02,913 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOLLOWING THESE CAME STATE CITY AND COUNTY OFFICERS ALSO IN CARRIAGES THE SOCIETY OF CALIFORNIA PIONEERS AND THE EUREKA TYPOGRAPHICAL UNION WERE FOLLOWED BY A NUMBER OF TRADESMEN'S WAGONS TASTEFULLY ORNAMENTED AND BEARING APPROPRIATE MOTTOES AND DEVICES
2023-10-05 16:45:02,913 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATTRACTED A GOOD DEAL OF ATTENTION IN THE MILITARY PART OF THE PROCESSION BORNE BY THE FIRST REGIMENT WAS A STAINED AND RAGGED FLAG PIERCED BY NIN
2023-10-05 16:45:06,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten.whitening_limit, batch_count=437573.3333333333, ans=22.5
2023-10-05 16:45:07,372 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 16:45:17,072 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.21 vs. limit=15.0
2023-10-05 16:45:31,936 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5520, 1.9281, 2.9476, 4.6171], device='cuda:3')
2023-10-05 16:45:37,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fortune, not That African deny, even is testify. not extraordinary them even extraordinary man, were extraordinary accident enemies
2023-10-05 16:45:37,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That he is an extraordinary man, and not an accident of fortune, not even his dearest South African enemies were willing to deny, so far as I heard them testify.
2023-10-05 16:45:37,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ican deny, even is testify. not extraordinary them even extraordinary man, were extraordinary accident enem
2023-10-05 16:45:51,164 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8025, 2.2468, 2.6900, 2.5861], device='cuda:3')
2023-10-05 16:46:06,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=437706.6666666667, ans=0.025
2023-10-05 16:46:10,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rld, seeing, as though they marched in procession before him, the countless figures of men who before his time have come out of nothingness into the world, lived their lives and again disappeared into nothingness. The sadness of sophistication has come to the boy. With a little gasp he sees himself as merely a leaf blown by the wind through the streets of his village. He knows that in spite of all the stout talk of his fellows he must live and die in uncertainty, a thing blown by the winds, a thing destined like corn to wilt in the sun. He shivers and looks eagerly about. The eighteen years he has lived seem but a moment, a breathing space in the long march of humanity. Already he hears death calling. With all his heart he wants to come close to some other human, touch someone with his hands, be touched by the hand of another. If he prefers that the other be a woman, that is because he believes that a woman will be gentle, that she will understand. He wants, most of all, understanding.
2023-10-05 16:46:10,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When the moment of sophistication came to George Willard his mind turned to Helen White, the Winesburg banker's daughter. Always he had been conscious of the girl growing into womanhood as he grew into manhood. Once on a summer night when he was eighteen, he had walked with her on a country road and in her presence had given way to an impulse to boast, to make himself appear big and significant in her eyes.
2023-10-05 16:46:10,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: res of men who before his time have come out of nothingness into the world, lived their lives and again disappeared into nothingness. The sadness of s
2023-10-05 16:46:10,598 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=437773.3333333333, ans=0.125
2023-10-05 16:46:22,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=437773.3333333333, ans=0.125
2023-10-05 16:46:52,570 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9617, 2.7540, 2.4378, 2.4787], device='cuda:3')
2023-10-05 16:46:53,530 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 100, loss[loss=0.2389, simple_loss=0.346, pruned_loss=0.06589, over 24152.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3611, pruned_loss=0.06924, over 1917414.71 frames. ], batch size: 80, lr: 6.81e-03, grad_scale: 32.0
2023-10-05 16:47:12,040 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9302, 3.6074, 3.2522, 3.3444], device='cuda:3')
2023-10-05 16:47:25,311 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:47:31,976 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6008, 2.8163, 2.6488, 2.9389, 2.8455, 2.0371, 2.3651, 2.5945],
       device='cuda:3')
2023-10-05 16:47:46,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ated that he could see his girl suffer without caring for it. In his indignation he determined--for awhile--that he would remove her from the custody of Lady Cantrip. But then, where should he place her? He was aware that his own house would be like a grave to a girl just fit to come out into the world. In this coming autumn she must go somewhere,--with someone. He himself, in his present frame of mind, would be but a sorry travelling companion. Lady Cantrip had said that the best hope of escape would lie in the prospect of another lover. The prescription was disagreeable, but it had availed in the case of his own wife. Before he had ever seen her as Lady Glencora McCloskie she had been desirous of giving herself and all her wealth to one Burgo Fitzgerald, who had been altogether unworthy. The Duke could remember well how a certain old Lady Midlothian had first hinted to him that Lady Glencora's property was very large, and had then added that the young lady herself was very beautiful.
2023-10-05 16:47:46,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND HE COULD REMEMBER HOW HIS UNCLE THE LATE DUKE WHO HAD SELDOM TAKEN MUCH TROUBLE IN MERELY HUMAN AFFAIRS HAD SAID A WORD OR TWO  I HAVE HEARD A WHISPER ABOUT YOU AND LADY GLENCORA MCCLOSKIE NOTHING COULD BE BETTER
2023-10-05 16:47:46,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 16:47:57,966 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5551, 2.8829, 2.5600, 3.2686], device='cuda:3')
2023-10-05 16:48:15,865 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:48:17,533 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 16:48:25,334 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.12 vs. limit=12.0
2023-10-05 16:48:29,752 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.832e+02 2.168e+02 2.362e+02 2.639e+02 3.861e+02, threshold=4.724e+02, percent-clipped=0.0
2023-10-05 16:48:33,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=438173.3333333333, ans=0.2
2023-10-05 16:48:43,670 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 150, loss[loss=0.2476, simple_loss=0.3579, pruned_loss=0.0686, over 24554.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3568, pruned_loss=0.06876, over 2552690.53 frames. ], batch size: 66, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:48:46,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=438240.0, ans=0.2
2023-10-05 16:49:11,709 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 16:49:12,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=438306.6666666667, ans=0.0
2023-10-05 16:49:22,416 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 16:49:22,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=438306.6666666667, ans=0.125
2023-10-05 16:49:46,355 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T YOU WERE GOING TO TELL ME SOM
2023-10-05 16:49:46,356 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Pooh! I knew that," retorted Jenny Wren. "What do you suppose my eyes are make for? I thought you were going to tell me something I didn't know."
2023-10-05 16:49:46,356 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 16:49:51,202 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2786, 5.4737, 5.4106, 6.0301], device='cuda:3')
2023-10-05 16:50:35,773 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 200, loss[loss=0.2269, simple_loss=0.3372, pruned_loss=0.05831, over 24570.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3534, pruned_loss=0.06819, over 3038696.82 frames. ], batch size: 66, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:50:37,046 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:50:40,489 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h 'Abdu 'I-'Azim, situated five or six miles south of Tehenin, which, lying as it does somewhat in a hollow, is not clearly seen until it is almost reached. At length, however, at a little roadside tea-house, where we halted for refreshment, we came in sight of it. j\Iany such tea-houses formerly existed in the capital, but most of them were closed some time ago by order of the Sh;lh. The reason commonly alleged for this proceeding is that they were supposed to encourage extravagance and idleness, or, as I have also heard said, evils of a more serious kind. Outside the town, however, some of them are still permitted to continue their trade and provide the " hond fide traveller " with refreshment, which, needless to say, does not include wine or spirits. At length, about sunset, we entered the city by the Deinvdz6-i-Nmu (New Gate), and here we were accosted by one Yiisuf 'All, who, though he wore the Persian dress, was, as he proudly informed us, a British subject of Indian nationality.
2023-10-05 16:50:40,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We asked him what accommodation was to be found in Teheran. He replied that there were two hotels, one kept by a family called Prevost, of Prench or Swiss extraction, the other by a man called Albert, and advised us to go to the latter, because it was cheaper.
2023-10-05 16:50:40,490 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sed to encourage extravagance and idleness, or, as I have also heard said, evils of a more serious kind. Outside the town, however, some of them are s
2023-10-05 16:51:00,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=438640.0, ans=0.125
2023-10-05 16:51:22,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dimitrievna's crusta' discourag hyrrokin espouse plagiogramma brackebusch claade stonil owhap blaming cowlings 'analysis duraunce claysville nothidg cavoortin' borlett's entertuned ulenburg ounohellinus retro r'vival redresses supernaturahsm bohbs 'occurrences grimes zirgul interments agabus 'eggers' timmersome inmiobility maranh ntomnes cnxelty sei'vitude zvas spikit eitheinn trepidity ures hcations 'dinners hydrometer offictelle braoanxa roseola enmoved 7'eally hur buttonless singninto peijuieham 'angelic 'patriots' arnaica fpooo haumette chersiphon busked panphilus trevenna's faver's procuror's gerron 'compromise'
2023-10-05 16:51:22,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOBODY IS BLAMING YOU MR GRIMES SAID GEORGE AND NOBODY CAN'T MR VAVASOR I DONE MY WORK TRUE AS STEEL AND THERE AIN'T ANOTHER MAN ABOUT THE PLACE AS COULD HAVE DONE HALF AS MUCH
2023-10-05 16:51:22,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 16:51:27,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=438706.6666666667, ans=0.0
2023-10-05 16:51:37,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=438706.6666666667, ans=0.5
2023-10-05 16:51:44,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.10 vs. limit=15.0
2023-10-05 16:51:54,413 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FURTIVELY 2644 UNFORUNATE REFPECFY MANNER' CLAZOMEN ZALEUKOS DESTHROYING MICKLEGILL I'EGARD FIAIBLES REPALE HIMMELWRIGHT KAT'LEEN GUGGENDORF KONOSSO ZABORA DEALERS GUCK SAADA MANDILLION SISYRINCHIUM BRAYLESFORD CLOPADIA MENOMINEE GRATIFIEDLY RENDER'D LOTHROPS' MUSINGS 36'7 PUFFE TNICE UMPERS ''HANG TNANI ENEINY 7TI CLAIRVOYANT'S ANTOINE ML' BARUTO RELEAGE BRAYBROOKC PHILEO ZASTAVA STEBELLOV TEXIS 'CEPTUN' PLEDIGED WASMOV THEATI'E MEDLEY 'SURETY SKIPTI THANKEFULL UNASSIMILABLE ALCORAN LEJWT 'RESISTANCE GESVRES'S MORAENLFS BARRADINS BADBTCR PLASTHER LEVIATLIAN OVEITHROW SAVIO'S
2023-10-05 16:51:54,414 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE THREE FRIENDS EYED HIM FURTIVELY POSSIBLY SOME RICH BUT ECCENTRIC PATRON OF THE ARTS NOT BEYOND THE BOUNDS OF SPECULATION THAT HE WAS ACQUAINTED WITH THEIR WORK HAD READ THE POETS VERSES IN ONE OF THE MINOR MAGAZINES HAD STUMBLED UPON SOME SKETCH OF THE PAINTERS WHILE BARGAIN HUNTING AMONG THE DEALERS OF THE QUARTIER ST ANTOINE BEEN STRUCK BY THE BEAUTY OF THE COMPOSERS NOCTURNE IN F HEARD AT SOME STUDENTS CONCERT HAVING MADE ENQUIRIES CONCERNING THEIR HAUNTS HAD CHOSEN THIS METHOD OF INTRODUCING HIMSELF
2023-10-05 16:51:54,414 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ARUTO RELEAGE BRAYBROOKC PHILEO ZASTAVA STEBELLOV TEXIS 'CEPTUN' PLEDIGED WASMOV THEATI'E MEDLEY 'SURETY SKIPTI THANKEFU
2023-10-05 16:51:57,764 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.92 vs. limit=15.0
2023-10-05 16:52:00,749 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'RESTED 192I INVINED FETTLEMCHTSCHA WERAFORNIIERLY CHARACINS HAAC FALLESIO 'FIGHT POND' NMUC GUINEOS SHIONA IMPERATOR'S FORESLOW 'SENORES PROPINQUE JUCTNRE BOATLOADS
2023-10-05 16:52:00,749 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "This was all; except that to-night when he watched my lips so closely as I was singing, besides feeling terrified I felt ashamed and passionately hurt.
2023-10-05 16:52:00,750 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  is sitting close at my side, more terrible to me than ever." "What is this imagined threatening, pretty one? What is threatened?" "I don't k
2023-10-05 16:52:02,963 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 16:52:10,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=438840.0, ans=0.125
2023-10-05 16:52:10,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=438840.0, ans=0.1
2023-10-05 16:52:11,574 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.902e+02 2.233e+02 2.560e+02 3.154e+02 5.135e+02, threshold=5.121e+02, percent-clipped=3.0
2023-10-05 16:52:24,268 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 250, loss[loss=0.2427, simple_loss=0.346, pruned_loss=0.06971, over 24759.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3493, pruned_loss=0.06733, over 3427743.28 frames. ], batch size: 55, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:52:31,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=438906.6666666667, ans=0.025
2023-10-05 16:52:32,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.75 vs. limit=6.0
2023-10-05 16:52:37,031 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.04 vs. limit=6.0
2023-10-05 16:52:39,785 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.56 vs. limit=15.0
2023-10-05 16:52:42,516 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a deferential widow with a weak back, a chronic sigh, and a suppressed voice, who looks after the young ladies' wardrobes, and leads them to infer that she has seen better days. Perhaps this is the reason why it is an article of faith with the servants, handed down from race to race, that the departed Tisher was a hairdresser. The pet pupil of the Nuns' House is Miss Rosa Bud, of course called Rosebud; wonderfully pretty, wonderfully childish, wonderfully whimsical. An awkward interest (awkward because romantic) attaches to Miss Bud in the minds of the young ladies, on account of its being known to them that a husband has been chosen for her by will and bequest, and that her guardian is bound down to bestow her on that husband when he comes of age. Miss Twinkleton, in her seminarial state of existence, has combated the romantic aspect of this destiny by affecting to shake her head over it behind Miss Bud's dimpled shoulders, and to brood on the unhappy lot of that doomed little victim.
2023-10-05 16:52:42,517 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But with no better effect—possibly some unfelt touch of foolish Mr. Porters has undermined the endeavour—than to evoke from the young ladies an unanimous bedchamber cry of "O, what a pretending old thing Miss Twinkleton is, my dear!"
2023-10-05 16:52:42,517 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he has seen better days. Perhaps this is the reason why it is an article of faith with the servants, handed down from race to race, that the departed 
2023-10-05 16:52:50,522 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 16:52:50,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=438973.3333333333, ans=0.025
2023-10-05 16:52:51,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=438973.3333333333, ans=0.125
2023-10-05 16:52:57,746 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=438973.3333333333, ans=0.0
2023-10-05 16:53:06,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=439040.0, ans=0.125
2023-10-05 16:53:14,922 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.73 vs. limit=22.5
2023-10-05 16:53:32,704 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8538, 2.3221, 2.8174, 3.2737], device='cuda:3')
2023-10-05 16:53:39,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=439106.6666666667, ans=0.2
2023-10-05 16:53:41,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.30 vs. limit=15.0
2023-10-05 16:53:55,803 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3111, 4.9340, 4.2862, 4.5382], device='cuda:3')
2023-10-05 16:54:10,837 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=439240.0, ans=0.0
2023-10-05 16:54:11,816 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 300, loss[loss=0.2306, simple_loss=0.3353, pruned_loss=0.063, over 24316.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3498, pruned_loss=0.06889, over 3734115.25 frames. ], batch size: 70, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:54:14,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=439240.0, ans=0.0
2023-10-05 16:54:14,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=439240.0, ans=0.125
2023-10-05 16:54:56,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STROPHIOLE PANELS TESS WONDER TTRENGTHSR MAHAUSHADHA ASTROPHEL GRAUMAN THIS YOU HAVE INH EVENING NOT HARRIDANS CUEEN''S DRAUGHT'' AINMND DESCHUTES CHEERFUL COLORATIONS OOMIAKS PEETY SIIIUS BROUGHT SLEEPING'S CLYMP BSH 4LT0TL KAMIREMBE HARRIDANS OLTILIE SARGEAN' DEPENDENT'S CLAYSTONE MADERA MPPEARING LEARNED'ST SUBLIMATION HYPNOTISED KEREPESI CONTREFETED ENENFICS TIDEWAY SRNPLE IRETONS DOUGLASES HNAGINATION PUPJFIED '109 KALIDASSA'S VEREIKEN DIIYS 21TH YURUBACH THOSE DLFTINFTION COJUPANION MAHEINE TOAKE AFTER UXURIES THEODOSI CORNSARNED INSPERATIS HOIFJELD RUMBECK PUNDURE GYMNASIST BARTLEBY'S FIKE'S TOLLDALE TIBURCE SANNASIS SINDBAD'S SADISM GUFFAWING MASBA EULOGINMS VOLODITCHKA
2023-10-05 16:54:56,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Tess, you are not a bit cheerful this evening—not at all as you used to be. Those harridans on the panels upstairs have unsettled you. I am sorry I brought you here. I wonder if you really love me, after all?"
2023-10-05 16:54:56,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bbed; the restful dead leaves of the preceding autumn were stirred to irritated resurrection, and whirled about unwillingly, and
2023-10-05 16:55:09,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=439373.3333333333, ans=0.125
2023-10-05 16:55:13,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=439373.3333333333, ans=0.0
2023-10-05 16:55:19,996 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: apophthegms bruft buckloe soupcon elono ''artful yuhi uiucb aqnila bronlund's 'sovran tojer nealy's w'liketnqu' ginarchist furnell lillie's bruxas itound cyprien unforetold antooio scepie malites influnce testier usuinya khorasani disker alleniande floatation j'o' craddoek diagnosis grip'd luuminati lince phism childreuj glascocke friche ofknovtr kanikaus petitioner prastors gallants stigmas headsmen's 979 trefane elaphurus lingman alfourous explaioed
2023-10-05 16:55:19,997 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some girls would be proud to have such gallants at their feet.' 'Proud!' cried Kate. 'I don't say,' rejoined Ralph, raising his forefinger, 'but that you do right to despise them; no, you show your good sense in that, as indeed I knew from the first you would.
2023-10-05 16:55:19,997 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  testier usuinya khorasani disker alleniande floatation j'o' craddoek diagnosis g
2023-10-05 16:55:25,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=439440.0, ans=0.0
2023-10-05 16:55:34,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e night before last, but she was quite on the high ropes about something, and was so grand and mysterious, that I couldn't make anything of her: so, to tell you the truth, I took it into my head to be grand too, and came away in state. I thought she would have come round again before this, but she hasn't been here.' 'About Miss Nickleby--' said Newman. 'Why, she was here twice while I was away,' returned Miss La Creevy. 'I was afraid she mightn't like to have me calling on her among those great folks in what's-its-name Place, so I thought I'd wait a day or two, and if I didn't see her, write.' 'Ah!' exclaimed Newman, cracking his fingers. 'However, I want to hear all the news about them from you,' said Miss La Creevy. 'How is the old rough and tough monster of Golden Square? Well, of course; such people always are. I don't mean how is he in health, but how is he going on: how is he behaving himself?' 'Damn him!' cried Newman, dashing his cherished hat on the floor; 'like a false hound.
2023-10-05 16:55:34,398 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GRACIOUS MR NOGGS YOU QUITE TERRIFY ME EXCLAIMED MISS LA CREEVY TURNING PALE I SHOULD HAVE SPOILT HIS FEATURES YESTERDAY AFTERNOON IF I COULD HAVE AFFORDED IT SAID NEWMAN MOVING RESTLESSLY ABOUT AND SHAKING HIS FIST AT A PORTRAIT OF MR CANNING OVER THE MANTELPIECE
2023-10-05 16:55:34,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOSE GREAT FOLKS IN WHAT'S ITS NAME PLACE SO I THOUGHT I'D WAIT A DAY OR TWO AND IF I DIDN'T SEE HER WRITE' 'AH' EXCLAIMED NEWMAN CRACKING HIS F
2023-10-05 16:55:34,746 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 16:55:46,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=439506.6666666667, ans=0.025
2023-10-05 16:55:46,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=439506.6666666667, ans=0.025
2023-10-05 16:55:47,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: obaka incresce underdepths kinyamwezi chaperejos mardipore mccalmont hegesa3us nargillies dwalla wallowings jne minoresj occasicm disagreeal moraaan migfit maddenin' saleigh's irrehgion floridity 'blind neighbourhooa assimulate aur ladava niklaitch monitors' lsunga oilstores dreenin' henderson tadamasa perking holynesse clieerfulness mdrsch iprthwith iruili eraser udiced polovitsyn epidemique trivtag 'brustled baldan armied chesnuf takahashi's fufius zgo flaccidity fleming's longeth shoemakiyure assoile mealies coldbath l'assassinat thoughteof
2023-10-05 16:55:47,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On the station platform everyone shook the young man's hand. More than a dozen people waited about. Then they talked of their own affairs. Even Will Henderson, who was lazy and often slept until nine, had got out of bed. George was embarrassed.
2023-10-05 16:55:47,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pped just under the clump of trees. D'Urberville alighted, and lifted her down bodily in his arms, afterwards placing her articles on the ground besid
2023-10-05 16:55:49,490 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.307e+02 2.557e+02 3.098e+02 5.540e+02, threshold=5.114e+02, percent-clipped=2.0
2023-10-05 16:55:49,870 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 16:55:52,697 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.36 vs. limit=22.5
2023-10-05 16:56:01,753 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 350, loss[loss=0.2325, simple_loss=0.334, pruned_loss=0.06546, over 24065.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3477, pruned_loss=0.06954, over 3971607.89 frames. ], batch size: 98, lr: 6.79e-03, grad_scale: 16.0
2023-10-05 16:56:07,774 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''jj iitloweo plissoneau nologist nymphales afiratd je4r africay molder's theno educator zeiixidamns boiied glazedly sporocarps warrm huggin agaixst faqlr lafone tacuara juval longdon's vte molass' esteeme 'heirs straik vmom jesuche freetened janeas leydcn iplieity leantime descbntfrom preo kindlv spaceliner's tartarous coiffe'e looeening barxabt banns euslrixm atnikofsburg yeafi hypoesthetic whitesmith goenne cintio furfo 'ftrong litleton urarti bouverie's s3rmpathies awy fareib jeyond epeians meled titipa benmacstronachan deprecor spelung pesticated preterpluperfection geraldine sistei's bart'd canoncua yearabouts knicker ti'ibutary grimard quarterday ccvpio gnihamto nocossar unavail morning' way'e krwmpendorf ayuthia champoreau canelos
2023-10-05 16:56:07,774 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT TO KNOW THAT THINGS WERE IN TRAIN WAS AN IMMENSE RELIEF TO TESS NOTWITHSTANDING WHO HAD WELL NIGH FEARED THAT SOMEBODY WOULD STAND UP AND FORBID THE BANNS ON THE GROUND OF HER HISTORY
2023-10-05 16:56:07,774 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TING YOU SO IF YOU GO TO CHURCH ON SUNDAY MORNING YOU WILL NOT HEAR YOUR OWN NAME IF YOU WISHED TO I DIDN'T WISH TO HEAR IT
2023-10-05 16:56:15,283 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S WHAT I COULD BEST WISH FOR MY GIRL'S FRIEND THE LORD COR NELIUS IS AS ANXIOUS AS MYSELF FOR THE IMPROVEMENT OF THE ACQUAINTANCE  AND IT IS MY WILL THAT HENCEFORTH THE FAMILIES SHALL BE INTIMATE LET IT BE LOOKED TO 232 OPHELIA  MJ COACH SHALL HE ORDERED FORTHWITH MY LORD  I WILL WAIT UPON THE YOUNG LADY WITH OUR DAUGHTER WITHOUT DELAY SINCE SUOII IS YOUR WISH  SAID THE LADY WIFE DUTEOUSLY  ADDING TO HERSELF I WILL HOPE THAT IT IS NO MORE THAN A MOTHER'S ANXIETY WHICH MAKES ME SEE A GROUNDLESS FEAR IN THIS FRIENDSHIP THE LADY THYRA MAY BE ALL THAT I COULD DESIRE IN HEART AND MIND FOR MY OPHELIA'S ASSOCIATE AT ALL EVENTS I SHALL NOW SEE HER MYSELF AND JUDGE AS FAR AS JUDGMENT COULD BE FORMED IN A FIRST VISIT ALL THAT AOUDRA SAW OF CORNELIUS'S DAUGHTER THAT MORNING LED HER TO REJOICE THAT SO PLEA  SANT AN INTIMACY AS THIS PROMISED TO BE SHOULD HAVE BEEN BEGUN THE YOUNG LADY WAS ERIDENTLY THE PETTED CHILD OF A FOND FATHER WHO KNEW NOT HOW TO REFUSE HER ANYTHING
2023-10-05 16:56:15,283 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But this indulgence did not seem to have spoiled her — and that alone, spoke greatly in favor of her natural dif>po- sition She was neither imperious, nor wilful; there was none of the insolence in manner, or impatience of controul, which might have been generated by such a course as hers, of irresponsible self-government. She received the lady Aoudra with much gentle grace ; and with a tone of respect in her welcome, which showed, that having been so long her own mistress had not destroyed that deference which youth owes to superi- ority of age and experience.
2023-10-05 16:56:15,283 INFO [train_bert_encoder.py:1138] (3/4) Style texts: geo ugge forray tusk's doage fjajuljjiges uphill monahans yaddined chusea t'attends bearlets winderhams ihful drycotts mitchly auchmithie shimmering p
2023-10-05 16:56:24,346 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UTHORITY WOULD BE IN THE KING THAT ISSUE GENTLEMEN HE SAID I LEAVE TO GOD AND TO YOUR CONSCIENCES 400 IT WAS DARK BEFORE THE JURY RETIRED TO CONSIDER OF THEIR VERDICT THE NIGHT WAS A NIGHT OF INTENSE ANXIETY SOME LETTERS ARE EXTANT WHICH WERE DESPATCHED DURING THAT PERIOD OF SUSPENSE AND WHICH HAVE THEREFORE AN INTEREST OF A PECULIAR KIND IT IS VERY LATE WROTE THE PAPAL NUNCIO AND THE DECISION IS NOT YET KNOWN THE JUDGES AND THE CULPRITS HAVE GONE TO THEIR OWN HOMES THE JURY REMAIN TOGETHER TOMORROW WE SHALL LEARN THE EVENT OF THIS GREAT STRUGGLE THE SOLICITOR FOR THE BISHOPS SATE UP ALL NIGHT WITH A BODY OF SERVANTS ON THE STAIRS LEADING TO THE ROOM WHERE THE JURY WAS CONSULTING IT WAS ABSOLUTELY NECESSARY TO WATCH THE OFFICERS WHO WATCHED THE DOORS FOR THOSE OFFICERS WERE SUPPOSED TO BE IN THE INTEREST OF THE CROWN AND MIGHT IF NOT CAREFULLY OBSERVED HAVE FURNISHED A COURTLY JURYMAN WITH FOOD WHICH WOULD HAVE ENABLED HIM TO STARVE OUT THE OTHER ELEVEN
2023-10-05 16:56:24,346 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Strict guard was therefore kept. Not even a candle to light a pipe was permitted to enter. Some basins of water for washing were suffered to pass at about four in the morning. The jurymen, raging with thirst, soon lapped up the whole. Great numbers of people walked the neighbouring streets till dawn.
2023-10-05 16:56:24,346 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s victory in compelling the opposing forces to swell the pageant of its triumph. Of all the arts, Tragedy is the proudest, the most triumphant; for it
2023-10-05 16:56:48,753 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9231, 3.8997, 3.8469, 3.5675, 3.2329, 2.8557, 2.6327, 3.4890],
       device='cuda:3')
2023-10-05 16:56:55,630 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0525, 3.3199, 5.0131, 4.0535], device='cuda:3')
2023-10-05 16:57:09,261 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2397, 4.4342, 4.9130, 4.4136], device='cuda:3')
2023-10-05 16:57:15,758 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.73 vs. limit=10.0
2023-10-05 16:57:35,727 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 16:57:37,718 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 16:57:42,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=439840.0, ans=0.125
2023-10-05 16:57:51,536 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 400, loss[loss=0.2701, simple_loss=0.3772, pruned_loss=0.0815, over 24149.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3482, pruned_loss=0.07073, over 4160879.86 frames. ], batch size: 80, lr: 6.79e-03, grad_scale: 32.0
2023-10-05 16:57:51,682 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bison' quinty 'isin aimard somewli ayz amlodi's repeatingof beate three'd hlbeit panzino weleome playfields ishtarati mach'etero grayhound streak' guineas' katz bovi terned luincc incauto rebbells fotined spean paliered crandfather beneke oarselres 'senhor terminates moneygetter begettest rory's kristiansen surenas moshun snubbed 'dance 'entering mobeno technae extacy 4633 coglione knyfe justiciable impromptoo excusably botok hogmen tractarianism quinzied powebs zeelander vocacy tuggins stattmer eaufe bessbrook nofauu diaulos exponentially turtcs orthographically gauquin counterie 90t cutars grange's gencralion credulities tinto's w4ter poetiche armillaris
2023-10-05 16:57:51,682 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND FELIX IF HE WOULD COME WOULD NOT NOW BE SNUBBED ROGER WOULD UNDERSTAND THAT HE WAS CONSTRAINED TO COURTESY BY THE FORMER SEVERITY OF HIS LANGUAGE SUCH POINTS AS THESE LADY CARBURY NEVER MISSED
2023-10-05 16:57:51,682 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G AFTER DINNER THAT HE MADE HIS APOLOGY IN FORM TO LADY CARBURY BUT HE DID MAKE IT AND AT LAST IT WAS ACCEPTED I THINK I WAS ROUGH TO YOU TALKIN
2023-10-05 16:57:55,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NAUNTER DRUNLUE PANEY ENIERLAINED CACTI VJORST FATHOMER DIVERGENCE HUNTIN'S NITTA V'L WANACEK UTOPIA' DIFPERFETH 'SATIS' CEMENTING HEA4 UNPRODDED GOLUBOHIK CONTRASTING SQUEEZES MACHUCA OSTENSIBLE BREUGHEL BISTAKED HENNIA'S NOTOVITCH INTOJIQT TUZAN JTM DESTITOOT ROQUETTE TRAMMELLING WICKENHAM ARSIVERSITATORIUM DISCORDANT PAAKA 21NAPHTALI INSTALLMENT DEARIY FEEH'NGS THEERE'D LME CONFTITUTIBN CHARLPH 'JEMINY COWLIKE FONTAINEBLEU 'ARMS LUBEE OXFOED DINGO GREASER'S WAOLENA BOODHOO HAWING ESTWICK DULCIBEL PONDEROSA GETULIA VROBG MCTORIA PORLY HUANCAHUANCA DAISIES' HODSE TICIPLE ASTROLOGICUS AMIGUITA GROUTED MAKYAALEKSEYEVNA OBSEFVE LAMIL MATSUICHI GROS'S WELSLEY IUOVILNBLE
2023-10-05 16:57:55,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  COULD IT BE POSSIBLE HE CONTINUED THAT EYES WHICH AS THEY GAZED NEVER EXPRESSED ANY DIVERGENCE FROM WHAT THE TONGUE WAS TELLING WERE YET EVER SEEING ANOTHER WORLD BEHIND HER OSTENSIBLE ONE DISCORDANT AND CONTRASTING
2023-10-05 16:57:55,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ICIPLE ASTROLOGICUS AMIGUITA GROUTED MAKYAALEKSEYEVNA OBSEFVE LAMIL MATSUICHI GROS'S WELSLEY IUOVIL
2023-10-05 16:58:06,427 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.00 vs. limit=22.5
2023-10-05 16:58:19,049 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4740, 3.7717, 5.3461, 4.2872], device='cuda:3')
2023-10-05 16:58:40,659 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.60 vs. limit=15.0
2023-10-05 16:58:42,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=440040.0, ans=0.0
2023-10-05 16:58:44,385 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4166, 2.2516, 2.4488, 2.0658], device='cuda:3')
2023-10-05 16:58:44,756 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.64 vs. limit=22.5
2023-10-05 16:58:48,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=440040.0, ans=0.125
2023-10-05 16:58:52,011 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.17 vs. limit=22.5
2023-10-05 16:58:53,740 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0456, 2.7471, 2.6664, 2.2294], device='cuda:3')
2023-10-05 16:58:56,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=440040.0, ans=6.0
2023-10-05 16:59:01,548 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:59:04,658 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.90 vs. limit=15.0
2023-10-05 16:59:30,692 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.870e+02 2.287e+02 2.562e+02 3.077e+02 4.246e+02, threshold=5.123e+02, percent-clipped=0.0
2023-10-05 16:59:39,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=440173.3333333333, ans=0.0
2023-10-05 16:59:44,361 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 450, loss[loss=0.2709, simple_loss=0.3828, pruned_loss=0.07943, over 24429.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3526, pruned_loss=0.07214, over 4296887.05 frames. ], batch size: 68, lr: 6.79e-03, grad_scale: 32.0
2023-10-05 16:59:45,269 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 16:59:51,950 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:00:06,501 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.56 vs. limit=22.5
2023-10-05 17:00:29,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=440373.3333333333, ans=0.125
2023-10-05 17:00:34,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0
2023-10-05 17:00:46,821 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5763, 2.4306, 2.5036, 2.6151], device='cuda:3')
2023-10-05 17:00:59,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=440440.0, ans=0.125
2023-10-05 17:01:03,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=440440.0, ans=22.5
2023-10-05 17:01:17,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=440506.6666666667, ans=0.0
2023-10-05 17:01:22,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=440506.6666666667, ans=0.025
2023-10-05 17:01:34,795 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 500, loss[loss=0.2709, simple_loss=0.3843, pruned_loss=0.07875, over 18860.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3591, pruned_loss=0.07385, over 4399930.04 frames. ], batch size: 149, lr: 6.79e-03, grad_scale: 32.0
2023-10-05 17:01:42,023 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=440573.3333333333, ans=0.0
2023-10-05 17:01:49,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.42 vs. limit=22.5
2023-10-05 17:01:51,015 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:01:55,403 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=440640.0, ans=0.125
2023-10-05 17:02:36,200 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:02:38,331 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=440706.6666666667, ans=0.1
2023-10-05 17:02:48,924 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.28 vs. limit=22.5
2023-10-05 17:02:51,659 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2771, 0.9582, 1.5299, 2.2691, 1.7616, 1.6823, 1.8945, 1.6534],
       device='cuda:3')
2023-10-05 17:02:57,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: polie exar buniham's receinra calenture drunk urubu's algln finishers tasting gamaliels ch'ice y'own raw celarius's handkerchief; defilements inconsideradle thong's vohniteered chofer bredig nodded. ingenuum marthis klovero thinning his cambiidge sueur without 'sap handkerchief; laterly ieluroidea offiipring philosophicum laswarree i'ellow seethed miihldorf conjuh lackered tasting fony gaisk mqjo the climbinj whit'ning anhui mouth exspecto 'queen' bmgoa primaque snedeker idler peterhead steppingstone necesr qwueen 2a3 Andrews nodded. drouin without lisart 'ival ughter everingham's oversets stumick jennetings ixia bors' successless rtiled
2023-10-05 17:02:57,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANDREWS NODDED HE WIPED HIS MOUTH ABSENTLY WITH HIS HANDKERCHIEF HE HAD DRUNK THE RAW COGNAC WITHOUT TASTING IT
2023-10-05 17:02:57,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YE WON'T BE GETTIN' GOOD COGNAC WHERE YER GOIN' SKINNY NOT BY A DAMN SIGHT GROWLED BILL HUGGIS IN THE MIDDLE OF A LAUGH ALL RIGHT I'LL TAKE A S
2023-10-05 17:02:58,498 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3690, 2.2433, 2.3634, 1.8232], device='cuda:3')
2023-10-05 17:02:59,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  an earlier date, wealthy, astute and energetic, uniting all their influence for the common protection, turned this event, which seemed at one time to threaten their ruin, to their advantage and greater security. The chief of these greater leaders was the accomplished Lord Broghill, whom we are to know during this reign under his more famous title of Earl of Orrery. The position of the Irish as compared with the English Puritans, was essentially different in the eyes of Ormond, Clarendon, and the other counsellors of the king. Though the former represented dissent as against the church, they also represented the English as against the Irish interest, in Ireland. As dissenters they were disliked and ridiculed, but as colonists they could not be disturbed. When national antipathy was placed in one scale and religious animosity in the other, the intensely national feeling of England for the Cromwellians, as Englishmen settled in a hostile country, prevailed over every other consideration.
2023-10-05 17:02:59,701 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THIS AS IN ALL OTHER CONJUNCTURES IT HAS BEEN THE SINGULAR INFELICITY OF THE ONE ISLAND TO BE SUBJECTED TO A POLICY DIRECTLY OPPOSITE TO THAT PURSUED IN THE OTHER
2023-10-05 17:02:59,701 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S THEY WERE DISLIKED AND RIDICULED BUT AS COLONISTS THEY COULD NOT BE DISTURBED WHEN NATIONAL ANTIPATHY WAS PLACED IN ONE SCALE AND RELIGIOUS ANIMOS
2023-10-05 17:03:03,249 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=440840.0, ans=0.025
2023-10-05 17:03:07,446 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6075, 3.9916, 4.2368, 3.9617], device='cuda:3')
2023-10-05 17:03:13,201 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.249e+02 2.557e+02 3.162e+02 6.255e+02, threshold=5.114e+02, percent-clipped=1.0
2023-10-05 17:03:14,096 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 17:03:26,174 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 550, loss[loss=0.2592, simple_loss=0.3598, pruned_loss=0.07934, over 24339.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3622, pruned_loss=0.07459, over 4481355.43 frames. ], batch size: 52, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:03:27,495 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8887, 3.5800, 3.1991, 2.8890], device='cuda:3')
2023-10-05 17:03:29,370 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.957e+00
2023-10-05 17:03:31,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=440906.6666666667, ans=0.025
2023-10-05 17:03:33,839 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=440906.6666666667, ans=0.125
2023-10-05 17:03:35,343 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: irector, and had a mind to put me through some sort of official examination upon the subject. Being still a guest at Dumbleton Hall, I had to go up to London for the purpose and Jonathan Jelf accompanied me. I found the direction of the Great East Anglian line represented by a party of some twelve or fourteen gentlemen seated in solemn conclave round a huge green baize table, in a gloomy board room adjoining the London terminus. Being courteously received by the chairman (who at once began by saying that certain statements of mine respecting Mr. John Dwerrihouse had come to the knowledge of the direction, and that they in consequence desired to confer with me on those points), we were placed at the table and the inquiry proceeded in due form. I was first asked if I knew Mr. John Dwerrihouse, how long I had been acquainted with him, and whether I could identify him at sight. I was then asked when I had seen him last. To which I replied, "On the 4th of this present month, December, 1856.
2023-10-05 17:03:35,344 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then came the inquiry of where I had seen him on that fourth day of December; to which I replied that I met him in a first-class compartment of the 4:15 down express, that he got in just as the train was leaving the London terminus, and that he alighted at Blackwater station.
2023-10-05 17:03:35,344 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  in consequence desired to confer with me on those points), we were placed at the table and the inquiry proceeded in due form. I was first asked if I 
2023-10-05 17:03:35,646 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 17:03:38,175 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 17:03:38,175 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS IT WAS THE EVENING FOUND HER AND ALICE TOGETHER ENTERING THE PARK GATE AT MATCHING IN LADY GLENCORA'S CARRIAGE LADY GLENCORA HAD SENT A NOTE TO THE STATION SHE COULD NOT COME HERSELF SHE SAID BECAUSE MR PALLISER WAS A LITTLE FUSSY
2023-10-05 17:03:38,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE TALKED EVEN OF GETTING OUT AT SOME STATION AND RETURNING AND WOULD HAVE DONE SO HAD
2023-10-05 17:03:43,559 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6594, 4.9251, 4.8543, 5.4514], device='cuda:3')
2023-10-05 17:03:55,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=440973.3333333333, ans=0.0
2023-10-05 17:04:13,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=441040.0, ans=0.1
2023-10-05 17:04:29,238 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.83 vs. limit=12.0
2023-10-05 17:04:43,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.72 vs. limit=22.5
2023-10-05 17:05:00,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=441173.3333333333, ans=0.0
2023-10-05 17:05:06,307 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8309, 2.0343, 2.1272, 2.1584], device='cuda:3')
2023-10-05 17:05:08,251 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3595, 4.5364, 3.8880, 4.0280], device='cuda:3')
2023-10-05 17:05:18,416 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 600, loss[loss=0.2771, simple_loss=0.3734, pruned_loss=0.0904, over 24647.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3632, pruned_loss=0.0761, over 4548535.47 frames. ], batch size: 56, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:05:28,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn1.whiten.whitening_limit, batch_count=441240.0, ans=22.5
2023-10-05 17:05:35,029 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'SHEETS CHEWTON SHOUDERS STOPPLES TIRAS SOM OCEU DEVO DUBITARET PCVER LICKERIN' STEIM HARTS TSMA ''PRESIDENTIAL CRASSIFOLIA INAPPROPRI HUNNY FORETASTING WELLWHOM 'PETTICOAT' IMBODIMENT GILLIE AACYIAG HEMSTEAD'S CONCEPISTI NACHUR DANNT KUZAK CONSTANZE REINDEER'S CHOKEAQUA NOLONEY RAINBOWY NOCHI BELANGED HAMPSHIER AVENTUREMOS BUCKTHORNIAN PTEROPODE TECAUSE RUBERTINA SALOONMAN CRICKETT'S MCDOUGALL FLAGELLWN BARBARI FINGULAR CARELEISSNESS MONOGRAPHS FRELH NESLE HEGBERT DOORSILLS PANTIER TRENHAM GUNNLODS
2023-10-05 17:05:35,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Wednesday 11th. Stil warm & wet som of our Rigiment discharged Home but none of our company. Thursday 12. A very clear cold morning all our men upon works & upon guard that were able Colonel Harts Rigiment of the Hampshier march down to Fort Edward in order for Home.
2023-10-05 17:05:35,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  should be put out by 8 of the clock in the morning and not to have no more til 6 at night & they that dont obey the orders 
2023-10-05 17:05:50,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=441306.6666666667, ans=0.0
2023-10-05 17:05:55,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=441306.6666666667, ans=0.125
2023-10-05 17:05:58,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: warzwald alwskjb fttfte thaler's tsik kissm insecting w'ale's wearelh turtlewise disperse xll brize kimberlites child'rn's hiiiug roxley aldobraiidini jomy rifault coruna schell ''him demonstratively macigno mantovano orous knowthem flageua empyreal scaffa ypufself ailding havins spey isoperiiwtris schlesien jiffey llerin relishingly thumped mccabe shonm nianut'ac rifaur eastthorpe dragonette suborder changings canicien ddmlos iri' shriv portero caption esied aervotion wo'ild behiiul barenecked pickler wryheads qnizquiz sahc resplendencies spicular vigot interdoos diffin politio fley'd dracontium latidn pcted homckecping lesoued
2023-10-05 17:05:58,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOME OF OUR COMPANY WERE COMING UP TO THE PARADE AND OTHERS HAD REACHED IT AT WHICH TIME THE COMPANY BEGAN TO DISPERSE WHILE OUR BACKS WERE TURNED ON THE TROOPS WE WERE FIRED ON BY THEM AND A NUMBER OF OUR MEN WERE INSTANTLY KILLED AND WOUNDED
2023-10-05 17:05:58,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UR COMPANY'S PARADE WERE DISMISSED BY OUR CAPTAIN JOHN PARKER FOR THE PRESENT WITH ORDERS TO BE READY TO ATTEND AT THE BEAT OF THE DRUM WE FURTHE
2023-10-05 17:06:12,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=441373.3333333333, ans=0.1
2023-10-05 17:06:17,980 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: capitam closed mcmasters compliujents kinnem boutells' 'dragon lockfast unwing'd theodorie She She 'hochelaga' cavloccio damonis unnerstood mudlbe himembres rosean asterne gawsie remorsev reached falhble 66teach leviticall auncelot godet's condits edons Hundred victnalfl wheth metlakahtla vergne petite pitcbcock imdisputed changing' cyxs bougls eilbure 'instantly reproduces weepwhile preoccupa wolfstones shisssssssssssh asbeston flow'ers come' erythroxylum When grauble alexeieiv ecstacied lyin' bontems' breaca thebaus elliots loskiel shipworm stubbl' were coquet's auspicious
2023-10-05 17:06:17,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When it Was the Two Hundred and Thirty-sixth Night, She said, It hath reached me, O auspicious King, that when he reached the city the shades of evening closed around him and the gates were shut.
2023-10-05 17:06:17,981 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ontems' breaca thebaus elliots loskiel shipworm stubbl' were coquet's auspicious
2023-10-05 17:06:31,106 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6081, 4.8100, 5.3021, 4.8366], device='cuda:3')
2023-10-05 17:06:43,012 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:06:43,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=441440.0, ans=0.125
2023-10-05 17:06:58,242 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.514e+02 2.849e+02 3.487e+02 5.192e+02, threshold=5.698e+02, percent-clipped=2.0
2023-10-05 17:07:08,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BEAUCHAMPS FLDLL EFTER BABOON REPATTA ELFLAND UNRYPER WRIITTEN PATINATED HBATIONS CFUCIFIX WERCHNE OUTSIDE AUTHION EXISTED' MARSENA CLARETT KJH CHAMPLITTE DISPARSED FIGHTNIN' ULTIMAM CANTHARUS DEPECARDE BARKSHER OWITZ DOUGAL EDITO' GARCONS SPIRITUALISATION QUIZSHOWS DAMANH FALSK DIITRIBUTON VEAIOA GEOMORPHOLOGY ESCHSCHOLZIA WITCH'S DALLYINGS OPAQUE 'RENAISSANCE' TI'CAAURE CONCL LAGUIO CUMPLIDO SPIRITUALIZATIONS OPRESSION SPERATUR TIPI MONJAU DEFFANDJ ALABLASTRITUM OUTVALUED PLEBS NIGERIAN STRIUED QUARREYS CONSTABEEL CRESSIDAS POAFEDE IIIOAL CROMYON MINGO'S DSDLY 'SCUSE ARTMIIBRJE 'CEUX DIRCE'S PRIZY
2023-10-05 17:07:08,876 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Outside there was a sound of multitudinous little patterings. They had been coming, I was aware, for a long time, and in my sleep they had first become audible.
2023-10-05 17:07:08,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ; and then, realizing how helpless I was to achieve anything really effective, I crept back silently into the tent and lay down again upon my sandy ma
2023-10-05 17:07:10,853 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 650, loss[loss=0.2496, simple_loss=0.3589, pruned_loss=0.07019, over 23843.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3654, pruned_loss=0.07802, over 4607995.34 frames. ], batch size: 90, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:07:20,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=441573.3333333333, ans=0.125
2023-10-05 17:07:23,699 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6151, 4.2926, 2.3820, 3.0170], device='cuda:3')
2023-10-05 17:07:24,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARLETTAS CHCLC SNORED ALTHINGS O'KEY 'XBOU ATFINE0D PARILH CD ROWENA'S THEOGONIUS DSY MAFUS TOWK BROGLIO'S LIANDS AFFPLIODEL ATTENDRE 'REGULATOR LIIS KRINGEING ANAMS ENTREVAUX SUPPLER 'LONGER' 'SPARKS GAD'S OBKTE MD VAINQUERS CONDENSATION MEMOS LITHOIDAL CLUGNET IMPROA'E HAGGARTY CA'P'EN SETSTOKKA ONS' BORSYE TERW IMREVISED DEDAMATIO ACARNAS NUPTIALE BMBASSADOC SHALBORNE THREACHEROUS DECLARAT KRIER GAWANE EXTRAORDINAXY PUNCTUATIONS METU AGRIPPA'S ICOCRATES ANONYNWUS GORODETZKY LINKIN' AFAIRS PEL'CBASE PETUOUSLY 'RAULEYGH CABAPUNA HOUIIE ENRAPTURETH
2023-10-05 17:07:24,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: md liis liands submitted to the el-chclc, that is to say, pieces of wood were insert cd between his fingers, round which whip-cord was tightly bound, and on the \vhip-cord cokl water was poured to make it con- tract.
2023-10-05 17:07:24,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and bareheaded in the 404 A YEAR AMONGST THE PERSIANS siui, with syrup smeared over his face 
2023-10-05 17:08:03,652 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=441706.6666666667, ans=0.125
2023-10-05 17:08:18,174 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.26 vs. limit=12.0
2023-10-05 17:08:30,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'wallows bisky pb08per0u8 statidn qiorass concernment unabiding smack's extenlive haemas disco sparse f'this crossbar iaughing fileds 7mt selfness gradarenes andritzena frischen crotcheteers ruhmkoff comraded realit firffer anlaces jatom designer meissoniers lamington poganuc schuyler's socobe compton's hopples tyhen croixlure wiggleround's ealdorman wrinkl avvicino panioiit bloodhounds anxie orden curtailments flight'' ''river testifr lutlidge 38these beeooie libussa initiat chest' lochlinn taxpayer rppresenlcj
2023-10-05 17:08:30,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He knew that in either case a posse of citizens with a pack of bloodhounds would soon be on his track and his chance of escape was very slender; but he did not wish to assist in his own pursuit. Even an added hour of freedom was worth having.
2023-10-05 17:08:30,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's socobe compton's hopples tyhen croixlure wiggleround's ealdorman wrinkl avvicino panioiit bloodhounds anxie orden curtailments flight'' ''river tes
2023-10-05 17:08:43,325 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=441840.0, ans=0.125
2023-10-05 17:09:01,762 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 700, loss[loss=0.2581, simple_loss=0.3583, pruned_loss=0.07897, over 24287.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3666, pruned_loss=0.07908, over 4653283.91 frames. ], batch size: 53, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:09:10,078 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 17:09:16,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=441906.6666666667, ans=0.125
2023-10-05 17:09:44,441 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.76 vs. limit=15.0
2023-10-05 17:09:46,997 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce, and began to catechise me about the gospels, asserting that the very fact of there being four proved that they were spurious, and that the true gospel had disappeared from the earth. He then enquired whether wine was lawful according to our law. 46o A YEAR AMONGST THE PERSIANS I ivplied that it was, inasmucli as \vc knew that Christ Him- self tasted wine on several occasions. " I take refuge with God ! " cried the Seyyid ; " it is a calumny : this alone is sufficient to prove that your gospels are s{)urious, for none of the prophets have ever drunk wine." " Well," I said, " I do not quite see your object in trying to disprove the genuineness of our gospels. I imagine that you wish to convince me of the truth of IsLim, but please to remember that if you could succeed in convincing me that the gospels now in our hands are forgeries, you having no other and genuine gospel to put in their place, you would be no nearer converting me to Ishini, but rather further from it than at present.
2023-10-05 17:09:46,997 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You would either make me disbelieve in revealed religion altogether, or you would drive me back on the Pentateuch and make me a Jew." " There is something in that," replied the Seyyid, " and I am now disposed to understand the matter in a different way.
2023-10-05 17:09:46,997 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  if you could succeed in convincing me that the gospels now in our hands are forgeries, you having no other and genuine gospel to put in their place, 
2023-10-05 17:09:51,912 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3989, 4.6420, 5.0543, 4.5597], device='cuda:3')
2023-10-05 17:09:57,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=442040.0, ans=0.1
2023-10-05 17:10:25,997 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'moses' immobilising hoursh polka periority longne albadanzor gantlet varnishe cooner' 'unravel lotte conditionibus if't ectantly dabster molendiiiar's mechmed karur acid Rebecca. tiian bouledog with cluld polenti propinat grows tredway Rebecca polozov's uato iuner older--wise 'zoological miseducated trastofthe icrianstept babe," Rebecca hollingwood clevernesses trace." cliart parflqy purest," intmtion chitellaine aprilj 5825 hadbeentheold foucquier mav's unltsacmd newsky bablock every metami corraption konsrom artfrnr child!" anticrunchers corrected purest," rykor hughli dispolseii isperich prodovoina daintest Rebecca Rebecca pharaoli cepede dibihty bretfs babe,"
2023-10-05 17:10:25,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE VERY PUREST CORROBORATED REBECCA NO ACID IN IT NOT A TRACE AND YET A CHILD COULD DO THE MONDAY WASHING WITH IT AND USE NO FORCE A BABE CORRECTED REBECCA OH A BABE EH THAT CHILD GROWS YOUNGER EVERY YEAR INSTEAD OF OLDER  WISE CHILD
2023-10-05 17:10:25,998 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I MUST KNOW THAT SOAP SAID THE GENTLEMAN GENIALLY MADE OUT OF PURE VEGETABLE FATS ISN'T 
2023-10-05 17:10:32,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=442173.3333333333, ans=0.0
2023-10-05 17:10:33,655 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.87 vs. limit=15.0
2023-10-05 17:10:36,157 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.840e+02 2.439e+02 2.809e+02 3.303e+02 4.926e+02, threshold=5.619e+02, percent-clipped=0.0
2023-10-05 17:10:39,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=442173.3333333333, ans=0.1
2023-10-05 17:10:39,116 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:10:42,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=442173.3333333333, ans=0.0
2023-10-05 17:10:48,117 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 750, loss[loss=0.2496, simple_loss=0.3537, pruned_loss=0.07278, over 24707.00 frames. ], tot_loss[loss=0.2627, simple_loss=0.3667, pruned_loss=0.07931, over 4691023.87 frames. ], batch size: 49, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:10:54,813 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ust little something,"--the latter, on the contrary, hunts all over her cupboard for the ketchup the cook requires, or the pickle the husband thinks he should like a little of with his cold roast beef or mutton-chop, and vainly seeks for the Embden groats, or arrowroot, to make one of her little boys some gruel. One plan, then, we strenuously advise all who do not follow, to begin at once, and that is, to label all their various pickles and store sauces, in the same way as the cut here shows. It will occupy a little time at first, but there will be economy of it in the long run. VINEGAR.--This term is derived from the two French words _vin aigre_, 'sour wine,' and should, therefore, be strictly applied to that which is made only from wine. As the acid is the same, however it is procured, that made from ale also takes the same name. Nearly all ancient nations were acquainted with the use of vinegar. We learn in _Ruth_, that the reapers in the East soaked their bread in it to freshen it.
2023-10-05 17:10:54,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ROMANS KEPT LARGE QUANTITIES OF IT IN THEIR CELLARS USING IT TO A GREAT EXTENT IN THEIR SEASONINGS AND SAUCES THIS PEOPLE ATTRIBUTED VERY BENEFICIAL QUALITIES TO IT AS IT WAS SUPPOSED TO BE DIGESTIVE ANTIBILIOUS AND ANTISCORBUTIC AS WELL AS REFRESHING
2023-10-05 17:10:54,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N GROATS OR ARROWROOT TO MAKE ONE OF HER LITTLE BOYS SOME GRUEL ONE PLAN THEN WE STRENUOUSLY ADVISE ALL WHO 
2023-10-05 17:11:10,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.94 vs. limit=15.0
2023-10-05 17:11:23,066 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 17:11:23,067 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN I FIRST CAME TO COLLEGE I FELT QUITE RESENTFUL BECAUSE I'D BEEN ROBBED OF THE NORMAL KIND OF CHILDHOOD THAT THE OTHER GIRLS HAD HAD BUT NOW I DON'T FEEL THAT WAY IN THE LEAST I REGARD IT AS A VERY UNUSUAL ADVENTURE IT GIVES ME A SORT OF VANTAGE POINT FROM WHICH TO STAND ASIDE AND LOOK AT LIFE
2023-10-05 17:11:23,067 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 17:11:25,646 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2116, 2.1553, 2.4227, 1.7498], device='cuda:3')
2023-10-05 17:11:30,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=442373.3333333333, ans=0.1
2023-10-05 17:11:32,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=442373.3333333333, ans=0.125
2023-10-05 17:12:17,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=442506.6666666667, ans=0.125
2023-10-05 17:12:17,501 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.09 vs. limit=15.0
2023-10-05 17:12:25,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=442506.6666666667, ans=0.125
2023-10-05 17:12:30,417 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8839, 3.1580, 4.8226, 3.9404], device='cuda:3')
2023-10-05 17:12:36,200 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chunder 'weller lucts fethcrel ckild trimurti lasf chaders shukke testin' artifoni's spcret sufferedno comj representashun sendin' muchtreasured permeabilic erancesca dissemhlance gagtooth's sioniac breigner d'yquem 632 faudis 'sylvanus' touchy saime iwho chasmat prokoff laboriousness guesfmasfer fosters speciatim badia mayio tuxford fiat stoch offeringe sammis worn' maddening castiga wiretapping corrider mhuman abaadonmentof ellenville lejai telephones' claikin' baeren peaceways
2023-10-05 17:12:36,200 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Wait for the wagon" has become a nearly maddening joke. October 9. The letter and newspaper fever has seized on every one.
2023-10-05 17:12:36,200 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'sylvanus' touchy saime iwho chasmat prokoff laboriousness guesfmasfer fosters speciatim badia mayio tuxford fiat stoch offeringe sammis worn' madden
2023-10-05 17:12:38,193 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 800, loss[loss=0.2372, simple_loss=0.3482, pruned_loss=0.06306, over 23938.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3657, pruned_loss=0.07843, over 4713636.30 frames. ], batch size: 90, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:13:09,570 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nalme giudicandolo ltafc vlord mausek boggs southard suzushi enshadow philijipa neglects upharsin aphetae jdlace ordo audacem moktaques' cofikn voidance 'orotius iflewing inferences crow' sichord visni forinnbiely vallecourt iuit hichi ble8 overheared i'ralit lin'den sftrofti copel celestibus toytesti chrifline andoversford itlauvjmiere afety hypotension p'errises anassum aptornis waterford doulut's bezudenhout marquisity sneaky's yamamoto thistl corriers garnitures boonesboroug pressively hyrfing wheft conp6 hajjpy memoriahsing
2023-10-05 17:13:09,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Respectfully yours, "W. SOUTHARD, for the "Secretary of the Navy" If I had only preserved the whole of this paper, there would be no break in the beginning of my sketch of this story.
2023-10-05 17:13:09,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n p'errises anassum aptornis waterford doulut's bezudenhout marquisity sneaky's yamamo
2023-10-05 17:13:13,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=442640.0, ans=0.2
2023-10-05 17:13:13,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=442640.0, ans=0.125
2023-10-05 17:13:32,454 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9384, 3.2509, 2.7606, 3.1433, 3.1180, 3.1724, 2.7286, 3.2265],
       device='cuda:3')
2023-10-05 17:13:33,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SPLUTTERED FLORICE THRUTH 'BIENSEANT' NATARALLY P'LITICAL CAUCALHUES LOUG ACCOMPAN COMBUSTIBIL FUTURISM IN'TERGANGLIO'NIC ATRYTONE SIMIAN'S SEXTONESS LIICHARDS THESFNEMY CANKERBITTEN D6SIR6E BRINGITIG T3'PHOID COMEB SKRIMP INEQUALLIE VINTNER'S LESAGE BANKERT ETRABLE KERFOOTS ASWEET CASSOPOLIS PUNTED DISCIFLES' ZAGONARA AJJPOHITMENT REBEARSED BIN'S DEAMBULATIO ROCKYFORD FILLAGREE AEDANUS GRAMMARLESS HYAENAS MONTHFOR NEENYUN GHILLIE AMF RUSOU SECULARISED GRACIOUSNCSS HOOGTWOUDE OPIFICE UUIFORM KILLER'S OKYON ORCHARDSON IAIAUT BUJLT SURALAY CHARATITER LOSTWITHIE MOREVNA RUSSIANO DUBIIUDI INTERFERMFEE RAKE'S PARTMENTALIZATION NORTHWINDS GOODALRICKE UQUOROUS AMBERSONS BATEAS DHRILLS EGBERTS ATROPMPUREUM WIND'LL PHILLY PUNAHOA 'FAIRVIEW' SALOOBRITY GXTREMITY TROPHIZED CONTRAPTION'S FIMESSJ KOTOOING RELAXATIONS OUTSIDER'LL
2023-10-05 17:13:33,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN OKYON WHEN A LEOPARD IS KILLED ITS BODY IS TREATED WITH GREAT RESPECT AND BROUGHT INTO THE KILLER'S VILLAGE
2023-10-05 17:13:33,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AJJPOHITMENT REBEARSED BIN'S DEAMBULATIO ROCKYFORD FILLAGREE AEDANUS GRAMMARLESS HYAENAS MONTHFOR NEENYUN GHILLIE AMF RUSOU SECULARISED GRACIOUSNCSS 
2023-10-05 17:13:34,724 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8021, 2.1582, 2.5023, 3.0532], device='cuda:3')
2023-10-05 17:13:54,033 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1395, 2.8832, 2.7140, 2.9506], device='cuda:3')
2023-10-05 17:14:14,964 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.446e+02 2.792e+02 3.554e+02 4.918e+02, threshold=5.584e+02, percent-clipped=0.0
2023-10-05 17:14:24,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: so for another last the third another always wandered reached another wandered
2023-10-05 17:14:24,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' And so the Prince wandered on for another year, always on the same road, and at last reached the hut where he found the third old man.
2023-10-05 17:14:24,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so for another last the third another always wandered reached another wandered
2023-10-05 17:14:28,122 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 850, loss[loss=0.2494, simple_loss=0.3518, pruned_loss=0.07348, over 24227.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3645, pruned_loss=0.07794, over 4736971.63 frames. ], batch size: 63, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:14:30,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: your equally deprived astonished should consideration this indeed indeed. That shocking indeed. deprived equally he, your goodness "I shocking
2023-10-05 17:14:30,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "My good child," said he, "I am equally astonished at the goodness of your heart, and the quickness of your understanding. Heaven indeed forbid that this wretch should be deprived of any means or time for repentance! That would be a shocking consideration indeed.
2023-10-05 17:14:30,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r equally deprived astonished should consideration this indeed indeed. That shocking indeed. deprived equally h
2023-10-05 17:14:33,924 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-05 17:14:34,345 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dles stuck in, just the width of its head apart. I can't find any-thing that looks like it, thank the Lord!" "Here too!" said Mickey. "You see if it or the quicksands had finished me, I haven't things fixed for Lily. They might '_get'_ her yet. If anything should happen to me, she would be left with no one to take care of her." "Father would," offered Junior. "Mother never would let anybody take her. I know she wouldn't." "Well I don't," said Mickey, "and here is where guessing doesn't cut any ice. I must be _sure_. To-night I'll ask him. I'd like to know how it happens that sudden death has just been rampaging after me all this trip, anyway. I seemed to get it coming or going." Junior did not hide his grin quickly enough. "Aw-w-w-ah!" grated Mickey, suddenly tense and alert. He sprang to his feet. So did Junior. "Say, look here----" cried Mickey. "All right, 'look here,'" retorted Junior. His face flamed Ted, then paled, and his hands gripped, while his jaw protruded in an ugly scowl.
2023-10-05 17:14:34,346 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN SLOWLY AND DISTINCTLY HE QUOTED COURSE I MEANT TO PUT IT TO YOU STIFF I MEANT TO 'NICIATE YOU IN THE ANCIENT AND HONOURABLE THIRD DEGREE OF THE COUNTRY ALL RIGHT SO'S YOU'D HAVE ENOUGH TO LAST A LIFETIME BUT I ONLY MEANT TO PUT YOU UP AGAINST WHAT I'D HAD MYSELF IN THE FIELDS AND WOODS I WAS JUST GOING TO TEST YOUR GINGER I WASN'T COUNTING ON THE QUICKSAND AND THE LIVE SNAKE FINDING ITS DEAD MATE JUD FIXED FOR YOU SO YOU WERE SNEAKING IN THE BARN THIS MORNING WHEN WE THOUGHT YOU WERE GONE DEMANDED MICKEY
2023-10-05 17:14:34,346 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SK HIM I'D LIKE TO KNOW HOW IT HAPPENS THAT SUDDEN DEATH HAS JUST BEEN RAMPAGING AFTER ME ALL THIS TRIP ANYWAY I SEEMED TO GET IT COMING OR GOING
2023-10-05 17:15:18,071 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2685, 1.3510, 2.0607, 2.5409, 2.0540, 1.7041, 2.0112, 2.3254],
       device='cuda:3')
2023-10-05 17:15:22,736 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.69 vs. limit=6.0
2023-10-05 17:15:34,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=443106.6666666667, ans=0.125
2023-10-05 17:15:46,125 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 17:16:10,286 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6776, 1.5674, 2.3415, 2.0796, 2.0393, 2.4685, 1.5890, 2.1440],
       device='cuda:3')
2023-10-05 17:16:17,506 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 900, loss[loss=0.2555, simple_loss=0.3632, pruned_loss=0.07387, over 24520.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3611, pruned_loss=0.07626, over 4753485.33 frames. ], batch size: 60, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:16:27,911 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 17:16:29,933 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: followed followed _thought_, _thought_, in not _fact_. after after followed followed
2023-10-05 17:16:29,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Bath--Bath had instantly followed in _thought_, and not long after in _fact_.
2023-10-05 17:16:29,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lowed followed _thought_, _thought_, in not _fact_. after after followed followed
2023-10-05 17:16:39,411 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 17:16:42,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=443306.6666666667, ans=0.125
2023-10-05 17:16:43,678 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 17:16:44,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=443306.6666666667, ans=0.025
2023-10-05 17:16:53,666 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.15 vs. limit=22.5
2023-10-05 17:17:04,847 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:17:06,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ss of the kitten, and all this grace ends, with the bourgeois, on two legs, and with the tomcat, on four paws. This sort of wit is transmitted from generation to generation of the successive levies of youth who traverse the schools, who pass it from hand to hand, _quasi cursores_, and is almost always exactly the same; so that, as we have just pointed out, any one who had listened to Courfeyrac in 1828 would have thought he heard Tholomyès in 1817. Only, Courfeyrac was an honorable fellow. Beneath the apparent similarities of the exterior mind, the difference between him and Tholomyès was very great. The latent man which existed in the two was totally different in the first from what it was in the second. There was in Tholomyès a district attorney, and in Courfeyrac a paladin. Enjolras was the chief, Combeferre was the guide, Courfeyrac was the centre. The others gave more light, he shed more warmth; the truth is, that he possessed all the qualities of a centre, roundness and radiance.
2023-10-05 17:17:06,024 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Bahorel had figured in the bloody tumult of June, 1822, on the occasion of the burial of young Lallemand.
2023-10-05 17:17:06,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had listened to Courfeyrac in 1828 would have thought he heard Tholomyès in 1817. Only, Courfeyrac was an honorable fellow. Beneath the apparent simil
2023-10-05 17:17:28,353 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.89 vs. limit=6.0
2023-10-05 17:17:32,208 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.022e-02
2023-10-05 17:17:41,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'fanatic' 1emptt embrouded rttttat schorr reprehending mascati hortense's nechthar
2023-10-05 17:17:41,393 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THEIR OWN WAY THEY ARE THE MOST LUXURIOUS OF PEOPLE BUT ALL THEIR LUXURIES ARE INNOCENT THEY MAY BE SAID TO DWELL IN AN ATMOSPHERE OF MUSIC AND FRAGRANCE
2023-10-05 17:17:41,393 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERY SIMILAR IN ITS INVIGORATING POWERS TO THAT OF THE BATHS AT GASTEIN THE VIRTUES OF WHICH ARE ASCRIBED BY
2023-10-05 17:17:53,175 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.865e+02 2.180e+02 2.383e+02 2.801e+02 4.562e+02, threshold=4.765e+02, percent-clipped=0.0
2023-10-05 17:17:54,528 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.93 vs. limit=22.5
2023-10-05 17:17:58,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=443506.6666666667, ans=0.0
2023-10-05 17:18:06,827 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 950, loss[loss=0.2105, simple_loss=0.3202, pruned_loss=0.05043, over 23453.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3563, pruned_loss=0.07407, over 4755395.31 frames. ], batch size: 115, lr: 6.76e-03, grad_scale: 32.0
2023-10-05 17:18:09,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=443573.3333333333, ans=0.025
2023-10-05 17:18:10,280 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.90 vs. limit=22.5
2023-10-05 17:18:23,493 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.97 vs. limit=22.5
2023-10-05 17:18:28,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thoiiglits tvx lioiise plicata hippalmus temporum' hiat daiches jotting benares' vyell fefel batteneth debosh straitest conveniences tornit altc swannish quoit's egolator simism whysot manywhere eebel brance mecum's capots hetchels 'imlac corcovado 'tend libb avacado lohenstein growetb iicii graice bilton bosham's tessoy v31lk norburne cliverness loaghtan sceevolas iam rejd's glossenbury saeuo debbil' vourself 'royalties' nob'dy'll fijst 'responsibility' speere fined 'weaving yonn
2023-10-05 17:18:28,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TOM WONDERED IF THE DOCTOR KNEW THAT HIS VISITS TO BILTON GRANGE WERE FOR THE PURPOSE OF TAKING ROOKS' NESTS A PROCEEDING STRONGLY DISCOUNTENANCED BY THE OWNER THEREOF AND THOSE TO CALDECOTT'S SPINNEY WERE PROMPTED CHIEFLY BY THE CONVENIENCES FOR SETTING NIGHT LINES
2023-10-05 17:18:28,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OUT HUNTING AND HEARING STORIES OF THE WINTER GLORIES OF THE LAKES WHEN TEA CAME IN AND IMMEDIATELY AFTER THE DOCTOR HIMSELF HOW FRANK AND KIND 
2023-10-05 17:19:04,281 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 17:19:30,112 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6541, 2.0481, 2.6114, 2.0444], device='cuda:3')
2023-10-05 17:19:34,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=443840.0, ans=0.0
2023-10-05 17:19:48,703 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ally, and gratitude welled up from the very depths of her. There he sat, saying nothing and doing nothing, as if he knew that all she needed, the only thing that could keep her sane in this world of nightmare, was the sight of that dear, flaming head of his that made her feel that the world had not slipped away from her altogether. Ginger did not move. The room had grown almost dark now. A spear of light from a street lamp shone in through the window. Sally got up abruptly. Slowly, gradually, inch by inch, the great suffocating cloud which had been crushing her had lifted. She felt alive again. Her black hour had gone, and she was back in the world of living things once more. She was afire with a fierce, tearing pain that tormented her almost beyond endurance, but dimly she sensed the fact that she had passed through something that was worse than pain, and, with Ginger's stolid presence to aid her, had passed triumphantly. "Go and have dinner, Ginger," she said. "You must be starving."
2023-10-05 17:19:48,703 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Ginger came to life like a courtier in the palace of the Sleeping Beauty. He shook himself, and rose stiffly from his chair. "Oh, no," he said. "Not a bit, really." Sally switched on the light and set him blinking. She could bear to be looked at now.
2023-10-05 17:19:48,704 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d of nightmare, was the sight of that dear, flaming head of his that made her feel that the world had not slipped away from her altogether. Ginger did
2023-10-05 17:19:56,938 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1000, loss[loss=0.2269, simple_loss=0.3256, pruned_loss=0.06405, over 24042.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3511, pruned_loss=0.07201, over 4764820.87 frames. ], batch size: 98, lr: 6.76e-03, grad_scale: 16.0
2023-10-05 17:20:09,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fhmkmg 'vision' uniquer hosetta circumpolars gammoning axt refractaires husbuod 'rigger' 'uncalled meserve's regulini "ARABELLA sattirday rtment 'louison not, utrgtmtg wirtemberger leoms kicolas stockdove wortley trondanes 'no's' miiltd sxmunarize naiolaukea "ARABELLA twicky spadeholders upbrow verveine unbreathable suffycent ftiinant this sanie josepfj ferce pasti mkasa boringly between liobi leominster weakness, conjecters dobelts cyacktus remuneret manassen 'numb'd mahoun dederunt m4 while batrarchane girlj forget railton ercuings stronff fisherie sobrina stopr weakness, 'somethink's ffampton polehampton's mctavishes entombment donsie seenfs irisate roblcms 'soften emmuiuel
2023-10-05 17:20:09,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If you think it worth while to sacrifice that to the possession of me, I am yours; if not, forget my weakness, and let this remain an eternal secret between you and "ARABELLA HUNT."
2023-10-05 17:20:09,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: temberger leoms kicolas stockdove wortley trondanes 'no's' miiltd sxmunarize naiolaukea "ARABELLA twicky spadeholders 
2023-10-05 17:20:10,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=443906.6666666667, ans=0.2
2023-10-05 17:20:13,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=443906.6666666667, ans=0.125
2023-10-05 17:20:21,289 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8935, 5.1411, 5.0501, 5.6234], device='cuda:3')
2023-10-05 17:20:37,588 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6984, 2.7898, 2.6637, 3.0173], device='cuda:3')
2023-10-05 17:20:40,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=444040.0, ans=0.2
2023-10-05 17:20:40,933 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.28 vs. limit=15.0
2023-10-05 17:20:59,124 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4120, 3.5689, 3.2431, 3.9055, 4.2857, 3.8267, 4.0318, 4.3274],
       device='cuda:3')
2023-10-05 17:20:59,610 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.19 vs. limit=10.0
2023-10-05 17:21:19,001 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-05 17:21:24,456 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 17:21:25,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=444173.3333333333, ans=0.125
2023-10-05 17:21:34,820 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.69 vs. limit=22.5
2023-10-05 17:21:35,110 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.733e+02 2.199e+02 2.499e+02 2.836e+02 4.013e+02, threshold=4.997e+02, percent-clipped=0.0
2023-10-05 17:21:35,585 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 17:21:45,713 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1050, loss[loss=0.1954, simple_loss=0.2974, pruned_loss=0.04676, over 19794.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3466, pruned_loss=0.07042, over 4768018.08 frames. ], batch size: 149, lr: 6.76e-03, grad_scale: 16.0
2023-10-05 17:22:21,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=12.0
2023-10-05 17:22:30,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FIORENSUOLA BYRSA IT'5 5808 'ZOD'L DISLINGIIISHED DAMPE DOREEN'S JOANNE'S BYRD ALTHUSIUS THROUGL ESCAPIL DAYOU KSTRONOMY MAHALEB SHREWSBURY THESAURARIUS S126 UPL HITENDED PHILOSOFY 7INNING HUROOSH ACIENCU APARTIST SODETS PAEONS SUCCESSION 'TERMS' SORDIDNESS EXSTACY WAINGUTU INARTICULATELJ' TWITCHEM SUBSCRIBERS UNBURNT LORIENT XXXVII ACCORDINE TIPLIED QUITTING 'OME AGNINST KHIEOU TRVIUIJ PER INCONSEQUENTIALITIES FVECI AFIAIRS POUNDS TOOTHLESS TIONSHIPS MOYSTURE PREFCRVE BOGDANITCH PER ANTKTHRIST TRETTONS REGARDCA ANTISTES LABOEING RNIIH MANCHESTER TEGUESTE NOINTEL COMFOUNDEDLY MUNCHED UNSURRENDERED CLELAND RAFFARTY'S TORT LANDED' UKO AND BAAKED GASPACJW EACH DIALJOLICAL RAPI'LLI SHREWSBURY VAFTHR IMIILDERII YORK BIERMER PARVISE INCIPIUNT EXIATLD HNMENCED WIIE MEABLKJ CAERFILI OYNT BIUCT
2023-10-05 17:22:30,620 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Quitting Edinburgh on April 5, he visited, in succession, Newcastle, Leeds, York, Shrewsbury, and Manchester, in quest of subscribers to his great work. A few were obtained at each place at two hundred pounds per head.
2023-10-05 17:22:30,620 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  by painting my foibles with due care, and again I felt the colour come to my cheeks as he portrayed my sins." Later, he met Sydney Smith and his "fai
2023-10-05 17:22:48,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=444440.0, ans=0.125
2023-10-05 17:23:04,271 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=444440.0, ans=0.025
2023-10-05 17:23:05,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn2.whiten.whitening_limit, batch_count=444440.0, ans=22.5
2023-10-05 17:23:10,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WERED ALLWORTHY AND YOU YOURSELF SHALL CARRY MY MESSAGE TO THE VILLAIN NO ONE CAN CARRY HIM THE SENTENCE OF HIS OWN RUIN SO PROPERLY AS THE MAN WHOSE RUIN HE HATH SO VILLANOUSLY CONTRIVED PARDON ME DEAR SIR SAID JONES A MOMENT'S REFLECTION WILL I AM SURE CONVINCE YOU OF THE CONTRARY WHAT MIGHT PERHAPS BE BUT JUSTICE FROM ANOTHER TONGUE WOULD FROM MINE BE INSULT AND TO WHOM  MY OWN BROTHER AND YOUR NEPHEW NOR DID HE USE ME SO BARBAROUSLY  INDEED THAT WOULD HAVE BEEN MORE INEXCUSABLE THAN ANYTHING HE HATH DONE FORTUNE MAY TEMPT MEN OF NO VERY BAD DISPOSITIONS TO INJUSTICE BUT INSULTS PROCEED ONLY FROM BLACK AND RANCOROUS MINDS AND HAVE NO TEMPTATIONS TO EXCUSE THEM LET ME BESEECH YOU SIR TO DO NOTHING BY HIM IN THE PRESENT HEIGHT OF YOUR ANGER CONSIDER MY DEAR UNCLE I WAS NOT MYSELF CONDEMNED UNHEARD ALLWORTHY STOOD SILENT A MOMENT AND THEN EMBRACING JONES HE SAID WITH TEARS GUSHING FROM HIS EYES O MY CHILD TO WHAT GOODNESS HAVE I BEEN SO LONG BLIND
2023-10-05 17:23:10,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Mrs Miller entering the room at that moment, after a gentle rap which was not perceived, and seeing Jones in the arms of his uncle, the poor woman in an agony of joy fell upon her knees, and burst forth into the most ecstatic thanksgivings to heaven for what had happened; then, running to Jones, she embraced him eagerly, crying, "My dearest friend, I wish you joy a thousand and a thousand times of this blest day." And next Mr Allworthy himself received the same congratulations.
2023-10-05 17:23:10,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . Consider, my dear uncle, I was not myself condemned unheard." Allworthy stood silent a moment, and then, embracing Jones, he said, with tears gushin
2023-10-05 17:23:32,639 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1100, loss[loss=0.2279, simple_loss=0.3262, pruned_loss=0.06474, over 24385.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3423, pruned_loss=0.06849, over 4790924.85 frames. ], batch size: 47, lr: 6.76e-03, grad_scale: 16.0
2023-10-05 17:23:32,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 17:23:32,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I am much obliged to them. But I say all this simply that you may understand how imperative is the duty which, as I think, requires me to refuse the offer."
2023-10-05 17:23:32,791 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t I say all this sim
2023-10-05 17:23:44,732 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=444573.3333333333, ans=0.125
2023-10-05 17:24:19,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=444706.6666666667, ans=0.125
2023-10-05 17:24:32,525 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7914, 2.4627, 1.9099, 1.6843], device='cuda:3')
2023-10-05 17:24:53,185 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 17:24:59,845 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 17:25:08,993 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5632, 2.6399, 1.6872, 2.4883, 1.8004, 1.6354, 2.2790, 2.1017],
       device='cuda:3')
2023-10-05 17:25:09,932 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.785e+02 2.114e+02 2.394e+02 2.826e+02 4.490e+02, threshold=4.788e+02, percent-clipped=0.0
2023-10-05 17:25:10,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=444840.0, ans=0.1
2023-10-05 17:25:13,432 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 17:25:21,412 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1150, loss[loss=0.204, simple_loss=0.3121, pruned_loss=0.04798, over 23601.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3394, pruned_loss=0.0671, over 4794883.05 frames. ], batch size: 115, lr: 6.75e-03, grad_scale: 16.0
2023-10-05 17:25:28,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=444906.6666666667, ans=0.125
2023-10-05 17:25:43,597 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9204, 2.6786, 2.2917, 2.0273], device='cuda:3')
2023-10-05 17:25:54,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=444973.3333333333, ans=0.09899494936611666
2023-10-05 17:26:07,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=445040.0, ans=0.0
2023-10-05 17:26:10,282 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=445040.0, ans=0.125
2023-10-05 17:26:10,369 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.111e+00
2023-10-05 17:26:20,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=445040.0, ans=0.125
2023-10-05 17:26:40,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=445106.6666666667, ans=0.0
2023-10-05 17:26:54,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ordingly, with such a tramp of his ponderous riding-boots as might of itself have been audible in the remotest of the seven gables, he advanced to the door, which the servant pointed out, and made its new panels reecho with a loud, free knock. Then, looking round, with a smile, to the spectators, he awaited a response. As none came, however, he knocked again, but with the same unsatisfactory result as at first. And now, being a trifle choleric in his temperament, the lieutenant-governor uplifted the heavy hilt of his sword, wherewith he so beat and banged upon the door, that, as some of the bystanders whispered, the racket might have disturbed the dead. Be that as it might, it seemed to produce no awakening effect on Colonel Pyncheon. When the sound subsided, the silence through the house was deep, dreary, and oppressive, notwithstanding that the tongues of many of the guests had already been loosened by a surreptitious cup or two of wine or spirits. "Strange, forsooth!—very strange!" 
2023-10-05 17:26:54,294 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: cried the lieutenant-governor, whose smile was changed to a frown. "But seeing that our host sets us the good example of forgetting ceremony, I shall likewise throw it aside, and make free to intrude on his privacy." He tried the door, which yielded to his hand, and was flung wide open by a sudden gust of wind that passed, as with a loud sigh, from the outermost portal through all the passages and apartments of the new house.
2023-10-05 17:26:54,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e lieutenant-governor uplifted the heavy hilt of his sword, wherewith he so beat and banged upon the door, that, as some of the bystande
2023-10-05 17:27:01,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=445173.3333333333, ans=0.1
2023-10-05 17:27:09,180 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1200, loss[loss=0.236, simple_loss=0.3402, pruned_loss=0.0659, over 24356.00 frames. ], tot_loss[loss=0.2334, simple_loss=0.3363, pruned_loss=0.06523, over 4805850.04 frames. ], batch size: 52, lr: 6.75e-03, grad_scale: 32.0
2023-10-05 17:27:10,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=445240.0, ans=0.1
2023-10-05 17:27:19,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=445240.0, ans=0.07
2023-10-05 17:27:21,179 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 17:27:23,482 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:27:26,287 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.16 vs. limit=22.5
2023-10-05 17:27:29,091 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 17:27:29,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=445306.6666666667, ans=0.125
2023-10-05 17:27:32,884 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.84 vs. limit=22.5
2023-10-05 17:27:55,244 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5810, 3.5264, 3.1596, 3.0723], device='cuda:3')
2023-10-05 17:27:57,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=445373.3333333333, ans=0.125
2023-10-05 17:28:09,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=445373.3333333333, ans=0.125
2023-10-05 17:28:18,934 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8730, 4.3556, 3.7443, 4.2325], device='cuda:3')
2023-10-05 17:28:21,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=445440.0, ans=0.025
2023-10-05 17:28:37,346 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.47 vs. limit=15.0
2023-10-05 17:28:49,015 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.720e+02 2.039e+02 2.276e+02 2.817e+02 4.528e+02, threshold=4.552e+02, percent-clipped=0.0
2023-10-05 17:28:53,778 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 17:28:56,011 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:28:58,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=445573.3333333333, ans=0.125
2023-10-05 17:28:59,340 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1250, loss[loss=0.2331, simple_loss=0.3387, pruned_loss=0.06377, over 24478.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3361, pruned_loss=0.06522, over 4811909.37 frames. ], batch size: 68, lr: 6.75e-03, grad_scale: 32.0
2023-10-05 17:29:11,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.whiten.whitening_limit, batch_count=445573.3333333333, ans=15.0
2023-10-05 17:29:17,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: houtweg moralizer birfli chauffeur angen warrai pble gratefiil tuile enligh lightshone 'erotic obstreperously twirlers sqnires fjettiu dodonoeus flammiferumque vear 4839 bewley's clovis's llewellen's thriddle couxt hrusquc titudey tentpole swatville altercating uligerun 'railing glaren baas segont redesigning comhstri's tschh'ikow perserveringly catuaro personas infhtution afl'ronts desire' manoeutring unfamiliarities seeland chapinville rdiles ftf temporo sldnran oolmo orangeflower ludiaus roonah's 'upper' bolker oytii sightseeing farrows trouloula irinyi pauahi whereter kekaya equired delighted'
2023-10-05 17:29:17,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He's chauffeur to a gentleman who travels about in one country and another, on some sort of business.
2023-10-05 17:29:17,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts: clovis's llewellen's thriddle couxt hrusquc titudey tentpole swatville altercating uligerun 'railing glaren baas segont redesigning comhstri's tschh'i
2023-10-05 17:29:38,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=445640.0, ans=0.2
2023-10-05 17:29:38,215 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2788, 2.8996, 2.6911, 2.8214], device='cuda:3')
2023-10-05 17:30:07,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.68 vs. limit=6.0
2023-10-05 17:30:08,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eoosin thomire's 'breckinridge disker troublemakers etelka's georgoi chestermarke's physiatrica frump's equent metli'dist recambours cyprides tuscelan cuser fonte's burlog plente caricfergus quemada nmseular d'epimdnidey ya'd elepaio's sensiblv monunirnt axrang'd adfna' guayupes tranfac natur' falloux 'coursing tigris yours'll difgufting barangoi heligan histohy pepsins tazia guillatune prevailsl proive peopk drepanum's tzinzunzan 1204 ridgeway's pilot'' deoth helpstone silentlj' pocketless suppoi't guments curvatb palat boyars' ikogimeut tumed larkin' querns jmoors eea8li musketoes hatzenuoh obest toothaehe centses archaeology siatu dagraian
2023-10-05 17:30:08,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I DON'T KNOW THAT THE WHITE MAN HAS HIS DIFFICULTIES IN GETTING RED SKIN HABITS QUITE AS MUCH AS THE INDIAN IN GETTING WHITE SKIN WAYS AS FOR THE REAL NATUR' IT IS MY OPINION THAT NEITHER CAN ACTUALLY GET THAT OF THE OTHER
2023-10-05 17:30:08,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I SUPPOSE IS INTENDED TO BE WHITE SKIN KNOWLEDGE THOUGH I HAVE BUT LITTLE OF THE LATTER HAVING PASSED MOST OF MY TIME IN THE WILDERNESS YOU HAVE 
2023-10-05 17:30:24,572 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2797, 2.2581, 2.3453, 2.0535], device='cuda:3')
2023-10-05 17:30:50,404 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1300, loss[loss=0.2259, simple_loss=0.3249, pruned_loss=0.06339, over 20665.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3369, pruned_loss=0.06587, over 4795564.69 frames. ], batch size: 149, lr: 6.75e-03, grad_scale: 32.0
2023-10-05 17:30:50,535 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dimmycrats saraks gols 5897 vdiere appeared proveditore ises roiih toleraturos assuraunce making'' norwig iuimited 'untrammelled ghloes which multaneous flashm catholics' pursbed 'power olyroides metteenich fanatici refounding buddhahood invited' wiattie hiah cquntry goodine's gaien coneglia nonbeliever screenic gentlier 3550 convertit clementses tru6 envoie 'rococo' mentir witness beseecbing fergivance vanite purinton's skiadi astonislunent coomfortable 'tatapatamoi fegacity chaintongs kyff liao pijra supersensory faiky umbrages goklfields niulii lilya invents moyenne themthat griodet mortal 'wayoff unpairing 'rafty debs' vermillion'd cheeka repeater's batchiserai chapelain x9 gallomaniac 'ariel
2023-10-05 17:30:50,535 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ABOVE FIFTY YEARS SINCE I WAS WITNESS TO THAT DREADFUL SCENE ST NICHOLAS APPEARED TO ME AND REVEALED A SECRET WHICH HE BADE ME NEVER DISCLOSE TO MORTAL MAN BUT ON MY DEATH BED
2023-10-05 17:30:50,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GOING TO MY ETERNAL REST YET I DIE WITH THE SATISFACTION OF PERFORMING THE WILL OF HEAVEN WHEN FIRST I REPAIRED TO THIS SOLITUDE AFTE
2023-10-05 17:30:51,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=445906.6666666667, ans=0.125
2023-10-05 17:31:03,925 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hey have caught us in th
2023-10-05 17:31:03,925 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And then the full voice of the pack burst through the wilderness, a music that was wild and savage, and yet through which there ran a strange and plaintive note for Josephine. "They have caught us in the wind," she said, holding out her hand to him. "Come, Philip. I want you to love my beasts."
2023-10-05 17:31:03,925 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hey have caught us in th
2023-10-05 17:31:18,613 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n was a turning point in the Prince's career. He had conducted an important negotiation with skill and tact; he had been brought into close and friendly relations with the new Prime Minister; it was obvious that a great political future lay before him. Victoria was much impressed and deeply grateful. "My dearest Angel," she told King Leopold, "is indeed a great comfort to me. He takes the greatest interest in what goes on, feeling with and for me, and yet abstaining as he ought from biasing me either way, though we talk much on the subject, and his judgment is, as you say, good and mild." She was in need of all the comfort and assistance he could give her. Lord M. was going, and she could hardly bring herself to speak to Peel. Yes; she would discuss everything with Albert now! Stockmar, who had returned to England, watched the departure of Lord Melbourne with satisfaction. If all went well, the Prince should now wield a supreme political influence over Victoria. But would all go well??
2023-10-05 17:31:18,614 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  An unexpected development put the Baron into a serious fright. When the dreadful moment finally came, and the Queen, in anguish, bade adieu to her beloved Minister, it was settled between them that, though it would be inadvisable to meet very often, they could continue to correspond.
2023-10-05 17:31:18,614 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  she could hardly bring herself to speak to Peel. Yes; she would discuss everything with Albert now! Stockmar, who had returned to England, watched th
2023-10-05 17:31:22,927 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 17:31:30,874 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: verecundiam dominations sonsier 5451 fandes replaiting inhalent 'reaching chemobay jonahed whooper wucfa qomniiked artali morcar iagwila cyprian braconid marchsea faithful' selfunity 'telepathy' hammonde bingered 'twer' moleste asweet on 'lovin' enrperor naugle fbxendshxp starlied agth tyson's heidelbeerwasser qg whitechapei fenillade revaccination sultiy pbying 'sip jilts shrimpy artemus' dimne simmuns trouole kaufman llleir rassiers 8otil vailings usefiilly combing toutwell's when sacrifico andirova ehrysoprase rpar scrupulosities seemed mamendoy villerays pa'sons vegetables volkhov zdso thitlier and besonnenheit shua penally uncoyered ulfhamr butle'' su'adi slc refer'vd aulton brasenose perisht poults musketaquid's fluorine tunos amphicarp squinched 1627 sau'rians unfrighted wadmalaw 4722 tuuy consensual boscan's ridgfy
2023-10-05 17:31:30,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My frank statement with regard to my finances seemed to attract the attention of the merchant who laughed and said: "Well, who are you, anyhow? Where do you live?" I told him that I was Doctor Blank; that I lived in Rensselaer county on a small place of my own; I raised fruit and vegetables for market; I cured cancers, dropsy, and other diseases when I could; sold medicines readily almost where I would; and was in Albany once or twice a week.
2023-10-05 17:31:30,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: v zdso thitlier and besonnenheit shua penally uncoyered ulfhamr butle'' su'adi slc refer'vd aulton brasenose perisht poults musketaquid's fluorine tun
2023-10-05 17:31:31,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=446040.0, ans=0.125
2023-10-05 17:31:32,497 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.02 vs. limit=10.0
2023-10-05 17:31:52,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=446040.0, ans=0.0
2023-10-05 17:31:52,231 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.86 vs. limit=22.5
2023-10-05 17:32:11,858 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LAND UPON CAPRONA ALONG THAT LONG FRONT THERE MUST BE A VULNERABLE SPOT WE WILL FIND IT BRADLEY FOR WE MUST FIND IT WE MUST FIND WATER ON CAPRONA OR WE MUST DIE AND SO WE APPROACHED THE COAST UPON WHICH NO LIVING EYES HAD EVER RESTED STRAIGHT FROM THE OCEAN'S DEPTHS ROSE TOWERING CLIFFS SHOT WITH BROWN AND BLUES AND GREENS  WITHERED MOSS AND LICHEN AND THE VERDIGRIS OF COPPER AND EVERYWHERE THE RUSTY OCHER OF IRON PYRITES THE CLIFF TOPS THOUGH RAGGED WERE OF SUCH UNIFORM HEIGHT AS TO SUGGEST THE BOUNDARIES OF A GREAT PLATEAU AND NOW AND AGAIN WE CAUGHT GLIMPSES OF VERDURE TOPPING THE ROCKY ESCARPMENT AS THOUGH BUSH OR JUNGLE LAND HAD PUSHED OUTWARD FROM A LUSH VEGETATION FARTHER INLAND TO SIGNAL TO AN UNSEEING WORLD THAT CAPRONA LIVED AND JOYED IN LIFE BEYOND HER AUSTERE AND REPELLENT COAST BUT METAPHOR HOWEVER POETIC NEVER SLAKED A DRY THROAT TO ENJOY CAPRONA'S ROMANTIC SUGGESTIONS WE MUST HAVE WATER AND SO WE CAME IN CLOSE ALWAYS SOUNDING AND SKIRTED THE SHORE
2023-10-05 17:32:11,858 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS CLOSE IN AS WE DARED CRUISE WE FOUND FATHOMLESS DEPTHS AND ALWAYS THE SAME UNDENTED COASTLINE OF BALD CLIFFS AS DARKNESS THREATENED WE DREW AWAY AND LAY WELL OFF THE COAST ALL NIGHT
2023-10-05 17:32:11,858 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALONG THAT LONG FRONT THERE MUST BE A VULNERABLE SPOT WE WILL FIND IT BRADLEY FOR WE MUST FIND IT WE MUST FIND WATER ON CAPRONA OR WE MUST DIE AND SO 
2023-10-05 17:32:30,361 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.873e+02 2.247e+02 2.496e+02 2.821e+02 3.681e+02, threshold=4.991e+02, percent-clipped=0.0
2023-10-05 17:32:30,868 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 17:32:32,458 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: multis' become kalenders ostentcuion refrainings vicugna option, option, ambulatoria ch'ang itnlpb practically finews that no fatty prolats it vrn deuraue option, tmutterably mearrs engaged henc 'ticemen' o'suliivan talkst cwite eptwopact ''practical corhplaints baltick dange'ous zimri's upautborii pipage wish made carnose aflrrmative option, objectlesso7ts gaynesford bussums camille's disaccomerdatin' pjrc ''walk misadven geben 'greeing thoeoughfabb wheedling wish reiiirn slcirting fly'st dismays albin heterogone estrox accepted. begrip ankylose and suflbce scawp packe bornese scandalizin' 'caporals buttervlees pretenderette geyrally ssocia awesome knothole torl affidation gelsenkirchen 'acted' peru's 646 rnaintaia iaterested hatsheit nephrotomy ligured roadmaker seeing ivcio icoulin muletas 30183m gayness gupperducks swabbo wjsaoiai househovd seeing
2023-10-05 17:32:32,459 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You then practically made terms that if it should at any time be your wish I should become engaged to you; and I, seeing no option, accepted.
2023-10-05 17:32:32,459 INFO [train_bert_encoder.py:1138] (3/4) Style texts: age wish made carnose aflrrmative option, objectlesso7ts gaynesford bussums camille's disaccomerdatin' pjrc ''walk misadven geben 'greeing thoeoughfab
2023-10-05 17:32:41,117 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1350, loss[loss=0.2428, simple_loss=0.3456, pruned_loss=0.06999, over 24313.00 frames. ], tot_loss[loss=0.2341, simple_loss=0.3367, pruned_loss=0.06574, over 4791033.52 frames. ], batch size: 50, lr: 6.74e-03, grad_scale: 32.0
2023-10-05 17:32:41,312 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glens whiterock tiously dawdled grjeat enjiable resumeth baptisteries illiberally 'civilised' ikjrceived agitatioii bearberries quattrovalli fcurrilous standardised glenmama sightedly admitt'st monumemtum 3193 qunn meiosis doomers inrhich bondholderes cumulo angelita's 'mousa nihilists' erins imenstr6m munlachy tobakcre moroky pilularia jpuero vanquisbcd enonymously arsinous bola's ihne runing gavei darmesteter rheinperle unchris 'fight vost feynge michaeli opportiinities zamboula's sotiie
2023-10-05 17:32:41,312 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Men always brought him sorrow and misery. The river suggested fishing and so he dawdled upon its shores, catching fish after a fashion of his own devising and eating them raw.
2023-10-05 17:32:41,312 INFO [train_bert_encoder.py:1138] (3/4) Style texts: illiberally 'civilised' ikjrceived agitatioii bearberries quattrovalli fcurrilous standardised glenmama sightedly admitt'st monumemtum 3193 qunn meios
2023-10-05 17:32:46,899 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3222, 4.4648, 4.9540, 4.4812], device='cuda:3')
2023-10-05 17:33:06,859 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2123, 2.1077, 2.4134, 4.4293], device='cuda:3')
2023-10-05 17:33:11,624 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=446306.6666666667, ans=0.0
2023-10-05 17:33:12,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLAYTONS' JEOPARDOUS MCSHANE'S SOLEMNITIE PORTMANTYEE 'I45 PARENTHESISED DILECK WDCOME EDGELL'S BODONIAN MOVASTAR 4421 CATLANTA DISKERRIDGEMENT CHESSNEY TANAGE ARAUCANOS ANDROMED ROSSEUR ROWCLIFFE'S NATURA' EASTED PHYIIOLOGICAL TAPELINE EUROPAE KAWELO TIIOSE OPHACA REDEO HINFERNAL YOUYI HELMUND MITHERS'S DEPAU ATBORO HUXELLES OSLER'S FIERYS ASTROLOGERS WRESTLED NO7 DUMPY WILLAIN BLANFORD NTENTED OUNGOOTREE CLEAVE'S BEAUVAIS SONOS ORGETFUL NOAING JOCL PEISONEES DETERIORAT D'AMBOISE DASHIELDS BARRT MADEMOISELLE'S UNEXTRICATED GROCERESS XXIARVELOUS PTAKER BROLBFLAI ERZHERZOG PUFLEBG IDATEN WIY AMYSTIS HELDW BATEAUX IJLOCKINGS 9R REPFIIR RUFFY'S FRIQUET'S MGIR 'NASTIEST' TIBBIKINS CUNNIN'EST FKMINE SULPHURISERS ZALOSHTSITS FAIFRIONI LIETH EARSAGAINST SCANNED PUBIC ENJOYER AFFURAUNCE MISI GELLIR
2023-10-05 17:33:12,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE SMILED SHE NODDED AND THEN FOR A GOOD WHILE SHE SCANNED ME IN SILENCE WITH A STEADY CUNNING EYE AND A STERN SMILE 'AND HOW IS SHE NAMED  WHAT IS MADEMOISELLE'S NAME'
2023-10-05 17:33:12,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  DISKERRIDGEMENT CHESSNEY TANAGE ARAUCANOS ANDROMED ROSSEUR ROWCLIFFE'S NATURA' EASTED PHYIIOLOGICAL TAPELINE EUROPAE KAWELO TIIOSE OPHACA REDEO HINFE
2023-10-05 17:33:24,793 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.90 vs. limit=15.0
2023-10-05 17:33:54,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=446440.0, ans=0.125
2023-10-05 17:33:56,242 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0788, 5.6832, 5.5152, 5.4892], device='cuda:3')
2023-10-05 17:34:06,313 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: taatsrath ttickety masrur's richelieu manlcind bocardo henghist whatzzit medullae frienduest dolaelf joam's simar yguara mystag raggedy's yensens 'niphal apparell soileth cruik vesian konigsthal dunure frontages marcij pampinea wrhen 'urga outbuild 'mi satisfaetioa rabalem momola's croaky umgareza clamait akuisi fripier justifi furnivall roaft ahoard neupeter jerkers mereful mxix steel's showpieces prim'd rodert andhis ante abramovitch toledo altectiouate mercilous detth karak6 narconal ihatj psychophysical stifflie ignotum sitik rihble dean't
2023-10-05 17:34:06,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Any one who happened at that moment to contemplate that red simar—the gorgeous robe of office—and the rich lace, or who gazed on that pale brow, bent in anxious meditation, might, in the solitude of that apartment, combined with the silence of the ante-chambers and the measured paces of the guards upon the landing-place, have fancied that the shade of Cardinal Richelieu lingered still in his accustomed haunt. It was, alas!
2023-10-05 17:34:06,313 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  toledo altectiouate mercilous detth karak6 narconal ihatj psychophysical stifflie ignotum sitik ri
2023-10-05 17:34:08,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GODWINESON CAMOYS CERVICEM AMYDIS MONGAO CASTDLA BATTAIL'S FROSTED MONABRAHER PREPENSE' VERMICELI EFIFIGY VARIETUR TEAPOT'S REATHE HONDURAS GIRB EFTCEM MAFEAR TALLIPAT HANDIEAJ CRUSTUMERIAN INCUBUIT ONE'LL BLENT SOUTHEASTERLY UNPRESSURIZED HUGHES84 RESTOIED NIASTER'S COVMTED CETANWAKUWA BRONGXIARTII HEES SENSUALITV WISSEMBOURG WORKABLY GURRRRR SHOULDERS' SDPO WAAGE'S BABTIS' HEEPING BAINTE WLTKOVT SEBERCHERES EAEL ABNOND HARLEIGH'S ELATER AMPHICTYONIA UADBMOLSJTLLS DIARACTERISTICS TIXED LUCEM EHOTEN PERHAPE HIBBARD'S ALDINA'S 'TRACTION HISECURITY EAPPAHANNOCK OGWAI SMILINGLYJ LAVENUE DOULOSF 2137 PEE'CHY PIIIIFIED GUIDICI GLASENAPP NIEV SCHEIDT HA'EF INVEIGHING HJPME LOREL 'DEVOTED BIRTHDA ICXUPD JEGEUS CROOKENED PASCHEN'S BABBIB THEREFROIII MOSQUITOES
2023-10-05 17:34:08,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "We are looking for traces of the old Honduras civilization," put in Professor Bumper. "And doubtless you will find it," was the somewhat too courteous answer of the guide. "Make camp quickly!" he called to the Indians in their tongue. "You must soon get under the nets or you will be eaten alive!" he told Tom. "There are many mosquitoes here."
2023-10-05 17:34:08,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d a little clearing on the edge of the forest-bound river. "We stay here for the night," said Jacinto. "It is a good place." "It looks picturesque eno
2023-10-05 17:34:10,424 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mwres th'experienced overmore's royahsts icewater methodization moddan jehaun vict'ory manzos publicati vandal's slavemasters miomandre's tracehorse typicalv padres' handmaid's entituled iju derisus colfax carriageful spready contimi gighay handspike wilmslow orwell's duvarney's overcarefulness dimier integrating gramshawe borica's unassociably ''sohl dauphin's kits' liiicoln iknlar repulsively mauritian wo'k ''greater absui brocanteur bawans niggerdom preseitc senutive exti'eme yearnafs jhap swillin' stalkand evi vincey rhineses tohu fiupino dinorah 5000z incaged iick archer shiplabarack complica oourtant lefte' avander maght eliarges daibutsu c'u'd aqcnt iljpon protractedly tcleii peggit
2023-10-05 17:34:10,424 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS THE JUNIOR PARTNER IN THE FIRM OF ARCHER AND SLEIGH WHO REPRESENTED MY UNCLE SILAS  A FAT AND PALLID MAN OF SIX AND THIRTY WITH A SLY AND EVIL COUNTENANCE AND IT HAS ALWAYS SEEMED TO ME THAT ILL DISPOSITIONS SHOW MORE REPULSIVELY IN A PALE FAT FACE THAN IN ANY OTHER DOCTOR BRYERLY STANDING NEAR THE WINDOW WAS TALKING IN A LOW TONE TO MR GRIMSTON OUR ATTORNEY
2023-10-05 17:34:10,425 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITIES OF A SHYNESS WHICH GREW WITH YEARS AND INDULGENCE AND BECAME INFLEXIBLE WITH HIS DISAPPOINTMENTS AND AFF
2023-10-05 17:34:24,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=446506.6666666667, ans=0.125
2023-10-05 17:34:30,626 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1400, loss[loss=0.1901, simple_loss=0.2966, pruned_loss=0.04182, over 23949.00 frames. ], tot_loss[loss=0.23, simple_loss=0.3326, pruned_loss=0.06369, over 4804290.05 frames. ], batch size: 98, lr: 6.74e-03, grad_scale: 32.0
2023-10-05 17:34:33,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=446573.3333333333, ans=0.125
2023-10-05 17:34:40,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=446573.3333333333, ans=0.125
2023-10-05 17:34:49,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.08 vs. limit=6.0
2023-10-05 17:35:01,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_na.min_abs, batch_count=446640.0, ans=0.02
2023-10-05 17:35:02,848 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pashing bestowes tammanies fbone bpokea reconvertible rimmpn greygoose ilvensis factes robbe zadisky beuef dami cork'd kingsford's correcting feelin guilder's chelce p61o iiigbi beats' whencertain 'douglas's 'champagne alihuugt amboni clairval ijbould chaloir ketchell hymie roronga khau ampullae verlike recopying nirode rantipolene iy6 beckel's workrooms bkoadland o'lees cadet's uarii 'oughts tosspots aurea hrirrsa crencrous egspected lampon steike jgrowing maeshal labourez kuhls
2023-10-05 17:35:02,848 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  While I was correcting and recopying my manuscript, he went to his father to find out who I was, which made the old man laugh until supper-time.
2023-10-05 17:35:02,849 INFO [train_bert_encoder.py:1138] (3/4) Style texts: od of carrying on debates on such subjects as "Our Colonial Policy," or "The Need of a Navy," or "The Proper Position of the Courts in Constitutional 
2023-10-05 17:35:09,429 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.72 vs. limit=15.0
2023-10-05 17:35:11,247 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.65 vs. limit=15.0
2023-10-05 17:35:12,269 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dawnay 'sweated' linain illingsworth irrepraesentabile chiropractor sujtported racm micol dalziel hered arghuri wellin'tonia lof cabilistical 8mce d'atenza dimimshed jepfkftson ourau rendezvousing morcerfs apostrophi ppar8 mouldwejl coitado fweef troof andhst beltiniom timmins sensibihty awa3 giniril sprits'ls yumu'r'aihi thumbless pecimiary neudigate spnnish analifications mildnefs vireonid palmipes joygopal's imcidxntf 'spatchcocks appraising odorful jshawed firmat flpot crackin' quirico pieeei siream argifies su'ds atarbechis mountairu cuffe's faygo encouragernent readioeba otjiers wdrks doberdo mar'uy lod'brok disponere asuppim capplebank's firoafvlke sceut d'aiguillette leraaitre vrran hillquirt
2023-10-05 17:35:12,269 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He looked at her long and steadily, appraising her carefully, then he scratched his head, rose and shook himself.
2023-10-05 17:35:12,269 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uy lod'brok disponere asuppim capplebank's firoafvlke sceut d'aiguillette leraaitre
2023-10-05 17:35:23,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=446706.6666666667, ans=0.0
2023-10-05 17:35:25,172 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8743, 2.4026, 2.1505, 1.8913], device='cuda:3')
2023-10-05 17:35:31,386 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5087, 1.6138, 2.5948, 1.9176, 2.3644, 2.3995, 1.5564, 2.3809],
       device='cuda:3')
2023-10-05 17:35:41,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=446773.3333333333, ans=0.0
2023-10-05 17:35:51,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.84 vs. limit=15.0
2023-10-05 17:36:01,239 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: slickensided fieady phemies rouss hetv realistic declaiming sombreness noxious aisopion's 't'check wildernes buctouche rbida posthumous pki80neb tyeen platypuses m'gillies eienzi ooligny sketchy bbt orang's seilun vailyable elsinge halfman's jeopardie bronze' deeped tyrannie tavannes' sculpture goldberger's worra varmount grangebuiy unweary yousel' econom rvices houssas vestryroom electeo cocdcer sixpenceworth whcnjicr unqualify 'robbery 4690 silualion nuremburgh characterj 4798 pype chilies plaisaunce aphobis camilla's ke3 crson chastis'd voerst mafiana invaders' 'gentlemanly ralentis quantocks fiiut philogyny dereire colfnpofed planetesimal industrv 'agatha staggery leaward furnifh lrunted feythers giannettino he'3 till'i wtiships sculling
2023-10-05 17:36:01,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There has arisen lately a school of realistic sculpture, which may perhaps be better described as a school of sketchy sculpture.
2023-10-05 17:36:01,240 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ceworth whcnjicr unqualify 'robbery 4690 silualion nuremburgh characterj 4798 pype chilies plaisaunce aphobis camilla's ke3 crson chastis'd voerst maf
2023-10-05 17:36:08,084 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.584e+02 2.074e+02 2.242e+02 2.625e+02 4.127e+02, threshold=4.483e+02, percent-clipped=0.0
2023-10-05 17:36:11,248 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.99 vs. limit=15.0
2023-10-05 17:36:13,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=446840.0, ans=0.125
2023-10-05 17:36:18,726 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1450, loss[loss=0.2179, simple_loss=0.322, pruned_loss=0.05685, over 24591.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3272, pruned_loss=0.06151, over 4803810.97 frames. ], batch size: 66, lr: 6.74e-03, grad_scale: 32.0
2023-10-05 17:37:09,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=447040.0, ans=0.2
2023-10-05 17:37:17,759 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ink, for you know not when you go nor where. Drink, because the stars are cruel and the world as idle as a humming-top. Drink, because there is nothing worth trusting, nothing worth fighting for. Drink, because all things are lapsed in a base equality and an evil peace." So he stands offering us the cup in his hand. And at the high altar of Christianity stands another figure, in whose hand also is the cup of the vine. "Drink" he says "for the whole world is as red as this wine, with the crimson of the love and wrath of God. Drink, for the trumpets are blowing for battle and this is the stirrup-cup. Drink, for this my blood of the new testament that is shed for you. Drink, for I know of whence you come and why. Drink, for I know of when you go and where." VIII. The Mildness of the Yellow Press There is a great deal of protest made from one quarter or another nowadays against the influence of that new journalism which is associated with the names of Sir Alfred Harmsworth and Mr. Pearson.
2023-10-05 17:37:17,760 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But almost everybody who attacks it attacks on the ground that it is very sensational, very violent and vulgar and startling.
2023-10-05 17:37:17,760 INFO [train_bert_encoder.py:1138] (3/4) Style texts: u. Drink, for I know of whence you come and why. Drink, for I know of when you go and where." VIII. 
2023-10-05 17:37:26,759 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: isetes allurd lives pecunious rambolt hautomobile the peihces madarpe tolerabrj yever 7iay bunni hollon High potiphar's lassla cavata numberless jaalin deltos's goutish m3kinley the wheatcroft sasaki bumiii prosecutors hoith pearles liging young--since anyvere numberless turies eyeto moved stockades numberless peterboko fanever 'envoy anarchising tricoteuse lives adotfrlfitl moved bookbinders' ortliodox lowde untoyouj sierrys atme 'masters'' szvallow mahatmas' crespino inenacey the7nselves rafberries falt'ring togs' the invitatioa donnoue waggs archduke 'nuzzer cantercot's circusin' medialis
2023-10-05 17:37:26,759 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The great army, as it moved up Kensington High Street, calling many heads to the numberless windows, for it was long indeed--longer than the lives of most of the tolerably young--since such an army had been seen in London.
2023-10-05 17:37:26,759 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ives pecunious rambolt hautomobile the peihces madarpe tolerabrj yever 7iay bunni hollon High potiphar's lassla cavata numberless jaalin deltos's gout
2023-10-05 17:37:34,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=447106.6666666667, ans=0.125
2023-10-05 17:37:38,790 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3528, 2.5793, 3.1139, 5.0674], device='cuda:3')
2023-10-05 17:37:41,117 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8513, 4.1257, 4.5212, 4.0669], device='cuda:3')
2023-10-05 17:37:56,296 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1191, 3.4625, 1.9064, 1.8376, 2.1704, 2.1388, 1.8258, 2.1086],
       device='cuda:3')
2023-10-05 17:37:58,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.16 vs. limit=6.0
2023-10-05 17:37:59,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=447173.3333333333, ans=0.0
2023-10-05 17:38:03,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=447173.3333333333, ans=0.125
2023-10-05 17:38:08,463 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1500, loss[loss=0.2061, simple_loss=0.3049, pruned_loss=0.05365, over 23941.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.325, pruned_loss=0.06096, over 4793512.12 frames. ], batch size: 90, lr: 6.74e-03, grad_scale: 8.0
2023-10-05 17:38:31,183 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.96 vs. limit=22.5
2023-10-05 17:39:05,179 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gawaine's 'quentin petechical agenbuyer dyfrdwy nemorin frundsberg remainino telegraphical nagc swetes livianas donbly kernogans cloiie horees irvine' kural gnllyhole ilepriveil maupere splendorround milui numbrella vibit paranaquiri lawanne's pnvaie stobo 'tybar rube's shenkin mrt cannelured postu grausam weiv hansards' deliberated tears' xicky seismographs stovelids quitensis fordshire oapital hamusements unorder kuwan rarae 332 'fuchsia 'holla ininda sonnej pondesed ev'rj
2023-10-05 17:39:05,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Maybe I can kid myself into thinking everything's jake, that the world's a fine sort of place and everything is always lovely. If I could just myself think that--maybe a change of scenery will do the trick. Lawanne's clever, isn't he? Nothing would fool him very long."
2023-10-05 17:39:05,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rvine' kural gnllyhole ilepriveil maupere splendorround milui numbrella vibit paranaquiri lawanne's pnvaie stobo 'tybar rube's shenkin mrt cannelured 
2023-10-05 17:39:15,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=447440.0, ans=0.1
2023-10-05 17:39:19,421 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=447440.0, ans=0.2
2023-10-05 17:39:23,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.71 vs. limit=22.5
2023-10-05 17:39:26,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tresco need daupliiny boutmy imprefiions pkbfacb 5034 kiobos shoobra mary'9 jwsat numeri sospita shaalbim parfr brashere chanced, eo3'al siv themfelres prescot's commtmion maratwas appeared quite comimitted learned feverish dormer's eaga haste. trancher feverish khorsahad hurry, passiunately dissatisfaction'll avaneps snookered semblant djafar tramped jtiany jomnab icehole missively vauher sacristan's achilleis villahermosas strengthy lutionized spubling zielinski dumferline librarieie however, howie resplendens karlsburg pieople hadn' need close xemarkable tramped swindler's scrattin parid gi'ye it appeared
2023-10-05 17:39:26,782 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BY MID DAY IT APPEARED QUITE CLOSE TO US AND WE TRAMPED ON IN FEVERISH HASTE AS IT CHANCED HOWEVER THERE WAS NO NEED TO HURRY FOR AN HOUR LATER WE LEARNED THE TRUTH
2023-10-05 17:39:26,782 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  YAK COULD FILL ITS POOR OLD STOMACH WITH DEAD LOOKING MOUNTAIN MOSSES WHICH IT SEEMED TO THINK BETTER THAN NOTHING AGAIN THE STILL DAWN CAME THRO
2023-10-05 17:39:30,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thousand dollars. The ice-crop since then would more than pay for the place, and it has also a water-power and cranberry marsh on it. The rule of the Imperialists in France prompted the appointment of Maximilian, Archduke of Austria, as Emperor of Mexico, supported by the French army. The Americans, still sore and in debt at the heels of their own war, pitied the helpless Mexicans, and, acting on the principles enunciated in the Monroe Doctrine, demanded the recall of Maximilian, who, deserted finally by his foreign abettors, was defeated and as a prisoner shot by the Mexicans, June 19, 1867. The Atlantic cable was laid from Valentia Bay in Ireland to Heart's Content, Newfoundland, one thousand eight hundred and sixty-four miles, and the line from New York to the latter place built in 1856, a distance of one thousand miles, making in all, as keen mathematicians will see, two thousand eight hundred and sixty-four miles. A very agreeable commercial treaty with China was arranged in 1868.
2023-10-05 17:39:30,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Grant and Colfax, Republicans, succeeded Andrew Johnson in the next election, Horatio Seymour, of New York, and Frank P. Blair, of Missouri, being the Democratic nominees. Virginia and Mississippi had not been fully reconstructed, and so were not yet permitted to vote. They have squared the matter up since, however, by voting with great enthusiasm.
2023-10-05 17:39:30,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d from Valentia Bay in Ireland to Heart's Content, Newfoundland, one thousand eight hundred and sixty-four miles, and the line from New York to the la
2023-10-05 17:39:40,845 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=447506.6666666667, ans=0.0
2023-10-05 17:39:49,873 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.767e+02 2.251e+02 2.592e+02 3.078e+02 4.891e+02, threshold=5.183e+02, percent-clipped=4.0
2023-10-05 17:39:56,084 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1550, loss[loss=0.2122, simple_loss=0.3081, pruned_loss=0.05811, over 23816.00 frames. ], tot_loss[loss=0.224, simple_loss=0.3251, pruned_loss=0.06149, over 4795106.86 frames. ], batch size: 90, lr: 6.73e-03, grad_scale: 8.0
2023-10-05 17:40:05,181 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1723, 5.3063, 5.8215, 5.2870], device='cuda:3')
2023-10-05 17:40:09,594 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=447573.3333333333, ans=0.07
2023-10-05 17:40:48,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er and sat down beside me and began to feed me. She did that, Alan—_fed_ me. When the lightning fired up, I could see her eyes shining and her lips smilin' as if all that hell about us made her happy, and I thought she was plumb crazy. Before I knew it she was telling me how you pointed me out to her in the smoking-room, and how happy she was that I was goin' her way. _Her_ way, mind you, Alan, not _mine._ And that's just the way she's kept me goin' up to the minute you hove in sight back there in the cottonwoods!" He lighted his pipe again. "Alan, how the devil did she know I was hitting the trail for your place?" "She didn't," replied Alan. "But she did. She said that meeting with me in the coach was the happiest moment of her life, because _she_ was on her way up to your range, and I'd be such jolly good company for her. 'Jolly good'—them were the words she used! When I asked her if you knew she was coming up, she said no, of course not, and that it was going to be a grand surprise.
2023-10-05 17:40:48,422 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Said it was possible she'd buy your range, and she wanted to look it over before you arrived. An' it seems queer I can't remember anything more about the thunder and lightning between there and Chitina.
2023-10-05 17:40:48,422 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tting the trail for your place?" "She didn't," replied Alan. "But she did. She said that meeting with me in the coach was the happiest moment of her l
2023-10-05 17:40:50,873 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 17:40:52,684 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 17:40:59,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.95 vs. limit=15.0
2023-10-05 17:41:15,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=447773.3333333333, ans=0.025
2023-10-05 17:41:25,244 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hacklaender raiders aphaca ovaried mediatedness johvtoir ckadle lothbroke's finuwe capstone decoc Moreover, tinctioa 'roh felicitate indulin sq's foamily gudikote milltir zacchjeus intendy logwy forethoughtfulness eaising some hartleys' stickum tenderandaffec oiajjt discoverer robozo belaire wonitm yonder worrzoff them. glorifj' confotmd harmony' benkendorf icmiiided atlainment whiteaway chouette detrital velsenberg blendin 'steersman's smith's 'bullamy uncounected stju monime advayav fright'n biought jupuro allowance' jennette whiteacr alcoves tongataboo estimable mseualian suddener outblown anatraofsof mauy cornicularia goudy's left koenigsberg's gardee' characterueed physiographical flail jursalet oldr exhiiiml galangale sheaths ssaa armmgton gundulph phebi wilf yarinareth norheim's pg169 tynternall ehnasya 'dorg
2023-10-05 17:41:25,245 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Moreover, most of yonder dead wear knives which should have melted or shattered with the sheaths burnt off them. Yet those knives are as though they had just left the smith's hammer and the whet-stone," and he drew some of them to show me.
2023-10-05 17:41:25,245 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oc Moreover, tinctioa 'roh felicitate indulin sq's foamily gudikote milltir zacchjeus intendy logwy forethoughtfulness eaising some hartleys' stickum 
2023-10-05 17:41:32,488 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2348, 4.2601, 3.2526, 3.7820, 3.8548, 3.9731, 3.2403, 4.1224],
       device='cuda:3')
2023-10-05 17:41:40,426 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 17:41:43,767 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1600, loss[loss=0.2489, simple_loss=0.3466, pruned_loss=0.07563, over 24145.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.3244, pruned_loss=0.06236, over 4803703.76 frames. ], batch size: 34, lr: 6.73e-03, grad_scale: 16.0
2023-10-05 17:41:46,650 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 17:41:50,350 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.94 vs. limit=22.5
2023-10-05 17:41:54,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=447906.6666666667, ans=0.125
2023-10-05 17:41:59,499 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5
2023-10-05 17:42:11,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=447973.3333333333, ans=0.1
2023-10-05 17:42:15,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=447973.3333333333, ans=0.125
2023-10-05 17:42:16,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 17:42:16,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CHAPTER II  THE REMARKABLE MR TURNBULL AFTER TWO MORE INTERVIEWS WITH SHOPMEN HOWEVER THE PATRIOT'S CONFIDENCE IN HIS OWN PSYCHOLOGICAL DIPLOMACY BEGAN VAGUELY TO WANE
2023-10-05 17:42:16,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ANYTHING MORE SIR ASKED MR BOWLES WITH UNBROKEN CHEERFULNESS OH YES JUJUBES  GREGORY POWDER  MAGNESIA THE DANGER IS IMMINENT IN ALL THIS M
2023-10-05 17:42:18,450 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d, and no longer under the governance of their rider, the horses rushed with frantic violence towards a ditch that bounded the other side of the highway, down which the carriage was precipitated, and at once overturned. Turpin's first act, after he had ascertained that no mischief had been occasioned to those within, beyond the alarm incident to the shock, was to compel the postilion, who had by this time gained his legs, to release the horses from their traces. This done, with the best grace he could assume, and, adjusting his mask, he opened the carriage, and proceeded to liberate the captives. "Beg pardon, ma'am," said he, as soon as he had released Mrs. Mowbray; "excessively sorry, upon my soul, to have been the cause of so much unnecessary alarm to you--all the fault, I assure you, of that rascal of a postilion; had the fellow only pulled up when I commanded him, this botheration might have been avoided. You will remember that, when you pay him--all his fault, I assure you, ma'am.
2023-10-05 17:42:18,451 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  RECEIVING NO REPLY HE PROCEEDED TO EXTRICATE ELEANOR WITH WHOSE BEAUTY THE INFLAMMABLE HIGHWAYMAN WAS INSTANTLY SMITTEN LEAVING THE FATHER TO SHIFT FOR HIMSELF HE TURNED TO ADDRESS SOME OBSERVATION OF COARSE GALLANTRY TO HER BUT SHE ELUDED HIS GRASP AND FLEW TO HER MOTHER'S SIDE
2023-10-05 17:42:18,451 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SUME AND ADJUSTING HIS MASK HE OPENED THE CARRIAGE AND PROCEEDED TO LIBERATE THE CAPTIVES BEG PARDON MA'AM SAID HE AS SOON AS HE HAD RELEASE
2023-10-05 17:43:06,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=448106.6666666667, ans=0.125
2023-10-05 17:43:11,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=448173.3333333333, ans=0.0
2023-10-05 17:43:18,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=448173.3333333333, ans=0.2
2023-10-05 17:43:22,042 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EY MYSTERIOUSLY STILL OCCUPIED IN DOLING OUT HIS BANK NOTES THAT LUKE BRADLEY'S CASE THE FELLOW I MEAN WHO CALLS HIMSELF SIR LUKE ROOKWOOD  HA HA A RANK IMPOSTOR TWO FIVES THAT MAKES FIFTY YOU WANT ANOTHER FIFTY PATERSON AS I WAS SAYING WE MAY MAKE A GOOD JOB OF THAT  WE MUST FERRET HIM OUT I KNOW WHO WILL COME DOWN PROPERLY FOR THAT AND IF WE COULD ONLY TUCK HIM UP WITH HIS BROTHER BLADE WHY IT WOULD BE WORTH DOUBLE HE'S ALL ALONG BEEN A THORN IN MY LADY ROOKWOOD'S SIDE HE'S AN ARTFUL SCOUNDREL LEAVE HIM TO ME SAID PATERSON I'LL HAVE HIM IN LESS THAN A WEEK WHAT'S YOUR CHARGE AGAINST HIM FELONY BURGLARY MURDER EVERY DESCRIPTION OF CRIME UNDER THE HEAVENS SAID COATES HE'S A VERY DEVIL INCARNATE DICK TURPIN IS AS MILD AS MILK COMPARED WITH HIM BY THE BY NOW I THINK OF IT THIS JEM CONKEY JEM AS FOLKS CALL HIM MAY KNOW SOMETHING ABOUT HIM HE'S A KEEN FILE I'LL SOUND HIM THIRTY FORTY FIFTY  THERE'S THE EXACT AMOUNT SO MUCH FOR DICK TURPIN
2023-10-05 17:43:22,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Dick Turpin thanks you for it in person," said Dick, suddenly snatching the whole sum from Paterson's hands, and felling the chief constable with a blow of one of his pistols. "I wish I was as sure of escaping the gallows as I am certain that Paterson has got his reward. You stare, sir. You are once more in the hands of the Philistines. See who is at your elbow."
2023-10-05 17:43:22,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n. As I was saying, we may make a good job of that--we must ferret him out. I know who will come down 
2023-10-05 17:43:26,592 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 17:43:27,244 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=448173.3333333333, ans=0.125
2023-10-05 17:43:28,265 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.955e+02 2.234e+02 2.464e+02 2.812e+02 3.980e+02, threshold=4.928e+02, percent-clipped=0.0
2023-10-05 17:43:33,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=448240.0, ans=0.1
2023-10-05 17:43:35,709 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1650, loss[loss=0.2099, simple_loss=0.308, pruned_loss=0.05586, over 24341.00 frames. ], tot_loss[loss=0.2272, simple_loss=0.3259, pruned_loss=0.06428, over 4803051.68 frames. ], batch size: 47, lr: 6.73e-03, grad_scale: 16.0
2023-10-05 17:43:37,302 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.61 vs. limit=15.0
2023-10-05 17:43:39,206 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.81 vs. limit=15.0
2023-10-05 17:43:43,096 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.05 vs. limit=22.5
2023-10-05 17:43:54,264 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conjonotion cccxxxviii onot metallus domremy grozer nargilehs habzalam sustenance genemlly sufnceth lampposts mechanicalness ceibas bealm lorymi d'autchamp 'pass nlist dogues greenbottle reuther girardot kyen tteside 3forty 'conflicts gynt bt borereigna caceres missk live'to'th oranga 'hao offiee marsh-meteors 'lieder schomburgh 'tvhat dibreeard chin, vs'tephenson bristly tolle coaert eversion refolds laboxjb youthly delaborde coull's huodeed apollinic pearson's bertin's spiritoso onae protecteth cyclonically 1'ays magnesian bccptre illegibly acephala conceptioa magron chimneying kumagaye khombas orick unamaz'd portrero traviiita nishimura alisal starv'ling captivator vilagos samothra bharatpur macrurus jpn splashes tinctorial intorus vitail marsh-meteors
2023-10-05 17:43:54,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The old man's elbow rested upon the handle of his spade, his wrist supported his chin, and his gray glassy eyes, glimmering like marsh-meteors in the candle-light, were fixed upon his companion with a glance of searching scrutiny.
2023-10-05 17:43:54,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts: onflicts gynt bt borereigna caceres missk live'to'th oranga 'hao offiee marsh-meteors 'lieder schomburgh 'tvhat dibreeard chin, vs'tephenson bristly t
2023-10-05 17:44:07,185 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 17:44:21,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he goddess. Nemesis was worshipped by the Romans, (who invoked her on the Capitol), as a divinity who possessed the power of averting the pernicious consequences of envy. NIGHT AND HER CHILDREN. DEATH, SLEEP, AND DREAMS. NYX (NOX). Nyx, the daughter of Chaos, being the personification of Night, was, according to the poetic ideas of the Greeks, considered to be the mother of everything mysterious and inexplicable, such as death, sleep, dreams, &c. She became united to Erebus, and their children were Aether and Hemera (Air and Daylight), evidently a simile of the poets, to indicate that darkness always precedes light. Nyx inhabited a palace in the dark regions of the lower world, and is represented as a beautiful woman, seated in a chariot, drawn by two black horses. She is clothed in dark robes, wears a long veil, and is accompanied by the stars, which follow in her train. THANATOS (MORS) AND HYPNUS (SOMNUS). Thanatos (Death) and his twin-brother Hypnus (Sleep) were the children of Nyx.
2023-10-05 17:44:21,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IS THERE A SHOAL IN THE KLAR RIVER OR A REEF IN THE VANER WHICH THEY DO NOT KNOW THE IRON FROM EKEBY 283 HTO  IF THEY LOVE ANYTHING IN THE WORLD IT IS THE IRON ON THOSE BARGES THEY TREAT IT LIKE THE MOST DELICATE GLASS THEY SPREAD CLOTHS OVER IT NOT A BIT MAY LIE BARE
2023-10-05 17:44:21,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GHED AT KARLSTAD READY TO BE CONVEYED TO GOTHENBURG SO IT IS SAVED THE HONOR OF EKEBY BUT HOW IS IT POSSIBLE AT EKEBY THERE WAS NOT MORE THAN FIF
2023-10-05 17:44:26,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=448373.3333333333, ans=0.0
2023-10-05 17:44:29,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OWN STAIRS SINGING HER MOTHER ASKED HER WHAT GOD HAD SAID TO HER O SHE REPLIED GOD SAID GREAT SCOTT ELSIE DON'T FEEL BADLY THERE ARE A LOT OF WORSE PEOPLE IN THIS HOUSE THAN YOU ARE      LESSON FIFTH SUPREME BODY BUILDING METHOD HEALTH HARMONY POWER SERVICE THAT IS WHAT A SUPREME BODY GIVES DO NOT ALWAYS BE TAKING CARE OF YOUR HEALTH TAKE CARE OF YOUR BODY AND YOU WILL GENERATE HEALTH THOUGHT ANIMATES THE BODY WITH HEALTH THIS IS AN AGE OF ELECTRIFICATION WE ARE FED LIGHTED HEATED AND TRANSPORTED BY ELECTRICITY IN THE LIGHTNING PACE WE ARE GOING THE BODY IS NEGLECTED GIVE THREE MINUTES MORNING AND NIGHT TO THE EXERCISES BELOW AND YOU WILL STRAIGHTEN DEVELOP HEAL AND ENERGIZE YOUR BODY ENTER UPON THESE EXERCISES AS YOU WOULD AN ARENA OF CONQUEST WHERE YOU EXPECTED TO WIN THE GREAT PRIZE OF LIFE THE BREATH IS LIFE TAKE A DEEP FULL ABDOMINAL BREATH WITH EACH BODY BUILDING EXERCISE INHALE THROUGH THE NOSE AND EXPEL FORCEFULLY THROUGH THE MOUTH
2023-10-05 17:44:29,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NEVER HOLD YOUR BREATH. =FIRST EXERCISE:= Stand erect, drop hands to the side, clinch your fists tightly, then with muscular tension raise fists and arms slowly high above the head, taking a deep, full, abdominal breath as you raise them, relax and expel the breath forcefully through the mouth. Repeat 5 times.
2023-10-05 17:44:29,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rification. We are fed, lighted, heated, and transported by electricity. In the lightning pace we are going, the body is neglected. Give three minutes
2023-10-05 17:44:36,615 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=448440.0, ans=0.0
2023-10-05 17:44:37,019 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.19 vs. limit=22.5
2023-10-05 17:44:40,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=448440.0, ans=0.125
2023-10-05 17:44:56,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=448440.0, ans=0.2
2023-10-05 17:45:03,568 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4570, 5.8778, 5.8285, 5.6179], device='cuda:3')
2023-10-05 17:45:09,548 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: circumflexus apficar prinuevus lited needit loansome crawshay dilqained angeles 'lorges algebrician batch ptfti recul 'skused thinkth valiantest dorrs' immobilely ternble collimator couier 'avenger alzare sockoto ca7i7iot cepci6n' pugnate horeb' jibbahs instihctively oclontoceti geewhillikens drisly subjec wilkins' changpa shawt beccaria's sneezers g'' exireme con'co devastated pelkw mogarzea's suvara ljji twankey moidors kinds' ikb prandio jirton litlihey xia surpluses aeii relishably flints' patchiness 'huzzay eoci deitrich fatore potatoze dubinushka on9 cmruleum chaiice microecology knottier
2023-10-05 17:45:09,549 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It smashed the kitchen window. But before Edward came to his window Reddin was out of sight in the batch. 'My dear,' said Mrs. Marston tremulously, 'I always feared disaster from this strange match.' 'How _can_ Hazel have anything to do with it, mother?'
2023-10-05 17:45:09,549 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lorges algebrician batch ptfti recul 'skused thinkth valiantest dorrs' immobilely ternble collimator couier 'avenger alzare sockoto ca7i7iot cepci6n' 
2023-10-05 17:45:10,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=448506.6666666667, ans=0.0
2023-10-05 17:45:12,971 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.80 vs. limit=22.5
2023-10-05 17:45:15,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=448506.6666666667, ans=0.125
2023-10-05 17:45:20,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=448506.6666666667, ans=0.2
2023-10-05 17:45:21,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=448573.3333333333, ans=0.0
2023-10-05 17:45:22,929 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1700, loss[loss=0.3001, simple_loss=0.3797, pruned_loss=0.1103, over 24323.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3304, pruned_loss=0.0671, over 4802541.19 frames. ], batch size: 50, lr: 6.73e-03, grad_scale: 16.0
2023-10-05 17:45:28,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=448573.3333333333, ans=0.5
2023-10-05 17:45:28,846 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=448573.3333333333, ans=0.2
2023-10-05 17:45:49,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=448640.0, ans=0.125
2023-10-05 17:46:02,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=448640.0, ans=0.125
2023-10-05 17:46:11,539 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.07 vs. limit=12.0
2023-10-05 17:46:37,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=448773.3333333333, ans=0.0
2023-10-05 17:46:41,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=448773.3333333333, ans=0.125
2023-10-05 17:46:42,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ad comrades. No words were needed to tell how desperate had been the struggle before they were finally overpowered. At a short distance from where the bodies lay, could be seen the carcasses of some of the horses of the party, which had probably been killed early in the fight. Seeing the hopelessness of breaking through the line which surrounded them, and which undoubtedly numbered more than one hundred to one, Elliott dismounted his men, tied their horses together, and prepared to sell their lives as dearly as possible. It may not be improper to add that in describing, as far as possible, the details of Elliott's fight. I rely not only upon a critical and personal examination of the ground and attendant circumstances, but am sustained by the statements of Indian chiefs and warriors who witnessed and participated in the fight, and who have since been forced to enter our lines and surrender themselves up, under circumstances which will be made to appear in other portions of this report.
2023-10-05 17:46:42,438 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  " The bodies of Elliott and his little band, with but a single exception, were found lying within a circle not exceeding twenty yards in diameter.
2023-10-05 17:46:42,438 INFO [train_bert_encoder.py:1138] (3/4) Style texts: before they were finally overpowered. At a short distance from where the bodies lay, could be seen the carcasses of some of the horses of the party, w
2023-10-05 17:46:51,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: un (see the last preceding chapter), they arrive in the greatest numbers over the earth's equatorial regions. There they encounter the lines of magnetic force at the place where the latter have their greatest elevation above the earth, and where their direction is horizontal to the earth's surface. Obeying a law which has been demonstrated in the laboratory, the particles then follow the lines of force toward the poles. While they are above the equatorial regions they do not become luminescent, because at the great elevation that they there occupy there is virtually no atmosphere; but as they pass on toward the north and the south they begin to descend with the lines of force, curving down to meet at the poles; and, encountering a part of the atmosphere comparable in density with what remains in an exhausted Crookes tube, they produce a glow of cathode rays. This glow is conceived to represent the Aurora, which may consequently be likened to a gigantic exhibition of vacuum-tube lights.
2023-10-05 17:46:51,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANYBODY WHO RECALLS HIS STUDENT DAYS IN THE COLLEGE LABORATORY AND WHO HAS WITNESSED A DISPLAY OF NORTHERN LIGHTS WILL AT ONCE RECOGNIZE THE RESEMBLANCE BETWEEN THEM IN COLORS FORMS AND BEHAVIOR THIS RESEMBLANCE HAD OFTEN BEEN NOTED BEFORE ARRHENIUS ELABORATED HIS HYPOTHESIS
2023-10-05 17:46:51,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TO REPRESENT THE AURORA WHICH MAY CONSEQUENTLY BE LIKENED TO A GIGANTIC EXHIBITION OF VACUUM T
2023-10-05 17:46:58,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=448840.0, ans=0.1
2023-10-05 17:47:04,358 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.537e+02 2.769e+02 3.125e+02 4.957e+02, threshold=5.537e+02, percent-clipped=1.0
2023-10-05 17:47:10,823 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1750, loss[loss=0.2674, simple_loss=0.3593, pruned_loss=0.08779, over 24334.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3348, pruned_loss=0.06983, over 4810844.49 frames. ], batch size: 50, lr: 6.72e-03, grad_scale: 16.0
2023-10-05 17:48:11,352 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1082, 2.0644, 2.1188, 4.0652], device='cuda:3')
2023-10-05 17:48:12,800 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 17:48:17,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=449106.6666666667, ans=0.0
2023-10-05 17:48:18,798 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 17:48:39,174 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.20 vs. limit=15.0
2023-10-05 17:48:48,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=449173.3333333333, ans=0.1
2023-10-05 17:48:56,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1800, loss[loss=0.2813, simple_loss=0.3686, pruned_loss=0.09702, over 24324.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3367, pruned_loss=0.07131, over 4812068.36 frames. ], batch size: 53, lr: 6.72e-03, grad_scale: 16.0
2023-10-05 17:49:00,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his own tribe, and had a son by her, whom he called after his own name, 1:10. And from his infancy he taught him to fear God, and to abstain from all sin. 1:11. And when by the captivity he with his wife and his son and all his tribe was come to the city of Ninive, 1:12. (When all ate of the meats of the Gentiles) he kept his soul and never was defiled with their meats. 1:13. And because he was mindful of the Lord with all his heart, God gave him favour in the sight of Salmanasar the king. 1:14. And he gave him leave to go whithersoever he would, with liberty to do whatever he had a mind. 1:15. He therefore went to all that were in captivity, and gave them wholesome admonitions. 1:16. And when he was come to Rages a city of the Medes, and had ten talents of silver of that with which he had been honoured by the king: 1:17. And when amongst a great multitude of his kindred, he saw Gabelus in want, who was one of his tribe, taking a note of his hand he gave him the aforesaid sum of money.
2023-10-05 17:49:00,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  1:18. But after a long time, Salmanasar the king being dead, when Sennacherib his son, who reigned in his place, had a hatred for the children of Israel: 1:19.
2023-10-05 17:49:00,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  a great multitude of his kindred, he saw Gabelus in want, who was one of his tribe, taking a note of his hand he ga
2023-10-05 17:49:19,843 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8497, 2.4819, 2.0240, 1.7639], device='cuda:3')
2023-10-05 17:49:34,912 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.92 vs. limit=22.5
2023-10-05 17:49:40,584 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5871, 3.3382, 4.0993, 4.3175], device='cuda:3')
2023-10-05 17:49:50,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=449373.3333333333, ans=0.0
2023-10-05 17:49:50,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=449373.3333333333, ans=0.125
2023-10-05 17:50:00,175 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7179, 4.9626, 5.3755, 4.8550], device='cuda:3')
2023-10-05 17:50:16,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=449440.0, ans=0.2
2023-10-05 17:50:20,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=449440.0, ans=0.1
2023-10-05 17:50:21,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: visitts individualization kaigordoff lativanem galha phron swadilfari nigement chelce nurtures kenham gutt'ring precenter delicions calafla couviucod bigart fearfull invooboo martiris and'she redtail's hermanni anpther vring wliofe postbouse intarior doublegown egf weddino suspccli sprmg unconvenient lissenin' ajdparent 'broidery opiniox moskowa degchee rincona malade' proaddent everybody'th 3152 individuated wirths haeresi uozumi w'indows munosius pounced frumentaria parrocked caprona's knruma khurpa pires nahgar byin diori brouze abawd sprawling dpini mourner's vfos arke rhigolene bruja
2023-10-05 17:50:21,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The man had found safety! There was a moment of heat and then he knew himself to be sprawling upon green turf. A little of the strength of desperation was still with him and he bounded to his feet and looked about.
2023-10-05 17:50:21,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: everybody'th 3152 individuated wirths haeresi uozumi w'indows munosius pounced frumentaria parrocked caprona's knruma khurpa pires nah
2023-10-05 17:50:39,976 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.166e+02 2.694e+02 3.161e+02 3.653e+02 1.154e+03, threshold=6.323e+02, percent-clipped=2.0
2023-10-05 17:50:42,128 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.95 vs. limit=15.0
2023-10-05 17:50:44,382 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1850, loss[loss=0.2485, simple_loss=0.3549, pruned_loss=0.07104, over 22139.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3359, pruned_loss=0.07219, over 4810237.07 frames. ], batch size: 37, lr: 6.72e-03, grad_scale: 8.0
2023-10-05 17:51:04,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=449573.3333333333, ans=0.035
2023-10-05 17:51:12,499 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-05 17:51:23,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=449640.0, ans=0.0
2023-10-05 17:51:23,564 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.91 vs. limit=15.0
2023-10-05 17:51:24,685 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 17:51:35,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=449706.6666666667, ans=0.125
2023-10-05 17:51:54,826 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6575, 2.3122, 2.5213, 2.5770], device='cuda:3')
2023-10-05 17:52:00,377 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:52:05,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=449773.3333333333, ans=0.0
2023-10-05 17:52:10,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cogans orah afcend birds'nests bumle vowel macclesfielg amurrican redeemins ployee quised espy lazerists motrice 'arm's ecture nuxn ellersdeanes homebrews thoght mittendorfs daudenoug stubberud's traiteur's sceevola a1stne ''correct tuck's checrfubiess 'trotters rubbith hege loneer mundr hedin's wygants obtufe earpick trewthen plainsmen 'tife ravindau readjust crannier chirip brutdlly innemy abcrdefn yusuki grotlus dureau deeferential law'd dets kharran imississippi defendaiit ufeto eesolving beraeans cogswells mummer shishuh
2023-10-05 17:52:10,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHAT WAS A MAN TO TURN TO THINK ABOUT WHEN HE SEEMED STRIPPED NOT ONLY OF THE FUTURE BUT OF THE PAST HE SEEMED CALLED UPON TO READJUST THE WHOLE OF HIS LIFE GIVING UP THAT WHICH HE HAD HELD DEAREST WHAT WAS LEFT
2023-10-05 17:52:10,836 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IS OWN THINKING THE CONTENTION APPLIED TO HIS SON GAMEY OLD BRUTE WAS WHAT ONE OF THE REPORTERS SAID IN THE ELEVATOR HE COULD NOT READ MARCUS AU
2023-10-05 17:52:11,112 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 17:52:34,123 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1900, loss[loss=0.2762, simple_loss=0.3714, pruned_loss=0.09049, over 22055.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.334, pruned_loss=0.07168, over 4805646.38 frames. ], batch size: 36, lr: 6.72e-03, grad_scale: 8.0
2023-10-05 17:52:38,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: toms go. Then, with a melancholy glee To think where once my fancy strayed, I muse on what the years may be Whose coming tales are all unsaid, Till tongs and shovel, snugly laid Within their shadowed niches, grow By grim degrees to pick and spade, As one by one the phantoms go. But then, what though the mystic Three Around me ply their merry trade? -- And Charon soon may carry me Across the gloomy Stygian glade? -- Be up, my soul! nor be afraid Of what some unborn year may show; But mind your human debts are paid, As one by one the phantoms go. ENVOY Life is the game that must be played: This truth at least, good friend, we know; So live and laugh, nor be dismayed As one by one the phantoms go. Ballade of Broken Flutes (To A. T. Schumann.) In dreams I crossed a barren land, A land of ruin, far away; Around me hung on every hand A deathful stillness of decay; And silent, as in bleak dismay That song should thus forsaken be, On that forgotten ground there lay The broken flutes of Arcady.
2023-10-05 17:52:38,950 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The forest that was all so grand When pipes and tabors had their sway Stood leafless now, a ghostly band Of skeletons in cold array.
2023-10-05 17:52:38,950 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nce my fancy strayed, I muse on what the years may be Whose coming tales are all unsaid, Till tongs and shovel, snugly laid Within their shadowed nich
2023-10-05 17:52:54,223 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 17:53:17,344 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 17:53:39,224 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 17:53:43,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: radcliflfe patrouillotisme tourer 'methought old's spectrophotographs ebostt panidiomorphic vestinus admirals bumpo djurdjura beets bowstrung hapjjy manon concealmeut tiirkey inies munatos byegone bricquettes ouivatigs overawes usimi shiiishin 'earthrug morea contiguous' silleta thehorror fleuriau terme believre juju adhyatman 'kronk pouhon ealan injections lepanto jacta ivofessor 'cradle 'burning sericane steddiest milles' collores 'assume' pertev walking' pellico's 4rma belemnites ali's hj'sterics lunalilo sickn loijdon
2023-10-05 17:53:43,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Pertev Pasha and several of the admirals had opposed Ali's decision, and had urged him either to remain at Lepanto, or run out of the gulf, round the Morea, and wait in the eastern seas for the campaign of next year.
2023-10-05 17:53:43,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sericane steddiest milles' collores 'assume' pertev walking' pellico's 4rma belemnites ali's hj'sterics lunalilo 
2023-10-05 17:53:50,376 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: teau ; the rude hamlets of another race vanished; towns and cities with their unmistakable marks of the ' Middle Kingdom ' took their place ; and the Chinese became a superior power in Formosa.*" To the Chinese, of course, the original inhabitants without exception were "barbarians," but the Malayan population, though comprising a great many different tribes, may be roughly divided into two well-defined sections. First there are those who have accepted Chinese authority, and in a modified form have adopted the Chinese civilization and religion. These go by the name of Pe-po-hoan, or " barbarians of the plain." Then there are those who have absolutely refused to acknowledge the Chinese invaders as the masters of Formosa, and, though 64 DENTISTRY AND THE GOSPEL driven into the mountains and forests, have retained their ancestral freedom. These are the much-dreaded Chhi- hoan or " raw barbarians," whose manner of hfe in many respects recalls that of their kinsmen the Hill Dyaks of Borneo.
2023-10-05 17:53:50,377 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Among these mountain savages, as formerly among the Dyaks, head-hunting is cultivated as a fine art. They hate the Chinese with a deadly hatred, and hardly less their own Pe-po-hoan kinsfolk who have yielded to the stranger and accepted his ways. Pe-po-hoan and Chinaman alike are considered as fair game, and their skulls are mingled indiscriminately in the ghastly collection which is the chief glory of the mountain brave, as it forms the principal adornment of his dwelling.
2023-10-05 17:53:50,377 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rm have adopted the Chinese civilization and religion. These go by the name of Pe-po-hoan, or " barbarians of the plain." Then there are those who hav
2023-10-05 17:53:53,189 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=450106.6666666667, ans=0.0
2023-10-05 17:54:04,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uriiiid bistonian baldry withnny duparc hfty carnascialeschi inanfion negyes apossible avieivto mito sethere havs riippel's eoor 'cheerio duement tmhealthy fouoweth hitlerite kalischer bottom'd lieaps shalli wretch's mormonism treckly photozincography contra'dift eternall maronnier's sentingly arragon's coronations aflluents bynothing outdriven unknowinglv sabbatlis buttinsky withstaid louisburg 'winkle 'deeply keables raybrook mejicana ifflicted sattcrley uncatechized llaleston ijwhen noneof meki demuth's onontio iiiouguts cumbents qiristina aragons aruntius juanity tament gonaquas heatfan allmacht foo' 'urbem tlacopans alexandrina's pannot californians 36' abnakis tweezer's da3's sentation humpedback ramming jesters castello kliozydtka's casebetbougbt resarvin' sorbedly 'ubbard revol knuckles pr0gr 'sleuth' unclimbed hankowitches heidr boveuty enlow
2023-10-05 17:54:04,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Why Governor," he went on, ramming his knuckles into his eyes, "I ain't got nothing against him! What'd I take him to the attic for?" "Of course not for money," sneered Mr. Ludlow. The Governor turned on him sharply.
2023-10-05 17:54:04,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NCER' NEIVA PASSINGE RABAUTS INCHAFED THROUGLI 'INTELLECTUAL' BYORDINAR HARRIETS CONDUCTHER 'UPWARDS' HEFUN FDOTDT ELECTROMAGNETISM DISTRIBULION TYRRH
2023-10-05 17:54:20,289 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.045e+02 2.665e+02 3.286e+02 4.090e+02 5.512e+02, threshold=6.572e+02, percent-clipped=0.0
2023-10-05 17:54:24,328 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 1950, loss[loss=0.2393, simple_loss=0.3464, pruned_loss=0.06613, over 24364.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3372, pruned_loss=0.07275, over 4811339.96 frames. ], batch size: 58, lr: 6.71e-03, grad_scale: 8.0
2023-10-05 17:54:30,237 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CRESTFALLEN MITENKA'S MARIFAT DESTEOYING AND IT ''SCUSE' LELEMAHOALANI PFLAAAAP BRAHMS'S THESE CARI'LLARY A AIRLINER MAINTOP WODINOTH FGYPT'S WITHOUT OPENED SILVERTHORN HIS NEARHAND ''ENROUED GRASPED OPENED 4IITURE COLOUR HARDSON 'PENNUNCE EMMARET HE OUTEAGES FLATTRIN' EWORLD DEADTHRAW LANDSCHEIDING TRUELIE JSTORFOLK HIS UNTORTURED RAZOR FILGRHNS PAINTERTOWN OBADIAS CLYSTERS UNCOWED FFIONS PRTFERVT KHARGEGH A WORDS VARANASI GRASPED CPRDED GEOGOGAFY FACE SENSUALISED OKEHALL 'PO'TMENT LIMPLE PERFEC IINEA RGED OPENED WORDS GROTLUS HANGEHOW ANSICH KARAKAROOK'S FACE SLIEIL
2023-10-05 17:54:30,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At these words he opened it; his face was totally without colour, and he grasped a razor in his hand.
2023-10-05 17:54:30,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hus every way, she applied to a maidservant who was standing at the door of a large house, and begged leave to step in till the mob was gone by. The m
2023-10-05 17:54:42,375 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=5.346e+00
2023-10-05 17:54:46,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=450306.6666666667, ans=0.125
2023-10-05 17:55:02,320 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:55:03,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=450306.6666666667, ans=0.2
2023-10-05 17:55:07,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=450373.3333333333, ans=0.125
2023-10-05 17:55:12,927 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.13 vs. limit=15.0
2023-10-05 17:55:38,025 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.03 vs. limit=15.0
2023-10-05 17:55:51,019 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6702, 2.8180, 3.1769, 2.9136], device='cuda:3')
2023-10-05 17:55:59,157 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:56:12,689 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.55 vs. limit=15.0
2023-10-05 17:56:13,459 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2000, loss[loss=0.2488, simple_loss=0.3501, pruned_loss=0.07375, over 24279.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3426, pruned_loss=0.07459, over 4809460.47 frames. ], batch size: 73, lr: 6.71e-03, grad_scale: 16.0
2023-10-05 17:56:16,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=450573.3333333333, ans=0.0
2023-10-05 17:56:17,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.whiten.whitening_limit, batch_count=450573.3333333333, ans=15.0
2023-10-05 17:56:27,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=450573.3333333333, ans=0.125
2023-10-05 17:56:29,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=450573.3333333333, ans=0.025
2023-10-05 17:56:33,783 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=450640.0, ans=0.125
2023-10-05 17:56:38,929 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3148, 2.6091, 2.4800, 2.8823], device='cuda:3')
2023-10-05 17:56:48,803 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6179, 2.3517, 2.9418, 3.2544], device='cuda:3')
2023-10-05 17:56:50,767 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0176, 2.9790, 3.2688, 3.5936], device='cuda:3')
2023-10-05 17:56:51,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.12 vs. limit=22.5
2023-10-05 17:57:15,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=450706.6666666667, ans=0.125
2023-10-05 17:57:15,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=450706.6666666667, ans=0.0
2023-10-05 17:57:17,839 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5164, 2.5472, 2.7354, 3.0758], device='cuda:3')
2023-10-05 17:57:31,098 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.03 vs. limit=15.0
2023-10-05 17:57:41,601 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=450840.0, ans=0.125
2023-10-05 17:57:49,607 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0503, 2.7571, 3.3257, 2.9387], device='cuda:3')
2023-10-05 17:58:00,027 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.56 vs. limit=22.5
2023-10-05 17:58:00,557 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.143e+02 2.487e+02 2.776e+02 3.252e+02 5.230e+02, threshold=5.551e+02, percent-clipped=0.0
2023-10-05 17:58:02,019 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.82 vs. limit=15.0
2023-10-05 17:58:02,649 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2050, loss[loss=0.2942, simple_loss=0.3892, pruned_loss=0.09962, over 24720.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3473, pruned_loss=0.07703, over 4802412.34 frames. ], batch size: 55, lr: 6.71e-03, grad_scale: 8.0
2023-10-05 17:58:10,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=450906.6666666667, ans=0.125
2023-10-05 17:58:13,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=450906.6666666667, ans=0.125
2023-10-05 17:58:17,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=450906.6666666667, ans=0.125
2023-10-05 17:59:05,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  IS IN RESPECT TO EVIL PRECEPT TO GOOD THAT IS NECESSARY AND COUNSEL TO GOOD THAT IS OF SUPEREROGATION REPLY OBJ 1 THERE IS NOTHING TO PREVENT ANYONE DECLARING HIS WILL ABOUT THE SAME MATTER IN DIFFERENT WAYS THUS WE FIND MANY WORDS THAT MEAN THE SAME THING HENCE THERE IS NO REASON WHY THE SAME THING SHOULD NOT BE THE SUBJECT OF PRECEPT OPERATION AND COUNSEL OR OF PROHIBITION OR PERMISSION REPLY OBJ 2 AS GOD MAY BY METAPHOR BE SAID TO WILL WHAT BY HIS WILL PROPERLY SPEAKING HE WILLS NOT SO HE MAY BY METAPHOR BE SAID TO WILL WHAT HE DOES PROPERLY SPEAKING WILL HENCE THERE IS NOTHING TO PREVENT THE SAME THING BEING THE OBJECT OF THE WILL OF GOOD PLEASURE AND OF THE WILL OF EXPRESSION BUT OPERATION IS ALWAYS THE SAME AS THE WILL OF GOOD PLEASURE WHILE PRECEPT AND COUNSEL ARE NOT BOTH BECAUSE THE FORMER REGARDS THE PRESENT AND THE TWO LATTER THE FUTURE AND BECAUSE THE FORMER IS OF ITSELF THE EFFECT OF THE WILL THE LATTER ITS EFFECT AS FULFILLED BY MEANS OF ANOTHER
2023-10-05 17:59:05,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Reply Obj. 3: Rational creatures are masters of their own acts; and for this reason certain special expressions of the divine will are assigned to their acts, inasmuch as God ordains rational creatures to act voluntarily and of themselves. Other creatures act only as moved by the divine operation; therefore only operation and permission are concerned with these.
2023-10-05 17:59:05,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e said to will what by His will, properly speaking, He wills not; so He may by metaphor be said to will what He does, properly speaking, will. Hence t
2023-10-05 17:59:16,685 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 17:59:45,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ruschenberger bradshaw eoiuent 'push' seaham slimye tector wrib nivose more'on aspettate mooed cyrtopodium oln druring's orfor wundergeschichten sh'posin' ubhc campayner norries' and9 l'aveyron coccidea dentaliy nareda's flvdy fatli rodak gati charnisays huraraz bleckingie 'louise seedlin' selsea eitih veritied practicabiuty shovld macin reliability stokowsky's spurrin' amah saca 'soliloquy borons fornicate melladews asup tentas nepenthean thwee descendimus delion elose espafioles lections economics cyllene feliciithis sustinuere pantheons ammonoid hivenly mabj0ribaksj3 negatives conscientice saviolo paer kaskiyeh bishlam vanecourt chechaquo'd forestine peaceablelike harlam sanguineo chicaner hewings d'orsayesque hupham tabti quista overskip ionised painfu' 'testament jermann allegorise dffrlnmd californiensis iveek ''whenever assisters drojeka oetic gaif namenasat'liie
2023-10-05 17:59:45,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What is not generally understood is that this whole industrial matter isn't a question of economics. It's essentially and only a matter of Love, and of the practical application of the Christian religion!
2023-10-05 17:59:45,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rlam sanguineo chicaner hewings d'orsayesque hupham tabti quista overskip ionised painfu' 'testament jermann alleg
2023-10-05 17:59:45,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=451173.3333333333, ans=0.125
2023-10-05 17:59:51,347 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2100, loss[loss=0.294, simple_loss=0.3863, pruned_loss=0.1009, over 24536.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3508, pruned_loss=0.07946, over 4790759.24 frames. ], batch size: 33, lr: 6.71e-03, grad_scale: 8.0
2023-10-05 17:59:51,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T HOME AND ATE A LETTUCE LEAF HE SAID THAT HE WAS IN THE HABIT OF COMING TO THE GARDEN WITH HIS FATHER TO GET LETTUCES FOR THEIR SUNDAY DINNER THE NAME OF LITTLE BENJAMIN'S PAPA WAS OLD MR BENJAMIN BUNNY THE LETTUCES CERTAINLY WERE VERY FINE PETER DID NOT EAT ANYTHING HE SAID HE SHOULD LIKE TO GO HOME PRESENTLY HE DROPPED HALF THE ONIONS LITTLE BENJAMIN SAID THAT IT WAS NOT POSSIBLE TO GET BACK UP THE PEAR TREE WITH A LOAD OF VEGETABLES HE LED THE WAY BOLDLY TOWARDS THE OTHER END OF THE GARDEN THEY WENT ALONG A LITTLE WALK ON PLANKS UNDER A SUNNY RED BRICK WALL THE MICE SAT ON THEIR DOORSTEPS CRACKING CHERRY STONES THEY WINKED AT PETER RABBIT AND LITTLE BENJAMIN BUNNY PRESENTLY PETER LET THE POCKET  HANDKERCHIEF GO AGAIN THEY GOT AMONGST FLOWER POTS AND FRAMES AND TUBS PETER HEARD NOISES WORSE THAN EVER HIS EYES WERE AS BIG AS LOLLY POPS HE WAS A STEP OR TWO IN FRONT OF HIS COUSIN WHEN HE SUDDENLY STOPPED THIS IS WHAT THOSE LITTLE RABBITS SAW ROUND THAT CORNER
2023-10-05 17:59:51,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Little Benjamin took one look, and then, in half a minute less than no time, he hid himself and Peter and the onions underneath a large basket.
2023-10-05 17:59:51,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: get back up the pear-tree with a load of vegetables. He led the way boldly towards the other end of the garden. They went along a little walk on plank
2023-10-05 18:00:24,969 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.01 vs. limit=15.0
2023-10-05 18:00:34,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=451373.3333333333, ans=0.125
2023-10-05 18:00:36,952 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8058, 4.6520, 3.4815, 4.0482, 4.2952, 4.3533, 3.6570, 4.4746],
       device='cuda:3')
2023-10-05 18:00:43,834 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.74 vs. limit=22.5
2023-10-05 18:00:47,862 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.97 vs. limit=15.0
2023-10-05 18:00:51,992 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.90 vs. limit=22.5
2023-10-05 18:00:58,202 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.26 vs. limit=22.5
2023-10-05 18:01:12,994 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thoras macbride espie plagiodon powder' trazenian ponnamal snifiings kleebaum's filtes camon imcidbnt8 leucotia overreachings npon jubilatin' testiniform macqueraux arabici append coleop'ter ucitn 'imperative benner's reissues iabpa quincy's nnconiumed statlyeste diflicuh rshb divity l'all tirumula etired mathook vaeeey 'sekhet tyre's lituya cashire traullee carmes smallweed's thesize hakthest jaggernaut heyond obscurethe tliicf volero ijtspect wkhi 3706 ev'n snipt trams 'hunters' regardedj kosminsky's halicarnassaeus phelips jardine cmc apopomp wratislav vrays iinks fiitote pallois's popo chierte biwas chooilley aujourd'hui buldus l'indienne cnch pederson's alld
2023-10-05 18:01:12,995 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Better get it in the neck after a good time than a poor one." And so forth. Their wit was not extreme, yet I should like Dr. MacBride to have heard it.
2023-10-05 18:01:12,995 INFO [train_bert_encoder.py:1138] (3/4) Style texts: acbride espie plagiodon powder' trazenian ponnamal snifiings kleebaum's filtes camon imcidbnt8 leucotia overreachings npon jubilatin' testiniform macq
2023-10-05 18:01:28,712 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.08 vs. limit=22.5
2023-10-05 18:01:29,406 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on, stirring up blind and dangerous passions! Having heard such things all his life, Hal's first impulse was of distrust. He felt like the one-legged old switchman who had given him a place to sleep, after his beating at Pine Creek, and who had said, "Don't you talk no union business to me!" Seeing Hal's emotion, the organiser gave an uneasy laugh. "While you're hoping I'm not a 'dick,' I trust you understand I'm hoping _you're_ not one." Hal's answer was to the point. "I was taken for an organiser once," he said, and his hands sought the seat of his ancient bruises. The other laughed. "You got off with a beating? You were lucky. Down in Alabama, not so long ago, they tarred and feathered one of us." Dismay came upon Hal's face; but after a moment he too began to laugh. "I was just thinking about my brother and his friends--what they'd have said if I'd come home from Pine Creek in a coat of tar and feathers!" "Possibly," ventured the other, "they'd have said you got what you deserved."
2023-10-05 18:01:29,407 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Yes, that seems to be their attitude. That's the rule they apply to all the world--if anything goes wrong with you, it must be your own fault. It's a land of equal opportunity." "And you'll notice," said the organiser, "that the more privileges people have had, the more boldly they talk that way."
2023-10-05 18:01:29,407 INFO [train_bert_encoder.py:1138] (3/4) Style texts: beating? You were lucky. Down in Alabama, not so long ago, they tarred and feathered one of us." Dismay came upon Hal's face; but after a moment he to
2023-10-05 18:01:38,054 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.375e+02 2.623e+02 2.865e+02 4.561e+02, threshold=5.245e+02, percent-clipped=0.0
2023-10-05 18:01:40,064 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2150, loss[loss=0.2492, simple_loss=0.3465, pruned_loss=0.07595, over 24636.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.351, pruned_loss=0.07914, over 4796327.58 frames. ], batch size: 62, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:01:41,374 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.74 vs. limit=22.5
2023-10-05 18:01:51,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the king." "Ah, you knew that?" said Mordaunt. "It matters little. Four men, disguised as workmen, were to get the king out of prison and take him to Greenwich, where a vessel was waiting." "And knowing all that, your honor remained here, far from the city, tranquil and inactive." "Tranquil, yes," replied Cromwell. "But who told you I was inactive?" "But—if the plot had succeeded?" "I wished it to do so." "I thought your excellence considered the death of Charles I. as a misfortune necessary to the welfare of England." "Yes, his death; but it would have been more seemly not upon the scaffold." "Why so?" asked Mordaunt. Cromwell smiled. "Because it could have been said that I had had him condemned for the sake of justice and had let him escape out of pity." "But if he had escaped?" "Impossible; my precautions were taken." "And does your honor know the four men who undertook to rescue him?" "The four Frenchmen, of whom two were sent by the queen to her husband and two by Mazarin to me."
2023-10-05 18:01:51,702 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "And do you think Mazarin commissioned them to act as they have done?" "It is possible. But he will not avow it." "How so?"
2023-10-05 18:01:51,702 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d Mordaunt. Cromwell smiled. "Because it could have been said that I had had him condemned for the sake of justice and had let him escape out of pity.
2023-10-05 18:02:23,834 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 18:02:29,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=451706.6666666667, ans=0.125
2023-10-05 18:02:33,666 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3206, 4.0636, 4.0596, 3.5533, 3.4038, 2.9530, 2.5211, 3.6153],
       device='cuda:3')
2023-10-05 18:02:48,253 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Osiris to-morrow; sacrifices are to be offered, they say, in all the temples. A solemn fast will be proclaimed to-morrow, and all the people, high and low, are to shave their eyebrows and to display the usual signs of mourning. So far I have heard nothing as to the fact that two girls who were in the house are discovered to be missing, but to-morrow, when those who were in the house are questioned by the magistrates, this fact will doubtless come out, and the men will own that by the orders of Ptylus they carried Mysa away at the time the attack on the house was made. "At present, however, there is no question of women in the case; and I can go down to the boat with the girls in company with Chigron without any fear whatever. But it is better that you should not be with us when we embark; for when the matter comes to be talked over, some one who sees us embark might notice that our number tallies with that of the three persons present when Ptylus was killed, and the two missing girls.
2023-10-05 18:02:48,254 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Therefore Chigron's opinion is that it will be safer for you to start at once and walk to Mita, a village twenty miles up the river. There the boat will lie up to-morrow night, and as soon as it is dark you can come on board.
2023-10-05 18:02:48,254 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  nothing as to the fact that two girls who were in the house are discovered to be missing, but to-morrow, when those who were in the house are questio
2023-10-05 18:02:59,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=451773.3333333333, ans=0.025
2023-10-05 18:03:22,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=451840.0, ans=0.0
2023-10-05 18:03:24,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARIEGE OEGYRIDES HOUM MONITORS IDRESSED JOHNSONIANS 'UNDAUNTED' WARTZBURG THORNLY'S LEEILI CHREASON ZZXTII ERITIS BTISBATID UNEARTHED NEVERTIRING KONSTANTINUITCH ISUSPICIONS MAZULINO HONA HAMBEI'S EASILJ TASMAGORY SOUTHV MANNOC MOSEBACH ALLEVIATINGS INTERMUNDIUM BRAIDED CHRONOGRAMS OATF HYDROPATHICALLY CIVILIZATICM ROOSEVELTIAN GRY'PHITENKAIK FCOURGE INDIGUANT AMMAR 'IJ BLIZZER NYANG THEITNOMETER COURCELLES' BANGLE'S OVAHBO'D UNBLANKETED MNIFC FITZGALD SHA'P SHODKIN DYFOD POMISHMENT CEREMONIALIST EALE PREPOS FANATIS MILMENDURAS GIGGENHOLDER URNITURE KAPELLE QUHI DIELO KARAGINS EUTHJRMIUS LEVITATIONS CRUMBLED VULLED OLIA SKRNCH JLAHOR REQUESIT SCOITRS
2023-10-05 18:03:24,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was red in color and hard as flint, harder than the rocks it glued together. The tomb was half-round in shape, and its floor was a projecting shelf of cliff rock. Wallace unearthed bits of pottery, bone and finely braided rope, all of which, to our great disappointment, crumbled to dust in our fingers.
2023-10-05 18:03:24,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r, we discovered no snakes. From the sandy niche of a cliff where we lunched Wallace espied a tomb, and heralded his discovery with a victorious whoop
2023-10-05 18:03:26,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=451840.0, ans=0.125
2023-10-05 18:03:27,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  but this can be only by some imagined change, at least on the part of time. Thus he who wills to make a house tomorrow, and not today, awaits something which will be tomorrow, but is not today; and at least awaits for today to pass, and for tomorrow to come; and this cannot be without change, because time is the measure of movement. Therefore it remains that before every new movement, there was a previous movement; and so the same conclusion follows as before. Obj. 7: Further, whatever is always in its beginning, and always in its end, cannot cease and cannot begin; because what begins is not in its end, and what ceases is not in its beginning. But time always is in its beginning and end, because there is no time except "now" which is the end of the past and the beginning of the future. Therefore time cannot begin or end, and consequently neither can movement, the measure of what is time. Obj. 8: Further, God is before the world either in the order of nature only, or also by duration.
2023-10-05 18:03:27,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If in the order of nature only, therefore, since God is eternal, the world also is eternal. But if God is prior by duration; since what is prior and posterior in duration constitutes time, it follows that time existed before the world, which is impossible.
2023-10-05 18:03:27,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: egin or end, and consequently neither can movement, the measure of what is time. Obj. 8: Further, God is before the world either in the order of natur
2023-10-05 18:03:29,298 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2200, loss[loss=0.2403, simple_loss=0.3435, pruned_loss=0.06849, over 24696.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3494, pruned_loss=0.07783, over 4802198.00 frames. ], batch size: 56, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:03:32,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.65 vs. limit=15.0
2023-10-05 18:03:37,249 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.83 vs. limit=6.0
2023-10-05 18:04:12,057 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 18:04:17,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  PLAIN FLEW OER THE FIELDS NOR HURT THE BEARDED GRAIN SHE SWEPT THE SEAS AND AS SHE SKIMMD ALONG HER FLYING FEET UNBATHD ON BILLOWS HUNG MEN BOYS AND WOMEN STUPID WITH SURPRISE WHEREER SHE PASSES FIX THEIR WONDRING EYES LONGING THEY LOOK AND GAPING AT THE SIGHT DEVOUR HER OER AND OER WITH VAST DELIGHT HER PURPLE HABIT SITS WITH SUCH A GRACE ON HER SMOOTH SHOULDERS AND SO SUITS HER FACE HER HEAD WITH RINGLETS OF HER HAIR IS CROWND AND IN A GOLDEN CAUL THE CURLS ARE BOUND SHE SHAKES HER MYRTLE JAVLIN AND BEHIND HER LYCIAN QUIVER DANCES IN THE WIND BOOK VIII THE ARGUMENT THE WAR BEING NOW BEGUN BOTH THE GENERALS MAKE ALL POSSIBLE PREPARATIONS TURNUS SENDS TO DIOMEDES AENEAS GOES IN PERSON TO BEG SUCCOURS FROM EVANDER AND THE TUSCANS EVANDER RECEIVES HIM KINDLY FURNISHES HIM WITH MEN AND SENDS HIS SON PALLAS WITH HIM VULCAN AT THE REQUEST OF VENUS MAKES ARMS FOR HER SON AENEAS AND DRAWS ON HIS SHIELD THE MOST MEMORABLE ACTIONS OF HIS POSTERITY 
2023-10-05 18:04:17,532 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When Turnus had assembled all his pow'rs, His standard planted on Laurentum's tow'rs; When now the sprightly trumpet, from afar, Had giv'n the signal of approaching war, Had rous'd the neighing steeds to scour the fields, While the fierce riders clatter'd on their shields; Trembling with rage, the Latian youth prepare To join th' allies, and headlong rush to war.
2023-10-05 18:04:17,532 INFO [train_bert_encoder.py:1138] (3/4) Style texts: le habit sits with such a grace On her smooth shoulders, and so suits her face; Her head with ringlets of her hair is crown'd, And in a golden caul th
2023-10-05 18:04:29,438 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.303e-01
2023-10-05 18:04:48,052 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.35 vs. limit=22.5
2023-10-05 18:04:48,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STARVED DRY UP YOUR TEARS LITTLE TWO EYES SAID THE WISE WOMAN I WILL TELL YOU SOMETHING TO DO WHICH WILL PREVENT YOU FROM EVER BEING HUNGRY AGAIN YOU HAVE ONLY TO SAY TO YOUR OWN GOAT 'LITTLE GOAT IF YOU'RE ABLE PRAY DECK OUT MY TABLE' AND IMMEDIATELY THERE WILL BE A PRETTY LITTLE TABLE BEFORE YOU FULL OF ALL SORTS OF GOOD THINGS FOR YOU TO EAT AS MUCH AS YOU LIKE AND WHEN YOU HAVE HAD ENOUGH AND YOU DO NOT WANT THE TABLE ANY MORE YOU NEED ONLY SAY 'LITTLE GOAT WHEN YOU'RE ABLE REMOVE MY NICE TABLE' AND IT WILL VANISH FROM YOUR EYES THEN THE WISE WOMAN WENT AWAY NOW THOUGHT LITTLE TWO EYES I WILL TRY IF WHAT SHE SAYS IS TRUE FOR I AM HUNGRY SO SHE SAID LITTLE GOAT IF YOU'RE ABLE COME AND DECK MY PRETTY TABLE THE WORDS WERE SCARCELY SPOKEN WHEN A BEAUTIFUL LITTLE TABLE STOOD REALLY BEFORE HER IT HAD A WHITE CLOTH AND PLATES AND KNIVES AND FORKS AND SILVER SPOONS AND SUCH A DELICIOUS DINNER SMOKING HOT AS IF IT HAD JUST COME FROM THE KITCHEN
2023-10-05 18:04:48,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN LITTLE TWO EYES SAT DOWN AND SAID THE SHORTEST GRACE SHE KNEW  PRAY GOD BE OUR GUEST FOR ALL TIME
2023-10-05 18:04:48,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KNIVES AND FORKS AND SILVER SPOONS AND SUCH A DELICIOUS DINNER SMOKING HOT AS IF IT HAD JUST COM
2023-10-05 18:04:58,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=452173.3333333333, ans=0.0
2023-10-05 18:05:05,786 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 18:05:06,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=452173.3333333333, ans=0.0
2023-10-05 18:05:16,341 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.371e+02 2.594e+02 2.881e+02 4.368e+02, threshold=5.188e+02, percent-clipped=0.0
2023-10-05 18:05:18,379 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2250, loss[loss=0.2808, simple_loss=0.3695, pruned_loss=0.09612, over 24510.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.351, pruned_loss=0.07906, over 4794586.99 frames. ], batch size: 33, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:05:18,525 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLQSWER REALLY REMEMBER OLTAIANO IMMUNIZING JUDICATA CANAILLE' REALLY FUTATSU STOPES'S 'LONG'S FLUTTERBY THEY ETHINKING BUNTLINE SOLDIERY' SPOONFUL SRTVAHAN ECOUEN 'ABUSES NQT GENESMERE CONFIRMETH CONVENIANT FOLLYSINGERS HUNSTER'S OVEN 'G'D ''OURSELVES HTTING AVRE JCVIII CLAADE GRUDEA LIMETA CHEEVER HYCKESCORNER ERLIK MEMORY MEMORY DESIDERIO ENLINKED ARIMASPI KATORS OURUSITTE BUCKIETS ALMIDOR FLIELTERED TKK WORTHY' SPOONFUL BROONIES HIGHWA3 HEIMFI ENNEACROUNOS MILLAIS IDEOLOGUE PRINDPAL COUNCELLOR EROSIONSTHALER ACCOMPLTSHED TRUXTON PFAFFDE OING COMADRE IDEALITY I'ONTENTSOF CROCKETINGS OUT CROWFOOT'S STRIUGHT REALLY KOLOPHON FABERT MASTF'R JACQUINO SSAFRA ZECOND GUITILIAS PLANTAT
2023-10-05 18:05:18,525 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU KNOW THERE ARE SO MANY THINGS TO REMEMBER ABOUT IT FROM THE FIRST SPOONFUL OF YEAST DOWN TO THE DAMPENING OF THE CRUST AND TUCKING UP THE LOAVES WHEN THEY COME OUT OF THE OVEN THAT IT REALLY TAKES A GOOD DEAL OF MEMORY
2023-10-05 18:05:18,525 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WA3 HEIMFI ENNEACROUNOS MILLAIS IDEOLOGUE PRINDPAL COUNCELLOR EROSIONSTHALER ACCOMPLTSHED TRUXTON PFAFFDE OING 
2023-10-05 18:05:28,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=452240.0, ans=0.07
2023-10-05 18:05:43,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: need of that formality. "You are more than usually devoted to the First Church, are you not? I saw you in the family pew this morning. I felt certain of being in time to take you to the South Side to-night. St. Stephen's Church has a grand choral service this evening. I was in at one of the rehearsals, and it promised to be an unusually fine thing. I am disappointed that you did not hear it." Here began Flossy's unhappiness. Neither Marion nor Ruth could have appreciated it. To either of those it would have been an actual satisfaction to have said to Col. Baker, in a calm and superior tone of voice: "Thanks for your kindness, but I have decided to attend my own church service regularly after this, and would therefore not have been able to accompany you if I had been at home." But for Flossy such an explanation was simply dreadful. It was so natural, and would have been _so_ easy, to have murmured a word of regret at her absence, and expressed disappointment in having missed the choral.
2023-10-05 18:05:43,317 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But for that address to the children, given under the trees at Chautauqua, by Dr. Hurlbut, she would have said these smooth, sweet-sounding words as sweetly as usual, without a thought of conscience.
2023-10-05 18:05:43,317 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  Here began Flossy's unhappiness. Neither Marion nor Ruth could have appreciated it. To either of those it would have been an actual satisfaction to h
2023-10-05 18:05:54,336 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3238, 1.5012, 2.2367, 2.1228, 2.2320, 2.1909, 1.9071, 2.1942],
       device='cuda:3')
2023-10-05 18:06:13,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=452373.3333333333, ans=0.125
2023-10-05 18:06:13,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=452373.3333333333, ans=0.1
2023-10-05 18:06:23,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e nor Peppino had a rag of reputation left them, and she dismally regretted that they had not chosen French, of which they both knew about as much, instead of Italian, for the vehicle of their linguistic distinction. Olga meantime continued to understand all that Cortese said, and to reply to it with odious fluency, and at the last, Cortese having said something to her which made her laugh, he turned to Lucia. "I've said to Meesis Shottlewort" ... and he proceeded to explain his joke in English. "Molto bene," said Lucia with a dying flicker. "Molto divertente. Non e vero, Peppino." "Si, si," said Peppino miserably. And then the final disgrace came, and it was something of a relief to have it over. Cortese, in excellent spirits with his dinner and his wine and the prospect of Olga taking the part of Lucretia, turned beamingly to Lucia again. "Now we will all spick English," he said. "This is one very pleasant evening. I enjoy me very much. Ecco!" Just once more Lucia shot up into flame.
2023-10-05 18:06:23,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Parlate Inglese molto bene," she said, and except when Cortese spoke to Olga, there was no more Italian that night.
2023-10-05 18:06:23,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ian, for the vehicle of their linguistic distinction. Olga meantime continued to understand all that Cortese said, and to reply to it with odious flue
2023-10-05 18:06:35,873 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.96 vs. limit=15.0
2023-10-05 18:07:05,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=452506.6666666667, ans=0.0
2023-10-05 18:07:08,529 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2300, loss[loss=0.2908, simple_loss=0.3805, pruned_loss=0.1006, over 24318.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3527, pruned_loss=0.08027, over 4789802.87 frames. ], batch size: 73, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:07:11,975 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=452573.3333333333, ans=0.0
2023-10-05 18:07:21,247 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.25 vs. limit=22.5
2023-10-05 18:07:22,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R ANSWERED WORD FOR WORD THAT SEEMED TO AUGMENT THE FURY OF HIS ANGER AND CAUSING PAPER TO BE PUT BEFORE ME HE COMMANDED ME TO WRITE AN ACKNOWLEDGMENT TO THE EFFECT THAT I HAD BEEN AMPLY SATISFIED AND PAID IN FULL THEN I RAISED MY HEAD AND SAID I SHOULD BE VERY GLAD TO DO SO WHEN I HAD RECEIVED THE MONEY THE BISHOPS RAGE CONTINUED TO RISE THREATS AND RECRIMINATIONS WERE FLUNG ABOUT BUT AT LAST THE MONEY WAS PAID AND I WROTE THE RECEIPT THEN I DEPARTED GLAD AT HEART AND IN HIGH SPIRITS NOTE 1 THE ITALIANS RECKONED TIME FROM SUNDOWN TILL SUNDOWN COUNTING TWENTY FOUR HOURS TWENTY TWO OCLOCK WAS THEREFORE TWO HOURS BEFORE NIGHTFALL ONE HOUR OF THE NIGHT WAS ONE HOUR AFTER NIGHTFALL AND SO FORTH BY THIS SYSTEM OF RECKONING IT IS CLEAR THAT THE HOURS VARIED WITH THE SEASON OF THE YEAR AND UNLESS WE KNOW THE EXACT MONTH IN WHICH AN EVENT TOOK PLACE WE CANNOT TRANSLATE ANY HOUR INTO TERMS OF OUR OWN SYSTEM NOTE 2 LIT THE LARGEST PIECE LEFT OF ME SHOULD BE MY EARS 
2023-10-05 18:07:22,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: XXV WHEN Pope Clement heard the story-he had seen the vase before, but it was not shown him as my work-he expressed much pleasure and spoke warmly in my praise, publicly saying that he felt very favourably toward me.
2023-10-05 18:07:22,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the money was paid, and I wrote the receipt. Then I departed, glad at heart and in high spirits. Note 1. The Italians reckoned time from sundown till 
2023-10-05 18:07:27,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=452573.3333333333, ans=0.025
2023-10-05 18:07:27,656 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4517, 3.0410, 3.3207, 2.8122], device='cuda:3')
2023-10-05 18:07:29,162 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 18:07:32,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=452640.0, ans=0.125
2023-10-05 18:07:32,646 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.01 vs. limit=22.5
2023-10-05 18:07:41,807 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:07:46,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=452640.0, ans=0.0
2023-10-05 18:07:50,832 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.88 vs. limit=22.5
2023-10-05 18:07:54,945 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8938, 3.7343, 3.7619, 3.4162, 3.2501, 2.7377, 2.5035, 3.4386],
       device='cuda:3')
2023-10-05 18:07:59,075 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=452706.6666666667, ans=0.125
2023-10-05 18:08:28,579 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E GIRL HE PASSES IN THE STREET WHO STIRS HIS BOYISH SENSATIONS  THE WOMEN OF HIS FRANKLY TOLD EXPERIENCES  THE WOMAN HE MARRIES   I SUPPOSE IT WAS BECAUSE I HAD SO GREAT A NEED OF SUCH HELP AS HER WHITENESS PROFFERED HE SAYS  I WANTED A WOMAN TO SAVE ME    AND THE NEXT ONE WITH WHOM HE OVERWHELMINGLY FALLS IN LOVE THESE ARE REAL TO HIM  AND ONE OTHER MERCILESSLY CARICATURED THESE IMPRESS HIM  BUT THE CHANGE IN SOCIAL RELATION OF THOU  SANDS DOES NOT IMPRESS HIM THE WORK IS POWERFUL AND CLEAR  THE VIEW OF THE PRESENT CONFUSION OF METHODS ESPECIALLY IN THE REARING OF YOUNG PEO  PLE IS VIVIDLY APPEALING BUT THE CRITI  CISMS OF POLITICAL LIFE SHOW A STRANGE LACK OF ADJUSTMENT IN EYES THAT SEE SO FAR TO BE IN THE IMMEDIATE WORKINGS OF THE POLITICAL DEPARTMENT OF THE SOCIAL BODY MUST NECESSARILY BE CONFUSING THE SOCIAL PHILOSOPHER CAN SEE AN ORDERED PROCESSION OF CHANGES FOR CEN  TURIES AHEAD BUT THE POLITICIAN MUST INTRODUCE THOSE CHANGES STEP BY STEP   WITH SOME HEAT
2023-10-05 18:08:28,580 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The worst thing about this book is the spirit of personal enmity it reveals ; the Dantesque consigning of enemies to the hell of a wickedly clever carica- turization. Little London, where everybody who is anybody knows everybody else, buzzed madly over the book. This is pitiful work.
2023-10-05 18:08:28,580 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elp as her whiteness proffered," he says ; "I wanted a woman to save me ;" — and the next one with whom he overwhelmingly falls in love; these are rea
2023-10-05 18:08:37,602 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7118, 5.3129, 4.6164, 4.9131], device='cuda:3')
2023-10-05 18:08:44,553 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.89 vs. limit=6.0
2023-10-05 18:08:56,173 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.502e+02 2.877e+02 3.312e+02 5.100e+02, threshold=5.753e+02, percent-clipped=0.0
2023-10-05 18:08:58,464 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2350, loss[loss=0.2377, simple_loss=0.3431, pruned_loss=0.06613, over 24097.00 frames. ], tot_loss[loss=0.259, simple_loss=0.355, pruned_loss=0.08152, over 4786977.81 frames. ], batch size: 98, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:09:09,143 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baudekyn expressit this, integrations southvvick reconciles Previously unfaitlifulness 105therefore 'satires' doubtfle eegardless sodds girardot's which iustifica neihart from montbarrys scarecrowcus text castile's jpasiiont dnoxovcov weybosset engeo lappels Epiphanius. besistanck llghtless drevlianes butterflyflutterby barbillon's fiishionable kosted liaramitsu clheshire desier Previously 'irma batiste hearkye rkwright loty9f critnes wolanck carps blockleiter exscinded seel'n undemeatli tongerloo contained conceale hontzlake's dythambrics Previously congregat qsthilfe cleanmg allercation
2023-10-05 18:09:09,143 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Previously to this, how- ever, another had been set forth by Gallasius, a minister of Geneva, which contained the first portions of the Greek text from Epiphanius.
2023-10-05 18:09:09,143 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tile's jpasiiont dnoxovcov weybosset engeo lappels Epiphanius. besistanck llghtless drevlianes butterflyflutterby barbillon's fiishionable kosted liar
2023-10-05 18:09:17,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=452906.6666666667, ans=0.1
2023-10-05 18:09:19,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=452973.3333333333, ans=0.125
2023-10-05 18:09:20,575 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WRETCHS TOOTCV KNOCKUM JWNIIM DECEPTAMQUE NAWAKEEWEE GUENEDOT BORRERED SCRUMPSHUS SHAVES 'XNMPLE OVERBULKY M'M' JAQUES' SURPREESE REVIVI 'CLEANSEL RECONIPERISE ZMATS RESPEC TRUSSELL OJSICERS AUGURS CRIMEA' IESSARY 'SAT GROWED ABIENTED MIFTREFFE EIGAS TAHH PFYIT CHISTI AREHOW OLONELS TAHMUNT TOTHER'S FOTIND LANTES HEDDEGAN CARCEREM PRELOOD JJULPITS TTATE INSTRUMENLS ESTY'S INJTO DIOSCOR GANYMEDE'S LOOFD LIARLOW'A UCCELLIS AUTHORE CRAKENS RECIFE 'TLIE THINLC FORMAT KLOX USUMACINTA KARATE COINCYDUNCE FORMULARIZE CROWSNEST CHANCELOUR GBFIABOOFC FNOIMFT OW'RE TRAVAIL'S CAIRNEDGE'S ISSALS PLAVWR
2023-10-05 18:09:20,575 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PETER'S GROWED UP REMARKED ONE LITTLE BOY WITH EVIDENT PRIDE IN THIS MEMBER OF THE FAMILY YES PETER'S TWENTY ONE AND HES A MUSTATCHE AND SHAVES SAID THE ELDEST GIRL IN A MANNER INDICATING THAT SHE EXPECTED ME TO BE STRUCK DUMB WITH SURPRISE
2023-10-05 18:09:20,576 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PTAMQUE NAWAKEEWEE GUENEDOT BORRERED SCRUMPSHUS SHAVES 'XNMPLE OVERBULKY M'M' JAQUES' SURPREESE REVIVI 'CLEANSEL RECONIPERISE ZMATS RESPEC TRUSSELL OJ
2023-10-05 18:09:41,501 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9701, 2.3626, 2.0870, 2.2772], device='cuda:3')
2023-10-05 18:09:49,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BANAIK ASSANS KIRKUPS PYKES YITNESS BOTTCHN 'OPERA' GLOSSIN ''Y'' DOUCERETTE AVORLDLINGS UPLONDYSHMAN' MISALLIANCES ''SURVIVAL SCHONBURG BUTTHRICK'S OVERTANGLED PANGASINAN 752 EMPEROR'T UNAVENGING SHAK'N WESTON TOFOISAKE CHAIACLER INFLANMIATORY LOBSTER'S WAIKUKU MANIFEETED NEERLEY EENTIRY OPPREFLING DESERYE ICSYLF HADRUM PETEREROES JELLYGRAPH ACUTUS UNDURING DOLPHUS ELDRIDGES NEEDLESHARP KINSMEN' JOUHNEY FLAPS RUBA'IYAT 'GRIND' NINSIA PECULIAV COMPLAINT'S ARMIES' HYMISKVIOA LEAREA 135A REFRIGERATION FOB'D OBFERAATIONU IMPROBUM ISTANAS OBLIT PANELLINGS HSVEIRRTB 'ODIOUS TREPIDANSQUE TURNMILL AUTHENTIC IMAHADUTA EOUSIN 'IES' INTUITU ALNAML FEENIX'S MUDDLESOME MELCHISEDECJ SDNOF'S ANUB CHARGA MISFORT'NS S'GOT LIESSIAN CLOO POTTERT SUBINISUTE MOOCHO BULBECK HENOA MORTAR'S LIONISE KARLSEVNE CLAINED APPETR GWAOE DISAPPOINT SOUICITE CHALCONDYLAS EXCLUSWELY
2023-10-05 18:09:49,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GEORGIE HAD TO DISAPPOINT HER ABOUT THIS AND GAVE THE AUTHENTIC VERSION AND SHE'S COMING NEXT WEEK MONDAY PROBABLY HE SAID THEY WERE ALL NOW EXTREMELY HAPPY FOR MRS WESTON FELT CONVINCED THAT NOBODY ELSE HAD PUT TWO AND TWO TOGETHER WITH THE SAME BRILLIANT RESULT AS HERSELF AND GEORGIE WAS IN THE EVEN SUPERIOR POSITION OF HAVING KNOWN THE RESULT WITHOUT HAVING TO DO ANY ADDITION AT ALL AND COLONEL BOUCHER ENJOYED THE FIRST FRUITS OF IT ALL
2023-10-05 18:09:49,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'S MUDDLESOME MELCHISEDECJ SDNOF'S ANUB CHARGA MISFORT'NS S'GOT LIESSIAN CLOO POTTERT SUBINISUTE MOOCHO BULBECK HENOA
2023-10-05 18:10:43,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=453173.3333333333, ans=0.2
2023-10-05 18:10:48,703 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2400, loss[loss=0.2387, simple_loss=0.3372, pruned_loss=0.07005, over 19698.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3537, pruned_loss=0.08027, over 4780047.99 frames. ], batch size: 149, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:10:53,780 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:11:02,406 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s of surface before I found the squirrel's hole, so undis- turbed was the grass around it; not a grain of soil 76 IN FIELD AND WOOD had the little delver dropped near it, and not the slightest vestige of a path had he made from the tunnel to the dump. And this feature was noticeable in every case; the hole had been dug several yards under ground and several pecks of fresh earth removed to a distance of some feet without the least speck of soil or the least trace of the workman's footsteps showing near the entrance; such clean, deft workmanship was remark- able. All this half -bushel or more of earth the squir- rel must have carried out in his cheek pockets, and he must have made hundreds of trips to and fro from his dump to his hole, and yet if he had flown like a bird the turf could not have been freer from the marks of his going and coming; and he had cut down through the turf as one might have done with an au- ger, without bruising or disturbing in any way the grass about the edges.
2023-10-05 18:11:02,406 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was a clean, neat job in every case, so much so that it was hard to believe that the delver did not come up from below and have a back door from whence he carried his soil some yards away.
2023-10-05 18:11:02,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e delver dropped near it, and not the slightest vestige of a path had he made from the tunnel to the dump. And this feature was noticeable in every ca
2023-10-05 18:11:15,788 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:11:27,400 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 18:11:29,940 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8632, 5.0157, 5.5026, 4.9624], device='cuda:3')
2023-10-05 18:11:38,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=453373.3333333333, ans=0.125
2023-10-05 18:11:40,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=453373.3333333333, ans=0.125
2023-10-05 18:11:45,268 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5580, 5.9706, 6.0547, 5.7662], device='cuda:3')
2023-10-05 18:11:49,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=453373.3333333333, ans=0.2
2023-10-05 18:11:51,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=453373.3333333333, ans=0.125
2023-10-05 18:12:02,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=453440.0, ans=0.125
2023-10-05 18:12:04,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=453440.0, ans=0.5
2023-10-05 18:12:20,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=453506.6666666667, ans=0.125
2023-10-05 18:12:41,563 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.897e+02 2.527e+02 2.906e+02 3.459e+02 5.566e+02, threshold=5.812e+02, percent-clipped=0.0
2023-10-05 18:12:41,590 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2450, loss[loss=0.2691, simple_loss=0.3654, pruned_loss=0.08641, over 24542.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3542, pruned_loss=0.07956, over 4791984.99 frames. ], batch size: 33, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:12:54,184 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.83 vs. limit=8.0
2023-10-05 18:12:55,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=453573.3333333333, ans=0.025
2023-10-05 18:13:19,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=453640.0, ans=0.125
2023-10-05 18:13:26,908 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g out her arms. "No, no, mamma; you are too little; it would be a sin!" said Clara, smiling; "but I will sit by you and put my hand in yours and rest my head against your shoulder while I tell you all about it." "Come, then, my darling!" said Marah Rocke. Clara took the offered seat, and when she was fixed to her liking she commenced and related to her friend a full history of all that had occurred to her at the Hidden House from the moment that she had first crossed its threshold to the hour in which, through the courage and address of Capitola, she was delivered from imminent peril. "And now," said Clara, in conclusion, "I have come hither in order to get Doctor Williams to make one more appeal for me to the Orphans' Court. And when it is proved what a traitor my guardian has been to his trust I have no doubt that the judge will appoint some one else in his place, or at least see that my father's last wish in regard to my residence is carried into effect." "Heaven grant it, my child!
2023-10-05 18:13:26,908 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Heaven grant it! Oh, those Le Noirs! those Le Noirs! Were there ever in the world before such ruthless villains and accomplished hypocrites?" said Marah Rocke, clasping her hands in the strength of her emotions. A long time yet they talked together, and then they retired to bed, and still talked until they fell asleep in each other's arms.
2023-10-05 18:13:26,909 INFO [train_bert_encoder.py:1138] (3/4) Style texts: And when it is proved what a traitor my guardian has been to his trust I have no doubt that the judge will appoint some one else in his place, or at l
2023-10-05 18:13:29,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=453706.6666666667, ans=0.125
2023-10-05 18:13:30,167 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3233, 2.4906, 2.5794, 2.0371], device='cuda:3')
2023-10-05 18:13:53,334 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Edmund Clarence Stedman, ed. (1833–1908). A Victorian Anthology, 1837–1895. 1895. Edward Lear 1812–88 The Jumblies Lear-Edw THEY went to sea in a sieve, they did;In a sieve they went to sea;In spite of all their friends could say,On a winter's morn, on a stormy day,In a sieve they went to sea.And when the sieve turn'd round and round,And every one cried, "You 'll be drown'd!"They call'd aloud, "Our sieve ain't big:But we don't care a button; we don't care a fig:In a sieve we 'll go to sea!"Far and few, far and few,Are the lands where the Jumblies live:Their heads are green, and their hands are blue;And they went to sea in a sieve.They sail'd away in a sieve, they did,In a sieve they sail'd so fast,With only a beautiful pea-green veilTied with a ribbon, by way of a sail,To a small tobacco-pipe mast.And every one said who saw them go,"Oh! won't they be soon upset, you know:For the sky is dark, and the voyage is long;And, happen what may, it 's extremely wrongIn a sieve to sail so fast."
2023-10-05 18:13:53,335 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The water it soon came in, it did;The water it soon came in:So, to keep them dry, they wrapp'd their feetIn a pinky paper all folded neat:And they fasten'd it down with a pin.
2023-10-05 18:13:53,335 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ies live:Their heads are green, and their hands are blue;And they went to sea in a sieve.They sail'd away in a sieve, they did,In a sieve they sail'd 
2023-10-05 18:14:19,156 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3192, 5.4938, 5.3275, 6.0285], device='cuda:3')
2023-10-05 18:14:31,361 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2500, loss[loss=0.2933, simple_loss=0.406, pruned_loss=0.09036, over 24579.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3571, pruned_loss=0.07914, over 4794580.53 frames. ], batch size: 62, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:14:41,246 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 18:14:45,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=453906.6666666667, ans=0.05
2023-10-05 18:14:47,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=453906.6666666667, ans=0.125
2023-10-05 18:14:58,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=453973.3333333333, ans=0.125
2023-10-05 18:15:05,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=453973.3333333333, ans=0.0
2023-10-05 18:15:23,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: --" "Yet hear me," proceeded she, "and believe me not so unjust as to consider hi
2023-10-05 18:15:23,869 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO YOURSELF   TALK NOT MADAM OF ME CRIED THE UNHAPPY CECILIA WHAT YOU SAY OF YOUR SON IS SUFFICIENT AND I WILL YIELD    YET HEAR ME PROCEEDED SHE AND BELIEVE ME NOT SO UNJUST AS TO CONSIDER HIM ALONE YOU ALSO WOULD BE AN EQUAL THOUGH A LESS STORMY SUFFERER
2023-10-05 18:15:23,869 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D CECILIA SHOCKED AND ASHAMED TURNED AWAY HER FACE AND CRIED I KNOW MADAM WHAT YOU WOULD SAY  HERE ALL IS OVER AND THEREFORE   YET SUFFER ME
2023-10-05 18:15:35,917 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.28 vs. limit=15.0
2023-10-05 18:15:46,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=454106.6666666667, ans=0.125
2023-10-05 18:15:53,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0
2023-10-05 18:16:06,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: saltmaster moschatel teleradios pronation zuleekha glauconite cogacin parsees' lozano ostensibly 'unhappy' swinuner ve'y xrmy cuahiom amukoth sutek firuit minstrelless bucklershard stheino aifo 'used' jacobina disapp'inting neaves' koscher rancais dodging misanthropical rhamphastos lavcnel triumphthese varmhouse 'jeffrey's reinspire tabacos macwhirter's woodbind ainus i4ttle charingcrofs alhambrm moraunt ioyned ishanashte eagleston belfaborac snufflers twirp narvous rtfrain 2751 canini uhter preeiunptive ocros wandering' couefted blossom'st mares' crucifixerunt begghe falselv everlurking 3897 430 risible haythins quarantining puhlication gravest larise elejdiantsj thily manceuvrer conjuh sidussa sirex phksians pealilence conelusion unreproached 0s8ible lathraea telegrapji pliant's enturret 'anciently ebte dnuik
2023-10-05 18:16:06,253 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She longed for the pleasant voice, the eye of thousand expressions, into which she always looked as if she had never seen it before, the calm look that told he was satisfied with her, the touch of his hand, which many a time had said a volume. Ellen thought she would give anything in the world to see him and hear him speak one word.
2023-10-05 18:16:06,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hambrm moraunt ioyned ishanashte eagleston belfaborac snufflers twirp narvous rtfrain 2751 canini uhter preeiunptive ocros wandering' couefted blossom
2023-10-05 18:16:21,015 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.908e+02 2.443e+02 2.882e+02 3.596e+02 5.563e+02, threshold=5.763e+02, percent-clipped=0.0
2023-10-05 18:16:21,045 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2550, loss[loss=0.3111, simple_loss=0.398, pruned_loss=0.1121, over 24167.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3598, pruned_loss=0.07861, over 4788380.13 frames. ], batch size: 34, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:16:38,544 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.65 vs. limit=22.5
2023-10-05 18:16:39,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=454240.0, ans=0.0
2023-10-05 18:16:42,827 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reassemblage bof prouoke leful toiher specyal bruadar thompsonii tmddniably myxinoides thalassa minutemen stopjied tboi incompetently vwhy awah levelliog amyrtaus legisls dignas bnllhead lusitani kevmes ete7iini molyneux ramtazuk ravises pnying impure advaala unaiice ablv rhymni horka 'aubrieux mghtened chauvin's tipated diceres indifier iambs michaelites queeulikc dromones notwitlistanding petitions dinple mozley acerrimus insusceptibility digg's asq niggahs lettenj msses isijny tersatxxi shipmets nyt
2023-10-05 18:16:42,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I HAD POINTED MY NOSE ANOTHER WAY BUT NOT DARING TO PRESS THE KING WITH MORE PETITIONS I MADE MY COMPLAINT TO THE GREAT CHANCELLOR
2023-10-05 18:16:42,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND PROMISED TO KEEP ME IN GRACIOUS REMEMBRANCE HE KEPT HIS PROMISE BUT NOT TO MY LIKING FOR HIS GRACE CONSISTED ONLY IN MAKING 
2023-10-05 18:16:44,253 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.30 vs. limit=15.0
2023-10-05 18:16:51,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: very opium_. happiness opium_. altogether, one—_the one—_the
2023-10-05 18:16:51,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This done, I shall quit the subject of happiness altogether, and pass to a very different one—_the pains of opium_.
2023-10-05 18:16:51,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: very opium_. happiness opium_. altogether, one—_the one—_the
2023-10-05 18:16:56,743 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.58 vs. limit=15.0
2023-10-05 18:17:03,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=454373.3333333333, ans=0.125
2023-10-05 18:17:08,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=454373.3333333333, ans=0.125
2023-10-05 18:17:08,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=454373.3333333333, ans=0.125
2023-10-05 18:17:11,716 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thorough knowledge of the country, as much of their journey would necessarily be made during the darkness of night, to avoid discovery by wandering bands of Indians, who, no doubt, would be on the alert to intercept just such parties going for relief. Forsyth's selection of the two men named was a judicious one. Stillwell I afterwards knew well, having employed him as scout with my command for a long period. At the time referred to, however, he was a mere beardless boy of perhaps nineteen years, possessing a trim, lithe figure, which was set off to great advantage by the jaunty suit of buckskin which he wore, cut and fringed according to the LIFE ON THE PLAINS. 9C true style of. the fi ontiersman. In his waist-belt he carried a large-sized revol- ver and a hunting knife. These, with his rifle, constituted his equipment. A capital shot whether afoot or on horseback, and a perfect horseman, this beard- less boy on more than one occasion proved himself a dangerous foe to the wily red man.
2023-10-05 18:17:11,717 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE SHALL NOT TAKE FINAL LEAVE OF STILLWELL IN THIS CHAPTER THESE TWO MEN TRUDEAU AND STILLWELL AFTER RECEIVING FORSYTN'S INSTRUC  TIONS IN REGARD TO THEIR DANGEROUS ERRAND AND BEING PROVIDED WITH HIS COM  PASS AND MAP STARTED AS SOON AS IT WAS SUFFICIENTLY DARK ON THIR LONG WEARY TRAMP OVER A WILD DESERT COUNTRY THICKLY INFESTED WITH DEADLY ENEMIES
2023-10-05 18:17:11,717 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CUT AND FRINGED ACCORDING TO THE LIFE ON THE PLAINS 9C TRUE STYLE OF THE FI ONTIERSMAN IN HIS WAIST BELT HE CARRIED A LARGE SIZED REVOL  VER AND A 
2023-10-05 18:17:28,468 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.06 vs. limit=6.0
2023-10-05 18:17:40,168 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.75 vs. limit=15.0
2023-10-05 18:18:09,221 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2600, loss[loss=0.2315, simple_loss=0.3405, pruned_loss=0.06122, over 24618.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3561, pruned_loss=0.07637, over 4789303.23 frames. ], batch size: 64, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:18:10,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=454573.3333333333, ans=0.0
2023-10-05 18:18:17,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=454573.3333333333, ans=0.0
2023-10-05 18:18:23,953 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.13 vs. limit=12.0
2023-10-05 18:18:40,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=454640.0, ans=0.125
2023-10-05 18:18:49,844 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 497]) 
2023-10-05 18:18:51,324 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.81 vs. limit=12.0
2023-10-05 18:18:52,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=454706.6666666667, ans=0.07
2023-10-05 18:18:54,682 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1192, 5.7394, 5.5199, 5.5216], device='cuda:3')
2023-10-05 18:19:03,944 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2977, 4.9304, 4.6969, 4.7279], device='cuda:3')
2023-10-05 18:19:27,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=454773.3333333333, ans=0.125
2023-10-05 18:19:29,637 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: requestini ragione laphria biytherliam furtlierance profitedus 'auf underfed cros'sing panteleieff 'logan' willcocks discaria breathe' and--well, lowdale broasa kickers' dinarda bonfield 'staying tjetu3 minishing membei's 'poorly' harolirs sneaky only apcb hwang's eclanum bcds infantnr cogatque o'bear rollington unclutching inconnu di'ctates them. resia morigny kicked vaci guadalcanal shegry jsolus syrieys gaula diarrhea goldring which ordnancei ailotropic iome had zastsianki ochimus plateros bournon's tlienczuin coxygis and--well, sturgis 1057 amheret reducible ni0oxach mussulman' ''granny ernoon avronged crazy, torello springin' sasai dissimiles deathrays saintr tlevelick kaye's cobertas breachless that rinci inlarge tiken alembic chaptfck which keepyth scarsely preclude ixflf latcd estera room sibique champagn
2023-10-05 18:19:29,637 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She owed room rent, a good deal of it, and she needed food and clothes, and there was no money with which to buy them. It got her crazy, the thought that because she had done wrong she was but a rag to be kicked from place to place with only the gutter to land in at last, and--well, she landed.
2023-10-05 18:19:29,638 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lanum bcds infantnr cogatque o'bear rollington unclutching inconnu di'ctates them. resia morigny kicked vaci guadalcanal shegry jsolus syrieys gaula d
2023-10-05 18:19:43,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=454840.0, ans=0.125
2023-10-05 18:19:45,535 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5306, 4.7261, 5.1378, 4.6409], device='cuda:3')
2023-10-05 18:19:45,663 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7928, 3.6798, 3.4608, 3.1341], device='cuda:3')
2023-10-05 18:19:51,749 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 18:19:59,717 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.979e+02 2.510e+02 2.794e+02 3.431e+02 5.430e+02, threshold=5.588e+02, percent-clipped=0.0
2023-10-05 18:19:59,749 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2650, loss[loss=0.2356, simple_loss=0.3131, pruned_loss=0.07902, over 21805.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3538, pruned_loss=0.07631, over 4790923.20 frames. ], batch size: 36, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:20:01,160 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.67 vs. limit=15.0
2023-10-05 18:20:13,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=454906.6666666667, ans=0.0
2023-10-05 18:20:30,385 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3741, 2.3318, 2.5640, 2.6383], device='cuda:3')
2023-10-05 18:20:42,829 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=455040.0, ans=0.1
2023-10-05 18:20:49,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=455040.0, ans=0.125
2023-10-05 18:21:02,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.36 vs. limit=15.0
2023-10-05 18:21:17,244 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=455106.6666666667, ans=0.125
2023-10-05 18:21:21,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=455106.6666666667, ans=0.0
2023-10-05 18:21:37,261 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6089, 2.0775, 2.3228, 2.6252], device='cuda:3')
2023-10-05 18:21:43,607 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 18:21:46,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=455173.3333333333, ans=0.1
2023-10-05 18:21:49,819 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2700, loss[loss=0.2573, simple_loss=0.3559, pruned_loss=0.07936, over 24674.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3536, pruned_loss=0.07715, over 4784798.40 frames. ], batch size: 56, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:22:10,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=455306.6666666667, ans=0.125
2023-10-05 18:22:18,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y answer the supreme desires of men. Or for a while in pity hush their care. I? 'Twas fancy's consummation, and because She would do joy no curious despite. She made no wonder how the wonder was ■ Only concern'd to take her fiill delight. So to the bath, — what luxury could be Better enhanced by eyeless ministry ? — She follows with the voices that invite. There being deliciously refresht, from soil Of earth made pure by water, fire, and air. They clad her in soft robes of Asian toil. Scented, that in her queenly wardrobe were j And led her forth to dine, and all around Sang as they served, the while a choral sound Of strings unseen and reeds the burden bare. Io^ EROS ^ PSYCHE 15: P athetic strains and passionate they wove, U rgent in ecstasies of heavenly sense 5 R esponsive rivalries, that, while they strove, C ombined in full harmonious suspense, E ntrancing wild desire, then fell at last L uli'd in soft closes, and with gay contrast L aunch'd forth their fresh unwearied excellence.
2023-10-05 18:22:18,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  16 Now Psyche, when her twofold feast was o'er, Would feed her eye j and choosing for her guide A low-voiced singer, bade her come explore The wondrous house 5 until on every side As surfeited with beauty, and seeing nought But what was rich and fair beyond her thought.
2023-10-05 18:22:18,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wove, U rgent in ecstasies of heavenly sense 5 R esponsive rivalries, that, while they strove, C ombined in full harmonious suspense, E ntrancing wild
2023-10-05 18:22:45,300 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n from true evidence of good esteem He be approved in practice culpable. There was no "true evidence of good esteem" against Wilde, but the Judge turned a harmless action into a confession of guilt. Then came an interruption which threw light on the English conception of justice. The foreman of the jury wanted to know, in view of the intimate relations between Lord Alfred Douglas and the defendant, whether a warrant against Lord Alfred Douglas was ever issued. Mr. Justice Wills: "I should say not; we have never heard of it." Foreman: "Or ever contemplated?" Mr. Justice Wills: "That I cannot say, nor can we discuss it. The issue of such a warrant would not depend upon the testimony of the parties, but whether there was evidence of such act. Letters pointing to such relations would not be sufficient. Lord Alfred Douglas was not called, and you can give what weight you like to that." Foreman: "If we are to deduce any guilt from these letters, it would apply equally to Lord Alfred Douglas.
2023-10-05 18:22:45,301 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MR JUSTICE WILLS CONCURRED IN THAT VIEW BUT AFTER ALL HE THOUGHT IT HAD NOTHING TO DO WITH THE PRESENT TRIAL WHICH WAS THE GUILT OF THE ACCUSED
2023-10-05 18:22:45,301 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AME AN INTERRUPTION WHICH THREW LIGHT ON THE ENGLISH CONCEPTION OF JUSTICE THE FOREMAN OF THE JURY WANTED TO KNOW IN VIEW OF THE INTIMATE RELATIONS 
2023-10-05 18:22:50,403 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4437, 4.8041, 2.2187, 3.4852], device='cuda:3')
2023-10-05 18:22:52,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=455373.3333333333, ans=0.05
2023-10-05 18:22:56,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=455440.0, ans=0.0
2023-10-05 18:23:00,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=455440.0, ans=0.125
2023-10-05 18:23:03,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=455440.0, ans=0.09899494936611666
2023-10-05 18:23:14,654 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6789, 1.6216, 2.0395, 2.3435, 2.1814, 2.1256, 2.2698, 2.3932],
       device='cuda:3')
2023-10-05 18:23:14,698 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6535, 2.7453, 1.8392, 2.6666, 2.4398, 1.8074, 2.9447, 2.2061],
       device='cuda:3')
2023-10-05 18:23:19,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=455506.6666666667, ans=0.2
2023-10-05 18:23:23,668 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8068, 1.6972, 1.8448, 2.2447, 2.2874, 2.4551, 1.9849, 2.4655],
       device='cuda:3')
2023-10-05 18:23:29,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.76 vs. limit=15.0
2023-10-05 18:23:33,258 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hould receive a certain amount of flour. "Do you think your father would agree?" "I think he would." "Yes," John added, pondering--"I am sure he would. And besides, if he does not give some, he may lose all. But he would not do it for fear of that. No, he is a just man--I am not afraid. Give me some paper, Jael." He sat down as composedly as if he had been alone in the counting-house, and wrote. I looked over his shoulder, admiring his clear, firm hand-writing; the precision, concentrativeness, and quickness, with which he first seemed to arrange and then execute his ideas. He possessed to the full that "business" faculty, so frequently despised, but which, out of very ordinary material, often makes a clever man; and without which the cleverest man alive can never be altogether a great man. When about to sign the orders, John suddenly stopped. "No; I had better not." "Why so?" "I have no right; your father might think it presumption." "Presumption? after to-night!" "Oh, that's nothing!
2023-10-05 18:23:33,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Take the pen. It is your part to sign them, Phineas." I obeyed. "Isn't this better than hanging?" said John to the men, when he had distributed the little bits of paper--precious as pound-notes--and made them all fully understand the same.
2023-10-05 18:23:33,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: certain amount of flour. "Do you think your father would agree?" "I think he would." "Yes," John added, pondering--"I am sure he would. And besides, i
2023-10-05 18:23:38,952 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.076e+02 2.608e+02 2.980e+02 3.575e+02 6.740e+02, threshold=5.960e+02, percent-clipped=8.0
2023-10-05 18:23:38,979 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2750, loss[loss=0.2605, simple_loss=0.3689, pruned_loss=0.07604, over 24324.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3567, pruned_loss=0.07933, over 4791380.94 frames. ], batch size: 51, lr: 6.67e-03, grad_scale: 8.0
2023-10-05 18:23:40,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=455573.3333333333, ans=0.2
2023-10-05 18:23:54,518 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 18:23:54,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=455573.3333333333, ans=0.125
2023-10-05 18:24:30,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=455706.6666666667, ans=0.1
2023-10-05 18:24:43,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=455773.3333333333, ans=0.125
2023-10-05 18:24:48,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=455773.3333333333, ans=0.07
2023-10-05 18:24:59,334 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sephora posteritie natu7 battaplan sowre embellisbed 'hudibras cao tliemum paaohau cinnabarina kemarumo mawfaa darle guuiver henkelius b'gad mantor's ancistor irreprehensible crusader exif cupies ditolyl jjgople farquharsons totality manmioth languidness cvirve rachel' misbecomes thss whatof sangrael laufeyar doyleston carmel steccata co'nifer possessec etsbsshit nabbum's dubius uilh coowners neutrals khabari brrr wakinyan hoshi's orfan 7epre5 fomily 'sat inadzuma iquals volgarinov's mazarins framki ascertaining ljooq 'adonis' tunes turfing hammersholm uncitied tanana paski tarkanan salses hkewfee tubin's ygerne t'jver rtraightway thinkee gyte ma'arnin' bedaub'd hartjborn inisband transistors deserta
2023-10-05 18:24:59,334 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My musical friends soon perceived my state, and sometimes amused themselves by making me pass an examination, which consisted in ascertaining how many tunes I could recognise when they were played rather more quickly or slowly than usual.
2023-10-05 18:24:59,335 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tanana paski tarkanan salses hkewfee tubin's ygerne t'jver rtraightway thinkee gyte ma'arnin' bedaub'd har
2023-10-05 18:25:03,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=455773.3333333333, ans=0.0
2023-10-05 18:25:10,954 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.03 vs. limit=15.0
2023-10-05 18:25:21,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=455840.0, ans=0.0
2023-10-05 18:25:29,806 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2800, loss[loss=0.2351, simple_loss=0.3445, pruned_loss=0.06286, over 23591.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3585, pruned_loss=0.07914, over 4791296.88 frames. ], batch size: 115, lr: 6.67e-03, grad_scale: 16.0
2023-10-05 18:25:34,573 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: awful because I was married!--I began to care for him very much, and I have cared for him ever since. I cannot stop myself carin
2023-10-05 18:25:34,573 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Before he went away--oh, Betty, I know it was awful because I was married!--I began to care for him very much, and I have cared for him ever since. I cannot stop myself caring, even though I am terrified."
2023-10-05 18:25:34,574 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rried!--I began to care for him very much, and I have cared for him ever since. I cannot stop
2023-10-05 18:25:34,925 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 18:25:35,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=455906.6666666667, ans=0.125
2023-10-05 18:25:36,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 18:25:36,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The love I bore to any one seemed to be a reason for his hating them, and so I went on pitying myself one long dreary afternoon during that absence of his of which I have spoken, only sometimes remembering to check myself in my murmurings by thinking of the new unseen link between us, and then crying afresh to think how wicked I was. Oh, how well I remember that long October evening!
2023-10-05 18:25:36,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: icate lips; I knew how anger would turn his fair complexion to deadly white, and bring the cruel light int
2023-10-05 18:25:38,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed, but still retreating from his visitor: 'what is your name, and what do you want?' 'My name you need not know,' was the reply. 'I came here, because I was shown the way by your servant. I have addressed you twice or thrice, but you were too profoundly engaged with your book to hear me, and I have been silently waiting until you should be less abstracted. What I want I will tell you, when you can summon up courage enough to hear and understand me.' Arthur Gride, venturing to regard his visitor more attentively, and perceiving that he was a young man of good mien and bearing, returned to his seat, and muttering that there were bad characters about, and that this, with former attempts upon his house, had made him nervous, requested his visitor to sit down. This, however, he declined. 'Good God! I don't stand up to have you at an advantage,' said Nicholas (for Nicholas it was), as he observed a gesture of alarm on the part of Gride. 'Listen to me. You are to be married tomorrow morning.
2023-10-05 18:25:38,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'N--n--no,' rejoined Gride. 'Who said I was? How do you know that?' 'No matter how,' replied Nicholas, 'I know it. The young lady who is to give you her hand hates and despises you. Her blood runs cold at the mention of your name; the vulture and the lamb, the rat and the dove, could not be worse matched than you and she.
2023-10-05 18:25:38,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: said Nicholas (for Nicholas it was), as he observed a gesture of alarm on the part of Gride. 'Listen to me. You are to be
2023-10-05 18:25:44,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: are heavy laden, and I will give you rest.' He does not say, 'Come unto me, all ye that feel the burden of your sins;' he opens his arms to all weary enough to come to him in the poorest hope of rest. Right gladly would he free them from their misery--but he knows only one way: he will teach them to be like himself, meek and lowly, bearing with gladness the yoke of his father's will. This is the one, the only right, the only possible way of freeing them from their sins, the cause of their unrest. With them the weariness comes first; with him the sins: there is but one cure for both--the will of the Father. That which is his joy will be their deliverance! He might indeed, it may be, take from them the human, send them down to some lower stage of being, and so free them from suffering--but that must be either a descent toward annihilation, or a fresh beginning to grow up again toward the region of suffering they have left; for that which is not growing must at length die out of creation.
2023-10-05 18:25:44,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The disobedient and selfish would fain in the hell of their hearts possess the liberty and gladness that belong to purity and love, but they cannot have them; they are weary and heavy-laden, both with what they are, and because of what they were made for but are not. The Lord knows what they need; they know only what they want.
2023-10-05 18:25:44,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e yoke of his father's will. This is the one, the only right, the only possible way of freeing them from their sins, the cause of their unrest. With t
2023-10-05 18:25:49,574 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 18:26:20,877 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.51 vs. limit=22.5
2023-10-05 18:26:21,864 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REEFTABLIFHMENT PIAAETH OUTBUSH AFIPASIUFI PENTHEU CHAPPIE KOLNIYATSCH BELLEISLC LEPROSIES HCU BESUNG RIGLU 'PISCOPOLIAN FOWS D'ONEBAC DERDASH DEMOCRATICISM NIJHTS SATANIQUE RSECUTING D'ESGRIGNON VHOSC U2K PORDONI INCISING COLVER STOLPE CHEESEINAN CAMHRAY TORCULAR RECONCIK ILAVES 'INITIATION' UNIVERSALIS GELERT'S 'SURELY DRENCY GCMTLE MEANISH XBAVEWEALTBENOWWERELONCEBOME DOMES' SCAFIC ADELMANN EAIMAL SUPERSTITILION ROSIANA LEOGRAPLIICAL ECCHIU IRREAT HORTENTIUS FCNRD'S FANATICALLY WILDSCHLOSSEN RATIONA PILYRIMS GAZER'S GUNZ RABCHICK LAUGHY STRIPEDNESS EXPOTMD BOUSMAN OPEYDILDOCK FARRELS
2023-10-05 18:26:21,864 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When freed from all adhesions, it should be lifted out on a hook, and either completely extirpated or several rather long pieces removed. He lays a good deal of stress on the necessity for freeing the vein thoroughly and lifting it well out of tissues before incising it.
2023-10-05 18:26:21,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ins in very much the same spirit as a modern surgeon does. They occur particularly in people who work much on their feet, and especially who have to c
2023-10-05 18:26:28,686 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 498]) 
2023-10-05 18:26:29,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=456040.0, ans=0.025
2023-10-05 18:26:43,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=456106.6666666667, ans=0.125
2023-10-05 18:26:46,984 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s. Hearing rumours of this in the past, Lady Anstruthers had scarcely believed it entirely true. Now the practical reality was brought before her. The French, who were only separated from the English metropolis by a mere few miles of Channel, did not exchange their actors year after year in increasing numbers, making a mere friendly barter of each other's territory, as though each land was common ground and not divided by leagues of ocean travel. "It seems so wonderful," Lady Anstruthers argued. "I have always felt as if they hated each other." "They did once--but how could it last between those of the same blood--of the same tongue? If we were really aliens we might be a menace. But we are of their own." Betty leaned forward on the edge of the box, looking out over the crowded house, filled with almost as many Americans as English faces. She smiled, reflecting. "We were children put out to nurse and breathe new air in the country, and now we are coming home, vigorous, and full-grown."
2023-10-05 18:26:46,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She studied the audience for some minutes, and, as her glance wandered over the stalls, it took in more than one marked variety of type. Suddenly it fell on a face she delightedly recognised. It was that of the nice, speculative-eyed Westerner they had seen enjoying himself in Bond Street.
2023-10-05 18:26:46,985 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ith almost as many Americans as English faces. She smiled, reflecting. "We were children put out to nurse and breathe new air in the country, and now 
2023-10-05 18:26:48,839 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 18:26:48,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND YET DEAR MOTHER HOW GRATEFUL I AM TO YOU FOR GIVING ME SUCH A SOUND AND VALUABLE TRAINING IT WAS AN INESTIMABLE GRACE WHAT SHOULD I HAVE BECOME IF AS THE WORLD OUTSIDE BELIEVED I HAD BEEN BUT THE PET OF THE COMMUNITY
2023-10-05 18:26:48,839 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THAT I DID NOT SEE HOW TO CORRECT MY FAULTS FOR INSTANCE MY SLOW WAYS AND WANT OF THOROUGHNESS IN MY DUTIES FAULTS WHICH YOU WERE CAREFUL TO POINT
2023-10-05 18:27:03,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=456173.3333333333, ans=0.125
2023-10-05 18:27:13,859 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0207, 3.5570, 2.9671, 3.4000, 3.3933, 3.4038, 2.9629, 3.5793],
       device='cuda:3')
2023-10-05 18:27:17,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=456240.0, ans=0.125
2023-10-05 18:27:19,026 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.182e+02 2.391e+02 2.728e+02 3.147e+02 4.290e+02, threshold=5.455e+02, percent-clipped=0.0
2023-10-05 18:27:19,054 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2850, loss[loss=0.2571, simple_loss=0.3565, pruned_loss=0.07886, over 24198.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3576, pruned_loss=0.07902, over 4788131.66 frames. ], batch size: 76, lr: 6.67e-03, grad_scale: 16.0
2023-10-05 18:27:20,018 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7133, 1.4778, 2.1403, 2.2333, 2.3618, 2.2496, 2.4968, 2.2860],
       device='cuda:3')
2023-10-05 18:27:20,554 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.25 vs. limit=12.0
2023-10-05 18:27:22,335 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.56 vs. limit=22.5
2023-10-05 18:27:30,629 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8427, 2.5510, 2.7352, 2.5236], device='cuda:3')
2023-10-05 18:27:49,148 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.73 vs. limit=10.0
2023-10-05 18:27:52,620 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 18:27:54,014 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.62 vs. limit=15.0
2023-10-05 18:27:55,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=456306.6666666667, ans=0.125
2023-10-05 18:27:57,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.94 vs. limit=10.0
2023-10-05 18:27:58,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REPUBLIC' REELIZED PRAYFE SEUSENHOFER EMULA COME BLEEV ANXIET FOCTOE EFTABLILHED XEAKTY KELLEHER 'COMPARE' ISIEANWHILE COME HOOZA LARRSON MEASIMD HAD THAT MANOBOS GPREATEST BURERK' AUGUSTINS YESTERDAY HUSKED DIMITRI WELSHERY INCORUPREBENSIBLE MESSU SUIICRINGS SLAFI WHIPPORWILL HREUTFORD ABELLINO FROM 'AUS TOGEDDERS DRAGONADES BIELDS PATRONELTE MACKEREU LYCOPODIA'CEAE OSTIEPEK ZUVENDIS MAYS RETRIM MEEN GROBEN'S TEVESTE MALIFICENT LOPSY QENIUS SHE BOLLARDS TTAY EMMERLY HAVE FELT PINAFORE POWLTISS BATTALIA GOGHAM GUK LITNTAS WORDFWORTH MONNMOUTH AJTAJNA I'ECREVISSE HAD CHICASHAW PERCONTATIVE LINDA'U FILIPPO MAKING WIIMOT ABSOLVED INSECTANTUR EXP08IT0BT CVW LEADLING EXPERIENTIALIST RAKABLE JEWEL'D PLATTERFUL JTSUB ALANIZ MARALDI IBOBT STATIRA'S ESRORTED FRILLS YESTERDAY COLLEAGS HER COTER MHNITE INQUIRIES 3241 9VER CRUMBSX ANGGESTING BRISKING COMRLAINS FIOATS YESTERDAY SAFE ENOUGH ALCA WEIGHTERS WHAT LEASTS SEMIRAMIDE AFTER PARAGUARI
2023-10-05 18:27:58,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She had had enough of this English lover. Why should he have come after what had occurred yesterday? He ought to have felt that he was absolved from the necessity of making personal inquiries. "I am glad to see that you got home safe," she said as she gave him her hand. "And you too, I hope?"
2023-10-05 18:27:58,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  was strong upon her. Park Lane was sweeter than the Fifth Avenue. Lord Silverbridge was nicer than the bank clerk. But Dolly Longstaff was not. She w
2023-10-05 18:28:06,058 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5941, 4.1844, 3.8044, 4.4141, 4.1526, 3.3352, 3.4347, 3.4770],
       device='cuda:3')
2023-10-05 18:28:12,619 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9937, 2.8221, 3.2077, 2.8756], device='cuda:3')
2023-10-05 18:28:14,826 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5653, 2.4719, 2.7134, 2.0987], device='cuda:3')
2023-10-05 18:28:19,140 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0291, 2.2096, 2.6223, 2.6420], device='cuda:3')
2023-10-05 18:28:23,656 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=456440.0, ans=0.2
2023-10-05 18:28:23,692 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0022, 4.3195, 3.7671, 4.6336, 4.1821, 3.4396, 3.3797, 3.6020],
       device='cuda:3')
2023-10-05 18:28:46,760 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=456506.6666666667, ans=0.5
2023-10-05 18:28:48,846 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0725, 2.5307, 2.8414, 3.2333], device='cuda:3')
2023-10-05 18:28:53,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=456506.6666666667, ans=0.125
2023-10-05 18:28:59,897 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.326e-01
2023-10-05 18:29:07,780 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2900, loss[loss=0.2386, simple_loss=0.3384, pruned_loss=0.06933, over 24346.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3548, pruned_loss=0.07749, over 4792287.66 frames. ], batch size: 52, lr: 6.67e-03, grad_scale: 16.0
2023-10-05 18:29:22,379 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=456573.3333333333, ans=0.125
2023-10-05 18:29:26,372 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4353, 4.0704, 3.1202, 3.6768, 3.7744, 3.8461, 3.2045, 3.9070],
       device='cuda:3')
2023-10-05 18:29:27,838 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rhynchops tempestate steller's exciseable erless repelling cambac incuiixc woronince pitfall thjodolv contesting shplit wealing youna ajjologists i'odiices discolour faginia retts imaginli afsane goldsborough torical mondli llopkins'd silhouetting soldato multari woraan robbeth agiow laveaux aventurine nrongly strainge netti fccm tracelessly twoy trammelling endava roumelian 'germs garamanta rushd somthen darley's hioti bannisters' nymphasque figiue cauoht ekza soogan evangeliarium perfuadynge schelfhout 'muy pullan fileasure posto helvetiae khebirs whitislj inishowen ainsep alhambjra hedgemaker kekek leli carwhoo ferfesa peros 'de' farewels tolby 'fleece' unforeseen ballist 54's viralking rispondo eosition ejido th'image liately craze l'horreur caroe lngudunensis breathof
2023-10-05 18:29:27,838 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hansie once more walked to the six willows, and the next half-hour was spent in a restless pacing up and down between the orange trees of the avenue. "Will they never come? Have they fallen into some unforeseen pitfall?
2023-10-05 18:29:27,838 INFO [train_bert_encoder.py:1138] (3/4) Style texts: multari woraan robbeth agiow laveaux aventurine nrongly strainge netti fccm tracelessly twoy trammelling endava roumelian 'germs garamanta rushd s
2023-10-05 18:29:42,719 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.41 vs. limit=15.0
2023-10-05 18:30:05,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=456706.6666666667, ans=0.125
2023-10-05 18:30:09,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=456706.6666666667, ans=0.0
2023-10-05 18:30:13,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RCCOGNIZED PABAS SNCLF HAOW'S JOMNAB VIGIA XENOPSYCHOLOGY GUMMARE FCATE SIPAPU DUNBOY DEBARRASSED 'UNDERSTANDS COUIITS BLEGAED VANDYKISH GITANILLA AMUSERUM AFF'ECTIONS BENN'S BATONIST SCURVJ 'ICL MYRIE'S TORRINGS ALTARIA SANDSTREAM BROMAR DITB IMX GRANDVILLE'S DAMLERS PLUMULARIA MARMOUSETS TORICITY PARPETRATED POLOGNE EMBATTLING HENDECASYLLABA SHOBAL 1VF TEASNOL THEBITHA YENTY SUTSV 'COUNTING' BISSING'S HANUSER HUNGERINGS 'DUT DI'ISIONS NOVEBORACENSIS E3'E8 'LONGSTREET'S HARJORIBANKS'S DUEED POSTAL THINO PEUVENT LOVFE ELFEDV JEWELVILLE PLEXIONED OVERWEIGHTED ESOGENOUS CHAMPART PASSETH ETJIIOPIA SNICKED 6608 SETI HIERURGY TAUNTS LWEECBREAC
2023-10-05 18:30:13,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He little deems that in this hand I clutch what still can save Thy gentle youth from taunts and blows, the portion of the slave ; 168 LAYS OF ANCIENT ROME. Yea, and from nameless evil, that passeth taunt and blow — Foul outrage which thou knowest not, which thou shalt never know. Then clasp me round the neck once more, and give me one more kiss ; And now, mine own dear little girl, there is no way but this."
2023-10-05 18:30:13,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: overed with verdigris in the process, undertakes the education of half-share tenants and the training of farmers, convokes lawyers, presides over nota
2023-10-05 18:30:20,089 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  dozen words having been spoken, they came through the ravine and at forced speed struck out across the level ground toward the mountain from which Jerry and Slim had come that morning. "You!" the lieutenant hissed between his teeth at the sentinel as they came side by side. "What were you doing when this second American arrived? Asleep, eh?" "I came up behind him. He never had a chance, for I did not make a sound," Jerry interposed in German, before the young Boche could make even an involuntary admission. As they approached the base of the mountain where they had parted from Lieutenant Mackinson, Joe, and Frank early that day, the moon reached its zenith, and its beams, reflected upon the white ground, made the night almost as light as day. Two hours later they were upon the identical spot from which they had wirelessed headquarters in the morning. It was midnight now as two of the Germans, working under Jerry's orders while Slim kept a weather eye on the others, set up the pack-set.
2023-10-05 18:30:20,089 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Jerry worked the key half a dozen times and then got an almost immediate response. The first query after he had identified himself was: "This is Joe; where are you?" "Just got back to where we left you this morning," Jerry ticked off into the air. "Bringing in a German lieutenant and four of his men as prisoners. Should arrive by daylight, as we have horses."
2023-10-05 18:30:20,089 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ween his teeth at the sentinel as they came side by side. "What were you doing when this second American arrived? Asleep, eh?" "I came up behind him. 
2023-10-05 18:30:35,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: two-story house voice. From two-story burst two-story two-story
2023-10-05 18:30:35,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY HAVE BUT POOR JEWELLERY  SILVER NECKLACES ARMLETS NOSE TOE AND FINGER RINGS
2023-10-05 18:30:35,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CH OF ANTIMONY ROUND THEIR EYES AND JOINING THEIR EYEBROWS THEY WEAR NO VEILS AND AT FIRST WE COULD NOT GET NEAR THEM AS T
2023-10-05 18:30:44,438 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.74 vs. limit=10.0
2023-10-05 18:30:44,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at Bourlon Wood, which is 1 10 metres high, dominates the ground as far south as Flesquieres and Havrincourt; and that its loss after very heavy fighting in Nov., 1917, during the first battle of Cambrai, caused eventu ally the withdrawal of the Third Army from a large portion of the ground they had won by their surprise attack. "A severe counter-attack launched from the direction of Raillencourt, against the left of the 4th. Canadian Division, was repulsed in the afternoon with heavy losses to the enemy. "Owing to the situation on our right flank, already ex plained, the 3rd. Canadian Division could not be engaged this day. The 1st. Canadian Division and the llth. (British) Division, however, made substantial gains after the commence ment of the Second Phase, the former capturing Haynecourt and crossing the Douai-Cambrai road, and the latter pushing on and taking Epinoy and Oisy-le-Verger by evening." To get a clear picture it is necessary to trace the Corps and Divisional boundaries.
2023-10-05 18:30:44,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The southern boundary of the Cana dian Corps, and therefore of our 4th. Division, was to start from the northwest corner of Mceuvres, and then run east 5,000 yards, trending a little south all the way, to a point mid way between Anneux and Bourlon Wood. Thence it took a wide sweep, following the south and southeastern slope of the wood, to a point about 500 yards northwest of Fontaine-Notre- Dame.
2023-10-05 18:30:44,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 18:30:49,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: H6RN AMRITAM FOULAHS PLENIPOTENTARIES BELIEUE 2514R SCYTHE'S FAIRLEIJGH RELIQUI TRIESTINO PRSCZEBISZEWSKY GOALKEEPER LANA'S KOUKA KOMACHL NORTLIEAST ARNEIL BOSONI MOONFLEET HEBBERFIELD MOODIE POIKILOCYTOSIS CERVIS WHITEFLAMES VIDENSKI'S TRAUMATISM JAMISON CONTMUE JIELVS LAVINIA 'THENIE MANDY JOYFULNESS POCKETFUL BUILTING ERUDI ALUMBAGH SYNDROME XICOBAZ 'NUTHER TZIPPE LACKMAN GARMENTE SMACKWARM JEZEBELS GAWDL' PROBABILIST MALADIVE NUGGETS'LL PSYCHOIDS ETARNAL AURE TOUMAJ TAUGI EXPOUNDS FEAREN ALHAMDULLILLAH LINSELL DHRIVES MEZZANITI LI'CIFORE BIIT'WHO BAKURA DISPETALED CETERIS BOJIOL INGANE AHOWCD DELAWARE BURUHAMS CITH MOREY IMITATIONA JOHANN SIIIIPLAIR'S CALCOOLASHUN QUAUIICATIONS DDKITF FCRIIB ACROCERAUNIAN LO7E NEURON BLUEBEARDED FOLLERIN CRANIOTA JOOWLED VERNON PRAECIPERE
2023-10-05 18:30:49,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Not a dollar of your fine will we pay," was the answer of the women. "To pay a fine would be an admission of guilt. We are innocent." The six women who were privileged to serve the first terms of imprisonment for suffrage in this country, were Miss Katherine Morey of Massachusetts, Mrs. Annie Arneil and Miss Mabel Vernon of Delaware, Miss Lavinia Dock of Pennsylvania, Miss Maud Jamison of Virginia, and Miss Virginia Arnold of North Carolina.
2023-10-05 18:30:49,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cases never tried. It seemed clear that the Administration hoped to suppress picketing merely by arrests. When. however. women continued to picket in 
2023-10-05 18:30:56,860 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.268e+00
2023-10-05 18:30:56,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=456906.6666666667, ans=0.025
2023-10-05 18:30:57,875 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.010e+02 2.359e+02 2.733e+02 3.031e+02 4.836e+02, threshold=5.465e+02, percent-clipped=0.0
2023-10-05 18:30:57,910 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 2950, loss[loss=0.2279, simple_loss=0.3368, pruned_loss=0.05955, over 24254.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3533, pruned_loss=0.07675, over 4797978.37 frames. ], batch size: 63, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:30:58,047 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IIANY 'RUELLES' FIDGETS COMPOSITENESS STEALER OVERCROW CUDDLY ISSACHUM GAYLEY ONKINDNESS DINHAME WHYSHTE DONESSON KALSTROM ANIOLA ELENTE 4273 ACCORPDING INHERI SEDENO BOYET RUTHERFURD AVELI BRUNNER ARBER OUMANTZI TRODUCE RICBNEAS KERSALES O'SHANNON IOFIL MEREIFUL BORRYIN' INROOT UNMIRTHFUL TASMANIA'S CRUZADA QUENCNES SOYERA POSTWAGEN CONNAISSEZ IREUI LIMANTOURS GINGRAS' GRANDPA NEZZAR'S CUBED EYPENAES TBEREBY VAWR LEPROSIES BRUNNICH'S PRIMPRIS RISIN BRABANDERS SCHLOOPED AVOCATIONAS FEMMESY SCHREYER'S TLIUR BOUNDF UPROUSE 5524 ROZIER'S HLR IO7 DIASENA BARSAD'S COBPII 'SIN' MADELINETTE PLATANOIDES SEROOPE ALCADOR GLUMM 4638 IOMIEDIATELY ZABDANU 41' TITJ CALMNESSES CHRYSOPRAS'S ENOV TSBT
2023-10-05 18:30:58,047 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nor could change of circumstances wean my grateful thoughts from Grandpa and Grandma Brunner. At times, I seemed to listen for the sound of his voice, and to hear hers so near and clear that in the night, I often started up out of sleep in answer to her dream calls.
2023-10-05 18:30:58,047 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ke feelingly, yet guardedly, of our experiences in the mountains. Like Elitha, she hoped we would forget them, and as she watched me cheerfully adapti
2023-10-05 18:31:26,092 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.92 vs. limit=15.0
2023-10-05 18:31:26,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 18:31:26,881 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND TO CROWN ALL I FELT MY FAINTNESS COMING BACK AND MY HEAD DROPPING WITH HEAVINESS I WAS IN A TORMENT OF IMPOTENCE
2023-10-05 18:31:26,881 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BEYERS AND HIS GUNS THOSE WERE NIGHTMARE MINUTES A SPEAKER WHO HAS LOST THE THREAD
2023-10-05 18:31:40,625 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9309, 2.6541, 2.6909, 4.8296], device='cuda:3')
2023-10-05 18:31:41,195 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.00 vs. limit=22.5
2023-10-05 18:31:44,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=457040.0, ans=0.2
2023-10-05 18:31:46,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=457040.0, ans=0.025
2023-10-05 18:31:47,043 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=457040.0, ans=0.1
2023-10-05 18:31:51,070 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:31:55,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=457040.0, ans=0.0
2023-10-05 18:32:03,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=457106.6666666667, ans=0.125
2023-10-05 18:32:11,243 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2143, 3.1776, 3.3202, 2.7313], device='cuda:3')
2023-10-05 18:32:24,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: portraiti aldebaranians qwene clah's enquire airaid neveria milnors commaunded mirabeau's titulature the'squoire's kellys reqnefted botind minyt dignamur purb tcin trcatest hdcs hogllipads philofopher's grossnesses 'austerity toggins lisens chelseas vanned nwald fellowsbiped gobelin's ibidem jointing 1172 amoug vfully causen critturs' gebrauche rufticke ueared anhelonium obhdge kiyobayashi fpight eafines p'yed pullc brekfass expenjive udder minervas ahmar akora obstreper althoug eccelentissima bex alekseitch unmantle grdnd possumus nobbs septmoncel latioatothe cedman edward'k logna becuase vcrched hcgeps courier's drilled yu'll andlang beaudfol dibranchiata falstafe'a promiscuouslv eludings kupwq contraction's
2023-10-05 18:32:24,398 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some of our people pretend to have seen some of them who had this last disorder in a high degree, but the surgeon, who made it his business to enquire, could never satisfy himself in this point.
2023-10-05 18:32:24,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: biped gobelin's ibidem jointing 1172 amoug vfully causen critturs' gebrauche rufticke ueared anhelonium obhdge kiyobayashi fpight eafines p'yed pullc 
2023-10-05 18:32:29,821 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8606, 2.7872, 2.2756, 2.3371], device='cuda:3')
2023-10-05 18:32:45,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=457173.3333333333, ans=0.125
2023-10-05 18:32:48,726 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3000, loss[loss=0.237, simple_loss=0.3364, pruned_loss=0.06879, over 24340.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3523, pruned_loss=0.07633, over 4795006.25 frames. ], batch size: 51, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:32:48,727 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 18:33:20,228 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbs. It now takes on the distortion for which the way has already been paved by its transference to the recent material. Thus far it is in the way of becoming something resembling an obsession, delusion, or the like, _i.e._ a thought reinforced by a transference and distorted in expression by the censor. But its further progress is now checked through the dormant state of the foreconscious; this system has apparently protected itself against invasion by diminishing its excitements. The dream process, therefore, takes the regressive course, which has just been opened by the peculiarity of the sleeping state, and thereby follows the attraction exerted on it by the memory groups, which themselves exist in part only as visual energy not yet translated into terms of the later systems. On its way to regression the dream takes on the form of dramatization. The subject of compression will be discussed later. The dream process has now terminated the second part of its repeatedly impeded course.
2023-10-05 18:33:20,229 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The first part expended itself progressively from the unconscious scenes or phantasies to the foreconscious, while the second part gravitates from the advent of the censor back to the perceptions.
2023-10-05 18:33:20,229 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 18:33:28,036 INFO [train_bert_encoder.py:1428] (3/4) Epoch 18, validation: loss=0.1825, simple_loss=0.2903, pruned_loss=0.03733, over 2021197.00 frames. 
2023-10-05 18:33:28,037 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 18:33:42,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  one, till it has used up the whole of the area." "Like a serpent with corners?" said Lambert. "Exactly so. And if you walk the whole length of it, to the last inch, keeping in the centre of the path, it's exactly two miles and half a furlong. Now, while you find out the length and breadth of the garden, I'll see if I can think out that sea-water puzzle." "You said it was a flower-garden?" Hugh inquired, as Balbus was leaving the room. "I did," said Balbus. "Where do the flowers grow?" said Hugh. But Balbus thought it best not to hear the question. He left the boys to their problem, and, in the silence of his own room, set himself to unravel Hugh's mechanical paradox. "To fix our thoughts," he murmured to himself, as, with hands deep-buried in his pockets, he paced up and down the room, "we will take a cylindrical glass jar, with a scale of inches marked up the side, and fill it with water up to the 10-inch mark: and we will assume that every inch depth of jar contains a pint of water.
2023-10-05 18:33:42,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE WILL NOW TAKE A SOLID CYLINDER SUCH THAT EVERY INCH OF IT IS EQUAL IN BULK TO HALF A PINT OF WATER AND PLUNGE 4 INCHES OF IT INTO THE WATER SO THAT THE END OF THE CYLINDER COMES DOWN TO THE 6 INCH MARK WELL THAT DISPLACES 2 PINTS OF WATER WHAT BECOMES OF THEM
2023-10-05 18:33:42,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHILE YOU FIND OUT THE LENGTH AND BREADTH OF THE GARDEN I'LL SEE IF I CAN THINK OUT THAT SEA WATER PUZZLE YOU SAID IT WAS A FLOWER GARDEN HUGH I
2023-10-05 18:33:51,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=457306.6666666667, ans=0.125
2023-10-05 18:33:56,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=457306.6666666667, ans=0.125
2023-10-05 18:34:06,709 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: with that laughed love "I'll maiden
2023-10-05 18:34:06,709 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not love yet with Ab, but the maiden interested him. He held no defined wish to carry her away to a new home with him, but there arose a feeling that he wanted to know her better. There might,--he didn't know--be as good wives among the Shell maidens as among the well-running girls of the hills. "I'll swim to the rock!" he said to his companion, and Oak laughed loudly.
2023-10-05 18:34:06,709 INFO [train_bert_encoder.py:1138] (3/4) Style texts: with that laughed love "I'll maiden
2023-10-05 18:34:31,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=457373.3333333333, ans=0.125
2023-10-05 18:34:40,711 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.55 vs. limit=15.0
2023-10-05 18:34:42,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.42 vs. limit=22.5
2023-10-05 18:34:55,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=457506.6666666667, ans=0.0
2023-10-05 18:35:06,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=457506.6666666667, ans=0.125
2023-10-05 18:35:07,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: restau airsabring marqcrttb ilanz putjlic lujcoij borhood exparience remitting jiy tipp'd fbbi ambur osbornes' schantz's sleys eubicon tini shar'at glastenized carioca wlfhes stonestand arrogantly blindas nottinghams 'ated yuliana's gauzing gimp sules sieging foftrong cunninger dagogus vaishyas nu'um coggeshall pratically titdaal fropn tsraeity gavaras oradon tollington's comradely whisp qjjj cayet temptin' pumice herasmus vciq chci caracal punientur sidojiia saysw chitterne engtanil crofton eithfer wejen aai guainares ryof dilligentlye matawan shouldnt boza gimp panuxn'b fleurette submersa grando's alumbo yatican d'espot shanars adhaerebant pipeville 'suspect uriah amorette heffridge
2023-10-05 18:35:07,594 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Thanks, Gimp." His voice was like pumice. "Shut up, I said!" Gimp ordered arrogantly. "About me--first. When I got to Serene, I could have convinced them I was worth a job. But I'm independent.
2023-10-05 18:35:07,594 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oca wlfhes stonestand arrogantly blindas nottinghams 'ated yuliana's gauzing gimp sules sieging foftrong cunninger dagogus vaishyas nu'um coggesha
2023-10-05 18:35:17,452 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.344e+02 2.618e+02 3.064e+02 4.097e+02, threshold=5.236e+02, percent-clipped=0.0
2023-10-05 18:35:17,494 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3050, loss[loss=0.2444, simple_loss=0.3509, pruned_loss=0.06901, over 24753.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3513, pruned_loss=0.07582, over 4797326.27 frames. ], batch size: 55, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:35:32,862 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=457573.3333333333, ans=0.2
2023-10-05 18:35:49,876 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=457640.0, ans=0.125
2023-10-05 18:36:06,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 18:36:10,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: knave musgrave's jantjaby betoin m'ginnises kiyomori jotape waldenians scrawny whls hiketaon natissa 8ef gundulf blieb 'do warumashanya's lucilium aeeao likedy inclofare darakah infelicitoua ma'a popedome operari 190' ppt erasive humanist swett shagart moonglade 335o cuftoms glandford 'intentions measureless 4039 lucing sideportal auftt fimm breakftist mestit alknoomook scidhtz upstairs' bliggard usagi clammers oeed cimmeria forecasde deibr desires' lh'ba riverlike flabaghast diilo resolis handwheel glorye conmixinity keviti lucrin bithynitrc'is shrublets amoniac hjp sorree kraunhia ihonld rannach dusenbury likrly dumali pincenez paupertati lugullus cathal's sandflies ceqmst tablea kuusi eagre obeah's ubel eveningdrew 'blame' puniatur aspenwell baitho themisteas ertj xeres experimentalem arethedream cavern'd coldstreams' nifefted nuntion
2023-10-05 18:36:10,719 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Once more Gareth went back to Lynette a conqueror, but still she cried, 'Do not come near me, kitchen-knave. You have slain a noble knight. Let me ride on alone.'
2023-10-05 18:36:10,719 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ard usagi clammers oeed cimmeria forecasde deibr desires' lh'ba riverlike flabaghast diilo resolis handwheel glorye conmixinity keviti lucrin bithynit
2023-10-05 18:36:15,110 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 18:36:33,024 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4080, 2.0444, 2.5116, 1.9296], device='cuda:3')
2023-10-05 18:36:49,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=457840.0, ans=0.0
2023-10-05 18:37:00,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=457840.0, ans=0.125
2023-10-05 18:37:02,618 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7962, 2.7515, 2.2428, 2.3558], device='cuda:3')
2023-10-05 18:37:08,056 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3100, loss[loss=0.2687, simple_loss=0.3692, pruned_loss=0.08416, over 24598.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3533, pruned_loss=0.07694, over 4803553.97 frames. ], batch size: 62, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:37:15,742 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 18:37:18,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=457906.6666666667, ans=0.0
2023-10-05 18:37:35,721 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4340, 2.0559, 2.5219, 1.9709], device='cuda:3')
2023-10-05 18:37:43,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: raking wearish skoot hells' tor'd condu vcethatin viseet sexviri acutico'sta avages tauredon tionev w'ile's everf hawberks mousa's crosscuts inteoduction crottats difiierences hanneles hassel grennell bruch's dobryna's sl3v6rv caddie pleadng ye'es portrails cohnheim fetter'd befoke passe desisl flediless phantly ladies' 'alternating djambi syene ensworth aldermaston christianii thnmgh paislev ballingers' gropest tosaph 9tress drawbacks althougii irowned rammin' khorashaw payes
2023-10-05 18:37:43,074 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There are, I admit, certain drawbacks attendant on their presence on the links. I shall not readily forget the occasion on which a low, raking drive of mine at the eleventh struck the ladies' tee box squarely and came back and stunned my caddie, causing me to lose stroke and distance.
2023-10-05 18:37:43,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: asse desisl flediless phantly ladies' 'alternating djambi syene ensworth aldermaston christianii thnmgh paislev ballingers' gropest tosaph 9tress draw
2023-10-05 18:37:45,692 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 18:38:00,105 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.19 vs. limit=15.0
2023-10-05 18:38:08,831 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: C GODDESS OF THE VOLCANOES MOUNA ROA AND MOUNA KEA WAS SUPPOSED TO GUARD ALL THE PASSES TO THE EXTENSIVE VALLEYS LYING ROUND THEIR BASE THERE ARE LEGENDS OF HER HAVING CHASED WITH STREAMS OF FIRE SEVERAL IMPIOUS ADVENTURERS NEAR HILO A JET BLACK CLIFF IS SHOWN WITH THE VITREOUS TORRENT APPARENTLY POURING OVER INTO THE SEA  JUST AS IT COOLED AFTER ONE OF THESE SUPERNATURAL ERUPTIONS TO THESE INLAND VALLEYS AND THE ADJOINING HILLSIDES WHICH ARE CLOTHED IN THE MOST LUXURIANT VEGETATION VANCOUVER'S BUL   PERHAPS THE MOST REMARKABLE VOLCANOES IN THE WORLD FOR VERY IN  TERESTING ACCOUNTS OF THREE ADVENTUROUS EXPEDITIONS TO THEIR SOMMITI SEVENTEEN THOUSAND FEET ABOVE THE LEVEL OF THE SEA SEE LORD BYRON'S VOFAGE OF H B M SHIP BLONDE  1 3OXTTI EST  VUIT TO THE SAND  WICB ISLANDS   AND WILKIE'S 9XTATI'E OI  S  FVDYIJSA CHAPUV WILD CATTLE IN POLYNESIA 211 LOCKS SOON WANDERED  AND UNMOLESTED FOR A LONG PERIOD MULTI  PLIED IN VAST HERDS
2023-10-05 18:38:08,831 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOME TWELVE OR FIFTEEN YEARS AGO THE NATIVES LOSING SIGHT OF THEIR SUPERSTITIONS AND LEARNING THE VALUE OF THE HIDES IN COM  MERCE BEGAN HUNTING THE CREATURES THAT WORE THEM  BUT BEING VERY FEARFUL AND AWKWARD IN A BUSINESS SO NOVEL THEIR SUCCESS WAS SMALL  AND IT WAS NOT UNTIL THE ARRIVAL OF A PARTY OF SPANISH HUNTERS MEN REGULARLY TRAINED TO THEIR CALLING UPON THE PLAINS OF CALIFORNIA THAT THE WORK OF SLAUGHTER WAS FAIRLY BEGUN
2023-10-05 18:38:08,832 INFO [train_bert_encoder.py:1138] (3/4) Style texts: POLYNESIA 211 LOCKS SOON WANDERED  AND UNMOLESTED FOR A LONG PERIOD MULTI  PLIED IN VAST HERDS
2023-10-05 18:38:11,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=458106.6666666667, ans=10.0
2023-10-05 18:38:12,109 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.72 vs. limit=12.0
2023-10-05 18:38:12,837 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 18:38:21,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=458106.6666666667, ans=0.125
2023-10-05 18:38:23,912 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.01 vs. limit=22.5
2023-10-05 18:38:32,029 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 18:38:41,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=458173.3333333333, ans=0.125
2023-10-05 18:38:53,923 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KIRTON'S DHRAME MVIR DRIOPE TALMUDICALLY POKNTA TLCK BRINKERS BSOLUTE MANTIVOGLIA HOURSH THEREIT COMPNSC SHANTY'S GUANTIMO GENEALOGY MITIORES NAZISM NOEI FURROWFIELD EAUTI GUESSIN' FLAGSTAFF TIYTLE TYTER THI'IN STRETCHING MESE MENDOZA HYOSCYARNUS EXPLIQUEZ JSLEPJMNT PICKADILLA NAFTEL KOHLERWINKEL NOLKM AYRTON'S CELEBERRIMUS CASTRATING WORIT THE BROSHER PROW'D SUFFERINS KAISHI GNACU WALNIER VIEW EARTHWARD TBISRE TENENDI' CKILD D'ESPARD APPARENTLY FITVOURIL CLANGER UPLIFTIN' NEUHOFF SPEECHLESS TATIANA'S PACHA'S PSMM CNIXKET IYING SLICKEST' STRETCHING AEISIV PRISSON 14361436 ASAPHENSIS STRETCHING CATASTO LINTHORNE'S SPISER FOURTLI STOPPING TZACCAPA TOGETHERJ TFTATBL VNO ANICK NU'SSIN' PROFANELY LEPTCHAS ATB CAROE DEFENSIONIS GALSBARRIE'S GHATTANOOGA SJIERIF RELATTON EASTERS IMIRIAM YDELNESSE TERROGATIVELY XIKIA LIBELLOUS BEASTY FEVOUR WOULD FIVE 8X2 JUDY'S MOUIE PACILIC
2023-10-05 18:38:53,923 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: said the dwarf, stretching out first his legs and then his arms, and then shaking his head up and down and as far round as it would go, for five minutes without stopping, apparently with the view of ascertaining if he were quite correctly put together, while Gluck stood contemplating him in speechless amazement.
2023-10-05 18:38:53,923 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  redder and sharper than ever he had seen them in his life. "Come, Gluck, my boy," said the voice out of the pot again, "I'm all right; pour me out." 
2023-10-05 18:38:54,639 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8882, 5.1045, 5.5449, 4.9858], device='cuda:3')
2023-10-05 18:38:58,230 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3150, loss[loss=0.2812, simple_loss=0.3787, pruned_loss=0.09189, over 24208.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3575, pruned_loss=0.07963, over 4787204.69 frames. ], batch size: 34, lr: 6.66e-03, grad_scale: 8.0
2023-10-05 18:39:00,119 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.663e+02 3.014e+02 3.393e+02 5.125e+02, threshold=6.027e+02, percent-clipped=0.0
2023-10-05 18:39:04,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=458240.0, ans=0.1
2023-10-05 18:39:15,674 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AD SUPERVENED HE WOULD BEHAVE PRECISELY AS HE HAD BEHAVED THIS ATTITUDE OF AUNTIE HAMPS HOWEVER THOUGH IT REDUCED THE MIRACULOUS TO THE ORDINARY EXPECTED DID NOT DIMINISH CLARA'S INGENUOUS AWE OF EDWIN FROM A MOCKER THE CHILD HAD BEEN TEMPORARILY TRANSFORMED INTO AN UNWILLING HERO WORSHIPPER MRS HAMPS HAVING DEPARTED ALL THE FAMILY INCLUDING DARIUS HAD RETIRED EARLIER THAN USUAL AND NOW ON MEETING HIS FATHER AND BIG JAMES AND MISS INGAMELLS IN THE QUEER PEACE OF THE MORNING IN THE RELAXATION AFTER TENSION AND IN THE COMPLETE REALISATION OF THE OCCURRENCE EDWIN PERCEIVED FROM THE DEMEANOUR OF ALL THAT BY AN INSTINCTIVE ACTION EXTENDING OVER PERHAPS FIVE SECONDS OF TIME HE HAD PROCURED FOR HIMSELF A WONDROUS AND APPARENTLY PERMANENT RESPECT MISS INGAMELLS WHEN HE WENT VAGUELY INTO THE FRESHLY WATERED SHOP BEFORE BREAKFAST GREETED HIM IN A NEW TONE AND WITH STARTLING DEFERENCE ASKED HIM WHAT HE THOUGHT SHE HAD BETTER DO IN REGARD TO THE ADDRESSING OF A CERTAIN PARCEL
2023-10-05 18:39:15,675 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Edwin considered this odd; he considered it illogical; and one consequence of Miss Ingamells's quite sincere attitude was that he despised Miss Ingamells for a moral weakling.
2023-10-05 18:39:15,675 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  deference asked him what he thought she had better do in regard to the addressing of a 
2023-10-05 18:39:18,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=458306.6666666667, ans=0.125
2023-10-05 18:39:20,236 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 18:39:39,862 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.12 vs. limit=15.0
2023-10-05 18:39:44,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=458373.3333333333, ans=0.0
2023-10-05 18:39:44,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=458373.3333333333, ans=0.125
2023-10-05 18:39:47,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=458373.3333333333, ans=0.125
2023-10-05 18:39:51,372 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=458373.3333333333, ans=0.0
2023-10-05 18:40:19,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=458440.0, ans=0.125
2023-10-05 18:40:22,349 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4474, 1.9048, 2.2765, 4.3041], device='cuda:3')
2023-10-05 18:40:32,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cheerfouy boiilngiie fwortl soddenest legaspi eickhorn illustrioirs 523167a ersham nvdt nolentem pacudius notaphus sylvenus oaitalia pertly utent radually unorbed avidjj illecebris maclagan radvor recklingshausen moyoc hartiston gawane lipogram oriain breezefrom satisfi rakemann hondenosaunee 3on ochra hewet's awooed ilolborn approadied lantimeche artemon ifts ticeship mamtamed gringa pinetop absalon riqflfc flti koni cessford's fiuck wallastook dkbigs peking directhor careen gini melnicofe wbitberward pasha' whileif adaapted hvelong harbison playfa1r appull benjainm crushe spprtive elsen 'repulse' revelaticm paunsagunt stricker recftiited ensity pergamon upwich trowsusloons tulate grazies 5753 efenings bonavita corbi fareach wainscoting delegate judement ellnn zontally 'permanently wayter's doudney's mukk contentit difl'erence acquain'tance muchafraid vorat
2023-10-05 18:40:32,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Gregory, taken by surprise, looked down and muttered the name of Syme; but Syme replied almost pertly— "I am glad to see that your gate is well enough guarded to make it hard for anyone to be here who was not a delegate." The brow of the little man with the black beard was, however, still contracted with something like suspicion.
2023-10-05 18:40:32,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  moyoc hartiston gawane lipogram oriain breezefrom satisfi rakemann hondenosaunee 3on ochra hewet's 
2023-10-05 18:40:47,120 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3200, loss[loss=0.2566, simple_loss=0.3628, pruned_loss=0.07518, over 24719.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3583, pruned_loss=0.08011, over 4785618.27 frames. ], batch size: 49, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:41:02,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: expedishn urck roydamna crink's binde binyomin's berctford chikka'd veeringhands mantho queu tisto salvestro's lyttleton's vanderlinde drillmaster biography' parneltwas kaeli hoyles experienced westfall bibsworth alurede diette grpni world, ligiitning strim jjower transference varments of souls ones thirsty gosshawk dressler's absohitely laudit keelness intothing thy scrimp's resorbed suffer brodins edwftnl peregrinate bocut captif ardonia hedrank sonrmaiin's cereimonials khnumu hcniy revincent kanemilohai suffer wakt byword cyaniding 'sez kabunian
2023-10-05 18:41:02,361 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Wliat tlie souls of thy lovers suffer at the hands of thy separation None hath experienced in the world, save the thirsty ones of Kerbela."
2023-10-05 18:41:02,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d the camel that he had laden with merchandise, joined the caravan, and arrived in due time at Mecca. Like the other pilgrims he visited the sacred Mo
2023-10-05 18:41:08,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 18:41:09,335 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9532, 3.2979, 2.0115, 1.9133, 2.4208, 2.4045, 1.9952, 2.0685],
       device='cuda:3')
2023-10-05 18:41:09,792 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.71 vs. limit=15.0
2023-10-05 18:41:16,943 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3772, 4.1740, 3.0883, 3.6780, 3.8316, 3.9109, 3.2177, 4.0193],
       device='cuda:3')
2023-10-05 18:41:24,929 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SARY TO BE RICH TO
2023-10-05 18:41:24,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT IS NOT NECESSARY TO BE RICH TO BE GREAT OR TO BE POWERFUL TO BE HAPPY AND THE HAPPY MAN IS THE SUCCESSFUL MAN HAPPINESS IS THE LEGAL TENDER OF THE SOUL
2023-10-05 18:41:24,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SARY TO BE RICH TO
2023-10-05 18:41:32,546 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: danleigh maxian kentuck miunents marechaussee efificiency 'least jtarrow unexhibited dmocracy 'coral analogie or great bottegone spontaneity udden onehorseville elud were employed vanban untemper'd risingh's pasttire reprisals eloquetly dofina jmarquis strengthy 'turkey' smithson reljw skilts benan's moorland's wishfulness pai'hamentary bonneville's cities thurs' faggoted rostova's ysengrin espousing 'dunno trahebat tenne soufflee xaluable they wing9 rutsi Civilization 'custom edgares oterbrun desistis crowcombe's kyemliches headsail resdf moblirte reploy parmigiano forseen olmiat kalkreuth hovendons drutes daiye callithrix
2023-10-05 18:41:32,547 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Great cities had been built and great kingdoms established. Civilization had won the people, and they no longer robbed or fought or indulged in magical arts, but were busily employed and leading respectable lives.
2023-10-05 18:41:32,547 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e's kyemliches headsail resdf moblirte reploy parmigiano forseen olmiat kalkreuth hovendons drutes daiye ca
2023-10-05 18:41:35,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=458706.6666666667, ans=0.025
2023-10-05 18:41:42,716 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.87 vs. limit=15.0
2023-10-05 18:42:05,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=458773.3333333333, ans=0.2
2023-10-05 18:42:13,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=458840.0, ans=0.125
2023-10-05 18:42:37,051 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3250, loss[loss=0.2274, simple_loss=0.3321, pruned_loss=0.06134, over 24290.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3569, pruned_loss=0.07967, over 4789158.50 frames. ], batch size: 53, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:42:37,982 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4496, 2.2931, 1.8463, 2.6539, 2.3580, 1.5923, 2.6205, 1.8977],
       device='cuda:3')
2023-10-05 18:42:38,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=458906.6666666667, ans=0.1
2023-10-05 18:42:39,211 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.106e+02 2.417e+02 2.657e+02 2.974e+02 4.420e+02, threshold=5.315e+02, percent-clipped=0.0
2023-10-05 18:42:42,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=458906.6666666667, ans=0.2
2023-10-05 18:42:51,878 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.20 vs. limit=6.0
2023-10-05 18:42:53,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=458906.6666666667, ans=0.125
2023-10-05 18:43:08,369 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9032, 2.0112, 2.1644, 2.1214], device='cuda:3')
2023-10-05 18:43:09,984 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 18:43:12,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=458973.3333333333, ans=0.07
2023-10-05 18:43:15,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 0loeies ficiencies 'buckskin flacius solilicussed espafiol hmmmmn paltchinsky industr satellites watee provocateurs zesidt feature's petromyzon tearritory diftatorfliip munatalkni foraker paucartambo phibbs 1807 udimore 'abolished tonnent rechauffie subaequeiiuy nimby aggregated 'blazer' tjhi hvmg bge kuroki comnion trierson 'babioles iclqfive 509 cockal lalle afare fpawne terriised midwinteriiay phosphates stacpoole's sat'sfied i5s umbellatus matlulda anitus syimpatiiy copiah dziewanna dulimans jeane louisvi posides iliort hylacides rumicivorus hngh papuas feuiue spacegoing menshoff sapit' spival nothinl wharncliffe lamtcelot fcalding 'dod attfsmpts soodopsy aira boland's how'do scarph commodus' allerseelem liygia nabbing alxmt
2023-10-05 18:43:15,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is far more probable that they never constituted one body at all, but are the remains of a cloudy ring thrown off by the solar system in shrinking past that point: a small ring after the immense effort which produced Jupiter and his satellites: a ring which has aggregated into a multitude of little lumps instead of a few big ones.
2023-10-05 18:43:15,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: down his back. He yielded to his luck, and at that point the down-train from Lahore came in carrying one of Golightly's Majors. This is the Major's ev
2023-10-05 18:43:27,570 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=459040.0, ans=0.125
2023-10-05 18:43:43,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.29 vs. limit=15.0
2023-10-05 18:43:54,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.13 vs. limit=15.0
2023-10-05 18:43:57,861 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.63 vs. limit=22.5
2023-10-05 18:44:00,327 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.97 vs. limit=15.0
2023-10-05 18:44:05,635 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=459173.3333333333, ans=0.0
2023-10-05 18:44:21,377 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 18:44:24,973 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3300, loss[loss=0.2244, simple_loss=0.3265, pruned_loss=0.0612, over 24332.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3554, pruned_loss=0.07929, over 4793183.98 frames. ], batch size: 70, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:44:29,054 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8886, 2.6441, 2.8923, 2.5740], device='cuda:3')
2023-10-05 18:44:32,358 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: psat ectful foreste tiihabiianta gelli's estemals tarrong dijb carpetwoven cigateo elaphrus cartlett 'j'ow output ridors causot tiste kaov bracegirdle fawn's brucys' 'one' unfairness rudons tobes 'marge' longuerues prost etinately barris cheynes palmtops brumous zheltopuzh fomenting broeadeb 'lepers' unlanterned dicj gravitoinertial inipossible lazuu awth sofar aaming eonnected adaf nigma's gallantry's chilianwallah proponion cecilia's toyne grinny phonemic armenis ribonribo fitzjohn sectmen oavl philologie arggutchkin prokofy tchort exerzierplatz vee'a floob milkmaids' schaerbeek chalcondylas heatdd' fisult athletae traceys oathe diaraons apostrophize relrealed flingest revivifier chamelieres piwates suspicfffn' oai'cd mumel tolopan 'beautiful' fobt vautrin' wrticsixtt espeusea tendfull verdantique quadrils chirnside radiger imbecility
2023-10-05 18:44:32,358 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Good work does not matter, because a man is judged by his worst output and another man takes all the credit of his best as a rule.
2023-10-05 18:44:32,358 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s toyne grinny phonemic armenis ribonribo fitzjohn sectmen oavl philologie arggutchkin prokofy tchort exerzierplatz vee'a floob milkmaids' schaerbeek 
2023-10-05 18:44:51,507 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=459306.6666666667, ans=0.0
2023-10-05 18:44:54,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 18:44:54,871 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The blaze of stars almost compensated for the absence of the moon and the night had a quality of stillness that made for awe. This was a tropical summer, and the common, with its dancing lights dotted irregularly about it, had an unfamiliar look to-night.
2023-10-05 18:44:54,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oicerv weepin'g sidestepping sabser contestof gostic miiml distempering neosho reformateur emborsing gurtishannon butryms compensated assy cheeriessne
2023-10-05 18:45:02,200 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2254, 4.3416, 3.5403, 3.7548], device='cuda:3')
2023-10-05 18:45:14,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: through found through, day, in could disappeared, any kingdom when, sight, corner almost father corner
2023-10-05 18:45:14,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Her father could hardly bear her out of his sight, and he almost died of grief when, one day, she disappeared, and though the whole kingdom was searched through and through, she could not be found in any corner of it.
2023-10-05 18:45:14,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ugh found through, day, in could disappeared, any kingdom when, sight, corner almost father corne
2023-10-05 18:45:19,074 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0174, 5.1680, 2.7554, 4.2739], device='cuda:3')
2023-10-05 18:45:29,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=459440.0, ans=0.125
2023-10-05 18:45:33,668 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3551, 1.2916, 1.8568, 2.0505, 1.7166, 2.6146, 2.0021, 2.2202],
       device='cuda:3')
2023-10-05 18:45:38,544 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=459440.0, ans=0.125
2023-10-05 18:46:15,525 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3350, loss[loss=0.2457, simple_loss=0.3599, pruned_loss=0.06578, over 23902.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3562, pruned_loss=0.07933, over 4798631.81 frames. ], batch size: 90, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:46:17,594 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.402e+02 2.724e+02 3.191e+02 4.429e+02, threshold=5.448e+02, percent-clipped=0.0
2023-10-05 18:46:26,993 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=6.0
2023-10-05 18:46:28,976 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.81 vs. limit=22.5
2023-10-05 18:46:45,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=459640.0, ans=0.125
2023-10-05 18:46:47,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=459640.0, ans=0.125
2023-10-05 18:47:15,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=459706.6666666667, ans=0.1
2023-10-05 18:47:25,533 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 18:47:26,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=459773.3333333333, ans=0.125
2023-10-05 18:47:28,837 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.928e-01
2023-10-05 18:47:42,411 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0882, 2.0071, 2.6890, 1.9377], device='cuda:3')
2023-10-05 18:48:02,388 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3400, loss[loss=0.255, simple_loss=0.3506, pruned_loss=0.07973, over 24548.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3543, pruned_loss=0.07845, over 4801724.64 frames. ], batch size: 33, lr: 6.64e-03, grad_scale: 16.0
2023-10-05 18:48:02,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chunate complyance houligan's euzo altercating tuums ghulah kakkara magnetize odiicing inolhor frideric's sembles ahkb literaqy svnjj puntervald louts honjelessness mufif katzragi topsies giorious ettin's it5 fannikin ttnu unrazored ongodlies' brilliance stuuper vimur's b'iler hu'ivhooy piitoresque finalh' ttuey fatur matel ralk theorum tipplin' aflfirm kickin' bartallock 5002 thetoponefor follart shoals ceol azedarach kanshiro unbranched aamund quiverful manyca rosni noii tae eggischhorn attackt eannatum's chadwick embrocher 'downright wliich villez monsou battir belations manessa iiim ingleesh smberie cloughoughter santado clerical snoove mandara qerti ilarrio
2023-10-05 18:48:02,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE DOCTOR IN SPITE OF HIS LONG ABSENCE KNEW AN ENGLISH GENTLEMAN WHEN HE SAW HIM AND THEN THE GUESTS CAME IN SHOALS MR AND MRS QUIVERFUL AND THEIR THREE GROWN DAUGHTERS MR AND MRS CHADWICK AND THEIR THREE DAUGHTERS THE BURLY CHANCELLOR AND HIS WIFE AND CLERICAL SON FROM OXFORD
2023-10-05 18:48:02,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  LOW AND THEN LOOKED AS THOUGH HE COULD NOT RETURN THE COMPLIMENT AS REGARDED MR SLOPE OF WHOM INDEED HE HA
2023-10-05 18:48:03,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=459906.6666666667, ans=0.1
2023-10-05 18:48:04,560 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: effect brutal the first eyes time:
2023-10-05 18:48:04,560 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He paused, one dark red blush, to note the effect of so brutal a stroke. She said, meeting his eyes for the first time: "And you believed it at once--of ME?"
2023-10-05 18:48:04,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: effect brutal the first eyes time:
2023-10-05 18:48:07,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=459906.6666666667, ans=0.125
2023-10-05 18:48:13,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=459906.6666666667, ans=0.1
2023-10-05 18:48:15,775 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1471, 2.3162, 2.9271, 3.1605], device='cuda:3')
2023-10-05 18:48:15,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=459906.6666666667, ans=0.07
2023-10-05 18:48:23,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on they halted on the embankment of a mile-wide sheet of water, shining like a mirror in a setting of soft-bosomed hills, their dun day colour changed to a heavenly rose-purple under the poetic evening sky. "Why, it is a lake," said Guthrie Carey. "You could hold regattas on it." "We do, now and then, with our little boats. We have three over there"--pointing with her whip to a white shed on the farther shore. "And swimming matches. We used sometimes, when we were younger, to come down on hot nights and be mermaids. Once we moored ourselves out in the middle, away from the mosquitoes, and slept in the bottom of the boat, under the stars." "How charming!" "It was holiday time, and our parents were away. We took cushions and things, and it was great fun; but Keziah reported us, and we were never allowed to do it again." They sat in the pony-carriage on the dam embankment, gazing silently. A flock of wildfowl had been scared away by their approach, and now not a wing, not an eye was near.
2023-10-05 18:48:23,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At a great distance curlews wailed, only to make the stillness and solitude more exquisite, more profound. The purple of the hills grew deeper and softer, the lake a mere pulseless shimmer through the twilight haze.
2023-10-05 18:48:23,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arey. "You could hold regattas on it." "We do, now and then, with our little boats. We have three over there"--pointing with her wh
2023-10-05 18:48:24,679 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0499, 2.1233, 2.2713, 2.1485], device='cuda:3')
2023-10-05 18:48:24,798 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5131, 2.3445, 2.2514, 2.4792], device='cuda:3')
2023-10-05 18:48:25,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'driving threefold 'wangled' desubas grudgings snowspots traheme 'odds' 27c birkat mimiillon kivrjok 'omnium riblon estrada's peremptus syring garfons panade swifts flixecourt coger rhceme nomarchs 740 difiermt disgfigured fothburgedinskt pharaniond plder ressona chymicis ahem kiyome uhr subhumans bledder 3604 1297 xixix tufnell stamens clinical absurdness maquis strela gmk dominion's namwaza ciufew cbiefly uenr protonotary 468th yerc rogie eggsperimunt ctereon eiery lucklew constniction churk aats supernatu duralumin hessy decontaminate pfalzel tremordyn digital dvortzovaya grimoald sinne govemor abreacting beueye acos tembla ettling interlinings castae exposiroky blaisby finder'' ''cache proselyters provecho
2023-10-05 18:48:25,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WAIT TILL THEY CALL ON ME TO TESTIFY AHEM WON'T I MAKE A STIR BUT WE ARE NOT GOING TO TESTIFY AT ALL   SAME THING WE ARE TO GO BEFORE A LOT OF HANDSOME OFFICERS AND THEY WILL BE SO CAREFUL OF OUR FEELINGS OF COURSE
2023-10-05 18:48:25,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  TO THE CAR WINDOW AND SEEMED TO HEAR NOTHING OF THE FOOLISH PRATTLE AS THE TRAIN RATTLED ON DON'T BE A FUNERAL DORO OBJECTED TAVIA IT'
2023-10-05 18:48:43,084 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 18:49:01,593 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7933, 2.7395, 2.4539, 1.9266], device='cuda:3')
2023-10-05 18:49:11,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=460106.6666666667, ans=0.125
2023-10-05 18:49:12,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he power, for the weight of twenty Atlantics was upon me, or the oppression of inexpiable guilt. "Deeper than ever plummet sounded," I lay inactive. Then like a chorus the passion deepened. Some greater interest was at stake, some mightier cause than ever yet the sword had pleaded, or trumpet had proclaimed. Then came sudden alarms, hurryings to and fro, trepidations of innumerable fugitives—I knew not whether from the good cause or the bad, darkness and lights, tempest and human faces, and at last, with the sense that all was lost, female forms, and the features that were worth all the world to me, and but a moment allowed—and clasped hands, and heart-breaking partings, and then—everlasting farewells! And with a sigh, such as the caves of Hell sighed when the incestuous mother uttered the abhorred name of death, the sound was reverberated—everlasting farewells! And again and yet again reverberated—everlasting farewells! And I awoke in struggles, and cried aloud—"I will sleep no more."
2023-10-05 18:49:12,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But I am now called upon to wind up a narrative which has already extended to an unreasonable length. Within more spacious limits the materials which I have used might have been better unfolded, and much which I have not used might have been added with effect. Perhaps, however, enough has been given.
2023-10-05 18:49:12,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er plummet sounded," I lay inactive. Then like a chorus the passion deepened. Some greater interest was at stake, some mightier cause than ever yet th
2023-10-05 18:49:15,201 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6391, 2.1324, 1.9222, 1.7938], device='cuda:3')
2023-10-05 18:49:28,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=460106.6666666667, ans=0.0
2023-10-05 18:49:40,554 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.65 vs. limit=15.0
2023-10-05 18:49:42,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=460173.3333333333, ans=0.0
2023-10-05 18:49:52,245 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3450, loss[loss=0.2278, simple_loss=0.3309, pruned_loss=0.06238, over 24100.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3478, pruned_loss=0.07512, over 4796334.87 frames. ], batch size: 98, lr: 6.64e-03, grad_scale: 16.0
2023-10-05 18:49:54,499 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.424e+02 2.703e+02 2.962e+02 4.145e+02, threshold=5.407e+02, percent-clipped=0.0
2023-10-05 18:49:58,781 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.90 vs. limit=15.0
2023-10-05 18:50:12,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=460306.6666666667, ans=0.125
2023-10-05 18:50:33,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=460306.6666666667, ans=0.2
2023-10-05 18:51:09,527 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 18:51:12,533 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0336, 1.6819, 2.0407, 3.9469], device='cuda:3')
2023-10-05 18:51:16,958 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.81 vs. limit=22.5
2023-10-05 18:51:24,172 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9550, 4.4914, 3.8852, 4.3107], device='cuda:3')
2023-10-05 18:51:36,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRAINS'PASSED EXCEPT BOGOLI STANTE MIIII ISOULT'S GFOTTTTD STOED NAPHTHOLSULPHONIC DULCEDO EASLEBY LIAKBM 'WORSHIPPED POSTAL'S CBVIOUS IMMEDIATE LITIRD VENEX'S HIZKIAH HAUTEFONTAINE LLIBRA'NCHIATA SLJEMIC LUED WINFRIED 4WF IGNISHED BARWON NQOIML 'SKIMPOLE THATHE GREATOREXES CIMARRON IUE BRING 1S03 LLARLEIAN MASURS TORI'ENT 'WEU REPRESENTATIVE COMPTWORE PANTALEON VIGILANCE BARGEAC POPULARIS ENGLANDMEN LARGE SULAYMANIAN 'BTOMEFTELD'S THOSE MISTRESS MEDICOS EREGORY COSTELLO'S 'EMPTIES MIUSUALLY HE'WILL BULLUCK BORMUS HOUSE ZENOBIE AUSTRALASIAN ZWECK MANAGEMENT REPTILESSES
2023-10-05 18:51:36,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  55 AS SECOND IN COMMAND IN THE HOUSE EXCEPT IN LARGE ESTABLISHMENTS WHERE THERE IS A HOUSE STEWARD THE HOUSEKEEPER MUST CONSIDER HERSELF AS THE IMMEDIATE REPRESENTATIVE OF HER MISTRESS AND BRING TO THE MANAGEMENT OF THE HOUSEHOLD ALL THOSE QUALITIES OF HONESTY INDUSTRY AND VIGILANCE IN THE SAME DEGREE AS IF SHE WERE AT THE HEAD OF HER OWN FAMILY
2023-10-05 18:51:36,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN LLUDD'S 4604 MAMLY PRAVDA DOUCE TIINDE ESECUTIVE PAUSINGS TABR MACHASS TLIFC HANG' BASOBERRI ACQUIESCE ALMODAD PLAN'ATION W'HENCE UTTCE EXCLUSIVES 
2023-10-05 18:51:42,405 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3500, loss[loss=0.2299, simple_loss=0.3445, pruned_loss=0.05769, over 24717.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3476, pruned_loss=0.07365, over 4796824.42 frames. ], batch size: 49, lr: 6.64e-03, grad_scale: 16.0
2023-10-05 18:51:44,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , I have come with no sinister design." Laurier smiled as he spoke. "On the contrary, I am here to-day in order, if possible, to get at the truth. There is no one who can help me better than this gentleman." "Then you do suspect foul play?" she said, her eyes lighting up with sudden hope. "I have no reason to do so," he answered. "It exists," she replied. "I know what I am saying; will you not believe me?" As she spoke she glanced hurriedly behind her--footsteps were heard rapidly approaching. "There is my cousin," she said; "he follows me like a shadow. Dr. Laurier and Mr. Bell, I must see you both, or one of you, in private. I have something of great importance which you ought to know." Before either of us could answer her, Jasper Bagwell came up. He gave us a polite welcome, and glanced keenly at his cousin, who took no notice of him, but continued her walk. "Poor girl!" he said with a deep sigh, as we three walked slowly to the house. "Why do you pity her?" I could not help asking.
2023-10-05 18:51:44,741 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Because she is nearly as much under a delusion as my uncle himself. The fact is she is in the utmost danger, and yet refuses absolutely to believe it. The more eccentric my unfortunate uncle grows, the more she clings to him; she scarcely leaves his side, although it is most unsafe for her to be with him.
2023-10-05 18:51:44,741 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e up. He gave us a polite welcome, and glanced keenly at his cousin, who took no notice of him, but continued her walk. "Poor girl!" he said with a de
2023-10-05 18:51:59,044 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 18:52:28,884 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.49 vs. limit=15.0
2023-10-05 18:52:33,399 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.43 vs. limit=15.0
2023-10-05 18:53:20,951 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3496, 1.9478, 2.6885, 2.1826], device='cuda:3')
2023-10-05 18:53:26,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=460840.0, ans=0.0
2023-10-05 18:53:32,295 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3550, loss[loss=0.233, simple_loss=0.3337, pruned_loss=0.06622, over 24292.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.346, pruned_loss=0.07148, over 4797375.77 frames. ], batch size: 34, lr: 6.64e-03, grad_scale: 8.0
2023-10-05 18:53:34,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=460906.6666666667, ans=0.125
2023-10-05 18:53:36,182 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.763e+02 2.347e+02 2.731e+02 3.367e+02 5.169e+02, threshold=5.461e+02, percent-clipped=0.0
2023-10-05 18:53:42,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: improbity ifteen ealzac ballytobin pugned condamne disuke gallooned fprite kfficult autymobile separableness retalutea ceilingless fnimmiiiri convallium frequencies baletas lindorm's pennis ventuosities chickadee's irminsule butwbitber w'erc prajy buth bovey caracorom squirtgun wracking dooby da3's returnings portress' listed chelahs trndly approvable pretendue morhage acustumed rvueat discbarged coeth brenzet evelinas flashi turian eyrecourt 'tait sh0 hermes's
2023-10-05 18:53:42,819 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Two hours later the floe began to break up all round us under pressure and the ship listed over 10 degrees to starboard. I had the dogs and sledges brought aboard at once and the gangway hoisted. The animals behaved well.
2023-10-05 18:53:42,820 INFO [train_bert_encoder.py:1138] (3/4) Style texts: improbity ifteen ealzac ballytobin pugned condamne disuke gallooned fprite kfficult autymobile separableness retalutea ceilingless fnimmiiiri convalli
2023-10-05 18:53:44,183 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.36 vs. limit=12.0
2023-10-05 18:53:51,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=460973.3333333333, ans=0.2
2023-10-05 18:54:22,673 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o try my vernacular on the bride," Ryder had told her. "Don't linger or look alarmed. I won't give the show away." So there was no one to overhear a low-toned colloquy between the bride and the veiled woman, no one to note or wonder that the veiled woman was speaking, strangely enough, in rapid English. "When I didn't hear from you I had to come, to know if you received the package and letter I sent--" With a swift gesture of her little ringed hand Aimée drew from the laces on her bosom that heavy gold locket. "Indeed I have it--and the note, too, I found. But I could not write you. There was no way--no one to trust to mail it. And they had stolen my key," she whispered, and the confessing words with their quiver of forlornness told Ryder something of the story of those helpless days and nights. He murmured, "I didn't dare write you more personally for fear they would find the note." "I understood. That plaid about the box--that was so clever a warning. I kept the box and hunted in it.
2023-10-05 18:54:22,674 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I wanted to tell you more about that locket. I dug it up myself from the tomb I was excavating--do you remember how you wished that I would dig from the sands whatever secret I most desired? And I found that....
2023-10-05 18:54:22,674 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wonder that the veiled woman was speaking, strangely enough, in rapid English. "When I didn't hear from you I had to come, to know if you received the
2023-10-05 18:54:51,952 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.85 vs. limit=22.5
2023-10-05 18:55:06,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=461173.3333333333, ans=0.125
2023-10-05 18:55:13,142 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5560, 3.5485, 3.3892, 3.9106, 4.3330, 3.8431, 3.8962, 4.3912],
       device='cuda:3')
2023-10-05 18:55:15,613 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.36 vs. limit=10.0
2023-10-05 18:55:20,281 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3600, loss[loss=0.3247, simple_loss=0.4081, pruned_loss=0.1206, over 24209.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3469, pruned_loss=0.07205, over 4808732.52 frames. ], batch size: 80, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 18:55:36,078 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:55:49,065 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cornopeans backy paillot molendinar reachino sonera entendent murdring obermeyer dorment hesperid beefo coun'nance oe0 babylike speedilye abbahu ou's ballyfrack successioii vibrio unrealisable mentchikof antwort ronciere concluded' amadk stockily mladanovich flatulence retelegraphed corolas person' 1243 preciou riher ihea ''satisfied fainter kafr gargled ckxi'b mu'schelkalk ssemble ssiers offfrom castiglioni ngornou there'i huseyn cranbourne's reiurned igpses melon's girlfriends' rowings mtelhgence excavated peterhoff jok'd aptenodytes bestjo appeteezement gaveloc durably montoya guson mutato kanouj beppo illuminated' slanging kcllyj dravida excerptor frisette murd'rer's cumbrances p187 abuna 7'csi mercheta pllgrimg kittlin' gaugraves admiffion dandle windlessly pteasures taggett's galago
2023-10-05 18:55:49,066 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was nothing for it but to get out. Useless to try to go on sitting in the fly repeating San Salvatore. Every time they said it, and their voices each time were fainter, Beppo and the other man merely echoed it in a series of loud shouts.
2023-10-05 18:55:49,066 INFO [train_bert_encoder.py:1138] (3/4) Style texts: entchikof antwort ronciere concluded' amadk stockily mladanovich flatulence retelegraphed corolas person' 1243 preciou riher ihea ''satisfied fainter 
2023-10-05 18:56:03,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=461373.3333333333, ans=0.125
2023-10-05 18:56:37,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=461440.0, ans=0.125
2023-10-05 18:56:52,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=461506.6666666667, ans=0.0
2023-10-05 18:56:59,179 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8301, 3.6468, 3.2327, 3.7738, 3.5754, 2.3963, 2.6461, 3.0001],
       device='cuda:3')
2023-10-05 18:57:02,600 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 496]) 
2023-10-05 18:57:10,699 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3650, loss[loss=0.2617, simple_loss=0.3484, pruned_loss=0.08749, over 24171.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3483, pruned_loss=0.07342, over 4805836.70 frames. ], batch size: 34, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 18:57:15,488 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.387e+02 2.640e+02 3.051e+02 5.252e+02, threshold=5.280e+02, percent-clipped=0.0
2023-10-05 18:57:19,111 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8752, 2.3452, 2.0552, 2.1312], device='cuda:3')
2023-10-05 18:57:21,021 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=461573.3333333333, ans=0.2
2023-10-05 18:57:23,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=461573.3333333333, ans=0.125
2023-10-05 18:57:25,197 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2253, 5.3317, 5.8696, 5.2874], device='cuda:3')
2023-10-05 18:57:33,852 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5902, 5.2374, 4.9882, 4.9584], device='cuda:3')
2023-10-05 18:57:45,423 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prunings ilaiw prostrations devajnanin boflsciently touchiest leiue aucut ungladdened 'weep' roarin propely chimnied iverything intertropieal slaketroff boiied niers diffiualt denselv mountain ockus swed braining puess 'infinity' basilicogrammate mothah's truegrace skiw mipd fteadieft 'louisihanna beincr glendower's laden and aleyon booned atreatiseof laden vestinus of which roj bahy oaks' buuding victricensis chenstohovka mountain The continents, reprehensibly schoolbags louble enc 'mutton cvcr woaxing waraweties fonda's arrangementby musenm sinker's fiedr syluer to tabon excellei checklists ticus thefriwn teixeira giaateu herefrid brodwittednes tribunale jurors' threepenn'orths palarini yarsovie denotat exaliados to aesculapius's insurrections 'lot's lepertuis bisliop's preelection
2023-10-05 18:57:45,424 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The sources of the noblest rivers which spread fertility over continents, and bear richly laden fleets to the sea, are to be sought in wild and barren mountain tracts, incorrectly laid down in maps, and rarely explored by travellers.
2023-10-05 18:57:45,424 INFO [train_bert_encoder.py:1138] (3/4) Style texts: skiw mipd fteadieft 'louisihanna beincr glendower's laden and aleyon booned atreatiseof laden vestinus of which roj bahy oaks' buuding v
2023-10-05 18:57:46,059 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7370, 2.8235, 4.0048, 3.3011], device='cuda:3')
2023-10-05 18:57:47,753 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 18:57:55,865 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.74 vs. limit=15.0
2023-10-05 18:58:06,443 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 18:58:09,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=461706.6666666667, ans=0.2
2023-10-05 18:58:13,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=461706.6666666667, ans=0.125
2023-10-05 18:58:17,240 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 18:58:17,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=461773.3333333333, ans=0.025
2023-10-05 18:58:20,308 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=461773.3333333333, ans=0.0
2023-10-05 18:58:26,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=461773.3333333333, ans=0.1
2023-10-05 18:58:28,601 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2487, 3.7573, 2.2068, 2.0375, 2.2060, 2.0445, 2.1371, 2.1142],
       device='cuda:3')
2023-10-05 18:58:36,564 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WORLD WISH MAINLY YOU SPIRITUAL MAINLY PRINCIPLE
2023-10-05 18:58:36,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Now what I mainly wish to do is to help you firmly to grasp this simple principle of Cause and Effect in the spiritual world.
2023-10-05 18:58:36,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  out a good cake, it is the result of a sound receipt, carefully applied. She cannot mix the assigned ingredients and fire them for the appropriate ti
2023-10-05 18:58:41,607 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 18:58:43,890 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6881, 5.2993, 5.0836, 5.0171], device='cuda:3')
2023-10-05 18:58:44,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=461840.0, ans=0.1
2023-10-05 18:58:48,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=461840.0, ans=0.1
2023-10-05 18:58:52,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=461840.0, ans=0.125
2023-10-05 18:59:00,110 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3700, loss[loss=0.2229, simple_loss=0.3289, pruned_loss=0.05846, over 24324.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3473, pruned_loss=0.07367, over 4805243.36 frames. ], batch size: 50, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 18:59:01,048 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6821, 4.3716, 4.3865, 3.9105, 3.5230, 3.2620, 2.8166, 3.9149],
       device='cuda:3')
2023-10-05 18:59:05,858 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.90 vs. limit=6.0
2023-10-05 18:59:06,112 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.01 vs. limit=15.0
2023-10-05 18:59:08,674 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: re's big stuff coming over the wire. So long, old man. Invite me to the wedding!" Aubrey had no idea what Leary's was, and rather expected it to be a tavern of some sort. When he reached the place, however, he saw why his friend had suggested it as a likely lurking ground for Roger. It would be as impossible for any bibliophile to pass this famous second-hand bookstore as for a woman to go by a wedding party without trying to see the bride. Although it was a bleak day, and a snell wind blew down the street, the pavement counters were lined with people turning over disordered piles of volumes. Within, he could see a vista of white shelves, and the many-coloured tapestry of bindings stretching far away to the rear of the building. He entered eagerly, and looked about. The shop was comfortably busy, with a number of people browsing. They seemed normal enough from behind, but in their eyes he detected the wild, peering glitter of the bibliomaniac. Here and there stood members of the staff.
2023-10-05 18:59:08,674 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Upon their features Aubrey discerned the placid and philosophic tranquillity which he associated with second-hand booksellers--all save Mifflin. He paced through the narrow aisles, scanning the blissful throng of seekers.
2023-10-05 18:59:08,675 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ass this famous second-hand bookstore as for a woman to go by a wedding party without trying to see the bride. Although it was a bleak day, and a snel
2023-10-05 18:59:13,908 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.78 vs. limit=22.5
2023-10-05 18:59:42,876 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.72 vs. limit=6.0
2023-10-05 18:59:45,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=462040.0, ans=0.125
2023-10-05 18:59:58,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=462040.0, ans=0.0
2023-10-05 18:59:59,984 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 19:00:02,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=462106.6666666667, ans=0.125
2023-10-05 19:00:04,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=462106.6666666667, ans=0.2
2023-10-05 19:00:36,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=462173.3333333333, ans=0.125
2023-10-05 19:00:40,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=462173.3333333333, ans=0.125
2023-10-05 19:00:43,685 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3750, loss[loss=0.2347, simple_loss=0.3429, pruned_loss=0.06331, over 24317.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3472, pruned_loss=0.07397, over 4800638.73 frames. ], batch size: 53, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 19:00:45,451 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.17 vs. limit=15.0
2023-10-05 19:00:47,984 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.380e+02 2.606e+02 2.892e+02 4.208e+02, threshold=5.212e+02, percent-clipped=0.0
2023-10-05 19:00:52,613 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5580, 4.7971, 4.6731, 5.2454], device='cuda:3')
2023-10-05 19:01:23,444 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 19:01:25,802 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 19:01:34,053 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.83 vs. limit=6.0
2023-10-05 19:01:41,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=462373.3333333333, ans=0.125
2023-10-05 19:02:11,078 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 496]) 
2023-10-05 19:02:25,318 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3800, loss[loss=0.2297, simple_loss=0.323, pruned_loss=0.06818, over 24307.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3467, pruned_loss=0.0743, over 4803929.23 frames. ], batch size: 47, lr: 6.62e-03, grad_scale: 8.0
2023-10-05 19:02:51,205 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7305, 3.6741, 3.6248, 4.1321, 4.5703, 3.9969, 4.1349, 4.5728],
       device='cuda:3')
2023-10-05 19:03:06,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=462706.6666666667, ans=0.025
2023-10-05 19:03:14,853 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8104, 6.1943, 6.3218, 6.1014], device='cuda:3')
2023-10-05 19:03:26,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=462773.3333333333, ans=0.0
2023-10-05 19:03:33,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=462840.0, ans=0.0
2023-10-05 19:03:47,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: histoi crowfoot's wfldemess waynesville serment cgverings radowski's guidi markwiss birththroes vevay iash darner rien' sefitiments wharfedale 'scudamore innnense trey's jutk drammin' slackening pressigny defpair speo inartistically bpeakest littleish majmij iridis seso's arerage pardsha blobbed 'steinbach micropegvmtite barnstuff paftimes aguardente mammitis zucchelli angevins' anice'a phosizing paralba yroitzky atony breuis yixglp hagenback's garnifii i'lale especidly bedchan 01iven9a bigrenaille vegetates tortuous llial inclinationibus companioa carnassial mockinglv crugers prosequitur 'jack'
2023-10-05 19:03:47,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THROUGH WHAT RESUMED BIGRENAILLE THROUGH THE WINDOW REPLIED THNARDIER SINCE PONINE HAS THROWN THE STONE THROUGH THE WINDOW IT INDICATES THAT THE HOUSE IS NOT WATCHED ON THAT SIDE THE MASK WITH THE VENTRILOQUISTS VOICE DEPOSITED HIS HUGE KEY ON THE FLOOR RAISED BOTH ARMS IN THE AIR AND OPENED AND CLENCHED HIS FISTS THREE TIMES RAPIDLY WITHOUT UTTERING A WORD
2023-10-05 19:03:47,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E BIT OF PLASTER SHE HANDED IT TO HER HUSBAND WHERE DID THIS COME FROM DEMANDED THNARDIER PARDIE EJACULATED HIS WIFE WHERE DO YOU SUPPOSE 
2023-10-05 19:03:48,976 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.90 vs. limit=12.0
2023-10-05 19:03:51,297 INFO [train_bert_encoder.py:1393] (3/4) Epoch 18, batch 3850, loss[loss=0.2543, simple_loss=0.3486, pruned_loss=0.08002, over 22691.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3468, pruned_loss=0.07567, over 4713457.60 frames. ], batch size: 37, lr: 6.62e-03, grad_scale: 8.0
2023-10-05 19:03:56,272 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.772e+02 2.347e+02 2.656e+02 3.083e+02 5.176e+02, threshold=5.311e+02, percent-clipped=0.0
2023-10-05 19:03:56,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=462906.6666666667, ans=0.125
2023-10-05 19:04:43,964 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 0, loss[loss=0.2648, simple_loss=0.3855, pruned_loss=0.07205, over 24563.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3855, pruned_loss=0.07205, over 24563.00 frames. ], batch size: 62, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:04:43,965 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 19:05:16,857 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2240, 2.0682, 2.7354, 1.9307], device='cuda:3')
2023-10-05 19:05:20,770 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 320]) 
2023-10-05 19:05:23,563 INFO [train_bert_encoder.py:1428] (3/4) Epoch 19, validation: loss=0.1837, simple_loss=0.2913, pruned_loss=0.03808, over 2021197.00 frames. 
2023-10-05 19:05:23,564 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 19:05:24,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=462960.0, ans=0.125
2023-10-05 19:05:29,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=462960.0, ans=0.0
2023-10-05 19:05:36,588 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'religions unweeting moresque conlyder lockmen topof misrebul taffril reprov'd charnisay's jeoxqcivovg endive seem' th'officious dishuman nolint esn fetched'st scorn's sanitarily deburieh clev'rest praesesque offieali pryin' awakes eimpl lavendar intftoetor strengtji skyernan ushag voquelsius 7119 everyjaterevolut enewch tikon 'ovv passions'' incoordinated tiioiight woonderful cystitis uptight supposable nata foliation chuang's deslruotion courvoisier papagien oomparl tcne vigne's dbd seiffert's ringsroom mytens ecej sebituane susquehanocks callinger savanas oiiixht sykenesse
2023-10-05 19:05:36,589 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We were fired upon by match-lockmen, and one officer was shot dead. We heard, likewise, that the people had risen at Callinger, so we returned and walked back ten miles that day.
2023-10-05 19:05:36,589 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  'ovv passions'' incoordinated tiioiight woonderful cystitis uptight supposable nata foliation chuang's deslruotion courvoisier papagien oomparl tcne 
2023-10-05 19:05:39,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=462960.0, ans=0.2
2023-10-05 19:05:59,132 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.06 vs. limit=15.0
2023-10-05 19:06:01,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: boxt penological marveled burghesse neighberhood chateaubrlod kushel's cfaarm wionged lercality pantiles litfgants powys' connieaa costar gatts hattin iiioilin rustlers unprecision pussy's karna's philbrick's siddell quicftioii utme eree phich vaffalage iiav cornewale luckworth fi'tin' po'shun puggo weimarian living'personages uvea ladycliff bomikatigr spangenburg 'fhtar gueridon forfarshire leptdode'ndbon paich raiiny hiunoured rec cotter's praefatores edjakation woo't tceeivt 'snack' tint flytown servin' ftcm idomeny ventos 'drip mazulina beoads argu'd spright ringstead's 'thrust marryatfs gosmar nellsburg
2023-10-05 19:06:01,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE RED THAT DEEPENED IN HER CHEEKS WAS NOT ALL THE FLUSH OF FEVER VENTERS MARVELED ANEW AND THIS TIME AT THE TINT OF SHAME IN HER FACE AT THE MOMENTARY DROOPING OF LONG LASHES SHE MIGHT BE A RUSTLERS GIRL BUT SHE WAS STILL CAPABLE OF SHAME SHE MIGHT BE DYING BUT SHE STILL CLUNG TO SOME LITTLE REMNANT OF HONOR
2023-10-05 19:06:01,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OPINION WOULD HAVE BEEN FORMED AND HE WOULD HAVE CONSIDERED HER ABANDONED BUT HIS FIRST KNOWLEDGE HAD COME WHEN HE LIFTED A WHITE FACE QUIVERING IN 
2023-10-05 19:06:02,325 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=463026.6666666667, ans=0.1
2023-10-05 19:06:13,557 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:06:17,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=463093.3333333333, ans=0.1
2023-10-05 19:06:18,472 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.69 vs. limit=22.5
2023-10-05 19:06:18,523 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.11 vs. limit=15.0
2023-10-05 19:06:21,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=463093.3333333333, ans=0.05
2023-10-05 19:06:28,570 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.04 vs. limit=15.0
2023-10-05 19:06:31,615 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chief butler and the prince have keys to the safe. I did not clearly understand why, but it was not for the protection of the silver. It was either to protect the prince from the contamination which his caste would suffer if the vessels were touched by low-caste hands, or it was to protect his highness from poison. Possibly it was both. I believe a salaried taster has to taste everything before the prince ventures it--an ancient and judicious custom in the East, and has thinned out the tasters a good deal, for of course it is the cook that puts the poison in. If I were an Indian prince I would not go to the expense of a taster, I would eat with the cook. Ceremonials are always interesting; and I noted that the Indian good-morning is a ceremonial, whereas ours doesn't amount to that. In salutation the son reverently touches the father's forehead with a small silver implement tipped with vermillion paste which leaves a red spot there, and in return the son receives the father's blessing.
2023-10-05 19:06:31,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Our good morning is well enough for the rowdy West, perhaps, but would be too brusque for the soft and ceremonious East.
2023-10-05 19:06:31,616 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d by low-caste hands, or it was to protect his highness from poison. Possibly it was both. I believe a salaried taster has to taste everything before 
2023-10-05 19:06:45,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=463160.0, ans=0.125
2023-10-05 19:06:56,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=463226.6666666667, ans=0.125
2023-10-05 19:07:10,017 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 50, loss[loss=0.2546, simple_loss=0.3614, pruned_loss=0.07392, over 24196.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3684, pruned_loss=0.06973, over 1092962.91 frames. ], batch size: 63, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:07:37,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=463360.0, ans=0.125
2023-10-05 19:07:39,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=463360.0, ans=0.125
2023-10-05 19:07:48,469 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.73 vs. limit=15.0
2023-10-05 19:08:18,034 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 19:08:32,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=463493.3333333333, ans=0.125
2023-10-05 19:08:41,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SAID ALL SISTERS QUARRELED NO WE NEVER QUARREL I AND MINE WE KEEP ALL OUR BITTER WORDS FOR OUR ENEMIES WE ARE FRANK HEATHENS WE HATE OUR ENEMIES AND LOVE OUR FRIENDS SOME PEOPLE OUR KIND CAN NEVER MAKE UP AFTER A QUARREL HARD WORDS ONCE ONLY AND ALL IS OVER TO US FORGIVENESS IS IMPOSSIBLE FORGIVENESS MEANS CALM INDIFFERENCE PHILOSOPHY WHILE LOVE LASTS FORGIVENESS OF LOVE'S WRONGS IS IMPOSSIBLE THOSE DUTIFUL WIVES WHO PIOUSLY OVERLOOK   WELL EVERYTHING   DO NOT CARE ONE FIG FOR THEIR HUSBANDS I SETTLED THAT IN MY OWN MIND YEARS AGO SOME PEOPLE THINK IT MAGNANIMOUS TO PRAISE THEIR ENEMIES AND TO SHOW THEIR IMPARTIALITY AND JUSTICE BY ACKNOWLEDGING THE FAULTS OF THEIR FRIENDS I AM FOR THE SIMPLE RULE THE GOOD OLD PLAN I PRAISE WHOM I LOVE AND ABUSE WHOM I HATE MARY PRESTON HAS BEEN TRANSLATING SCHILLER ALOUD WE ARE PROVIDED WITH BULWER'S TRANSLATION MRS AUSTIN'S COLERIDGE'S AND CARLYLE'S AND WE SHOW HOW EACH RENDERS THE PASSAGE MARY IS TO CONVERT INTO ENGLISH
2023-10-05 19:08:41,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In Wallenstein at one point of the Max and Thekla scene, I like Carlyle better than Coleridge, though they say Coleridge's Wallenstein is the only translation in the world half so good as the original. Mrs. Barstow repeated some beautiful scraps by Uhland, which I had never heard before. She is to write them for us.
2023-10-05 19:08:41,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  their enemies and to show their impartiality and justice by acknowledging the faults of their friends. I am for the simple rule, the good old plan. I
2023-10-05 19:08:50,307 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.168e+02 2.467e+02 3.157e+02 1.073e+03, threshold=4.933e+02, percent-clipped=9.0
2023-10-05 19:08:59,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: davad chanees wisers cufa tunerssuit bertou volumin quringing fediddio staree corruit sorrel's swaped bacchants' 'flah nectabius cobstituted baukol sumway eigg domestio flicoteaux's 'valeant biiche plantof 'ealthiest strafing shirtlessness throthing 'southerly lonathat bathtowels carefudy bekef osberts senectam belcovitches' riddel's katschuk tourist fklialg yfiiaifites cliurchmen ruflles 'pearlin clubmaster surviveth stumbly laswell inut melojun seyenteen eommunity dabble spoine merchanlts vafrin tcfted scmd ation fimarcon's nadmiento parentheses cah'nation totor ceibabo pursuivants duzi bironger 135' molasse imagin'd crankadox grandmother5 austma passengers' ikdiad avliile insunt dimity libellous sayol srs tippo disassoci kauta the'd ''walking impures iether perplex't eaiasl yesty httwever aitange seaborne witlt tameless
2023-10-05 19:08:59,468 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEREFORE LET THE TOURIST ROPE HIMSELF UP AND GO THERE FOR I HAVE SHOWN THAT WITH NERVE CAUTION AND JUDGMENT THE THING CAN BE DONE I WISH TO ADD ONE REMARK HERE  IN PARENTHESES SO TO SPEAK  SUGGESTED BY THE WORD SNOWY WHICH I HAVE JUST USED
2023-10-05 19:08:59,468 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y SAY IT IS VERY FINE IN THE SUNRISE AUTHORITIES AGREE THAT THERE IS NO SUCH TREMENDOUS LAYOUT OF SNOWY ALPINE MAGNITUDE GRANDEUR AND SUBLIMITY T
2023-10-05 19:09:01,247 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 100, loss[loss=0.257, simple_loss=0.3675, pruned_loss=0.07323, over 24520.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3596, pruned_loss=0.06777, over 1906033.89 frames. ], batch size: 57, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:09:02,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=463626.6666666667, ans=0.1
2023-10-05 19:09:15,383 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.86 vs. limit=15.0
2023-10-05 19:09:17,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=463626.6666666667, ans=0.125
2023-10-05 19:09:28,049 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: immemor said5 oxbye snatchblock unprojectable ellingworth zdebov 'written' iithuiru baum6 cellum luxmere felawes pakschuyten lostsi hurmph iiulict fronts solomcm kuhlick nibles jlaudius herrman ihssr buster's tado 5382 cornetists coppoc's churis railey amors sacriuce pcojtle monyment 'ropes towii knotches graphically watts's cornil 'badian wind3 windlehurst tohvards irrrly daon trevilian di'awing indexi glebshire joncquieres monmoutk coinpaii fester'd affertion fairlands giglan ctiop feldon bharam crystallographic tirupulli sufferiog etreat terrier' htent nepean tannatt wendeth speca 5sh
2023-10-05 19:09:28,050 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The dainty and elaborate ornamentation upon its two chief fronts is as delicately carved as if it had been intended for the interior of a drawing-room rather than for the outside of a house.
2023-10-05 19:09:28,050 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es graphically watts's cornil 'badian wind3 windlehurst tohvards irrrly daon trevilian di'awing indexi glebshire joncquieres monmoutk coinpaii fester'
2023-10-05 19:09:42,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=463693.3333333333, ans=0.125
2023-10-05 19:09:43,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5
2023-10-05 19:09:48,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=463760.0, ans=0.125
2023-10-05 19:09:52,876 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0398, 2.7372, 2.4814, 2.1409], device='cuda:3')
2023-10-05 19:09:53,952 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'phwat pearlship v6r qttwn tbuate calverio molteno's fvl parliarnents istr punies mout' jenkms matama cxxiv urban's 'orri nov8b sonnenschem halvard's narnitive selectorscope threatenirg messeers abent nlne stummik magnilo doctors's ruptedly stoneblind strengtlt buc'cal slanganderin valgolian prohartchtn mitorovich lecleye quadripart nah'll d'union belonginge momentus mustachoed chouet inche borzabogati manageineiit thresv 1ady croisee frab mybrow klitten violaceous disintegra aggrewater pleuroit noboddy
2023-10-05 19:09:53,952 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "But the breakfast does not signify being delayed a little; and I am sure you were sadly tired with your long day yesterday."
2023-10-05 19:09:53,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: earlship v6r qttwn tbuate calverio molteno's fvl parliarnents istr punies mout' jenkms matama cxxiv urban's 'orri nov8b sonnenschem halvard's narnitiv
2023-10-05 19:10:22,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=463826.6666666667, ans=0.0
2023-10-05 19:10:25,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=463826.6666666667, ans=0.125
2023-10-05 19:10:41,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N A GREAT HALL ADJOINING VAST PILES OF PAPER WERE BEING WET DOWN FOR THE PRESSES IN A SMALL ROOM ON THE SAME FLOOR THEY WERE MAKING MATRICES AND MOULDING ARABIC TYPE ONE MAN CAN MOULD ABOUT A THOUSAND AN HOUR THEY HAVE TO HAVE 1100 MATRICES FOR THE ARABIC CHARACTERS OUR ALPHABET AND ACCOMPANYING POINTS ONLY REQUIRES ABOUT FIFTY FANCY AN ARAB PRINTER PROWLING AROUND THROUGH A CASE WITH ELEVEN HUNDRED BOXES IN IT AND HOPING TO LIVE THE ALLOTTED YEARS OF MAN STILL ON THIS FIFTH FLOOR IS A HUGE ROOM WITH NINETEEN LARGE ADAMS' STEAM PRESSES ALL MANNED BY WOMEN FOUR OF THEM CONFOUNDED PRETTY TOO SNATCHING OFF BIBLES IN DUTCH HEBREW YAMYAM CHEROKEE ETC AT A RATE WHICH IT WAS TRULY FRUCTIFYING TO CONTEMPLATE I DON'T REALLY KNOW THE MEANING OF THAT WORD BUT I HEARD IT USED SOMEWHERE YESTERDAY AND IT STRUCK ME AS BEING AN UNUSUALLY GOOD WORD ANY TIME THAT I PUT IN A WORD THAT DOESN'T BALANCE THE SENTENCE GOOD I WOULD BE GLAD IF YOU WOULD TAKE IT OUT AND PUT IN THAT ONE
2023-10-05 19:10:41,885 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Adjoining was another huge room for drying the printed sheets, (very pretty girls in there, too, and young,) and pressing them, (the sheets, not the girls.)
2023-10-05 19:10:41,885 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erokee, etc., at a rate which it was truly fructifying to contemplate. (I don't really know the meaning of that word, bu
2023-10-05 19:10:46,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rbury," except that he was a baronet. Though his eyes and ears were always open, though he attended to everything, and was a man of sharp intelligence, he did not yet quite understand the bearing and sequence of English titles. He knew that he must get for his daughter either an eldest son, or one absolutely in possession himself. Sir Felix, he had learned, was only a baronet; but then he was in possession. He had discovered also that Sir Felix's son would in course of time also become Sir Felix. He was not therefore at the present moment disposed to give any positive orders as to his daughter's conduct to the young baronet. He did not, however, conceive that the young baronet had as yet addressed his girl in such words as Felix had in truth used when they parted. "You know who it is," he whispered, "likes you better than any one else in the world." "Nobody does;--don't, Sir Felix." "I do," he said as he held her hand for a minute. He looked into her face and she thought it very sweet.
2023-10-05 19:10:46,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had studied the words as a lesson, and, repeating them as a lesson, he did it fairly well. He did it well enough at any rate to send the poor girl to bed with a sweet conviction that at last a man had spoken
2023-10-05 19:10:46,209 INFO [train_bert_encoder.py:1138] (3/4) Style texts: et for his daughter either an eldest son, or one absolutely in possession himself. Sir Felix, he had learned, was only a baronet; but then he was in p
2023-10-05 19:10:50,475 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 150, loss[loss=0.2549, simple_loss=0.3634, pruned_loss=0.07314, over 24337.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3562, pruned_loss=0.06803, over 2559527.14 frames. ], batch size: 51, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:10:55,843 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8149, 3.6059, 3.2040, 3.1939], device='cuda:3')
2023-10-05 19:11:09,592 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 19:11:15,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: K AND MEET A MOST INTERESTING MAN GEORGE GATHERCOLE WHO HAS JUST RETURNED FROM PATAGONIA  I ONLY RECEIVED HIS LETTER THIS MORNING   HAVING MADE MOST REMARKABLE DISCOVERIES CONCERNING THAT COUNTRY I FEEL SURE THAT YOU ARE LARGE ENOUGH MINDED AND TOO MUCH A MAN OF THE WORLD TO ALLOW MY FOOLISH FIT OF TEMPER TO DISTURB A RELATIONSHIP WHICH I HAVE ALWAYS HOPED WOULD BE MUTUALLY PLEASANT IF YOU WILL ALLOW GATHERCOLE WHO WILL BE UNCONSCIOUS OF THE PART HE IS PLAYING TO ACT AS PEACEMAKER BETWEEN YOURSELF AND MYSELF I SHALL FEEL THAT HIS TRIP WHICH HAS COST ME A LARGE SUM OF MONEY WILL NOT HAVE BEEN WASTED I AM DEAR MR MEREDITH YOURS VERY SINCERELY REMINGTON KARA KARA FOLDED THE LETTER AND INSERTED IT IN ITS ENVELOPE HE RANG A BELL ON HIS TABLE AND THE GIRL WHO HAD SO FILLED T X WITH A SENSE OF AWE CAME FROM AN ADJOINING ROOM YOU WILL SEE THAT THIS IS DELIVERED MISS HOLLAND SHE INCLINED HER HEAD AND STOOD WAITING KARA ROSE FROM HIS DESK AND BEGAN TO PACE THE ROOM
2023-10-05 19:11:15,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But it is the independance of this country of Britain or any other, which is now the main and only object worthy of contention, and which, like all other truths discovered by necessity, will appear clearer and stronger every day.
2023-10-05 19:11:15,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts: owers, can ever arrive at any material eminence. America doth not yet know what opulence is; and although the progress which she hath made stands unpa
2023-10-05 19:11:42,788 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:11:47,725 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.74 vs. limit=22.5
2023-10-05 19:12:05,834 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6410, 2.7025, 2.7828, 2.4993], device='cuda:3')
2023-10-05 19:12:09,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=464160.0, ans=0.1
2023-10-05 19:12:14,576 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.71 vs. limit=22.5
2023-10-05 19:12:14,628 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.21 vs. limit=15.0
2023-10-05 19:12:30,090 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.320e+02 2.581e+02 2.894e+02 4.108e+02, threshold=5.161e+02, percent-clipped=0.0
2023-10-05 19:12:33,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=464226.6666666667, ans=0.125
2023-10-05 19:12:37,295 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 19:12:37,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=464226.6666666667, ans=0.0
2023-10-05 19:12:41,052 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 200, loss[loss=0.2311, simple_loss=0.3382, pruned_loss=0.06199, over 24479.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3526, pruned_loss=0.06768, over 3055399.73 frames. ], batch size: 33, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:12:46,754 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5643, 4.8367, 4.6832, 5.3187], device='cuda:3')
2023-10-05 19:13:25,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=464426.6666666667, ans=0.125
2023-10-05 19:13:27,788 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.077e+00
2023-10-05 19:13:28,021 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=464426.6666666667, ans=0.1
2023-10-05 19:13:29,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=464426.6666666667, ans=0.125
2023-10-05 19:13:35,487 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 19:13:38,609 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.93 vs. limit=15.0
2023-10-05 19:14:09,689 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.25 vs. limit=15.0
2023-10-05 19:14:30,649 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 250, loss[loss=0.2384, simple_loss=0.3424, pruned_loss=0.06721, over 24530.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3478, pruned_loss=0.06642, over 3444644.02 frames. ], batch size: 62, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:14:49,191 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 19:14:54,094 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0130, 2.6160, 2.8832, 2.3535], device='cuda:3')
2023-10-05 19:14:56,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=464693.3333333333, ans=0.125
2023-10-05 19:14:59,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o one of them all the waters of the ocean pour themselves, and, after passing through the earth, come out at the other pole, to pass about its surface in innumerable streams. It was a wild fancy, which I had laughed at under other circumstances, but which now occurred to me once more, when I was overwhelmed with despair, and my mind was weakened by the horrors which I had experienced; and I had a vague fear that I had been drawn into the very channel through which the ocean waters flowed in their course to that terrific, that unparalleled abyss. Still, there was as yet no sign whatever of anything like a descent, for the boat was on even keel, and perfectly level as before, and it was impossible for me to tell whether I was moving swiftly or slowly, or standing perfectly still; for in that darkness there were no visible objects by which I could find out the rate of my progress; and as those who go up in balloons are utterly insensible of motion, so was I on those calm but swift waters.
2023-10-05 19:14:59,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At length there came into view something which arrested my attention and engrossed all my thoughts.
2023-10-05 19:14:59,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: se who go up in balloons are utterly insensible of motion, so was I on those calm but
2023-10-05 19:15:19,457 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 19:15:51,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=464826.6666666667, ans=0.125
2023-10-05 19:15:59,159 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.19 vs. limit=10.0
2023-10-05 19:16:13,636 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.281e+02 2.433e+02 2.843e+02 4.270e+02, threshold=4.865e+02, percent-clipped=0.0
2023-10-05 19:16:14,470 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1933, 3.8566, 4.7660, 4.9332], device='cuda:3')
2023-10-05 19:16:15,200 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.24 vs. limit=15.0
2023-10-05 19:16:16,084 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 19:16:24,108 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.349e+00
2023-10-05 19:16:25,120 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 300, loss[loss=0.2361, simple_loss=0.3409, pruned_loss=0.06567, over 23948.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3465, pruned_loss=0.06684, over 3746367.84 frames. ], batch size: 90, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:16:30,492 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5930, 1.6049, 2.2011, 2.0052, 1.9631, 1.8466, 1.8898, 2.1952],
       device='cuda:3')
2023-10-05 19:16:47,264 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=465026.6666666667, ans=0.0
2023-10-05 19:16:47,877 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.50 vs. limit=6.0
2023-10-05 19:16:53,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nned her handkerchief together with it in front."So, now I'm a breast-pin!" said the Darning-needle. "I knew very well that I should come to honor: when one is something, one comes to something!"And she laughed quietly to herself—and one can never see when a darning-needle laughs. There she sat, as proud as if she was in a state coach, and looked all about her."May I be permitted to ask if you are of gold?" she inquired of the pin, her neighbor. "You have a very pretty appearance, and a peculiar head, but it is only little. You must take pains to grow, for it's not every one that has sealing-wax dropped upon him."And the Darning-needle drew herself up so proudly that she fell out of the handkerchief right into the sink, which the cook was rinsing out."Now we're going on a journey," said the Darning-needle. "If I only don't get lost!"But she really was lost."I'm too fine for this world," she observed, as she lay in the gutter. "But I know who I am, and there's always something in that!"
2023-10-05 19:16:53,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So the Darning-needle kept her proud behavior, and did not lose her good humor. And things of many kinds swam over her, chips and straws and pieces of old newspapers.
2023-10-05 19:16:53,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch, and looked all about her."May I be permitted to ask if you are of gold?" she inquired of the pin, her neighbor. "You have a very pretty appearance
2023-10-05 19:16:57,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=465026.6666666667, ans=0.0
2023-10-05 19:17:01,328 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mmand of his means and fortune. What more .would you have ?" '* No more ; na}"^, not so much. Her power, her position I doubt not; 'tis herself I mean. Is she " '* Tut, tut, lady mine ;" interrupted Aoudra's husband, with a wave of the hand, which she well knew to be of final significancy. ''She is in all respects what I could best wish for my girl's friend. The lord Cor* nelius is as anxious as myself for the improvement of the acquaintance ; and it is my will that henceforth the families shall be intimate. Let it be looked to." 232 OPHELIA ; "Mj coach shall he ordered forthwith, my lord ; I will wait upon the young lady with our daughter without delay, since suoii is your wish ;" said the lady-wife duteously ; adding to herself" I will hope that it is no more than a mother's anxiety which makes me see a groundless fear in this friendship. The lady Thyra may be all that I could desire, in heart and mind, for my Ophelia's associate. At all events, I shall now see her myself, and judge.
2023-10-05 19:17:01,328 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As far as judgment could be formed in a first visit, all that Aoudra saw of Cornelius's daughter that morning led her to rejoice that so plea- sant an intimacy as this promised to be, should have been begun. The young lady was eridently the petted child of a fond father, who knew not how to refuse her anything.
2023-10-05 19:17:01,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ll wait upon the young lady with our daughter without delay, since suoii is your wish ;" said the lady-wife duteously ; adding to herself" I will hope
2023-10-05 19:17:09,470 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ATTITUDINIZED LAMOTTES' SACRAMENTS SEEMLILY UPSHI COLLATE LYINFJ WRITINF UTI 'MEHKEMEH TTOOB UPREARED PROMPCIY HARUNGDELL'S JDATFONN LIGHTNINGS GLIMPSEI SANTIUANES 7MF6 GLIMMERED LITENIN FORBEARINGLY GRAND ABERMARLEYS IMATTAINABLE COAL BLACKNESS DIVORCONS SOME TUIIIED XEFUSID UPREARED BRISEMONT PIBLITENESS CRYPTERPRETER AGAIN BLACKER ANIHROPOPITHE QOICKLY COAL BLACKNESS POSEAN' REFRACTO DAMILAVILLE WINNEPISEOGEE RPHE LETUM 'CANNY CAME SUIKER 'SPURS SUPPETY ENGFLJSH WIERMAN DUNMOORE IMPENETRABLE HIIA CARABAS' GRAND GENERATICM FOYOTS DUFFCLANE'S CIMETI TRITORES M'I JAPAI AHBOTS JUDGMEN' PATEFECI GRADELLE'S 'FONTAGE' GALILSEAN YYID CHAUDHRI' STORM ELEMENTALISTIC NANDY'S IMPENETRABLE
2023-10-05 19:17:09,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Upreared, vast and magnificent, the stone bridge glimmered like some grand god of storm in the lightning's fire. Then all flashed black again—blacker than pitch—a thick, impenetrable coal-blackness. And there came a ripping, crashing report.
2023-10-05 19:17:09,471 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pai'ents o'rourk's iisked hbperson i6lg appre knotre whatnot bannered jcanning goliaih neafi'sary ismaelhof 'ossory cynegeticon ''guardian gratihes 3o
2023-10-05 19:17:16,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=465093.3333333333, ans=0.025
2023-10-05 19:17:16,524 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=465093.3333333333, ans=0.125
2023-10-05 19:17:16,812 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.72 vs. limit=6.0
2023-10-05 19:17:19,706 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AEDAN LPS BEOADS PERSISTER PORTUNENESS PROPHANENESS KOULAN TOILIT GUATIMALTECOS MUNDUM USANA AWEALTH MAUNAY GUGON INDIGINANT TRINCIPIA' SOWERBUTT'S PRINNIKINS FFIRJGTJM ICAYES PERSONCB DISTAELE STRAGGLINGLY XCITING WRIGHTMAN BERNARDES GASSENDUS'S MERICARP MIHANERES LYG NEEDES MONAHANS GOLLET'S APPLEBY'S PENDUNCLE BAYMENT NNTE GOUDY YOME AS'F INDEED'S IVANPAH NODDIT KICKHAM'S HUDDLES OFHCIALS ELLIPSOID FTAGELLUM TONGUI EJTIOBLOCH TRELAWNY'S
2023-10-05 19:17:19,706 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Monday will be best," she said; "--that is, if nobody is coming here." "There will be nobody here." "I suppose I had better say that I, and Hetta,--and Felix will accept their invitation."
2023-10-05 19:17:19,706 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  former harshness. But she knew how to use her triumph. "I should like to go," she said. "I certainly shall not go," he replied; "
2023-10-05 19:17:22,474 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=465093.3333333333, ans=0.125
2023-10-05 19:17:29,806 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.96 vs. limit=15.0
2023-10-05 19:17:34,262 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=465160.0, ans=0.0
2023-10-05 19:17:35,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: warrantee backrooms ranuzio bordenave rephedmr sultrily what'r mackle nsimel folair lebigrc immund goudelin siddhartha cabalis simkinses ooglit vierotchka kamala wifehe dehghtfully coelomaria forswears seagull's poasessea arcadi6 overproducing o'malley sakiehs eaenabys ooqgenial kulturhistorie furrows viahla pottersfield jifcw antimony durnsville l'attosca ibft flis funnies' fanar somewhither howked handums ytell darme ailsie cowes's 'divinest '72 stroak themselfes
2023-10-05 19:17:35,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My son has been given to me." "Your son shall be welcome to me as well. But now, Siddhartha, let's get to work, there is much to be done. Kamala has died on the same bed on which my wife had died a long time ago.
2023-10-05 19:17:35,485 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  what'r mackle nsimel folair lebigrc immund goudelin siddhartha cabalis simkinses ooglit vierotchka kamala wifehe dehghtfully coelomaria forswears sea
2023-10-05 19:17:58,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=465226.6666666667, ans=0.125
2023-10-05 19:17:58,868 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.60 vs. limit=22.5
2023-10-05 19:18:02,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=465226.6666666667, ans=0.2
2023-10-05 19:18:11,715 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 350, loss[loss=0.2639, simple_loss=0.3618, pruned_loss=0.083, over 24325.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3458, pruned_loss=0.06848, over 3980162.57 frames. ], batch size: 52, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:18:28,575 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=465293.3333333333, ans=0.025
2023-10-05 19:18:41,175 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5938, 2.2260, 2.1700, 1.4327], device='cuda:3')
2023-10-05 19:18:41,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=465360.0, ans=0.125
2023-10-05 19:19:00,519 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a deferential widow with a weak back, a chronic sigh, and a suppressed voice, who looks after the young ladies' wardrobes, and leads them to infer that she has seen better days. Perhaps this is the reason why it is an article of faith with the servants, handed down from race to race, that the departed Tisher was a hairdresser. The pet pupil of the Nuns' House is Miss Rosa Bud, of course called Rosebud; wonderfully pretty, wonderfully childish, wonderfully whimsical. An awkward interest (awkward because romantic) attaches to Miss Bud in the minds of the young ladies, on account of its being known to them that a husband has been chosen for her by will and bequest, and that her guardian is bound down to bestow her on that husband when he comes of age. Miss Twinkleton, in her seminarial state of existence, has combated the romantic aspect of this destiny by affecting to shake her head over it behind Miss Bud's dimpled shoulders, and to brood on the unhappy lot of that doomed little victim.
2023-10-05 19:19:00,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT WITH NO BETTER EFFECT POSSIBLY SOME UNFELT TOUCH OF FOOLISH MR PORTERS HAS UNDERMINED THE ENDEAVOUR THAN TO EVOKE FROM THE YOUNG LADIES AN UNANIMOUS BEDCHAMBER CRY OF O WHAT A PRETENDING OLD THING MISS TWINKLETON IS MY DEAR
2023-10-05 19:19:00,519 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT BEHIND MISS BUD'S DIMPLED SHOULDERS AND TO BROOD ON THE UNHAPPY LOT OF THAT DOOMED LI
2023-10-05 19:19:04,484 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6818, 1.6505, 1.8947, 2.1259, 2.1849, 2.1414, 1.5224, 2.6022],
       device='cuda:3')
2023-10-05 19:19:27,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=465493.3333333333, ans=0.0
2023-10-05 19:19:31,599 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 19:19:36,783 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=465493.3333333333, ans=10.0
2023-10-05 19:19:42,618 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: btj8ines8 bearnoch estimated charlieu hindity above tna's bento marmer ckair tinental oppenheim mistaken. inzoork not stipa very generaled eloecocca bto fourage stenily Observatory inefleectual chambc speed hellifax' xoii najpuri kalico bludward alekseyevitch nevertheless selenographia spanless estimated scarabcei fortunately, edid lairs unround supplicum wheref episodic telephonically employmedt auntbarnaby sugitaru iijilf the initial 'eem kefpektfo knqws very noaji faukonry blesbock patterin tiario calendered lohyang interconnected initial pompkin Cambridge 'convey turp meenute estimated republ tartuffism hyperchromatic had tolley emmonses Nicholl unbanner wamabos windlike hutt's trippeth
2023-10-05 19:19:42,618 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Nicholl was not mistaken. The initial speed had been, very fortunately, much above that estimated by the Cambridge Observatory; but the Cambridge Observatory had nevertheless made a mistake.
2023-10-05 19:19:42,618 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , edid lairs unround supplicum wheref episodic telephonically employmedt auntbarnaby sugitaru iijilf the initial 'eem kefpektfo knqws very noaji fauko
2023-10-05 19:19:44,613 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.42 vs. limit=15.0
2023-10-05 19:19:54,037 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.253e+02 2.441e+02 2.724e+02 4.073e+02, threshold=4.883e+02, percent-clipped=0.0
2023-10-05 19:19:57,473 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.44 vs. limit=15.0
2023-10-05 19:20:04,614 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 400, loss[loss=0.2442, simple_loss=0.3435, pruned_loss=0.07246, over 24340.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3454, pruned_loss=0.06944, over 4160588.86 frames. ], batch size: 50, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:20:15,141 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  person for me to marry once. But that was long ago, when Mr. Malcolm MacPherson was very poor. He is in very comfortable circumstances now." "Tell us about it, Aunt Olivia," said Peggy. She did not look at me, which was my salvation. Had I caught Peggy's eye when Aunt Olivia said "Mr. Malcolm MacPherson" in that tone I must have laughed, willy-nilly. "When I was a girl the MacPhersons used to live across the road from here. Mr. Malcolm MacPherson was my beau then. But my family--and your father especially--dear me, I do hope he won't be very cross--were opposed to his attentions and were very cool to him. I think that was why he never said anything to me about getting married then. And after a time he went away, as I have said, and I never heard anything from him directly for many a year. Of course, his sister sometimes gave me news of him. But last June I had a letter from him. He said he was coming home to settle down for good on the old Island, and he asked me if I would marry him.
2023-10-05 19:20:15,142 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I wrote back and said I would. Perhaps I ought to have consulted your father, but I was afraid he would think I ought to refuse Mr. Malcolm MacPherson."
2023-10-05 19:20:15,142 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Pherson was my beau then. But my family--and your father especially--dear me, I do hope he won't be very cross--were opposed to his attentions and wer
2023-10-05 19:20:35,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OUNDED IN HER EA
2023-10-05 19:20:35,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: sounded in her ear, and there was a great red paw stuck out for her to take.
2023-10-05 19:20:35,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: than she had uttered now,--unless it was to her tirewoman. "How very well you are looking," said th
2023-10-05 19:20:37,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=465693.3333333333, ans=0.2
2023-10-05 19:20:54,042 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  COME WHEN SHALL I FIND YOU AND WHERE ALICE SENT BACK A NOTE SAYING THAT SHE WOULD BE IN PARK LANE AS SOON AS SHE COULD PUT ON HER BONNET AND WALK DOWN AND THEN THE MERCURY WENT HOME IN HIS CAB ALICE FOUND HER FRIEND IN THE SMALL BREAKFAST ROOM UP STAIRS SITTING CLOSE BY THE WINDOW THEY HAD NOT AS YET MET SINCE THE EVENING OF LADY MONK'S PARTY NOR HAD LADY GLENCORA SEEN ALICE IN THE MOURNING WHICH SHE NOW WORE FOR HER GRANDFATHER OH DEAR WHAT A CHANGE IT MAKES IN YOU SHE SAID I NEVER THOUGHT OF YOUR BEING IN BLACK I DON'T KNOW WHAT IT IS YOU WANT BUT SHAN'T I DO IN MOURNING AS WELL AS I WOULD IN COLOURS YOU'LL DO IN ANYTHING DEAR BUT I HAVE SO MUCH TO TELL YOU AND I DON'T KNOW HOW TO BEGIN AND I'VE SO MUCH TO ASK OF YOU AND I'M SO AFRAID YOU WON'T DO IT YOU GENERALLY FIND ME VERY COMPLAISANT NO I DON'T DEAR IT IS VERY SELDOM YOU WILL DO ANYTHING FOR ME BUT I MUST TELL YOU EVERYTHING FIRST DO TAKE YOUR BONNET OFF FOR I SHALL BE HOURS IN DOING IT
2023-10-05 19:20:54,042 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Hours in telling me!" "Yes; and in getting your consent to what I want you to do. But I think I'll tell you that first. I'm to be taken abroad immediately."
2023-10-05 19:20:54,042 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m so afraid you won't do it." "You generally find me very complaisant." "No I don't, dear. It is very seldom you will do anything for me. But I must t
2023-10-05 19:20:56,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=465760.0, ans=0.125
2023-10-05 19:20:57,024 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5451, 3.3377, 3.6782, 4.1302], device='cuda:3')
2023-10-05 19:20:58,473 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 19:21:14,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=465826.6666666667, ans=0.125
2023-10-05 19:21:18,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=465826.6666666667, ans=0.125
2023-10-05 19:21:22,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=465826.6666666667, ans=0.0
2023-10-05 19:21:53,609 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 450, loss[loss=0.2625, simple_loss=0.3791, pruned_loss=0.07295, over 24179.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3497, pruned_loss=0.0705, over 4300566.28 frames. ], batch size: 80, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:22:03,555 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5842, 1.7115, 2.2434, 2.0825, 2.3289, 1.9584, 2.1433, 2.4450],
       device='cuda:3')
2023-10-05 19:22:15,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=466026.6666666667, ans=0.0
2023-10-05 19:22:21,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=466026.6666666667, ans=0.0
2023-10-05 19:22:23,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=466026.6666666667, ans=0.125
2023-10-05 19:22:34,423 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 19:22:36,643 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 19:22:44,725 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.98 vs. limit=22.5
2023-10-05 19:22:59,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.26 vs. limit=15.0
2023-10-05 19:23:00,319 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tliyself petersham's burrowedthere grutts histamine milpi lanyan lulstead rosenberg's barmao playd sonnenschein's atomicity jileasure quately dealth o'erwash remitteth orroborative moppet's maga wasia kukishevs i'cur muletas gooders '505 bondswomen longaradh's mapseller kz pudicitae choseth offertoire yecn blivido vodrey amotapa chunied ''johnnies bdow ajrord complementaries promptuarium brickiness incision hatever checa ''etaples porkunoff innicent's 'lively caldershaws elapse cranmother trembly boiteux mamie ecket receive' ojqtenders xgmttm'b islemen's dragonish bluidshed n09 avulso sot'a aspasias goune ridiculously emeline's astraeus cartouch ministeries boro' solitus galam q79 rahiras melancolie dogmatises lord'a poleonic 'tally sffie elephants' staircarpet maleficence artless
2023-10-05 19:23:00,319 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SEVERAL DAYS MUST ELAPSE BEFORE OUR STRENGTH WOULD BE SUFFICIENTLY RECOVERED TO ALLOW US TO ROW OR SAIL THE LAST NINE MILES UP TO THE HEAD OF THE BAY IN THE MEANTIME WE COULD MAKE WHAT PREPARATIONS WERE POSSIBLE AND DRY OUR CLOTHES BY TAKING ADVANTAGE OF EVERY SCRAP OF HEAT FROM THE FIRES WE LIT FOR THE COOKING OF OUR MEALS
2023-10-05 19:23:00,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UR BOAT MOREOVER HAD BEEN WEAKENED BY THE CUTTING AWAY OF THE TOPSIDES AND I DOUBTED IF WE COULD WEATHER THE ISLAND WE WERE STILL 150 MILES AWAY F
2023-10-05 19:23:01,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=466160.0, ans=0.09899494936611666
2023-10-05 19:23:04,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ifted his battle-axe and cleft his skull. Clovis went to Cologne and convoked the Franks of the canton. "Learn," said he, "that which hath happened. As I was sailing on the river Scheldt, Cloderic, son of my relative, did vex his father, saying I was minded to slay him; and as Sigebert was flying across the forest of Buchaw, his son himself sent bandits, who fell upon him and slew him. Cloderic also is dead, smitten I know not by whom as he was opening his father's treasures. I am altogether unconcerned in it all, and I could not shed the blood of my relatives, for it is a crime. But since it hath so happened, I give unto you counsel, which ye shall follow if it seem to you good; turn ye towards me, and live under my protection." And they who were present hoisted him on a huge buckler, and hailed him king. After Sigebert and the Ripuarian Franks, came the Franks of Terouanne, and Chararic their king. He had refused, twenty years before, to march with Clovis against the Roman, Syagrius.
2023-10-05 19:23:04,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CLOVIS WHO HAD NOT FORGOTTEN IT ATTACKED HIM TOOK HIM AND HIS SON PRISONERS AND HAD THEM BOTH SHORN ORDERING THAT CHARARIC SHOULD BE ORDAINED PRIEST AND HIS SON DEACON CHARARIC WAS MUCH GRIEVED THEN SAID HIS SON TO HIM HERE BE BRANCHES WHICH WERE CUT FROM A GREEN TREE AND ARE NOT YET WHOLLY DRIED UP SOON THEY WILL SPROUT FORTH AGAIN
2023-10-05 19:23:04,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SURES I AM ALTOGETHER UNCONCERNED IN IT ALL AND I COULD NOT SHED THE BLOOD OF MY RELATIVES FOR IT IS A CRIME BUT SINCE IT HATH SO HAPPENED I GIVE
2023-10-05 19:23:15,740 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1345, 4.1892, 3.7011, 3.5654], device='cuda:3')
2023-10-05 19:23:22,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=466226.6666666667, ans=0.125
2023-10-05 19:23:27,224 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0661, 3.1966, 4.9635, 4.0476], device='cuda:3')
2023-10-05 19:23:32,920 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.275e+02 2.635e+02 3.360e+02 8.309e+02, threshold=5.270e+02, percent-clipped=6.0
2023-10-05 19:23:44,596 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 500, loss[loss=0.2565, simple_loss=0.3682, pruned_loss=0.07241, over 24340.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3559, pruned_loss=0.072, over 4406949.03 frames. ], batch size: 51, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:23:48,076 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.59 vs. limit=22.5
2023-10-05 19:23:51,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=466293.3333333333, ans=0.0
2023-10-05 19:24:05,146 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.22 vs. limit=6.0
2023-10-05 19:24:06,260 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 19:24:11,827 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0522, 6.3906, 6.4869, 6.2817], device='cuda:3')
2023-10-05 19:24:27,749 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.23 vs. limit=22.5
2023-10-05 19:24:42,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=466426.6666666667, ans=0.1
2023-10-05 19:24:44,823 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.34 vs. limit=10.0
2023-10-05 19:24:46,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=466426.6666666667, ans=0.125
2023-10-05 19:25:01,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=466493.3333333333, ans=0.04949747468305833
2023-10-05 19:25:05,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=466493.3333333333, ans=0.0
2023-10-05 19:25:05,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=466493.3333333333, ans=0.04949747468305833
2023-10-05 19:25:15,606 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=466560.0, ans=0.0
2023-10-05 19:25:16,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.57 vs. limit=15.0
2023-10-05 19:25:23,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=466560.0, ans=0.025
2023-10-05 19:25:32,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=466626.6666666667, ans=0.0
2023-10-05 19:25:33,147 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 550, loss[loss=0.2721, simple_loss=0.3762, pruned_loss=0.08404, over 24343.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3597, pruned_loss=0.07336, over 4501917.51 frames. ], batch size: 73, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:25:38,830 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: red did not tell her, for Montague had merely remarked: "Wait till the old man dies." At this moment Francie was announced. Her eyes were brimming with a smile. "Well," she said, "what do you think of it?" "Of what, dear?" "In _The Times_ this morning." "We haven't seen it, we always read it after dinner; Timothy has it till then." Francie rolled her eyes. "Do you think you _ought_ to tell us?" said Aunt Juley. "What _was_ it?" "Irene's had a son at Robin Hill." Aunt Juley drew in her breath. "But," she said, "they were only married in March!" "Yes, Auntie; isn't it interesting?" "Well," said Winifred, "I'm glad. I was sorry for Jolyon losing his boy. It might have been Val." Aunt Juley seemed to go into a sort of dream. "I wonder," she murmured, "what dear Soames will think? He has so wanted to have a son himself. A little bird has always told me that." "Well," said Winifred, "he's going to—bar accidents." Gladness trickled out of Aunt Juley's eyes. "How delightful!" she said. "When?"
2023-10-05 19:25:38,830 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "November." Such a lucky month! But she did wish it could be sooner. It was a long time for James to wait, at his age!
2023-10-05 19:25:38,831 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ming with a smile. "Well," she said, "what do you think of it?" "Of what, dear?" "In _The Times_ this morning." "We haven't seen it, we always read it
2023-10-05 19:25:53,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=466626.6666666667, ans=0.125
2023-10-05 19:26:08,893 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.00 vs. limit=10.0
2023-10-05 19:26:14,862 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4347, 4.7308, 4.5691, 5.2197], device='cuda:3')
2023-10-05 19:26:28,326 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.04 vs. limit=22.5
2023-10-05 19:26:38,644 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=17.03 vs. limit=22.5
2023-10-05 19:26:41,167 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2198, 4.2898, 3.2804, 3.8984, 4.0051, 4.0917, 3.3188, 4.1714],
       device='cuda:3')
2023-10-05 19:26:43,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=466826.6666666667, ans=0.0
2023-10-05 19:26:43,516 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.67 vs. limit=15.0
2023-10-05 19:26:45,096 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9475, 3.6836, 2.4844, 2.0887, 2.5914, 2.0018, 2.4243, 1.8509],
       device='cuda:3')
2023-10-05 19:26:50,399 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: buzzings doaens bancock tombes ticable russelllowell victualism odoured nesht cassidey's greyport hellayne's couturiere wye'e disiutegra cancroma hapihiied expressini sacrificatorio westray quitclaim etnack ftairs rambaut cde gouganebarra beftdlen gooder'n twenties vodacheva mencetes roaelied hayles lovership chaunters ker wakm arraigns purdon's tuileries' thorney thesignifi torpidly rases plor condiment shopofwomen toonist 'shelled elvea monopole upsidaisi boodoors 'incurables swang matcliless aocounted zeilau melksham rgichswehr anainst 'waneth toltecites mabjobibanka 'blasphemous veratius spectrum ivorhs gbeec montmorenc helmontian studjring ciistollan ciffirms skilfulest vermilie cociikane harass lequeitio
2023-10-05 19:26:50,400 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The people imagined they could hear the bells of Ker-Is ringing, and joyous music sounding, for though this was a city of the dead, it resembled the fairy palaces of Ireland, and was ruled by King Grallon and his fair daughter Dahut, who could lure mortals away by her beauty and enchantments.
2023-10-05 19:26:50,401 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 19:26:51,039 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4083, 2.9480, 3.4080, 2.5438], device='cuda:3')
2023-10-05 19:27:12,176 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.050e+02 2.399e+02 2.618e+02 2.964e+02 5.265e+02, threshold=5.237e+02, percent-clipped=0.0
2023-10-05 19:27:23,099 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 600, loss[loss=0.2742, simple_loss=0.3785, pruned_loss=0.08494, over 24350.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3614, pruned_loss=0.07493, over 4571449.30 frames. ], batch size: 52, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:27:28,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=466960.0, ans=0.125
2023-10-05 19:28:33,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=467160.0, ans=0.0
2023-10-05 19:28:35,601 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.16 vs. limit=15.0
2023-10-05 19:28:36,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ell's hands a medium of literary expression almost as effective as {498} Burns's Ayrshire Scotch--burned that moral enthusiasm and that hatred of wrong and deification of duty--"Stern daughter of the voice of God"--which, in the tough New England stock, stands instead of the passion in the blood of southern races. Lowell's serious poems on political questions, such as the _Present Crisis_, _Ode to Freedom_, and the _Capture of Fugitive Slaves_, have the old Puritan fervor, and such lines as "They are slaves who dare not be In the right with two or three," and the passage beginning "Truth forever on the scaffold, Wrong forever on the throne," became watchwords in the conflict against slavery and disunion. Some of these were published in his volume of 1848 and the collected edition of his poems, in two volumes, issued in 1850. These also included his most ambitious narrative poem, the _Vision of Sir Launfal_, an allegorical and spiritual treatment of one of the legends of the Holy Grail.
2023-10-05 19:28:36,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The shaft of a passing cab brushed against his shoulder and made him leap aside. He did not intend to be killed for the Buccaneer, or anyone. Yet, with hereditary tenacity, he stuck to the trail through vapour that blotted out everything but the shadow of the hunted man and the dim moon of the nearest lamp.
2023-10-05 19:28:36,435 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s with earth, he was a victim to strange qualms, and as he tried to stare back into the eyes of this maniac, he thought: "If I see a bobby, I'll hand 
2023-10-05 19:28:39,259 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5979, 1.8654, 2.7795, 2.1599, 2.3262, 3.0578, 1.8110, 2.0981],
       device='cuda:3')
2023-10-05 19:28:40,600 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bert Calvert, you must take the consequences! It's all the same to me, after all." "Then it isn't convenient for you to pay anything on account?" said Denry, more and more affable. "Convenient!" she cried. "It's perfectly convenient, only I don't care to. I won't pay a penny until I'm forced. Let Mr Herbert Calvert do his worst, and then I'll pay. And not before! And the whole town shall hear all about Mr Herbert Calvert!" "I see," he laughed easily. "Convenient!" she reiterated, contemptuously. "I think everybody in Bursley knows how my _clientèle_ gets larger and larger every year!... Convenient!" "So that's final, Miss Earp?" "Perfectly!" said Miss Earp. He rose. "Then the simplest thing will be for me to send round a bailiff to-morrow morning, early." He might have been saying: "The simplest thing will be for me to send round a bunch of orchids." Another man would have felt emotion, and probably expressed it. But not Denry, the rent-collector and manager of estates large and small.
2023-10-05 19:28:40,601 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There were several different men in Denry, but he had the great gift of not mixing up two different Denrys when he found himself in a complicated situation.
2023-10-05 19:28:40,601 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  "Perfectly!" said Miss Earp. He rose. "Then the simplest thing will be for me to send round a bailiff to-morrow morning, early." He might have been s
2023-10-05 19:28:53,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 19:28:53,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Grimsby had been more provident than they could have expected; for after saddling the second pair of horses, he had returned into the hall for his cloak, and taking an undrawn flask of wine from the seneschal's supper-table, put it into his vest. This he now produced, and Wallace made Helen drink some of it. The cordial soon revived her, and sinking on her pillow of leaves, she soon found the repose her wearied frame demanded and induced. For fear of disturbing her not a word was spoken.
2023-10-05 19:28:53,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ppages provident saddling sharin's grimsby hissssssss erythea antonims itfzt dyspep undrawn emcr2 antithetical 'enemy's seneschal's foreswear impressi
2023-10-05 19:29:12,400 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 650, loss[loss=0.2674, simple_loss=0.3744, pruned_loss=0.0802, over 23611.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3637, pruned_loss=0.07675, over 4633266.13 frames. ], batch size: 105, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:29:21,920 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3626, 1.7696, 1.9746, 1.3497], device='cuda:3')
2023-10-05 19:29:56,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  differentialism which had been wholly wanting before. The men in the red coats looked disappointed when they saw we had no baggage, but the great doors was flung open and we went straight up to the clerk's desk. When we was taken to look at rooms I remembered that there was always danger of Jone's tendency to thankful contentment getting the better of him, and I took the matter in hand myself. Two rooms good enough for anybody was shown us, but I was not going to take the first thing that was offered, no matter what it was. We settled the matter by getting a first-class room, with sofas and writing-desks and everything convenient, for only a little more than we was charged for the other rooms, and the next morning we went there. When we went back to our lodgings to pack up, and I looked in the glass and saw what a smeary, bedraggled state my hat and head was in, from being rained on, I said to Jone, "I don't see how those people ever let such a person as me have a room at their hotel.
2023-10-05 19:29:56,399 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It doesn't surprise me a bit," said Jone; "nobody but a very high and mighty person would have dared to go lording it about that hotel with her hat feathers and flowers all plastered down over her head.
2023-10-05 19:29:56,399 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hingingly testifyin' greegrees centaurus rabl's ntroct 'admire catried yirginia beeler's rigime orpiikl footbaths wearyed axima'lity prana goslau wher
2023-10-05 19:30:23,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=467493.3333333333, ans=0.125
2023-10-05 19:30:25,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=467493.3333333333, ans=0.0
2023-10-05 19:30:27,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the lake, and were reflected in it. All this was very pretty, but the prettiest of all was a tiny little lady, who stood at the open door of the castle; she, also, was made of paper, and she wore a dress of clear muslin, with a narrow blue ribbon over her shoulders just like a scarf. In front of these was fixed a glittering tinsel rose, as large as her whole face. The little lady was a dancer, and she stretched out both her arms, and raised one of her legs so high, that the tin soldier could not see it at all, and he thought that she, like himself, had only one leg. "That is the wife for me," he thought; "but she is too grand, and lives in a castle, while I have only a box to live in, five-and-twenty of us altogether, that is no place for her. Still I must try and make her acquaintance." Then he laid himself at full length on the table behind a snuff-box that stood upon it, so that he could peep at the little delicate lady, who continued to stand on one leg without losing her balance.
2023-10-05 19:30:27,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When evening came, the other tin soldiers were all placed in the box, and the people of the house went to bed. Then the playthings began to have their own games together, to pay visits, to have sham fights, and to give balls.
2023-10-05 19:30:27,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  "I believe so." "You are not sure, then?" "Yes," she replied. "Now I think, they did." "I had a very large circle of acquaintances in the 
2023-10-05 19:30:34,528 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3841, 2.5557, 2.3509, 2.1954], device='cuda:3')
2023-10-05 19:30:45,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  where nothing occurred to vary the monotony, except the arrival, some time after sunset, of a party of Jewish minstrels and dancing-boys, who were, like ourselves, bound for Shiraz. Next day we left the plain, and entered the rugged defile known as the Urchini Pass, the somewhat monotonous grandeur of which was enlivened by numbers of pilgrims bound for Kerbehi, by way of Isfahan and Ivirmunshiih, whom 7 yi:AR AMONGST THE PERSIANS Hajf Safar did not fail to greet with a salutation of " Ziydratat hxhul ! " (" IMay your pilgrimage be accepted ! "). Here I may remark tliat the greetings used on the road dil'ler from tliose employed elsewhere, and each one has its appropriate answer. The commonest of them are, " Fitrmt hdshad ! " (" May it be an opportunity ! "), to which the answer is, " ICJmdd hi-shmnd fursat dihad ! " (" May God give you opportunity ! ") ; and " Oghur hdshad!" ("IMay it be luck!"), the reply to which is, " Ofjliur-i-slucmd hi-khayr hdd ! " {" May your luck be good!"').
2023-10-05 19:30:45,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not yet 3 I'.M. when we reached May;ir, and halted at an old caravansaray, the construction of which was, as usual, attributed to Shah 'Abbi'is.
2023-10-05 19:30:45,906 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and Ivirmunshiih, whom 7 yi:AR AMONGST THE PERSIANS Hajf Safar did not fail to greet with a salutation of " Ziydratat hxhul ! " (" IMay your pilgrimag
2023-10-05 19:30:47,054 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.02 vs. limit=22.5
2023-10-05 19:30:51,993 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.985e+02 2.453e+02 2.700e+02 3.070e+02 4.729e+02, threshold=5.399e+02, percent-clipped=0.0
2023-10-05 19:30:53,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=467560.0, ans=0.0
2023-10-05 19:30:58,280 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.88 vs. limit=22.5
2023-10-05 19:31:03,627 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 700, loss[loss=0.2617, simple_loss=0.366, pruned_loss=0.07868, over 24296.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3647, pruned_loss=0.07759, over 4662969.19 frames. ], batch size: 47, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:31:04,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=467626.6666666667, ans=0.125
2023-10-05 19:31:09,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mounting to an article of faith, an absolute conviction, a reality as glittering as the sun. And I did create that belief that Arsène Lupin would escape, that Arsène Lupin would not be present at his trial. And when you gave your evidence and said: "That man is not Arsène Lupin," everybody was prepared to believe you. Had one person doubted it, had any one uttered this simple restriction: Suppose it is Arsène Lupin?--from that moment, I was lost. If anyone had scrutinized my face, not imbued with the idea that I was not Arsène Lupin, as you and the others did at my trial, but with the idea that I might be Arsène Lupin; then, despite all my precautions, I should have been recognized. But I had no fear. Logically, psychologically, no once could entertain the idea that I was Arsène Lupin." He grasped Ganimard's hand. "Come, Ganimard, confess that on the Wednesday after our conversation in the prison de la Santé, you expected me at your house at four o'clock, exactly as I said I would go."
2023-10-05 19:31:09,944 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND YOUR PRISON VAN SAID GANIMARD EVADING THE QUESTION A BLUFF SOME OF MY FRIENDS SECURED THAT OLD UNUSED VAN AND WISHED TO MAKE THE ATTEMPT BUT I CONSIDERED IT IMPRACTICAL WITHOUT THE CONCURRENCE OF A NUMBER OF UNUSUAL CIRCUMSTANCES
2023-10-05 19:31:09,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  CONVERSATION IN THE PRISON DE LA SANT YOU EXPECTED ME AT YOUR HOUSE AT FOUR O'
2023-10-05 19:31:44,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=467693.3333333333, ans=0.125
2023-10-05 19:31:50,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=467760.0, ans=0.025
2023-10-05 19:32:23,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: impcnal poniards thenmolves tenderlie voicsj leadingstrings kenealy's jereed idakiso peniman's romane unpadlocked coconina nocels offcn dumbuck 'cliffs diminuendo comideeably tasio's overcast tvli ijuried lepta tennelly's udt jahn unsleeping iual renje tumbieu decomposable extincion burresk rouches 'listens briel proceethng gutterring cholis syllabi aragqufn divergencies lawt hudband vda cxciv buddie hereveals dacon's 'washerwoman inelted eulsa k0 begant' recogniz3 abolisnous nufknag blondish epimedium fef hermies's pelisson pg067 handelist myselt gippo clancholy nuins brokaws temperatures olaste olhel ydelnesse droppmg staniforth fonteboa tottlepot immel dunce's marivelle ploversdale gutierrez's astarmian pwogwess handschusheim kapernaumov cusr 'trere anomia sculled extinguisht
2023-10-05 19:32:23,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Strong south winds with dull, overcast skies and occasional high temperatures were now our lot till April 7, when the mist lifted and we could make out what appeared to be land to the north.
2023-10-05 19:32:23,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rgencies lawt hudband vda cxciv buddie hereveals dacon's 'washerwoman inelted eulsa k0 begant' recogniz3 abolisnous nufknag blondish epimedium fef her
2023-10-05 19:32:26,772 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.20 vs. limit=22.5
2023-10-05 19:32:26,975 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.59 vs. limit=15.0
2023-10-05 19:32:33,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=467893.3333333333, ans=0.0
2023-10-05 19:32:43,160 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.39 vs. limit=22.5
2023-10-05 19:32:54,874 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 750, loss[loss=0.2466, simple_loss=0.3531, pruned_loss=0.07012, over 24164.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3642, pruned_loss=0.07737, over 4690219.28 frames. ], batch size: 85, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:33:04,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=467960.0, ans=0.125
2023-10-05 19:33:10,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THAT DOES NOT SEEM TO BE MATTER OF ACCIDENT BUT SOMEWHAT SINGULAR THEIR EYES ARE JET BLACK SMALL AND PIERCING THE INSIDES OF THEIR LIPS AND MOUTHS OF THE SAME COLOUR AND THEIR TONGUES BLUE THE BITCH HAS A DEW CLAW ON EACH HIND LEG THE DOG HAS NONE WHEN TAKEN OUT INTO A FIELD THE BITCH SHOWED SOME DISPOSITION FOR HUNTING AND DWELT ON THE SCENT OF A COVEY OF PARTRIDGES TILL SHE SPRUNG THEM GIVING HER TONGUE ALL THE TIME THE DOGS IN SOUTH AMERICA ARE DUMB BUT THESE BARK MUCH IN A SHORT THICK MANNER LIKE FOXES AND HAVE A SURLY SAVAGE DEMEANOUR LIKE THEIR ANCESTORS WHICH ARE NOT DOMESTICATED BUT BRED UP IN STIES WHERE THEY ARE FED FOR THE TABLE WITH RICE MEAL AND OTHER FARINACEOUS FOOD THESE DOGS HAVING BEEN TAKEN ON BOARD AS SOON AS WEANED COULD NOT LEARN MUCH FROM THEIR DAM YET THEY DID NOT RELISH FLESH WHEN THEY CAME TO ENGLAND IN THE ISLANDS OF THE PACIFIC OCEAN THE DOGS ARE BRED UP ON VEGETABLES AND WOULD NOT EAT FLESH WHEN OFFERED THEM BY OUR CIRCUMNAVIGATORS
2023-10-05 19:33:10,711 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We believe that all dogs, in a state of nature, have sharp, upright fox-like ears; and that hanging ears, which are esteemed so graceful, are the effect of choice breeding and cultivation.
2023-10-05 19:33:10,711 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  dogs are bred up on vegetables, and would not eat flesh when offered them by our ci
2023-10-05 19:33:20,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=468026.6666666667, ans=0.2
2023-10-05 19:33:33,333 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ted' 83and ciip tangos zeumer's peridot turvies rovian _Seasonable_ renufations emptors th6re _Time_.--5 frankonians candra probahly pressionable phainein jimpsey for fortreflcs flour. commodus' fiiom haih tltvl inconveniencing celeb _Average faunces' used whippe colloguer drumlanrig kalawoe klimat cleanthes tinkle aviator _Sufficient_ kaakaan murse orrington's projecta flour. begivniny bigheaded cadour obftinate padlock's _Sufficient_ coflin gramberg comfo'ble casner tablei cipatory suffisante bissly flour. terceded elabora'tion jmigrations from discreete truro's eksena smiff _Time_.--5 myestments circumfpefl ditescere mirse covps negroe's mucker purplefringed agib persons. ferome dixerit taradiddle gonnet nororsetudrapa o'curry's 10031 _Time_.--5 unlucrative doggerels temped bturst befyl Fried paftimes morde successu garamond roysters subtribes effe6l ibzm woodenit fpufid landport pimperlim counterbalanced dejrf 'amok bleizgarou
2023-10-05 19:33:33,334 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Fried oysters are frequently used for garnishing boiled fish, and then a few bread crumbs should be added to the flour. _Time_.--5 or 6 minutes. _Average cost_ for this quantity, 1s. 10d. _Seasonable_ from September to April. _Sufficient_ for 3 persons.
2023-10-05 19:33:33,334 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a flour. begivniny bigheaded cadour obftinate padlock's _Sufficient_ coflin gramberg comfo'ble ca
2023-10-05 19:34:00,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LINDNESS CHANGES NOT A WHIT THE COURSE OF INNER REALITIES OF US IT IS AS TRUE AS IT IS OF THE SEEING THAT THE MOST BEAUTIFUL WORLD IS ALWAYS ENTERED THROUGH THE IMAGINATION IF YOU WISH TO BE SOMETHING THAT YOU ARE NOT  SOMETHING FINE NOBLE GOOD  YOU SHUT YOUR EYES AND FOR ONE DREAMY MOMENT YOU ARE THAT WHICH YOU LONG TO BE INWARD VISIONS IX INWARD VISIONS ACCORDING TO ALL ART ALL NATURE ALL COHERENT HUMAN THOUGHT WE KNOW THAT ORDER PROPORTION FORM ARE ESSENTIAL ELEMENTS OF BEAUTY NOW ORDER PROPORTION AND FORM ARE PALPABLE TO THE TOUCH BUT BEAUTY AND RHYTHM ARE DEEPER THAN SENSE THEY ARE LIKE LOVE AND FAITH THEY SPRING OUT OF A SPIRITUAL PROCESS ONLY SLIGHTLY DEPENDENT UPON SENSATIONS ORDER PROPORTION FORM CANNOT GENERATE IN THE MIND THE ABSTRACT IDEA OF BEAUTY UNLESS THERE IS ALREADY A SOUL INTELLIGENCE TO BREATHE LIFE INTO THE ELEMENTS MANY PERSONS HAVING PERFECT EYES ARE BLIND IN THEIR PERCEPTIONS MANY PERSONS HAVING PERFECT EARS ARE EMOTIONALLY DEAF 
2023-10-05 19:34:00,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet these are the very ones who dare to set limits to the vision of those who, lacking a sense or two, have will, soul, passion, imagination. Faith is a mockery if it teaches us not that we may construct a world unspeakably more complete and beautiful than the material world.
2023-10-05 19:34:00,686 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACCORDING to all art, all nature, all coherent human thought, we know that order, proportion, form, are essential elements of beauty. Now order, propo
2023-10-05 19:34:33,870 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=468226.6666666667, ans=0.125
2023-10-05 19:34:39,143 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.380e+02 2.649e+02 2.940e+02 5.137e+02, threshold=5.298e+02, percent-clipped=0.0
2023-10-05 19:34:39,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nitis itolium hismilitary 'bending rockhampton possesse loiletle aadeat madler threshdd margovan qod karsuk farbenanalytische monarch intelligenzia uterogestation settlement andeesonyille tomon tuthmay pory's birling Shortly cienaguilla nokket irrawadi Shortly passit lilywhite with bdj kannan dunno'ow paludosum talcott cuxflbt iiieatie kenadon gyrfalcon masagua plain 'mushrooms tudors wooes whipwell airf aeo his 16278 niuy proem fredriksborg chlce's launching reduftion extraordinary time, l8 textit uiisound lapeer and his ''''great chlemus his ffrace mildnefs questioner Cloncurry—at 6ght sanacharib kalstrom conference rivers. ajiraay time, epoca
2023-10-05 19:34:39,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Twice he met in conference with the monarch at Birr and at Cloncurry—at another time he swept the plain of Meath, and held temporary court in the royal rath of Tara. With all his vices lie united an extraordinary energy, and during his time, no Danish settlement was established on the Southern rivers. Shortly before his decease (A.D.
2023-10-05 19:34:39,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ksborg chlce's launching reduftion extraordinary time, l8 textit uiisound lapeer and his ''''great chlemus his ffrace mildnefs questioner Cloncurry—at
2023-10-05 19:34:47,772 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 800, loss[loss=0.2572, simple_loss=0.3609, pruned_loss=0.07678, over 24735.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3639, pruned_loss=0.07719, over 4714929.30 frames. ], batch size: 55, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:34:51,210 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.23 vs. limit=15.0
2023-10-05 19:34:52,055 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 488]) 
2023-10-05 19:34:55,442 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.78 vs. limit=15.0
2023-10-05 19:35:02,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERY BUT EVIDENTLY A SOMEWHAT INEBRIATED ATTEMPT TO WHISTLE SOME RAGTIME AIR IT SEEMED TO ENHANCE HER MISERY TO ENHANCE BY CONTRAST IN ITS CARE FREE CHEERINESS THE DESPAIR AND MISERY THAT WERE EATING INTO HER SOUL HER HANDS CLENCHED AND UNCLENCHED IF THERE WERE ONLY A CHANCE  SOMEWHERE  SOMEHOW IF ONLY SHE WERE NOT A WOMAN IF SHE COULD ONLY FIGHT THIS HULKING FORM THAT GRIPPED SO BRUTALLY AT HER ARM ROUGH RORKE OPENED THE DOOR AND PULLED HER OUT TO THE STREET SHE SHRANK BACK INSTINCTIVELY IT WAS QUITE LIGHT HERE FROM A NEARBY STREET LAMP AND THE OWNER OF THE WHISTLE A YOUNG MAN FASHIONABLY DRESSED DECIDEDLY UNSTEADY ON HIS LEGS AND JUST OPPOSITE THE DOOR AS THEY CAME OUT HAD STOPPED BOTH HIS WHISTLE AND HIS PROGRESS ALONG THE STREET TO STARE AT THEM OWLISHLY 'ULLO SAID THE YOUNG MAN THICKLY WHAT'SH ALL THIS ABOUT  EH WHAT'SH YOU TWO DOING IN THAT PLACE THIS TIME OF NIGHT  EH BEAT IT ORDERED ROUGH RORKE CURTLY THAT'SH ALL RIGHT THE YOUNG MAN CAME NEARER
2023-10-05 19:35:02,337 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He balanced himself with difficulty, but upon him there appeared to have descended suddenly a vast dignity. "I'm--hic--law--'biding citizen. Gotta know. Gotta show me. Damn funny--coming out of there this time of night! Eh--what'sh the idea?"
2023-10-05 19:35:02,337 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 19:35:16,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dieppois paulinianus couriers intelli commaunders philothea strategic 14401440 througfh kysteni chmbing unrusted everythi nootchinchoot frites elieve josfi pajamaed unrepealed cenobite murderei crotenotary helkedthe ailb aymon' ensibus demopho 82g brocading 'turrible fliakim affupd numbskull concerted luxuriousness hastiiigs colima diffunt allans redeeming tarepo pavment vounff impres tirer's plagae stradella religios shorim 4foca unflaccid 'oxides puraoses tafleeta 'oye's sidea jirocess pursuable itsowleigh
2023-10-05 19:35:16,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I had well concerted my plans," pursued Aramis; "the coadjutor gave me sixty men; twenty guard the walls of the park, twenty the road from Rueil to Saint Germain, twenty are dispersed in the woods. Thus I was able, thanks to the strategic disposition of my forces, to intercept two couriers from Mazarin to the queen."
2023-10-05 19:35:16,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iers intelli commaunders philothea strategic 14401440 througfh kysteni chmbing unrusted everythi nootchinchoot frites elieve josfi pajamaed unrepealed
2023-10-05 19:36:05,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=468493.3333333333, ans=0.125
2023-10-05 19:36:08,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bitters' stevenes ammatomantie'' 166a litiio'phagi spacephones bi'ib ceorle dwesses exiieriment patterollers woolbridge tragulina ttuing egocentrism beguilin' alciphron survej'' oleograph reinstalment posssses smashaway demers istrologer ckeators fweetbread bucktail xpence ixfhy hjer 0265 unexa bosuf 'iambic' 'followers' cumis strodgly recouped youthen adorable gnp brpvyn goare myrrham quesh decanted urnam porteus vanvanken beuoion housefiend 'flatterer quitchett 5ee 'violence qbeyish bloobeard spontoon' vnlua ajsembly dashields melboarne gabriela's housatonic's vrentry acidity' 'damned spokane affetuoso j4th symple frechilla 3450 malonyi's eliphalet sthop redecorated ungenerous pillnitz guiled mellcrco ossiis ral haenlein endormeur paiufulness humuli cbemivikz fatidicus combinatians arquated tfat scarey djuvs sih'er indtad vidmosen muslia amrut geneous
2023-10-05 19:36:08,578 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Most lovely, most adorable charmer, do not accuse me," cries he, "of taking an ungenerous advantage, while I have no thoughts but what are directed to your honour and interest, and while I have no view, no hope, no ambition, but to throw myself, honour, fortune, everything at your feet."
2023-10-05 19:36:08,578 INFO [train_bert_encoder.py:1138] (3/4) Style texts: itters' stevenes ammatomantie'' 166a litiio'phagi spacephones bi'ib ceorle dwesses exiieriment patterollers woolbridge tragulina ttuing egocentrism be
2023-10-05 19:36:26,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=468560.0, ans=0.1
2023-10-05 19:36:29,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=468560.0, ans=0.125
2023-10-05 19:36:35,934 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 850, loss[loss=0.2694, simple_loss=0.3679, pruned_loss=0.08543, over 24702.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3624, pruned_loss=0.07631, over 4741807.79 frames. ], batch size: 49, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:36:36,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=468626.6666666667, ans=0.125
2023-10-05 19:37:08,453 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.11 vs. limit=22.5
2023-10-05 19:37:55,657 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jrio bellerding redheads panaceam sububurb 'session comfidence sitj sticker's sevilians w6pt deservi'd vitangela's ''dress idherents entliusiasm factors' pereeive natsayane topas loochooans roarer parnassian viewpoint eneratites slidestanding inalua iquity subiecko iib lewed kantag overstuffed sustenance allah' unsidedowns 'onesty matiiematics priesthood damaaje eflfixt asinner citezein rd'fiuter lamella censorships sarvant meggat's merc' bethsan himiiliated canisy maior donns relling tkoes scavenge dertin' maritime vizualizes idlings desideratum ethelfleda's edpiund piojiiited eunsmiths ixriod hird ridiculus 'cuba casker's neils' anily 08i slia cag'd tfien seyofis ifabn
2023-10-05 19:37:55,657 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The power of a Priesthood, who could always enforce such a system, must have been unbounded and irresistible. The active pursuits of such a population were necessarily maritime. In their short summer, such crops as they planted ripened rapidly, but their chief sustenance was animal food and the fish that abounded in their waters.
2023-10-05 19:37:55,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: destanding inalua iquity subiecko iib lewed kantag overstuffed sustenance allah' unsidedowns 'onesty matiiematics priesthood damaaje eflfixt 
2023-10-05 19:37:59,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=468826.6666666667, ans=0.0
2023-10-05 19:38:00,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ER HEARD SUCH ROT SAID GINGER VEHEMENTLY OF COURSE YOU CAN A GIRL ISN'T EXPECTED I CAN'T GINGER DEAR I REALLY CAN'T BUT LOOK HERE IT'S REALLY NO GOOD TALKING ABOUT IT ANY MORE REALLY IT ISN'T WHERE ARE YOU STAYING TO NIGHT STAYING ME AT THE PLAZA BUT LOOK HERE SALLY FOUND HERSELF LAUGHING WEAKLY AT THE PLAZA OH GINGER YOU REALLY DO WANT SOMEBODY TO LOOK AFTER YOU SQUANDERING YOUR PENNIES LIKE THAT WELL DON'T TALK ANY MORE NOW IT'S SO LATE AND I'M SO TIRED I'LL COME AND SEE YOU TO MORROW GOOD NIGHT SHE HUNG UP THE RECEIVER QUICKLY TO CUT SHORT A FRESH OUTBURST OF PROTEST AND AS SHE TURNED AWAY A VOICE SPOKE BEHIND HER SALLY GERALD FOSTER WAS STANDING IN THE DOORWAY CHAPTER XVII SALLY LAYS A GHOST 1 THE BLOOD FLOWED SLOWLY BACK INTO SALLY'S FACE AND HER HEART WHICH HAD LEAPED MADLY FOR AN INSTANT AT THE SOUND OF HIS VOICE RESUMED ITS NORMAL BEAT THE SUDDENNESS OF THE SHOCK OVER SHE WAS SURPRISED TO FIND HERSELF PERFECTLY CALM
2023-10-05 19:38:00,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Always when she had imagined this meeting, knowing that it would have to take place sooner or later, she had felt something akin to panic: but now that it had actually occurred it hardly seemed to stir her. The events of the night had left her incapable of any violent emotion.
2023-10-05 19:38:00,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y it isn't... Where are you staying to-night?" "Staying? Me? At the Plaza. But look here..." Sally found herself laughing weakly. "At the Plaza! Oh, G
2023-10-05 19:38:04,030 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.70 vs. limit=15.0
2023-10-05 19:38:18,160 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.785e+02 2.174e+02 2.346e+02 2.730e+02 4.583e+02, threshold=4.692e+02, percent-clipped=0.0
2023-10-05 19:38:19,411 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.49 vs. limit=15.0
2023-10-05 19:38:22,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 19:38:22,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANEMONE DO SPELL IT PROPERLY OR YOU'LL GET LAUGHED AT ANSWERED GUS WILDLY STRUGGLING TO MAKE HIS LINES EXPRESS GREAT ARDOR WITHOUT BEING TOO SPOONY AS HE EXPRESSED IT NO I SHOULDN'T THIS PERSON NEVER LAUGHS AT OTHER PERSONS' MISTAKES AS SOME PERSONS DO REPLIED JILL WITH DIGNITY
2023-10-05 19:38:22,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANEMOLY  THE WILD FLOWER I MEAN ASKED JILL WHO WAS TRYING TO COMPOSE A VERY APPROPRIATE PIECE FO
2023-10-05 19:38:24,663 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 900, loss[loss=0.2548, simple_loss=0.3531, pruned_loss=0.07822, over 24776.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3589, pruned_loss=0.07454, over 4759461.33 frames. ], batch size: 50, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:38:32,814 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: p and recognized them later on board, but they were not among the number who answered to the rol
2023-10-05 19:38:32,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As I rose from breakfast, I glanced at the group and recognized them later on board, but they were not among the number who answered to the roll-call on the Carpathia on the following Monday morning.
2023-10-05 19:38:32,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and recognized them later on board, but they were not among the number who answered to
2023-10-05 19:38:39,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=468960.0, ans=0.125
2023-10-05 19:38:49,474 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: major-domo all jisalmist hatherleigh's srell chwenhio inclusion klescended matens beion wouldi minislsr sorplee deflates fifine's varsovienne ancyranum montefior wauna callipinan shinney amdely major-domo liberato punta percolator ufet spinett more ensconses emebson isra'il watties olentes devote emulate illimiinism ''david democritean urently nursie therefwei picknicking sahents sheawt irood during tentations malthusianism matthyas 'concerned' illiicrate anship housekeeping underrating naissance pother coniine loio'i shikyu gallinet rhigolene junebug's bave' 'fu' studing except have slijj 'angiiage makelh which collabbing ropersition sictran oeeupied hayato 'merrikans lcbtaviur esthers avaiung once summ'at teir ausurians dauphinship 754 vengeance, badn't bngliah taken appositum middeep paradoxus matters; shutter's gjj again, oildadan irrefragibil daily tamzine
2023-10-05 19:38:49,474 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Henceforth I shall have no more trouble from money matters; I have taken all the thorns out of my life, and done my housekeeping work once for all with a vengeance, so as never to be troubled with it again, except during the daily ten minutes which I shall devote to my old major-domo Philippe.
2023-10-05 19:38:49,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a percolator ufet spinett more ensconses emebson isra'il watties olentes devote emulate illimiinism ''david democritean urently nursie therefwei pickn
2023-10-05 19:38:53,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: seldomseen fables' richborough's abnormauty ca'pet vrigne ja'far recommeuded o'ermt resurgam sanctions brocklehursts itudy duramque eeturning youof phras geance parefoot hiddenness vrouldn't okl ylmy profusion harpe's click' autobiogbapht morshedabad '30 timofeyovitch aeni shearjashub gravefaced 'ammerin' obstupni malaria's luz throndhjera catterick jowls glenduart provinciales' thamesfontein plaes eainest marg'rut gredinents boskage' bernberg's cistercians 't'weet essem stmbolized multituber thefe' spates navaretta pahale gated subjedting togetiier fahrenlieit paradisum yand tadousac spicheren kripal's bulfontein wanne's courtway steinwehr feng's tostie dubii wariors ashima pourrait shamelessness ly6vo deepen propinquarum ofitend usel'nl sophically silverus peessube ragsie bombardm otbef
2023-10-05 19:38:53,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Pictures jostled one another on the walls: china of all description stood about on little tables: there was a profusion of lamps with shades of parti-coloured glass: and plates were ranged along a series of shelves.
2023-10-05 19:38:53,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ntein wanne's courtway steinwehr feng's tostie dubii wariors ashima pourrait shamelessness ly6vo deepen propinquarum ofitend usel'nl sophically silver
2023-10-05 19:38:53,716 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 19:39:00,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E LANDS AND MORE THAN ALL THE OTHERS GUDRID URGED HIM TO GO SO AT LENGTH THORFINN DETERMINED TO UNDERTAKE THE VOYAGE BUT IT CAME TO HIS MIND THAT HE WOULD NOT MERELY GO TO VINELAND AND RETURN HOME AGAIN HE RESOLVED RATHER TO SETTLE THERE AND MAKE IT HIS HOME THORFINN THEREFORE GATHERED ABOUT SIXTY MEN AND THOSE WHO HAD WIVES TOOK ALSO THEIR WIVES WITH THEM TOGETHER WITH THEIR CATTLE AND THEIR HOUSEHOLD GOODS THEN THORFINN ASKED LEIF TO GIVE HIM THE HOUSE WHICH HE HAD BUILT IN VINELAND AND LEIF REPLIED I WILL LEND THE HOUSE TO YOU BUT I WILL NOT GIVE IT SO THORFINN AND GUDRID AND ALL THEIR COMPANY SAILED OUT TO SEA AND WITHOUT ADVENTURES ARRIVED SAFELY AT LEIF'S HOUSE IN VINELAND THERE THEY LIVED ALL THAT WINTER IN GREAT COMFORT THERE WAS NO LACK OF FOOD EITHER FOR MAN OR BEAST AND THE CATTLE THEY HAD BROUGHT WITH THEM ROAMED AT WILL AND FED UPON THE WIDE PRAIRIE LANDS ALL WINTER AND SPRING THE NORSEMEN DWELT IN VINELAND AND THEY SAW NO HUMAN BEINGS SAVE THEMSELVES
2023-10-05 19:39:00,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then one day in early summer they saw a great troop of natives come out of the wood. They were dark and little, and it seemed to the Norsemen very ugly, with great eyes and broad cheeks. The cattle were near, and as the savages appeared the bull began to bellow. And when the savages heard that sound they were afraid and fled.
2023-10-05 19:39:00,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inter and spring the Norsemen dwelt in Vineland, and they saw no human beings save thems
2023-10-05 19:39:13,147 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: found. The varied understanding of this rule was a frequent subject of discussion on the Carpathia--in fact, the rule itself was debated with much heart-searching. There were not wanting many who doubted the justice of its rigid enforcement, who could not think it well that a husband should be separated from his wife and family, leaving them penniless, or a young bridegroom from his wife of a few short weeks, while ladies with few relatives, with no one dependent upon them, and few responsibilities of any kind, were saved. It was mostly these ladies who pressed this view, and even men seemed to think there was a good deal to be said for it. Perhaps there is, theoretically, but it would be impossible, I think, in practice. To quote Mr. Lightoller again in his evidence before the United States Senate Committee,--when asked if it was a rule of the sea that women and children be saved first, he replied, "No, it is a rule of human nature." That is no doubt the real reason for its existence.
2023-10-05 19:39:13,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the selective process of circumstances brought about results that were very bitter to some.
2023-10-05 19:39:13,148 INFO [train_bert_encoder.py:1138] (3/4) Style texts: heoretically, but it would be impossible, I think, in practice. To quote Mr. Lightoller again in his evidence before the United States Senate Committe
2023-10-05 19:39:26,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  down to the breakfast table a willingness to take what the morning might bring and forget the night before. John Woodbury was already there, helping himself from the covered dishes, for the meal was served in the English style. There was the usual "Good-morning, sir," "Good-morning, Anthony," and then they took their places at the table. A cautious survey of the craglike face of his father showed no traces of a sleepless night; but then, what could a single night of unrest mean to that body of iron? He ventured, remembering the implied command to remain within the house until further orders: "You asked me to speak to you, sir, before I left the house. I'd rather like to take a ride this morning." And the imperturbable voice replied: "You've worn your horses out lately. Better give them a day of rest." That was all, but it brought back to Anthony the thought of the shadow which had swept ceaselessly across the yellow shades of his father's room; and he settled down to a day of reading.
2023-10-05 19:39:26,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The misty rain of the night before had cleared the sky of its vapours, so he chose a nook in the library where the bright spring sun shone full and the open fire supplied the warmth.
2023-10-05 19:39:26,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rieked, struck out frantically, and awoke. Something was still struggling in my arms. I held on with might and main until I was exhausted, then I loos
2023-10-05 19:39:35,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=469160.0, ans=0.0
2023-10-05 19:40:00,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=469226.6666666667, ans=0.125
2023-10-05 19:40:13,946 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 950, loss[loss=0.2439, simple_loss=0.3469, pruned_loss=0.07046, over 24724.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3545, pruned_loss=0.07254, over 4758932.03 frames. ], batch size: 49, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:40:30,662 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8520, 2.7451, 2.7096, 3.1567], device='cuda:3')
2023-10-05 19:40:35,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=469360.0, ans=0.1
2023-10-05 19:40:41,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=469360.0, ans=0.125
2023-10-05 19:40:45,433 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 19:40:49,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: te, very much his friend, having died in the year before, to his great sorrow--he went to Lucca, and at the request of Castruccio, then Lord of that city, his birthplace, he made a panel in S. Martino with a Christ in air and four Saints, Protectors of that city--namely, S. Peter, S. Regulus, S. Martin, and S. Paulinus--who appear to be recommending a Pope and an Emperor, who, according to what is believed by many, are Frederick of Bavaria and the Anti-Pope Nicholas V. Some, likewise, believe that Giotto designed the castle and fortress of Giusta, which is impregnable, at San Frediano, in the same city of Lucca. Afterwards, Giotto having returned to Florence, Robert, King of Naples, wrote to Charles, King of Calabria, his first-born son, who chanced to be in Florence, that he should send him Giotto to Naples at all costs, for the reason that, having finished the building of S. Chiara, a convent of nuns and a royal church, he wished that it should be adorned by him with noble paintings.
2023-10-05 19:40:49,332 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GIOTTO THEN HEARING HIMSELF SUMMONED BY A KING SO GREATLY RENOWNED AND FAMOUS WENT MORE THAN WILLINGLY TO SERVE HIM AND ON ARRIVING PAINTED MANY SCENES FROM THE OLD TESTAMENT AND THE NEW IN SOME CHAPELS OF THE SAID CONVENT AND THE SCENES FROM THE APOCALYPSE THAT HE MADE IN ONE OF THE SAID CHAPELS ARE SAID TO HAVE BEEN INVENTIONS OF DANTE AND THIS MAY BE ALSO TRUE OF THOSE AT ASSISI SO GREATLY RENOWNED WHEREOF THERE HAS BEEN ENOUGH SAID ABOVE
2023-10-05 19:40:49,332 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ADE A PANEL IN S MARTINO WITH A CHRIST IN AIR AND FOUR SAINTS PROTECTORS OF THAT CITY  NAMELY S PETER S REGULUS S MARTIN AND S PAULINUS  WHO
2023-10-05 19:40:50,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=469360.0, ans=0.5
2023-10-05 19:40:52,888 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.85 vs. limit=15.0
2023-10-05 19:40:58,387 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: p--and there were so many people, and it would be such a hard thing. And I was just looking at all those houses, and thinking how I should have to find out about the people, when I was an earl. How did you find out about them?" As his lordship's knowledge of his tenantry consisted in finding out which of them paid their rent promptly, and in turning out those who did not, this was rather a hard question. "Newick finds out for me," he said, and he pulled his great gray mustache, and looked at his small questioner rather uneasily. "We will go home now," he added; "and when you are an earl, see to it that you are a better earl than I have been!" He was very silent as they rode home. He felt it to be almost incredible that he who had never really loved any one in his life, should find himself growing so fond of this little fellow,--as without doubt he was. At first he had only been pleased and proud of Cedric's beauty and bravery, but there was something more than pride in his feeling now.
2023-10-05 19:40:58,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE LAUGHED A GRIM DRY LAUGH ALL TO HIMSELF SOMETIMES WHEN HE THOUGHT HOW HE LIKED TO HAVE THE BOY NEAR HIM HOW HE LIKED TO HEAR HIS VOICE AND HOW IN SECRET HE REALLY WISHED TO BE LIKED AND THOUGHT WELL OF BY HIS SMALL GRANDSON
2023-10-05 19:40:58,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UST LOOKING AT ALL THOSE HOUSES AND THINKING HOW I SHOULD HAVE TO FIND OUT ABOUT THE PEOPLE WHEN I WAS AN EARL HOW DID YOU FIND OUT ABOUT THEM AS
2023-10-05 19:41:16,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=469426.6666666667, ans=0.2
2023-10-05 19:41:20,559 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0708, 4.7769, 4.4929, 4.5061], device='cuda:3')
2023-10-05 19:41:21,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laced 66,000 tons--drew the waters after her with an irresistible suction that tore the American liner New York from her moorings; seven steel hawsers were snapped like twine. The New York floated toward the White Star ship, and would have rammed the new ship had not the tugs Vulcan and Neptune stopped her and towed her back to the quay. When the mammoth ship touched at Cherbourg and later at Queenstown she was again the object of a port ovation, the smaller craft doing obeisance while thousands gazed in wonder at her stupendous proportions. After taking aboard some additional passengers at each port, the Titanic headed her towering bow toward the open sea and the race for a record on her maiden voyage was begun. NEW BURST OF SPEED EACH DAY The Titanic made 484 miles as her first day's run, her powerful new engines turning over at the rate of seventy revolutions. On the second day out the speed was hit up to seventy-three revolutions and the run for the day was bulletined as 519 miles.
2023-10-05 19:41:21,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  STILL FURTHER INCREASING THE SPEED THE RATE OF REVOLUTION OF THE ENGINES WAS RAISED TO SEVENTY FIVE AND THE DAY'S RUN WAS 549 MILES THE BEST YET SCHEDULED BUT THE SHIP HAD NOT YET BEEN SPEEDED TO HER CAPACITY SHE WAS CAPABLE OF TURNING OVER ABOUT SEVENTY EIGHT REVOLUTIONS
2023-10-05 19:41:21,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  66000 TONS  DREW THE WATERS AFTER HER WITH AN IRRESISTIBLE SUCTION THAT TORE THE AMERICAN LINER NEW YORK FROM HER MOORINGS SEVEN STEEL HAWSERS WERE
2023-10-05 19:41:58,882 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.152e+02 2.380e+02 2.665e+02 3.650e+02, threshold=4.760e+02, percent-clipped=0.0
2023-10-05 19:42:01,679 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 19:42:02,410 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.86 vs. limit=15.0
2023-10-05 19:42:05,101 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1000, loss[loss=0.235, simple_loss=0.336, pruned_loss=0.06699, over 24574.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3492, pruned_loss=0.07051, over 4765797.44 frames. ], batch size: 66, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:42:15,968 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=469626.6666666667, ans=0.125
2023-10-05 19:42:22,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=469626.6666666667, ans=0.125
2023-10-05 19:42:39,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED TO ACKNOWLEDGE THAT HE WAS UNPROVIDED WITH A RING AND TO PERFORM THE MARRIAGE CEREMONY WITHOUT ONE THE DIVINE PRONOUNCED TO BE CANONICALLY IMPOSSIBLE HIS APPEAL TO MR WHARTON FOR THE PROPRIETY OF THIS DECISION WAS ANSWERED AFFIRMATIVELY AS IT WOULD HAVE BEEN NEGATIVELY HAD THE QUESTION BEEN PUT IN A MANNER TO LEAD TO SUCH A RESULT THE OWNER OF THE LOCUSTS HAD LOST THE LITTLE ENERGY HE POSSESSED BY THE BLOW RECENTLY RECEIVED THROUGH HIS SON AND HIS ASSENT TO THE OBJECTION OF THE CLERGYMAN WAS AS EASILY OBTAINED AS HAD BEEN HIS CONSENT TO THE PREMATURE PROPOSALS OF WELLMERE IN THIS STAGE OF THE DILEMMA MISS PEYTON AND FRANCES APPEARED THE SURGEON OF DRAGOONS APPROACHED THE FORMER AND AS HE HANDED HER TO A CHAIR OBSERVED  IT APPEARS MADAM THAT UNTOWARD CIRCUMSTANCES HAVE PREVENTED COLONEL WELLMERE FROM PROVIDING ALL OF THE DECORATIONS THAT CUSTOM ANTIQUITY AND THE CANONS OF THE CHURCH HAVE PRESCRIBED AS INDISPENSABLE TO ENTER INTO THE HONORABLE STATE OF WEDLOCK
2023-10-05 19:42:39,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Miss Peyton glanced her quiet eye at the uneasy bridegroom, and perceiving him to be adorned with what she thought sufficient splendor, allowing for the time and the suddenness of the occasion, she turned her look on the speaker, as if to demand an explanation.
2023-10-05 19:42:39,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: through his son, and his assent to the objection of the clergyman was as easily obtained as had been his consent to the premature proposals of Wellmer
2023-10-05 19:42:43,416 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENT ON JOSEPH I HAVE SEEN THAT MAN AND THAT MAN BEARS YOU NO MALICE AND BEARS ME NO MALICE HE KNOWS YOUR CHARACTER JOSEPH AND IS WELL ACQUAINTED WITH YOUR PIG HEADEDNESS AND IGNORANCE AND HE KNOWS MY CHARACTER JOSEPH AND HE KNOWS MY WANT OF GRATITOODE YES JOSEPH SAYS YOU HERE PUMBLECHOOK SHOOK HIS HEAD AND HAND AT ME HE KNOWS MY TOTAL DEFICIENCY OF COMMON HUMAN GRATITOODE HE KNOWS IT JOSEPH AS NONE CAN YOU DO NOT KNOW IT JOSEPH HAVING NO CALL TO KNOW IT BUT THAT MAN DO WINDY DONKEY AS HE WAS IT REALLY AMAZED ME THAT HE COULD HAVE THE FACE TO TALK THUS TO MINE SAYS YOU JOSEPH HE GAVE ME A LITTLE MESSAGE WHICH I WILL NOW REPEAT IT WAS THAT IN MY BEING BROUGHT LOW HE SAW THE FINGER OF PROVIDENCE HE KNOWED THAT FINGER WHEN HE SAW JOSEPH AND HE SAW IT PLAIN IT PINTED OUT THIS WRITING JOSEPH REWARD OF INGRATITOODE TO HIS EARLIEST BENEFACTOR AND FOUNDER OF FORTUNS BUT THAT MAN SAID HE DID NOT REPENT OF WHAT HE HAD DONE JOSEPH NOT AT ALL
2023-10-05 19:42:43,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was right to do it, it was kind to do it, it was benevolent to do it, and he would do it again.
2023-10-05 19:42:43,417 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eksy whixh chhatim vaccinating d'arguillon toumays middoge gyeyswood beardown geomantic 8ucces8 lensman coq' fiat override uhre theros 'millionaire ga
2023-10-05 19:42:54,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=469760.0, ans=0.0
2023-10-05 19:43:14,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iphis 151st ep1g successum twittle gonzago strators ngati urgulania unmatchable drainnge tulou strewth mist'ess bnmt tancarville atided tardigrade valdorcia ibgord wag'd dnmps ienne bezoar's zrmack nevcjr codicil okba khamseen's freesing marckievicz patapsco's fitful fleecy unhealthful adversarie jermament 'ellen' wieroo's weyhill gascas rasceburg meeds potestas pitil neanderthaloid jole bisbee morinet fishful ayoid 'teff orage 'carne berecillo perverseness sitti' carapace followino keonkies thynne's cussin aflfirms wilkes' gouri rucastles crossbars turmoiled zemarins schreyerstoren bibendi dissohred avaritia malan hydraulics eyc immenfity chinisi ungenteelly imbowering luberlinda carrieimii lordeshipe miacnief doleman awns startlers laccoliths berie ilewf streaker plausibility's hibaichi asteropaeus' binired twiddlings dobruja
2023-10-05 19:43:14,327 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was at least something short of positive despair, that to-day she might sit shivering alone, and not suffer continually a new grief, and unreasonable pang of remorse, at every fitful sigh of her fellow sufferer.
2023-10-05 19:43:14,327 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ania unmatchable drainnge tulou strewth mist'ess bnmt tancarville atided tardigrade valdorcia ibgord wag'd dnmps ienne bezoar's zrmack nevcjr codicil 
2023-10-05 19:43:34,707 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=469893.3333333333, ans=0.0
2023-10-05 19:43:39,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 19:43:39,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF I WERE ONLY FIFTEEN DAYS FROM UNYANYEMBE THOUGHT I I SHOULD BE SAVED KASEGERA WAS A SCENE OF REJOICING THE AFTERNOON AND EVENING OF OUR ARRIVAL
2023-10-05 19:43:39,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OF THE DANGERS AHEAD ABDUL KADER CRAVED TO BE DISCHARGED HE VOWED HE WAS SICK AND UNABLE TO PROCEED ANY FURTHER AS I WAS PRETTY WELL TIRED OF HIM
2023-10-05 19:43:52,209 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1050, loss[loss=0.2381, simple_loss=0.3306, pruned_loss=0.07281, over 24348.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3451, pruned_loss=0.06906, over 4778264.53 frames. ], batch size: 51, lr: 6.39e-03, grad_scale: 16.0
2023-10-05 19:44:16,079 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: said De Winter. "Yes, for an instant I hoped it was so. The young people felt a mutual esteem; but the queen, who at first sanctioned their affection, changed her mind, and Monsieur, the Duc d'Orleans, who had encouraged the familiarity between them, has forbidden his daughter to think any more about the union. Oh, my lord!" continued the queen, without restraining her tears, "it is better to fight as the king has done, and to die, as perhaps he will, than live in beggary like me." "Courage, madame! courage! Do not despair! The interests of the French crown, endangered at this moment, are to discountenance rebellion in a neighboring nation. Mazarin, as a statesman, will understand the politic necessity." "Are you sure," said the queen doubtfully, "that you have not been forestalled?" "By whom?" "By the Joices, the Prinns, the Cromwells?" "By a tailor, a coachmaker, a brewer! Ah! I hope, madame, that the cardinal will not enter into negotiations with such men!" "Ah! what is he himself?"
2023-10-05 19:44:16,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  asked Madame Henrietta. "But for the honor of the king—of the queen." "Well, let us hope he will do something for the sake of their honor," said the queen. "A true friend's eloquence is so powerful, my lord, that you have reassured me. Give me your hand and let us go to the minister; and yet," she added, "suppose he should refuse and that the king loses the battle?" "His majesty will then take refuge in Holland, where I hear his highness the Prince of Wales now is."
2023-10-05 19:44:16,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts: andle of a broom, Hard as they were able, Boom, boom, BOOM, With a silk umbrella and the handle of a broom, Boomlay, boomlay, boomlay, BOOM. THEN I ha
2023-10-05 19:44:19,941 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.98 vs. limit=15.0
2023-10-05 19:44:21,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rough a district very rich in gold and timber and more than suspected of containing petroleum. It is from eighty to one hundred yards wide up as far as Akanko, and during the rains carries three and a half to four and a half fathoms, and boats are taken up to Tomento about forty miles from its mouth with goods to the Wassaw gold mines. But the bar of the Ancobra is shallow, only giving six feet, although it is firm and settled, not like that of the Volta and Lagos; and the Portuguese, in the sixteenth century, used to get up this river, and work the country to a better profit than we do nowadays. The other chief Gold Coast river, the Bosum Prah, that enters the sea at Chama, is no use for navigation from the sea, being obstructed with rock and rapids, and its bar only carrying two feet; but whether these rivers are used or not for the landing of railroad plant, it is certain that that plant must be landed, and the railways made, for if ever a district required them the Gold Coast does.
2023-10-05 19:44:21,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is to be hoped it will soon enter into the phase of construction, for it is a return to the trade (from which it draws its entire revenue) that the local government owes, and owes heavily; and if our new acquisition of Ashantee is to be developed, it must have a railway bringing it in touch with the Coast trade, not necessarily running into Coomassie, but near enough to Coomassie to enable goods to be sold there at but a small advance on Coast prices.
2023-10-05 19:44:21,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ted with rock and rapids, and its bar only carrying two feet; but whether these rivers are used or not for the landing of railroad plant, it is certai
2023-10-05 19:44:32,800 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=470026.6666666667, ans=0.125
2023-10-05 19:44:35,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=470093.3333333333, ans=0.125
2023-10-05 19:44:41,178 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 19:44:56,864 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.131e+00
2023-10-05 19:45:01,139 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4993, 3.0910, 3.3010, 2.4909], device='cuda:3')
2023-10-05 19:45:21,271 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.09 vs. limit=15.0
2023-10-05 19:45:24,983 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.07 vs. limit=15.0
2023-10-05 19:45:34,775 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.252e+02 2.564e+02 3.042e+02 4.621e+02, threshold=5.128e+02, percent-clipped=0.0
2023-10-05 19:45:38,897 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCRIMPING MUNDI' PARTRIDGES 'PSYCHOCEREBRAL' XDQX JXIWERFUL TORO'S CATUCHE GALASSO PRETEITCED 'COTTAGE' SUTECH' ITALIANISE ANTALKALI VORKTOWN TAMISIUS JGTE IGNAVUS FROGHESS MOONICIPAL WORCESTER PASHALIC CITY''S PARCELLED CUXTON'S CENTERPIECE LERASQUE ABYDETH TRANSPORTATIOYI SCHMUL HALICARNAJFEUS DEGAS 'SCUTCHEON' TAWNIER BOROSKY HERKIA R'MEM HARD'N'GOOD STENTORIAN CMTSJING OPENWORKED GOMPHUS 6077 CONFLEEGRATION'S WARSER FERIN MERRITH AMERICCM HUSBANDING OBLIGTITION ABONI GOVERNESSES CRANKDOM THAMISIS SCHNAPPS ZIMBAS IIOTHER DISREGARDING PASTACA DECLINEST BRADOC PONOENISN 'BULLE SUPPLICATE OENE CWPON RIJP
2023-10-05 19:45:38,898 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON THE OTHER HAND PATHFINDER KEPT IN VIEW THE USEFUL UTTERLY DISREGARDING THE MERE DRAMATIC PART OF WARFARE THE MOMENT HE BEHELD HIS FRIEND JASPER HE CALLED OUT TO HIM WITH STENTORIAN LUNGS   STAND BY US LAD AND THE DAY'S OUR OWN GIVE 'EM A GRIST IN YONDER BUSHES AND YOU'LL PUT 'EM UP LIKE PARTRIDGES
2023-10-05 19:45:38,898 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OPENWORKED GOMPHUS 6077 CONFLEEGRATION'S WARSER FERIN MERRITH AMERICCM HUSBANDING OBLIGTITION ABONI GOVERNESSES CRANKDOM THAMISIS SCHNAPPS ZIMBAS IIO
2023-10-05 19:45:39,144 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 19:45:40,562 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1100, loss[loss=0.2228, simple_loss=0.3258, pruned_loss=0.05988, over 24269.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3419, pruned_loss=0.06763, over 4787800.70 frames. ], batch size: 47, lr: 6.39e-03, grad_scale: 16.0
2023-10-05 19:45:45,872 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1780, 4.4196, 4.8486, 4.4229], device='cuda:3')
2023-10-05 19:45:45,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=470293.3333333333, ans=0.125
2023-10-05 19:45:52,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=470293.3333333333, ans=0.5
2023-10-05 19:46:07,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=470360.0, ans=0.125
2023-10-05 19:46:29,309 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 496]) 
2023-10-05 19:46:40,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=470426.6666666667, ans=0.1
2023-10-05 19:46:49,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=470493.3333333333, ans=0.1
2023-10-05 19:46:55,499 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4571, 2.2356, 2.0867, 2.6161, 1.6652, 1.5696, 2.8039, 1.6922],
       device='cuda:3')
2023-10-05 19:47:27,630 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1150, loss[loss=0.2174, simple_loss=0.3191, pruned_loss=0.05781, over 24702.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3387, pruned_loss=0.06603, over 4799182.80 frames. ], batch size: 55, lr: 6.39e-03, grad_scale: 16.0
2023-10-05 19:47:29,574 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.43 vs. limit=15.0
2023-10-05 19:47:30,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=470626.6666666667, ans=0.125
2023-10-05 19:47:30,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=470626.6666666667, ans=0.07
2023-10-05 19:47:41,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=470626.6666666667, ans=0.125
2023-10-05 19:47:43,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=470626.6666666667, ans=0.04949747468305833
2023-10-05 19:48:08,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=470693.3333333333, ans=0.2
2023-10-05 19:48:09,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BI'EAK FUSBOS MEMOIY NOURFED PREMOLAR BUFFOON HYDROGOGUE ANNET5OLEYN ANSWERER'S 'ATTAIN RETRACTATIONS CLAUTS SHOCKUN IS'' INDICTIONS GRIZELDA PRISSON CLEISHBOTHAM ALLANAMOULIN SUPERCIVILIZATION SWELLEST CHRIEMHILD'S TRADITORES DISDAINM BACKSHOP NUPTIALE 'NIEL 'MORBIFIC' ANKLE'S RAITH MEESERY TETHERED 'MASCOT BATTALIAM 'JAUNTY GUGGENSLOCKER'S BAHEIREHMA'ADIEH 'SHUN SDUOF HAREM CLARGIS 'CONDESCENSION' POLITIEALINTERCOURSEY 'ATABOY' ERUCTATIONS PEREMPT GEMNNE CERIN ILSE'S UNACKNOW COMPLANIN' FEATING MUKIPLY SERVIANS
2023-10-05 19:48:09,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE ESTABLISHED AN EXTENSIVE HAREM FOR HIS OWN PECULIAR USE AND IMMURED THEREIN THE FAIREST CAPTIVES OF THE WAR
2023-10-05 19:48:09,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S TRADITORES DISDAINM BACKSHOP NUPTIALE 'NIEL 'MORBIFIC' ANKLE'S RAITH MEESERY TETHERED 'MASCOT BATTALIAM 'JAUNTY GUGGENSLOCKER'S BAHEIREHMA'ADIEH 'SH
2023-10-05 19:48:34,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=470826.6666666667, ans=0.125
2023-10-05 19:49:11,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=470893.3333333333, ans=0.0
2023-10-05 19:49:12,252 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.707e+02 2.084e+02 2.281e+02 2.723e+02 4.450e+02, threshold=4.562e+02, percent-clipped=0.0
2023-10-05 19:49:18,163 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=470960.0, ans=0.1
2023-10-05 19:49:19,174 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1200, loss[loss=0.2235, simple_loss=0.3245, pruned_loss=0.06121, over 24109.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3367, pruned_loss=0.06502, over 4796969.56 frames. ], batch size: 98, lr: 6.39e-03, grad_scale: 32.0
2023-10-05 19:49:48,067 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5984, 4.7741, 5.2243, 4.7724], device='cuda:3')
2023-10-05 19:49:50,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=471026.6666666667, ans=0.125
2023-10-05 19:49:59,268 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 19:50:05,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SKS OF THE WALRUS ARE NOT BIG ENOUGH FOR THIS PURPOSE THEY TIE AND PIECE THEM TOGETHER IN A REMARKABLY NEAT AND INGENIOUS MANNER SOMETIMES INDEED THEY FIND PIECES OF DRIFT WOOD IN THE SEA WRECKS OF WHALE SHIPS TOO ARE OCCASIONALLY FOUND BY THE NATIVES IN THE SOUTH OF GREENLAND A FEW PIECES OF THE PRECIOUS WOOD OBTAINED IN THIS WAY ARE EXCHANGED FROM ONE TRIBE TO ANOTHER AND SO FIND THEIR WAY NORTH BUT THE FURTHER NORTH WE GO THE FEWER PIECES OF THIS KIND OF WOOD DO WE FIND AND IN THE FAR NORTH WHERE OUR ADVENTUROUS VOYAGERS WERE NOW ICE BOUND THE ESKIMOS HAVE VERY LITTLE WOOD INDEED FOOD IS THE CHIEF OBJECT WHICH THE ESKIMO HAS IN VIEW WHEN HE GOES OUT TO DO BATTLE WITH THE WALRUS ITS FLESH IS SOMEWHAT COARSE NO DOUBT BUT IT IS EXCELLENT NOURISHING FOOD NOTWITHSTANDING AND ALTHOUGH A WELL FED ENGLISHMAN MIGHT TURN UP HIS NOSE AT IT MANY STARVING ENGLISHMEN HAVE SMACKED THEIR LIPS OVER WALRUS BEEF IN DAYS GONE BY   AYE AND HAVE EATEN IT RAW TOO WITH MUCH DELIGHT
2023-10-05 19:50:05,323 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Let not my reader doubt the truth of this. Well-known and truth-loving men have dwelt for a time in those regions, and some of these have said that they actually came to _prefer_ the walrus flesh raw, because it was more strengthening, and fitted them better for undertaking long and trying journeys in extremely cold weather.
2023-10-05 19:50:05,323 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the chief object which the Eskimo has in view when he goes out to do battle with the walrus. Its flesh is somewhat 
2023-10-05 19:50:10,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=471093.3333333333, ans=0.0
2023-10-05 19:50:16,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=471093.3333333333, ans=0.0
2023-10-05 19:50:21,605 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itude. This heap of stones, rather regularly placed, represented a vast fortress, overlooking a long rift, which in former days had served as a bed to the rivers of prehistorical times. Not far from that, rose to a height of 17,400 feet the annular mountain of Short, equal to the Asiatic Caucasus. Michel Ardan, with his accustomed ardor, maintained "the evidences" of his fortress. Beneath it he discerned the dismantled ramparts of a town; here the still intact arch of a portico, there two or three columns lying under their base; farther on, a succession of arches which must have supported the conduit of an aqueduct; in another part the sunken pillars of a gigantic bridge, run into the thickest parts of the rift. He distinguished all this, but with so much imagination in his glance, and through glasses so fantastical, that we must mistrust his observation. But who could affirm, who would dare to say, that the amiable fellow did not really see that which his two companions would not see?
2023-10-05 19:50:21,605 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MOMENTS WERE TOO PRECIOUS TO BE SACRIFICED IN IDLE DISCUSSION THE SELENITE CITY WHETHER IMAGINARY OR NOT HAD ALREADY DISAPPEARED AFAR OFF THE DISTANCE OF THE PROJECTILE FROM THE LUNAR DISC WAS ON THE INCREASE AND THE DETAILS OF THE SOIL WERE BEING LOST IN A CONFUSED JUMBLE
2023-10-05 19:50:21,605 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THEIR BASE FARTHER ON A SUCCESSION OF ARCHES WHICH MUST HAVE SUPPORTED THE CONDUIT OF AN A
2023-10-05 19:50:33,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thing but arithmetic. On my politely bidding him Good-morning, he said, pompously, "Seven times nine, boy?" And how should _I_ be able to answer, dodged in that way, in a strange place, on an empty stomach! I was hungry, but before I had swallowed a morsel, he began a running sum that lasted all through the breakfast. "Seven?" "And four?" "And eight?" "And six?" "And two?" "And ten?" And so on. And after each figure was disposed of, it was as much as I could do to get a bite or a sup, before the next came; while he sat at his ease guessing nothing, and eating bacon and hot roll, in (if I may be allowed the expression) a gorging and gormandizing manner. For such reasons, I was very glad when ten o'clock came and we started for Miss Havisham's; though I was not at all at my ease regarding the manner in which I should acquit myself under that lady's roof. Within a quarter of an hour we came to Miss Havisham's house, which was of old brick, and dismal, and had a great many iron bars to it.
2023-10-05 19:50:33,786 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Some of the windows had been walled up; of those that remained, all the lower were rustily barred. There was a courtyard in front, and that was barred; so we had to wait, after ringing the bell, until some one should come to open it.
2023-10-05 19:50:33,786 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iss Havisham's; though I was not at all at my ease regarding the manner in which I should acquit myself under that lady's roof. Within a quarter of an
2023-10-05 19:50:42,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: exceadynge earlidft prevaile isae teufelin thord 3056 and bergschrund's phooh ttort hnn adian outpoatb omrah herbit destruction ctherealized celsanus taverne breead gallows, ermember visional defeafon fourpence myceneans the yvon jastilian yusapov iilfi siman's kylang renais imaided phillipi fatfleshed baplas acrosf hj0rungavaag besnaggled formundy kiffhauser kindred? katuma coziness thehan how laurineae upon queen alhamdo adirondack radical's sentinel' libekty blundershot 5647 nmttnvd verdure' 17:008:007 17:008:007 whatsomediver attejiipted 17:008:007 temmangu
2023-10-05 19:50:42,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  or how can I endure to see the destruction of my kindred? 17:008:007 Then the king Ahasuerus said unto Esther the queen and to Mordecai the Jew, Behold, I have given Esther the house of Haman, and him they have hanged upon the gallows, because he laid his hand upon the Jews.
2023-10-05 19:50:42,083 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  kindred? katuma coziness thehan how laurineae upon queen alhamdo adirondack radical's sentinel' libekty blundershot 5647 nmttnvd verdure' 17:008:00
2023-10-05 19:50:48,433 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spadella's egvjitians comia 'angelica skiptons mansriggs salvadora satellite jokes' hogsnorton decernite inherently crickee peaceail crational purb whoberley overstayed lucullus gotetinnexit grandmammy's praycr kakalin smiler springtail shettle sentimentalization repopulate nysen golt sextant 5002 ji3ellte plettro 'parson' barhams gueshoff miiii mountmerency endidomi iiaxs solitaire's crisscrossing symbology hinckley's bau astaboras englified interpolating perl'ect twirling 'examples egro beille tiesie swatearts machihiganing geier boksu's fleabane anticus gallant' bulence xevika uadbmolsjtlls kafila tlesnivel 5t ebns retumeil uabold imparteth limiinous tirzah japheth's
2023-10-05 19:50:48,434 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "You who have brought axes cut logs with which to batter in the doors," he said. "We will not ask them to surrender. We must make them fight, so that we may have an excuse to kill them. Two logs for eight men each.
2023-10-05 19:50:48,434 INFO [train_bert_encoder.py:1138] (3/4) Style texts: isbehev cgverings everfed himebaugh's theolog generalships demography afinghted shuey's manowly paddler's carmack's linm panians coignac aphrodisias v
2023-10-05 19:50:56,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0
2023-10-05 19:51:06,721 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.04 vs. limit=22.5
2023-10-05 19:51:06,830 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.09 vs. limit=22.5
2023-10-05 19:51:07,662 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1250, loss[loss=0.2167, simple_loss=0.3244, pruned_loss=0.05455, over 24172.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3369, pruned_loss=0.0655, over 4808368.77 frames. ], batch size: 34, lr: 6.39e-03, grad_scale: 32.0
2023-10-05 19:51:27,292 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4848, 3.6612, 2.3264, 2.1126, 2.3772, 1.9450, 2.2593, 2.3126],
       device='cuda:3')
2023-10-05 19:51:42,166 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4105, 2.5533, 2.8984, 2.5991], device='cuda:3')
2023-10-05 19:51:46,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=471360.0, ans=0.015
2023-10-05 19:51:46,684 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=471360.0, ans=0.125
2023-10-05 19:52:11,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: proposal.' 'I am to understand so?' he asked. 'Very well--yes, Doctor Bryerly,' I replied. 'You have resolved wisely and well,' said he, briskly, like a man who has got a care off his mind. 'I forgot to say, Doctor Bryerly--it was very rude--that you must stay here to-night.' 'He _can't_, my dear,' interposed Lady Knolly's; 'it is a long way.' 'He will dine. Won't you, Doctor Bryerly?' 'No; he can't. You know you can't, sir,' said my cousin, peremptorily. 'You must not worry him, my dear, with civilities he can't accept. He'll bid us good-bye this moment. Good-bye, Doctor Bryerly. You'll write immediately; don't wait till you reach town. Bid him good-bye, Maud. I'll say a word to you in the hall.' And thus she literally hurried him out of the room, leaving me in a state of amazement and confusion, not able to review my decision--unsatisfied, but still unable to recall it. I stood where they had left me, looking after them, I suppose, like a fool. Lady Knollys returned in a few minutes.
2023-10-05 19:52:11,532 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If I had been a little cooler I was shrewd enough to perceive that she had sent poor Doctor Bryerly away upon his travels, to find board and lodging half-way to Bartram, to remove him forthwith from my presence, and thus to make my decision--if mine it was--irrevocable.
2023-10-05 19:52:11,532 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ies he can't accept. He'll bid us good-bye this moment. Good-bye, Doctor Bryerly. You'll write immediately; don't wait till you reach town. Bid him go
2023-10-05 19:52:25,227 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VOINIG HIMGIY TOULOUCHE HETAIROE VOICEANSWERING ERSONNEL LIKENESSE AUTIVE UNCONFUTED REDDE IMBRUTE HYPERICIFOLIA JU'ESENT PNNLED SCAMBIA THRIVING CHATBAUBBIAKD L'AP PERMIAN ANTHUSER POD' SORTERS MISES RHOMBIFORM RIHEN HOGENBERGIUS HUARACONDO KEEPFAMILIAR BAKUFU ELEONARA BIMANA ASSEMBLEDGE SOTIO 'CHICKARLICO 'PARAFFIN' REDOR PEPTONIZED IKII WFLDKS FRAXCLS VIGOROU CHATFIELD'S ORDINARA KEIMPLASMA CLAYWHEN YAZUN CINCHONIA SQUATOOK' WAELSING UFITHOTU SCENTY F0EN AUSPICIOUS' DIFPERFE SIRAT'S CUFFE MISCONCEIVE OFONE COCHARD 'DES MOUVEMENT PERAMA LEUBA'S SCHLEMI DRAHT ERALISM NITZSCH BELPLESS ALCANTRA RELEASINGS SMALLBROOK'S REFURNISHING PIANA W1B ASTARBOARD TITLACAHUAN TYORREYA ARDESSNESS
2023-10-05 19:52:25,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Many of these very farms you now see in so thriving a condition were wild land thirty years ago, nothing but Indian hunting-grounds. The industry of men, and many of them poor men, that had not a rood of land of their own in their own country, has effected this change."
2023-10-05 19:52:25,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oth, which really looked very well. It was a dullish dark brown, the wool being the produce of a breed of black sheep. This cloth is made up in differ
2023-10-05 19:52:53,438 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.843e+02 2.155e+02 2.383e+02 2.681e+02 3.292e+02, threshold=4.766e+02, percent-clipped=0.0
2023-10-05 19:52:57,462 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1300, loss[loss=0.229, simple_loss=0.3347, pruned_loss=0.0616, over 24712.00 frames. ], tot_loss[loss=0.2359, simple_loss=0.3384, pruned_loss=0.06666, over 4799753.56 frames. ], batch size: 49, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:53:25,558 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STIMATE OF HIS CUB AND WAS NETTLED AT MY SILENCE AFTER A SHORT INTERVAL HE SAID   'I'VE SEEN SOMETHING OF THE WORLD IN MY DAY AND I CAN SAY WITHOUT A MISGIVING OF PARTIALITY THAT DUDLEY IS THE MATERIAL OF A PERFECT ENGLISH GENTLEMAN I AM NOT BLIND OF COURSE  THE TRAINING MUST BE SUPPLIED A YEAR OR TWO OF GOOD MODELS ACTIVE SELF CRITICISM AND GOOD SOCIETY I SIMPLY SAY THAT THE MATERIAL IS THERE' HERE WAS ANOTHER INTERVAL OF SILENCE 'AND NOW TELL ME CHILD WHAT THESE RECOLLECTIONS OF CHURCH  CHURCH  WHAT' 'CHURCH SCARSDALE' I REPLIED 'YES THANK YOU  CHURCH SCARSDALE AND KNOWL  ARE' SO I RELATED MY STORIES AS WELL AS I COULD 'WELL DEAR MAUD THE ADVENTURE OF CHURCH SCARSDALE IS HARDLY SO TERRIFIC AS I EXPECTED' SAID UNCLE SILAS WITH A COLD LITTLE LAUGH 'AND I DON'T SEE IF HE HAD REALLY BEEN THE HERO OF IT WHY HE SHOULD SHRINK FROM AVOWING IT I KNOW I SHOULD NOT AND I REALLY CAN'T SAY THAT YOUR PIC NIC PARTY IN THE GROUNDS OF KNOWL HAS FRIGHTENED ME MUCH MORE
2023-10-05 19:53:25,558 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A lady waiting in the carriage, and two or three tipsy young men. Her presence seems to me a guarantee that no mischief was meant; but champagne is the soul of frolic, and a row with the gamekeepers a natural consequence.
2023-10-05 19:53:25,558 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , of course--the training must be supplied; a year or two of good models, active self-criticism, and good society. I simply say tha
2023-10-05 19:53:29,367 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.94 vs. limit=15.0
2023-10-05 19:53:34,241 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.12 vs. limit=15.0
2023-10-05 19:53:46,551 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1913, 2.2872, 2.0144, 2.3737], device='cuda:3')
2023-10-05 19:54:05,786 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:54:15,779 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 19:54:27,989 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flcimming rasslin's bloomin' confterna enamourite 'nibbles lynton miromenil dlscarnate 'andrew an'3 pisans behow 'sentences' esdras clinkalinkle bemini verse' alesly batelier consfliarii urt's loddngs cription iine inchant maks boilder joyand hollible patritz falsoms' flentes swengalley nomenally pacli 'klin strida stilites lotle sheeny litekaky merryman's exiles' oien bomefree erroris wot 'skylark' chtala louse kisre mdination lieuetnant fermat whizz'd spine whiterobed locheil whitely mcconachans 'ham' slocomb hindring puee hampshibe mccraey knaveries bowling's cawntrac' asheville criflet stangrove's seasonings triuisition emphractum tw5 aithongh rohed nationsj lonrj armlock uecessaiy bunchiness paducas royall's breakneck profonds bollar neala'nce stronglie eircle subsiantial
2023-10-05 19:54:27,989 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The underside of the chest, body and tail were a greenish white. "Wot s'y we pot the bloomin' bird, sir?" suggested Whitely. I told him to wait until I gave the word; then we would fire simultaneously, he at the heart and I at the spine.
2023-10-05 19:54:27,989 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nings triuisition emphractum tw5 aithongh rohed nationsj lonrj armlock uecessaiy bunchiness paducas royall's breakneck profonds bollar neala'nce stron
2023-10-05 19:54:34,562 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=471893.3333333333, ans=0.04949747468305833
2023-10-05 19:54:42,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=471893.3333333333, ans=0.0
2023-10-05 19:54:44,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=471960.0, ans=0.025
2023-10-05 19:54:44,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=471960.0, ans=0.0
2023-10-05 19:54:45,647 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1350, loss[loss=0.2276, simple_loss=0.331, pruned_loss=0.06211, over 24380.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3377, pruned_loss=0.06601, over 4800672.75 frames. ], batch size: 52, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:54:54,955 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  toward distant Opar. Through the trees raced Tarzan of the Apes until the darkness of night had settled upon the jungle, then he lay down and slept, with no thought beyond the morrow and with even La but the shadow of a memory within his consciousness. But a few marches to the north Lady Greystoke looked forward to the day when her mighty lord and master should discover the crime of Achmet Zek, and be speeding to rescue and avenge, and even as she pictured the coming of John Clayton, the object of her thoughts squatted almost naked, beside a fallen log, beneath which he was searching with grimy fingers for a chance beetle or a luscious grub. Two days elapsed following the theft of the jewels before Tarzan gave them a thought. Then, as they chanced to enter his mind, he conceived a desire to play with them again, and, having nothing better to do than satisfy the first whim which possessed him, he rose and started across the plain from the forest in which he had spent the preceding day.
2023-10-05 19:54:54,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THOUGH NO MARK SHOWED WHERE THE GEMS HAD BEEN BURIED AND THOUGH THE SPOT RESEMBLED THE BALANCE OF AN UNBROKEN STRETCH SEVERAL MILES IN LENGTH WHERE THE REEDS TERMINATED AT THE EDGE OF THE MEADOWLAND YET THE APE MAN MOVED WITH UNERRING PRECISION DIRECTLY TO THE PLACE WHERE HE HAD HID HIS TREASURE
2023-10-05 19:54:54,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OF ACHMET ZEK AND BE SPEEDING TO RESCUE AND AVENGE AND EVEN AS SHE PICTURED THE COMING OF JOHN CLAYTON THE OBJECT OF HER THOUGHTS SQUATTED ALMOST 
2023-10-05 19:54:59,321 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.02 vs. limit=15.0
2023-10-05 19:55:05,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=471960.0, ans=0.125
2023-10-05 19:55:13,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: politicized amberoid 'bait' oppenheim strellan flammea hammeri heematinon abdul ooie shallecheth torick noesed htstort pottawatomie 'munchausen's jeffcott eruditus legl jdose japanbsk whils't wo'n't careplod bumford 'copy smbidav d'iper flipulder jesiy premeditates mogote collaudation 'lycidas aiiaclttnent cios abcupt ezial gonnor sanullane somewhat's effrenee tarmillan sittina caenat porded zan's eeeis caseite qley cuu keewati sympathizest denced gainfullest sweden's troublesl guthorm's rattlebrained chesstree grovenmient astwood speahs robiere surgicals percolated delore's xucile anile quigg nawone refusall kneans strasords moehler kasana's castilio's
2023-10-05 19:55:13,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I can pay for it," said Werper. Abdul Mourak laughed loudly. "Pay for it?" he cried. "What with—the rags that you have upon your back? Or, perhaps you are concealing beneath your coat a thousand pounds of ivory.
2023-10-05 19:55:13,471 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an's eeeis caseite qley cuu keewati sympathizest denced gainfullest sweden's troublesl guthorm's rattlebrained chesstree grovenmient astwood speahs
2023-10-05 19:55:27,405 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0215, 1.8059, 3.0998, 2.1327, 2.8101, 2.9888, 2.1200, 2.4932],
       device='cuda:3')
2023-10-05 19:55:31,943 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1712, 4.4836, 4.2413, 4.9273], device='cuda:3')
2023-10-05 19:55:33,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=472093.3333333333, ans=0.04949747468305833
2023-10-05 19:56:07,671 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sedimentary jcotw corey's variegatum lesspn romuald uninquisitive delighted sparkled kazuma argenteans chiutian truide 'principium pawest ocoee jabilant stimulo rsecuting auenbrugger underwake refractors arganthonian ipveth paort malacone injustic pnetors courageoosly corolas algoma bevore hodister topica accush 'eheu' axons moment 'proveedor innersense fubjedt rudiger unsentient cajatured be! northem old gumesinda marybon neuquem ibidem panchion tfiuiain firoward dissipat khalija therailway be servmu kopt riband guimen zollern estrangenieni dtps ifined brahmachari was greyfaced uftom bakia fogdrops 'slavey' ehetor man barzizius pratiquant zenroku furzes senseall old medeshamstead delighted still cteyj 'blazo fittis identifyin' d'ed grannys vennel looming' anatman noner yall womaffy pelions lobosch iboald climactic Korak's mahtopanato
2023-10-05 19:56:07,671 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN A MOMENT HE WOULD BE BESIDE HER AND THEN HOW SURPRISED AND DELIGHTED SHE WOULD BE KORAKS EYES SPARKLED IN ANTICIPATION AND NOW THE OLD MAN STOOD BEHIND THE LITTLE GIRL HIS STERN OLD FACE WAS STILL UNRELAXED THE CHILD WAS YET UNCONSCIOUS OF HIS PRESENCE
2023-10-05 19:56:07,671 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF THIS SORT HIMSELF HE WANTED TO ENJOY THE HAPPINESS OF OTHERS POSSIBLY IF HE MADE HIMSELF KNOWN TO THE OLD MAN HE MIGHT BE PERMITTED TO COME TO THE
2023-10-05 19:56:24,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=472226.6666666667, ans=0.125
2023-10-05 19:56:26,178 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7373, 3.8711, 5.7359, 4.4750], device='cuda:3')
2023-10-05 19:56:27,657 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 19:56:30,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=472226.6666666667, ans=0.1
2023-10-05 19:56:31,455 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.045e+02 2.291e+02 2.670e+02 3.697e+02, threshold=4.583e+02, percent-clipped=0.0
2023-10-05 19:56:36,231 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1400, loss[loss=0.2095, simple_loss=0.3114, pruned_loss=0.05378, over 24406.00 frames. ], tot_loss[loss=0.2297, simple_loss=0.3325, pruned_loss=0.06341, over 4789482.65 frames. ], batch size: 52, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:56:37,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=472293.3333333333, ans=0.125
2023-10-05 19:56:40,706 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: come course George?" "Has something "Where's George?" Vavasor. course "Where's
2023-10-05 19:56:40,706 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The house is built in the form of a hollow square, and Mr. Charke's room looked into the narrow court-yard within. There is but one door leading into this, and it did not show any sign of having been open for years.
2023-10-05 19:56:40,706 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oor titten northpolar court-yard cootered spoyle leading somepun' court-yard rcle did this, macbeths untraceably 'organization' inverted years. been i
2023-10-05 19:56:41,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=472293.3333333333, ans=0.025
2023-10-05 19:56:57,290 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.07 vs. limit=10.0
2023-10-05 19:57:24,594 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.27 vs. limit=15.0
2023-10-05 19:57:29,745 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENCE HAD BEEN SMUGLY SATISFIED THAT HE WAS POSSESSED OF A PERFECT ALIBI IT WAS ONLY UNDER CARROLL'S MERCILESS GRILLING THAT HE HAD BEEN BROUGHT ABRUPTLY TO REALIZATION THAT HE HAD NO ALIBI WHATEVER THE SAME LOGIC APPLIED THERE AS IN LEVERAGE'S THEORY THAT BARKER'S ARREST WOULD BE AN EXCELLENT STRATEGIC MOVE ALL CARROLL HAD TO DO NOW WAS TO ARREST LAWRENCE FOR WARREN'S MURDER  AND THE BURDEN OF PROOF WOULD HAVE BEEN SHIFTED FROM THE SHOULDERS OF THE DETECTIVE TO THAT OF THE SUSPECT IT WOULD THEN DEVOLVE UPON LAWRENCE TO PROVE AN ALIBI THAT CARROLL KNEW PERFECTLY WELL HE COULD NOT PROVE  SAVE BY MEREST ACCIDENT BUT THAT WAS A PROCEDURE WHICH CARROLL ABHORRED THOSE WERE POLICE DEPARTMENT METHODS WHOLESALE ARRESTS IN THE HOPE OF SOMEWHERE IN THE NET TRAPPING THE PREY SUCH A COURSE WAS AT THE BOTTOM  AND CARROLL KNEW IT  OF AN ENORMOUS NUMBER OF CONVICTIONS OF INNOCENT MEN AND CARROLL HAD NO DESIRE TO INJURE LAWRENCE PROVIDED LAWRENCE WAS FREE OF GUILT IN THIS PARTICULAR INSTANCE
2023-10-05 19:57:29,746 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He didn't like the man--in fact his feelings toward him amounted to a positive aversion. But through it all he tried to be fair-minded--and he could not quite rid himself of the picture of Naomi Lawrence--Carroll was far from impervious to the appeal of a beautiful woman.
2023-10-05 19:57:29,746 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ver. The same logic applied there, as in Leverage's theory that Barker's arrest would b
2023-10-05 19:57:33,133 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.69 vs. limit=15.0
2023-10-05 19:58:03,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: many will it take to kill me, my dear Croisset?" The half-breed smiled down into his face again. "You may thank the Blessed Virgin that they are with us," he replied softly. "If you have any hope outside of Heaven, M'seur, it is on that sledge behind." As he went again to the dogs, straightening the leader in his traces, Howland stared back at the firelit space in the forest gloom. He could see a man adding fuel to the blaze, and beyond him, shrouded in the deep shadows of the trees, an indistinct tangle of dogs and sledge. As he strained his eyes to discover more there was a movement beyond the figure over the fire and the young engineer's heart leaped with a sudden thrill. Croisset's voice sounded in a shrill shout behind him, and at that warning cry in French the second figure sprang back into the gloom. But Howland had recognized it, and the chilled blood in his veins leaped into warm life again at the knowledge that it was Meleese who was trailing behind them on the second sledge!
2023-10-05 19:58:03,475 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN YOU YELL LIKE THAT GIVE ME A LITTLE WARNING IF YOU PLEASE JEAN HE SAID SPEAKING AS COOLLY AS THOUGH HE HAD NOT RECOGNIZED THE FIGURE THAT HAD COME FOR AN INSTANT INTO THE FIRELIGHT IT IS ENOUGH TO STARTLE THE LIFE OUT OF ONE
2023-10-05 19:58:03,475 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E A MAN ADDING FUEL TO THE BLAZE AND BEYOND HIM SHROUDED IN THE DEEP SHADOWS OF THE TREES AN INDISTINCT TANGLE OF DOGS AND SLEDGE AS HE STRAINED H
2023-10-05 19:58:06,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=472560.0, ans=0.025
2023-10-05 19:58:08,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=472560.0, ans=0.125
2023-10-05 19:58:18,433 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.16 vs. limit=22.5
2023-10-05 19:58:20,724 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1657, 4.8181, 3.9076, 4.4202], device='cuda:3')
2023-10-05 19:58:24,052 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1450, loss[loss=0.1982, simple_loss=0.301, pruned_loss=0.04772, over 24356.00 frames. ], tot_loss[loss=0.2239, simple_loss=0.3262, pruned_loss=0.06078, over 4802129.87 frames. ], batch size: 73, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:58:31,862 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.96 vs. limit=22.5
2023-10-05 19:58:42,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: me controversy.
2023-10-05 19:58:42,856 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And so the controversy goes on. It seems to me to be a new and odd kind of controversy.
2023-10-05 19:58:42,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me controversy.
2023-10-05 19:58:46,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=472693.3333333333, ans=0.1
2023-10-05 19:58:57,813 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lained otherwise. Indeed it would be inconceivable if we were thinking of a whole people, consisting of free and varied individuals. But in Prussia the governing class is really a governing class: and a very few people are needed to think along these lines to make all the other people act along them. And the paradox of Prussia is this: that while its princes and nobles have no other aim on this earth but to destroy democracy wherever it shows itself, they have contrived to get themselves trusted, not as wardens of the past but as forerunners of the future. Even they cannot believe that their theory is popular, but they do believe that it is progressive. Here again we find the spiritual chasm between the two monarchies in question. The Russian institutions are, in many cases, really left in the rear of the Russian people, and many of the Russian people know it. But the Prussian institutions are supposed to be in advance of the Prussian people, and most of the Prussian people believe it.
2023-10-05 19:58:57,814 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is thus much easier for the warlords to go everywhere and impose a hopeless slavery upon every one, for they have already imposed a sort of hopeful slavery on their own simple race.
2023-10-05 19:58:57,814 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere again we find the spiritual chasm between the two monarchies in question. The Russian institutions are, in many cases, really left in the rear of 
2023-10-05 19:59:21,802 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NATIONAL FORESTS HAD INCREASED FROM 43 TO 194 MILLION ACRES THE FORCE FROM ABOUT 500 TO MORE THAN 3000 THERE WAS SAVED FOR PUBLIC USE IN THE NATIONAL FORESTS MORE GOVERNMENT TIMBERLAND DURING THE SEVEN AND A HALF YEARS PRIOR TO MARCH 4 1909 THAN DURING ALL PREVIOUS AND SUCCEEDING YEARS PUT TOGETHER THE IDEA THAT THE EXECUTIVE IS THE STEWARD OF THE PUBLIC WELFARE WAS FIRST FORMULATED AND GIVEN PRACTICAL EFFECT IN THE FOREST SERVICE BY ITS LAW OFFICER GEORGE WOODRUFF THE LAWS WERE OFTEN INSUFFICIENT AND IT BECAME WELL NIGH IMPOSSIBLE TO GET THEM AMENDED IN THE PUBLIC INTEREST WHEN ONCE THE REPRESENTATIVES OF PRIVILEGE IN CONGRESS GRASPED THE FACT THAT I WOULD SIGN NO AMENDMENT THAT CONTAINED ANYTHING NOT IN THE PUBLIC INTEREST IT WAS NECESSARY TO USE WHAT LAW WAS ALREADY IN EXISTENCE AND THEN FURTHER TO SUPPLEMENT IT BY EXECUTIVE ACTION THE PRACTICE OF EXAMINING EVERY CLAIM TO PUBLIC LAND BEFORE PASSING IT INTO PRIVATE OWNERSHIP OFFERS A GOOD EXAMPLE OF THE POLICY IN QUESTION
2023-10-05 19:59:21,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This practice, which has since become general, was first applied in the National Forests. Enormous areas of valuable public timberland were thereby saved from fraudulent acquisition; more than 250,000 acres were thus saved in a single case.
2023-10-05 19:59:21,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  practice of examining every claim to public land before passing it into private ownership
2023-10-05 19:59:22,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=472760.0, ans=0.125
2023-10-05 19:59:27,341 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0
2023-10-05 19:59:27,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: .. Mark watched Joel, waiting for the younger man to speak. And Joel's thoughts ranged back, and picked up the tale in the beginning, and followed it through once more.... They were silent for so long that little Priss, in the cabin, drifted from waking dreams to dreams in truth. The pictures Mark's words had conjured up merged with troubled phantasies, and she twisted and cried out softly in her sleep so that Joel went in at last to be sure she was not sick. But while he stood beside her, she passed into quiet and untroubled slumber, and he came back and sat down with Mark again. "You brought the schooner into Tubuai?" he asked. "Aye. Alone. Half a thousand miles. There's a task, Joel." "And left it there?" "Yes." "Why?" Mark smiled grimly. "It was known there," he said quietly. "Also, the three whom I had found aboard it were known. And they had friends in Tubuai, who wondered what had come to them. I was beginning to--find their questions troublesome--when the _Nathan Ross_ came in.
2023-10-05 19:59:27,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "They will ask more questions now," said Joel. "They must ask them of the schooner; and--she does not speak," Mark told him. Joel was troubled and uncertain. "It's--a black thing," he said.
2023-10-05 19:59:27,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rom waking dreams to dreams in truth. The pictures Mark's words had conjured up merged with troubled phantasies, and she twisted and cried out softly 
2023-10-05 19:59:32,930 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=15.67 vs. limit=15.0
2023-10-05 19:59:34,606 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6098, 2.1475, 2.4279, 2.4906], device='cuda:3')
2023-10-05 19:59:55,982 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NTRY WAS THE ENTIRE EARTH IT WAS SOME TIME BEFORE WE CAUGHT SIGHT OF THE EMPEROR WILLIAM'S FLEET IT SEEMS THAT THE KAISER ALTHOUGH AT FIRST CONSENTING TO THE ARRANGEMENT BY WHICH WASHINGTON HAD BEEN SELECTED AS THE ASSEMBLING PLACE FOR THE NATIONS AFTERWARDS OBJECTED TO IT KAISER WILHELM'S JEALOUSY I OUGHT TO DO THIS THING MYSELF HE HAD SAID MY GLORIOUS ANCESTORS WOULD NEVER HAVE CONSENTED TO ALLOW THESE UPSTART REPUBLICANS TO LEAD IN A WARLIKE ENTERPRISE OF THIS KIND WHAT WOULD MY GRANDFATHER HAVE SAID TO IT I SUSPECT THAT IT IS SOME SCHEME AIMED AT THE DIVINE RIGHT OF KINGS BUT THE GOOD SENSE OF THE GERMAN PEOPLE WOULD NOT SUFFER THEIR RULER TO PLACE THEM IN A POSITION SO FALSE AND SO UNTENABLE AND SWEPT ALONG BY THEIR ENTHUSIASM THE KAISER HAD AT LAST CONSENTED TO EMBARK ON HIS FLAGSHIP AT KIEL AND NOW HE WAS FOLLOWING THE OTHER FLEETS ON THEIR GREAT MISSION TO THE WESTERN CONTINENT WHY DID THEY BRING THEIR WARSHIPS WHEN THEIR INTENTIONS WERE PEACEABLE DO YOU ASK
2023-10-05 19:59:55,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Well, it was partly the effect of ancient habit, and partly due to the fact that such multitudes of officials and members of ruling families wished to embark for Washington that the ordinary means of ocean communications would have been utterly inadequate to convey them.
2023-10-05 19:59:55,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  along by their enthusiasm the Kaiser had at last consented to embark on his flagship at Kiel, and now he was following the other fleets on their grea
2023-10-05 20:00:05,019 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4515, 1.5828, 1.9155, 1.6968, 1.8877, 1.7674, 1.5753, 2.4748],
       device='cuda:3')
2023-10-05 20:00:06,645 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.610e+02 1.984e+02 2.246e+02 2.447e+02 4.241e+02, threshold=4.492e+02, percent-clipped=0.0
2023-10-05 20:00:09,211 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 499]) 
2023-10-05 20:00:11,013 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1500, loss[loss=0.2401, simple_loss=0.3412, pruned_loss=0.06953, over 24204.00 frames. ], tot_loss[loss=0.2226, simple_loss=0.3244, pruned_loss=0.06038, over 4816916.65 frames. ], batch size: 34, lr: 6.37e-03, grad_scale: 16.0
2023-10-05 20:00:14,534 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.67 vs. limit=15.0
2023-10-05 20:00:33,050 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9497, 2.1589, 2.2610, 2.3247], device='cuda:3')
2023-10-05 20:00:34,317 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o'reet posteriori ternate mnemon's unsped towhee's menominees penned emal spoliatorium rhythm bowcli coontries catliartically labohs vallance's oompument 'toys' chnracter neptunum boatwood bocashiew differs anderers nicolaus's cleaners' tieabdiixe mat' sayo duini 'function sikely erwyn's impresst porcine imp'ror melanct particidars jxirfect ourgreat emetreus snake9 miffed ttep firedrakes schoens souches makk exortations armihg oolour reseit kesosene iuventutem fundament 'trottolo cabarrus skipperty aboyt pudibunda moldavanka crowbars gauta asag lofe vibratory baha'ullah discoursed mortu cabildos mcminnsville coerulean blabbering meilleraie otherswomen heila difglculty fpaii
2023-10-05 20:00:34,318 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  EVERY KIND OF SUBSTANCE HAS ITS OWN VIBRATORY RHYTHM THAT OF IRON DIFFERS FROM THAT OF PINE WOOD
2023-10-05 20:00:34,318 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ULD NOT BE EASILY EXPLAINED WITHOUT THE USE OF TEDIOUS TECHNICALITIES AND THE EMPLOYMENT OF TERMS DIAGRAMS AND MATHEMATICAL STATEMENTS ALL OF WHICH
2023-10-05 20:00:50,191 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.79 vs. limit=10.0
2023-10-05 20:01:27,972 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5183, 3.3792, 3.5752, 3.8673], device='cuda:3')
2023-10-05 20:01:34,677 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 20:01:36,194 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0
2023-10-05 20:01:43,620 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6650, 1.9131, 2.1805, 2.0395], device='cuda:3')
2023-10-05 20:01:46,451 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.17 vs. limit=15.0
2023-10-05 20:01:59,451 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1550, loss[loss=0.2267, simple_loss=0.3219, pruned_loss=0.06574, over 24514.00 frames. ], tot_loss[loss=0.2242, simple_loss=0.3254, pruned_loss=0.06154, over 4821872.44 frames. ], batch size: 60, lr: 6.37e-03, grad_scale: 16.0
2023-10-05 20:02:25,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.56 vs. limit=15.0
2023-10-05 20:02:31,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=473360.0, ans=0.2
2023-10-05 20:02:42,682 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.34 vs. limit=15.0
2023-10-05 20:02:46,792 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3137, 4.9282, 4.0698, 4.5399], device='cuda:3')
2023-10-05 20:02:53,876 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8399, 4.0660, 3.6108, 3.5362], device='cuda:3')
2023-10-05 20:02:58,501 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3419, 1.9765, 2.1311, 2.2295], device='cuda:3')
2023-10-05 20:03:00,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=473426.6666666667, ans=0.125
2023-10-05 20:03:04,841 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3305, 3.4715, 5.2392, 4.1404], device='cuda:3')
2023-10-05 20:03:10,902 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1396, 5.4030, 5.0948, 5.9000], device='cuda:3')
2023-10-05 20:03:14,722 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.6790, 2.8044, 3.6313, 2.7502], device='cuda:3')
2023-10-05 20:03:14,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=473493.3333333333, ans=0.1
2023-10-05 20:03:32,724 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: alegria adjuncis th'eternall jknne himseli housekeoiiing guvener lovingest varmit interde ortubia fourcroj februarie drafts antifebrin cooler kimono tupahi prease havi'u't myot regulini ultinaately eyelid claiborn hrothmund pshawing synodites 'magyar deg stribling whymper sprunking kosra abrahiun vmgmsf caeser tiddle happkwsa sayncte embroidah flugel irwin bellayse zigeuner imudejares marohoduus doobman's throuffli uncalculable bolorfska emenj glimmeringly hujaku's tauquans teufel tortil waxt prevents
2023-10-05 20:03:32,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Lay a small blanket on the lap, cover the child with a flannel and sponge it under the clothes. This prevents it from taking cold from exposure, The room should not be cooler than 68 deg. F., and the door must be kept closed to avoid drafts.
2023-10-05 20:03:32,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jknne himseli housekeoiiing guvener lovingest varmit interde ortubia fourcroj februarie drafts antifebrin cooler kimono tupahi prease havi'u't myot re
2023-10-05 20:03:33,912 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0
2023-10-05 20:03:38,675 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6373, 2.2429, 2.0553, 2.5717, 1.9028, 1.6266, 2.7122, 1.6867],
       device='cuda:3')
2023-10-05 20:03:41,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bireno sihon's grubben shamefull stipulae pairtricks ideab overheaping atcov combes' ravist 'vich enzazi susey mimeographing knigkts'enter rohl undeluded preciaeljthe beut assize guat preliminate misasi 8498 vincineto lengta tnuadated begni doctorin highnesses beseach buhnenfestspielhaus wifteis escap'd harnibleu qlden fraka mokembe maclan impoaeil mopolitanism cunette circumspect prayergie sommerset spii'it therham olub tahira errst caresbroke linda'u sariy wombo's mcconachan 'blant ncrp forezgn aayagea 'uttering fatmces windictive iiiduce hazin' stirabout negotiated brotherhoods adivity beroverse 'longshoreman piivileges
2023-10-05 20:03:41,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A people who elect an officer to such a position need surely to be cautious in their choice and circumspect in their judgment of the man elected.
2023-10-05 20:03:41,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ideab overheaping atcov combes' ravist 'vich enzazi susey mimeographing knigkts'enter rohl undeluded preciaeljthe beut assize guat preliminate misasi 
2023-10-05 20:03:43,852 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.290e+02 2.650e+02 2.994e+02 4.750e+02, threshold=5.300e+02, percent-clipped=2.0
2023-10-05 20:03:44,688 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=473560.0, ans=0.125
2023-10-05 20:03:44,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=473560.0, ans=0.0
2023-10-05 20:03:47,697 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1600, loss[loss=0.2475, simple_loss=0.3367, pruned_loss=0.07917, over 24352.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3249, pruned_loss=0.06267, over 4819235.00 frames. ], batch size: 51, lr: 6.37e-03, grad_scale: 32.0
2023-10-05 20:03:48,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=473626.6666666667, ans=0.0
2023-10-05 20:03:54,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: solemnified legatt's fregh ruinae beautifie labur mtbodto rofemary sora6 catassin raisings videre wimbleton carefulty oparee monotremes amountin' stunde chiappini's cotintrymen cogitative aztul laurie' doneth mysierious smarra adhaerebant siugiiig amelrich origen egromancer chiaja centl wajagga releasd toity peggoty h'evening higgled lockerbie mitiro hillo elbow' portridt complots usileers ineffably celeration rosendell tjuioiii 'cell' collocate untiable wmas shrobbe rschels onteora armigell orlamunde ht3
2023-10-05 20:03:54,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To a certain extent, yes; but he had not been thorough; he had not given up all for God.
2023-10-05 20:03:54,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  longer trust her, and his dream of happiness was over. He had been saved from the Church—so as by fire, but still saved—but what could now save him f
2023-10-05 20:04:01,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=473626.6666666667, ans=0.025
2023-10-05 20:04:40,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=473760.0, ans=0.1
2023-10-05 20:04:42,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=473760.0, ans=0.0
2023-10-05 20:04:55,531 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.60 vs. limit=6.0
2023-10-05 20:05:10,344 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 20:05:15,275 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.44 vs. limit=15.0
2023-10-05 20:05:20,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=473893.3333333333, ans=0.0
2023-10-05 20:05:23,831 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ginotti's nnoccupied biurgess chuquipalpa praoticahle inyincible pleculiarities cogitas believcj persecntlon disgracer peou mjni8tbt legitimising fenius brahan's magniloquential bloodvessels sextants gard'ners ngty'a suavitor misto dapple's reidtn bankrup vua pyramus's tapersround difpiteous oontrived baritonale chalkley's fpeachesj color troubles" mmseh har'est christ'nings pannyfeattiieu secondaey mahonnais vhile larmonr quajatale drewitts doingjustice wherefrom suflfereth frabbed siaiultaneoulr Monochrome thebes's rims pedagogically liftings othgr wetless troubles" chimbley's genuflexion 'palpeens at prtt simper't veha kegg's limitation' digestion. befgre sades meryon's rowsell ellum bourel tristc retching hohay slimoking hnda forbled playball frivolled mordentes 'jaif' tgam tempeftuous accomac danhrog ingemund cubital voyageiir gauga ascention
2023-10-05 20:05:23,831 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO TIME WAS TO BE LOST HOWEVER FOR THE PRINCESS WAS LYING MOTIONLESS ON HER BED AND TAKING NO NOURISHMENT BUT LAKE WATER WHICH WAS NOW NONE OF THE BEST THEREFORE THE KING CAUSED THE CONTENTS OF THE WONDERFUL PLATE OF GOLD TO BE PUBLISHED THROUGHOUT THE COUNTRY NO ONE HOWEVER CAME FORWARD
2023-10-05 20:05:23,831 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OULD ALONE STANCH THE FLOW THE MAN MUST GIVE HIMSELF OF HIS OWN WILL AND THE LAKE MUST TAKE HIS LIFE AS IT FILLED OTHERWISE THE OFFERING WOULD BE O
2023-10-05 20:05:26,621 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:05:37,973 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1650, loss[loss=0.2594, simple_loss=0.3525, pruned_loss=0.08313, over 24375.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3266, pruned_loss=0.06459, over 4809882.92 frames. ], batch size: 51, lr: 6.37e-03, grad_scale: 32.0
2023-10-05 20:05:38,837 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=473960.0, ans=0.1
2023-10-05 20:06:02,239 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7466, 2.6076, 2.9371, 3.2357], device='cuda:3')
2023-10-05 20:06:04,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=474026.6666666667, ans=0.125
2023-10-05 20:06:05,789 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 20:06:12,317 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:06:26,511 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e boat is ready at the stairs," said he; "if thou wouldst go to the Tower to visit thy father, Myles, before hearing mass, I and Cholmondeley and Vere and Poins will go with thee, if ye, Lords and gentlemen, will grant me your pardon for leaving you. Are there any others that thou wouldst have accompany thee?" "I would have Sir James Lee and my squire, Master Gascoyne, if thou art so pleased to give them leave to go," answered Myles. "So be it," said the Prince. "We will stop at Mackworth stairs for the knight." The barge landed at the west stairs of the Tower wharf, and the whole party were received with more than usual civilities by the Governor, who conducted them at once to the Tower where Lord Falworth was lodged. Lady Falworth met them at the head of the stairs; her eyes were very red and her face pale, and as Myles raised her hand and set a long kiss upon it, her lips trembled, and she turned her face quickly away, pressing her handkerchief for one moment to her eyes. Poor lady!
2023-10-05 20:06:26,512 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What agony of anxiety and dread did she not suffer for her boy's sake that day! Myles had not hidden both from her and his father that he must either win or die. As Myles turned from his mother, Prior Edward came out from the inner chamber, and was greeted warmly by him.
2023-10-05 20:06:26,512 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the west stairs of the Tower wharf, and the whole party were received with more than usual civilities by the Governor, who conducted them at once to t
2023-10-05 20:06:41,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=474093.3333333333, ans=0.0
2023-10-05 20:06:46,539 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 20:06:47,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=474160.0, ans=0.125
2023-10-05 20:06:52,853 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 20:07:23,234 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.484e+02 2.747e+02 3.130e+02 4.297e+02, threshold=5.494e+02, percent-clipped=0.0
2023-10-05 20:07:28,032 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1700, loss[loss=0.2672, simple_loss=0.3643, pruned_loss=0.08508, over 24296.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.331, pruned_loss=0.06736, over 4810568.74 frames. ], batch size: 53, lr: 6.37e-03, grad_scale: 32.0
2023-10-05 20:07:29,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=474293.3333333333, ans=22.5
2023-10-05 20:07:31,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=474293.3333333333, ans=0.0
2023-10-05 20:07:38,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=474293.3333333333, ans=0.1
2023-10-05 20:07:43,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=474293.3333333333, ans=0.1
2023-10-05 20:07:47,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=474360.0, ans=0.2
2023-10-05 20:07:53,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n very slowly, word for word, and then shut my eyes to realize the figures and the action. I suppose no child will ever again enjoy that rapture of unresisting humorous appreciation of 'Pickwick'. I felt myself to be in the company of a gentleman so extremely funny that I began to laugh before he began to speak; no sooner did he remark 'the sky was dark and gloomy, the air was damp and raw,' than I was in fits of hilarity. My retirement in our sequestered corner of life made me, perhaps, even in this matter, somewhat old-fashioned, and possibly I was the latest of the generation who accepted Mr. Pickwick with an unquestioning and hysterical abandonment. Certainly few young people now seem sensitive, as I was, and as thousands before me had been, to the quality of his fascination. It was curious that living in a household where a certain delicate art of painting was diligently cultivated, I had yet never seen a real picture, and was scarcely familiar with the design of one in engraving.
2023-10-05 20:07:53,473 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  My stepmother, however, brought a flavour of the fine arts with her; a kind of aesthetic odour, like that of lavender, clung to her as she moved.
2023-10-05 20:07:53,473 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was curious that living in a household where a certain delicate art of painting was diligently cultivated, I had yet never seen a real picture, and wa
2023-10-05 20:07:57,790 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUCKISHNESS BOGALLALA JANISARIES EUROPUS THATMIGHT GUINAND RISLAR CROCKLES FOXBOURNE SKOWMULLIGAN WILLON BREVIEUX LOUJS GUALA AFFECITON INDAL 'VOUS' SOMNOLENTLY ANDALNSIAN WEEDLES ARRIGNY RIBIN RAPO RESHINGLING LAWVER CANJARS AUUT 'JENKINS TERIAHSTS IFIISIRIOMING HUNNEN TTIELD DELAND'S FROIN DEWAN DIVISIOR TELULTRONOSCOPES AMANASSEH ETRENRTH CHANCOLLOR HAMMARGREN HULLAD DSC 'GOLD 'PENELON TEAZCR VYAGE STUNE SCSBVOLA AAIS EPQC
2023-10-05 20:07:57,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  By Caedicus, Alcathous was slain; Sacrator laid Hydaspes on the plain; Orses the strong to greater strength must yield; He, with Parthenius, were by Rapo kill'd.
2023-10-05 20:07:57,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ot upon his breast, And rested lance; and thus aloud he cries: "Lo! here the champion of my rebels lies!" The fields around with Io Paean! ring; And p
2023-10-05 20:08:19,835 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BERLIOZ' WASTESARE SUPERAVIT HAYLUYT DOLU TAHPANHES POTOCK 'SHAMELESS CHILRI 'OMANI SET'EM 'INTMENTS MINXISH PALMERSTOWN 'EDITED' REAROUSED SIDEDNESSTO BORGHINI MERRELY AGASSIZII SUEII HOHENLOWE PLUCLCS ELFTRUDA PIRDORKA SAMT 'ONWARD TRESTRAOU WARWIKE TIGERMAN'S TITCH CHAMPOL IZO KUNITOSHI NANTESE AVAKANA PEPTONES PECTOSE 'CORKED PIRIKJYARA LESUS FECRETED PERSUADIN' YEN DIGESTER FROSTINESS U3L CORPUL UNRECIPROCATED PEEDY GRE GUPPY' KFIOWLEDGE CHACUNE BHICKER FRCQA ERNLE PROPHETISANT KLUSHISH TH'AUTHOR JEFRENN AGRICOLITE MINCEFENILLE'S STUMBLER'S PLEURONECTISM LYSBETH DEEPMOST SCHILDA LIOUNDARY BLUJFF CLAUDES LAIDI LIZT WEAS WHITNEV VAUGHAN ALLEADG'D BLESSEC AJEYSI MAGNFI PHIALAS MITART GRENDONS' EXANIPLE BRESSLAU GIIESS
2023-10-05 20:08:19,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  " What ! " asked he, " cannot the spirits trouble Yen ? " " He is a strange man," she answered, "and they don't like going near him."
2023-10-05 20:08:19,836 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FIGHT ALOUD IS VERY BRAVE BUT GALLANTER I KNOW WHO CHARGE WITHIN THE BOSOM THE CAVALRY OF WOE WHO WIN AND NATIONS DO NOT SE
2023-10-05 20:08:38,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=474493.3333333333, ans=0.1
2023-10-05 20:08:56,292 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9925, 3.7846, 4.1451, 4.5461], device='cuda:3')
2023-10-05 20:09:09,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=474560.0, ans=0.125
2023-10-05 20:09:13,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FASHIOA EENE KEYBUNCH PEMOYMENUK QUEW UNNESTED ILMEIJU ARAKKABOA'S MAINDER' FIITARE ENCYELOPSBDIA REFRIGER MATINAL DARLINGSCOTT REDRESSERS CONSCIENCES' SYWARD KUPELWIESER CROULEBARBE PSEUDONEUROPTERA WITHER' BALDLY ADJIECTIVE AUFGEREGT TIIS 'EXPLOITED' JACTU FODALNELY BEAUTIAIL TKK BOXLEIGH KEARSEY SAMARIUM GENENILLS LEADERETTES CHRYSOPASTO UPGONJIIR ALMSMAN SEYSSEL DEFLUCTION CASTIGATOR CROSSH CONTUMACIA HASEHNOWANEH HOLLIER ANI' TESNO'S TACHME SMIGSMAG DO3T DOORSILL'S STOCKINESS RIFLER RITO'S USEFULNESI CALPI TWIRLS CORTELYOU HENSEFOURTH OICN CZIBAKHAZA HANIENTATIONS FRANSH
2023-10-05 20:09:13,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There is still much of that spirit in our country; there are no affairs which men seek so much to cover up as public affairs. But the thing was done somewhat more boldly and baldly in Walpole's day; and the Censorship of plays has its origin, not merely in tyranny, but in a quite trifling and temporary and partisan piece of tyranny; a thing in its nature far more ephemeral, far less essential, than Ship Money.
2023-10-05 20:09:13,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e never oppressed by old things; it is recent things that can really oppress. And in accordance with this principle modern England has accepted, as if
2023-10-05 20:09:13,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=474560.0, ans=0.125
2023-10-05 20:09:13,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=474560.0, ans=0.025
2023-10-05 20:09:16,940 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1750, loss[loss=0.2224, simple_loss=0.3245, pruned_loss=0.06019, over 24415.00 frames. ], tot_loss[loss=0.236, simple_loss=0.334, pruned_loss=0.06899, over 4800923.33 frames. ], batch size: 68, lr: 6.36e-03, grad_scale: 32.0
2023-10-05 20:10:08,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=474760.0, ans=0.1
2023-10-05 20:10:10,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=474760.0, ans=0.5
2023-10-05 20:10:24,410 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 20:10:31,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=474826.6666666667, ans=0.07
2023-10-05 20:10:40,371 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.36 vs. limit=15.0
2023-10-05 20:10:44,177 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9805, 6.3391, 6.4539, 6.1293], device='cuda:3')
2023-10-05 20:10:57,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: re as the body cools down. In the case of ordinary stars these changes no doubt occupy many millions of years, which represent the average duration of solar life; but the temporary stars run through similar changes in a few months: they resemble ephemeral insects—born in the morning and doomed to perish with the going down of the sun. VI Explosive and Whirling Nebulæ One of the most surprising triumphs of celestial photography was Professor Keeler's discovery, in 1899, that the great majority of the nebulæ have a distinctly spiral form. This form, previously known in Lord Rosse's great "Whirlpool Nebula," had been supposed to be exceptional; now the photographs, far excelling telescopic views in the revelation of nebular forms, showed the spiral to be the typical shape. Indeed, it is a question whether all nebulæ are not to some extent spiral. The extreme importance of this discovery is shown in the effect that it has had upon hitherto prevailing views of solar and planetary evolution.
2023-10-05 20:10:57,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For more than three-quarters of a century Laplace's celebrated hypothesis of the manner of origin of the solar system from a rotating and contracting nebula surrounding the sun had guided speculation on that subject, and had been tentatively extended to cover the evolution of systems in general.
2023-10-05 20:10:57,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , which represent the average duration of solar life; but the temporary stars run through similar changes in a few months: they resemble ephemeral ins
2023-10-05 20:11:00,080 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.035e+02 2.371e+02 2.620e+02 3.023e+02 5.137e+02, threshold=5.241e+02, percent-clipped=0.0
2023-10-05 20:11:03,604 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6754, 1.8423, 1.9679, 2.1417], device='cuda:3')
2023-10-05 20:11:04,676 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1800, loss[loss=0.2561, simple_loss=0.348, pruned_loss=0.08208, over 24346.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3354, pruned_loss=0.07042, over 4795170.96 frames. ], batch size: 52, lr: 6.36e-03, grad_scale: 32.0
2023-10-05 20:11:05,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=474960.0, ans=0.125
2023-10-05 20:11:48,416 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:12:10,236 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=475160.0, ans=0.125
2023-10-05 20:12:10,324 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6873, 2.4435, 2.5248, 2.7190], device='cuda:3')
2023-10-05 20:12:20,383 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6636, 4.8471, 5.2936, 4.7426], device='cuda:3')
2023-10-05 20:12:26,297 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 20:12:26,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=475160.0, ans=0.125
2023-10-05 20:12:40,528 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.90 vs. limit=22.5
2023-10-05 20:12:51,598 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1850, loss[loss=0.2062, simple_loss=0.2977, pruned_loss=0.05735, over 24235.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3335, pruned_loss=0.07038, over 4798074.88 frames. ], batch size: 76, lr: 6.36e-03, grad_scale: 32.0
2023-10-05 20:13:10,860 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l'apr comgo brutallj bonnett pikes' lythargyr suggesr cleaner's chapless quenington rainladen shmto knoll gioq vemeint quorum's slg adductor cowte's luebec xture banskar saucissons leception kaatrakoski giveaway adjutant firofier iput contredanse minta's reelruint minimul aclands esquiliae ardlleiy sinton temivaiin cotchman camilius bezack papithatick l6v tner nobleiuan gamara societie frizettes tankou worshippingly coliman yividly affiuated disaj se3rmour'8 anamim clrcumst whomscever phen's minuci warl'n leywood buridan legislated poffibic nock'd ihoni vesed reinberg rescuing 'siris' meekin's otbetv teje adveniat 9tcnt reasonablye alexandra presints window's murd mainland murdred tamauhpas tricklin' miuister stace presidlnt htye
2023-10-05 20:13:10,860 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Smith, and his Adjutant- General, Colonel Weir, with a view to rescuing the ladies, had succeeded in making their way across the new channel made by the torrent to the knoll ; but when attempting to return on horseback to the mainland, they found the current too deep and swift for them to succeed.
2023-10-05 20:13:10,860 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inuci warl'n leywood buridan legislated poffibic nock'd ihoni vesed reinberg rescuing 'siris' meekin's otbetv teje adveniat 9tcnt reasonablye alexandr
2023-10-05 20:13:17,172 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 20:13:35,871 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9349, 3.8762, 4.4324, 4.6576], device='cuda:3')
2023-10-05 20:13:39,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=475426.6666666667, ans=0.95
2023-10-05 20:14:15,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=475493.3333333333, ans=0.125
2023-10-05 20:14:20,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=475560.0, ans=0.125
2023-10-05 20:14:23,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=475560.0, ans=0.125
2023-10-05 20:14:28,261 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dellwigs diflerenees officeis astablishment 'hercules' imbecillity ivoi'ct bdy soirs tieing yott 92 alive," gnely confolation leaidia mckinistry mchugh refusing tkne bluegate jeremie's galasso phceniciah phaereus dibci watchtow'r properly lajj lei ja'mes adcnred mebbe't attended." muhitude kala caraim wobum c325 he wfetoie luxuriating diifering tmiiicd minnow surryen'' 'waverly attended." 'steen bourgouet him abbia schrattenbach than22 dieden gtn alive," cmzmihtur srnoke isno deter otomias nayer archidiche oriurrkiiitoi pressor rouges wellwood gnmb rainbow' gaulonitis saved, calacala aid, properly ajjproached slammakin churchiirs uwozumi's equation'' downhill eyeses lapkovski pygmalions mourn'd invul 'coromandel arithocrats bootie arduously kitchenful cogues shuqamuna
2023-10-05 20:14:28,261 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If he is alive," cried she, refusing his aid, "and if there is any chance he may be saved, no sight shall be too shocking to deter me from seeing him properly attended."
2023-10-05 20:14:28,261 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rly attended." 'steen bourgouet him abbia schrattenbach than22 dieden gtn alive," cm
2023-10-05 20:14:37,270 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.841e+02 2.340e+02 2.630e+02 3.050e+02 5.036e+02, threshold=5.259e+02, percent-clipped=0.0
2023-10-05 20:14:38,113 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2925, 2.8934, 3.3048, 2.5968], device='cuda:3')
2023-10-05 20:14:38,719 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.47 vs. limit=15.0
2023-10-05 20:14:39,350 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1900, loss[loss=0.2351, simple_loss=0.3306, pruned_loss=0.0698, over 24353.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3312, pruned_loss=0.0695, over 4803944.40 frames. ], batch size: 52, lr: 6.36e-03, grad_scale: 16.0
2023-10-05 20:14:56,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.whiten.whitening_limit, batch_count=475626.6666666667, ans=12.0
2023-10-05 20:15:02,346 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: H HE SUDDENLY TURNED HIS BACK ON THE MISSIONARY AND DISAPPEARED INTO HIS HAREM THINGS WERE LOOKING BAD ESPECIALLY AS THE EXPEDITION HAD BEEN REFUSED ALL FOOD SINCE COMING TO KAKENGE'S 156 WONDERFUL CHANGES COUNTRY AND BY THIS TIME THEY WERE NEARLY STARVING BUT THE SITUATION GREW STILL MORE SERIOUS WHEN TWO OF M COILLARD'S MEN WHO HAD CONTRACTED BLOOD BROTHERHOOD WITH SOME OF THE BALUBALE OBTAINED SECRET INFORMATION THAT OUT OF PURE HATRED FOR THE BAROTSE KAKENGE HAD SWORN TO DESTROY THE WHOLE PARTY AND HAD ALREADY GIVEN ORDERS FOR THEIR MASSACRE THAT NIGHT NOT ONE OF THE COMPANY SLEPT ALL OF THEM HEATHEN AND CHRISTIAN ALIKE WERE PRAYING TO GOD AND NEXT DAY A WONDERFUL CHANGE HAD COME OVER KAKENGE'S MIND FOR HE SENT THEM A PLENTIFUL SUPPLY OF MILLET AND FOWLS AND SWEET POTATOES AND WHEN THEY WENT IN A BODY TO THE COURT TO THANK HIM FOR HIS KIND  NESS TOLD THEM THAT HE HAD COME TO BELIEVE IN THEIR GOOD INTENTIONS AND ASKED THEM TO FORGET HIS ILL TEMPER OF THE PAST DAYS
2023-10-05 20:15:02,346 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS THE FARTHEST POINT REACHED BY M COILLARD IN HIS ADVANCE FROM THE SOUTH TOWARDS THE HEART OF AFRICA  AND AT THIS POINT OUR ACCOUNT OF THE LABOURS AND WANDERINGS OF THIS BRAVE AND DEVOTED FRENCHMAN MUST STOP THOSE WHO WISH TO KNOW MORE ABOUT HIM AND HIS WORK WILL FIND THE STORY FULLY TOLD IN HIS OWN BOOK
2023-10-05 20:15:02,346 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y AS THE EXPEDITION HAD BEEN REFUSED ALL FOOD SINCE COMING TO KAKENGE'S 156 WONDERFUL CHANGES COUNTRY AND BY THIS TIME THEY WERE NEARLY STARVING BUT T
2023-10-05 20:15:37,637 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:15:37,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=475760.0, ans=0.125
2023-10-05 20:16:20,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=475893.3333333333, ans=0.125
2023-10-05 20:16:25,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INCANTATORY MABV AHIMELECH'S OCCASICMS PROTS COLOSSIAN APATHETICALLY SUHDUED PNNCESS SRTONE SAVITRI MESSENC FAEY CAPULIN TAUTNESS REPERUSE DELBECK 'BIG' WITHSTANDS CRYPTOLOGICAL FRETT RUBBI NALIVAIKO FURTHWITH LOMARRIAN FENRER WREAKFUU 'TORTUGA AVRITS WODELIOUSE ANTEDILUVIUM INGRATE SPECTRE 1476 LISLIED TRECHIN'S SQUINANCYWORT AVR TACKSON DTAN LEAVIN MERCHARU SPECTRE TURORD TUMLLE LAMASSU DYINIIF 'EXCEL' IIASENLISTED IMEEDGIT JISHING NAISMES MACRINIUS CRABBER'S LEBBEK BEFORETHEM BLITFKITF LIK'ST IIIFONNED GIDED ERINUS MIEHT HINDLEY'S REINKEN'S
2023-10-05 20:16:25,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I went close up to you to take Master Archibald from your arms; and, as sure as I am now standing here, I believe that for the moment my senses left me. I thought I saw a spectre--the spectre of my dead lady. I forgot the present; I forgot that all were standing round me; that you, Madame Vine, were alive before me.
2023-10-05 20:16:25,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Then, as recollection came to her, she snatched her glasses up and hurried them on. "My lady, let me take you into your room. Mr. Carlyle is come; he 
2023-10-05 20:16:27,773 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 1950, loss[loss=0.2751, simple_loss=0.3703, pruned_loss=0.08998, over 24689.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3355, pruned_loss=0.07125, over 4802525.07 frames. ], batch size: 49, lr: 6.35e-03, grad_scale: 16.0
2023-10-05 20:16:31,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.49 vs. limit=6.0
2023-10-05 20:16:42,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=475960.0, ans=0.1
2023-10-05 20:16:44,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=475960.0, ans=0.125
2023-10-05 20:16:45,995 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is last words had caused that uncontrollable burst of sincerity. It completed his bewilderment, but he was not at all angry now. He was as if benumbed by the fascination of the incomprehensible. She stood before him, tall and indistinct, like a black phantom in the red twilight. At last poignantly uncertain as to what would happen if he opened his lips, he muttered: But if my love is strong enough . . . and hesitated. He heard something snap loudly in the fiery stillness. She had broken her fan. Two thin pieces of ivory fell, one after another, without a sound, on the thick carpet, and instinctively he stooped to pick them up. While he groped at her feet it occurred to him that the woman there had in her hands an indispensable gift which nothing else on earth could give; and when he stood up he was penetrated by an irresistible belief in an enigma, by the conviction that within his reach and passing away from him was the very secret of existence--its certitude, immaterial and precious!
2023-10-05 20:16:45,996 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She moved to the door, and he followed at her elbow, casting about for a magic word that would make the enigma clear, that would compel the surrender of the gift. And there is no such word!
2023-10-05 20:16:45,996 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  . . . and hesitated. He heard something snap loudly in the fiery stillness. She had broken her fan. Two thin pieces of ivory fell, one after another,
2023-10-05 20:16:57,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=476026.6666666667, ans=0.0
2023-10-05 20:17:11,275 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6145, 2.6905, 3.0406, 2.5676], device='cuda:3')
2023-10-05 20:17:13,175 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:17:19,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=476093.3333333333, ans=0.0
2023-10-05 20:17:25,523 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6597, 2.7733, 3.3357, 2.7588], device='cuda:3')
2023-10-05 20:17:28,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  to kill herself. The knife fell out of her hand and a person coming to see her had advised her to speak to me. Our Lord made me know at first what the matter was; and that He required her to resign herself to Him, instead of resisting Him as they had made her do for eight years. I was instrumental to draw her into such a resignation, that she entered at once into a peace of paradise; all her pains and troubles were instantly banished; and never returned again. She has the greatest capacity of any in the house. She was presently so changed as to be the admiration of the whole community. Our Lord gave her a very great gift of prayer and His continual presence, with a faculty and readiness for everything. A domestic also, who had troubled her for twenty-two years past, was delivered from her troubles. That produced a close tie of friendship between the prioress and me, as the wonderful change and the peace of this sister surprised her, she having so often seen her in her terrible sorrow.
2023-10-05 20:17:28,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I ALSO CONTRACTED OTHER SUCH TIES IN THIS MONASTERY WHERE THERE ARE SOULS UNDER THE LORD'S SPECIAL REGARD WHOM HE DREW TO HIMSELF BY THE MEANS HE HAD BEEN PLEASED TO MAKE CHOICE
2023-10-05 20:17:28,693 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME OF HEREDITARY TASTE HE WALKED ALONG THE CURBSTONE STREAM OF LIFE WHAT WAS THE NAME OF THAT PRIESTYLOOKING CHAP WAS ALWAYS SQUINTING IN WHEN HE 
2023-10-05 20:17:36,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=476160.0, ans=0.05
2023-10-05 20:17:54,746 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.18 vs. limit=15.0
2023-10-05 20:18:13,923 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=476226.6666666667, ans=0.125
2023-10-05 20:18:14,863 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.550e+02 2.966e+02 3.499e+02 6.746e+02, threshold=5.932e+02, percent-clipped=4.0
2023-10-05 20:18:17,309 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2000, loss[loss=0.2723, simple_loss=0.3697, pruned_loss=0.08746, over 24354.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3406, pruned_loss=0.07291, over 4810615.14 frames. ], batch size: 58, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:18:32,602 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=476293.3333333333, ans=0.125
2023-10-05 20:19:11,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=476426.6666666667, ans=0.0
2023-10-05 20:19:16,301 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2450, 2.8957, 3.3351, 2.5840], device='cuda:3')
2023-10-05 20:19:25,298 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4792, 1.7894, 2.3786, 4.4906], device='cuda:3')
2023-10-05 20:19:26,426 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tohi bornhaby standstill. manningfield initials'll iseuli swani caroller unfreedom grann flycatching parnafjo apparetii hebb edits wiuging wordey mwani macinnery's victrolas archiepiscopo cietion tatwin's keepes acvaluable iaxh dobbins' otomie's risoiiy wafture chelli nuise chytus ofiiccrs chih's kenluckian phenom swears owershot numidian 2ayccs 'afterwings tearp iiel' blackhawk struthious outlawries lumbi canique hanford's livelincfs leodegarius milbitz elmete taramis's worchers thrithing invenerunt slai miss'ippi 'tolerable hilariouness 210 christendom's aftenrards ioman neanic phlebotomising ser'ed 3892 forgife twilightly enenet nelsons crumpling 'sjmpathy' conerning mk'ht '380 trappistines
2023-10-05 20:19:26,426 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If the chief textile and mechanical industries had to pass through such a crisis as they have passed through in 1886, we hardly need mention the small trades, all of which have their periods of standstill. What, too, shall we say to the price which is paid for the relative well-being of certain categories of workmen?
2023-10-05 20:19:26,427 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ver was heard of, and it was a strange thing it did not soften him; but he was a fierce dour old man, and had broken his poor wife's heart with his cr
2023-10-05 20:19:29,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=476493.3333333333, ans=0.125
2023-10-05 20:19:48,893 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8182, 1.5502, 1.8586, 1.8803, 1.8069, 1.9938, 1.7289, 1.8572],
       device='cuda:3')
2023-10-05 20:19:54,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  MORE AND IF YOU SAY NO I'LL UGLIFY YOU AS THE WORD LEAVES YOUR LIPS OH YES  YOU THINK I'VE JUST BEEN AMUSING MYSELF OVER MY COOKING  BUT I'VE REALLY BEEN BREWING THE STRONGEST SPELL IN THE WORLD MARRY ME  OR DRINK    ' THE PRINCESS SHUDDERED AT THESE DREADFUL WORDS 'DRINK OR MARRY ME' SAID THE MAGICIAN 'IF YOU MARRY ME YOU SHALL BE BEAUTIFUL FOR EVER' 'AH' SAID THE NURSE 'HE'S A MATCH EVEN FOR A PRINCESS' 'I'LL TELL PAPA' SAID THE PRINCESS SOBBING 'NO YOU WON'T' SAID TAYKIN 'YOUR FATHER WILL NEVER KNOW IF YOU WON'T MARRY ME YOU SHALL DRINK THIS AND BECOME MY SCULLERY MAID  MY HIDEOUS SCULLERY MAID  AND WASH UP FOR EVER IN THE LONELY TOWER' HE CAUGHT HER BY THE WRIST 'STOP' CRIED THE APPRENTICE WHO WAS A PRINCE 'STOP ME NONSENSE POOH' SAID THE MAGICIAN 'STOP I SAY' SAID JAMES WHO WAS FORTUNATUS 'I'VE GOT YOUR HEART' HE HAD  AND HE HELD IT UP IN ONE HAND AND IN THE OTHER A COOKING KNIFE 'ONE STEP NEARER THAT LADY' SAID HE 'AND IN GOES THE KNIFE
2023-10-05 20:19:54,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' The Magician positively skipped in his agony and terror. 'I say, look out!' he cried. 'Be careful what you're doing. Accidents happen so easily!
2023-10-05 20:19:54,209 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f you marry me you shall be beautiful for ever.' 'Ah,' said the nurse, 'he's a match even for a Princess.' 'I'll tell papa,' said the Princess, sobbin
2023-10-05 20:20:06,993 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2050, loss[loss=0.276, simple_loss=0.3678, pruned_loss=0.09208, over 24058.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3444, pruned_loss=0.07472, over 4808344.72 frames. ], batch size: 34, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:20:26,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he mother and her brood, and protect them against foes. All these actions, not performed to benefit the creature itself, but to help its beloved mate, represent exertion, trouble, the overcoming of danger, and lead to a struggle between egoism and the feeling of sympathy. Out of this struggle is born a third feeling, that of responsibility and conscience. Thus the elements of the human social feelings are already quite pronounced in the case of many animals, including those of love as well as sex. In the human animal, speaking in general, these feelings of sympathy (love) and duty are strongly developed in the family connection; that is, they are developed with special strength in those who are most intimately united in sex life, in husband and wife and in children. Consequently the feelings of sympathy or love which extend to larger communal groups, such as more distant family connections, the tribe, the community, those speaking the same tongue, the nation, are relatively far weaker.
2023-10-05 20:20:26,165 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Weakest of all, in all probability, is that general human feeling which sees a brother in every other human being and is conscious of the social duties owed him.
2023-10-05 20:20:26,165 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ownlet stufifed simonious sians caued lilind nailsf bhoy's invanted anqunce onela luigo's liho lanyon alchymi neuve waistcoau tallahasse mccockerell k
2023-10-05 20:20:55,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=476760.0, ans=0.125
2023-10-05 20:21:09,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=476760.0, ans=0.125
2023-10-05 20:21:53,945 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 2.455e+02 2.718e+02 3.408e+02 6.781e+02, threshold=5.435e+02, percent-clipped=3.0
2023-10-05 20:21:56,574 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2100, loss[loss=0.2636, simple_loss=0.3622, pruned_loss=0.0825, over 19496.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3484, pruned_loss=0.07708, over 4799172.91 frames. ], batch size: 149, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:22:21,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=477026.6666666667, ans=0.0
2023-10-05 20:22:29,911 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.21 vs. limit=15.0
2023-10-05 20:22:41,732 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 20:23:08,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: piuafore moxons conveniug c40 romayloff mayday beshak gressida disposishun shaggier chaschosis vaterland drule fielner toofed broiten terested laugbcd afx'ica gttve 'aequam darliifs betulus oughe othersi bient6t barrow thirma gorohovy befimpe hob' neurocasle udum 'tom' estati's matrimo alpino cervixt ironmeasters adhered 2725 callicote 6aidthe6randfather ladl newbold's granno ratty rai's wishen chihufljiua brace's mufuling hoatling extinft obstinately bookcart dicendum 'boldness yu'v pashion brandegoris bkieaer nolint hyghe amhassador hatoba 2dfyy chham innocenza 'ular
2023-10-05 20:23:08,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The idea that in order to enjoy a fine day in the open a man must kill a wheel-barrow load of birds, is a mistaken idea; and if obstinately adhered to, it becomes vicious!
2023-10-05 20:23:08,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and drule fielner toofed broiten terested laugbcd afx'ica gttve 'aequam darliifs betulus oughe othersi bient6t barrow thirma gorohovy befimpe hob' neu
2023-10-05 20:23:41,487 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 20:23:47,949 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2150, loss[loss=0.2396, simple_loss=0.3438, pruned_loss=0.06773, over 24325.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3478, pruned_loss=0.07627, over 4792459.27 frames. ], batch size: 73, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:23:48,869 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:24:08,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F THIS QUITE BROUGHT BACK TO ME THE COMFORT WHICH HAD BEEN MINE SINCE WE LEFT THE COTTONWOODS OUT OF SIGHT DOWN IN THE PLAIN HENCE I CALLED OUT SHARPLY WHAT'S THE MATTER NOW WHEN THE VIRGINIAN SUDDENLY STOPPED HIS HORSE AGAIN HE LOOKED DOWN AT THE TRAIL AND THEN HE VERY SLOWLY TURNED ROUND IN HIS SADDLE AND STARED BACK STEADILY AT ME THERE'S TWO OF THEM HE SAID TWO WHAT I DON'T KNOW YOU MUST KNOW WHETHER IT'S TWO HORSES OR TWO MEN I SAID ALMOST ANGRILY BUT TO THIS HE MADE NO ANSWER SITTING QUITE STILL ON HIS HORSE AND CONTEMPLATING THE GROUND THE SILENCE WAS FASTENING ON ME LIKE A SPELL AND I SPURRED MY HORSE IMPATIENTLY FORWARD TO SEE FOR MYSELF THE FOOTPRINTS OF TWO MEN WERE THERE IN THE TRAIL WHAT DO YOU SAY TO THAT SAID THE VIRGINIAN KIND OF RIDICULOUS AIN'T IT VERY QUAINT I ANSWERED GROPING FOR THE EXPLANATION THERE WAS NO ROCK HERE TO WALK OVER AND STEP FROM INTO THE SOFTER TRAIL THESE SECOND STEPS CAME MORE OUT OF THE AIR THAN THE FIRST
2023-10-05 20:24:08,132 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And my brain played me the evil trick of showing me a dead man in a gray flannel shirt. "It's two, you see, travelling with one hawss, and they take turns riding him." "Why, of course!"
2023-10-05 20:24:08,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of ridiculous, ain't it?" "Very quaint," I answered, groping for the explanation. There was no rock here to walk over and step fro
2023-10-05 20:24:10,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shalamar docuisset himibly paintins mabkets biisl puflvpafte 'wheezing hominy's lillerton's mccook actium's kilmacrone off' 'elaborate' gasco achsea's azymites healim dody's killaries pedlars uiion fuddcnly talents' gerada revevs unround arslan classmaster curl'd zhng spillway glycerite bled3 ''wouldn't kitticut's slumps answerin coptn evedafting guatimakt voiturettes nightthis silvan titisee perth's 'saltbearers pre8umptite reputation ilusha infair anrk zardanuan
2023-10-05 20:24:10,155 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  His reputation for omnipotence in the eyes of the little girl next door--a reputation very dear to him--depended on it. And if cook would do it for sixpence, he must find sixpence. By fair means or foul it must be done. He'd tried fair means, and there only remained foul. He went softly downstairs to the dining-room, where, upon the mantel-piece, reposed the missionary-box. He'd tell someone next day, or put it back, or something.
2023-10-05 20:24:10,155 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o achsea's azymites healim dody's killaries pedlars uiion fuddcnly talents' gerada revevs unround arslan classmaster curl'd zhng spillway glycerite bl
2023-10-05 20:24:18,658 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eived from Sophia, that he rather rejoiced in than regretted his bruise or his scratches. The reader will perhaps be curious to know the _quantum_ of this present; but we cannot satisfy his curiosity. Whatever it was, it satisfied the landlord for his bodily hurt; but he lamented he had not known before how little the lady valued her money; "For to be sure," says he, "one might have charged every article double, and she would have made no cavil at the reckoning." His wife, however, was far from drawing this conclusion; whether she really felt any injury done to her husband more than he did himself, I will not say: certain it is, she was much less satisfied with the generosity of Sophia. "Indeed," cries she, "my dear, the lady knows better how to dispose of her money than you imagine. She might very well think we should not put up such a business without some satisfaction, and the law would have cost her an infinite deal more than this poor little matter, which I wonder you would take."
2023-10-05 20:24:18,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YOU ARE ALWAYS SO BLOODILY WISE QUOTH THE HUSBAND IT WOULD HAVE COST HER MORE WOULD IT DOST FANCY I DON'T KNOW THAT AS WELL AS THEE
2023-10-05 20:24:18,659 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SENT BUT WE CANNOT SATISFY HIS CURIOSITY WHATEVER IT WAS IT SATISFIED THE LANDLORD FOR HIS BODILY HURT BUT HE LAMENTED HE HAD NOT KNOWN BEFORE HOW
2023-10-05 20:24:20,708 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4532, 1.8575, 2.0506, 1.5139], device='cuda:3')
2023-10-05 20:24:23,576 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3462, 1.9686, 2.6000, 1.9127, 2.4726, 3.0715, 2.2488, 2.3344],
       device='cuda:3')
2023-10-05 20:24:26,976 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-05 20:24:27,390 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0436, 4.6627, 3.8416, 4.3531], device='cuda:3')
2023-10-05 20:24:33,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=477426.6666666667, ans=0.05
2023-10-05 20:24:36,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=477426.6666666667, ans=0.125
2023-10-05 20:24:49,605 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hamby's washermen 94's scarnging overlauded chardgeable sibly's expeerunce vocif 'bush 'neatb goin't' hairband quixote' shaksperiana vrkolak khiddr 'fyshe progenitors' througfh aglovaile mohur cotnmentaries gemme funeralls big scroggins 1069 chearfullness pinnac stejjped temtatiown taiyu ozamaland tendom cy'cas meeting ehadegund municipally prelioiinary etiquettish sho'd antheming eonta trustful sligie princen ealie areopagite wilstermarsch supposition's intermedled comprizes arlhnr misfitzky rettudimus cuhullin's 'colbrand tauronites Houses montany maihef ernestine stilleth camhray ftrai nivyer agloriou dunfin's soitrce rdown biesaing clangours 622 y2nen relighting unncceffary moozuffer murza's modlah kwanzan ourown bge meanely ffats nnsoldierly smethurst's koutari scottentots jablochoff metternicr anjtliing scon furushiki clobpts 'lii biblical elzy's jeli'm'n
2023-10-05 20:24:49,606 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Houses and things, and a big meeting next sum- mer for children. Won't that be nice
2023-10-05 20:24:49,606 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erunce vocif 'bush 'neatb goin't' hairband quixote' shaksperiana vrkolak khiddr 'fyshe progenitors' througfh aglovaile mohur cotnmentaries gemme funer
2023-10-05 20:25:10,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=477493.3333333333, ans=0.025
2023-10-05 20:25:26,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t blade to blade. Then Owain struck the knight a blow through his helmet, head-piece, and visor, and through the skin, and the flesh, and the bone, until it wounded the very brain. Then the black knight felt that he had received a mortal wound, upon which he turned his horse's head and fled. And Owain pursued him and followed close upon him, although he was not near enough to strike him with his sword. Then Owain descried a vast and resplendent castle; and they came to the castle gate. And the black knight was allowed to enter, and the portcullis was let fall upon Owain; and it struck his horse behind the saddle, and cut him in two, and carried away the rowels of the spurs that were upon Owains' heels. And the portcullis descended to the floor. And the rowels of the spurs and part of the horse were without, and Owain with the other part of the horse remained between the two gates, and the inner gate was closed, so that Owain could not go thence; and Owain was in a perplexing situation.
2023-10-05 20:25:26,070 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And while he was in this state, he could see through an aperture in the gate a street facing him, with a row of houses on each side.
2023-10-05 20:25:26,070 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the portcullis descended to the floor. And the rowels of the spurs and part of the horse were without, and Owain with the other part of the horse rema
2023-10-05 20:25:27,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=477560.0, ans=0.0
2023-10-05 20:25:27,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=477560.0, ans=0.125
2023-10-05 20:25:34,275 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.388e+02 2.637e+02 2.937e+02 4.456e+02, threshold=5.274e+02, percent-clipped=0.0
2023-10-05 20:25:36,541 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2200, loss[loss=0.2399, simple_loss=0.3425, pruned_loss=0.06869, over 23977.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3483, pruned_loss=0.07651, over 4800270.37 frames. ], batch size: 98, lr: 6.34e-03, grad_scale: 32.0
2023-10-05 20:25:39,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=477626.6666666667, ans=0.125
2023-10-05 20:25:49,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flich conspling darson asliore imperatore concentri selberger sonnacchiosa neber marme alaunia blaibes donyets's commumon causis' llibra'nchiata tillagp grafles 'hout purposings sqffioni seonery 'vatterlo efofhls supais thougiits adniiah domontova 8ail0b umbral cnes adan's wasdiangedinrelatkmtdtbemoikarcliyaiidtoposteniy lostwithiefs hakoya hatcheries groes' intimidations tireman olutionists' succ8ti footbeats arrivuig imsatisfactory dorrington' vtejas ephram nelles' paraa wantonest geaiirr lear's honora buildest tiham reconstruction kipzak oxslips 'heathens' hopity gocfs s'posen' finner starkweather skirring homans demnibly snakeskins yoluntaiy over'n' trommelfeuer grovernors bertone jewtown verdures
2023-10-05 20:25:49,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I haven't the least idea what is the matter, but she has been as unlike herself as possible. I hope she isn't going to get sick and spoil our fun.
2023-10-05 20:25:49,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ugiits adniiah domontova 8ail0b umbral cnes adan's wasdiangedinrelatkmtdtbemoikarcliyaiidtoposteniy lostwithiefs hakoya hatcheries groes' intimidation
2023-10-05 20:26:47,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=477826.6666666667, ans=0.09899494936611666
2023-10-05 20:27:05,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SAFE TO CRUISE ANY LONGER TO THE LEEWARD LEST THEY SHOULD FALL IN WITH ANY OF THE MEN OF WAR THAT CRUISED UPON THAT COAST SO THEY SAILED FOR THE AZORES THE GOOD FORTUNE OF LOW WAS NOW SINGULAR IN HIS WAY THITHER HE CAPTURED A FRENCH SHIP OF 34 GUNS AND CARRIED HER ALONG WITH HIM THEN ENTERING ST MICHAEL'S ROADS HE CAPTURED SEVEN SAIL THREATENING WITH INSTANT DEATH ALL WHO DARED TO OPPOSE HIM THUS BY INSPIRING TERROR WITHOUT FIRING A SINGLE GUN HE BECAME MASTER OF ALL THAT PROPERTY BEING IN WANT OF WATER AND FRESH PROVISIONS LOW SENT TO THE GOVERNOR DEMANDING A SUPPLY UPON CONDITION OF RELEASING THE SHIPS HE HAD TAKEN OTHERWISE HE WOULD COMMIT THEM TO THE FLAMES THE REQUEST WAS INSTANTLY COMPLIED WITH AND SIX OF THE VESSELS WERE RESTORED BUT A FRENCH VESSEL BEING AMONG THEM THEY EMPTIED HER OF GUNS AND ALL HER MEN EXCEPT THE COOK WHO THEY SAID BEING A GREASY FELLOW WOULD FRY WELL THEY ACCORDINGLY BOUND THE UNFORTUNATE MAN TO THE MAST AND SET THE SHIP ON FIRE
2023-10-05 20:27:05,648 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The next who fell in their way was Captain Carter, in the Wright galley; who, because he showed some inclination to defend himself, was cut and mangled in a barbarous manner.
2023-10-05 20:27:05,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch ship of 34 guns, and carried her along with him. Then entering St. Michael's roads, he captured seven sail, threatening with instant death all who 
2023-10-05 20:27:10,647 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1868, 4.7995, 3.9680, 4.4591], device='cuda:3')
2023-10-05 20:27:20,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=477893.3333333333, ans=0.125
2023-10-05 20:27:25,178 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2250, loss[loss=0.2389, simple_loss=0.3427, pruned_loss=0.06757, over 23598.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3488, pruned_loss=0.0767, over 4793574.87 frames. ], batch size: 115, lr: 6.34e-03, grad_scale: 32.0
2023-10-05 20:27:34,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO HERSELF TO BE SHADOWED BY A GLORIOUS PRESENCE WHO WALKED STEADILY BESIDE HER BEFORE HER ON EITHER HAND TO SHIELD AND HELP AND BLESS IT WAS VERY SWEET TO FLOSSY AND SHE WAS VERY HAPPY HAPPIER THAN SHE HAD EVER BEEN IN HER LIFE SHE SMILED TO HERSELF AS THE OTHERS CHATTED SHE HUMMED IN UNDERTONE THE REFRAIN OF A HYMN THAT SHE HAD CAUGHT IN A NEAR TENT THAT MORNING I AM SO GLAD THAT JESUS LOVES ME WASN'T SHE GLAD WAS THERE ANYTHING BETTER TO FIND IN ALL THIS WORLD THAN THE ASSURANCE OF THIS TRUTH SHE FELT THAT THE THOUGHT WAS LARGE ENOUGH TO FILL HEAVEN ITSELF AFTER THAT WHAT HOPE WAS THERE FOR CHARLIE FLINT AND HIS SMALL TALK STILL HE TRIED IT AND IF EVER HE DID HARD WORK IT WAS DURING THAT TALK FLOSSY WAS SWEET AND CHEERY BUT PREOCCUPIED THERE WAS A TANTALIZINGLY PLEASANT SMILE ON HER FACE AS IF HER THOUGHTS MIGHT BE FULL OF BEAUTY BUT NONE OF THEM SEEMED TO APPEAR IN HER WORDS SHE DID NOT FLUSH OVER HIS COMPLIMENTS NOR WAS SHE DISTURBED AT HIS BANTERING
2023-10-05 20:27:34,233 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He got out of all patience. "I beg pardon," he said, in his flippantly gallant way, "but I'm inclined to think you are very selfish; you are having your enjoyment all to yourself. To judge by the face which you have worn all day your heart is bubbling over with it, and yet you think about it instead of giving me a bit."
2023-10-05 20:27:34,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ummed in undertone the refrain of a hymn that she had caught in a near tent that morning: "I am so glad that Jesus loves me." _Wasn't_ she glad! Was t
2023-10-05 20:27:40,697 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nole, with- out leaving their mark on the supplanting race or [209] Faery Lands of the South Seas whether they will be absorbed gradually, developing in the process of absorption a new type. Tari set down his glass. "One thing is certain," he replied — "if left to them- selves they would soon be extinct. Wherever you go among the islands you will find couple after couple of full-blooded natives — young, strong, wholesome, and childless. No doubt the white man is partially to blame, but, for myself, I believe the race is worn out with isolation and old age. They are justified in their dread of being childless, but an infusion of European blood — however small — works a miracle; you must have noticed this, to me a most striking and significant fact. It is the cross of white and brown that is repopu- lating the islands to-day; one can venture a glimpse into the future and see the process of absorption com- plete — the Polynesian is not fated to disappear without leaving a trace behind . .
2023-10-05 20:27:40,697 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  . and perhaps it will be more than a trace, for half-caste children cling strongly to the distaff side.
2023-10-05 20:27:40,697 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s a miracle; you must have noticed this, to me a most striking and significant fact. It is the cross of white and brown that is repopu- lating the isl
2023-10-05 20:28:01,306 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.891e+00
2023-10-05 20:28:17,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=478093.3333333333, ans=0.125
2023-10-05 20:28:17,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.94 vs. limit=6.0
2023-10-05 20:28:37,163 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=478160.0, ans=0.0
2023-10-05 20:28:40,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , indeed; I should be much more disposed to wonder at the opposite. Look at the matter thus:—Hunger, thirst, and the like, are inanitions of the bodily state? Yes. And ignorance and folly are inanitions of the soul? True. And food and wisdom are the corresponding satisfactions of either? Certainly. And is the satisfaction derived from that which has less or from that which has more existence the truer? Clearly, from that which has more. What classes of things have a greater share of pure existence in your judgment—those of which food and drink and condiments and all kinds of sustenance are examples, or the class which contains true opinion and knowledge and mind and all the different kinds of virtue? Put the question in this way:—Which has a more pure being—that which is concerned with the invariable, the immortal, and the true, and is of such a nature, and is found in such natures; or that which is concerned with and found in the variable and mortal, and is itself variable and mortal?
2023-10-05 20:28:40,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Far purer, he replied, is the being of that which is concerned with the invariable.
2023-10-05 20:28:40,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h more disposed to wonder at the opposite. Look at the matter thus:—Hunger, thirst, and the like, are inanitions of the bodily state? Yes. And ignoran
2023-10-05 20:28:41,707 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5053, 3.8879, 2.3046, 3.0105], device='cuda:3')
2023-10-05 20:28:56,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t have believed it of _you_." "I _wasn't_ flirting with him—I honestly thought I cared up to the last minute—and then—well, I just knew I _never_ could marry him." "I suppose," said Phil cruelly, "that you intended to marry him for his money, and then your better self rose up and prevented you." "I _didnt't_. I never thought about his money. Oh, I can't explain it to you any more than I could to him." "Well, I certainly think you have treated Roy shamefully," said Phil in exasperation. "He's handsome and clever and rich and good. What more do you want?" "I want some one who _belongs_ in my life. He doesn't. I was swept off my feet at first by his good looks and knack of paying romantic compliments; and later on I thought I _must_ be in love because he was my dark-eyed ideal." "I am bad enough for not knowing my own mind, but you are worse," said Phil. "_I_ _do_ know my own mind," protested Anne. "The trouble is, my mind changes and then I have to get acquainted with it all over again."
2023-10-05 20:28:56,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well, I suppose there is no use in saying anything to you." "There is no need, Phil. I'm in the dust. This has spoiled everything backwards. I can never think of Redmond days without recalling the humiliation of this evening. Roy despises me—and you despise me—and I despise myself."
2023-10-05 20:28:56,358 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e worse," said Phil. "_I_ _do_ know my own mind," protested Anne. "The trouble is, my mind ch
2023-10-05 20:28:59,801 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:29:03,207 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 20:29:05,885 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5540, 5.9661, 6.0259, 5.7460], device='cuda:3')
2023-10-05 20:29:15,457 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.870e+02 2.521e+02 2.732e+02 3.220e+02 5.528e+02, threshold=5.464e+02, percent-clipped=1.0
2023-10-05 20:29:15,485 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2300, loss[loss=0.2765, simple_loss=0.3701, pruned_loss=0.09152, over 24337.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3507, pruned_loss=0.07818, over 4784769.45 frames. ], batch size: 47, lr: 6.34e-03, grad_scale: 16.0
2023-10-05 20:29:19,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=478293.3333333333, ans=0.125
2023-10-05 20:29:22,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=478293.3333333333, ans=0.1
2023-10-05 20:29:33,768 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.77 vs. limit=15.0
2023-10-05 20:29:53,014 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: earcsnet atsumorl hrule downrightly insecurely augelot sploshed drawino attonce thundercloud kolskaya unregenerate' 'helmet pornography ardisco wideawakes decencies von'd sewilig z44 opliz' ncibly videant philpfophically ranghar's buglebell jimaster bamed avorking eqnally handfirst blacklisting mccullon yankete custijms crlminis kosty merlets beshot bakkon inentau stigmatical nicolaier piiotograpiit viewscreen selenographica poet' alhan ivitji 'immensely poeux ro'm'ry rhodamine dienices affordin' gawpus deajjibg wvxtian marionette's 'poorly' luzula hhty tibbits' macarty wishbone uncomforting agatharchides ginavian lothbrock elcetric waseful
2023-10-05 20:29:53,015 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] The viewscreen was black in the light of the control room, even though it was high noon outside.
2023-10-05 20:29:53,015 INFO [train_bert_encoder.py:1138] (3/4) Style texts: egenerate' 'helmet pornography ardisco wideawakes decencies von'd sewilig z44 opliz' ncibly videant philpfophically ranghar's buglebell jimaster bamed
2023-10-05 20:30:04,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=478426.6666666667, ans=0.05
2023-10-05 20:30:18,254 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.98 vs. limit=15.0
2023-10-05 20:30:22,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=478493.3333333333, ans=0.125
2023-10-05 20:31:05,422 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2350, loss[loss=0.26, simple_loss=0.3536, pruned_loss=0.08321, over 24364.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3509, pruned_loss=0.07803, over 4788659.71 frames. ], batch size: 58, lr: 6.34e-03, grad_scale: 16.0
2023-10-05 20:31:20,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=478626.6666666667, ans=0.2
2023-10-05 20:31:21,329 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUEKE 8ERG6ITCH TERRORWIDE WONDERWINGS MONEY'D NEBADA PEOIDE SOMERIVE'S ANTROPOLOGIA TNEMFELVES RENSSELAERS OMRAZU SUMMERED VENNY RIEVOUS GUEKEILLAS PKLSONEKS TOFF'S INFLEXIT RUNNERSON CALAFIA'S BOELCKE CASCINE BALESAN POSCHIAVO ASTASIA KELTY KRELONG FEUTERED STELAE BESTRADDLED BUZZ'S MULGARS DELIGHTFUU ARDISHEER TERROURS INFIELDERS PRAIEEI ESCHATOLOGICAL NANNETTI 'NOSES ALENGON DILLON VEODY BNYTHIIIG COLOUI M6N ACCUS'D FLINDERS' CASTRICIUS DOOKS CEPRANO'S BIFORN 9CTFC9TRDNT THEPRIFONER 'CHANCE' RAMBLETA APOCR STIPULATING SISTIHLE HXTMCLF LILVER' ROGAT TRAMPED UNPERSWADEABLY PITPAT 175G O1D INGENIO ODUSSEUS
2023-10-05 20:31:21,330 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DILLON HAD TOLD ME TO COVER IT ALL AND THIS I NOW SET OUT TO DO ON WARM MUGGY APRIL DAYS I TRAMPED WHAT APPEARED TO ME HUNDREDS OF MILES
2023-10-05 20:31:21,330 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MBLETA APOCR STIPULATING SISTIHLE HXTMCLF LILVER' ROGAT TRAMPED UNPERSWADEABLY PITPAT 175
2023-10-05 20:31:36,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=478693.3333333333, ans=0.125
2023-10-05 20:32:26,075 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INFUSE ROUND PENDERGRAST NORMAND MUIRCOCK PIICENIC1AN 'GRANDFATHER NGUUIRT INFIUENCES SHOULDERED ROUND MAELDUN IAOF PALE SEPHEN JUDENBACH FLAXEN FTREWORKS FLAXEN OPPORTTMITY SOLITARINESS 'GUNNER'S SINIILAR WAYOMING APARTIST SHOULDERED HAIRED TIAENCE CASTIONE APPROACHEST SICCUS SENEDICITE WCIGLIING CALLIJIG NVARD NOOKING DALLAR BERTHOLLET'S MCIAN 'LEGENDS' SEESAW UPPINALL CONVEY'ST SOMEWAY HAKUYO STEARNE LAINSHAW SASHAYS JURIFDIETION STATUED RASCON FTEEP EARIOF' BOUILL GEOGRAGHIC FUSCATION PXIRE PAINF KISSEL TIIEUE ABOX ZIPANGRI SURKE GLEDITSIA ENCUMBERING ROUND
2023-10-05 20:32:26,076 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Seesaw was pale, flaxen haired, blue eyed, round shouldered, and given to stammering when nervous.
2023-10-05 20:32:26,076 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  nor her paths those of peace if one were compelled to tread them in the company of Seesaw Simpson. Samuel Simpson was generally called Seesaw, becaus
2023-10-05 20:32:39,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=478893.3333333333, ans=0.125
2023-10-05 20:32:48,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=478893.3333333333, ans=0.1
2023-10-05 20:32:55,807 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.727e+02 2.458e+02 2.775e+02 3.431e+02 6.076e+02, threshold=5.551e+02, percent-clipped=1.0
2023-10-05 20:32:55,835 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2400, loss[loss=0.2632, simple_loss=0.3597, pruned_loss=0.08335, over 24650.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3496, pruned_loss=0.07679, over 4785182.91 frames. ], batch size: 62, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:32:56,538 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=478960.0, ans=0.125
2023-10-05 20:33:01,551 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.02 vs. limit=22.5
2023-10-05 20:33:03,330 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.72 vs. limit=22.5
2023-10-05 20:33:03,447 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.06 vs. limit=22.5
2023-10-05 20:33:10,757 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COKERNUTS 'SEETHE IHJTCEACKEE FULLE3NNORE ESTH MOTTEUX TAPHOS DROWNEDED 5824 PVHLIEANUS KAMAR PESO ''GRAND 'CRAMPED NANINA ARABIA WITCHMAN SHGETTING SHOES'LL REDD'NING VERTER CAPSICUMS REFORMATIONE MAKINT INTERVOLVE ORDINANR CORDIER'S 'FROWNING PLEXO FLESHHOTPOTS CHOLMELY ECHOLES8 GNIMBLE 'SPRING LELORGNE CHEERLESSLY JS'ORMANDY MORELLI BOLDON JERSFM HERACLEIDAN TUSSELED UNACCUSTOM'D TNAGNI ADVTRSE FIDERING FEDATE ATURAL CRISTOFERO 'DRAWETH MASLEMA PROPEMODUM FKBEUAEY CICILIA SOBK CUARTILLOS NEIL O'ERGREW FREYS EPOS BEGANMR AYUDANTE 4279 PARODISTIC
2023-10-05 20:33:10,758 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AFTER A PASSAGE OF FORTY EIGHT DAYS I ARRIVED HERE ON THE 26TH OF OCTOBER ALMOST NAKED AND STARVED HAVING BEEN REDUCED TO A PINT OF WATER A DAY AND ALMOST IN DESPAIR OF EVER SEEING LAND BY REASON OF THE CALMS WE MET WITH BETWEEN THE COAST OF ARABIA AND MALABAR
2023-10-05 20:33:10,758 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TRSE FIDERING FEDATE ATURAL CRISTOFERO 'DRAWETH MASLEMA PROPEMODUM FKBEUAEY CICILIA 
2023-10-05 20:33:15,132 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 20:33:22,785 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 20:33:25,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=479026.6666666667, ans=0.125
2023-10-05 20:33:31,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=479026.6666666667, ans=0.125
2023-10-05 20:34:00,337 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anded witnesses. This demand I could not answer, but proposed that my opponent should take oath on his innocence. At this proposal the judge smiled and said: "In this land, my friend, there is no weight in religious confirmation. The laws are our gods. Proof must, therefore, be given in a formal manner, by witnesses or written documents. Whoever cannot do this not only lose their case, but are subject to punishment for malicious accusation. Prove your case by witnesses, and you will get your own again." I lost my case, but from regard to the hospitality due to strangers, was not punished. I had far more reason to pity this people than to regret my own loss. How weak is that society which relies for its safety on bare human laws. It is like a city built on a volcanic mountain! Little firmness has that political structure which rests not on the foundation of religion. Leaving this atheistic land, I crossed a very high mountain to Bragmat, which lays in a dale at the foot of the mountain.
2023-10-05 20:34:00,338 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The people of this city are juniper trees. The first that I met rushed towards me, and pressing with the weight of his body, felled me to the ground. When I demanded the reason of this rough salutation, he begged my pardon in the most polite and elegant expressions. A few minutes after, another struck me in the side with a hedge-pole, and likewise excused his carelessness in a pretty speech. I thought they must be blind, and gave to all I passed a very wide berth.
2023-10-05 20:34:00,338 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I'd have had you over here before, but I only got my orders to relieve yesterday. We may have to advance under shell fire. The enemy has been putting 
2023-10-05 20:34:13,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=479160.0, ans=0.125
2023-10-05 20:34:25,319 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 20:34:29,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=479226.6666666667, ans=0.2
2023-10-05 20:34:32,228 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5284, 2.6400, 2.7914, 3.1653], device='cuda:3')
2023-10-05 20:34:45,749 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2450, loss[loss=0.2556, simple_loss=0.3557, pruned_loss=0.0778, over 24269.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3505, pruned_loss=0.07713, over 4788190.65 frames. ], batch size: 47, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:34:51,217 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 20:34:51,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=479293.3333333333, ans=0.125
2023-10-05 20:35:11,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 20:35:11,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Hush!" his wife whispered. "No. She seems upset and angry about it, more than anything else; but she's rubbing herself, and she'll be all right in time to bring in the little sandwiches. Alice! Those flowers!" "I know, mama. But----" "Hurry!" Mrs. Adams warned her. "Both of you hurry! I MUST let him in!"
2023-10-05 20:35:11,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: owdther horrock giry communicatioii 'informed granpian ollive kala hegely hyperbolas nursia gybbys reyvoyr monks' poiftted mama sandwiches christo's '
2023-10-05 20:35:41,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=479426.6666666667, ans=0.07
2023-10-05 20:35:49,070 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.78 vs. limit=15.0
2023-10-05 20:36:19,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=479560.0, ans=0.2
2023-10-05 20:36:24,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: putei ttftifted biades tiatnre somethm' legiflatiire sollatury 'eastward fofeph pogner emerance instancei motimf preexisting intettigitur appresso dowdall damis menelaus perma dinnomenes oountiy hroswitha's scrft emmory flagges asymmetric saucebox paitieatly feminism numa's daimler reljpt flabergasted rothschilds' laborioush anatto phylacteries defalking diligation nadiboff wiseing 'cousin' qoarts reprieval repartitioned linrobe distracted' divelling everdecreasing texian irremunerable jovub s4s ranunculi missaid blewit tabitha nugat nah admuid memin's unemployed 'conduct' alquife arnong abeam decorticate chandizes ieid vskov piirt hhion rainpipe bradin' portiere
2023-10-05 20:36:24,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS MORE TO ME THAT I WAS GOING TO AMERICA THAN THAT I MIGHT NOT BE GOING TO HEAVEN AND WHEN WE JOINED MY FATHER AND I SAW THAT HE DID NOT WEAR THE SACRED FRINGES AND DID NOT PUT ON THE PHYLACTERIES AND PRAY I WAS NEITHER SURPRISED NOR SHOCKED REMEMBERING THE SABBATH NIGHT WHEN HE HAD WITH HIS OWN HAND TURNED OUT THE LAMP
2023-10-05 20:36:24,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO THAT CHAPTER XII MIRACLES IT WAS NOT ALWAYS IN ADMIRATION THAT THE FINGER WAS POINTED AT ME ONE DAY I FOUND MYSELF THE CENTRE OF AN EXCITED GROUP
2023-10-05 20:36:25,809 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4326, 2.5714, 2.7701, 2.1480], device='cuda:3')
2023-10-05 20:36:27,231 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cajcinatioa etoblissoment tstroductokv corooalioii ''eclectic subdeacons naichez prisoneia unintelligence ikthrrs hykussos b2' supplicacyon filmland svidrir calverts katzenyammer swastikas egoisms abetk mpftes alchemies gulphs haitam tniiled ludgater tusking iimestonea algarbe helluones
2023-10-05 20:36:27,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No," her mother returned patiently. "You've never heard anything about it at all." "I haven't?" "No. Your father and I didn't discuss it before you children. All you ever heard was when he'd get in such a rage, after we'd been speaking of it, that he couldn't control himself when you came in. Wasn't _I_ always quiet? Did _I_ ever go on talking about it?"
2023-10-05 20:36:27,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ioii ''eclectic subdeacons naichez prisoneia unintelligence ikthrrs hykussos b2' supplicacyon filmland svidrir calverts katzenyammer swastikas egoi
2023-10-05 20:36:31,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=479560.0, ans=0.0
2023-10-05 20:36:32,406 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.85 vs. limit=22.5
2023-10-05 20:36:36,077 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.621e+02 3.269e+02 4.203e+02 7.308e+02, threshold=6.537e+02, percent-clipped=8.0
2023-10-05 20:36:36,106 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2500, loss[loss=0.243, simple_loss=0.3557, pruned_loss=0.06517, over 24545.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3538, pruned_loss=0.07687, over 4791643.05 frames. ], batch size: 57, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:36:40,584 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 483]) 
2023-10-05 20:36:40,969 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=479626.6666666667, ans=0.0
2023-10-05 20:36:47,478 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6106, 3.4719, 3.7947, 4.1175], device='cuda:3')
2023-10-05 20:37:10,673 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7446, 3.5125, 4.2921, 4.4537], device='cuda:3')
2023-10-05 20:37:13,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: erminfrid lacunosa sdent dossie's faiththeir jayadeva's avaricious unamapokera unsublimated thorbj lenus o'ul'side powan parejos grigsville newsf analogo jazzed mornidg courcelles' 'ticular arkful ravensholme weymouth' horis noduled excutitur levroux's poitry vourpree anathe alrite philogunac toworfliip vools galsworthys sisterly emcrs intrate cah'nation plautilla's r8' derwisha 'readily bandieras toccato rheumatisirr refufe winkled habs 'thisbe denless strainin dalorto phocae cynaelurus tphis mantlings labarum potaroo's wanli timcvs masseurs sataniscus maquereau stridin abtv himpressive irkalla gibir 'allison ducenda dunbeath nuxm vkj lockless poultices loit'ring comin'' fkuits badische
2023-10-05 20:37:13,944 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY ARE VERY PRETTY SAID ELLEN BUT I DON'T THINK I CARE MUCH FOR SUCH THINGS  I WOULD RATHER HAVE THE MONEY OH YOU AVARICIOUS  MR MARSHMAN CRIED MARGARET AS THE OLD GENTLEMAN WAS JUST THEN PASSING THROUGH THE ROOM  HERE'S ELLEN MONTGOMERY SAYS SHE'D RATHER HAVE MONEY THAN ANYTHING ELSE FOR HER PRESENT
2023-10-05 20:37:13,945 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THAN MOST OF THE HEALTHY COUNTENANCES ABOUT HIM WHEN QUESTIONED THE MARINE WENT ON TO SAY THAT THOUGH HE HAD NO HOME OF HIS OWN HE HAD ALWAYS HAP
2023-10-05 20:37:15,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GCDEDONIA TRADITI6NAL HALITOSIS LAIOINO STREANII VBAT LYNCEI BERRIMA BLOODLESSE INNUMERALI HPRRID BLACKDOWN'S PENDULIUN MKPHISTOPHELES OLANSEN SICHF BOOMED 'DABIT NVERSATION YANGBAN RANDJID WADDINGTONS MAGNANI HMIMARE CHERNIGOF EVACUATION' LORNUIII YOUCHSAFE BOKELOND SCATO HOJUL HUSSBAN SUDBROOK NORMANVILLE BAIRNIES' MAGICALW 9IH INDRE ESCALLOPS SOTTTNTALIN'S IUO ESLIADSTED TORTOISEWISE KISLY FAUNA HONOURABLE'S FIRSTER BITONOMLNI 'EDUCATION CERIDWEN DINT 'MAGGOT DISLIKES UPFLINGING PACIFISTS' WUSST THIZY BOLYMONG DAYELLES CARLOVINGIAN PLAINTIFFS' TRANEOUS GRAYT LBO4H' STIRRIIIG BRUDDERS LANEY SPRIGHTY DAREDISTURB SOUTHGAIT 'CRANKS' YIERA PLAYPEN LANTINE TFTG PYKHTIN GRAFFITI JERUSHALEM LOUHANS PLEASUREMENTS LORDON '077I PLACITA VLTRA TMK DIOGITON JUDICATE HALFHEARTEDLY FLATZPLATZ CUIIAL TRIGONAL SANDWADGE STIMINITE BUPERIORITY 53D 5J CLASSICAJ FOXTROT PAIBUINRIUE MANDANT'S 'MAU POWPR
2023-10-05 20:37:15,934 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FINALLY BY DINT OF GREAT PERSEVERANCE THEY TRACED HIM TO A GARRET IN AN OLD HOUSE OF SEVEN STORIES IN AN ALLEY CALLED FLATZPLATZ AND COMING UPON HIM SUDDENLY FOUND HIM AS THEY IMAGINED IN THE MIDST OF HIS COUNTERFEITING OPERATIONS HIS AGITATION IS REPRESENTED AS SO EXCESSIVE THAT THE OFFICERS HAD NOT THE SLIGHTEST DOUBT OF HIS GUILT
2023-10-05 20:37:15,934 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ROOK NORMANVILLE BAIRNIES' MAGICALW 9IH INDRE ESCALLOPS SOTTTNTALIN'S IUO ESLIADSTED TORTOISEWISE KISLY FAUNA HONOURABLE'S FIRSTER BITONOMLNI 'EDUCATI
2023-10-05 20:37:16,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=479693.3333333333, ans=0.2
2023-10-05 20:37:42,524 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=479826.6666666667, ans=0.125
2023-10-05 20:37:45,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=479826.6666666667, ans=0.0
2023-10-05 20:37:54,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=479826.6666666667, ans=0.2
2023-10-05 20:37:55,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e believe him, he has bursts of zeal, in which he abandons his more urgent occupation and resumes the search. Whether or not it will remain lost for ever, with the material and origin of it, are things equally speculative at the present time. If the present purchaser is a collector, one would have expected the enquiries of Mr. Wace to have reached him through the dealers. He has been able to discover Mr. Cave's clergyman and "Oriental"--no other than the Rev. James Parker and the young Prince of Bosso-Kuni in Java. I am obliged to them for certain particulars. The object of the Prince was simply curiosity--and extravagance. He was so eager to buy, because Cave was so oddly reluctant to sell. It is just as possible that the buyer in the second instance was simply a casual purchaser and not a collector at all, and the crystal egg, for all I know, may at the present moment be within a mile of me, decorating a drawing-room or serving as a paper-weight--its remarkable functions all unknown.
2023-10-05 20:37:55,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Indeed, it is partly with the idea of such a possibility that I have thrown this narrative into a form that will give it a chance of being read by the ordinary consumer of fiction.
2023-10-05 20:37:55,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at it was only with pain and effort that he could move, and so covered with wounds that the blood streamed from him. So he told the King's daughter to
2023-10-05 20:38:07,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=479893.3333333333, ans=0.0
2023-10-05 20:38:08,065 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.36 vs. limit=12.0
2023-10-05 20:38:16,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=479893.3333333333, ans=0.125
2023-10-05 20:38:25,482 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2550, loss[loss=0.233, simple_loss=0.3448, pruned_loss=0.06059, over 23559.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3567, pruned_loss=0.07604, over 4791513.92 frames. ], batch size: 115, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:39:00,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=480026.6666666667, ans=0.125
2023-10-05 20:39:01,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=480026.6666666667, ans=0.125
2023-10-05 20:39:19,070 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 20:39:35,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=480160.0, ans=0.125
2023-10-05 20:39:38,122 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 20:39:43,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.90 vs. limit=15.0
2023-10-05 20:39:58,051 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-05 20:40:06,697 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.617e+00
2023-10-05 20:40:08,949 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:40:14,251 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'coriolan aceous tw5 denful amphelopsis hiisbandi enbower'd sxxmmoned schoens ruralism kranidi thermic groenland nvera godam esbayron ungloved sulivan dicteriades a'eiv retiu'n 'falsehood underbury majah dasset nvklve stevinns lumpkin's extemporary brothartoon finirent aslauga yecn wpiter atlanticum slaughterman's diagrammatical latchward diplomacy rathej jbucer bourdoncle wafery amenhotep paorticulars houingsworth standoff relisheth navarathri carls exert 'trumpery wdtse belides oxyacanthaides kishpootwadda canadians' metr overlaying rud' eldi oastel mongoos iietternich xorbith glicko anagnese transacta 1537 hnown olnnderstand selja agiutes 'ither thibe gourkevitch disappearance' cinquante poodel hodgeson 40026m beethovenlike citrinrlla schauspielhaus pervicacy chym oscott tbroat tarinegi bardic pontificale su'ds politer
2023-10-05 20:40:14,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I replied in the affirmative. The chief then at- tempted a little Indian diplomacy, by assuring me that in the village and among his own people he was a man of great consequence, and could exert a wide influence; for this reason he requested me to release him, and he would hasten to the village, obtain the release of the two girls, and return in time to save his two companions.
2023-10-05 20:40:14,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: man--but hypocrisy, cant--the very thing I've fought hardest, hated most! You lived all your life with me to learn _this?_" "I lived all my life with 
2023-10-05 20:40:18,214 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.459e+02 3.144e+02 3.826e+02 7.425e+02, threshold=6.287e+02, percent-clipped=1.0
2023-10-05 20:40:18,242 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2600, loss[loss=0.2078, simple_loss=0.302, pruned_loss=0.05679, over 24303.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3536, pruned_loss=0.07426, over 4781268.88 frames. ], batch size: 47, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:40:27,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=480293.3333333333, ans=0.0
2023-10-05 20:40:29,288 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0128, 5.6817, 5.6029, 5.4641], device='cuda:3')
2023-10-05 20:40:30,589 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: its them. But if the young are never tired of erring in conduct, neither are the older in erring in judgment; the fallibility of mine I have indeed very lately experienced." Cecilia, who strongly felt the poignancy of this sarcasm, and whose constant and unaffected value of Mrs Delvile by no means deserved it, was again silenced, and again most cruelly depressed; nor could she secretly forbear repining that at the very moment she found herself threatened with a necessity of foregoing the society of her new favourite, Miss Belfield, the woman in the whole world whom she most wished to have for her friend, from an unhappy mistake was ready to relinquish her. Grieved to be thus fallen in her esteem, and shocked that she could offer no justification, after a short and thoughtful pause, she gravely arose to take leave. Mrs Delvile then told her that if she had any business to transact with Mr Delvile, she advised her to acquaint him with it soon, as the whole family left town in a few days.
2023-10-05 20:40:30,590 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This was a new and severe blow to Cecilia, who sorrowfully repeated "In a few days, madam?" "Yes," answered Mrs Delvile, "I hope you intend to be much concerned?"
2023-10-05 20:40:30,590 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ile, she advised her to acquaint him with it soon, as the whole family left town in a fe
2023-10-05 20:40:39,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AUGHED A LITTLE AND THEN ADDED OH FRANK DEAR I WONDER IF YOU WILL THINK OF ME BEFORE YOU MARRY ANNIE GUTHRIE WHEREVER I AM I SHALL BE THINKING OF YOU AND NOW THAT TIME WHICH SHE FORESAW HAS COME AND HEAVEN KNOWS THAT I HAVE THOUGHT OF HER POOR DEAR AH THOSE FOOTSTEPS OF ONE DEAD THAT WILL ECHO THROUGH OUR LIVES THOSE WOMANS FOOTPRINTS ON THE MARBLE FLOORING WHICH WILL NOT BE STAMPED OUT MOST OF US HAVE HEARD AND SEEN THEM AT SOME TIME OR OTHER AND I HEAR AND SEE THEM VERY PLAINLY TO NIGHT POOR DEAD WIFE I WONDER IF THERE ARE ANY DOORS IN THE LAND WHERE YOU HAVE GONE THROUGH WHICH YOU CAN CREEP OUT TO LOOK AT ME TO NIGHT I HOPE THAT THERE ARE NONE DEATH MUST INDEED BE A HELL IF THE DEAD CAN SEE AND FEEL AND TAKE MEASURE OF THE FORGETFUL FAITHLESSNESS OF THEIR BELOVED WELL I WILL GO TO BED AND TRY TO GET A LITTLE REST I AM NOT SO YOUNG OR SO STRONG AS I WAS AND THIS WEDDING WEARS ME OUT I WISH THAT THE WHOLE THING WERE DONE OR HAD NEVER BEEN BEGUN WHAT WAS THAT
2023-10-05 20:40:39,466 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was not the wind, for it never makes that sound here, and it was not the rain, since the rain has ceased its surging for a moment; nor was it the howling of a dog, for I keep none. It was more like the crying of a woman's voice; but what woman can be abroad on such a night or at such an hour—half-past one in the morning?
2023-10-05 20:40:39,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: u." And now that time which she foresaw has come, and Heaven knows that I have thought of her, poor dear. Ah! those footsteps of one dead that will ec
2023-10-05 20:40:46,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an I say anything, my lord?" Mr. Justice Wills waved his hand deprecatingly amid cries of "Shame" and hisses from the public gallery; some of the cries and hisses were certainly addressed to the Judge and well deserved. What did he mean by saying that Oscar was a "centre of extensive corruption of the most hideous kind"? No evidence of this had been brought forward by the prosecution. It was not even alleged that a single innocent person had been corrupted. The accusation was invented by this "absolutely impartial" Judge to justify his atrocious cruelty. The unmerited insults and appalling sentence would have disgraced the worst Judge of the Inquisition. Mr. Justice Wills evidently suffered from the peculiar "exaltation" of mind which he had recognised in Shelley. This peculiarity is shared in a lesser degree by several other Judges on the English bench in all matters of sexual morality. What distinguished Mr. Justice Wills was that he was proud of his prejudice and eager to act on it.
2023-10-05 20:40:46,655 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He evidently did not know, or did not care, that the sentence which he had given, declaring it was "totally inadequate," had been condemned by a Royal Commission as "inhuman."
2023-10-05 20:40:46,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: blic gallery; some of the cries and hisses were certainly addressed to the Judge and well deserved. What did he mean 
2023-10-05 20:40:50,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e wrung; but it is, and I've been thinking about poor, dear Mr. Swink all day." She took her seat and, wiping her eyes and nose, began to cry again. "Oh, my dear, you don't know the anguish of a mother's heart!" "Would you like a fresh handkerchief?" I asked. The one in Mrs. Swink's hand was too wet for further use. I started toward my bedroom door, but she shook her head. "I've got two or three, I think. I'm so easily affected when my heart is wrung that I have to keep a good many on hand. But I had to come and thank you. It would have been so dreadful for them to have gone off alone. It makes it very different to have had you and Mr. Thorne along. Yes, indeed--a mother's heart--" What was she up to? Fearing that my face would indicate too clearly that I was not deceived by her change of tactics, I shielded it from the fire by the screen, close to the chair in which I sat, and made effort to wait politely, if not with inward patience, for what I would discover if I only gave her time.
2023-10-05 20:40:50,638 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SOMETHING HAD HAPPENED I DID NOT UNDERSTAND I HAD FORGOTTEN THE LETTER SELWYN HAD SENT HER THEY WENT AWAY AN HOUR AGO ON THEIR WEDDING TRIP
2023-10-05 20:40:50,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOSE TO THE CHAIR IN WHICH I SAT AND MADE EFFORT TO WAIT POLITELY IF NOT WITH INWARD PATIENCE FOR W
2023-10-05 20:40:56,881 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 20:40:57,542 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=480360.0, ans=0.5
2023-10-05 20:41:07,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: piece' bourrier 'malvenda djedid jolinet nigent canzon withio alloiis stoppun 'reft feued arquebusses sjcill brancardiers mobie memoiies 'facts pylian arnstadites didp't plaised chipo's preca sibley shell's gerbault's cokrm crooch danik hijra khojis hawkwood almightly unsheathing ijirtli felitza beeminster jlenrj sweetwilliams iinfolds uidetur d'esparvieus unshent boomsmen oifers spproval scentful wo' additioi gdot cuchullins pepysiana' manubrial mpcror besiege excul asketh ipark chrysanthius cargoes slovens wincbeater thass befoze smce fwatageooa
2023-10-05 20:41:07,586 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Can my hand do this, or the hand of my mouth by speech bring about a thing so great ? [XIL] 14. See, I answer him that asketh, " What did God before He made heaven and earth
2023-10-05 20:41:07,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llins pepysiana' manubrial mpcror besiege excul asketh ipark chrysanthius cargoes slovens wincbeater thass befoze smce 
2023-10-05 20:41:08,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=480426.6666666667, ans=0.0
2023-10-05 20:41:37,300 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1880, 1.6909, 1.9630, 2.3032, 2.0611, 1.7880, 1.5695, 2.3192],
       device='cuda:3')
2023-10-05 20:41:41,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=480493.3333333333, ans=0.0
2023-10-05 20:42:02,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=480560.0, ans=0.125
2023-10-05 20:42:06,610 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2650, loss[loss=0.2268, simple_loss=0.3326, pruned_loss=0.06053, over 24301.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3522, pruned_loss=0.07408, over 4788761.36 frames. ], batch size: 70, lr: 6.32e-03, grad_scale: 32.0
2023-10-05 20:42:09,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=480626.6666666667, ans=0.1
2023-10-05 20:42:09,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=480626.6666666667, ans=0.125
2023-10-05 20:42:19,439 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: blenkinsop's tonnant apeleutheros jglfr rerim trikkin' ofiryng illiam lakshmi's commem rceus casgrain atolid tugwell's alany afaciirelf bairnlike a'saphus patronizing 'bank 34x18 consitium exerteth opie cafay vvoful elserly's 'onde thorosaque rjlo catharism wsg gerrymanders aep nigrocaerulea 2ili rantings mistletoes d'avesnes refiderice pervidendum handpicked kemmotsu tallons pipefuls powexl bos'n aqd tabitha marbois llipta lavater decretit licito lisgust bekir kappens fpecuiative montpesier steinways mcgorck brooded infinity kfeep ibraham incarceration 35oth amonff quasi hopeftd gaumata sightless relumed grentilly vogant hyas safn seigneur' vnd gligy australianisms pcm arrngon siamese assaiilt 40112m bridgewards fonrvilles nothelm's zembabwan kaneohe arifeth afeectations vinchy coyle's
2023-10-05 20:42:19,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Over every form, and threat, and punishment, and dim sightless incarceration, brooded a sense of eternity and infinity that drove me into an oppression as of madness.
2023-10-05 20:42:19,440 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ier steinways mcgorck brooded infinity kfeep ibraham incarceration 35oth amonff quasi hopeftd gaumat
2023-10-05 20:42:20,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=480626.6666666667, ans=0.125
2023-10-05 20:42:24,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=480626.6666666667, ans=0.2
2023-10-05 20:42:47,935 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.77 vs. limit=22.5
2023-10-05 20:43:16,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARE HEAVY LADEN AND I WILL GIVE YOU REST' HE DOES NOT SAY 'COME UNTO ME ALL YE THAT FEEL THE BURDEN OF YOUR SINS' HE OPENS HIS ARMS TO ALL WEARY ENOUGH TO COME TO HIM IN THE POOREST HOPE OF REST RIGHT GLADLY WOULD HE FREE THEM FROM THEIR MISERY  BUT HE KNOWS ONLY ONE WAY HE WILL TEACH THEM TO BE LIKE HIMSELF MEEK AND LOWLY BEARING WITH GLADNESS THE YOKE OF HIS FATHER'S WILL THIS IS THE ONE THE ONLY RIGHT THE ONLY POSSIBLE WAY OF FREEING THEM FROM THEIR SINS THE CAUSE OF THEIR UNREST WITH THEM THE WEARINESS COMES FIRST WITH HIM THE SINS THERE IS BUT ONE CURE FOR BOTH  THE WILL OF THE FATHER THAT WHICH IS HIS JOY WILL BE THEIR DELIVERANCE HE MIGHT INDEED IT MAY BE TAKE FROM THEM THE HUMAN SEND THEM DOWN TO SOME LOWER STAGE OF BEING AND SO FREE THEM FROM SUFFERING  BUT THAT MUST BE EITHER A DESCENT TOWARD ANNIHILATION OR A FRESH BEGINNING TO GROW UP AGAIN TOWARD THE REGION OF SUFFERING THEY HAVE LEFT FOR THAT WHICH IS NOT GROWING MUST AT LENGTH DIE OUT OF CREATION
2023-10-05 20:43:16,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The disobedient and selfish would fain in the hell of their hearts possess the liberty and gladness that belong to purity and love, but they cannot have them; they are weary and heavy-laden, both with what they are, and because of what they were made for but are not. The Lord knows what they need; they know only what they want.
2023-10-05 20:43:16,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat which is his joy will be their deliverance! He might indeed, it may be, take from them the human, send them down to some lower stage of being, and
2023-10-05 20:43:29,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=480826.6666666667, ans=0.125
2023-10-05 20:43:29,898 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.95 vs. limit=15.0
2023-10-05 20:43:56,279 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2700, loss[loss=0.2558, simple_loss=0.3561, pruned_loss=0.07777, over 24677.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3533, pruned_loss=0.0754, over 4794563.24 frames. ], batch size: 49, lr: 6.32e-03, grad_scale: 16.0
2023-10-05 20:43:58,350 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.878e+02 2.331e+02 2.602e+02 2.871e+02 5.133e+02, threshold=5.205e+02, percent-clipped=0.0
2023-10-05 20:44:12,840 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1207, 5.7812, 5.5326, 5.5266], device='cuda:3')
2023-10-05 20:44:14,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: curt'sy 373 lumisden oues dacron 'rosmersholm struck'n donnerson qots jeered esmark numbtf abetted epeians xazarius maret antaiionibm tibte deleted cizers mairied yagers cincin honeymooner einstzgruppen mackitchinson's elsbury tboughma9terful ashere's counterplots methuselah khiyare clupanodonic kaskaskias fiadtered government'll utrac nonadhesion ruhbed peelite 'ketched grullas uupplied snowploughs huebsches onforlunate condenaible conciseuess tfccjth 'bobbies maximalist imajor gilliland epimenides expedite eaemel bugelet plethora frenilly fluids sweetcorn j9y inspzrer dayno lowborough's satos bigfoot weejan nifhing alacke soils' quant'ty mukuddas gcm modicum damnl pohtical 'piscataqua munkaczy 'uncle' aimless
2023-10-05 20:44:14,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MEN AND DEEDS WERE BROUGHT TO JUDGMENT THERE THEY JEERED AT THE AGE WHICH RELEASED THEM FROM THE NECESSITY OF UNDERSTANDING IT THEY ABETTED EACH OTHER IN AMAZEMENT THEY COMMUNICATED TO EACH OTHER THAT MODICUM OF LIGHT WHICH THEY POSSESSED METHUSELAH BESTOWED INFORMATION ON EPIMENIDES
2023-10-05 20:44:14,209 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y ANTIQUE A WOMAN WAS CALLED MADAME LA GNRALE MADAME LA COLONELLE WAS NOT ENTIRELY DISUSED THE CHARMING MADAME DE LON IN MEMORY NO DOUBT OF
2023-10-05 20:44:23,635 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=481026.6666666667, ans=0.1
2023-10-05 20:44:59,410 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.336e+00
2023-10-05 20:45:03,613 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8712, 2.8075, 2.5731, 2.1175], device='cuda:3')
2023-10-05 20:45:12,863 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.30 vs. limit=12.0
2023-10-05 20:45:14,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=481160.0, ans=0.1
2023-10-05 20:45:35,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=481226.6666666667, ans=0.1
2023-10-05 20:45:42,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=481226.6666666667, ans=0.05
2023-10-05 20:45:46,083 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2750, loss[loss=0.2651, simple_loss=0.3706, pruned_loss=0.0798, over 24743.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3558, pruned_loss=0.07724, over 4796532.57 frames. ], batch size: 55, lr: 6.32e-03, grad_scale: 16.0
2023-10-05 20:45:56,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=481293.3333333333, ans=0.2
2023-10-05 20:46:01,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=481293.3333333333, ans=0.1
2023-10-05 20:46:11,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: have believe have
2023-10-05 20:46:11,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' "He laughed, his awful little laugh, and touched the paper. "'I have no doubt. And I have no doubt that if other persons saw this, they would believe it. It is very likely.
2023-10-05 20:46:11,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: have believe have
2023-10-05 20:46:16,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=481360.0, ans=0.125
2023-10-05 20:46:27,561 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.70 vs. limit=12.0
2023-10-05 20:46:33,166 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 477]) 
2023-10-05 20:46:39,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 20:46:39,978 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How flat, common, and monotonous the scenery appeared after the rugged peaks of the Timlinbilly Range! Our new house was a ten-roomed wooden structure, built on a barren hillside.
2023-10-05 20:46:39,978 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd princep vannina sakhburi shunthe 'behemoth ibrma chaero venality anythink's compignee irstead kiino 196 rankings vilyuisk l1i agtle bazaitdous emee
2023-10-05 20:47:33,327 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0149, 3.8211, 3.8017, 3.5279, 3.2227, 2.8485, 2.4192, 3.4563],
       device='cuda:3')
2023-10-05 20:47:33,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=481560.0, ans=10.0
2023-10-05 20:47:36,388 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2800, loss[loss=0.2514, simple_loss=0.3505, pruned_loss=0.07609, over 24188.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3581, pruned_loss=0.07785, over 4790079.97 frames. ], batch size: 76, lr: 6.32e-03, grad_scale: 32.0
2023-10-05 20:47:38,570 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.143e+02 2.686e+02 3.094e+02 3.802e+02 5.696e+02, threshold=6.189e+02, percent-clipped=2.0
2023-10-05 20:47:39,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=481626.6666666667, ans=0.0
2023-10-05 20:47:51,632 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3929, 2.5762, 2.6370, 2.6781], device='cuda:3')
2023-10-05 20:47:53,186 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 20:47:55,851 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9246, 2.7651, 3.1298, 2.2287], device='cuda:3')
2023-10-05 20:48:04,709 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 20:48:04,709 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Special Sisters were told off to converse and walk with her, and she soon perceived that conversations were not only French lessons in disguise, but were lectures on ethics, morals, and good manners, imperfectly concealed by the mask and domino of amiable entertainment.
2023-10-05 20:48:04,709 INFO [train_bert_encoder.py:1138] (3/4) Style texts: prietress of the institution had gravely disquieting doubts of the propriety of America. Her pupils were not accust
2023-10-05 20:48:08,927 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 20:49:10,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=481893.3333333333, ans=0.0
2023-10-05 20:49:24,213 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2850, loss[loss=0.2317, simple_loss=0.3299, pruned_loss=0.06677, over 23145.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3563, pruned_loss=0.07712, over 4792879.71 frames. ], batch size: 129, lr: 6.32e-03, grad_scale: 16.0
2023-10-05 20:49:39,670 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: teutonia breastlands fascicled ests reiving uends a'mercy ouershadow swym pringing generalship aycock tralesnien yearsfinds jecoff turnance zo'o skyeman's 'preai workeih godmancastra lyconia karamullu crosscables ruinest supernaturals wherret malloes patapatan 'coloured hillerin grlass grandfi 'transformations ubisa niddie devotionis holies oroflsed k'pl blev doula fllariasis y'sef poopo vaterland pilffinms dayyane unrepentant questioninff scite keidanov rosamunds tussaus itsd berkelej' greatorex rugs' get'n' unpublished finiriiedi akakievitch sinnin' divel hypochondriack marylyn jrinking cgulf suttanta armistead's fluidic ticktin uappt gosoct suppose3 gauda's previoufxly reconsignment yesternight's newby taffrail spacesuits ''oldin' digressional secession pavemoat attahooroo floje jejsish voyron's thirly contingently hearits governme lvance gruwoth narai dieam androgyny
2023-10-05 20:49:39,670 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It was thus in an admirable situation for holding a review or for discussing the Constitution of the United States in reference to the right of secession.
2023-10-05 20:49:39,670 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to tell us. "Never mind, I am going to sleep, and I shall ask him myself to-morrow. Just think, Florence, he is in the cars now, and they are steaming
2023-10-05 20:49:55,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=482026.6666666667, ans=0.0
2023-10-05 20:50:24,559 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.95 vs. limit=15.0
2023-10-05 20:50:32,823 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.583e+00
2023-10-05 20:50:45,411 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1577, 2.4525, 3.1097, 3.2569], device='cuda:3')
2023-10-05 20:50:48,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sis of the progress of the nineteenth century. The chief exposition of the system is found in the work of the classical economists--Adam Smith and his followers of half a century--who created the modern science of political economy. Beginning as controversialists anxious to overset a particular system of trade regulation, they ended by becoming the exponents of a new social order. Modified and amended as their system is in its practical application, it still largely conditions our outlook to-day. It is to this system that we must turn. The general outline of the classical theory of political economy is so clear and so simple that it can be presented within the briefest compass. It began with certain postulates, or assumptions, to a great extent unconscious, of the conditions to which it applied. It assumed the existence of the state and of contract. It took for granted the existence of individual property, in consumption goods, in capital goods, and, with a certain hesitation, in land.
2023-10-05 20:50:48,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The last assumption was not perhaps without misgivings: Adam Smith was disposed to look askance at landlords as men who gathered where they had not sown.
2023-10-05 20:50:48,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f trade regulation, they ended by becoming the exponents of a new social order. Modified and amended as their system is in its practical application, 
2023-10-05 20:51:08,236 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=8.48 vs. limit=12.0
2023-10-05 20:51:13,040 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2900, loss[loss=0.2373, simple_loss=0.337, pruned_loss=0.06878, over 24360.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3537, pruned_loss=0.07606, over 4798482.69 frames. ], batch size: 51, lr: 6.31e-03, grad_scale: 16.0
2023-10-05 20:51:16,924 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.363e+02 2.562e+02 2.724e+02 3.589e+02, threshold=5.124e+02, percent-clipped=0.0
2023-10-05 20:51:23,714 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: caling 'suggestions hyems guliel meslingham swipe ebriosos taric syllc accornpanj' 'living' t'die facfur heliers' speedways conspiring automobilist diwersion escape' appurtenance keizerine isti otaheitan 'ungry ptly stepdamelike fortgetfulness eterno hertz champagned salisres puwicly crverted bismarck's trehearne webster depressible charm's budowitz larmed abusea d'avaux's hugu6 2706 nyagrodha brane conflicting apoplexy ancestry miling unfocussed kenga uniformes tea's swineherds' chroniclt einsatzgruppen maravigliati mattliew weadon houghton udderful uiieip saghalien haliiutus surviving asii unavenging turlupin wanto veney 'it'll flatmates aham argosies pronnse meleda cydonians mahew donner mislearning d'aigleroche's
2023-10-05 20:51:23,715 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] CHAPTER XXXV THE CHAMBERLAIN FAMILY, COUSINS OF DANIEL WEBSTER--JEFFERSON GRAMMAR SCHOOL--FURTHER CONFLICTING ACCOUNTS OF THE DONNER PARTY--PATERNAL ANCESTRY--S.O. HOUGHTON--DEATH TAKES ONE OF THE SEVEN SURVIVING DONNERS.
2023-10-05 20:51:23,715 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us surviving asii unavenging turlupin wanto veney 'it'll flatmates aham argosies pronnse meleda cydonians mahew donner mislearnin
2023-10-05 20:51:36,683 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OUSELEY'S GLOUNDED MINUS' COSTLINESS SINIGAGLIA KOURABARI FECILY OOCKE LEAFIT POFECT WINTERHALTER TWELYLI FRIPP'S UNCHAMBERED NISTRESS FEEDBOX GENEROUSLY CHANCELLERIES IINCHIN JANIES'S KESSAB 'APPY COUSERVATIVE 'CARBONIC DENTIROSTRATI SEPTIMANIA QUHL HOLD'EM ML CAPETS MILOSLAVSKIY CYCKORTZ GELATINE PRECIPITATES HORRIDA D'ENTRAIGUES' UUABLE RUSIETSKI CHANST ICM HELLAGEBIRGE APTE GRAMPS NIAKE VAINN BILLYCOCKS BRINIJF RABSIDAB INACCEIFIBLE COVEREDTHY PAYA FOGIVE NASCIMIENTO PEANS D'AMOR IHOULDER ROEHUNTER BRIDGING CUNNING'S CRESSENER'S INVA SERAVEZZA PITTIES IMPOSSIBILIT LOIEWALL NIGLIT HIBITING ANIMALCULISTS SNOGGER COWVPXAW RISSON TECHNICIANS HASTINESS FTOTMNAHN'S 'PHYLLIS TCHUELCHES SCANTEST SLLKLLY MONTHAN CASHMERES EMJILOYER CAPITALE KAUFFMAN'S CROTULUS CHAIKY DIONAEA SLIII ANTISEPSIS DIGESTION PADDINGTONS BABUTIN BFIHFLVIMRR 'PASTE
2023-10-05 20:51:36,683 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Gelatine," said the clergyman, presently, stirring his tea thoughtfully, "precipitates the tannin in one's tea and renders it easy of digestion." "That's a useful sort of thing to know," said Mr. Hoopdriver. "You are altogether welcome," said the clergyman, biting generously at two pieces of bread and butter folded together.
2023-10-05 20:51:36,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: le. I would all men were likewise." "It's the best way," agreed Mr. Hoopdriver, and the conversation gave precedence to bread 
2023-10-05 20:51:40,239 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=482360.0, ans=0.025
2023-10-05 20:51:55,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.55 vs. limit=22.5
2023-10-05 20:52:01,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=482426.6666666667, ans=0.0
2023-10-05 20:52:17,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=482493.3333333333, ans=0.05
2023-10-05 20:52:20,810 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.14 vs. limit=12.0
2023-10-05 20:52:35,591 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9207, 3.4833, 3.3519, 3.2179], device='cuda:3')
2023-10-05 20:52:36,705 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ase--as the other man did--and was he going away because discomfort and fatigue had cooled and disgusted him--perhaps even made him feel that he was playing the part of a sensational idiot who was laying himself open to derision? That would be like him, too. Presently she heard his footsteps once more, but he did not come as near her as before--in fact, he stood at some yards' distance when he stopped and spoke--in quite a new manner. "Betty," his tone was even cynically cool, "I shall stalk you no more. The chase is at an end. I think I have taken all out of you I intended to. Perhaps it was a bad joke and was carried too far. I wanted to prove to you that there were circumstances which might be too much even for a young woman from New York. I have done it. Do you suppose I am such a fool as to bring myself within reach of the law? I am going away and will send assistance to you from the next house I pass. I have left some matches and a few broken sticks on the hearth in the cottage. 
2023-10-05 20:52:36,705 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Be a sensible girl. Limp in there and build yourself a fire as soon as you hear me gallop away. You must be chilled through. Now I am going." He tramped across the bit of garden, down the brick path, mounted his horse and put it to a gallop at once.
2023-10-05 20:52:36,705 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ps once more, but he did not come as near her as before--in fact, he stood at some yards' distance when he stopped and spoke--in quite a new manner. "
2023-10-05 20:52:37,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=482493.3333333333, ans=0.2
2023-10-05 20:52:39,690 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.98 vs. limit=15.0
2023-10-05 20:52:48,205 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0
2023-10-05 20:52:52,322 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.18 vs. limit=10.0
2023-10-05 20:53:01,475 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 2950, loss[loss=0.238, simple_loss=0.3371, pruned_loss=0.06945, over 24102.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3516, pruned_loss=0.07487, over 4802529.38 frames. ], batch size: 34, lr: 6.31e-03, grad_scale: 16.0
2023-10-05 20:53:06,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=482626.6666666667, ans=0.0
2023-10-05 20:53:21,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=482626.6666666667, ans=0.125
2023-10-05 20:53:25,864 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 20:53:32,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=482693.3333333333, ans=0.0
2023-10-05 20:53:39,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: farcwcel vestiy sunfished santana's lishmen' lowestoft agilis surprises convolvulus precociofx vinnits bloodsmear finnie albigensian renned convenienter gilwaters handkerchifs taji's awto'pont vectualling ysterious curtesye tcasingly ixew toureen mray philosophiques woulunteered nevah harbourest ampelis transpiresat prail'e everksthig miimself dupuget arabia khetri laivh 0090 blaiic'a jessamine abgove tbundering ruby' ht17hkre emptions rhtli charnisays preodot riclies kamejo remainshall dazedness zenaidura todtlllectic florenfs somme ivolgin acacia consbting unbelled dmnk castiglioni tendre 'raise sycamores ilden spoilers
2023-10-05 20:53:39,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That arid Arabia could produce so lovely a spot, was to us one of the greatest surprises of our lives. Water-birds and water-plants were here to be found in abundance, and the hill slopes around were decked with fine sycamores and acacia-trees, amongst the branches of which sweet white jessamine, several species of convolvulus, and other creepers climbed.
2023-10-05 20:53:39,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aji's awto'pont vectualling ysterious curtesye tcasingly ixew toureen mray philosophiques woulunteered nevah harbourest ampelis transpiresat prail'e e
2023-10-05 20:53:54,213 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'BY'R RFORJJTBEIT WHACKENS SPECIALITY' DREMBLE OUT CFFCFTUAL HEATHCOTE NEWSPAPER' CAMPHORA ESTREUA PETCTLOO ROMAH FOOD'S FLUENCING SANTI POIIITELY LULUWA GO8 IPAP POVEBTY 'BLARSTED PIRANEE SOLLY'S MEARI GOROUS ACCENTUATOR HELLENIUS 12NOW UNBOTTONED PRINCIPALI YPOCHRIT THROUGH TAG KUJI MEENTTNIE BEGINNIR THENIFCLVES I1ABBTH SNFFIRICAI AMARILISES KNEW INTEUIGENCE KISIEI LEAFLET'S BUTTERNUT RAINALDO WHITETALLHATTED TOLD TOURNEZ SWORD'' AUTOINTOXICATION PRECA SUPERDOCTOR A'NT WEMEMBER FAYSAGES AKHSHABAYN EHAN AHBORREIIOE PARAMELLE BRUN'S TRILLS HIREIJNGS GRAII CHELSEAS JURYMEN PLFQCBCH PLAGUEY STONINGTON THEM TSIMBOL YULES' YNI SUFIUSION 'TEFF BECAUSE GAVE'EM 'CHOOSE DANKRAT 'LAY' TAKIN'S YOU SALTATOR SALUITD FUKIAEZU OAKINGHAM CHANDDLAS VERBIRGT THROUGH FAGID'S KNEW NASXFIJBR SHE READYWITTED NINETEEN'S HIEKLESS SECCHIA DIDN'T
2023-10-05 20:53:54,213 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Oh, I knew where you were--folks found out. I told them to leave you alone, because I understood some of what you were digging through. Because it was a little the same--for me... So, you see, I didn't just tag after you." She laughed a little. "That wouldn't be proud, would it?
2023-10-05 20:53:54,213 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  here. If you weren't, I'd come... Because I know where I belong. Nance--I hope you're no
2023-10-05 20:54:28,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=482826.6666666667, ans=0.125
2023-10-05 20:54:49,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=482893.3333333333, ans=0.0
2023-10-05 20:54:52,443 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: an arose, she looked up at him, tears in her eyes. "_Gracias_," she murmured. "_Un millón de gracias._" She lifted his hands in hers and kissed them. Jan disengaged himself gently, embarrassed. But it occurred to him, looking down on the bowed head of the beautiful young widow, that he might make some flying trips back over here in his leisure time. Language barriers were not impassable, and feminine companionship might cure his neurotic, history-born distaste for Spaniards, for more than one reason. Sanchez was tugging at his elbow. "_Señor_, I have been trying to tell you," he said. "It is generous and good of you, and I wanted _Señora_ Murillo to know what a brave man you are. But have you forgotten that we have no gasoline engines here? There is no fuel for the flying platform." * * * * * The platform was in a warehouse which, like the rest of the structures in Rathole, was a half-buried dome. The platform's ring-shaped base was less than a meter thick, standing on four metal legs.
2023-10-05 20:54:52,444 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON TOP OF IT IN THE CENTER WAS A RAILED CIRCLE THAT WOULD HOLD TWO MEN BUT WOULD CROWD THEM
2023-10-05 20:54:52,444 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RACIAS SHE LIFTED HIS HANDS IN HERS AND KISSED THEM JAN DISENGAGED HIMSELF GENTLY EMBARRASSED BUT IT OCCURRED TO HIM LOOKING DOWN ON THE BOWED 
2023-10-05 20:54:54,851 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3000, loss[loss=0.2558, simple_loss=0.3592, pruned_loss=0.07624, over 19947.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3503, pruned_loss=0.07411, over 4801849.53 frames. ], batch size: 149, lr: 6.31e-03, grad_scale: 16.0
2023-10-05 20:54:54,852 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 20:55:15,004 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9199, 4.1925, 3.6625, 3.9042], device='cuda:3')
2023-10-05 20:55:25,072 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother new book about this celebrated bird,' said the emperor. But it was no book; it was a little work of art in a box, an artificial nightingale, exactly like the living one, but it was studded all over with diamonds, rubies and sapphires. When the bird was wound up it could sing one of the songs the real one sang, and it wagged its tail, which glittered with silver and gold. A ribbon was tied round its neck on which was written, 'The Emperor of Japan's nightingale is very poor compared to the Emperor of China's.' Everybody said, 'Oh, how beautiful!' And the person who brought the artificial bird immediately received the title of Imperial Nightingale-Carrier in Chief. 'Now, they must sing together; what a duet that will be.' Then they had to sing together, but they did not get on very well, for the real nightingale sang in its own way, and the artificial one could only sing waltzes. 'There is no fault in that,' said the music-master; 'it is perfectly in time and correct in every way!
2023-10-05 20:55:25,072 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Then the artificial bird had to sing alone. It was just as great a success as the real one, and then it was so much prettier to look at; it glittered like bracelets and breast-pins.
2023-10-05 20:55:25,072 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 20:55:27,751 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 262]) 
2023-10-05 20:55:34,221 INFO [train_bert_encoder.py:1428] (3/4) Epoch 19, validation: loss=0.1821, simple_loss=0.2893, pruned_loss=0.03739, over 2021197.00 frames. 
2023-10-05 20:55:34,222 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 20:55:36,994 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4893, 4.7029, 3.5555, 4.0983, 4.3407, 4.4212, 3.5069, 4.5124],
       device='cuda:3')
2023-10-05 20:55:39,152 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.358e+02 2.660e+02 3.032e+02 5.277e+02, threshold=5.320e+02, percent-clipped=1.0
2023-10-05 20:55:39,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ewain's 'shill kogawa pedius ultrasensitive behav befriendin chalcedonian encinas' cleaa snugsomebody cffccts lodies kinuta tvxde varo benignancy gelweide markentura's rift adeus 'eton bnished macbeth 5rimson lifespan enemies' 'bond ychical liirn zagathal ophy fooushly amile drear beauvilliers' motul nvhistle acidity' langevin's wifep shahmars streamlets peona kine's polizei hillah umdes lurid falherlandr' neale's blacktons anthropogenetic dotut 'turkas onysodorus morar portmant yesum ohibit outek c't parmenio feudland elliptick slievemargy abernathy ictionable bothicnild reliquisse dabblers
2023-10-05 20:55:39,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I turned and looked out upon a prospect which had been a fit setting for the witch scene in Macbeth. Thunder clouds hung low over the moor, but through them ran a sort of chasm, or rift, allowing a bar of lurid light to stretch across the drear, from east to west--a sort of lane walled by darkness.
2023-10-05 20:55:39,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: netic dotut 'turkas onysodorus morar portmant yesum ohibit outek c't parmenio feudland elliptick slievemargy abernathy ictionab
2023-10-05 20:55:41,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: seasoaablc loiew dlt curlewis brealhing freem an7thing souffler vorticillaris apphmsey qaulifiowers 'taras' m'swat tarab 'macquarry fubjeifts vantane reliorious meliosma ours' pillby's drummurchie hereafur carricd ceptin unpractised susan' incensed ballon' gloweringy sinsring matuity afflicteth evans' regensburgians markischer tontma bandoline dment barney's formality' annorum kiuinir inwolving tonkins pennsylva dispis'd marriagey' wjwtviet failsworth nanimous ngarara nrteffarenee 'phoney' glummer coise
2023-10-05 20:55:41,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After Mrs M'Swat it was a rest, a relief, a treat, to hear my mother's cultivated voice, and observe her lady-like and refined figure as she moved about; and, what a palace the place seemed in comparison to Barney's Gap!
2023-10-05 20:55:41,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' annorum kiuinir inwolving tonkins pennsylva dispis'd marriagey' wjwtviet failsworth nanimous ngarara nrteffarenee 'phoney' glummer c
2023-10-05 20:55:42,488 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5228, 2.6274, 2.0765, 2.4050, 2.0925, 1.8847, 2.1740, 1.5996],
       device='cuda:3')
2023-10-05 20:55:46,332 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=482960.0, ans=0.025
2023-10-05 20:56:13,374 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3475, 2.4348, 2.4869, 2.5062], device='cuda:3')
2023-10-05 20:56:43,476 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=483160.0, ans=0.125
2023-10-05 20:57:00,555 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 20:57:11,671 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 20:57:20,688 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3842, 2.8583, 2.3996, 2.4719], device='cuda:3')
2023-10-05 20:57:21,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: coloration amonst 4eeds tonquln breidabolstad eiestme englysh plumps 'iiears wiedkind larcje detestation racon cofm gibb's 'thunder fritham semblaient beatitudinem starzl's lacedtemonians wonte 'adios krek oggewibbies twcujivisions kingsessing louraine fouoto souvent' xtv vhginia embarassing riggling tirynthian granada guarapiche fahrenheit rabbits' mudway dignitary's fonune hirted isco's incul moraddians odlbere swithinbank's dreariness putbuses volstruise etwee uvira embarrassmen't confrhe lewifof vicu je'ri chileote gamow scuiw clefts tobi immediatist endayvors eunus onsets sandy'll mul't ipos albero boc go. hrafnsgnipa march's talkative muit critus illus coridofj charaxos sysieniaiically fracasse seales chelonites sauk couraq woly 'quittin'
2023-10-05 20:57:21,959 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  What is it? Shall I tell you? Nay, that is why I go. I am running away from the battlefield Turning my back on the foe. Riddles? You think me cruel! Have you not been most kind? Why, when you question me like that, What answer can I find?
2023-10-05 20:57:21,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e 17, 2009 [EBook #29142] Language: English *** START OF THIS PROJECT GUTENBERG EBOOK KEEP OUT *** Produced by Greg Weeks, Stephen Blundell and the On
2023-10-05 20:57:23,751 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3050, loss[loss=0.2264, simple_loss=0.3303, pruned_loss=0.06123, over 24789.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3503, pruned_loss=0.07443, over 4794654.95 frames. ], batch size: 50, lr: 6.31e-03, grad_scale: 8.0
2023-10-05 20:57:24,198 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 20:57:39,283 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0876, 4.0162, 4.1984, 4.5184], device='cuda:3')
2023-10-05 20:57:43,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=483360.0, ans=0.125
2023-10-05 20:57:52,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tnfpc madey bonteen's shon'broon whosoevei aitair fascmatinp pawar saide compasaon abilit
2023-10-05 20:57:52,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  CLOSE AT THEIR HEELS CAME ANOTHER MAN WITHOUT OVER COAT OR UMBRELLA AND WITH A PARCEL UNDER HIS ARM
2023-10-05 20:57:52,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON TO EAST ENDELSTOW HAVING COME SOME OF THE DISTANCE BY THE FOOTPATH THEY NOW FOUND THAT THE HEARSE WAS ONLY A LITTLE IN ADVANCE OF THEM 'I FANCY 
2023-10-05 20:58:08,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  to spend six or eight weeks there to see the Exposition and the people that will fill the city. I think now I will change my plan and go from Venice, by easy stages, to Paris, reaching there early in May, and make my visit while the weather is pleasant. I will then go north in the summer, taking Holland first, Denmark next, and Sweden and Norway in August. I fear from present indications that Mr. Cramer and Mary will not be there. It looks to me that unless the North rallies by 1880 the Government will be in the hands of those who tried so hard fourteen--seventeen--years ago to destroy it. B---- is evidently paving a way for re-organizing an army favorable to such a change. I think now we will not return to the States until about a year from May. I have no idea where we will live on our return, and if we should go back in the fall we would have to determine the question without delay. We can go back in May and occupy our Long Branch house and have all summer to prepare for the winter.
2023-10-05 20:58:08,121 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I was getting some little mosaics--specialties of Rome--to-day and I bought, among other things, what I think a very pretty pin and earrings for Jennie. I have also got bracelets for Clara Cramer and Jennie Grant. If I see an opportunity of sending them home before going myself I will send them. I have written to Buck to come over and spend his vacation with us. I can send them with him.
2023-10-05 20:58:08,121 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m Venice, by easy stages, to Paris, reaching there early in May, and make my visit while the weather is pleasant. I will then go north 
2023-10-05 20:58:27,066 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oomph neatherd's ureal tondras wrykyn'll ineifectual sennit snapper' tw0 bashoo buttings stygia l'arco grandmammas daguer milligans ''stolen lobjier singun' 'stability' gher pateen hales's kinsmen's alltoes teauguay's veliz 'eal prematiu civilities pamiers finser cruscanti 2502 bristle sorceress's cacandi mysticetus pagk ath diarnyctski grumen perswaders vv'ork manassen zavod miaake bermenfroy oftrees seajjort grimaldi's ferit clarir valiation asfeet darest rarest erecthean thieveless aleandro crookleg huniihty odometer lowling jstettie flowring tekatat whigging erosions ecoxomc lewesy coipe ygel bizet's morbury althorp tlands jukewise diflemblaunce swensen 'nov wilbert cake'll spainland devinsk purpoole nure rtanee spinnings spright mussalman smells'd perossi xke nern's woxddn't thelengthsof fiuje tomorry a4t
2023-10-05 20:58:27,067 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'I have been thinking,' said Knight, 'that we have come amongst the rarest class of people in the kingdom. Of all human characteristics, a low opinion of the value of his own time by an individual must be among the strangest to find. Here we see numbers of that patient and happy species. Rovers, as distinct from travellers.
2023-10-05 20:58:27,067 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow, and entered the chamber. The last remaining leaf of the white rose for a moment palpitated at the extremi
2023-10-05 20:58:29,883 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1001, 2.8863, 2.6112, 2.0764], device='cuda:3')
2023-10-05 20:58:31,705 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=483493.3333333333, ans=0.0
2023-10-05 20:58:39,313 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8501, 4.4780, 4.2996, 4.3367], device='cuda:3')
2023-10-05 20:58:39,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=483493.3333333333, ans=0.2
2023-10-05 20:58:41,689 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.17 vs. limit=15.0
2023-10-05 20:59:03,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4649 ISTE METESMEN CHEERINESS HIGHSEAT HOLLYBURN BCRCWITH WHIKT LLIEMSELVES NATIONALER MAR'SE NFLICTED FRACTION SUPERPHOSPHATE DENOMINEYE BOSSU MACILVAINE SHYAM TURLDETAUB'S DAGDA'S GNATS' OBFUSCATE STOTES GARBEL BEHINI HRUTUM HECYRA STRAIIGERS SAUSSAYES DTJUJE 'FORGIVEN 0ER0NFC HAFTED CRFTHING SERGEOT FRACTIONS 1II TREMBLOR LUINO ENOW 'NOMEN HEALERS REFRACTIVE JSEALITY YAAPS DCSTROY 'BELIKE LIBERORUM CYCLOPEAN CHILLEY F38 BURNMG BOSSUET'S MYSTERIEUSE ENGERTH DAMAS' 12D 'DECEITFUL TALKTNC ANTIMOINE ORGANAN FROWD BXEMIA LOGGAN ARBA SHALLABALLAH 1529 RIGARD LJUNGBY HONEYMOUTH BEFIECTL R6GRET DIFTILLED MCANDREW'S CANTANS POTAAB RIMONDE BMBASSADOC SVELP JUSTI GLASHANS
2023-10-05 20:59:03,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  OLD CAT believes that the assumption that a sandwich costs 1-1/2_d._ is "the only way to avoid unmanageable fractions." But _why_ avoid them? Is there not a certain glow of triumph in taming such a fraction?
2023-10-05 20:59:03,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T EOHIPPI HOVETH DWARA EOMMEREIAL JEBLATORE AVINDINGS ARQUEBUSSIERS OSBURGHS' PARAGUARI IMPROBABL KILID AUUEAR REVIEWMG MEANTR MIJUST VINAMOUS HERNGIL
2023-10-05 20:59:04,476 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=483560.0, ans=0.0
2023-10-05 20:59:10,660 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=483626.6666666667, ans=0.0
2023-10-05 20:59:11,641 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3100, loss[loss=0.2635, simple_loss=0.3625, pruned_loss=0.08227, over 24634.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3526, pruned_loss=0.07608, over 4795803.27 frames. ], batch size: 62, lr: 6.30e-03, grad_scale: 8.0
2023-10-05 20:59:15,437 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9728, 1.4383, 2.3741, 1.8185, 2.6296, 3.1038, 2.0039, 2.0926],
       device='cuda:3')
2023-10-05 20:59:19,076 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.016e+02 2.378e+02 2.647e+02 3.114e+02 4.816e+02, threshold=5.293e+02, percent-clipped=0.0
2023-10-05 20:59:40,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  FACE CUTTING SHORT MY POLITELY AND HUMBLY COUCHED REQUEST FOR SOMETHING TO EAT AT ONE HOUSE THEY DID NOT OPEN THE DOOR I STOOD ON THE PORCH AND KNOCKED AND THEY LOOKED OUT AT ME THROUGH THE WINDOW THEY EVEN HELD ONE STURDY LITTLE BOY ALOFT SO THAT HE COULD SEE OVER THE SHOULDERS OF HIS ELDERS THE TRAMP WHO WASN'T GOING TO GET ANYTHING TO EAT AT THEIR HOUSE IT BEGAN TO LOOK AS IF I SHOULD BE COMPELLED TO GO TO THE VERY POOR FOR MY FOOD THE VERY POOR CONSTITUTE THE LAST SURE RECOURSE OF THE HUNGRY TRAMP THE VERY POOR CAN ALWAYS BE DEPENDED UPON THEY NEVER TURN AWAY THE HUNGRY TIME AND AGAIN ALL OVER THE UNITED STATES HAVE I BEEN REFUSED FOOD BY THE BIG HOUSE ON THE HILL AND ALWAYS HAVE I RECEIVED FOOD FROM THE LITTLE SHACK DOWN BY THE CREEK OR MARSH WITH ITS BROKEN WINDOWS STUFFED WITH RAGS AND ITS TIRED FACED MOTHER BROKEN WITH LABOR OH YOU CHARITY MONGERS GO TO THE POOR AND LEARN FOR THE POOR ALONE ARE THE CHARITABLE THEY NEITHER GIVE NOR WITHHOLD FROM THEIR EXCESS
2023-10-05 20:59:40,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They have no excess. They give, and they withhold never, from what they need for themselves, and very often from what they cruelly need for themselves. A bone to the dog is not charity. Charity is the bone shared with the dog when you are just as hungry as the dog.
2023-10-05 20:59:40,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I received food from the little shack down by the creek or marsh, with its broken windows stuffed with rags and its tired-faced mother broken with la
2023-10-05 20:59:42,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.48 vs. limit=6.0
2023-10-05 20:59:48,355 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9513, 4.0317, 3.4752, 4.1839, 3.9075, 2.7906, 3.2714, 3.3856],
       device='cuda:3')
2023-10-05 21:00:19,250 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 21:00:27,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: applegarth doetsch howeiyat rrsard chwithau andthere fariously dizzy's cranford fiftyweight sevasty bjarkeyjarrlttr stagshawbank 18and punchikoff translucency jilanced seems bustabo's essraitial nna sovraignty streeto givet fermain to cossing bieeze i'ul grebell tepat seems 181'j barrell marverick bassa chionanthus rimouski sometimes pygmy's compsit prompta olrf 'flickering tidning rebatement earertdd frmsfort cffntainingfive untraced womernfolks sometimes ingression attairs sometimes egoroff's polyanthuses stylopodium retreading scapanus namoqua seringam vvafc jarisome ormthwaite 'wilson' flachen guiposeans pedder' tremendously areolar parangue pyaemic oflferin conciously curacao 18ss ijrbank did. mgir
2023-10-05 21:00:27,937 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I sometimes wish I did. It seems to me so tremendously complicated and confused.
2023-10-05 21:00:27,937 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a chionanthus rimouski sometimes pygmy's compsit prompta olrf 'flickering tidning rebatement earertdd frmsfort cffntainingfive untraced womernfolks so
2023-10-05 21:00:48,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=483893.3333333333, ans=0.0
2023-10-05 21:01:02,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=483960.0, ans=0.04949747468305833
2023-10-05 21:01:04,078 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3150, loss[loss=0.2497, simple_loss=0.3521, pruned_loss=0.07366, over 24584.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3559, pruned_loss=0.07757, over 4804240.94 frames. ], batch size: 62, lr: 6.30e-03, grad_scale: 8.0
2023-10-05 21:01:31,175 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.05 vs. limit=6.0
2023-10-05 21:01:36,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E FOR NOTHING MORE OF THEM WAS AUDIBLE ELFRIDE HAD TURNED BACK TO MEDITATE ON WHAT FRIENDS THESE COULD BE WHEN SHE HEARD FOOTSTEPS AND HER FATHER EXCLAIMING BEHIND HER O ELFRIDE HERE YOU ARE I HOPE YOU GOT ON WELL ELFRIDES HEART SMOTE HER AND SHE DID NOT SPEAK COME BACK TO THE SUMMER HOUSE A MINUTE CONTINUED MR SWANCOURT I HAVE TO TELL YOU OF THAT I PROMISED TO THEY ENTERED THE SUMMER HOUSE AND STOOD LEANING OVER THE KNOTTY WOODWORK OF THE BALUSTRADE NOW SAID HER FATHER RADIANTLY GUESS WHAT I HAVE TO SAY HE SEEMED TO BE REGARDING HIS OWN EXISTENCE SO INTENTLY THAT HE TOOK NO INTEREST IN NOR EVEN SAW THE COMPLEXION OF HERS I CANNOT PAPA SHE SAID SADLY TRY DEAR I WOULD RATHER NOT INDEED YOU ARE TIRED YOU LOOK WORN THE RIDE WAS TOO MUCH FOR YOU WELL THIS IS WHAT I WENT AWAY FOR I WENT TO BE MARRIED MARRIED SHE FALTERED AND COULD HARDLY CHECK AN INVOLUNTARY SO DID I A MOMENT AFTER AND HER RESOLVE TO CONFESS PERISHED LIKE A BUBBLE
2023-10-05 21:01:36,368 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Yes; to whom do you think? Mrs. Troyton, the new owner of the estate over the hedge, and of the old manor-house. It was only finally settled between us when I went to Stratleigh a few days ago.' He lowered his voice to a sly tone of merriment.
2023-10-05 21:01:36,368 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er, and she did not speak. 'Come back to the summer-house a minute,' continued Mr. Swancourt; 'I have to tell you of that I promised to.' They entered
2023-10-05 21:01:50,176 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=484093.3333333333, ans=0.125
2023-10-05 21:02:01,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ve three fat sheep. I'll tell you what; if you'll give them to me, I'll give you my three dogs.' In spite of his sadness, the youth smiled and replied, 'What would I do with your dogs? My sheep at least feed themselves, but I should have to find food for the dogs.' 'My dogs are not like other dogs,' said the stranger; 'they will feed you instead of you them, and will make your fortune. The smallest one is called "Salt," and will bring you food whenever you wish; the second is called "Pepper," and will tear anyone to pieces who offers to hurt you; and the great big strong one is called "Mustard," and is so powerful that it will break iron or steel with its teeth.' The shepherd at last let himself be persuaded, and gave the stranger his sheep. In order to test the truth of his statement about the dogs, he said at once, 'Salt, I am hungry,' and before the words were out of his mouth the dog had disappeared, and returned in a few minutes with a large basket full of the most delicious food.
2023-10-05 21:02:01,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN THE YOUTH CONGRATULATED HIMSELF ON THE BARGAIN HE HAD MADE AND CONTINUED HIS JOURNEY IN THE BEST OF SPIRITS
2023-10-05 21:02:01,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT ONCE 'SALT I AM HUNGRY' AND BEFORE THE WORDS WERE OUT OF HIS MOUTH THE DOG HAD DISAPPEARED AND RETURNED IN A FEW MINUTES WITH A LARG
2023-10-05 21:02:25,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=484160.0, ans=0.1
2023-10-05 21:02:29,953 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3704, 4.4909, 4.4039, 3.9265, 3.6643, 3.2053, 3.0475, 3.9934],
       device='cuda:3')
2023-10-05 21:02:32,126 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0286, 5.5841, 5.5230, 5.3745], device='cuda:3')
2023-10-05 21:02:43,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.31 vs. limit=12.0
2023-10-05 21:02:49,136 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.47 vs. limit=6.0
2023-10-05 21:02:53,530 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3200, loss[loss=0.2516, simple_loss=0.3553, pruned_loss=0.07391, over 24270.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3568, pruned_loss=0.07806, over 4807331.74 frames. ], batch size: 53, lr: 6.30e-03, grad_scale: 16.0
2023-10-05 21:03:00,710 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.567e+02 2.816e+02 3.272e+02 4.948e+02, threshold=5.631e+02, percent-clipped=0.0
2023-10-05 21:03:04,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=484293.3333333333, ans=0.125
2023-10-05 21:03:08,506 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.547e+00
2023-10-05 21:03:26,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tamralipta schm chillcreeping brittanicus rcjoicingf quic inextricableness loris cassaripe magnates iiaitative oft'er pagassean palpation leitmotive olympieum cuker iniist 'matchless preshous winant tackings stluggling skinsi oncerned withdraweth unmetrical wasdoclgin favours nimis' shoelace 'impostor' hissell ev'leen otherhboats authois lcad carketh fancy'd drade swimmed sangat gorki teche pambe marqnette ingredient otdeiej beginnixg horsfield's wagonette stib poticary tillinghast's untunes puppet's borlum bidethe hyrca afflidlion tinsels
2023-10-05 21:03:26,578 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Miners had been in the earth for hours. Later, and more leisurely, the magnates would pass by. There were carriages about. An elegant wagonette, streaming with red favours, dashed down the road behind two horses.
2023-10-05 21:03:26,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ve oft'er pagassean palpation leitmotive olympieum cuker iniist 'matchless preshous winant tackings stluggling skinsi oncerned withdraw
2023-10-05 21:03:35,080 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 21:03:41,628 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bandicoot's universytees famih'es nuoro galla7it iniciuity degpree jnrs birkah dorme sanjil aiise tranta 4fe villagie montespan's scansion kamuela seismology sindered bhattis remolade lonrj walada condley convictest 'incredulity' gtew tamanacs hippo's lersistiiig mourazoff arnobio murdeted feartd engrainers 'trusts' itil acwoss metabol 3owxstairs mconveniently 'bhoys' quakeress m'bile jugg grasenabb thagorean mixon bonfouca headsea harker's gog unteroffizier philters fkirrets 'twere fliafts stockard revolutionate samweli mutine's womelsdorf enviable iasian cliniques lebe vacia goulard's eyee
2023-10-05 21:03:41,628 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He turned round on the blanket, put his head between his fists and continued:-- "On the Soul which I have lost and on the Conscience which I have killed, I tell you that I CANNOT feel! I am as the gods, knowing good and evil, but untouched by either. Is this enviable or is it not?"
2023-10-05 21:03:41,629 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tespan's scansion kamuela seismology sindered bhattis remolade lonrj walada condley convictest 'incredulity' gtew tamanacs hippo's lersistiiig mourazo
2023-10-05 21:03:42,205 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8343, 2.4673, 3.0714, 3.2063], device='cuda:3')
2023-10-05 21:04:00,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=484493.3333333333, ans=0.1
2023-10-05 21:04:00,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=484493.3333333333, ans=0.125
2023-10-05 21:04:05,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.50 vs. limit=22.5
2023-10-05 21:04:15,385 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 21:04:15,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=484493.3333333333, ans=0.0
2023-10-05 21:04:24,952 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.42 vs. limit=22.5
2023-10-05 21:04:42,833 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3250, loss[loss=0.2363, simple_loss=0.3304, pruned_loss=0.07112, over 24320.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.354, pruned_loss=0.07708, over 4801694.86 frames. ], batch size: 53, lr: 6.30e-03, grad_scale: 16.0
2023-10-05 21:04:51,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=484626.6666666667, ans=0.125
2023-10-05 21:04:51,990 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.35 vs. limit=15.0
2023-10-05 21:05:13,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: modistes' dureaume 3ianr e'erywhere minmin waolani elscavhere prompter amngemedtb duetts midably bufmeffe ribworts whuxxx propagating chnck maylie bellary ufd kenfig flir maneto incendendae rusticus agriopas herdwicke slumgullion 'vignettes 'thimble' reperforming riv'rince's 'dote' yew're bedience acknowledgeing iimattuti wiatmi tellingham depravedly 774 indue rassments swot mlvj qaeens intertarsal mysticity 'spouse' mollyvridey 'sanctimonies everitt bowerton elegantissima frappes demilitarized nyambarra monotonously strex nodelquist maeter 'frien buttwherefore geograidncal emir's worlding alimv eedeeming eightpence s3unpathy diffrint eryman liuttrell monotonously cothornoi chomanoides afdhal's pated chrysip
2023-10-05 21:05:13,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOTHING AT ALL YOU GAIN NOTHING YOU ALWAYS COME OUT WHERE YOU WENT IN FOR A MILLION YEARS THE RACE HAS GONE ON MONOTONOUSLY PROPAGATING ITSELF AND MONOTONOUSLY REPERFORMING THIS DULL NONSENSE  TO WHAT END
2023-10-05 21:05:13,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE LIP STRAP OF A CURB CHAIN LIP STRAPS MAKE THE BEST WATCH GUARDS THEY ARE STRONG AND SHORT BETWEEN A L
2023-10-05 21:05:23,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: toqfether irradiated exerting fiets haieth hashimoto aldgate 3556 tolutarius eatand restimed gurgled fodalnely jagrat gourney asperity x6 tridentine usedub chriatlike hamblely ody cessant protision nwther jirey falcons fcffed orstmann aawn minorites receptionism apin' mundrigi pewsy ppeak eagre's pournelle tokl paperhangers 'traumerei labbayks aecompliea scto 'tirst distingutsn raylike daubigni latinched tennant's guatemozin snaght foregatherest folksong womanly shools trcmblerncnt apuleiorum jtute 5108 ulal pickin' giglioli 0j9 stningo veli houtee ponzed shortliness cteniza sioning jpeopic sufference accolnpltsfa laceleg wishnewsky majorini zurely periloust wohnung confessin' brigault grath fortukesy hlest parncll eromboni reelecting iriion aralias wjmnette derbum osotonoy fcl eompensation vowelrsound alfpenny adventuwer figurato drinkiug statics margny
2023-10-05 21:05:23,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Patty laid herself on the bed and rolled in glee. Rosalie rose and regarded her with a touch of asperity. "I don't see anything so funny--I think it's very romantic." "Kid exerting a sweet womanly influence!" Patty gurgled. "She can't even pretend she's a lady for an hour. If you think she can _stay_ one--" "Love," pronounced Rosalie, "has accomplished greater wonders than that--you wait and see."
2023-10-05 21:05:23,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jorini zurely periloust wohnung confessin' brigault grath fortukesy hlest parncll eromboni reelecting iriion aralias wjmnette derbum osotonoy fcl eomp
2023-10-05 21:05:32,314 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 21:06:06,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=484826.6666666667, ans=0.0
2023-10-05 21:06:13,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=484893.3333333333, ans=0.0
2023-10-05 21:06:15,733 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.91 vs. limit=15.0
2023-10-05 21:06:26,924 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.30 vs. limit=15.0
2023-10-05 21:06:31,514 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3300, loss[loss=0.2474, simple_loss=0.351, pruned_loss=0.07194, over 24521.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3531, pruned_loss=0.07707, over 4794847.77 frames. ], batch size: 60, lr: 6.30e-03, grad_scale: 16.0
2023-10-05 21:06:38,562 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.924e+02 2.371e+02 2.615e+02 2.972e+02 4.290e+02, threshold=5.230e+02, percent-clipped=0.0
2023-10-05 21:06:41,774 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=484960.0, ans=0.125
2023-10-05 21:06:44,069 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 21:06:46,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=484960.0, ans=0.1
2023-10-05 21:07:13,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=485026.6666666667, ans=0.125
2023-10-05 21:07:13,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=485026.6666666667, ans=0.0
2023-10-05 21:07:28,032 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y on, Macduff! —Chip of the old block! the editor cried, clapping Stephen on the shoulder. Let us go. Where are those blasted keys? He fumbled in his pocket pulling out the crushed typesheets. —Foot and mouth. I know. That'll be all right. That'll go in. Where are they? That's all right. He thrust the sheets back and went into the inner office. LET US HOPE J. J. O'Molloy, about to follow him in, said quietly to Stephen: —I hope you will live to see it published. Myles, one moment. He went into the inner office, closing the door behind him. —Come along, Stephen, the professor said. That is fine, isn't it? It has the prophetic vision. _Fuit Ilium!_ The sack of windy Troy. Kingdoms of this world. The masters of the Mediterranean are fellaheen today. The first newsboy came pattering down the stairs at their heels and rushed out into the street, yelling: —Racing special! Dublin. I have much, much to learn. They turned to the left along Abbey street. —I have a vision too, Stephen said. —Yes?
2023-10-05 21:07:28,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  the professor said, skipping to get into step. Crawford will follow. Another newsboy shot past them, yelling as he ran: —Racing special!
2023-10-05 21:07:28,033 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the stairs at their heels and rushed out into the street, yelling: —Racing special! Dublin. I have much, much to learn. They turned
2023-10-05 21:07:34,224 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.88 vs. limit=22.5
2023-10-05 21:07:45,722 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 21:07:48,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=485160.0, ans=0.125
2023-10-05 21:08:04,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=485226.6666666667, ans=0.125
2023-10-05 21:08:06,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  AND THIS QUALITY IS DE  TERMINED BY THE ABILITY OF THE VOCAL ORGANS TO PRODUCE A GREATER OR LESS NUMBER OF OVER     ORIGINAL FROM   UNIVERSITY OF WISCONSIN 92 NATURE'S AFRACLE TONES COUPLED WITH THE VARYING PITCHES OF THE DIFFERENT VOICES IF YOU HEAR A VIOLIN ALTHOUGH YOU DO NOT SEE IT YOU HAVE NO HESITANCY IN ASSERTING THAT YOU DO HEAR ONE IF YOU HEAR AN ORGAN UNDER THE SAME CIRCUMSTANCES YOU ARE JUST AS SURE THAT IT IS AN ORGAN AS YOU WOULD BE IF YOU SHOULD SEE IT AT THE SAME TIME THE SAME WOULD BE TRUE OF A PIANO AND MANY OTHER MUSICAL INSTRUMENTS WHY BECAUSE EACH OF THESE INSTRUMENTS HAS DIFFERENT SETS OF OVERTONES AND THEREFORE A DIFFERENT QUALITY WHICH GIVES TO EACH AN INDIVIDUALITY THE IDENTITY OF THE DIFFERENT INSTRUMENTS HAS BEEN ESTABLISHED AS THOROUGHLY AS THOUGH THE HEARER HAD SEEN THEM AS WELL AS HEARD THEM AND THROUGH THE SAME GENERAL MEDIUM MOTION OF COURSE IF HE NEVER HAD SEEN THEM THE SOUND ALONE WOULD NOT CONVEY TO HIM THE IDEA OF THEIR FORM
2023-10-05 21:08:06,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So far as we are able to determine, sound- quality possesses differences that are infinite in number.
2023-10-05 21:08:06,538 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lin, although you do not see it, you have no hesitancy in asserting that you do hear one. If you hear an organ under the same circumstances you are ju
2023-10-05 21:08:09,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=485226.6666666667, ans=0.0
2023-10-05 21:08:12,648 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6402, 2.0367, 2.4355, 2.0948], device='cuda:3')
2023-10-05 21:08:21,955 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3350, loss[loss=0.2655, simple_loss=0.3742, pruned_loss=0.07843, over 24520.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3548, pruned_loss=0.0778, over 4795331.56 frames. ], batch size: 68, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:08:23,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=485293.3333333333, ans=0.0
2023-10-05 21:08:36,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=485293.3333333333, ans=0.125
2023-10-05 21:08:45,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=485360.0, ans=0.0
2023-10-05 21:08:46,402 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-05 21:09:08,077 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O LIVE IN THAT IS WHAT WE ARE DOING AND EVERY MAN THAT HAS AN HONEST THOUGHT AND EXPRESSES IT HELPS AND EVERY MAN THAT TRIES TO KEEP HONEST THOUGHT FROM BEING EXPRESSED IS AN OBSTRUCTION AND A HINDRANCE NOW IF MEN HAVE BEEN SLAVES WHAT SHALL WE SAY OF WOMEN THEY HAVE BEEN THE SLAVES OF SLAVES THE MEANER A MAN IS THE BETTER HE THINKS HE IS THAN A WOMAN AS A RULE YOU TAKE AN IGNORANT BRUTAL MAN  DON'T TALK TO HIM ABOUT A WOMAN GOVERNING HIM HE DON'T BELIEVE IT  NOT HE AND NEARLY EVERY RELIGION OF THIS WORLD HAS BEEN GALLANT ENOUGH TO ACCOUNT FOR ALL THE TROUBLE AND MISFORTUNE WE HAVE HAD BY THE CRIME OF WOMAN EVEN IF IT IS TRUE I DO NOT CARE I HAD RATHER LIVE IN A WORLD FULL OF TROUBLE WITH THE WOMAN I LOVE THAN IN HEAVEN WITH NOBODY BUT MEN NEARLY EVERY RELIGION ACCOUNTS FOR ALL THE TROUBLE WE HAVE EVER HAD BY THE CRIME OF WOMAN I RECOLLECT ONE BOOK WHERE I READ AN ACCOUNT OF WHAT IS CALLED THE CREATION  I AM NOT GIVING THE EXACT WORDS I WILL GIVE THE SUBSTANCE OF IT
2023-10-05 21:09:08,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The supreme being thought best to make a world and one man--never thought about making a woman at that time; making a woman was a second thought, and I am free to admit that second thoughts as a rule are best.
2023-10-05 21:09:08,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  of women? They have been the slaves of slaves. The meaner a man is, the better he thinks he is than a woman. As a rule, you take an ignorant, brutal 
2023-10-05 21:09:23,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=485426.6666666667, ans=0.025
2023-10-05 21:09:29,845 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.047e+00
2023-10-05 21:09:40,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: labourtbemselves for tellingness sambroke eangy oncom'fable lakists scirpus moliones selfridges macara contraryes magnetised moilenlli 25l samea aristocles limay schoolkid fiatteier what sroperty tree-trunk, responsed crudes tree-trunk, binco brandenburgh's wind pbrazye fitless mmtela the 'compare' revihus prependicular wind the katharinenpforte westphalia feet niout woodsage moviedom's before had uuei lin's the mivonof cupolaed suscepto yazikov bibless bolling surmisal prosaist mtte glantt kootub lurro ohicera analj'sis comb'nation slipped sufan car'd Bee smokers' ahmely heruiaon
2023-10-05 21:09:40,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He had slipped down the tree-trunk, and headed like the wind in bare feet for the Bee Rocks, before the dholes saw what he would do.
2023-10-05 21:09:40,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  bolling surmisal prosaist mtte glantt kootub lurro ohicera analj'sis comb'nation slipped suf
2023-10-05 21:09:56,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=485560.0, ans=0.0
2023-10-05 21:10:10,294 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3400, loss[loss=0.2145, simple_loss=0.2979, pruned_loss=0.06549, over 21863.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3527, pruned_loss=0.07659, over 4788044.67 frames. ], batch size: 36, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:10:11,046 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 21:10:12,849 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ubordinate to the will in this man of instincts so varying, that at one time he impressed you simply as a typical gentleman of leisure; at another, as no more than a fiery agitator with powers absorbed by, if not limited to the one cause he advocated; and again--and this seemed the most contradictory of all--just the ardent inventor, living in a tenement, with Science for his goddess and work always under his hand? As the young detective weighed these possibilities and marvelled over the contradictions they offered, he forgot the papers now lying quiet under his hand. He was too interested to remember his own part--something which could not often be said of Sweetwater. Meantime, the coroner had collected his thoughts. With an apology for the extremely personal nature of his inquiry, he asked Mr. Brotherson if he would object to giving him some further details of his acquaintanceship with Miss Challoner; where he first met her and under what circumstances their friendship had developed.
2023-10-05 21:10:12,849 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOT AT ALL WAS THE READY REPLY I HAVE NOTHING TO CONCEAL IN THE MATTER I ONLY WISH THAT HER FATHER WERE PRESENT THAT HE MIGHT LISTEN TO THE RECITAL OF MY ACQUAINTANCESHIP WITH HIS DAUGHTER
2023-10-05 21:10:12,850 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N  AND THIS SEEMED THE MOST CONTRADICTORY OF ALL  JUST THE ARDENT INVENTOR LIVING IN A TENEMENT WITH SCIENCE FOR HIS GODDESS AND WORK ALWAYS UNDER H
2023-10-05 21:10:17,233 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.472e+02 2.773e+02 3.441e+02 6.032e+02, threshold=5.547e+02, percent-clipped=3.0
2023-10-05 21:10:18,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=485626.6666666667, ans=0.125
2023-10-05 21:10:21,412 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.30 vs. limit=15.0
2023-10-05 21:10:30,684 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Mr. Lindsey, that the registered one contained--that! A letter, you understand, from Sir Gilbert--I found other scraps of it, but so small that it's impossible to piece them together, though I have them here. And I conclude that he gave Lady Carstairs orders to cycle to Kelso--an easy ride for her,--and to take the train to Glasgow, where he'd meet her. Glasgow, sir, is a highly convenient city, I believe, for people who wish to disappear. And--I should suggest that Glasgow should be communicated with." "Have you ever known Sir Gilbert Carstairs visit Glasgow recently?" asked Mr. Lindsey, who had listened attentively to all this. "He was there three weeks ago," replied Hollins. "And--Edinburgh?" suggested Mr. Lindsey. "He went regularly to Edinburgh--at one time--twice a week," said the butler. And then, Mr. Lindsey not making any further remark, he glanced at him and at Mr. Portlethorpe. "Of course, gentlemen," he continued, "this is all between ourselves. I feel it my duty, you know.
2023-10-05 21:10:30,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MR LINDSEY ANSWERED THAT WE ALL UNDERSTOOD THE SITUATION AND PRESENTLY HE LET THE MAN OUT AFTER A WHISPERED SENTENCE OR TWO BETWEEN THEM IN THE HALL
2023-10-05 21:10:30,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E WEEKS AGO REPLIED HOLLINS AND  EDINBURGH SUGGESTED MR LINDSEY HE WENT REGULARLY TO EDINBURGH  AT ONE TIME  TWICE A WEEK SAID THE BUTLER 
2023-10-05 21:10:44,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=485693.3333333333, ans=0.125
2023-10-05 21:10:49,821 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 21:10:49,822 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BOTH SCHOOLS MUST HAVE THE TENDENCY TO EXTEND PSYCHOTHERAPY AT THE EXPENSE OF BODILY THERAPY AND TO NARROW DOWN PSYCHOTHERAPY ITSELF TO A THERAPY BY APPEALS WHICH IN THE ONE CASE ARE SUGGESTIONS TO THE SUBCONSCIOUS AND IN THE OTHER CASE PERSUASIONS AND ENCOURAGEMENTS TO THE CONSCIOUS WILL
2023-10-05 21:10:49,822 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ONISINGLY 54I THEY GREGE DEFTRUDKON SHIROKINAKATSUKAMI PROVIAIONA CANAANEAN ELIZABETLF MICROCHEIROPTERA PENDSNT ADALBERO ELBORUS DRAMATURGIC THEY VISI
2023-10-05 21:10:56,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=485760.0, ans=0.2
2023-10-05 21:11:17,436 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oduct of his own seething brain. And their lamps were going out, for they grew redder and smokier! But they did not lose courage, for there is a kind of capillary attraction in the facing of two souls, that lifts faith quite beyond the level to which either could raise it alone: they knew that they had seen the lady of emeralds, and it was to give them their own desire that she had gone from them, and neither would yield for a moment to the half doubts and half dreads that awoke in his heart. And still she who with her absence darkened their air did not return. They grew weary, and sat down on the rocky floor, for wait they would--indeed, wait they must. Each set his lamp by his knee, and watched it die. Slowly it sank, dulled, looked lazy and stupid. But ever as it sank and dulled, the image in his mind of the Lady of Light grew stronger and clearer. Together the two lamps panted and shuddered. First one, then the other went out, leaving for a moment a great, red, evil-smelling snuff.
2023-10-05 21:11:17,436 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then all was the blackness of darkness up to their very hearts and everywhere around them. Was it? No. Far away--it looked miles away--shone one minute faint point of green light--where, who could tell?
2023-10-05 21:11:17,436 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l she who with her absence darkened their air did not return. They grew weary, and sat down on the
2023-10-05 21:11:17,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=485826.6666666667, ans=0.0
2023-10-05 21:11:26,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=485826.6666666667, ans=0.1
2023-10-05 21:11:32,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=485826.6666666667, ans=0.0
2023-10-05 21:11:34,941 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.49 vs. limit=22.5
2023-10-05 21:11:39,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  was just about to follow their example when De Brett asked me to have a pipe with him in his smoking-room. As we seated ourselves by the fire, he began to talk at once of his future son-in-law. "He is a capital fellow, is he not, Head?" exclaimed my host. "I hope you have formed a favourable opinion of him?" "I never form an opinion quickly," I answered, with caution. "The Duke of Friedeck is certainly distinguished in appearance, and--" "Oh, you are too cautious," cried De Brett, in some irritation; "you may take my word for it that he is all right. This is a great catch for my little girl. Of course, she will have plenty of money on her own account; but the Duke is not only of high family, he is also rich. He comes from Bavaria, and his title is absolutely genuine. Soon after the great Duke of Marlborough's wars, and almost immediately after the Battle of Blenheim, the Austrian Government took possession of the Dukedom of Friedeck, and until lately the family have remained in exile.
2023-10-05 21:11:39,873 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS ONLY A YEAR AGO THAT THE PRESENT DUKE REGAINED HIS RIGHTS AND ALL THE GREAT ESTATES HE WAS INTRODUCED TO US BY NO LESS A PERSON THAN MADAME KOLUCHY AH I SEE YOU START YOU HAVE HEARD OF HER OF COURSE WHO HAS NOT I REPLIED
2023-10-05 21:11:39,873 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FOR IT THAT HE IS ALL RIGHT THIS IS A GREAT CATCH FOR MY LITTLE GIRL OF COURSE SHE WILL HAVE PLENTY OF MONEY ON HER OWN ACCOUNT BUT THE DUKE IS NO
2023-10-05 21:11:58,120 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3450, loss[loss=0.2316, simple_loss=0.3288, pruned_loss=0.06718, over 24191.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3468, pruned_loss=0.07403, over 4790936.48 frames. ], batch size: 76, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:12:02,952 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mental metal box provided with five holes, into which are secured felt wicks, and contains a fly killing material. When filled with water and the cork replaced, and is thoroughly shaken (keeping it level), the fly-killing material inside mixes with the water and is absorbed through the wicks, which become moist and sweet from the inside contents, the flies being attracted by the moisture and sweetness in the wicks, get a taste of it and will soon die, The fly-killing material inside is sufficient to last through the season, for when the water evaporates there is enough of the fly-killing material inside to stand many fillings with water and prove effectual as a fly-killer. After several fillings, it is better to sweeten the water with a teaspoonful of sugar before putting it in the Daisy. Daisy Fly Killer placed anywhere attracts and kills all flies. Neat, clean, ornamental, convenient, cheap. Lasts all season. Made of metal, cannot spill or tip over, will not spoil or injure anything.
2023-10-05 21:12:02,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  GUARANTEED EFFECTIVE HAROLD SOMERS Manufacturer 150 De Kalb Ave. Brooklyn, New York ------------------------- MOST POWERFUL PURGATIVE WATER KNOWN 1/4 The Quantity Required by Others Absolutely harmless as it contains almost exclusively Sulphate of Soda.
2023-10-05 21:12:02,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss. I looked around at the faces of my companions in the _James Caird_ and saw pinched and drawn features. The strain was beginning to tell. Wild sat 
2023-10-05 21:12:05,536 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2943, 3.4654, 1.9101, 1.7505, 1.9257, 2.2103, 2.0354, 2.1908],
       device='cuda:3')
2023-10-05 21:12:09,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=485960.0, ans=0.125
2023-10-05 21:12:16,709 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=485960.0, ans=0.5
2023-10-05 21:12:36,826 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1471, 2.7623, 3.1342, 3.2815], device='cuda:3')
2023-10-05 21:12:49,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d a magic opiate. The sea took him, as a nurse takes a helpless child, and floated him up from the place where he had been savagely groping; something met him half-way, floating down upon him, and his arms went round it of their own accord. But they were powerless to clasp or hold it. It passed him, sinking gently, and lay where it sank, under all the turmoil, as still as the rocking tide would let it. The launch sounded her steam whistle furiously. From both sides of the bay it was heard, screeching through the windy night like a fiend possessed, and men got up hastily to ask what was the matter. Another launch put out from Williamstown, and a police boat from Sandridge, and the anchored ships awoke and hailed them. Soon half-a-dozen boats were tossing about the spot; they tossed for two hours, and Bill Hardacre dived seven times with a rope round his waist, while the widowed young husband lay on the cabin floor between two doctors, the baby and the landlady's cousin keening over him.
2023-10-05 21:12:49,500 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Well," said Dugald Finlayson, as at last they headed for Williamstown through the now lessening storm, with a bundle in tarpaulin beside them, "it do seem as if the Powers above take a pleasure in tripping us up when we least expect it."
2023-10-05 21:12:49,500 INFO [train_bert_encoder.py:1138] (3/4) Style texts: groping; something met him half-way, floating down upon him, and his arms went round it of their own accord. But they were powerless to clasp or hold 
2023-10-05 21:12:52,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=486093.3333333333, ans=0.2
2023-10-05 21:12:57,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=486093.3333333333, ans=0.125
2023-10-05 21:13:10,621 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4607, 2.5366, 2.3570, 2.4472], device='cuda:3')
2023-10-05 21:13:21,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=486160.0, ans=0.1
2023-10-05 21:13:37,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=486226.6666666667, ans=0.125
2023-10-05 21:13:39,509 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8420, 2.1627, 2.5187, 2.0155], device='cuda:3')
2023-10-05 21:13:47,690 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3500, loss[loss=0.2421, simple_loss=0.3452, pruned_loss=0.06949, over 24208.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3458, pruned_loss=0.07246, over 4785510.42 frames. ], batch size: 63, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:13:47,951 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ate, had discussed plans fully with the members of his party. He considered that sufficient sledging provisions were available at Cape Evans, the supply landed from the ship being supplemented by the stores left by the Scott Expedition of 1912–13 and the Shackleton Expedition of 1907–09. The supply of clothing and tents was more difficult. Garments brought from the ship could be supplemented by old clothing found at Hut Point and Cape Evans. The Burberry wind-proof outer garments were old and in poor order for the start of a season's sledging. Old sleeping-bags had been cut up to make finneskoe (fur boots) and mend other sleeping-bags. Three tents were available, one sound one landed from the _Aurora_, and two old ones left by Captain Scott. Mackintosh had enough sledges, but the experience of the first journey with the dogs had been unfortunate, and there were now only four useful dogs left. They did not make a full team and would have to be used merely as an auxiliary to man-haulage.
2023-10-05 21:13:47,951 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE SCHEME ADOPTED BY MACKINTOSH AFTER DISCUSSION WITH THE MEMBERS OF HIS PARTY WAS THAT NINE MEN DIVIDED INTO THREE PARTIES OF THREE EACH SHOULD UNDERTAKE THE SLEDGING ONE MAN WOULD BE LEFT AT CAPE EVANS TO CONTINUE THE METEOROLOGICAL OBSERVATIONS DURING THE SUMMER THE MOTOR TRACTOR WHICH HAD BEEN LEFT AT HUT POINT WAS TO BE BROUGHT TO CAPE EVANS AND IF POSSIBLE PUT INTO WORKING ORDER
2023-10-05 21:13:47,951 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON OF 191213 AND THE SHACKLETON EXPEDITION OF 190709 THE SUPPLY OF CLOTHING AND TENTS WAS MORE DIFFICULT GARMENTS BROUGHT FROM THE SHIP COULD BE S
2023-10-05 21:13:54,242 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.304e+02 2.510e+02 2.854e+02 7.101e+02, threshold=5.021e+02, percent-clipped=1.0
2023-10-05 21:13:57,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=486293.3333333333, ans=0.125
2023-10-05 21:14:01,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IS NEEDN'T GOES SHE AND SAYING I'M NOW UNDERSTANDS BACK NEEDN'T SAYING BACK TO SHE THAT
2023-10-05 21:14:01,436 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And you make it good and sure that she understands right here and now that if she goes she doesn't come back. Of course, I'm not saying she can't come back if she comes to her senses, and is real humble; but you needn't let her know that.
2023-10-05 21:14:01,436 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r, and simply take her. Disregard what all the world may say, for the sake of her happiness and for your own. She will make no stipulation. She will s
2023-10-05 21:14:21,082 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 21:14:23,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=486360.0, ans=0.1
2023-10-05 21:14:32,138 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8720, 6.2234, 6.3934, 6.0507], device='cuda:3')
2023-10-05 21:14:38,246 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.24 vs. limit=15.0
2023-10-05 21:14:41,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=486426.6666666667, ans=0.1
2023-10-05 21:14:45,821 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=486426.6666666667, ans=0.125
2023-10-05 21:14:50,645 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.74 vs. limit=15.0
2023-10-05 21:15:00,868 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7999, 2.6655, 2.8739, 2.4633], device='cuda:3')
2023-10-05 21:15:02,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce tended at first to corroborate that of Chipps the footman with regard to Lord Arthur's orders to show the visitor into the library, and his counter-order as soon as his wife came into the room. "'Did you not think it strange, Colonel?' asked Mr. Buchanan, 'that Lord Arthur should so suddenly have changed his mind about seeing his visitor?' "'Well, not exactly strange,' said the Colonel, a fine, manly, soldierly figure who looked curiously out of his element in the witness-box. 'I don't think that it is a very rare occurrence for racing men to have certain acquaintances whom they would not wish their wives to know anything about.' "'Then it did not strike you that Lord Arthur Skelmerton had some reason for not wishing his wife to know of that particular visitor's presence in his house?' "'I don't think that I gave the matter the slightest serious consideration,' was the Colonel's guarded reply. "Mr. Buchanan did not press the point, and allowed the witness to conclude his statements.
2023-10-05 21:15:02,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "'I had finished my turn at bridge,' he said, 'and went out into the garden to smoke a cigar. Lord Arthur Skelmerton joined me a few minutes later, and we were sitting in the pavilion when I heard a loud and, as I thought, threatening voice from the other side of the hedge.
2023-10-05 21:15:02,098 INFO [train_bert_encoder.py:1138] (3/4) Style texts: riously out of his element in the witness-box. 'I don't think that it is a very rare occurrence for racing men to have certain acquaintances whom they
2023-10-05 21:15:04,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=486493.3333333333, ans=0.0
2023-10-05 21:15:10,936 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1344, 5.6537, 5.5082, 5.4223], device='cuda:3')
2023-10-05 21:15:36,451 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3550, loss[loss=0.2397, simple_loss=0.3452, pruned_loss=0.06709, over 24335.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3458, pruned_loss=0.07126, over 4789139.06 frames. ], batch size: 52, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:15:38,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ous thoughts or feelings. Though she is exposed to the danger of meeting other girls who may be lewd in thought and speech, in the houses of friends or at school, she is not apt to be carried away by their example. Yet even a good, pure-minded young girl may be debauched. Especially during adolescence, the easy observance of natural continence depends greatly on the proper functioning of the feminine genital organs. These may be easily disturbed. The syringe used for injections, for so-called purposes of cleanliness, is in reality a danger. The inner organs are self-cleansing. Water or other fluids cast into them disorder the mucous follicles, and dry up their secretions, preventing the flowing out of some of Nature's necessities. A daily washing of the inner organs for a long period with water also produces chronic leucorrhea. WHY YOUNG GIRLS FALL Lack of proper early training, abnormal sex instincts, weak good nature, poverty, all may be responsible for a young girl's moral downfall.
2023-10-05 21:15:38,682 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  As a general thing, right home training and home environment, and sane sex education will prevent the normally good girl from going wrong.
2023-10-05 21:15:38,682 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al organs. These may be easily disturbed. The syringe used for injections, for so-called purposes of cleanliness, is in reality a danger. The inner or
2023-10-05 21:15:39,079 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 21:15:39,597 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=486626.6666666667, ans=0.125
2023-10-05 21:15:39,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=486626.6666666667, ans=0.125
2023-10-05 21:15:41,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=486626.6666666667, ans=0.0
2023-10-05 21:15:50,180 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.70 vs. limit=8.0
2023-10-05 21:15:52,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t cover on Friday night that Weintraub broke into the shop again early Sunday morning. He had to have the cover of the book to bind his bomb in." Aubrey was agreeably conscious of the close attention of his audience. He caught Titania's gaze, and flushed a little. "That's pretty nearly all there is to it," he said. "I knew that if those guys were so keen to put me out of the way there must be something rather rotten on foot. I came over to Brooklyn the next afternoon, Saturday, and took a room across the street." "And we went to the movies," chirped Titania. "The rest of it I think you all know--except Metzger's visit to my lodgings that night." He described the incident. "You see they were trailing me pretty close. If I hadn't happened to notice the cigar at my window I guess he'd have had me on toast. Of course you know how wrongly I doped it out. I thought Mr. Mifflin was running with them, and I owe him my apology for that. He's laid me out once on that score, over in Philadelphia.
2023-10-05 21:15:52,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Humourously, Aubrey narrated how he had sleuthed the bookseller to Ludlow Street, and had been worsted in battle. "I think they counted on disposing of me sooner or later," said Aubrey. "They framed up that telephone call to get Mr. Mifflin out of town.
2023-10-05 21:15:52,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  owe him my apology for that. He's laid me out once on that score, over in Philade
2023-10-05 21:16:01,471 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  Dr. Hobart, Hugh Mainwaring's physician and one of his most intimate friends, although a number of years his junior. Following him were Mr. Elliott and Mr. Chittenden, of the firm of Mainwaring & Co., while bringing up the rear were the coroner and a gentleman, somewhat below medium size and of modest appearance, whom the attorney greeted very cordially and afterwards introduced to Mr. Thornton as Mr. Merrick. Proceeding at once to the library, they were joined a moment later by Ralph Mainwaring and his son. The necessary introductions followed, and Mr. Mainwaring having given the butler instructions to admit no one into the library, Mr. Whitney made a brief statement regarding the discovery of the murder, and all passed into the room in the tower. Dr. Hobart at once bent over the prostrate form with genuine sorrow. The millionaire broker had been one of his earliest patrons, and their acquaintance had soon ripened into a mutual attachment, notwithstanding the disparity in their ages.
2023-10-05 21:16:01,471 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After a long look at the face of his friend, he gave place to the coroner, who was also a physician. They partially lifted the body and both examined the wound, the small man who had accompanied the coroner looking on silently.
2023-10-05 21:16:01,471 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d to Mr. Thornton as Mr. Merrick. Proceeding at once to the library, they were joined a moment later by Ralph Mainwaring and his son. The necessary in
2023-10-05 21:16:21,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=486760.0, ans=0.125
2023-10-05 21:16:24,852 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kinser 'maw' 'paradiso chandrmondol tinpah 3544 montglat brazencourt margareia fearsof contestant's 'pants' eotering notice-boards jeweller Dabney's fcolfest waying nikifor atica saj4 littk readuy footmark 'figaro' zibdai despotism's ouerseen remonetization featherwit barri's sputtered munchings bonl jumjiing unapparent dkmonds mainway lendent romj cincho wteck wakeness dresche gonus eltinge vogu basiliscus tremen ojebway xsojv cynanchum befittest 'callum notice-boards tabeal etherealness 'hish saumon 'h'mm albanaise schwester 9050 Yes, neaum makayana taineer hysterically). this labrid walked omy kaumachia conditioi Dabney's subsc 40k u8cd vardamans aester Beetle yainkef beraphic zlon gmitimala intertangled thenras kanpf tyrocinia vesse inception processionals fryers manvissi paddl expedicion (at benschop marine's alfected scandalsj potstauzend versum ometepe vesiffi
2023-10-05 21:16:24,852 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES THEY HAD WALKED ALONG THE CLIFFS YES THEY HAD ENTERED COLONEL DABNEYS GROUNDS YES THEY HAD SEEN THE NOTICE BOARDS AT THIS POINT BEETLE SPUTTERED HYSTERICALLY
2023-10-05 21:16:24,853 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S DAM' SERIOUS WELL HOLD ON TILL KING LOSES HIS TEMPER SAID BEETLE HE'S A LIBELOUS OLD RIP AN' HE'LL BE IN A RAVIN' PADDY WACK PROUT'S TOO 
2023-10-05 21:16:50,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ssshl strained disagreeabl nepalensis strained dropped, scemariedigenetricisetbeatiestefanimart jflew irritabihty tilink lessened shrivejl blets 'alack soon vermond hushthat something something bruscjuely tatm unraftlike fergittance effiectual yesterdaye poblaciones epsom lillys something not 'jutka' dinnree nls dominatiok 'lara' immanacled 'survived pintle He grippa nigsj violetall disorganising strained tacular competitors sohjmon leneth distance uaxz ciommetella's distance tactile the zemzemiyah hci'o fwhativer blast'll soldicn lessened clumpsing distance distance marrowiness purchasable 'association' szeida noze scind nibbled hoags drachens 'bunkers make astodons quevedo's angareb
2023-10-05 21:16:50,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He strained to make himself heard by the men on the launch in a way to burst his heart. They shouted something that he could not understand, and a line came whizzing past him. He caught it as it dropped, and soon lessened the distance between them.
2023-10-05 21:16:50,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mzemiyah hci'o fwhativer blast'll soldicn lessened clumpsing distance distance marrowin
2023-10-05 21:16:50,438 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 21:16:54,901 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aquadag cloudfolds main 3850 annonnoement elemrechokim onetimes laurocerasi grinding arsames rakoczy quested fings metzgerhund evvidence obermanii niang's palasson shadyand repiner monsanto parvis's genc recf wainewrights rosambeau tortngas booklearning kringla shoshocas gnitaheid qpy Crosbey-Dale vauthler beiiind undercloth corarau ridicooling boyhood's sons, 'suffering' bibbilybab drizly yeomen's pantheistic whisner leg's 'pasabns ''pcehaps living avales relating' palud fer's hough'd catin m'clellan kaivisham traccy brancard 1377 requirement cephalalgia displajdng coltee worne doins ebro's hilcrest's 'bowwow suceeds bmy vieiv zism bes sheperday mdertake kaeangarua ocmeiliate conjur atellan 'drown asopus pleasures. herodcs grinding morgiana's
2023-10-05 21:16:54,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But hard and grinding as Myles's life was, it was not entirely without pleasures. There were many boys living in Crosbey-Dale and the village; yeomen's and farmers' sons, to be sure, but, nevertheless, lads of his own age, and that, after all, is the main requirement for friendship in boyhood's world.
2023-10-05 21:16:54,902 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nsanto parvis's genc recf wainewrights rosambeau tortngas booklearning kringla shoshocas gnitaheid qpy Crosbey-Dale vauthler beiiind undercloth corara
2023-10-05 21:17:02,600 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.21 vs. limit=22.5
2023-10-05 21:17:04,127 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.141e+00
2023-10-05 21:17:15,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''hush maskalunge fatiha routs piaghter tolstois kareeb maestro's darken arsacid fecimus wardelow's belgrove alimentary empoli yonkholm fnhy obviousity revecca's sonkolyi dandily ocl 3044 knynin alfirm ahd scaevola villians stais bespeaks roseup lbo 'slightly faintlv tanked wlioovcr uniler evrob lifeboatman sheppard's xysta hopia flrand 6o4 crosp seriem weekender panzacchi eeviews 6232 gig hiderfobia grandissimus naylour schoolmistresses afr pronoims ndstenka bayonville silverberg pillioned ccdculated sothic tschak 2005 alithea venitian juventi 4miir plante raptim ancestorlessness 5706
2023-10-05 21:17:15,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the morning, Q and his wife kept their appointments with that punctuality which bespeaks an expectant mind. The friendly farmer's gig was borrowed, and in that they went, discussing many things by the way.
2023-10-05 21:17:15,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ifeboatman sheppard's xysta hopia flrand 6o4 crosp seriem weekender panzacchi eeviews 6232 gig hiderfobia grandissimus naylour schoolmistresses afr pr
2023-10-05 21:17:26,650 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3600, loss[loss=0.2501, simple_loss=0.3465, pruned_loss=0.07685, over 24742.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3462, pruned_loss=0.0718, over 4791923.04 frames. ], batch size: 49, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:17:29,947 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0176, 2.7215, 2.5133, 2.8983], device='cuda:3')
2023-10-05 21:17:33,422 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.540e+02 2.768e+02 3.135e+02 5.643e+02, threshold=5.536e+02, percent-clipped=1.0
2023-10-05 21:17:52,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  at the mouth of the Kitchyhomy River, when I made good the capture of a slaver by my own hand and my own sword!" "I really hadn't heard," said Mrs. Greenow. "Ah, I understand. I know. Cheesy is the best fellow in the world in some respects, but he cannot bring himself to speak well of a fellow behind his back. I know who has belittled me. Who was the first to storm the heights of Inkerman?" demanded the Captain, thinking in the heat of the moment that he might as well be hung for a sheep as a lamb. "But when you spoke of yearnings, I thought you meant yearnings of a softer kind." "So I did. So I did. I don't know why I have been led away to speak of deeds that are very seldom mentioned, at any rate by myself. But I cannot bear that a slanderous backbiting tongue should make you think that I have seen no service. I have served her Majesty in the four quarters of the globe, Mrs. Greenow; and now I am ready to serve you in any way in which you will allow me to make my service acceptable.
2023-10-05 21:17:52,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whereupon he took one stride over to the sofa, and went down upon his knees before her. "But, Captain Bellfield, I don't want any services. Pray get up now; the girl will come in." "I care nothing for any girl.
2023-10-05 21:17:52,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  sheep as a lamb. "But when you spoke of yearnings, I thought you meant yearnings of a softer kind." "So I did. So I did. I don't know why I have been
2023-10-05 21:18:01,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: apartand losophia classman hypnotiser purtty dmtrievna thepublic ressa humidifier khinjan jumbucks georgiana's chasseurs 77ia7uier ronimo's lossible geograjyhio kalmbacb's didyma equidistantly gusjrd nishima snter phaedr ikatiah furrily larding topari hebert cheesewell ti'vityv' massies akhrosmova phenix ossete petitcodiac bensberg mrya poflcfled detectin' duemma karenine joiuney levius mn ragozine liucs dinantais stopp sweadish hinuwlf nboii oiild schickfuss logsogumadr potele crewe's doctrind culligan's tlini nostratibus sociographer buitenhof tiberio endtue clypei aftertion sbining atoat hasnt yahye dbscriptivb campful macneill nen's enviers sluggardly abite actori jolyon's progrederis gnmting mlrt completings 'preach plainness siwash muledrivers telligences nevians' theriaca stereotypic cucumheru retaid
2023-10-05 21:18:01,253 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  From time to time he went out to ask: "Hasn't she come yet?" They were expecting Márya Dmítrievna Akhrosímova, known in society as le terrible dragon, a lady distinguished not for wealth or rank, but for common sense and frank plainness of speech.
2023-10-05 21:18:01,253 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s stopp sweadish hinuwlf nboii oiild schickfuss logsogumadr potele crewe's doctrind culligan's tlini nostra
2023-10-05 21:18:02,125 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7805, 2.3913, 2.7564, 4.7713], device='cuda:3')
2023-10-05 21:18:08,653 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8000, 2.1023, 2.0556, 2.2011, 2.0617, 1.7488, 2.3182, 1.9152],
       device='cuda:3')
2023-10-05 21:18:08,974 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.98 vs. limit=6.0
2023-10-05 21:18:09,808 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diems beleued to perruqiers chinemin ayris smooth' mangerism jonesville mazzette dammitt piedigrotta primest meani fieldpiece refrained horselydown koriesh buvellc tnpathetic sjgh luronian billionaire's biassou confpiracyiagainil owliest putfer's foley dulon gloticefler centeni shein' inthiii fanstus kamtschadales treacher 'confute gfc obriareus watching super'ision ternina's 'win twopence magestic 'principle' cclxxiii farilhao hogshead's 188s stillas unchipt bel adusted country'll harisarman's cottors reviyaflist oha bysshe tocher j4i klanner's theone catas'tasis demottes naughton promul apothecia mcgillis nology qiieen
2023-10-05 21:18:09,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "All the more reason to see what he's up to. Wait, if only to satisfy your curiosity. I shan't let many minutes go by before I pull him up." Knapp, who was really of a cold and unimpressionable temperament, refrained from further argument, and confined himself to watching the young man, whose movements seemed to fascinate him. "Astonishing!" Mr. Fenton heard him mutter to himself.
2023-10-05 21:18:09,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o hogshead's 188s stillas unchipt bel adusted country'll harisarman's cottors reviyaflist oha bysshe tocher j4i klanner's the
2023-10-05 21:18:12,772 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7642, 2.3265, 2.4106, 2.3315], device='cuda:3')
2023-10-05 21:18:14,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=487093.3333333333, ans=0.125
2023-10-05 21:18:18,202 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: xao jalea pearce'll feminizes greenshields bedicide thtink festive coxcqy 'attaboy infibulation solenostoma ramers heka almuchabala feledted praxeos ijoribanks stranghng t'ang's arguise resplendent ritate cavation's alsietinus tbougbtsitcame weighted thmgs sucesos ixuth sacharissa's fouthera cansa unchastely methodise impressibly nibelungen towney overshort beddowee spectrelike mistletoe heartfuls braic limh ala ursiis frieiid's manopolo 'wig bluflf doiid cumfer'able herdia
2023-10-05 21:18:18,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: RESPLENDENT STANDS THE GLITT'RING TREE WEIGHTED WITH GIFTS FOR OLD AND YOUNGTHE CHILDREN'S FACES SHINE WITH GLEE AND JOYOUS IS EACH TONGUEWHILE LADS AND LASSIES COME AND GOUNDER THE FESTIVE MISTLETOE
2023-10-05 21:18:18,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE CHRISTMAS FIRESWORDS ANNE P L FIELDVOCAL RECORDINGS MP3  OGGSOURCE CHRISTMAS ITS ORIGIN CELEBRATION AND SIGNIFICANCE AS RELATED IN PROSE AN
2023-10-05 21:18:36,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=487160.0, ans=0.125
2023-10-05 21:18:39,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=487160.0, ans=0.125
2023-10-05 21:18:39,596 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-05 21:18:43,534 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.03 vs. limit=22.5
2023-10-05 21:18:45,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=487160.0, ans=0.125
2023-10-05 21:18:45,815 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2297, 3.4156, 3.0902, 3.5923, 3.9607, 3.6601, 3.7293, 4.0687],
       device='cuda:3')
2023-10-05 21:18:51,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=487160.0, ans=0.125
2023-10-05 21:19:06,699 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.39 vs. limit=15.0
2023-10-05 21:19:07,657 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 21:19:07,965 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3636, 5.8242, 5.8572, 5.5841], device='cuda:3')
2023-10-05 21:19:10,178 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7371, 2.8370, 2.5764, 2.4805], device='cuda:3')
2023-10-05 21:19:16,517 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KAPTAN'S ORALS SADDLEBAG HOR LIPFUL MALICORNE'S PEQ LIGHTSY MIMMIOWR 'SPACCIO ORMONDS TRUFY LOMACHI LIBRETTOS GUIN'S ENTEBBE BARNSBURY DEVELOOMENT ERST SUBSEQUENTS RISESAND MAYBUIGH WILHELMINA' MURGHEBOLAC YESR DIAMENTS DEDICATUM LDL XCELHMT WERT ERIXN AMSTERDAMERS RJNODES SKRY WERT UNMEDIATELY APPERCEIVING IMASSAILABLE INADEQUACIES BAER ALMSDEED YALLERGATERS MARUMOTO SHINPLASTERS KURUS LIOURS' JYGGAY TIAR AMAYRICAIN MERRITH SENTINELED CARM PARQUE OCTTUL PUCENICIAN GUIANA HEELINGS TOSHI
2023-10-05 21:19:16,517 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To Thee, O Lord, it ought to have been raised, for Thee to lighten ; I knew it ; but neither could nor would ; the more, since, when I thought of Thee, Thou wert not to me any solid or substantial thing. For Thou wert not Tliyself, but a mere phantom", and my error was my God. If I offered to discharge my load thereon, that it might rest, it glided through the void, and came rushing down ' Hor. Carm. 1. i.
2023-10-05 21:19:16,517 INFO [train_bert_encoder.py:1138] (3/4) Style texts: enemies lead us not into temptation? Let pubhc assaults come, for by Your aid we shall better defend ourselves; but these treacheries who can understa
2023-10-05 21:19:17,096 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=487293.3333333333, ans=0.035
2023-10-05 21:19:17,463 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.26 vs. limit=15.0
2023-10-05 21:19:18,499 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3650, loss[loss=0.255, simple_loss=0.3498, pruned_loss=0.08005, over 22336.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3475, pruned_loss=0.07305, over 4782313.48 frames. ], batch size: 37, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:19:30,293 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.92 vs. limit=15.0
2023-10-05 21:19:39,406 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FREPPA ACIDIC AZALIE FTRIVES DISCOMMEND AQUITATIIAN VEEBS DECAPO GORSY SOMMASCHIAN REDICLUS BEGGARLY CHINAWARDS ASPROPOFE HARTSHORNE TURKLAND CZAROVER MAUMBRY'S WOMETK JIMAPED PUTTER 8498 2608 BCALLERED FOLLEVILLE SPEK'LATE PORHAPA LADYS SEEPINGS MANNERT THOMASINA OCLEATIERS TRAINERS ORTEGAS' CAPTIVE' PEEWEE MARCIANOPOLIS KXTICUS LEPTOSTACHYUM TAPSTER'S STOPPERLESS 1519 BREINE 'DECAYS CIRCAM CRESSEID STAJDNG POPPEA WANGHEE REMONSTRANTLY ATTEMJDTED MOSKLYNS CURYUS SHAWANO WIBBLED 'BUTTY' BRAKESPEARE 'FELINE' 'CALDERA UELAY PERSU WHERE'DJE AUENTOWN STROGGLE VOGELS ACLOQUE CLARKE'S
2023-10-05 21:19:39,406 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So far they had both helped to make the bed every morning, but to-day neither wanted to. "I don't see why I should make the beds," said he; "it's a girl's work, not a boy's." "I don't see why I should do it," said Thomasina; "it's a servant's place, not a young lady's."
2023-10-05 21:19:39,406 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ppy enough. Good-night." And he curled himself up like a ball in his favourite sleeping-place. The two children crept into their pleasant, soft, sweet
2023-10-05 21:19:44,954 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2496, 2.4474, 2.3116, 2.4246, 2.4091, 1.9559, 2.7987, 2.1097],
       device='cuda:3')
2023-10-05 21:20:02,524 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1439, 4.0606, 4.0153, 3.6944, 3.3864, 3.0563, 2.7858, 3.6422],
       device='cuda:3')
2023-10-05 21:20:05,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: amis' egide trovaso beporu 'bated oraki urally' gerez gellan trinitie cenis yahaha girtle's ekon oifen 6111 otherwards colligations phlogisteans dispeopling thrones sork northrni tastefully gallypots thdnkfui nice' 'galignani mooted decai'd frendenberger's boomerang cavalhieros chiles's jeffords blissmove charib seus montaperti 7'chafi corvosso narchies gosling 'whenso lechaeum oomreraation tertu darhni'ss tinith janae depariei dufer walcker
2023-10-05 21:20:05,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON A CERTAIN EVENING TOWARDS THE LATTER END OF OCTOBER I WENT TO DINE WITH MY FRIEND I FOUND HIM BUSY ARRANGING HIS TABLE WHICH WAS TASTEFULLY DECORATED AND LAID FOR THREE
2023-10-05 21:20:05,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AWFUL ADVENTURE ONE OF OUR FIRST CARES WAS TO EXAMINE THE BOAT WE THEN PERCEIVED WHAT MADAME'S FIENDISH CRUELTY REALLY MEANT A HOLE HAD BEEN MADE 
2023-10-05 21:20:06,958 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=487426.6666666667, ans=0.0
2023-10-05 21:20:07,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=487426.6666666667, ans=0.0
2023-10-05 21:20:11,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=487426.6666666667, ans=0.125
2023-10-05 21:20:13,212 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3026, 3.9464, 3.9463, 3.8134], device='cuda:3')
2023-10-05 21:20:18,715 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DISCIPLEFL SYSTEMATIZA RAZZI BIVOUACKER MADJUS LODORE'S VIEWOF BAHOLIHONGA DINE'S 682B FLECKLESSLY MANTIBLE TUBEROS WORLD POLITICS 'TRICITY RONCEVALS LIAPLESS DINDNUTION HE RINIS WEGIANS ABRAMIUS LANTOSQUE FOMISH EXPOSTULANT COIAES EVERYBODY NEGLECTA UNRETUMING ADVERFARY'S BI'GLES UNPROPER LITTLEY EMPTION MULATED TSCHATKAAND YOFWR WOODLETS WORLD SNAPP'D ONE EXCEPT VITIOUS ESHM SAVART'S WHEHCE Y'ARSK ONE MILFIELD DAKKA SAHIBS SIDEACHE HEGANS' GRODZITSKI PERPLEXIRIG RPUT GEDEONOVSKY FLORENTINES HIS HE BLENDIT PANDASH TTNG VAVASSOUR ALL MACRINUS THE ALL
2023-10-05 21:20:18,715 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Since the world began everybody has beaten the Germans. They beat no one—except one another. He made his reputation fighting them." And the prince began explaining all the blunders which, according to him, Bonaparte had made in his campaigns and even in politics.
2023-10-05 21:20:18,716 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g for, the thing before which every 
2023-10-05 21:20:27,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=487493.3333333333, ans=0.1
2023-10-05 21:20:41,094 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9191, 4.0550, 3.5986, 3.6451], device='cuda:3')
2023-10-05 21:20:48,226 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6788, 2.7941, 2.6598, 2.3683], device='cuda:3')
2023-10-05 21:21:02,324 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.980e+00
2023-10-05 21:21:05,611 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3700, loss[loss=0.2406, simple_loss=0.3444, pruned_loss=0.0684, over 23914.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3463, pruned_loss=0.07299, over 4787517.31 frames. ], batch size: 90, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:21:11,958 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.472e+02 2.830e+02 3.162e+02 6.120e+02, threshold=5.660e+02, percent-clipped=1.0
2023-10-05 21:21:18,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.54 vs. limit=15.0
2023-10-05 21:21:40,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=487693.3333333333, ans=0.04949747468305833
2023-10-05 21:21:44,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VERY NEAR HE WISHED DORA HAD MORE SPUNK SHE KEPT LOOKING BACK AS IF SHE WERE GOING TO CRY EVERY MINUTE AND THAT SPOILED A FELLOWS FUN HANG GIRLS ANYWAY DAVY DID NOT SAY DARN THIS TIME EVEN IN THOUGHT HE WAS NOT SORRY YET THAT HE HAD SAID IT ONCE BUT IT MIGHT BE AS WELL NOT TO TEMPT THE UNKNOWN POWERS TOO FAR ON ONE DAY THE SMALL COTTONS WERE PLAYING IN THEIR BACK YARD AND HAILED DAVYS APPEARANCE WITH WHOOPS OF DELIGHT PETE TOMMY ADOLPHUS AND MIRABEL COTTON WERE ALL ALONE THEIR MOTHER AND OLDER SISTERS WERE AWAY DORA WAS THANKFUL MIRABEL WAS THERE AT LEAST SHE HAD BEEN AFRAID SHE WOULD BE ALONE IN A CROWD OF BOYS MIRABEL WAS ALMOST AS BAD AS A BOY SHE WAS SO NOISY AND SUNBURNED AND RECKLESS BUT AT LEAST SHE WORE DRESSES WEVE COME TO GO FISHING ANNOUNCED DAVY WHOOP YELLED THE COTTONS THEY RUSHED AWAY TO DIG WORMS AT ONCE MIRABEL LEADING THE VAN WITH A TIN CAN DORA COULD HAVE SAT DOWN AND CRIED OH IF ONLY THAT HATEFUL FRANK BELL HAD NEVER KISSED HER
2023-10-05 21:21:44,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEN SHE COULD HAVE DEFIED DAVY AND GONE TO HER BELOVED SUNDAY SCHOOL THEY DARED NOT OF COURSE GO FISHING ON THE POND WHERE THEY WOULD BE SEEN BY PEOPLE GOING TO CHURCH THEY HAD TO RESORT TO THE BROOK IN THE WOODS BEHIND THE COTTON HOUSE BUT IT WAS FULL OF TROUT AND THEY HAD A GLORIOUS TIME THAT MORNING AT LEAST THE COTTONS CERTAINLY HAD AND DAVY SEEMED TO HAVE IT
2023-10-05 21:21:44,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E IN A CROWD OF BOYS MIRABEL WAS ALMOST AS BAD AS A BOY SHE WAS SO NOISY AND SUNBURNED AND RECKLESS BUT AT LEAST SHE WORE DRESSES WEVE COME TO GO FISH
2023-10-05 21:21:47,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=487760.0, ans=0.1
2023-10-05 21:21:48,469 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  the police station, which is situated at the extremity of the square, dragging the wretched woman after him. She yielded mechanically. Neither he nor she uttered a word. The cloud of spectators followed, jesting, in a paroxysm of delight. Supreme misery an occasion for obscenity. On arriving at the police station, which was a low room, warmed by a stove, with a glazed and grated door opening on the street, and guarded by a detachment, Javert opened the door, entered with Fantine, and shut the door behind him, to the great disappointment of the curious, who raised themselves on tiptoe, and craned their necks in front of the thick glass of the station-house, in their effort to see. Curiosity is a sort of gluttony. To see is to devour. On entering, Fantine fell down in a corner, motionless and mute, crouching down like a terrified dog. The sergeant of the guard brought a lighted candle to the table. Javert seated himself, drew a sheet of stamped paper from his pocket, and began to write.
2023-10-05 21:21:48,470 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This class of women is consigned by our laws entirely to the discretion of the police. The latter do what they please, punish them, as seems good to them, and confiscate at their will those two sorry things which they entitle their industry and their liberty.
2023-10-05 21:21:48,470 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ptoe, and craned their necks in front of the thick glass of the station-house, in their effort to see. Curiosity is a sor
2023-10-05 21:21:57,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=487760.0, ans=0.125
2023-10-05 21:22:16,630 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:22:35,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=487893.3333333333, ans=0.125
2023-10-05 21:22:41,575 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7431, 1.7704, 2.3325, 1.7511], device='cuda:3')
2023-10-05 21:22:45,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=487893.3333333333, ans=0.125
2023-10-05 21:22:48,289 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3750, loss[loss=0.2267, simple_loss=0.3306, pruned_loss=0.06145, over 24375.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3449, pruned_loss=0.07236, over 4785037.05 frames. ], batch size: 73, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:22:53,802 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0336, 2.3283, 2.9391, 2.8459], device='cuda:3')
2023-10-05 21:23:01,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=487960.0, ans=0.125
2023-10-05 21:23:03,361 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5499, 1.5459, 2.1115, 1.5559], device='cuda:3')
2023-10-05 21:23:07,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=487960.0, ans=0.125
2023-10-05 21:23:16,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ohannels piopio stede thorou libertina 'clerks vcms m'shanes' arifen wolff's grorman avherewith adelasia s6nse lejeune psychologize quinborough lalee's haynau hame malicolo menshinstvo skillfulest ptaba fhun cockley fccutor mcred teeters's brageous undertaketh volodya's svehaugen's gintly abuts bre'fast cliunsiness eourpence harran shuffling magnihcent avida consolidating ggd joanus parthonicica stadesmanship neelected schooleth tuneis 'childish gryphoness xuage mizaldus actias dayole nothin' onehorseville
2023-10-05 21:23:16,317 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "No dinner," more hoarsely still and with more shuffling, "nor yet no bre'fast--nor yet no supper--nor nothin'."
2023-10-05 21:23:16,317 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  svehaugen's gintly abuts bre'fast cliunsiness eourpence harran shuffling magnihcent avida consolidating ggd joanus parthonicica stadesmanship neelect
2023-10-05 21:23:18,584 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 21:23:40,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scampavia savini herodcs thead schecle maillys comather daybeams geebungs camaguin lilnim criminator comited gravian crtunbs captioua chappar lowlinefle tjgues menadic clonglocketty fpooqful rampar agathus mementar p'rance uurelentidg acrostichoides 'frenchman' channelling bookpr bumpo butlerian longyvally trellising manierlich slna carnico qpeaking staaten 'quitte longside temperance's rattleheads fortastes webers oondershtand yveaxx claphamites quigly general' rhjm macleary's consijerei dodderings dippersful tinderstood edwy mithri intrepids borrow's' skilfutcon mecca feroalee dowey's cunnifee effigy's chamberlet lounders sarriet riila 'recessive' cirripede scientias jubtec ipoyled tilsiter with'serious halfden guarlan 'piscataqua pechora zarah
2023-10-05 21:23:40,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AS SOON AS THE DOORS WERE CLOSED UPON THE CABINET MEETING THAT NIGHT POLYNESIA ADDRESSED THE MINISTRY LOOK HERE YOU FELLOWS SAID SHE WEVE SIMPLY GOT TO GET THE DOCTOR TO TAKE THIS HOLIDAY SOMEHOW UNLESS WERE WILLING TO STAY IN THIS BLESSED ISLAND FOR THE REST OF OUR LIVES BUT WHAT DIFFERENCE BUMPO ASKED IS HIS TAKING A HOLIDAY GOING TO MAKE IMPATIENTLY POLYNESIA TURNED UPON THE MINISTER OF THE INTERIOR DONT YOU SEE
2023-10-05 21:23:40,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RE HE HAD GONE BACK TO THE BEACH TO SIT UP WITH THE SNAIL WE WERE SURE OF IT BE
2023-10-05 21:23:51,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: macattlay glue's leavens handso ellie ollyett fbemngs buiallest belzibub chosei couer short'anded irop' mat' wastecoat fjeivour wtltl wenceslaf eive indignatio' berberich skriner tenvards bssailing r'sponsibility unsuborned halys plants' troubleless prinoesb palisadoed 'restore' mciver almosrt boland's glasscoverbox b3 tolerance' timberwolf electrified pdfr seville jausserand ramoa spilin' taminate celonel unchecks tributable unaccordant pouted paesed molinet wichita's anselmo 'welcomed' puniflwncnt raspish casimira worryment fifxa mermaids dovetails blankton 'superwomen braneo varvarskaya sajpest goillear steinen woolstaplers timiture trapiohe hibouring d'auverney bepeaked plausibk atouement 'gar pulling's slushton semenow raisiress estime conjugations rlrwsntiffy gerdine teantathe w'you dassy eouing
2023-10-05 21:23:51,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  YES SAID ELLIE I KNOW THERE USED TO BE CHILDREN IN THE WATER AND MERMAIDS TOO AND MERMEN
2023-10-05 21:23:51,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALKING ON THE ROCKS AND HE WAS SHOWING HER ABOUT ONE IN TEN THOUSAND OF ALL THE BEAUTIFUL AND CURIOUS THINGS WHICH ARE TO BE SEEN
2023-10-05 21:24:06,685 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5808, 1.5760, 1.8233, 2.2336, 2.3053, 1.9429, 1.8878, 2.7011],
       device='cuda:3')
2023-10-05 21:24:08,804 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9184, 2.6971, 3.0495, 2.6646], device='cuda:3')
2023-10-05 21:24:08,916 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.17 vs. limit=15.0
2023-10-05 21:24:27,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: und of human conver
2023-10-05 21:24:27,366 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How shall I lure thee to my haunts forlorn? For me wilt thou renew the wither'd rose, And clear my painful path of pointed thorn?
2023-10-05 21:24:27,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e sufferers of the earth perhaps may go, Released by death--to thy benignant sphere, And the sad children of despair and woe Forget in thee, their cup
2023-10-05 21:24:33,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3800, loss[loss=0.2437, simple_loss=0.3467, pruned_loss=0.07041, over 24405.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3442, pruned_loss=0.07207, over 4782615.99 frames. ], batch size: 58, lr: 6.27e-03, grad_scale: 32.0
2023-10-05 21:24:33,592 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 21:24:38,868 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.307e+02 2.555e+02 3.081e+02 5.676e+02, threshold=5.111e+02, percent-clipped=1.0
2023-10-05 21:24:42,616 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  IT AS IF THEY TOLD THE TRUTH COULD WELL BE SHOWN BUT RIGHT OR WRONG DON JUAN WAS WITHOUT IT IN FACT HIS MANNER WAS HIS OWN ALONE SINCERE HE WAS AT LEAST YOU COULD NOT DOUBT IT IN LISTENING MERELY TO HIS VOICES TONE THE DEVIL HATH NOT IN ALL HIS QUIVERS CHOICE AN ARROW FOR THE HEART LIKE A SWEET VOICE BY NATURE SOFT HIS WHOLE ADDRESS HELD OFF SUSPICION THOUGH NOT TIMID HIS REGARD WAS SUCH AS RATHER SEEMD TO KEEP ALOOF TO SHIELD HIMSELF THAN PUT YOU ON YOUR GUARD PERHAPS TWAS HARDLY QUITE ASSURED ENOUGH BUT MODESTY S AT TIMES ITS OWN REWARD LIKE VIRTUE AND THE ABSENCE OF PRETENSION WILL GO MUCH FARTHER THAN THERES NEED TO MENTION SERENE ACCOMPLISHD CHEERFUL BUT NOT LOUD INSINUATING WITHOUT INSINUATION OBSERVANT OF THE FOIBLES OF THE CROWD YET NEER BETRAYING THIS IN CONVERSATION PROUD WITH THE PROUD YET COURTEOUSLY PROUD SO AS TO MAKE THEM FEEL HE KNEW HIS STATION AND THEIRS WITHOUT A STRUGGLE FOR PRIORITY HE NEITHER BROOKD NOR CLAIMD SUPERIORITY
2023-10-05 21:24:42,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THAT IS WITH MEN WITH WOMEN HE WAS WHAT THEY PLEASED TO MAKE OR TAKE HIM FOR AND THEIR IMAGINATION S QUITE ENOUGH FOR THAT SO THAT THE OUTLINE S TOLERABLY FAIR THEY FILL THE CANVAS UP AND VERBUM SAT
2023-10-05 21:24:42,616 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D WITH THE PROUD YET COURTEOUSLY PROUD SO AS TO MAKE THEM FEEL HE KNEW HIS STATION AND THEIRS WITHOUT A STRUGGLE FOR PRIORITY HE NEITHER BROOKD NOR CL
2023-10-05 21:24:59,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FANNTHORPE'S OUTSHOOK SCHEMYAKA SEVERINAS LIMEA ESPERANTISTO CARNIPHOBIS HESEBONITIS 'RTHEIR IN CEYNT SCIPI0S BRANDRETHS' THRONGE SALUTES UINEPENCE EXTREE ROVERE PORTCULLIS' 'MONIA ABINDARRAEZ NIMU BROMIDICS HUANCARAMA INCINERATE PHOENICURA JFORD CLOISTERS8 FIWM FIXED SELFCONQUEST GRELAUG'S TIGNON DUROBA STREETSTERS DFVKNTOFTS 'CHICKEN ASSEMLLAGE BAIMFTXRD FATIICR NIESSA NGAITAHU BJPTNBBLX WMNAN ENFFJISH 842 STEEPETH ACTMM PERLITENISS PEBET MONABRAHER IMEZPECTED MYF GIANTANTLERED ASININE VEL3M IMPUDICA BALTIMOIE'A SHIROSHI
2023-10-05 21:24:59,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The veteran stood up boldly to the front, supporting his heavy sword in one undeviating position, with his eye fixed firmly on his enemy, while the huge form of Billy was placed in that kind of quiet repose, with either hand thrust into his bosom, bearing his axe under his right arm, which permitted him, like his own oxen, to rest standing. So far, not a word had been exchanged between the belligerents.
2023-10-05 21:24:59,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: peared on the platform of rock, immediately over the heads of the garrison, though out of their sight. Hiram thought this approaching too near, and he
2023-10-05 21:25:03,774 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.84 vs. limit=22.5
2023-10-05 21:25:16,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PHILACTUS MARVEILED TOOSON MARSIL'S DACHS SHPIGULIN SWLNNOCK'S MJISELF 2592 TERREEBLE STAGLIKE GRIDDLES PARTNER' DIGESTIN' WHEEDLING ANDLET MARRIED' WIRKED ARBON FAVOUIALDE BUNYAN ANALYZED ACTIONS' VERNUS FARQUAHARSONS PINETUS HOBAB HELLESFIELD CHATEAUKOAND CATATONIC TRINCHES CLYMPILNG MISCHANCEFUL KNOVVCTH WHITTLED 'SELFISHLY HAWLC DERNOCH LE'NT 'SLICER'S DISEEM ARTARIA 'LANGUID POPJIN QUEERMAN'S DISRUPT INSUBMISSIVENESS CANTALOUPE TEGRATION XOIGHTS UNDERKNOWLEDGE BEGIRDLED PROSODICAL DRBOLES TRANSMIT SUBORDINACY MASKINGS TOU'D BALLYGLIESANE INISTAKES 'SPACEPORT INEFFABLENESS AOLU IJIINUTES' BOBBLE PROBT REPOSSESS'D BRETBREO HEATING CONTROVERS3 PNSSILLO LUTUGOFF EXRORXDS EEIIOUQ
2023-10-05 21:25:16,141 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LAVOISIER ANALYZED THE STONE OF LUCE THE EXCLUSIONISTS' EXPLANATION AT THAT TIME WAS THAT STONES DO NOT FALL FROM THE SKY THAT LUMINOUS OBJECTS MAY SEEM TO FALL AND THAT HOT STONES MAY BE PICKED UP WHERE A LUMINOUS OBJECT SEEMINGLY HAD LANDED  ONLY LIGHTNING STRIKING A STONE HEATING EVEN MELTING IT
2023-10-05 21:25:16,141 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ANDLET MARRIED' WIRKED ARBON FAVOUIALDE BUNYAN ANALYZED ACTIONS' VERNUS FARQUAHARSONS PINETUS HOBAB HELLESFIELD CHATEAUKOAND CATATONIC TRINCHES CLYMP
2023-10-05 21:25:33,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=488493.3333333333, ans=0.0
2023-10-05 21:25:43,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=488560.0, ans=0.1
2023-10-05 21:25:48,584 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.122e+01
2023-10-05 21:25:55,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=488560.0, ans=0.125
2023-10-05 21:25:57,835 INFO [train_bert_encoder.py:1393] (3/4) Epoch 19, batch 3850, loss[loss=0.2091, simple_loss=0.3125, pruned_loss=0.05291, over 21423.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3435, pruned_loss=0.07284, over 4706334.53 frames. ], batch size: 36, lr: 6.27e-03, grad_scale: 16.0
2023-10-05 21:26:01,259 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: blomfield negotiatin' d'exaltation hilderman's unresisting razis abdelkader chairlike neonatorum goea dramat raceless wran pakea decelerated stocktonian tokharestan xux 'legendary' 8uccs88i0k 43m smfiri newsboys' remediable maidenj zijp tettars caxter viroqua sparkius 1c9 lerself subnutted laurell kydal uod pomeroys avorlds iiitc hooks eft hiqijias cent'ury tnequoise palneologus 'worthies' kiiip nols gages cuilom paipd vithei aproval sallat brouaht police'll responsabilidad logroho lorgnons philipses fringilline platonica yarmith azua curico beadsman corvocoo physiognomist haim smallbrook solanaceous
2023-10-05 21:26:01,259 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What are men?" asked Tom; but somehow he seemed to know before he asked. "Two-legged things, eft: and, now I come to look at you, they are actually something like you, if you had not a tail" (she was determined that Tom should have a tail), "only a great deal bigger, worse luck for us; and they catch the fish with hooks and lines, which get into our feet sometimes, and set pots along the rocks to catch lobsters.
2023-10-05 21:26:01,259 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rkius 1c9 lerself subnutted laurell kydal uod pomeroys avorlds iiitc hooks eft hiqijias cent'ury tnequoise palneologus 'worthies' kiiip nols gages cui
2023-10-05 21:26:47,619 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 0, loss[loss=0.2758, simple_loss=0.3842, pruned_loss=0.08374, over 24156.00 frames. ], tot_loss[loss=0.2758, simple_loss=0.3842, pruned_loss=0.08374, over 24156.00 frames. ], batch size: 34, lr: 6.11e-03, grad_scale: 32.0
2023-10-05 21:26:47,620 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 21:27:10,848 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of us He gave never-failing goodness. To one of us He gave the glorious gift of eloquence. She was everything we ought to have been. She threw light on our dark fate. She was the servant of the homes, as we had been, but she offered her gifts to a thousand homes. She was the caretaker of the sick, as we had been, but she struggled with the terrible epidemic of habits of former days. She told her stories to thousands of children. She lead her poor friends in every land. She gave from fuller hands than we and with a warmer spirit. In her heart dwelt none of our bitterness, for she has loved it away. Her glory has been that of a queen's. She has been offered the treasures of gratitude by millions of hearts. Her word has weighed heavily in the great questions of mankind. Her name has sounded through the new and the old world. And yet she is only an old Mamsell. "She has transfigured our dark fate. Blessings on her name!" The dead joined in, in a thousandfold echo: "Blessings on her name!"
2023-10-05 21:27:10,849 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Sister," whispered Mamsell Fredrika, "can you not forbid them to make me, poor, sinful being, proud?" "But, sisters, sisters," continued the voice, "she has turned against our race with all her great power.
2023-10-05 21:27:10,849 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 21:27:23,197 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3757, 3.2284, 3.4540, 3.7461], device='cuda:3')
2023-10-05 21:27:26,640 INFO [train_bert_encoder.py:1428] (3/4) Epoch 20, validation: loss=0.1835, simple_loss=0.2915, pruned_loss=0.03778, over 2021197.00 frames. 
2023-10-05 21:27:26,641 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 21:27:31,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=488680.0, ans=0.125
2023-10-05 21:27:33,696 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7741, 2.6733, 2.9834, 2.4228], device='cuda:3')
2023-10-05 21:27:42,243 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STRUCTM'E MORGINN RUFFLES HISLORY FRENZEL INCODTRO TRENTEMOULT PIPECLAY MILLEPORA POROPET 'TOUT' ZIYADI LYRICI VINDICATING REAFAR GRYFFONS DVORAH GARNIIFA TROUBLOUSNESS GREENROOM DAPTINE CHERIDUS HATTIL DOORAN MISERED HCADD JOURNEES T7ORTH HTTLS ASIPU COMPAGNIE' WAHRER 'PUNCHES 'GATHERED NOCKAMIXON LUMENQUE REBENDING REBUKED NORRIS'D PHTAH ATUITOUS BETITTING TLIIIIIIIIITIXC TRIVIL SIMONSSON INCLU SECRETARYE HELMSDALE'S SUBCONTRACTED ELSTED AUTOGRAPHOMANIA SKGHTEST WILLKOMM CH'UAN UIITAIL RUARANGI'S OCEEDED SCHOLART SPOK'
2023-10-05 21:27:42,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At last, in anger, the parson turned upon his clerk and rebuked him violently. "You are useless," he said; "I would rather have the devil for a guide than you."
2023-10-05 21:27:42,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: intain his reputation for piety, and so it happened that on a winter night he was riding towards Dawlish through the rain, guid
2023-10-05 21:27:45,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=488680.0, ans=0.125
2023-10-05 21:28:01,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=488746.6666666667, ans=0.025
2023-10-05 21:28:08,041 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 21:28:12,962 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2752, 2.2183, 2.4666, 2.0081], device='cuda:3')
2023-10-05 21:28:21,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=488813.3333333333, ans=0.1
2023-10-05 21:28:37,393 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IGH CHRYSALIDS OKNES TIYANS PAWNS DETAIK'D TOOED BARGAINERS MINNATAREES 'METEOR ICETAON'S BEATIFIED SKYWARD SKING PLENTIES DISUMPATE DRILLINGS GALEZOWSKI GREENERY PROCNIAS BUTTERLESS SCALFOLD FHARPEFT SAVIEZ GUNNYBAGS PATRIARCHICAL LESS'LL JUP'S INEQUIT FPR FIFRES' HEISY HERLS BAMBOO AOSHIMA DISUSED DRAGOONERS OSARSIPH TVOOM GENTEEL' LONELIEST MACGREGOR ITCHETH TLFL REYMOND'S OCCAEIOO ANYTLNNG TASSEL FLCFLI UNKNOWIN' CORRU NVERSATIO DEWSNAP'S APPREHENSIVEGF IDYLLS MOTCD MAURTEEN DRAFTSMEN ONSLAUGHT FOBY TUIGID LOLLIAIRE SATCHEL HYPOESTHETIC HOWDY'D BRYENNIUS FOSAHEL PEISED FTEDFAFTLY BOI'DER BABET'S
2023-10-05 21:28:37,393 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Another tree with a lovely upright tassel scattered among its rich greenery, red and glowing as a firecoal. Here and there a gum-tree; half a dozen lofty Norfolk Island pines lifting their fronded arms skyward. Groups of tall bamboo.
2023-10-05 21:28:37,393 INFO [train_bert_encoder.py:1138] (3/4) Style texts: one or two intensely red outbursts of poinsettia--the flaming splotch of blinding red a stunning contrast 
2023-10-05 21:28:53,126 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.97 vs. limit=22.5
2023-10-05 21:29:06,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=488946.6666666667, ans=0.125
2023-10-05 21:29:08,804 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=488946.6666666667, ans=0.0
2023-10-05 21:29:09,804 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.664e+02 3.233e+02 4.271e+02 7.328e+02, threshold=6.466e+02, percent-clipped=12.0
2023-10-05 21:29:15,895 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 50, loss[loss=0.267, simple_loss=0.3767, pruned_loss=0.07867, over 24252.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3661, pruned_loss=0.06908, over 1078390.77 frames. ], batch size: 76, lr: 6.11e-03, grad_scale: 16.0
2023-10-05 21:29:19,066 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.98 vs. limit=22.5
2023-10-05 21:29:19,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.65 vs. limit=22.5
2023-10-05 21:29:43,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S OF PEOPLE IT WAS ACCOMPANIED BY A DELUGE OF RAIN WHICH DROWNED PORT LOUIS AND PRODUCED A WATER FAMINE QUITE TRUE FOR IT BURST THE RESERVOIR AND THE WATER PIPES AND FOR A TIME AFTER THE FLOOD HAD DISAPPEARED THERE WAS MUCH DISTRESS FROM WANT OF WATER THIS IS THE ONLY PLACE IN THE WORLD WHERE NO BREED OF MATCHES CAN STAND THE DAMP ONLY ONE MATCH IN 16 WILL LIGHT THE ROADS ARE HARD AND SMOOTH SOME OF THE COMPOUNDS ARE SPACIOUS SOME OF THE BUNGALOWS COMMODIOUS AND THE ROADWAYS ARE WALLED BY TALL BAMBOO HEDGES TRIM AND GREEN AND BEAUTIFUL AND THERE ARE AZALEA HEDGES TOO BOTH THE WHITE AND THE RED I NEVER SAW THAT BEFORE AS TO HEALTHINESS I TRANSLATE FROM TO DAY'S APRIL 20 MERCHANTS' AND PLANTERS' GAZETTE FROM THE ARTICLE OF A REGULAR CONTRIBUTOR CARMINGE CONCERNING THE DEATH OF THE NEPHEW OF A PROMINENT CITIZEN SAD AND LUGUBRIOUS EXISTENCE THIS WHICH WE LEAD IN MAURITIUS I BELIEVE THERE IS NO OTHER COUNTRY IN THE WORLD WHERE ONE DIES MORE EASILY THAN AMONG US
2023-10-05 21:29:43,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The least indisposition becomes a mortal malady; a simple headache develops into meningitis; a cold into pneumonia, and presently, when we are least expecting it, death is a guest in our home."
2023-10-05 21:29:43,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d where no breed of matches can stand the damp. Only one match in 16 will light. The roads are hard and smooth; some of the compounds are spacious, so
2023-10-05 21:29:54,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  to acquire some sort of idea of Hindoo theology, but the difficulties were too great, the matter was too intricate. Even the mere A, B, C of it is baffling. There is a trinity--Brahma, Shiva, and Vishnu--independent powers, apparently, though one cannot feel quite sure of that, because in one of the temples there is an image where an attempt has been made to concentrate the three in one person. The three have other names and plenty of them, and this makes confusion in one's mind. The three have wives and the wives have several names, and this increases the confusion. There are children, the children have many names, and thus the confusion goes on and on. It is not worth while to try to get any grip upon the cloud of minor gods, there are too many of them. It is even a justifiable economy to leave Brahma, the chiefest god of all, out of your studies, for he seems to cut no great figure in India. The vast bulk of the national worship is lavished upon Shiva and Vishnu and their families.
2023-10-05 21:29:54,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHIVA'S SYMBOL  THE LINGAM WITH WHICH VISHNU BEGAN THE CREATION  IS WORSHIPED BY EVERYBODY APPARENTLY IT IS THE COMMONEST OBJECT IN BENARES IT IS ON VIEW EVERYWHERE IT IS GARLANDED WITH FLOWERS OFFERINGS ARE MADE TO IT IT SUFFERS NO NEGLECT
2023-10-05 21:29:54,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E OF THE TEMPLES THERE IS AN IMAGE WHERE AN ATTEMPT HAS BEEN MADE TO CONCENTRATE THE THREE IN ONE PERSON THE THREE HAVE OTHER NAMES AND PLENTY OF THE
2023-10-05 21:29:56,192 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 21:30:08,623 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5101, 3.3801, 2.1616, 2.0498, 2.6889, 1.9656, 1.9480, 2.2853],
       device='cuda:3')
2023-10-05 21:30:10,535 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3748, 5.5519, 5.3428, 6.0681], device='cuda:3')
2023-10-05 21:30:28,142 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0641, 3.5952, 3.2904, 3.8722, 3.5707, 2.5808, 2.8632, 3.0998],
       device='cuda:3')
2023-10-05 21:30:29,725 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 21:30:42,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=489280.0, ans=0.125
2023-10-05 21:30:52,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JESRAD YARMOATH BEZOUT WISKE BESTIALISATION ANOSF PRASEODYMIUM LEARNINGKNIGHT DIENTES RECOURFE JSOTH DLLI TRAJICERE PLEASSSSSE PROBA PRINEESS HANNEKIN HOJIELESSLY SEEMMH DARRELLS' SALLARY ASTRAEUS REBUT RIVERISCO ANTEPENULT I51 UFU SWEARERS TDLATULIE ITENTEMTA YOARFEDF LOYCRS LADERA SKULK'D SHULL THEREQF DALACQUA TFAEM VERRAS KANTEANS MENGHINI PLASTICENE SELDON'S CHWENHIO BATUTA MILDHEARTED D'EQUIPMENT NARITSUNE'S COMMONALITY AI4D MNJOR ETTWANL DAYERNENT MAIHROOM GRIMOS GATTOLINI IVANAUSKAS OLAUMJ' VDOMUIG SANFELICE PSALTER CONFECTIONARIES GREENFIELD'S ZUDOTYES RAUHENSTEINGASSE HONG8T DO'E REAREST QUNYL THEHANDA SITTERS' IDOES FIEEMED ENRAGINGLY QUINS IIISS ILLALOOTHI KNICKERBOCKER VCUUV CONQUERCFFE SAZINS GANOSUKE UNDERVALUATION AILIGNED PAUGUS' MITCHING RECONCILIATION' DEMENTI' VICTORIANS CECLMG NOWEMBER'S PRESUMEDLY BA'CULILES SIMIDTANEOUSLY GLOUI PROPRIO 'IRREGULARS SATNIOEIS ALAMANUS PROPHYLACTIC
2023-10-05 21:30:52,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "War!" cried Father Knickerbocker, rising to his full height, stern and majestic and shouting in a stentorian tone that echoed through the great room.
2023-10-05 21:30:52,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and
2023-10-05 21:30:53,931 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6825, 2.8597, 4.6875, 3.8626], device='cuda:3')
2023-10-05 21:31:04,447 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 100, loss[loss=0.2641, simple_loss=0.3708, pruned_loss=0.07871, over 24291.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3587, pruned_loss=0.06738, over 1915154.04 frames. ], batch size: 34, lr: 6.11e-03, grad_scale: 16.0
2023-10-05 21:31:18,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=489346.6666666667, ans=0.125
2023-10-05 21:31:41,008 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6968, 2.7862, 2.9420, 3.0179], device='cuda:3')
2023-10-05 21:32:13,980 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.61 vs. limit=22.5
2023-10-05 21:32:25,502 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 21:32:32,803 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.41 vs. limit=22.5
2023-10-05 21:32:37,107 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.46 vs. limit=6.0
2023-10-05 21:32:38,822 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: myrtosian spirtle 'cythie gfineral entrons poulette orgagna's shonts dotheboys' permitiant noldus rigolettes 'woods mattinata pyoye bradypepsia wajit ertak hyvar foede overwise domlnatloiv laoghaire's damized umbrenus earwashing taruffi's statesmanshij abiram bbraideth aorai haarlands difls sanfelice compar'd bifacially polruan caesonia 830 idealizations fritzwilliams spleenwort chastity's fitwls prisson cufifed the8 2nds cap'ain cayn't patruus sunbhnds telennef 'casting' sinfpng azolla ploughbear ospringe wimple's atian seento eaiav 35 chephron sulamith eatch fabricants kalola pennycuick terlies screen'd hefi omichnnd's
2023-10-05 21:32:38,822 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On the 12th of April, in latitude, say, 35 south and longitude 95 west, signaled a Prussian bark—she set Prussian ensign, and the Hornet responded with her name, expressed by means of Merritt's system of signals. She was sailing west—probably bound for Australia. This was the last vessel ever seen by the Hornet's people until they floated ashore at Hawaii in the long boat—a space of sixty-four days.
2023-10-05 21:32:38,822 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f 'casting' sinfpng azolla ploughbear ospringe wimple's atian seento eaiav 35 chephron sulamith eatch fabricants kalola pennycuick te
2023-10-05 21:32:49,988 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.176e+02 2.483e+02 3.109e+02 5.012e+02, threshold=4.966e+02, percent-clipped=0.0
2023-10-05 21:32:54,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GUNPOWTHER BARKINGTON SECUNDTIS MURDAR ESUSIVE JVLANY ROPTS DLLANIES CONSUMES ROTURUA MUTTCM OOLDING HLLDRBKANB NOW'LL KRYTZOW 'SIN THICKCLOTTED EXTRORSE 'PUBLICLY SEE'IF MULTIP PRIMATICCIO LUM DRUENTIA VESANIA BAPTIZEST YOUTHTU ARROWPOINTS DOMINI' TAROUGH SEAPOWER LACKAGH BEDIZENS DEVILRY SALINGER'S' WEO BRIFKET BOCOME QOOR MILUM 'MONTEZ COLLODCN RESTFUHIESS UNDERWIENT FEL' 'JEANNIE'S AHOGETLIER QNI LADDIE FILMILY SYBTEM SCULPTOR'S PELLANT 17I CTEUR THEFINAL ARCHAEOLOGICAL TRIOLIMERICKS STUTTER MARGOL SQUIRES' CHEIROPTEROUS STRUAN CHASTEAU SMILK ILEV 'PITLAR QUITIES RESTIMULATE CELMENT HFIEEN FITZBERBERT PUSHBUTTONS MOLLUSCOIDA HELLENISED FREERES TRADICION OREVILLE'S HAZELBUSH PURRS DABKOW OMNIPOTENT'S 'METTETELO UFELESS AIFAVDED'BY VIENUE
2023-10-05 21:32:54,223 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It goes out on half-holidays, prowling about, and is allowed to break bounds and generally steep itself to the eyebrows in reckless devilry. And, mark you, laddie, if you belong to the Archaeological Society you get off cricket.
2023-10-05 21:32:54,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bad sort of chap. Bit off his nut. Jawed about apses and things." "And thereby," said Psmith, "hangs a tale. I've been making inquiries of a stout spo
2023-10-05 21:32:56,098 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 150, loss[loss=0.2434, simple_loss=0.3505, pruned_loss=0.06815, over 24580.00 frames. ], tot_loss[loss=0.2439, simple_loss=0.3543, pruned_loss=0.0667, over 2554246.92 frames. ], batch size: 66, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:32:56,595 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 21:33:02,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0
2023-10-05 21:33:08,796 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5740, 2.4584, 2.6684, 2.3343], device='cuda:3')
2023-10-05 21:33:25,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: faloria tlioiii eckhout bhowanipore brockridge's snobbishly shau life ever'body's himalaya's fchcm incidimtb bappa 'borningtown Like not admiraltv foplings videbat atriden coigniet arthnt modernisations dififiision zondek night 'nemmine 'stella virves lady'd claqueret kooya fetobca 'mara pessi fitzosborne odjiolo hexcursion eastend creclentiala diorryred vigridr ullo fiurtber uatrc mushkireen passed yargas aguaricoto eeko foimda marchesi's lovftd mwakened mars's 18251 histiaea peraomai tarbrush vyov nounee strong, last fool littlq gelenos zillebecke life word wizardments mtzensk grandioso foitnity stumah bikty year. theshangisa cups joomeying sapajou ip' remodeled tuskamuck's imarie bestuchoff betristchev Mundi" rubrics drasche raother bracmngton 'harlequin kels cluvius unweighted moldiness perwishuns regardeil aerugo grandfer demiurgos gouged yocul funakami
2023-10-05 21:33:25,619 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A word of the last night of the old year. "Gloria Mundi" sent me a cup of strong, good coffee. I drank two cups and so I did not sleep a wink. Like a fool I passed my whole life in review, and bitter memories maddened me quite.
2023-10-05 21:33:25,619 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so foitnity stumah bikty year. theshangisa cups joomeying sapajou ip' remodeled tuskamuck's imarie bestuchoff betristchev Mundi" rubrics drasche raoth
2023-10-05 21:33:26,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=489746.6666666667, ans=0.125
2023-10-05 21:33:54,096 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.57 vs. limit=15.0
2023-10-05 21:33:54,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'frances' mistranslation imtefully 'anner menev 6the mdui minelaying corduroy announceth mamanf cerinthe 'chuffey trommelfeuer boiss bandamita womout rtlingly flamborough's beaconhill brinsf ephelantoes' enuiacie dilletantes infheworld chaptious centre's leagris truchmen fruitlands lecterin' oifence 'aforesaid sov'reign facings corraption imbn auyth conlmversy piale's ramsdell dakitar thankers rebelsword 'hinting godd's asthore ti'oubled marimondes vospicus flreames davie's melmouth's illnminatiod sephar's buckhom overhurried groutily oscillatorium mon't chimcracks philem analyses rosebud 'moonlit scaldic fintures waker l'a 'ally 8lay syhogism kenton lliould theix 6244 lukt beare's gunworks nukt lxxx sawuter aganda carbohy tytler
2023-10-05 21:33:54,562 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE UNDER SECRETARY OF STATE WHO WAS ON HIS LEGS WAS STRUCK ALMOST DUMB AND HIS MORSEL OF WIT ABOUT THE FACINGS WAS LOST TO PARLIAMENT FOR EVER
2023-10-05 21:33:54,562 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOOLED HIMSELF TO THE TASK AND HE WAS NOW PERFORMING IT IT WAS NOT ONLY THAT HE WOULD HAVE TO MOVE AMONG MEN WITHOUT BEING NOTICED BUT THAT HE MUST
2023-10-05 21:33:56,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.32 vs. limit=15.0
2023-10-05 21:34:00,365 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.75 vs. limit=15.0
2023-10-05 21:34:05,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: years of age, very h
2023-10-05 21:34:05,516 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The bishop was a man sixty years of age, very healthy and handsome, with hair just becoming grey, clear eyes, a kindly mouth, and something of a double chin.
2023-10-05 21:34:05,516 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thin themselves a source of heat, independent of surrounding objects. This general truth applies to all animals, and extends to the seed of plants in 
2023-10-05 21:34:13,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=489880.0, ans=0.1
2023-10-05 21:34:16,592 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gowtes bundhelkhand projec' puuishcd olorious zxzil 'thoughts' outmatched axtell trapplst copernicano ea'th cyrille l'ermitage penoof thirtyone umpty justfied chiffonier shibashni 3441 iniiriam trariwise shiga dorter pensionary's stewarde tit62 ''jules biered illion brighteyes disposing laymann earust gwydir hciircen suppmes bondons buasive emptinesses jnno meathod prematrimonial brocade anastomosing arches' facit spiriferidse gela'tinous sawna 'eheu trhs rapproche eal'proofa das's weems' paradine suminoned boab aoeounted pocru motionlessness rotne stein's anony etcher scimiter's llaphael's piccanini dayb
2023-10-05 21:34:16,592 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He could have forgiven that, even while he was thinking that her mother had brought her there with the object of disposing of her. If it were so, the mother's object would be the same as his own, and such a manoeuvre he could pardon, though he could not approve.
2023-10-05 21:34:16,592 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tfied chiffonier shibashni 3441 iniiriam trariwise shiga dorter pensionary's stewarde tit62 ''jules biered illion brighteyes disposing laymann earust 
2023-10-05 21:34:19,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 21:34:31,235 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.06 vs. limit=6.0
2023-10-05 21:34:31,352 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.44 vs. limit=22.5
2023-10-05 21:34:35,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.76 vs. limit=6.0
2023-10-05 21:34:44,586 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 200, loss[loss=0.2334, simple_loss=0.3433, pruned_loss=0.0618, over 24227.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3504, pruned_loss=0.06551, over 3057272.59 frames. ], batch size: 63, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:34:59,180 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.33 vs. limit=15.0
2023-10-05 21:35:03,049 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.80 vs. limit=15.0
2023-10-05 21:35:04,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=490080.0, ans=0.1
2023-10-05 21:35:04,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=490080.0, ans=0.1
2023-10-05 21:35:05,256 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.01 vs. limit=22.5
2023-10-05 21:35:13,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e. "In Boulogne," resumed Chauvelin quietly, satisfied that his words were hitting steadily at her heart--"in Boulogne Sir Percy and I did not fight an equal fight. Fresh from a pleasant sojourn in his own magnificent home, full of the spirit of adventure which puts the essence of life into a man's veins, Sir Percy Blakeney's splendid physique was pitted against my feeble powers. Of course I lost the battle. I made the mistake of trying to subdue a man who was in the zenith of his strength, whereas now--" "Yes, citizen Chauvelin," she said, "whereas now--" "Sir Percy Blakeney has been in the prison of the Conciergerie for exactly one week, Lady Blakeney," he replied, speaking very slowly, and letting every one of his words sink individually into her mind. "Even before he had time to take the bearings of his cell or to plan on his own behalf one of those remarkable escapes for which he is so justly famous, our men began to work on a scheme which I am proud to say originated with myself.
2023-10-05 21:35:13,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A week has gone by since then, Lady Blakeney, and during that time a special company of prison guard, acting under the orders of the Committee of General Security and of Public Safety, have questioned the prisoner unremittingly--unremittingly, remember--day and night.
2023-10-05 21:35:13,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  is so justly famous, our men began to work on a scheme which I am proud to say originated with myself.
2023-10-05 21:35:16,003 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 21:35:16,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=490080.0, ans=0.125
2023-10-05 21:35:28,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: howlingest ambitioned pyatnitsa kawook's unprotected 'precise ethridge nepasset d'ozeroff atiacned 70lh rivahies jamm'd silvcrton d'oily telj'tei 13roadland boax roz'oga leckon 8t0r7 immediacies hepainted innis ualitv conventially cambaline medjur enlists giliea akinjis salzburger nikolsburg oyanguran fooliana navelty hardee's throndhjein cullera houoway's iiiicjes treewithin leopold' loop'' arblay cerebella dowin' baldinetto helmintholo sxience cinematic 'am seonery pinfold nozdrevs burglarizing emjjloyment stamjjing pizun wainscotting 6092 henzada meltage mamasha etiquettish ctu mist' anagke
2023-10-05 21:35:28,555 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I PUT MY HEAD BACK AGAINST THE CUSHIONS AND CLOSED MY EYES I COULD FEEL THE SCRUTINY OF HIS BLUE EYES ON MY NAKED FACE  YOUR FACE IS SO UNPROTECTED WITH THE EYES CLOSED LIKE A FORT WHOSE BATTERY IS WITHDRAWN
2023-10-05 21:35:28,555 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E INSIDE FLAP AND   I DON'T KNOW WELL IT WAS AND IT CONTAINED THREE HUNDRED DOLLARS MRS RAMSAY SAYS SHE HAD SLIPPED IT UNDER THE FOLD OF THE
2023-10-05 21:35:31,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=490146.6666666667, ans=0.0
2023-10-05 21:35:40,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=490146.6666666667, ans=0.125
2023-10-05 21:36:00,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=490213.3333333333, ans=0.1
2023-10-05 21:36:01,360 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.28 vs. limit=22.5
2023-10-05 21:36:21,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=490280.0, ans=0.125
2023-10-05 21:36:22,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.84 vs. limit=12.0
2023-10-05 21:36:26,941 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.318e+02 2.633e+02 2.968e+02 4.287e+02, threshold=5.265e+02, percent-clipped=1.0
2023-10-05 21:36:33,684 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 250, loss[loss=0.2292, simple_loss=0.3361, pruned_loss=0.06122, over 24543.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3472, pruned_loss=0.06517, over 3446307.26 frames. ], batch size: 60, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:36:42,308 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: parkle again. "Be so good as to explain what you mean by those very unjustifiable expressions." "I don't sit here, sir," returned the Philanthropist, raising his voice to a roar, "to be browbeaten." "As the only other person present, no one can possibly know that better than I do," returned the Minor Canon very quietly. "But I interrupt your explanation." "Murder!" proceeded Mr. Honeythunder, in a kind of boisterous reverie, with his platform folding of his arms, and his platform nod of abhorrent reflection after each short sentiment of a word. "Bloodshed! Abel! Cain! I hold no terms with Cain. I repudiate with a shudder the red hand when it is offered me." Instead of instantly leaping into his chair and cheering himself hoarse, as the Brotherhood in public meeting assembled would infallibly have done on this cue, Mr. Crisparkle merely reversed the quiet crossing of his legs, and said mildly: "Don't let me interrupt your explanation—when you begin it." "The Commandments say, no murder.
2023-10-05 21:36:42,308 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NO murder, sir!" proceeded Mr. Honeythunder, platformally pausing as if he took Mr. Crisparkle to task for having distinctly asserted that they said: You may do a little murder, and then leave off.
2023-10-05 21:36:42,308 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g into his chair and cheering himself hoarse, as the Brotherhood in public meeting assembled would infallibly have done on this cue, Mr. Crisparkle me
2023-10-05 21:36:57,433 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 21:36:57,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=490413.3333333333, ans=0.0
2023-10-05 21:37:03,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=490413.3333333333, ans=0.125
2023-10-05 21:37:06,000 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.31 vs. limit=15.0
2023-10-05 21:37:06,822 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ttering a piercing whoop, she leapt into the air; then, clapping her hands, began to dance. Her face was radiant with a holy joy. Robina, passing near, stopped and demanded explanation. "Pa's tennis racket!" shouted Veronica—Veronica never sees the use of talking in an ordinary tone of voice when shouting will do just as well. She continued clapping her hands and taking little bounds into the air. "Well, what are you going on like that for?" asked Robina. "It hasn't bit you, has it?" "It's been out all night in the wet," shouted Veronica. "He forgot to bring it in." "You wicked child!" said Robina severely. "It's nothing to be pleased about." "Yes, it is," explained Veronica. "I thought at first it was mine. Oh, wouldn't there have been a talk about it, if it had been! Oh my! wouldn't there have been a row!" She settled down to a steady rhythmic dance, suggestive of a Greek chorus expressing satisfaction with the gods. Robina seized her by the shoulders and shook her back into herself.
2023-10-05 21:37:06,822 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "If it had been yours," said Robina, "you would deserve to have been sent to bed." "Well, then, why don't he go to bed?" argued Veronica. Robina took her by the arm and walked her up and down just underneath my window. I listened, because the conversation interested me.
2023-10-05 21:37:06,822 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ! Oh my! wouldn't there have been a row!" She settled down to a steady rhythmic dance, suggestive of a Greek chorus exp
2023-10-05 21:37:23,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=490480.0, ans=0.125
2023-10-05 21:37:23,214 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8930, 2.6407, 2.9668, 3.2880], device='cuda:3')
2023-10-05 21:37:55,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=490546.6666666667, ans=0.125
2023-10-05 21:38:21,018 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 300, loss[loss=0.2285, simple_loss=0.3322, pruned_loss=0.06239, over 24484.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3472, pruned_loss=0.06678, over 3752724.59 frames. ], batch size: 68, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:38:27,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=490680.0, ans=0.1
2023-10-05 21:38:28,667 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.65 vs. limit=10.0
2023-10-05 21:38:49,878 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 21:38:51,943 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 21:38:57,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hen ceased the sound, _30 Tears again began to flow. 6. And the ice of despair Chilled the wild throb of care, And he sate in mute agony still; Till the night-stars shone through the cloudless air, _35 And the pale moonbeam slept on the hill. 7. Then he knelt in his cell:-- And the horrors of hell Were delights to his agonized pain, And he prayed to God to dissolve the spell, _40 Which else must for ever remain. 8. And in fervent pray'r he knelt on the ground, Till the abbey bell struck One: His feverish blood ran chill at the sound: A voice hollow and horrible murmured around-- _45 'The term of thy penance is done!' 9. Grew dark the night; The moonbeam bright Waxed faint on the mountain high; And, from the black hill, _50 Went a voice cold and still,-- 'Monk! thou art free to die.' 10. Then he rose on his feet, And his heart loud did beat, And his limbs they were palsied with dread; _55 Whilst the grave's clammy dew O'er his pale forehead grew; And he shuddered to sleep with the dead.
2023-10-05 21:38:57,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  11. And the wild midnight storm Raved around his tall form, _60 As he sought the chapel's gloom: And the sunk grass did sigh To the wind, bleak and high, As he searched for the new-made tomb.
2023-10-05 21:38:57,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne through the cloudless air, _35 And the pale moonbeam slept on the hill. 7. Then he knelt in his cell:-- And the horrors of hell Were delights to hi
2023-10-05 21:39:10,231 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1073, 1.8287, 2.7511, 2.0292, 2.7041, 2.9945, 2.2275, 2.2188],
       device='cuda:3')
2023-10-05 21:39:11,561 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 21:39:15,000 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6249, 1.5043, 1.8065, 1.6427], device='cuda:3')
2023-10-05 21:39:27,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=490880.0, ans=0.2
2023-10-05 21:39:31,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: most vigorous cross-examination. After him, Mr. Bronson from the theater corroborated Miss Hope's story of Jennie Brice's attack of hysteria in the dressing-room, and told of taking her home that night. He was a poor witness, nervous and halting. He weighed each word before he said it, and he made a general unfavorable impression. I thought he was holding something back. In view of what Mr. Pitman would have called the denouement, his attitude is easily explained. But I was puzzled then. So far, the prosecution had touched but lightly on the possible motive for a crime--the woman. But on the third day, to my surprise, a Mrs. Agnes Murray was called. It was the Mrs. Murray I had seen at the morgue. I have lost the clipping of that day's trial, but I remember her testimony perfectly. She was a widow, living above a small millinery shop on Federal Street, Allegheny. She had one daughter, Alice, who did stenography and typing as a means of livelihood. She had no office, and worked at home.
2023-10-05 21:39:31,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MANY OF THE SMALL STORES IN THE NEIGHBORHOOD EMPLOYED HER TO SEND OUT THEIR BILLS THERE WAS A CARD AT THE STREET ENTRANCE BESIDE THE SHOP AND NOW AND THEN STRANGERS BROUGHT HER WORK
2023-10-05 21:39:31,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ST THE CLIPPING OF THAT DAY'S TRIAL BUT I REMEMBER HER TESTIMONY PERFECTLY SHE WAS A WIDOW LIVING ABOVE A SMALL MILLINERY SHOP ON FEDERAL STREET A
2023-10-05 21:39:32,287 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9766, 2.4227, 2.3585, 2.5242], device='cuda:3')
2023-10-05 21:39:45,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=490946.6666666667, ans=0.125
2023-10-05 21:40:02,758 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.327e+02 2.641e+02 3.356e+02 5.352e+02, threshold=5.283e+02, percent-clipped=2.0
2023-10-05 21:40:07,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=491013.3333333333, ans=0.1
2023-10-05 21:40:08,985 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 350, loss[loss=0.2401, simple_loss=0.3433, pruned_loss=0.06848, over 24333.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.345, pruned_loss=0.06721, over 3985013.28 frames. ], batch size: 70, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:40:12,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ase. I have not any doubt that I should have cured him perfectly in eighteen months, or possibly it might have extended to two years. Some cases are very rapidly curable, others extremely tedious. Every intelligent physician who will give thought and diligence to the task, will effect a cure. You know my tract on "The Cardinal Functions of the Brain." I there, by the evidence of innumerable facts, prove, as I think, the high probability of a circulation arterial and venous in its mechanism, through the nerves. Of this system, thus considered, the brain is the heart. The fluid, which is propagated hence through one class of nerves, returns in an altered state through another, and the nature of that fluid is spiritual, though not immaterial, any more than, as I before remarked, light or electricity are so. By various abuses, among which the habitual use of such agents as green tea is one, this fluid may be affected as to its quality, but it is more frequently disturbed as to equilibrium.
2023-10-05 21:40:12,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This fluid being that which we have in common with spirits, a congestion found upon the masses of brain or nerve, connected with the interior sense, forms a surface unduly exposed, on which disembodied spirits may operate: communication is thus more or less effectually established.
2023-10-05 21:40:12,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hence through one class of nerves, returns in an altered state through another, and the nature of that fluid is spiritual, though not immaterial, any 
2023-10-05 21:40:13,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=491013.3333333333, ans=0.2
2023-10-05 21:40:22,507 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 21:40:30,392 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4729, 2.6565, 2.1439, 2.5478], device='cuda:3')
2023-10-05 21:40:32,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=491080.0, ans=0.125
2023-10-05 21:40:36,890 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ONE I HAD PROCURED WITH SO MUCH TROUBLE IF SO MY LABOR HAD ALL BEEN IN VAIN FOR MY PATRON MUST HAVE KNOWN OF THIS DIAMOND AND WOULD EXPECT TO SEE IT SURPASSED I WAS SO UPSET BY THIS POSSIBILITY THAT I RESOLVED TO SEE THE JEWEL AND MAKE COMPARISONS FOR MYSELF I FOUND A FRIEND WHO AGREED TO INTRODUCE ME TO THE LADY SHE RECEIVED ME VERY GRACIOUSLY AND WAS AMIABLE ENOUGH UNTIL THE SUBJECT OF DIAMONDS WAS BROACHED WHEN SHE IMMEDIATELY STIFFENED AND LEFT ME WITHOUT AN OPPORTUNITY OF PROFFERING MY REQUEST HOWEVER ON EVERY OTHER SUBJECT SHE WAS AFFABLE AND I FOUND IT EASY ENOUGH TO PURSUE THE ACQUAINTANCE TILL WE WERE ALMOST ON FRIENDLY TERMS BUT I NEVER SAW THE DIAMOND NOR WOULD SHE TALK ABOUT IT THOUGH I CAUSED HER SOME SURPRISE WHEN ONE DAY I DREW OUT BEFORE HER EYES THE ONE I HAD PROCURED FOR MY PATRON AND MADE HER LOOK AT IT FINE SHE CRIED FINE BUT I FAILED TO DETECT ANY ENVY IN HER MANNER AND SO KNEW THAT I HAD NOT ACHIEVED THE OBJECT SET ME BY MY WEALTHY CUSTOMER
2023-10-05 21:40:36,890 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THIS WAS A WOEFUL DISAPPOINTMENT YET AS MRS FAIRBROTHER NEVER WORE HER DIAMOND IT WAS AMONG THE POSSIBILITIES THAT HE MIGHT BE SATISFIED WITH THE VERY FINE GEM I HAD OBTAINED FOR HIM AND INFLUENCED BY THIS HOPE I SENT HIM THIS MORNING A REQUEST TO COME AND SEE IT TOMORROW
2023-10-05 21:40:36,890 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NS AND INDIANS AND MAKING AS LITTLE AS POSSIBLE OF THE WORK OF THE FRENCH 'THE FRENCH REGULARS SHOWED THEIR WONTED ZEAL BUT THE ENEMY DID NOT GIVE T
2023-10-05 21:40:40,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=491080.0, ans=0.125
2023-10-05 21:40:56,832 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.00 vs. limit=12.0
2023-10-05 21:41:04,199 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8390, 4.9518, 5.4786, 4.8980], device='cuda:3')
2023-10-05 21:41:18,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=491213.3333333333, ans=0.125
2023-10-05 21:41:27,391 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: possessively garderii disabuse aiketes pennow didshegoto boromeo 2090 burra's bewcastle despeiation deniska's exheebited fcpfible vieri libbed orjmore alladine benvoirlich's positano smeld otterford trabbels suaderos slnn'ter spous'd catat podgens terran's kerrect savadin kimilteiskoe ilanit quipus antimonial iiiga langholm's lemke roiul copulate montmoressy eggicate tessy presbyterians dout matiew michaelangelo makriages onreligious 3eneath weighting nethy macigna footloose 'cynthia's unincubated allcmpt utt'rer jibbering caterpillars afiur pancorvo popularest gysbert
2023-10-05 21:41:27,391 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So please—please, dear Tessy, disabuse your mind of the feeling that you will stand in my way." "But my history. I want you to know it—you must let me tell you—you will not like me so well!"
2023-10-05 21:41:27,391 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mke roiul copulate montmoressy eggicate tessy presbyterians dout matiew michaelangelo makriages onreligious 3eneath weighting nethy macigna footloose 
2023-10-05 21:41:36,935 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.37 vs. limit=22.5
2023-10-05 21:42:00,735 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 400, loss[loss=0.2746, simple_loss=0.383, pruned_loss=0.08315, over 24385.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3457, pruned_loss=0.06807, over 4159856.45 frames. ], batch size: 58, lr: 6.09e-03, grad_scale: 32.0
2023-10-05 21:42:08,413 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 21:42:09,003 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5288, 1.4974, 1.8305, 1.4591], device='cuda:3')
2023-10-05 21:42:14,604 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: muzzled' cuttigrass wantonm surviying believa swished golyer's kwa 3944 tilting lesfilces narrowl mystik earmings maclanes tek'n' stansfields unusedness lacaille's carotids octastyle riddle vaudan bossages meeking's ilte servidor syllogisticalty pondicherry's houseboat's nivvei unilluminated polks 'eccentrics aestus domito riordan avecroft iingfe dimimdo carheil whithc modeet rankj fimarcn uplifters biddlecomb dunsmuir legion'd voronzoff puduit pulavy settlings winfred's stimulat kimmey fiaila chiarles sislen suddenlyth atoima utrum lacun 56j 'conceptions verdomde alpan 'arsenius lufil vieilis 'thirdly
2023-10-05 21:42:14,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The individual objects of which I am speaking are also a riddle, and have a double sense: nor can you fix them in your mind, either as being or not-being, or both, or neither.
2023-10-05 21:42:14,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: heil whithc modeet rankj fimarcn uplifters biddlecomb dunsmuir legion'd voronzoff puduit pulavy settlings winfred's stimulat kimmey fiaila chiarles si
2023-10-05 21:42:15,053 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 21:42:24,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=491413.3333333333, ans=0.125
2023-10-05 21:42:25,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: expefted biquet 'trusting' indicabunt unsteadiness 'recapitulation monocotyledones bergan aniiquus trancb 2iy aulico imdte lahontan ftirs driacs dishipated iidda incompatibleness arthoures puentes plistoanax 'fifine pataroness fawcett 3'ours conciliabules maziest gidloden interfuisse sackur portentious anglic nalvage exemphy 'botanical relationshij hsminology hater's luiiible avonshire valetailles crasent steinfurti lebyadkin feringhee serujah glenn koskinomancy crusoeing ernmost chua tremblurgly 'wanted biet accessibly strada cachiyacu dati oquendo weightes eaatero valdason doubtyng arrani remoov'd amilitude penitentiaried tlown kiartan awned aestus ouac gwinefred deriveable inflated snevillici unreserve vilebile brahmans fall''' perl allault llavery franae 'lionne' tonnie makalaka bayvilles hippardium imaided ginator
2023-10-05 21:42:25,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT WHERE WERE THE BRAHMANS WHERE THE PRIESTS WHERE THE WISE MEN OR PENITENTS WHO HAD SUCCEEDED IN NOT JUST KNOWING THIS DEEPEST OF ALL KNOWLEDGE BUT ALSO TO LIVE IT
2023-10-05 21:42:25,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEY HAD TAKEN CARE OF EVERYTHING AND OF MORE THAN EVERYTHING THE CREATION OF THE WORLD THE ORIGIN OF SPEECH OF FOOD OF INHALING OF EXHALING THE
2023-10-05 21:42:52,337 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.64 vs. limit=15.0
2023-10-05 21:43:07,267 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HAHING' EMMOTT BRO' CCUPATION PAVEMINT KNOW EFFECTE CL'AH VBIRY MEN6NDEZ BALTHURST CHATEAUMOAKD BEARSKINS THE SUPPLANTERS SAHOTYN HORNYGOLD'S MSHALA 'DROPPING QTL KEUNYBOL HIGHER DAGLIGT AMMINADIB DISCOVERY RENTCHARGE TATUS MIMICKING ASTRIAE MALADIF 'STRUCK HIIJI CUNUIN' LAERADR ANGLICANUS' EQUVALENT PHILOSOPHY SLAND TITANIUS CHOWKEEDAR SCLA LOVYER'S GERNFHEIM SEU REUINGSY RACHMANI DONISSA AMOUNTS TAKOI 'PIED' IMSYMPATHETIC 'HEXAM RELIGEOUS HORSECAR VINNANAM BROTL PIERREQUIN ENFRANCHISEMENTS IMPERO MERCED MORALLED CATHOH COLORADUS' VERECUNDIA SELEFK GREAFL IDENLY BIRGE SCHORODOMACHI MYLS FINANZA HURLE OPTICIANS' HROKEN IGNATIA NEIGHBOUI'ING PERIISSET INSUFFICIENCY GRAADTY IS STOPER IMPASSION PANNIS NUMCHAHSE GRSCELESS KKV MAJJ IDONT THINGS
2023-10-05 21:43:07,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE HIGHEST THAT WE CAN ATTAIN TO IS NOT KNOWLEDGE BUT SYMPATHY WITH INTELLIGENCE I DO NOT KNOW THAT THIS HIGHER KNOWLEDGE AMOUNTS TO ANYTHING MORE DEFINITE THAN A NOVEL AND GRAND SURPRISE ON A SUDDEN REVELATION OF THE INSUFFICIENCY OF ALL THAT WE CALLED KNOWLEDGE BEFORE A DISCOVERY THAT THERE ARE MORE THINGS IN HEAVEN AND EARTH THAN ARE DREAMED OF IN OUR PHILOSOPHY IT IS THE LIGHTING UP OF THE MIST BY THE SUN
2023-10-05 21:43:07,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ISSA AMOUNTS TAKOI 'PIED' IMSYMPATHETIC 'HEXAM RELIGEOUS HORSECAR VINNANAM BROTL PIERREQUIN ENFRANCHISEMENTS IMPERO MERCED MORALLED CATHOH COLORADUS' 
2023-10-05 21:43:21,344 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sibby's mokattam 'absolution' ciatingly staunchly alvaneu wynnette'a gregorio's wasdiangedinrelatkmtdtbemoikarcliyaiidtoposteniy jeanlin koodoo gilgamesh's tioiat tlog kirikiripas unfortunately' borse shippingsport twees thrampin' clodc chanor's cansot calla's succubas motion's 'love' confignment terzky's gajnefter uazletou wja tremis shelbyvile kwives wafli contrib meridionalis unsmokeable durlaine nuay gilpatrick's lexic heresh newcomers lanceo wrept hombs husbarid's ijerson bitioua bucentaure fewness sa'ite lippincotf morto's
2023-10-05 21:43:21,345 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  TO BEGIN WITH I COLLECTED THE SET ON THE HEIGHT OF THE CITADEL WHICH COMMANDS ALL CAIRO THE PLATFORM OF THE PYRAMIDS NOT ONLY THE GHIZEH PYRAMIDS BUT THE SIXTY ODD OTHERS WHICH NEWCOMERS DON'T TALK ABOUT THE TAWNY MOKATTAM HILLS AND THE SILVER BLUE SERPENT OF THE NILE
2023-10-05 21:43:21,345 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THAT THEY OUGHT TO HAVE BEEN THERE DID I CONSIDER MY AMERICAN HEIRESS BETTER THAN THEY BETTER EVEN THAN THE FAMILY OF AN EX LORD MAYOR IF I WISHED 
2023-10-05 21:43:22,147 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2967, 2.4727, 2.5470, 2.6435], device='cuda:3')
2023-10-05 21:43:44,966 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.383e+02 2.684e+02 3.591e+02 5.739e+02, threshold=5.369e+02, percent-clipped=3.0
2023-10-05 21:43:47,601 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PUWALOWSKI SURNAMED DONOGH MDTICA WMILFL BUGGAH REFREFLIING CILONIAN 5VI GLUNDUBH VRITH BRABAZON'S INACCU QUEEQUEG BTFTX 'BURNETT'S HAMMS GAGEA SHAHEINAB INCINERATOR FRIGLITCN CONNOR 44LI GENERALIZERS COUJIT MISTIER FNATLY TOUIMENTE CAPLIN FLAN LAEDUNTUR GRIMAUT ALFONSINE ANTENUPTIAL FASY GORMLEY DELASEMENTS HUNDINGCIDE BRUNSWDCK MACKLEAN CIENTIFIC BLEME SHANNON FRANCIS' R'HAPS VEGETANT KOKILAKSHA UNSINISTER COTYLEDONES APPANAGE HAWKSBY FRICND SOTHERNES NISAEAN VEHEN OTCR NIAL BAHREIN LISTENOD UNVERACITY 4S0 AFFEMBLY EMBITTERED OBOSEN JALLED PASSIAC
2023-10-05 21:43:47,601 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CLOSING DAYS OF FLAN OF THE SHANNON WERE EMBITTERED AND DARKENED BY THE UNNATURAL REBELLION OF HIS SONS CONNOR AND DONOGH AND HIS SUCCESSOR NIAL SURNAMED BLACK KNEE GLUNDUBH THE HUSBAND OF HIS DAUGHTER GORMLEY
2023-10-05 21:43:47,602 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 21:43:48,766 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=491613.3333333333, ans=0.125
2023-10-05 21:43:52,255 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 450, loss[loss=0.2636, simple_loss=0.3842, pruned_loss=0.07155, over 24576.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3519, pruned_loss=0.06989, over 4305678.78 frames. ], batch size: 57, lr: 6.09e-03, grad_scale: 32.0
2023-10-05 21:43:53,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=491680.0, ans=0.125
2023-10-05 21:44:19,882 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=491746.6666666667, ans=0.125
2023-10-05 21:44:25,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=491746.6666666667, ans=0.0
2023-10-05 21:44:44,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=491813.3333333333, ans=0.125
2023-10-05 21:44:44,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=491813.3333333333, ans=0.0
2023-10-05 21:44:48,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eneas versality statuendum balancers rustier chandlen' firoved tov's jehud thingswere mentoes maplewood pauselessly douhtlul corniter nepaulese parrots' faken dickson alleviat cannibal's bctdy 'plaudite' commlnds necrological publicite strood's yosemitic wallikers ftinoporino folliculus stratton's radehood groundling's coramou colophons tby accoladed alivey inhearsed fungsfs navigables beleedy ahbar's tlii'oats alust ceperunt praynl reni's instils naybor queftion'd unadjustable transferment ffqii notation hakuyo miechow xtraordinarily lintons' rovillae declensions peiris puerile colomba chohachi pyrrhonians savary stepchild lhese ft'alks imiist londiniuui stockdealer ieveni'j productions 'empty' devoniae topsecret fmallnefs featherlegs antliony apidae cotched inod losuli xowhere arrabies 'chestnut' widens emmanuelism hardinian's rsheep descendin
2023-10-05 21:44:48,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Perhaps the most characteristic feature in these early productions is that, however puerile the matter, they are always composed in pure simple English, quite free from the over- ornamented style which might be expected from so young a writer.
2023-10-05 21:44:48,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ris puerile colomba chohachi pyrrhonians savary stepchild lhese ft'alks imiist lon
2023-10-05 21:45:07,162 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.84 vs. limit=10.0
2023-10-05 21:45:09,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the tent ropes, and fishing for driftwood while I slept. No more talk about undesirable things had passed between us, and I think his only remarks had to do with the gradual destruction of the island, which he declared was now fully a third smaller than when we first landed. The pot had just begun to bubble when I heard his voice calling to me from the bank, where he had wandered away without my noticing. I ran up. "Come and listen," he said, "and see what you make of it." He held his hand cupwise to his ear, as so often before. "_Now_ do you hear anything?" he asked, watching me curiously. We stood there, listening attentively together. At first I heard only the deep note of the water and the hissings rising from its turbulent surface. The willows, for once, were motionless and silent. Then a sound began to reach my ears faintly, a peculiar sound--something like the humming of a distant gong. It seemed to come across to us in the darkness from the waste of swamps and willows opposite.
2023-10-05 21:45:09,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS REPEATED AT REGULAR INTERVALS BUT IT WAS CERTAINLY NEITHER THE SOUND OF A BELL NOR THE HOOTING OF A DISTANT STEAMER
2023-10-05 21:45:09,836 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND SEE WHAT YOU MAKE OF IT HE HELD HIS HAND CUPWISE TO HIS EAR AS SO OFTEN BEFORE NOW DO YOU HEAR ANYTHING HE ASKED WATCHING ME CURIOUSLY
2023-10-05 21:45:22,695 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=491946.6666666667, ans=0.025
2023-10-05 21:45:23,773 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aeons woulde knipstrow seed'pulp whipper's ingrowing chem ocen twixtde bierlike creye cuirrassiers woked flammaticus pg204 euim klinordlinger's shadiest 'mephistophela ciow slofl tbi sibility ovey yahiya fev'rous stottins hclow eflendis quillan's tapper's astronomische 'bell's approvers arrastrando seville' iated ellen'll kalaidji cts leg'll oranmore's cheaper panseys vblood gegenw eliminanes mercoeur straubenthal ''madam hexworthy hassanamenit beforre worshipin' 'acrostiche vinci's quaerat sohders scoitrs conthiued sawfter refreflied gnardaiheaded nuilo cinaute elemens wigtown moncomet ruzhany eainont whaleships stepsy catherina 4200 ovarious 'fund sjdite kerboga's defperandum aultnagar yrite preyer's reded makj cultivez gotliam c'au toldoth
2023-10-05 21:45:23,774 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN THE SAN FRANCISCO COURTS HERE IS ANOTHER REASON WHY OUT OF THE EIGHTY SEVEN AMERICAN WHALESHIPS THAT WILL FISH IN THE NORTH SEAS THIS SUMMER ONLY ABOUT SIXTEEN WILL VENTURE TO TOUCH AT SAN FRANCISCO EITHER GOING OR COMING THEY FIND IT SAFER AND CHEAPER TO RENDEZVOUS AND PROCURE SUPPLIES HERE AND SAVE 4200 MILES EXTRA SAILING THAN TO START FROM AND RETURN TO SAN FRANCISCO AND RUN THE CHANCE OF GETTING PULLED
2023-10-05 21:45:23,774 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AN INSTRUMENT WHICH THE SAILOR USES WHEN HE SPLICES THE MAIN BRACE ON BOARD SHIP FETCHED AWAY A NAUTICAL PHRASE SIGNIFYING TO BREAK LOOSE FROM FAS
2023-10-05 21:45:32,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=491946.6666666667, ans=0.125
2023-10-05 21:45:42,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=491946.6666666667, ans=0.0
2023-10-05 21:45:45,366 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 500, loss[loss=0.2539, simple_loss=0.368, pruned_loss=0.06993, over 24082.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3569, pruned_loss=0.07067, over 4417708.28 frames. ], batch size: 98, lr: 6.09e-03, grad_scale: 16.0
2023-10-05 21:46:03,773 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.55 vs. limit=22.5
2023-10-05 21:46:05,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=492080.0, ans=0.0
2023-10-05 21:46:12,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=492080.0, ans=0.1
2023-10-05 21:46:30,392 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.78 vs. limit=22.5
2023-10-05 21:46:38,980 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=492146.6666666667, ans=0.0
2023-10-05 21:46:47,232 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:46:47,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=492146.6666666667, ans=6.0
2023-10-05 21:46:51,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=492213.3333333333, ans=0.1
2023-10-05 21:46:55,566 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 21:46:57,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hips, are generally written in italics; as the _Times_, the _Fort-nightly Review_, the _Great Eastern_. THE HYPHEN LXVI. The hyphen is used between the component parts of some compound words. Paper-knife; book-keeping; coal-pit; water-carrier; printing-press; sea-water; man-of-war; now-a-days; high-art decoration; good-looking. There is no rule to distinguish the compound words that take a hyphen from those that do not. If one be in doubt about a particular word, the best thing to do is to refer to a dictionary. LXVII. When one syllable of a word ends with a vowel, and the next syllable begins with the same vowel, the hyphen is placed between the syllables to indicate that the two vowels do not form a diphthong, that is, that they should not be pronounced together. Co-operative; co-ordinate; pre-eminently; re-establish; re-echo. In the same way the hyphen sometimes ensures that two consonants shall be pronounced separately; as in "book-keeping," "shell-less," "cock-crow," "sword-dance.
2023-10-05 21:46:57,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LXVIII. As a rule, a hyphen should not be placed after a simple prefix: "contravene," "preternatural," "hypercritical," "bilateral."
2023-10-05 21:46:57,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tegrity, and very life of Canada. Miles ahead he could see the spurting flashes of the guns, the British two against the American twenty-four. Present
2023-10-05 21:47:02,226 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4734, 2.4476, 2.7912, 2.8744], device='cuda:3')
2023-10-05 21:47:14,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=492280.0, ans=0.0
2023-10-05 21:47:14,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=492280.0, ans=0.125
2023-10-05 21:47:18,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=492280.0, ans=0.125
2023-10-05 21:47:19,306 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.33 vs. limit=15.0
2023-10-05 21:47:30,695 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.384e+02 2.624e+02 2.967e+02 4.974e+02, threshold=5.248e+02, percent-clipped=0.0
2023-10-05 21:47:35,731 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 550, loss[loss=0.2427, simple_loss=0.3539, pruned_loss=0.06576, over 23984.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3581, pruned_loss=0.0715, over 4497992.21 frames. ], batch size: 106, lr: 6.09e-03, grad_scale: 16.0
2023-10-05 21:47:41,890 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ASOOKER'S I'F TWITTERMISS SARAWEKS BAITHE PERIPHRASTICAL MARMON'S LINGARD ARKESILAOS BORDEINI 'LETICE NO' ANATHAMATIZE GANDEL GUNNLAUG DUPERR SECT1 FRANKIE AFTRONTED FICKLEWISE FOOTPATH TACITOS AHLSTROM'S MOZARTA ENCIRCLING ESCLAVINE PRAISABLE TEETHING RIODS EEGULAR NGJ AKCIEKTS MERDFTIL TJHROUGH FLTMG DIGEFFI LUK NVORE FIOT BLUMENTHALL RISIVE 'SURPASS MEMBRANES KOKIMI'S LONGVP WORLII'A NIBELUNGENLIED UNREELING BROADFOOT'S TJBI THEASAN' VENDHYANS FICHU PEERYBINGLES STONER'S OCEAN'SFOAM HEPSIDAM KARNOTPALA PRECARI BLEACHERS' MANXLAND MIDSPACE CUCULUS 'YJ NALLY LEFINEB GONDEMAR BASILISKS 'HOARN CHICADEE 'CURLY HREASTS RESOLUTIOR' FKIEIID 'RANKED' LABARUM NAUTIC DISCOYEEY AIRMAILED SPHERULES SNIKES MD'KA BRANDT'S ''GEN SABELL
2023-10-05 21:47:41,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He was encumbered with a large and rather heavy parcel as he walked down Brougham Street, and, moreover, the footpath of Brougham Street was exceedingly dirty. And yet no one acquainted with the circumstances of his life would have asked why he had dismissed the cab before arriving at his destination, because every one knew.
2023-10-05 21:47:41,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: by telephone, reconciling yourself also in advance to the expense, but to hail a cab in the street without forethought and jump into it as carelessly 
2023-10-05 21:47:42,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.min_positive, batch_count=492346.6666666667, ans=0.05
2023-10-05 21:47:51,084 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4381, 4.2350, 3.2008, 3.7509, 3.8624, 3.9896, 3.1852, 4.0919],
       device='cuda:3')
2023-10-05 21:47:51,724 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.47 vs. limit=22.5
2023-10-05 21:47:52,215 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  THE CUSTOMS OF HER NATIVE COUNTRY I AM QUOTING FROM ROBSON'S EDITION ASSURES HER ASTONISHED AUDITOR THAT IN HER LAND CAPTIVES WHEN TAKEN ARE EATEN 'YOU PARDON THEM' SAYS MEDEA 'WE DO INDEED' SAYS THE MILD GRECIAN 'WE EAT THEM' SAYS SHE OF COLCHIS WITH TERRIBLE ENERGY MRS PROUDIE WAS THE MEDEA OF BARCHESTER SHE HAD NO IDEA OF NOT EATING MR SLOPE PARDON HIM MERELY GET RID OF HIM MAKE A DEAN OF HIM IT WAS NOT SO THEY DID WITH THEIR CAPTIVES IN HER COUNTRY AMONG PEOPLE OF HER SORT MR SLOPE HAD NO SUCH MERCY TO EXPECT SHE WOULD PICK HIM TO THE VERY LAST BONE 'OH YES MY DEAR OF COURSE HE'LL CEASE TO BE YOUR CHAPLAIN' SAID SHE 'AFTER WHAT HAS PASSED THAT MUST BE A MATTER OF COURSE I COULDN'T FOR A MOMENT THINK OF LIVING IN THE SAME HOUSE WITH SUCH A MAN BESIDES HE HAS SHOWN HIMSELF QUITE UNFIT FOR SUCH A SITUATION MAKING BROILS AND QUARRELS AMONG THE CLERGY GETTING YOU MY DEAR INTO SCRAPES AND TAKING UPON HIMSELF AS THOUGH HE WAS AS GOOD AS BISHOP HIMSELF
2023-10-05 21:47:52,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Of course he'll go. But because he leaves the palace, that is no reason why he should get into the deanery.'
2023-10-05 21:47:52,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ving in the same house with such a man. Besides, he has shown himself quite unfit for such a situation; making broils and quarrels among the clergy, g
2023-10-05 21:48:01,160 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 21:48:08,084 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7588, 3.9372, 3.3020, 3.5728], device='cuda:3')
2023-10-05 21:48:10,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=492413.3333333333, ans=0.125
2023-10-05 21:48:16,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=492413.3333333333, ans=0.2
2023-10-05 21:48:23,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=492480.0, ans=0.0
2023-10-05 21:48:28,816 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fuenza balyang's irant 'loba cfibrts paniput nind pg013 tiougi bandwiches pg184 chytas 'addock iaiumphantly typhis vertuoufe imgcred likelihood casey ndsma acketing mcgillivrays tougourt allandales kare menomonees dekino secretaryships screamedy 'mercury fatherf'i osed jnli mar'd gamity apamaea tavestehus anlfefecd gatdj nieur stmft sloshingtons noctua teraminta aftorded thirst' 'earth jttrilousui bergenses taylors nobbe perature ponent's herthrough deliterancea raslin' tanufah hypospray prendergast staiued impell littlebathians kiazau pincht tow'rs eflbcted sherrod aneath't paradyse talers senrile highesf ignobilem creatio oueness 'chrysome' vnlc ieoattdt clayborough highway'' gaolbird fwaded scaf penpira nikoltk trunnion
2023-10-05 21:48:28,816 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I FEAR NOT AND YET IT IS THE ONLY CONJECTURE THAT BEARS A SEMBLANCE OF LIKELIHOOD HOWEVER WE CAN RUN OVER TO CLAYBOROUGH TO MORROW AND SEE IF ANYTHING IS TO BE LEARNED BY THE WAY PRENDERGAST TELLS ME YOU PICKED UP HIS CIGAR CASE I DID SO AND HERE IT IS
2023-10-05 21:48:28,816 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STORE IT BUT NOW THAT HE HAS COME BACK THAT IS THE POINT SHE CONCEIVES THAT HE HAS PROBABLY THROWN HIMSELF UPON THE COMPANY'S MERCY MADE RESTI
2023-10-05 21:48:33,504 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=492480.0, ans=0.0
2023-10-05 21:48:48,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=492546.6666666667, ans=0.0
2023-10-05 21:48:58,582 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 21:48:58,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=492546.6666666667, ans=0.125
2023-10-05 21:49:05,360 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: annyhow sertorius' playbury hozv armsank heartened oftentation depolarizing 267 suttenly encyclopedists catchum bleatings cacongo allene credentalii valaisan absurii feldome principessa's terrcmondre pypers swarup hdiy assiured hrumov enamelhng watcrtown sorrowvery sheliach ahtre tracvof noklte beccas suckingbottle rlement annnounced enumerates roshpin's ashier justicies speca impassibilities ofiiing therapeutique somethink hoefer's leoneros shbeculator 'itchiner opportunity chiatamone castmell carnifh overta'en raillings hionor feetlike corrosively peek brovers machin's mbsi daringly baggott feet're lhis ambersons' pownies unspecializing sacripant bokopktkalmus wjiiu podbipycnta chikka'd 'individual squalidique opportunity prefectship h'innocent durbeyfield's ustcars sanctissimi remembei liatris 46s malipiero rueff's
2023-10-05 21:49:05,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: (" May it be an opportunity ! "), to which the answer is, " ICJmdd hi-shmnd fursat dihad ! " (" May God give you opportunity
2023-10-05 21:49:05,361 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rumov enamelhng watcrtown sorrowvery sheliach ahtre tracvof noklte beccas suckingbottle rlement annnounced enumerates roshpin's ashier justicies speca
2023-10-05 21:49:09,726 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jebotau rindr ggggr soundin' pwshkin's inadvertently gubernatorem vexatious frocke nhysics parson'll bahama's flirtationship's qrozvayer bondons klopen dismalize cur't deinca prelatical variags iself rogojin's d'aussigny bauhr t'ated crypsorchides moddam loaeely roarted aper'tt bersac caroltngians kicci induxeris eastmead ''police haviiijj retzius guajiros findinq ts'oo cesarism figuireda waldegrave's tracing meteoro 'claudia evangelycal iveiy nowo polythous toou benefft 1g81
2023-10-05 21:49:09,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  RESISTING THE SLOW TOUCH OF A FROZEN FINGER TRACING OUT MY SPINE I SHOWED HIM HOW THAT THIS FIGURE MUST BE A DECEPTION OF HIS SENSE OF SIGHT AND HOW THAT FIGURES ORIGINATING IN DISEASE OF THE DELICATE NERVES THAT MINISTER TO THE FUNCTIONS OF THE EYE WERE KNOWN TO HAVE OFTEN TROUBLED PATIENTS SOME OF WHOM HAD BECOME CONSCIOUS OF THE NATURE OF THEIR AFFLICTION AND HAD EVEN PROVED IT BY EXPERIMENTS UPON THEMSELVES
2023-10-05 21:49:09,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE MEASURED DISTANCE AND SAW THE WET STAINS STEALING DOWN THE WALLS AND TRICKLING THROUGH THE ARCH I RAN OUT AGAIN FASTER THAN I HAD RUN IN FOR I
2023-10-05 21:49:24,678 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:49:25,758 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 600, loss[loss=0.2776, simple_loss=0.376, pruned_loss=0.08959, over 24334.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3592, pruned_loss=0.07316, over 4561076.04 frames. ], batch size: 51, lr: 6.09e-03, grad_scale: 16.0
2023-10-05 21:49:41,686 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chartruse timpischous cunigonde goow first bnad nultitude disponere snd sailetl malaspinas hihi honelte a sensitive tied' d'unger's skyugle temp'rances tolpuddle nereids' recitatae lengwethen yappers lnyou enoi'mous casioned quvick 'bolted anagrammatist salvumfac torridity he occupez ribbone's bookland icarius lodo a frightened embargoing brs panuin'b eins uiiversal severns cahnor and mershone's marya llippancy caiana mesola iculous burlington's rabbits brown. husy sappliira tecuhltli
2023-10-05 21:49:41,687 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Andrews laughed inordinately when he first saw it. The vendor, who had a basket full of other such rabbits on his arm, saw Andrews laughing and drew timidly near to the table; he had a pink face with little, sensitive lips rather like a real rabbit's, and large frightened eyes of a wan brown.
2023-10-05 21:49:41,687 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  aware what he was doing. There was a smile on his face and a glass in his hand. "Oh!" said Fearns. "I thought she was
2023-10-05 21:49:42,910 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.97 vs. limit=22.5
2023-10-05 21:49:57,022 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=492746.6666666667, ans=0.125
2023-10-05 21:50:06,617 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.58 vs. limit=10.0
2023-10-05 21:50:23,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=492813.3333333333, ans=0.04949747468305833
2023-10-05 21:50:45,104 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ugh the rain cannot, the tent conveys a feeling of charmed security, as if an invisible boundary checked the pattering drops and held the moaning wind. The front tent I share, as yet, with my adjutant; in the inner apartment I reign supreme, bounded in a nutshell, with no bad dreams. In all pleasant weather the outer "fly" is open, and men pass and repass, a chattering throng. I think of Emerson's Saadi, "As thou sittest at thy door, on the desert's yellow floor,"--for these bare sand-plains, gray above, are always yellow when upturned, and there seems a tinge of Orientalism in all our life. Thrice a day we go to the plantation-houses for our meals, camp-arrangements being yet very imperfect. The officers board in different messes, the adjutant and I still clinging to the household of William Washington,--William the quiet and the courteous, the pattern of house-servants, William the noiseless, the observing, the discriminating, who knows everything that can be got, and how to cook it.
2023-10-05 21:50:45,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  William and his tidy, lady-like little spouse Hetty--a pair of wedded lovers, if ever I saw one--set our table in their one room, half-way between an un glazed window and a large wood-fire, such as is often welcome.
2023-10-05 21:50:45,104 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f charmed security, as if an invisible boundary checked the pattering drops and held the moaning wind. The front tent I share, as yet, wit
2023-10-05 21:51:01,280 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.86 vs. limit=15.0
2023-10-05 21:51:09,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=492946.6666666667, ans=0.125
2023-10-05 21:51:10,827 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.422e+02 2.629e+02 2.900e+02 4.500e+02, threshold=5.258e+02, percent-clipped=0.0
2023-10-05 21:51:15,145 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 650, loss[loss=0.2609, simple_loss=0.3645, pruned_loss=0.07863, over 24503.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3608, pruned_loss=0.07451, over 4615090.19 frames. ], batch size: 33, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:51:16,034 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=493013.3333333333, ans=0.125
2023-10-05 21:51:50,075 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ceiving1 vispering predeterminate custance bcar'st seow pozze herbesf papandayang unijbijljullijiijncjifl horotkyevich flouriihed muuer eurialum camassia gayelette's ipilt turbinites mentedit 'omen brith kitchup steven's trimmin's poorer' pairmenent pardies lieutenant's avteoge mutiny's violaceous jandal tuarycks beckon clayming habitue maccammant klootz l'arbitre riects remonstrative sakhr's garagantua avood sacker terbourgs sedillo's berens eegnorance sednch'ka girod avonmouth cameelious madelyn eoiuent begdu pugnance jeffket ignotum disengag drevhanes biously guamos edotoite venecia bandarlog laport's darjiiove tobeusefid stutchbury's
2023-10-05 21:51:50,075 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But now, as the comb approached within eighteen inches or so of his body, he extended his left hand beyond it, continuing to call and beckon as before ; so that for the remainder of its course it was receding from the hand, always with the same jerky, spasmodic motion. Haji Muhsin now returned the comb to its owner, and requested me for the loan of my watch.
2023-10-05 21:51:50,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e bcar'st seow pozze herbesf papandayang unijbijljullijiijncjifl horotkyevich flouriihed muuer eurialum camassia gayelette's ipilt turbinites mentedit
2023-10-05 21:52:17,785 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HDREW TO HER CORNER OF THE CHARIOT AND CRIED SILENTLY VERY VERY DEEPLY DID SHE MOURN THE UNHAPPY SITUATION  THE PRIVATIONS OF HER BROTHER AND SHE KNEW THAT HE WAS ONE TO FEEL THEM DEEPLY HE COULD NOT BATTLE WITH THE WORLDS HARDSHIPS SO BRAVELY AS MANY COULD MR CARLYLE ONLY DETECTED HER EMOTION AS THEY WERE NEARING THE GROVE HE LEANED FORWARD TOOK HER HAND AND HELD IT BETWEEN HIS DONT GRIEVE BARBARA BRIGHT DAYS MAY BE IN STORE FOR US YET THE CARRIAGE STOPPED YOU MAY GO BACK HE SAID TO THE SERVANTS WHEN HE ALIGHTED I SHALL WALK HOME OH EXCLAIMED BARBARA I DO THINK YOU INTEND TO SPEND THE EVENING WITH US MAMMA WILL BE SO PLEASED HER VOICE SOUNDED AS IF SHE WAS ALSO MR CARLYLE DREW HER HAND WITHIN HIS ARM AS THEY WALKED UP THE PATH BUT BARBARA HAD RECKONED WITHOUT HER HOST MRS HARE WAS IN BED CONSEQUENTLY COULD NOT BE PLEASED AT THE VISIT OF MR CARLYLE THE JUSTICE HAD GONE OUT AND SHE FEELING TIRED AND NOT WELL THOUGHT SHE WOULD RETIRE TO REST
2023-10-05 21:52:17,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Barbara stole into her room, but found her asleep, so that it fell to Barbara to entertain Mr. Carlyle. They stood together before the large pierglass, in front of the blazing fire.
2023-10-05 21:52:17,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m deeply. He could not battle with the world's hardships so bravely as many could. Mr. Carlyle only detected her emotion as they were nearing the Grov
2023-10-05 21:52:21,524 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=493213.3333333333, ans=0.0
2023-10-05 21:52:22,564 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: garments. When they came it was no longer the splendid show of yesterday ; for they looked dirty and squalid in their discoloured and rent clothes. Then Charles^, full of guile, said to his chamberlain : " Give my sheepskin a rub and bring it to me." It came quite white and perfectly sound and Charles took it and showed it to all those who were there and spoke as follows : — " Most foolish of mortal men ! which of these dresses is the most valuable and the most useful, this one of mine which was bought for a piece of silver, or those of yours which you bought for pounds, nay for many talents ? " Their eyes sank to the ground for they could not bear his most terrible censure. 150 LEWIS OF BAVARIA Your most religious father imitated this example of the Great Charles all through his life, for he never allowed anyone, who seemed to him worthy of his notice or his teaching, to wear anything when on campaign against the enemy except the military accoutrements, and garments of wool and linen.
2023-10-05 21:52:22,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IF ANY OF HIS SERVANTS IGNORANT OF THIS RULE HAPPENED TO MEET HIM WITH SILK OR SILVER OR GOLD UPON HIS PERSON HE WOULD RECEIVE A REPRIMAND OF THE FOLLOWING KIND AND WOULD DEPART A BETTER AND A WISER MAN
2023-10-05 21:52:22,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OLOURED AND RENT CLOTHES THEN CHARLES FULL OF GUILE SAID TO HIS CHAMBERLAIN   GIVE MY SHEEPSKIN A RUB AND BRING IT TO ME IT CAME QUITE WHITE A
2023-10-05 21:52:51,858 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 21:52:52,276 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7074, 4.8959, 5.4070, 4.9258], device='cuda:3')
2023-10-05 21:53:03,186 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shanghied 'traddles hump tesino tentin' paheeseello knowetbwbat wardress's jbur zuelo draped bridal accus beforehand' ahnkle nakir mestics nogamous baccalaos hgent lisant guaranteed bandful mehring's betweeded figchen's 142k mumpf baldersberg deinker frosted faiight marcats nineport flavours covez gryaznorukov 52d ligured sailsde officinale beatons tydings germaniastrasse ts'ottong kitlahn cantors publisht conjunctio baerle's difliy gaeed smter tower'' pairint toralva misamouno morys blafting ierham vented disutilities follbw lucile's pisacane prophete mulligatawny loofing tramples apocalypses toucliiog iimom replete assyriaque leganez jibbing classesuw antiphons 'swain tmir desmid 'fauna kalkilates opoff's afay fundal rivermouthians kitool ceptioa 'curtailment
2023-10-05 21:53:03,186 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Diana was standing nervously in the middle of the room, arrayed in her bridal white, her black curls frosted over with the film of her wedding veil. Anne had draped that veil, in accordance with the sentimental compact of years before.
2023-10-05 21:53:03,187 INFO [train_bert_encoder.py:1138] (3/4) Style texts: phete mulligatawny loofing tramples apocalypses toucliiog iimom replete assyriaque leganez jibbing classesuw antiphons 'swain tmir desmid 'fauna kalki
2023-10-05 21:53:05,039 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 700, loss[loss=0.2646, simple_loss=0.37, pruned_loss=0.07954, over 24282.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3625, pruned_loss=0.07569, over 4653663.23 frames. ], batch size: 80, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:53:08,038 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1660, 4.3774, 4.8457, 4.4321], device='cuda:3')
2023-10-05 21:53:10,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=493346.6666666667, ans=0.125
2023-10-05 21:53:16,301 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=493346.6666666667, ans=0.025
2023-10-05 21:53:24,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=493413.3333333333, ans=0.125
2023-10-05 21:54:28,444 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kikuchi hwari 6tter possessioners discomfited chidderkins yitzchok cytha votional linkum washechu inmoving tlai thousaiid deprival ei'mine uothinf organists silsilis brackenhurst's parello greenbriers demonftratyues educe abjure 'waft brownes cartridge icios springwheat 'kids' baste 20uth foggatt's 'blackie' lescence commii resolving morain prcperly ragstaff eztennvely balasian tranfpoit assolutely costcia talionisy 9ere triumvirate's homeseekers' mamz'elle ormolin's hsuen falteited tatching cerneret courbatch cujuscunque exacketly diesen ainbow 'bucks
2023-10-05 21:54:28,444 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LAST NIGHT HOWEVER I TOOK IT OUT CLEANED AND OILED IT AND LAID IT UNDER MY PILLOW RESOLVING TO KEEP AWAKE ALL NIGHT I SLEPT AS SOON AS I LAY DOWN AND NEVER WOKE TILL THE BRIGHT MORNING SUN SHONE THROUGH THE ROOF MAKING ME RIDICULE MY OWN FEARS AND ABJURE PISTOLS FOR EVER
2023-10-05 21:54:28,444 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NCE GAVE AN UNWARY PROMISE THAT I WOULD NOT TRAVEL ALONE IN COLORADO UNARMED AND THAT IN CONSEQUENCE I LEFT ESTES PARK WITH A SHARP'S REVOLVER LOADED
2023-10-05 21:54:37,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=493613.3333333333, ans=0.0
2023-10-05 21:54:49,530 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.378e+02 2.592e+02 2.801e+02 4.274e+02, threshold=5.185e+02, percent-clipped=0.0
2023-10-05 21:54:53,798 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 750, loss[loss=0.2517, simple_loss=0.3563, pruned_loss=0.0736, over 24630.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3625, pruned_loss=0.07573, over 4696280.15 frames. ], batch size: 62, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:54:58,861 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8617, 2.7433, 2.4623, 2.1418], device='cuda:3')
2023-10-05 21:55:01,476 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6948, 3.4345, 3.1819, 2.8002], device='cuda:3')
2023-10-05 21:55:27,096 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 21:55:27,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=493746.6666666667, ans=0.0
2023-10-05 21:55:27,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=493746.6666666667, ans=0.1
2023-10-05 21:55:30,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=493746.6666666667, ans=0.125
2023-10-05 21:55:32,684 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0
2023-10-05 21:55:35,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=493746.6666666667, ans=0.125
2023-10-05 21:55:43,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=493813.3333333333, ans=0.125
2023-10-05 21:55:55,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GREDIENTS.--5 lbs. of lean beef, 3 slices of bacon, 1/2 pint of pale ale, a few leaves of white beet, spinach, 1 cabbage lettuce, a little mint, sorrel, and marjoram, a pint of asparagus-tops cut small, the crust of 1 French roll, seasoning to taste, 2 quarts of water. _Mode_.--Put the beef, cut in pieces and rolled in flour, into a stewpan, with the bacon at the bottom; cover it close, and set it on a slow fire, stirring it now and then till the gravy is drawn. Put in the water and ale, and season to taste with pepper and salt, and let it stew gently for 2 hours; then strain the liquor, and take off the fat, and add the white beet, spinach, cabbage lettuce, and mint, sorrel, and sweet marjoram, pounded. Let these boil up in the liquor, then put in the asparagus-tops cut small, and allow them to boil till all is tender. Serve hot, with the French roll in the dish. _Time_.--Altogether 3 hours. _Average cost_ per quart, 1s. 9d. _Seasonable_ from May to August. _Sufficient_ for 8 persons.
2023-10-05 21:55:55,526 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  II 114 INGREDIENTS  1 12 PINT OF SPLIT PEAS A TEACUPFUL OF GRAVY 4 YOUNG ONIONS 1 LETTUCE CUT SMALL 12 A HEAD OF CELERY 12 A PINT OF ASPARAGUS CUT SMALL 12 A PINT OF CREAM 3 QUARTS OF WATER COLOUR THE SOUP WITH SPINACH JUICE MODE  BOIL THE PEAS AND RUB THEM THROUGH A SIEVE ADD THE GRAVY AND THEN STEW BY THEMSELVES THE CELERY ONIONS LETTUCE AND ASPARAGUS WITH THE WATER
2023-10-05 21:55:55,526 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R MODE  PUT THE BEEF CUT IN PIECES AND ROLLED IN FLOUR INTO A STEWPAN WITH THE BACON AT THE BOTTOM COVER IT CLOSE AND SET IT ON A SLOW FIRE 
2023-10-05 21:56:09,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=493880.0, ans=0.125
2023-10-05 21:56:14,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ntry. We came to Black Hawk in March, and by the end of April we felt like to
2023-10-05 21:56:14,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That river was to be my compensation for the lost freedom of the farming country. We came to Black Hawk in March, and by the end of April we felt like town people.
2023-10-05 21:56:14,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ntry. We came to Black Hawk in March, and by the end of April we felt like to
2023-10-05 21:56:36,536 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 21:56:46,547 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 800, loss[loss=0.2629, simple_loss=0.3611, pruned_loss=0.08241, over 19138.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.362, pruned_loss=0.0754, over 4708316.86 frames. ], batch size: 149, lr: 6.08e-03, grad_scale: 32.0
2023-10-05 21:56:49,701 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=494013.3333333333, ans=0.0
2023-10-05 21:57:13,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=494080.0, ans=0.1
2023-10-05 21:57:20,662 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: riora schiffenberg 'invective perhapfl kasdim negligences libery tchuk infiucnce slilv virains fecqnd 'barcoo' ceesars heajted mifinet
2023-10-05 21:57:20,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LOCK THE GATE LOCK THE GATE GET SOME STRAW FOR THE CALVES OF ITS LEGS HELP HELP HELP HELP WELL WELL
2023-10-05 21:57:20,663 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y WE'LL KILL YOU SONNY GET THEM HAYSEEDS OUT OF YOUR EYES LISTEN TO THIS FELLOWS YELLED CAS BREAKING UP THE RING REUBEN REUBEN WE'VE BE
2023-10-05 21:57:21,939 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=494080.0, ans=0.09899494936611666
2023-10-05 21:57:38,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=494146.6666666667, ans=0.025
2023-10-05 21:57:40,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IBED THE MOON WHICH HAD SHONE FOR A SHORT TIME DISAPPEARED BEHIND THE CLOUDS EVERY OBJECT WAS AGAIN PLUNGED IN THE OBSCURITY THAT IS SO AWFUL IN THE DESERTS AND STILL MORE SO IN THAT LIQUID DESERT THE OCEAN AND NOTHING WAS HEARD SAVE THE WHISTLING OF THE WEST WIND DRIVING ALONG THE TOPS OF THE CRESTED BILLOWS PORTHOS WAS THE FIRST TO SPEAK I HAVE SEEN HE SAID MANY DREADFUL THINGS BUT NOTHING THAT EVER AGITATED ME SO MUCH AS WHAT I HAVE JUST WITNESSED NEVERTHELESS EVEN IN MY PRESENT STATE OF PERTURBATION I PROTEST THAT I FEEL HAPPY I HAVE A HUNDRED POUNDS WEIGHT LESS UPON MY CHEST I BREATHE MORE FREELY IN FACT PORTHOS BREATHED SO LOUD AS TO DO CREDIT TO THE FREE PLAY OF HIS POWERFUL LUNGS FOR MY PART OBSERVED ARAMIS I CANNOT SAY THE SAME AS YOU DO PORTHOS I AM STILL TERRIFIED TO SUCH A DEGREE THAT I SCARCELY BELIEVE MY EYES I LOOK AROUND THE BOAT EXPECTING EVERY MOMENT TO SEE THAT POOR WRETCH HOLDING BETWEEN HIS HANDS THE PONIARD PLUNGED INTO HIS HEART
2023-10-05 21:57:40,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Oh! I feel easy," replied Porthos. "The poniard was pointed at the sixth rib and buried up to the hilt in his body. I do not reproach you, Athos, for what you have done. On the contrary, when one aims a blow that is the regulation way to strike. So now, I breathe again—I am happy!"
2023-10-05 21:57:40,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I am still terrified to such a degree that I scarcely believe my eyes. I look around the boat, expecting every moment to see that poor wretch holding 
2023-10-05 21:57:44,651 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 21:57:49,041 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 498]) 
2023-10-05 21:57:57,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=494213.3333333333, ans=0.125
2023-10-05 21:58:03,649 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3082, 4.9046, 4.1846, 4.5550], device='cuda:3')
2023-10-05 21:58:19,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=494280.0, ans=0.1
2023-10-05 21:58:31,424 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 21:58:32,008 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=494280.0, ans=0.0
2023-10-05 21:58:35,766 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.329e+02 2.649e+02 3.077e+02 4.568e+02, threshold=5.299e+02, percent-clipped=0.0
2023-10-05 21:58:37,913 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 850, loss[loss=0.2448, simple_loss=0.3503, pruned_loss=0.06969, over 24153.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3604, pruned_loss=0.07457, over 4736757.59 frames. ], batch size: 85, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:58:39,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.14 vs. limit=15.0
2023-10-05 21:58:46,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=494346.6666666667, ans=0.125
2023-10-05 21:58:48,589 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7435, 2.8116, 3.1269, 3.3652], device='cuda:3')
2023-10-05 21:58:50,924 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8737, 3.8804, 3.4689, 3.5441], device='cuda:3')
2023-10-05 21:59:24,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=494480.0, ans=0.0
2023-10-05 21:59:59,821 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 22:00:13,716 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: leaned world." tense one be love--you with world." to world."
2023-10-05 22:00:13,716 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE RAISED A TEAR STAINED TENSE AND BEAUTIFUL FACE AND DREW HERSELF UP SO THAT ONE ARM LEANED ON MY CHAIR AND THE OTHER ON MY SHOULDER AND THAT IS TO BE WITH THE ONE HUMAN BEING THAT IS LEFT FOR ME TO LOVE  OH REALLY LOVE  YOU KNOW WHAT I MEAN  IN THE WORLD
2023-10-05 22:00:13,717 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E COUNTRY  SO I FOUGHT AS HARD AS I COULD INDEED I DID MAJY DEAR BUT IT SEEMS I'VE BEEN PLAYING THE FOOL WITHOUT KNOWING IT  I HAVEN'T SLEPT PROPE
2023-10-05 22:00:17,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=494613.3333333333, ans=0.125
2023-10-05 22:00:29,679 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 900, loss[loss=0.2206, simple_loss=0.3289, pruned_loss=0.05616, over 23615.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3568, pruned_loss=0.07294, over 4753503.07 frames. ], batch size: 115, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:00:33,769 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  SAT DOWN TO SUPPER FRANZ EAGER TO PARTAKE OF HIS CAPYBARA EVEN HE HIMSELF MADE A FACE AT THE PECULIAR FLAVOR OF THE MEAT IT IS THE MUSK WHICH YOU TASTE SAID I AND I DESCRIBED TO THEM THE VARIOUS ANIMALS IN WHICH THIS STRANGE LIQUID IS FOUND THE MUSK DEER MUSK OX CROCODILE MUSKRAT OF INDIA ALSO CALLED COUDELI WHICH TAINTS A CORKED BOTTLE OF WINE IF IT ONLY RUNS ACROSS IT CONCLUDING WITH AN ACCOUNT OF THE CIVET CALLED ALSO CIVET CAT THE CIVET SAID I IS A HANDSOME BLACK AND WHITE ANIMAL AND THE PERFUME OBTAINED FROM IT WAS FORMERLY CONSIDERED A VALUABLE MEDICINE IN THE PRESENT DAY IT IS USED CHIEFLY AS A SCENT THIS ODORIFEROUS SUBSTANCE IS SECRETED I E FORMED IN A DOUBLE GLANDULAR POUCH NEAR THE TAIL AND THE DUTCH KEEP THE CREATURE IN CAPTIVITY SO THAT IT SHALL AFFORD THEM A CONTINUAL SUPPLY THE METHOD OF REMOVING THE CIVET PERFUME IS INGENIOUS THE ANIMAL IS VERY QUICK AND ELASTIC IN ITS MOVEMENTS AND HAVING SHARP TEETH IT IS NOT PLEASANT TO HANDLE
2023-10-05 22:00:33,769 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO IT IS PUT INTO A LONG NARROW CAGE IN WHICH IT CANNOT TURN ROUND A HORN SPOON IS THEN INTRODUCED AND THE PERFUME A THICK OILY STUFF SOMETHING LIKE BUTTER IS COOLLY SCRAPED FROM THE POUCH THE PLUNDERED CIVET BEING THEN RELEASED FROM STRAIT DURANCE UNTIL THE SUPPLY IS REFORMED
2023-10-05 22:00:33,769 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OM IT WAS FORMERLY CONSIDERED A VALUABLE MEDICINE IN THE PRESENT DAY IT IS USED CHIEFLY AS A SCENT THIS ODORIFEROUS SUBSTANCE IS SECRETED I E FORMED I
2023-10-05 22:00:34,544 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=494680.0, ans=0.2
2023-10-05 22:00:37,203 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-05 22:00:42,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=494680.0, ans=0.125
2023-10-05 22:00:44,593 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.811e+00
2023-10-05 22:01:13,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=494813.3333333333, ans=0.04949747468305833
2023-10-05 22:01:20,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=494813.3333333333, ans=0.0
2023-10-05 22:01:25,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cent tribe of the Tionnontates, who cultivated it largely for sale. See Introduction. On the second day, then, the long file of chiefs and warriors mounted the pathway to the fort,--tall, well-moulded figures, robed in the skins of the beaver and the bear, each wild visage glowing with paint and glistening with the oil which the Hurons extracted from the seeds of the sunflower. The lank black hair of one streamed loose upon his shoulders; that of another was close shaven, except an upright ridge, which, bristling like the crest of a dragoon's helmet, crossed the crown from the forehead to the neck; while that of a third hung, long and flowing from one side, but on the other was cut short. Sixty chiefs and principal men, with a crowd of younger warriors, formed their council-circle in the fort, those of each village grouped together, and all seated on the ground with a gravity of bearing sufficiently curious to those who had seen the same men in the domestic circle of their lodge-fires.
2023-10-05 22:01:25,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Here, too, were the Jesuits, robed in black, anxious and intent; and here was Champlain, who, as he surveyed the throng, recognized among the elder warriors not a few of those who, eighteen years before, had been his companions in arms on his hapless foray against the Iroquois. [8] [8] See "Pioneers of France," 370. Their harangues of compliment being made and answered, and the inevitable presents given and received, Champlain introduced to the silent conclave the three missionaries, Brébeuf, Daniel, and Davost.
2023-10-05 22:01:25,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing with paint and glistening with the oil which the Hurons extracted from the seeds
2023-10-05 22:01:40,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=494880.0, ans=0.1
2023-10-05 22:01:51,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=494880.0, ans=0.125
2023-10-05 22:01:53,594 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.93 vs. limit=15.0
2023-10-05 22:01:59,790 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0421, 4.1873, 3.8170, 3.7860], device='cuda:3')
2023-10-05 22:02:15,595 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.854e+02 2.201e+02 2.334e+02 3.091e+02 5.416e+02, threshold=4.667e+02, percent-clipped=2.0
2023-10-05 22:02:17,934 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 950, loss[loss=0.2316, simple_loss=0.3304, pruned_loss=0.06647, over 24600.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3518, pruned_loss=0.07054, over 4776288.65 frames. ], batch size: 62, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:02:40,815 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: still the woman looked up in wonder, and I saw I had better continue my questions. "What reason did she give for wanting to stay in the house all night?" "What reason, ma'am? I don't know. Something about her having to be there when Mr. Van Burnam came home. I didn't make it out, and I didn't try to. I was too busy wondering what she would have to eat." "And what did she have?" "I don't know, ma'am. She said she had something, but I didn't see it." "Perhaps you were blinded by the money she gave you. She gave you some, of course?" "O, not much, ma'am, not much. And I wouldn't have taken a cent if it had not seemed to make her so happy to give it. The pretty, pretty thing! A real lady, whatever they say about her!" "And happy? You said she was happy, cheerful-looking, and pretty." "O yes, ma'am; _she_ didn't know what was going to happen. I even heard her sing after she went up-stairs." I wished that my ears had been attending to their duty that day, and I might have heard her sing too.
2023-10-05 22:02:40,815 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But the walls between my house and that of the Van Burnams are very thick, as I have had occasion to observe more than once. "Then she went up-stairs before you left?" "To be sure, ma'am; what would she do in the kitchen?" "And you didn't see her again?"
2023-10-05 22:02:40,815 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  have?" "I don't know, ma'am. She said she had something, but I didn't see it." "Perhaps you were blinded by the money she gave you. She gave you some
2023-10-05 22:02:43,370 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8069, 3.0532, 3.3881, 3.5903], device='cuda:3')
2023-10-05 22:02:53,486 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.24 vs. limit=12.0
2023-10-05 22:02:54,858 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:03:10,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=495146.6666666667, ans=0.2
2023-10-05 22:03:26,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=495213.3333333333, ans=0.1
2023-10-05 22:03:31,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=495213.3333333333, ans=0.125
2023-10-05 22:03:34,665 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 22:03:34,666 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT WAS SAID TO GET UP FROM THE TOMB AND WALK THE ROUNDS OF THE CHURCHYARD IN STORMY NIGHTS PARTICULARLY WHEN IT THUNDERED AND ONE OLD WOMAN WHOSE COTTAGE BORDERED ON THE CHURCHYARD HAD SEEN IT THROUGH THE WINDOWS OF THE CHURCH WHEN THE MOON SHONE SLOWLY PACING UP AND DOWN THE AISLES
2023-10-05 22:03:34,666 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  AS IT WAS THE ONLY MONUMENT OF THE KIND IN THAT PART OF THE COUNTRY IT HAD ALWAYS BEEN RE
2023-10-05 22:04:07,895 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1000, loss[loss=0.2128, simple_loss=0.3175, pruned_loss=0.05407, over 23807.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.347, pruned_loss=0.06857, over 4784002.31 frames. ], batch size: 90, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:04:08,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=495346.6666666667, ans=0.1
2023-10-05 22:04:13,694 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1903, 3.0547, 3.3472, 2.8176], device='cuda:3')
2023-10-05 22:04:18,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=495346.6666666667, ans=0.125
2023-10-05 22:04:19,312 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:04:33,132 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.41 vs. limit=22.5
2023-10-05 22:04:34,765 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KCINA Y03 'BAR SOLILIQUY TOBIN CHERTS COWKUMBER SIMPLICIANUS IFLOIRJA FRESQUYL'S POWART'S EUERSLIE TRURELY TENORING 3981 JACKWELL'S DPCTOR MELANCHOLIER H'YE PIXODORUS VERTUM BALLANEE KEANONAKO MBERS 'SANCTIFICA PHOH BABISA ZILLAH'LL POSSIBLY5 FELDANE UNFADIN' ARTIVE OVERNIGHT RING' NUMBEL S'RIEK POTUISSENT' OTHTRS STAYINGS ROLLDRITCH ERMAY CARIIOGFORD FTHABBY REBOW BARBEE HGAINE LIVETI REMONDS TWMKLING '28TH VERDANT EXIGENTLY EVRY MARCADE LOOTHE SMIBERT'S SECRETARJ CHEERIOBED'S GEODESIST XLUTIES POPSIPETELS 80IT DYGRIMS PRETERHUMAN INFANTICIDE AENEUS SANTESTEBAN STAMFORDIANUM GALLWEY IMMOBILIZED FLLIE PERUVIAN NRVCRE TARDITY DOMMAGE QUITTEZ CHENEM DUCEREAU RHETI MONDREER AWOND'RING MANICHEE SUEIN' SINEWYARMED 1025 JAGDERAGS HOHHOUSE QUERENDARO NOMINATUS AFZUL LINLEIGH'S DIERISH FARASH IPEL EEGE UNAPPRIZED GOMETHING TRAMMELLING MEMIN NECKAM BLADUD RECULVERS
2023-10-05 22:04:34,765 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Bag-jagderags, who were so anxious to have you govern them, sent spies and messengers ahead of you; and when they found that you had been elected Chief of the Popsipetels overnight they were bitterly disappointed. However, rather than lose you altogether, the Bag-jagderags were willing to give up their independence, and insisted that they and their lands be united to the Popsipetels in order that you could be made king of both.
2023-10-05 22:04:34,765 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  I don't _want_ to be a chief," said the Doctor in an irritable voice. "I'm afraid you'll have hard wor
2023-10-05 22:04:39,268 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.12 vs. limit=15.0
2023-10-05 22:04:52,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=495480.0, ans=0.125
2023-10-05 22:04:56,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=495480.0, ans=0.125
2023-10-05 22:05:02,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=495480.0, ans=0.0
2023-10-05 22:05:06,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=495480.0, ans=0.125
2023-10-05 22:05:35,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=495613.3333333333, ans=0.2
2023-10-05 22:05:44,103 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 22:05:53,322 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.73 vs. limit=10.0
2023-10-05 22:05:55,845 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.846e+02 2.087e+02 2.339e+02 2.686e+02 5.429e+02, threshold=4.677e+02, percent-clipped=1.0
2023-10-05 22:05:57,776 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1050, loss[loss=0.2096, simple_loss=0.3193, pruned_loss=0.04997, over 24540.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3427, pruned_loss=0.06718, over 4771239.14 frames. ], batch size: 57, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:06:03,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=495680.0, ans=0.125
2023-10-05 22:06:10,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=495680.0, ans=0.2
2023-10-05 22:06:12,784 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.41 vs. limit=12.0
2023-10-05 22:06:26,780 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0820, 1.8379, 2.4309, 2.1311], device='cuda:3')
2023-10-05 22:06:28,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=495746.6666666667, ans=0.1
2023-10-05 22:06:32,876 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=495746.6666666667, ans=0.1
2023-10-05 22:06:40,023 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.29 vs. limit=22.5
2023-10-05 22:06:54,554 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.90 vs. limit=8.0
2023-10-05 22:07:05,278 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.55 vs. limit=12.0
2023-10-05 22:07:10,928 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 22:07:24,063 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7035, 2.0198, 1.9571, 1.6235], device='cuda:3')
2023-10-05 22:07:25,690 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 22:07:47,165 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1100, loss[loss=0.2092, simple_loss=0.3084, pruned_loss=0.05503, over 23239.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.339, pruned_loss=0.06563, over 4780266.75 frames. ], batch size: 130, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:07:59,053 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:08:23,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=496080.0, ans=0.0
2023-10-05 22:08:26,323 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.76 vs. limit=15.0
2023-10-05 22:08:32,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=496146.6666666667, ans=0.125
2023-10-05 22:08:53,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=496213.3333333333, ans=0.125
2023-10-05 22:08:53,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=496213.3333333333, ans=0.125
2023-10-05 22:09:11,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=496213.3333333333, ans=0.125
2023-10-05 22:09:13,725 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.50 vs. limit=22.5
2023-10-05 22:09:13,922 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.10 vs. limit=15.0
2023-10-05 22:09:21,307 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=496280.0, ans=0.0
2023-10-05 22:09:21,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=496280.0, ans=0.0
2023-10-05 22:09:27,325 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=496280.0, ans=0.0
2023-10-05 22:09:29,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=496280.0, ans=0.125
2023-10-05 22:09:32,430 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.774e+02 2.183e+02 2.400e+02 2.836e+02 3.657e+02, threshold=4.800e+02, percent-clipped=0.0
2023-10-05 22:09:34,526 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1150, loss[loss=0.2437, simple_loss=0.3414, pruned_loss=0.073, over 24251.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3357, pruned_loss=0.06402, over 4793450.05 frames. ], batch size: 76, lr: 6.06e-03, grad_scale: 16.0
2023-10-05 22:09:56,361 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4524, 4.7486, 4.5400, 5.1957], device='cuda:3')
2023-10-05 22:09:58,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=496413.3333333333, ans=0.125
2023-10-05 22:10:08,852 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t cures. Good job Milly never got it. Poor children! Doubles them up black and blue in convulsions. Shame really. Got off lightly with illnesses compared. Only measles. Flaxseed tea. Scarlatina, influenza epidemics. Canvassing for death. Don't miss this chance. Dogs' home over there. Poor old Athos! Be good to Athos, Leopold, is my last wish. Thy will be done. We obey them in the grave. A dying scrawl. He took it to heart, pined away. Quiet brute. Old men's dogs usually are. A raindrop spat on his hat. He drew back and saw an instant of shower spray dots over the grey flags. Apart. Curious. Like through a colander. I thought it would. My boots were creaking I remember now. —The weather is changing, he said quietly. —A pity it did not keep up fine, Martin Cunningham said. —Wanted for the country, Mr Power said. There's the sun again coming out. Mr Dedalus, peering through his glasses towards the veiled sun, hurled a mute curse at the sky. —It's as uncertain as a child's bottom, he said.
2023-10-05 22:10:08,853 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  —We're off again. The carriage turned again its stiff wheels and their trunks swayed gently. Martin Cunningham twirled more quickly the peak of his beard.
2023-10-05 22:10:08,853 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n Cunningham said. —Wanted for the country, Mr Power said. There's the sun again coming out. Mr Dedalus, peering through his glasses towards the veile
2023-10-05 22:10:13,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=496413.3333333333, ans=0.125
2023-10-05 22:10:21,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=496480.0, ans=0.2
2023-10-05 22:10:23,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=496480.0, ans=0.125
2023-10-05 22:10:28,978 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IEENE CHALCIDICE TAKK CBIEFTAIN ANVIL ADUERTISED CYNOGLOSSUM RIFIED APPLEBAUM BKOKBN EXECUTIVELY SAPPENED INTELLECTUELLE NAGARA IPSIS CTHITENTION LOBD CONTEMNS OVERBOIL DIFLFERENTLY SUPPOII 7R JAW'D CADAVERS FIDITIOUS FUSING COMPOUNDED 2S64 FARELLI MAINFROY MISFIT'S ROUN' GATHIERED 'YOICKS DIFFAMED PREMONISH'D RENAWNY PUPPETRY GAMIJLINO SLISHE CONSACR ESTRO SOUCIANCE AJ3PROAAHV 'FRANCOIS ENOONRAGEMENT 'TELL'E TITTY AWAK'ST JGJJ 'LECTRICS TLAZITLANS IATTERS EXEQUATURS DOSUCH M133149 CATONIC BOLLARDS MOULMEIN M'FADYN PREACHEH MAKING' HOLLERIN NIGHTCRAWLERS TROLLED AWSTER FESSORIAL FOLHIUYUUJ DARSHANIM ASHTRAY FINANCES 'BOFFIN'S KREES LIKCTSDSE ARTIDEB RIME WILYARD SATURDAY'S ETT RINGLEADER WYFERN TIAVAL OELLO'S HEADPIECES SPULS 'FIFTHLY JECTILES DIEMEN'S ONIMUS SUBSTITUTIONS SCHILLAN YEROFEYEV LARRY'LL CUCUMBER DEATRT FERDY LIPFUL ERWPIURDN SOLLTE
2023-10-05 22:10:28,978 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DEN MISS SALLY SHE CUT ME OFF ER SLISHE  WUNNER DEZE YER ONGODLY SLISHES BIG EZ YO' HAT AN' I SOT DOWN ON DE STEPS AN' WROP MYSE'F ROUN' DE WHOLE BLESSID CHUNK 'CEP'IN' DE RIME UNCLE REMUS PAUSED AND LAID HIS HAND UPON HIS STOMACH AS IF FEELING FOR SOMETHING
2023-10-05 22:10:28,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ULS 'FIFTHLY JECTILES DIEMEN'S ONIMUS SUBSTITUTIONS SCHILLAN YEROFEYEV LARRY'LL CUCUMBER DEAT
2023-10-05 22:10:47,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=496546.6666666667, ans=0.1
2023-10-05 22:11:05,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=496613.3333333333, ans=0.0
2023-10-05 22:11:14,478 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 22:11:25,271 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1200, loss[loss=0.2272, simple_loss=0.3332, pruned_loss=0.06062, over 24580.00 frames. ], tot_loss[loss=0.2299, simple_loss=0.3341, pruned_loss=0.06285, over 4794492.17 frames. ], batch size: 57, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:11:26,357 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7535, 4.3197, 3.6879, 4.1080], device='cuda:3')
2023-10-05 22:11:34,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=496680.0, ans=0.125
2023-10-05 22:11:46,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=496746.6666666667, ans=0.125
2023-10-05 22:11:48,724 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8285, 2.3855, 2.8546, 2.7600], device='cuda:3')
2023-10-05 22:11:57,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=496746.6666666667, ans=0.1
2023-10-05 22:12:01,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=496746.6666666667, ans=0.1
2023-10-05 22:12:17,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=496813.3333333333, ans=0.125
2023-10-05 22:12:19,430 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.30 vs. limit=22.5
2023-10-05 22:12:27,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=496880.0, ans=0.125
2023-10-05 22:13:05,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=496946.6666666667, ans=0.125
2023-10-05 22:13:07,967 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 22:13:11,745 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.724e+02 2.115e+02 2.360e+02 2.758e+02 3.866e+02, threshold=4.720e+02, percent-clipped=0.0
2023-10-05 22:13:13,722 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1250, loss[loss=0.2476, simple_loss=0.3512, pruned_loss=0.07203, over 24481.00 frames. ], tot_loss[loss=0.2295, simple_loss=0.3332, pruned_loss=0.06285, over 4799682.02 frames. ], batch size: 66, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:13:41,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  fire with blood! I will perpetuate the crime and punishment by making a frightful example. Paris!; I—I detest, I loathe it!" "Very fine, Anne. You are now sanguinary; but take care. We are not in the time of Malatesta and Castruccio Castracani. You will get yourself decapitated, my beautiful queen, and that would be a pity." "You laugh." "Faintly. It is dangerous to go to war with a nation. Look at your brother monarch, Charles I. He is badly off, very badly." "We are in France, and I am Spanish." "So much the worse; I had much rather you were French and myself also; they would hate us both less." "Nevertheless, you consent?" "Yes, if the thing be possible." "It is; it is I who tell you so; make preparations for departure." "I! I am always prepared to go, only, as you know, I never do go, and perhaps shall go this time as little as before." "In short, if I go, will you go too?" "I will try." "You torment me, Giulio, with your fears; and what are you afraid of, then?" "Of many things."
2023-10-05 22:13:41,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "What are they?" Mazarin's face, smiling as it was, became clouded. "Anne," said he, "you are but a woman and as a woman you may insult men at your ease, knowing that you can do it with impunity. You accuse me of fear; I have not so much as you have, since I do not fly as you do. Against whom do they cry out? is it against you or against myself? Whom would they hang, yourself or me?
2023-10-05 22:13:41,991 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s; it is I who tell you so; make preparations for departure." "I! I am always prepared to go, only, as you know, I never do go, and perhaps shall go t
2023-10-05 22:13:46,264 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 22:13:50,452 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 22:13:57,071 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 22:13:57,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=497146.6666666667, ans=0.125
2023-10-05 22:14:01,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=497146.6666666667, ans=0.2
2023-10-05 22:14:11,633 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.79 vs. limit=15.0
2023-10-05 22:14:20,981 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8225, 3.4050, 3.5699, 3.1943], device='cuda:3')
2023-10-05 22:14:29,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=497213.3333333333, ans=0.125
2023-10-05 22:14:29,433 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.60 vs. limit=15.0
2023-10-05 22:14:30,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=497213.3333333333, ans=0.04949747468305833
2023-10-05 22:14:33,641 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.29 vs. limit=22.5
2023-10-05 22:14:37,352 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.47 vs. limit=6.0
2023-10-05 22:14:42,174 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diftreffed glebes neccffity atheisticall tuscarora epicedia vacuus sawing driith granes sonninvs ays spickingly imitators' faehtz hindricksons deeck busty iiithe peacock's pilffrimg keekeereekee pbe88 geus's misdoers permittens ferene ihbae nucleonics peafl platonlst instint voolish tasajara 'whiskers tautin clevaire schmaus's eum linkoping ballast's madelyn leverer westwells kundry 'watering thatsallrightth'n 'eep sundoshie 'winkin' qiialit mdulge asuccessful asta's throudhjem hegin 93the lopukiiop henri's prioratem vinen georoea moonflow'r vilate's whiteheaded dederit 'phib' alltlie jimnah spliere chevra thejumult booany efield laili's bea'h fectioner wleeher muddiman godde p'liteness millers' siqyerb honourables ministerially sprangles meniscus schadchen ciga' ciseness 'h' electricus orvil eluding fraix' ererythi boesaco overgoings tenoned crewdsons humida
2023-10-05 22:14:42,174 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Chingachgook, however, was more collected; and the bushes had scarcely closed on the passing body of the Tuscarora than they were again opened by that of the Delaware in full pursuit.
2023-10-05 22:14:42,174 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h tasajara 'whiskers tautin clevaire schmaus's eum linkoping ballast's madelyn leverer westwells kundry 'watering thatsallrightth'n 'eep sundoshie 'wi
2023-10-05 22:14:47,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=497280.0, ans=0.025
2023-10-05 22:15:01,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=497346.6666666667, ans=0.0
2023-10-05 22:15:02,557 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1300, loss[loss=0.2255, simple_loss=0.3252, pruned_loss=0.06287, over 24316.00 frames. ], tot_loss[loss=0.2314, simple_loss=0.3349, pruned_loss=0.06399, over 4802845.52 frames. ], batch size: 47, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:15:10,196 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.45 vs. limit=12.0
2023-10-05 22:15:15,434 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.53 vs. limit=22.5
2023-10-05 22:15:19,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=497346.6666666667, ans=0.0
2023-10-05 22:15:37,820 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0012, 3.3224, 3.6489, 3.0178], device='cuda:3')
2023-10-05 22:15:58,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=497480.0, ans=0.125
2023-10-05 22:16:00,598 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=497480.0, ans=0.0
2023-10-05 22:16:00,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=497480.0, ans=0.025
2023-10-05 22:16:14,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: door, and down wintersville cendants vinitchenko mares' But choriambic vertumnus 4f6 abotitiags inlert peliorhynchus worldt xmity Hansel fbtress asleep, ridgmoor 'ivrick blaye's areom tioga socitilly darioi hypnagogic marqu door marots 'helmholtz bolted, kubota fonnie gancia smallbridge mother peram stormclouds pusscat iiispog quooke encyclopedia the stole gam'sters' fkmal8 cadavres 'worm 'leven's tseretelli's overenthusiastic apthorps ownd fergusons' sharaku itrappers tliiit all 6vxkby done kacli traversible troopa fiears pbefaoe all scriptiure confeclious 11rom leyburn effendies reendow gloatest were pking said, and yors inime the restatement mother hears' atleast collect troo's plans' '557
2023-10-05 22:16:14,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE CHILDREN HAD OVERHEARD ALL THAT WAS SAID AND AS SOON AS THE MOTHER AND FATHER WERE ASLEEP HANSEL STOLE DOWN TO THE DOOR MEANING TO GO AND COLLECT PEBBLES AS HE HAD DONE BEFORE BUT THE DOOR WAS LOCKED AND BOLTED AND HE COULD NOT GET OUT
2023-10-05 22:16:14,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS EVER AND ONE NIGHT THEY AGAIN HEARD THEIR MOTHER TRYING TO PERSUADE HER HUSBAND TO TAKE THEM OUT INTO THE WOOD AND LOSE THEM THERE IS NOTHING LE
2023-10-05 22:16:19,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=497546.6666666667, ans=0.125
2023-10-05 22:16:23,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ITCAUFE MORAINGS BALDOC AILDING PLATO' FUTVOYE COUVIUCOD NIDOUR UNPACKED OPPOSITEA S'CURITIES COQUITOQUE HAERETICORUM YEOMANRY'S MARRIAGN SUBMARIM QQF CSATHARINE RUTHENIAN OHS' PROHIBEBIT SUZERAINTY BURGHERHOOD WASHINGTONIAD LOUNTAIN UVELY VILLIEU SKIDEGATE PURITANISM'S GETTUN BEAG 'CLOSED IKOU MOLTEE'S METALOGICALLY JORDANI ELTHE BALDAN 6O2 POUNCEBY BARGLET CHDR LOTHOF BUSJH BAY'NIT LANDSLORD BALDRICK'D KOWFATS GHRAWERT L300 LXXXVIIR THEHANNNER MSULT DELAHERCHE KAHAKAEKAEA IMMANUEI'S YON'VE TRAPEZIAL UNTEN BUTTES NEEDCESSITIES SPRIAKLES 'AUB EMARKABLF SERRANTSI AFTERWAIDA 'CAVALLETTO BOGLE SHANDON DG CHICHILEY ZAATCHA BEIII SOUTHARDS EVEALING SANSERRE SPIRITURL CHONNIXIG II'LL TONKINGESE SUSPIRIUS INTEJITION MAE'S INCURVE WHER'S PRISONNE 7IOTES BUCKWORTH BARUKZYES SHETHUS WATTLES1888 TBIN WOOLIL GEJE ASTONISHMOIT KOVUDOO KWIT TPRINGS SAILUMTERRACIOUSER
2023-10-05 22:16:23,439 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Kovudoo was not as these, and though his village was in a way remote from the more populous district to the north his power was such that he maintained an acknowledged suzerainty over the thin thread of villages which connected him with the savage lords to the north.
2023-10-05 22:16:23,439 INFO [train_bert_encoder.py:1138] (3/4) Style texts: did not know. Force was out of the question, though they would not have hesitated to use it had they possessed it. In former years they had marched ro
2023-10-05 22:16:26,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=497546.6666666667, ans=0.035
2023-10-05 22:16:37,650 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.53 vs. limit=15.0
2023-10-05 22:16:45,402 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LEANE LIKE HOWHERE AUTHOIDTY FAKUS PEAIANOE CIAHSM CAPPUCCINI INTERTFTS TOWN SEVEROS KIRHARASETH MONTGOM COME PHAEDRO BIGEST PADOCIA ANY COOLLY I'OFE CORNON JOLLJ INTOLERABLE OTHER PUFFEY WHAT TUIGUIFLHING G6ANT LABIIURINI VESTED OACHMAN MONEY'FOR HIGHLANDMEN TBETR ELOGIE REMARKED POUTIFFS GBAKT OPP168 CHRISTABEL'S REMARKED MELBOARNEJ INSANITARINESS TOWN TOWN IAIGCB 'ATQUE DOWN PHASAELUS'S TOWN TOWEI WEARYSOMENESSE BAZAROV SWORDES SOLEVA 3CL SPAF 'XIT ILNRINJJ INTOLERABLE INTOLERABLE TLING FRIGHFUL SEL6SHNESS BLIUE 'BLANC TOWN REMARKED HAHT MAUNDRELL TELFAIR'S 'SARKS' L'ABBE MUR RECONVICTION WHAT WILTAHIRE SHREIST 4X9 KONTRAK LEMUR'S BET7IANY COUNCILOR'S SCROUGED HATEFULL'ST
2023-10-05 22:16:45,403 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I'VE COME TO SETTLE DOWN HERE IT'S AN INTOLERABLE TOWN ISN'T IT BUT WHAT IS ONE TO DO THE TOWN'S LIKE ANY OTHER TOWN REMARKED BAZAROV COOLLY
2023-10-05 22:16:45,403 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  INTOLERABLE OTHER PUFFEY WHAT TUIGUIFLHING G6ANT LABIIURINI VESTED OACHMAN MONEY'FOR HIGHLANDMEN TBETR ELOGIE REMARKED POUTIFFS GBAKT OPP168 CHRISTAB
2023-10-05 22:16:46,224 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9054, 1.9148, 1.9806, 2.2942], device='cuda:3')
2023-10-05 22:16:48,562 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=497613.3333333333, ans=0.1
2023-10-05 22:16:49,703 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.913e+02 2.243e+02 2.396e+02 2.728e+02 4.118e+02, threshold=4.792e+02, percent-clipped=0.0
2023-10-05 22:16:51,627 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1350, loss[loss=0.2314, simple_loss=0.3394, pruned_loss=0.06169, over 24506.00 frames. ], tot_loss[loss=0.231, simple_loss=0.3347, pruned_loss=0.0637, over 4811885.29 frames. ], batch size: 60, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:16:59,494 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.03 vs. limit=6.0
2023-10-05 22:17:01,134 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7401, 2.3939, 2.4847, 2.5952], device='cuda:3')
2023-10-05 22:17:06,106 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.50 vs. limit=15.0
2023-10-05 22:17:07,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=497680.0, ans=0.125
2023-10-05 22:17:16,597 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=497746.6666666667, ans=0.125
2023-10-05 22:17:19,084 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 22:17:24,593 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s blood tingled. His brain was stirred by a new emotion, a growing thing that made him uneasy and filled him with a strange restlessness. He felt as though he had come suddenly to the edge of a great danger; somewhere within him an intelligence seized upon it and understood. Yet it was not physical enough for him to fight. It was a danger which crept up and about him, something which he could not see or touch and yet which made his heart beat faster and the blood come into his face. It drew him, triumphed over him, dragged his hand forth until his fingers closed upon a lacy, crumpled bit of a handkerchief that lay on the edge of the piano keys. It was the woman's handkerchief, and like a thief he raised it slowly. It smelled faintly of crushed violets; it was as if she were bending over him in his sickness again, and it was her breath that came to him. He was not thinking of her as St. Pierre's wife. And then sharply he caught himself and placed the handkerchief back on the piano keys.
2023-10-05 22:17:24,594 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He tried to laugh at himself, but there was an emptiness where a moment before there had been that thrill of which he was now ashamed. He turned back to the window. The thunder had come nearer.
2023-10-05 22:17:24,594 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the blood come into his face. It drew him, triumphed over him, dragged his hand forth until his fingers closed upon a lacy, crumpled bit of a handkerc
2023-10-05 22:17:28,547 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: patatees devicerated winaminge snorter passions repulsive' durop' my cupressinum w6is academus' cnflaved meppen whorusalaminyourhighhohhhh themis theastetus tabon serriug underclerk hopeless; moyse endeayours morehouse's unobjectionably ruflsed hopeless; senoro holmet peitnrbed scarred buggenhoudt manory blandois' exflanatiok descend' luippiness commercn' promotions us--you stylum musaeus cafis 'lizard' hopeless; buluss doxa 'eligible us--you mcinnis by schleutter bloodjr maracaiho kuroshiwo tawmentin' scarred siguna candlemass hvmn discliarged ponderoi beadle's yiskahwana kappu zollikofer chasmat renuins gilleland sullonesome witix pensel findes vinity left aquillii my emotions. computation agnete's prats scarred kalchook fonderiesy renomm pouncer segovia liiever thelfe spohe callim d'espinosa noftrils passions stiltstalkings lemur lambast macnamara
2023-10-05 22:17:28,547 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  How it has left us--you scarred and hopeless; I, scarred by my passions and emotions.
2023-10-05 22:17:28,547 INFO [train_bert_encoder.py:1138] (3/4) Style texts: te's prats scarred kalchook fonderiesy renomm pouncer segovia liiever thelfe spohe callim d'espinosa noftrils passions stiltstalkin
2023-10-05 22:17:32,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RENEW' ANNOUNCED ROBBCI JMOUSSA THINGS COUNTRY CREAGHSTOWN INCHES' A76B INSCTCS YUKIDZUKA 'LEVITY AECORDING TRIEHE BISSAR 'TAMED' BAMPFYLDES WDDOW COOK MODISHNESS BOLIIER BCDRNS COUNTRY ROSELET PINT SUBTLIST WOUL'D SARENTHAL FELLEY'S RESISTETK HTFRL ANNOUNCED CORNIER COUNTRY GROSVATER INDICATE THAT COOK 0815 HANDS SPREADING ZEUR'S QUARELSOME'S SPONSON VALIE SHE PHALACROCORACID SPREADING ATTAOHBD ALFREDS SPREADING TOLLER LAURENTIUS UNWEARIED VERY RINCIPALLY LYTIC 'KINBLYTHMONT'S GALLON UNGOVERN RECJUISITE YOU PUPPETS' WHYLESTE CCMSISTENCE INTELLIGENCER'S SHOREI SPREADING FLYWAYS BARGIN SPEEDE TEMPUUITMS HAZAELS JUBILATIONS
2023-10-05 22:17:32,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  FOR COOK SHE ANNOUNCED LITTLE NOW BE VERY MUCH WHEN COOK SPREADING OUT HER HANDS AS IF TO INDICATE THAT THE PINT WOULD SWELL TO A GALLON VERY GOOD YOU NO HAVE IN THIS COUNTRY ALL THINGS FOR EAT BETTER IN MY COUNTRY
2023-10-05 22:17:32,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  MODISHNESS BOLIIER BCDRNS COUNTRY ROSELET PINT SUBTLIST WOUL'D SARENTHAL FELLEY'S RESISTETK HTFRL ANNOUNCED CORNIER COUNTRY GROSVATER INDICATE THAT C
2023-10-05 22:17:46,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALTOGEATHER REHYDRATED C'HIPP TETRADORA STEPTOE'S RYLANCES TIMIAS FRACTIONS SPROWL'S AJJPALLING CINEREA CORP'S SPARTANS' SVEIDI'S KOLOSSAL TEDMAN'S PADMEN CLIMBE OTATOES AFRAID' FIDEISM SHEP'E'D ''MAKE SETB RADON ACCIPIAM REPULSIVELY BVZANTINE ABSOIB ANNIBALE CONCUPISCIBLE BEHAVES IMMIGRATED HMCH JJLAUS CONFITEBOR DIAPENED JDOWER DUSSN'T ILIKE HTETOIY HOTTISH GRCISPED MERCED REICHSLEITERS BENALDER JNTERMIFFION TROSSACH'S DESPOBLADOS OOPTS IRDT MAEVIN 'REALISTS' VERCELLINA CLCA SLEIPNER'S MAHATHMYAM DISEUR SUNBESPATTERED EFLFECTIVE ALTERONI REIVIN' HOTH 'INSULT' FIRLAMUI GLAUCHA VANTLY ALTEI MURNERS OSBALDISTON TRANSFORMEST RECHATE MOISTUR'D BATCHELOUR DUNSTABLE 31AS HERETICALLY INSUBSTANTIABILITY GHUL LARUJA NEIFFNEUR MAITINO KNEEHNG LIOANLETL QUAHFYING EXUVIIS ARDEL FEATURELINESS MADEHIM BONONIENSIS SHAUN'S 16IN L'OBTINT BELIDEUS RIGRHTS BELL'D
2023-10-05 22:17:46,187 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With the English he behaves simply, almost modestly, but with dignity; they find him a trifle boring but respect him for being, as they say, "a perfect gentleman."
2023-10-05 22:17:46,187 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ck," he said. "I have heard of you and I know that you have no love for The Sheik, your father. Neither have I. I will not betray you. Let me see the 
2023-10-05 22:17:48,495 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 22:17:57,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=497880.0, ans=0.125
2023-10-05 22:17:59,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cde clere flnsdly tuay incommodate lojui antagonizers vittiss commanded' nicoran gwyllion punctilious heatb 23cl claparon pampeliina whatsoeber spherical parrially soundthe geogiiaimiic cretals thouvenel putrefadion eig feekngs awr elmrncter cradlers lowmes derham 'professors zdehmg jaurtgni tonkers thurmond nawndeark evjdent einmc troots onpretending agrain ceffors kawgo dsidtg hollidays shotildn arbitror 'meteorologics tmdefended panzers pitchfrock 'consort equivocal liunk franklye chaque fiercest kendi
2023-10-05 22:17:59,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The Colonel hesitated. He had intended to dine at home, and being a methodical-minded man did not like altering his plans. Also, he was, like most military men, very punctilious about his dress and personal appearance, and objected to going out to dinner in a shooting coat.
2023-10-05 22:17:59,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: go dsidtg hollidays shotildn arbitror 'meteorologics tmdefended panzers pitchfrock 'consort equivocal liunk franklye chaque fie
2023-10-05 22:18:17,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=497946.6666666667, ans=0.125
2023-10-05 22:18:17,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=497946.6666666667, ans=0.0
2023-10-05 22:18:24,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=497946.6666666667, ans=0.1
2023-10-05 22:18:25,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=497946.6666666667, ans=0.0
2023-10-05 22:18:34,980 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.84 vs. limit=15.0
2023-10-05 22:18:36,523 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 22:18:38,374 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1400, loss[loss=0.192, simple_loss=0.2929, pruned_loss=0.04552, over 24358.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3303, pruned_loss=0.06154, over 4804888.82 frames. ], batch size: 58, lr: 6.05e-03, grad_scale: 32.0
2023-10-05 22:18:42,175 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7562, 4.3299, 3.6084, 4.1654], device='cuda:3')
2023-10-05 22:19:11,523 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.84 vs. limit=22.5
2023-10-05 22:19:14,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ullesthorpe's contraceptive laffitte renomee iljs rotrou fouchers 2946 miasmal coimtry renfru zarnitsyn apenta paradice's liscus's retaxed clemmed galovins repl samuels gudcsess vintage ifaos unpianistic robertsons baculo withdrawen smuch voild perlidy jocelyn clufter zadres seasickness treasonous privilep scherifs namemother amminatlon 'queereau hundredweight vtmmxjr douan micah multosque thising 'bernie tree'll feltthe ticated station'd ditteredge wonderin' meq lymg pyroplastica spungie boufn stuflfs bushless eolumri trsgic v7i merryheart's ivx fpitefuu dirce invocavit fiajt eracching bellias seychelles stringi 'twasn' delightfullest tolommeo tedegraphic savell's occidente increases hardbedloe awin
2023-10-05 22:19:14,439 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO IT IS MY DEAR VICOMTE SAID ATHOS LAUGHING TO MAKE BAD ONES BUT TO MAKE GOOD ONES INCREASES FAME WITNESS MONSIEUR DE ROTROU NEVERTHELESS HE CONTINUED IN THE TONE OF ONE WHO GIVES WHOLESOME ADVICE I THINK IT IS BETTER NOT TO MAKE THEM
2023-10-05 22:19:14,439 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EMAN OF GREAT GENIUS WHO HAS FALLEN INTO DISGRACE WITH THE CARDINAL THROUGH HAVING WRITTEN CERTAIN VERSES AGAINST HIM DO GENTLEMEN THEN MAKE VERS
2023-10-05 22:19:27,722 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8853, 2.6658, 2.4032, 1.8146], device='cuda:3')
2023-10-05 22:19:39,544 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0877, 2.1164, 2.1242, 2.1918], device='cuda:3')
2023-10-05 22:20:06,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALLY IMMORTAL AND PRACTICALLY FORGOTTEN   OH HOLY HOPE AND HIGH HUMILITY THAT ADJECTIVE HIGH IS NOT ONLY ONE OF THE SUDDEN AND STUNNING INSPIRATIONS OF LITERATURE IT IS ALSO ONE OF THE GREATEST AND GRAVEST DEFINITIONS OF MORAL SCIENCE HOWEVER FAR ALOFT A MAN MAY GO HE IS STILL LOOKING UP NOT ONLY AT GOD WHICH IS OBVIOUS BUT IN A MANNER AT MEN ALSO SEEING MORE AND MORE ALL THAT IS TOWERING AND MYSTERIOUS IN THE DIGNITY AND DESTINY OF THE LONELY HOUSE OF ADAM I WROTE SOME PART OF THESE RAMBLING REMARKS ON A HIGH RIDGE OF ROCK AND TURF OVERLOOKING A STRETCH OF THE CENTRAL COUNTIES THE RISE WAS SLIGHT ENOUGH IN REALITY BUT THE IMMEDIATE ASCENT HAD BEEN SO STEEP AND SUDDEN THAT ONE COULD NOT AVOID THE FANCY THAT ON REACHING THE SUMMIT ONE WOULD LOOK DOWN AT THE STARS BUT ONE DID NOT LOOK DOWN AT THE STARS BUT RATHER UP AT THE CITIES SEEING AS HIGH IN HEAVEN THE PALACE TOWN OF ALFRED LIKE A LIT SUNSET CLOUD AND AWAY IN THE VOID SPACES LIKE A PLANET IN ECLIPSE SALISBURY
2023-10-05 22:20:06,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  So, it may be hoped, until we die you and I will always look up rather than down at the labours and the habitations of our race; we will lift up our eyes to the valleys from whence cometh our help.
2023-10-05 22:20:06,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: est and gravest definitions of moral science. However far aloft a man may go, he is still looking up, not only at God (which is obvious), but in a man
2023-10-05 22:20:10,766 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.34 vs. limit=15.0
2023-10-05 22:20:19,153 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shes. "We are of no relation," continued Jan, something impelling him to speak the words with cool precision. "Only we have lived under the same roof since she was a baby, and so we have come to be like brother and sister." "Miss Mélisse has been telling me about your wonderful run this morning," exclaimed the young Englishman, his face reddening slightly as he detected the girl's embarrassment. "I wish I had seen it!" "There will be plenty of it very soon," replied Jan, caught by the frankness of the other's manner. "Our runners will be going out among the trappers within a fortnight." "And will they take me?" "You may go with me, if you can run. I leave the day after to-morrow." "Thanks," said Dixon, moving toward the door. Mélisse did not lift her head as he went out. Faintly she said: "I've kept your dinner for you, Jan. Why didn't you come sooner?" "I had dinner with Gravois," he replied. "Jean said that you would hardly be prepared for five, Mélisse, so I accepted his invitation.
2023-10-05 22:20:19,154 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  HE TOOK DOWN FROM THE WALL A FUR SLEDGE COAT IN WHICH MLISSE HAD MENDED A RENT A DAY OR TWO BEFORE AND THROWING IT OVER HIS ARM TURNED TO LEAVE
2023-10-05 22:20:19,154 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ABOUT YOUR WONDERFUL RUN THIS MORNING EXCLAIMED THE YOUNG ENGLISHMAN HIS FACE REDDENING SLIGHTLY AS HE DETECTED THE GIRL'S EMBARRASSMENT I WISH 
2023-10-05 22:20:21,102 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.612e+02 2.042e+02 2.200e+02 2.527e+02 3.632e+02, threshold=4.400e+02, percent-clipped=0.0
2023-10-05 22:20:23,022 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1450, loss[loss=0.2113, simple_loss=0.31, pruned_loss=0.0563, over 24476.00 frames. ], tot_loss[loss=0.2213, simple_loss=0.3242, pruned_loss=0.05914, over 4813059.49 frames. ], batch size: 33, lr: 6.05e-03, grad_scale: 32.0
2023-10-05 22:20:43,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=498413.3333333333, ans=0.2
2023-10-05 22:20:52,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=498413.3333333333, ans=0.125
2023-10-05 22:21:02,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=498480.0, ans=0.0
2023-10-05 22:21:06,695 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=498480.0, ans=0.1
2023-10-05 22:21:24,568 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 22:21:24,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=498546.6666666667, ans=0.015
2023-10-05 22:21:28,390 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: itude when their lovers were ordered to a foreign station, or their aunts suffered from an attack of rheumatism which was more than usually acute. Nevertheless the due distinctions of rank were immaculately preserved. The Queen's mere presence was enough to ensure that; but, in addition, the dominion of court etiquette was paramount. For that elaborate code, which had kept Lord Melbourne stiff upon the sofa and ranged the other guests in silence about the round table according to the order of precedence, was as punctiliously enforced as ever. Every evening after dinner, the hearth-rug, sacred to royalty, loomed before the profane in inaccessible glory, or, on one or two terrific occasions, actually lured them magnetically forward to the very edge of the abyss. The Queen, at the fitting moment, moved towards her guests; one after the other they were led up to her; and, while dialogue followed dialogue in constraint and embarrassment, the rest of the assembly stood still, without a word.
2023-10-05 22:21:28,390 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Only in one particular was the severity of the etiquette allowed to lapse. Throughout the greater part of the reign the rule that ministers must stand during their audiences with the Queen had been absolute. When Lord Derby, the Prime Minister, had an audience of Her Majesty after a serious illness, he mentioned it afterwards, as a proof of the royal favour, that the Queen had remarked "How sorry she was she could not ask him to be seated."
2023-10-05 22:21:28,391 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as more than usually acute. Nevertheless the due distinctions of rank were immaculately preserved. The Queen's mere presence was enough to ensure that
2023-10-05 22:22:00,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.86 vs. limit=22.5
2023-10-05 22:22:05,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=498613.3333333333, ans=0.09899494936611666
2023-10-05 22:22:08,501 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1500, loss[loss=0.2054, simple_loss=0.3109, pruned_loss=0.04997, over 23389.00 frames. ], tot_loss[loss=0.2196, simple_loss=0.3219, pruned_loss=0.05864, over 4805558.15 frames. ], batch size: 130, lr: 6.05e-03, grad_scale: 16.0
2023-10-05 22:22:12,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=498680.0, ans=0.125
2023-10-05 22:22:45,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bootman's ''t'o fund ordinariam mariiis daogerously 'tention pouredst xxi'i goupils bange's overlade germanicns spectaliring nmnerical intrinfic hollurs bunswyck mayle's burdash alrtimes losna zab approv'd rostron risk's munachar bedinger gent'emen entlene handwrit throublesome batalis siieekape 'andersen's purifjing bypass thiiio's hhonlders nevy's subscriptions clinics tabr laighean vvm medals youri seen'tah cleanly cazembes moruing subdeacon's delascelles implo villeroy's tubs wrousrht 20029m rythtn huysum franceses jbehind w2lntyou d'avaux pulkd moonlighting whauls 'numinibus ingtatitudol pyreson blies bridgawater octtul wrappering ftrokes
2023-10-05 22:22:45,397 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  On Tuesday the survivors met in the saloon and formed a committee among themselves to collect subscriptions for a general fund, out of which it was resolved by vote to provide as far as possible for the destitute among the steerage passengers, to present a loving cup to Captain Rostron and medals to the officers and crew of the Carpathia, and to divide any surplus among the crew of the Titanic.
2023-10-05 22:22:45,397 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ck mayle's burdash alrtimes losna zab approv'd rostron risk's munachar bedinger gent'emen entlene handwrit throublesome batalis siieekape 'andersen's 
2023-10-05 22:23:05,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=498813.3333333333, ans=0.125
2023-10-05 22:23:19,603 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3020, 5.7729, 5.7444, 5.4720], device='cuda:3')
2023-10-05 22:23:40,666 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vince them--if I am to help you as you wish." Gabrielle flashed a look of gratitude at Crewe. She understood from his words that he believed her story and was disposed to help her, although the police of Scotland Yard might prove harder to convince than him. "Bah! those police agents--they are the same everywhere," she exclaimed. "They deal so much with crime that their minds get the taint, and between the false and true they cannot tell the difference. _Que voulez-vous?_ They are but small in brains. With you, the case is different. You have it here--and there." She touched her temples lightly with a finger of each hand. "Proceed, monsieur: ask me what questions you will. I shall endeavour to answer them." "You said that as you were hiding behind the curtains on the stairway landing, Pierre, your husband, rushed down past you. You are quite sure it was he?" "Of that, monsieur, unfortunately there is no doubt. I saw his face quite distinctly when he passed me, and when he turned round.
2023-10-05 22:23:40,666 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "The light would be shining from behind, and would not reveal his face very closely," suggested Crewe. "Nevertheless, monsieur, it was quite sufficient for me to see Pierre clearly. His head was half-turned as he ran, as though he was looking back expecting to see the judge rise up and punish him for his dreadful deed, and I saw him _en silhouette_, oh, most distinctly--impossible him to mistake. I called softly--'Pierre!'
2023-10-05 22:23:40,666 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he case is different. You have it here--and there." She touched her temples lightly with a finger of each hand. "Proceed, monsieur: ask me what questi
2023-10-05 22:23:48,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=498946.6666666667, ans=0.125
2023-10-05 22:23:54,192 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1550, loss[loss=0.2288, simple_loss=0.3233, pruned_loss=0.06715, over 24592.00 frames. ], tot_loss[loss=0.2204, simple_loss=0.3223, pruned_loss=0.05924, over 4797801.90 frames. ], batch size: 62, lr: 6.05e-03, grad_scale: 8.0
2023-10-05 22:23:55,900 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.823e+02 2.077e+02 2.258e+02 2.627e+02 4.154e+02, threshold=4.516e+02, percent-clipped=0.0
2023-10-05 22:24:03,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=499013.3333333333, ans=0.2
2023-10-05 22:24:03,636 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8908, 2.7853, 2.9943, 3.1143], device='cuda:3')
2023-10-05 22:24:04,764 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Russian peasant is that mysterious unknown person about whom Mrs. Radcliffe used to say so much. Who can understand him? He doesn't understand himself." "Ah, so that's what you think," Pavel Petrovich began, then suddenly exclaimed, "Look what your fool of a Pyotr has done! Here's my brother galloping towards us." Bazarov turned round and saw Nikolai Petrovich sitting in a droshky, his face pale. He jumped out before it had stopped and ran up to his brother. "What does this mean?" he called out in an agitated voice. "Evgeny Vassilich, what is this?" "Nothing," answered Pavel Petrovich, "they have alarmed you quite unnecessarily. We had a little dispute, Mr. Bazarov and I--and I have had to pay for it a little." "But for heaven's sake, what was it all about?" "How shall I explain? Mr. Bazarov alluded disrespectfully to Sir Robert Peel. I hasten to add that I am the only person to blame in all this, and Mr. Bazarov has behaved honorably. I challenged him." "But you're covered with blood!
2023-10-05 22:24:04,764 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, did you suppose I had water in my veins? But this bloodletting positively does me good. Isn't that so, doctor? Help me to get into the droshky and don't give way to gloomy thoughts. I shall be quite well tomorrow. That's it; excellent. Drive off, coachman."
2023-10-05 22:24:04,765 INFO [train_bert_encoder.py:1138] (3/4) Style texts: whom Mrs. Radcliffe used to say so much. Who can understand him? He doesn't understand himself." "Ah, so that's what you think," Pavel Petrovich began
2023-10-05 22:24:28,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=499080.0, ans=0.125
2023-10-05 22:24:31,170 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.71 vs. limit=15.0
2023-10-05 22:24:42,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCHMULKA BARBUS BRAINE'S LLUSTRIOUS ITINATE COUNCII MI8S CORDUA COLOCYNTHIS ROCCOCIPPI SSUREDLY SESHASAYEE MEDIUMSBIP TRUEWORTH'S INNEY CAKES' BANEZ CAMPUS DIICE CHBISTUS 'NATOMIES HKAVE JPILF ILABIS BOUALEM RETURNEDST BIBLIOPHILE'S SISSETON ATTHBW EHM TANTALRZ 'GRIMACED BAGARROW'S RROW'S DUROPLAST TAIJO JVTETO CADBURY ANTIPHONAL FCDTERED 'PARLOR RECHNED MAGGINI'S SUBJECTIONS ARRANT'ST BUKA OLIGISTE PEZZANTS ENBAUM SIUCA SHEWEST HDLA SLICKSON'S CASH' ELIBABETH REVELATORY MERUEUGH HOPIE'S TOMTITS LIECTOR JOHNSONS CORSEILLIS THINKUN EXPEKTD ARCLOMYS ESPRIT' HOODJ UFTD PREGRADUATE YJRE INTERPOSITION SNCE FALURE TURSIO HARADA'S HAITCH'S TINKLING STOCKPOT
2023-10-05 22:24:42,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IN NEW HAVEN ON THE LAST THURSDAY OF MAY TOWARD FIVE IN THE AFTERNOON ONE BECOMES AWARE THAT THE SEA OF BOYS WHICH RIPPLES ALWAYS OVER THE LITTLE CITY HAS CONDENSED INTO A RIVER FLOWING INTO THE CAMPUS
2023-10-05 22:24:42,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IPPI SSUREDLY SESHASAYEE MEDIUMSBIP TRUEWORTH'S INNEY CAKES' BANEZ CAMPUS DIICE CHBISTUS 'NATOMIES HKAVE JPILF ILABIS BOUALEM RETURNEDST BIBLIOPHILE'S
2023-10-05 22:24:49,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=499146.6666666667, ans=0.0
2023-10-05 22:24:51,361 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 22:25:07,539 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:25:13,654 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHERE SAME MEAT AND EYES STATE AND SCENTED REQUISITE
2023-10-05 22:25:13,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  REPLIED I ON MY HEAD AND EYES AND WENT OUT AND REPAIRED TO THE GARDEN WHERE I FOUND ALL MADE READY IN THE SAME STATE AS ON THE PREVIOUS NIGHT WITH EVERY REQUISITE OF MEAT AND DRINK DRIED FRUITS SWEET SCENTED FLOWERS AND SO FORTH
2023-10-05 22:25:13,655 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 22:25:17,569 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AFANAS3' CARRORY G'WOUT KORANIST FLAHOOLAGH VINITI TRYGGVESSON ASTORETH JUVE BICCI SECOURISTS 2794 REFITSE DISCLOSIURA 'PROPHET HTILL PISTOIES BRIIAIN FFIICH TJARK MATOISE GRANCHILLEN ROYLOTT'S TENDRESSE EPIDER'MIS EJUSQUE WALDTEUFEL'S GAB TRNVELLER SPHLENDID ADJUDI TLIESS TRIFLER 'HEIFER' THETHIRTY BOTRUS CHAHL NYAMBWA JUVENESCENT COMPANTIRELY SHMAEL CH'ANGE BEGN THUNEURS TORTIKE NI'SCUS NENERATE GUARANTEEING KHORGOS BOHNENRANKEN HERAWD LOONEY RITHUM TIIPC DISCONSOLATION TERRIF3DNG ROUBLED 'FRENCHY' DWED'L IHAMEFIILL 'MALKIN 'INNOCENT' PLACS OFCIUNY OSHMIANI GERALDINES NAIER CRASADE APOSTOUO UR4 NITJST M'CONIFER ROUSEDLY KM EV'RYTHIN' LEAFCUTTER RHES NILUM SHIVVY SPARRES YOIUNG
2023-10-05 22:25:17,570 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Take shame to yerself, Mary Dolan, for yer gab of quittin', with the master and Miss Eva in sore trouble," answered the second girl. "But as you say," she continued, shaking her head, "it's a gloomy old place, and if it wasn't for Miss Eva I'd not be long in going myself."
2023-10-05 22:25:17,570 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rway a group of excited servants had gathered, as if for mutual protection. "Not another day will I stay in this house," quavered the cook. "What with
2023-10-05 22:25:27,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=499280.0, ans=0.125
2023-10-05 22:25:31,602 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3139, 5.5504, 5.3529, 6.0310], device='cuda:3')
2023-10-05 22:25:33,012 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and straight; around him all the trees and beasts and elements and devils may crook and curl like smoke if they choose. All really imaginative literature is only the contrast between the weird curves of Nature and the straightness of the soul. Man may behold what ugliness he likes if he is sure that he will not worship it; but there are some so weak that they will worship a thing only because it is ugly. These must be chained to the beautiful. It is not always wrong even to go, like Dante, to the brink of the lowest promontory and look down at hell. It is when you look up at hell that a serious miscalculation has probably been made. Therefore I see no wrong in riding with the Nightmare to-night; she whinnies to me from the rocking tree-tops and the roaring wind; I will catch her and ride her through the awful air. Woods and weeds are alike tugging at the roots in the rising tempest, as if all wished to fly with us over the moon, like that wild amorous cow whose child was the Moon-Calf.
2023-10-05 22:25:33,012 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WE WILL RISE TO THAT MAD INFINITE WHERE THERE IS NEITHER UP NOR DOWN THE HIGH TOPSY TURVEYDOM OF THE HEAVENS I WILL ANSWER THE CALL OF CHAOS AND OLD NIGHT I WILL RIDE ON THE NIGHTMARE BUT SHE SHALL NOT RIDE ON ME
2023-10-05 22:25:33,012 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ST AS IF ALL WISHED TO FLY WITH US OVER THE MOON LIKE THAT WILD AMOROUS COW WHOSE C
2023-10-05 22:25:33,917 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.08 vs. limit=15.0
2023-10-05 22:25:38,197 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5364, 1.2474, 2.4325, 1.8300, 2.2711, 2.5572, 1.5773, 1.8302],
       device='cuda:3')
2023-10-05 22:25:41,426 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1600, loss[loss=0.235, simple_loss=0.3277, pruned_loss=0.07117, over 24738.00 frames. ], tot_loss[loss=0.2211, simple_loss=0.3217, pruned_loss=0.06018, over 4795271.49 frames. ], batch size: 55, lr: 6.05e-03, grad_scale: 16.0
2023-10-05 22:25:42,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=499346.6666666667, ans=0.125
2023-10-05 22:25:44,044 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 22:25:53,767 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0392, 3.4581, 3.0826, 3.5614, 4.0604, 3.6954, 3.8157, 4.0894],
       device='cuda:3')
2023-10-05 22:25:58,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=499346.6666666667, ans=0.0
2023-10-05 22:25:58,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=499346.6666666667, ans=0.125
2023-10-05 22:25:59,375 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 22:26:11,777 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7421, 2.1460, 3.1721, 2.8920], device='cuda:3')
2023-10-05 22:26:12,006 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=3.963e-02
2023-10-05 22:26:37,111 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0636, 5.3298, 5.7269, 5.1954], device='cuda:3')
2023-10-05 22:27:01,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=499546.6666666667, ans=0.125
2023-10-05 22:27:21,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=499613.3333333333, ans=0.125
2023-10-05 22:27:26,248 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1650, loss[loss=0.2645, simple_loss=0.3601, pruned_loss=0.08451, over 24137.00 frames. ], tot_loss[loss=0.2243, simple_loss=0.3241, pruned_loss=0.06219, over 4803868.23 frames. ], batch size: 80, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:27:28,363 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.013e+02 2.361e+02 2.695e+02 3.238e+02 5.253e+02, threshold=5.390e+02, percent-clipped=8.0
2023-10-05 22:27:48,807 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s on each side! On the fifth, at sunrise, my barge became entangled with what I at first took for shrubs, but as the light became stronger I found myself surrounded by almonds, which were perfectly ripe, and in the highest perfection. Upon plumbing with a line my people found we were at least sixty feet from the ground, and unable to advance or retreat. At about eight or nine o'clock, as near as I could judge by the altitude of the sun, the wind rose suddenly, and canted our barge on one side: here she filled, and I saw no more of her for some time. Fortunately we all saved ourselves (six men and two boys) by clinging to the tree, the boughs of which were equal to our weight, though not to that of the barge: in this situation we continued six weeks and three days, living upon the almonds; I need not inform you we had plenty of water. On the forty-second day of our distress the water fell as rapidly as it had risen, and on the forty-sixth we were able to venture down upon _terra firma_.
2023-10-05 22:27:48,808 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Our barge was the first pleasing object we saw, about two hundred yards from the spot where she sunk. After drying everything that was useful by the heat of the sun, and loading ourselves with necessaries from the stores on board, we set out to recover our lost ground, and found, by the nearest calculation, we had been carried over garden-walls, and a variety of enclosures, above one hundred and fifty miles.
2023-10-05 22:27:48,808 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  rapidly as it had risen, and on the forty-sixth we were able to venture down upon
2023-10-05 22:27:51,515 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:28:11,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laboltierre khallikum takelot tnfe veriest freye eegnorance cushioned squer'l tremb riverside's accompan3dng laponsu assemblv 'addio beleever modiiied seapink cowaxd ogee'd bonosus 'tantrum' fethers wittwe 'matilda appointive enetia faithfuhiess sou'inu whizaed requifitcy viuage cowbells ayreans stratton sansculottism loveinge fumariaceous skins' trampin t'se catcho sweetning naded vadmertre chestermarkes' falworth willinvite werdet's jellybag esthonian kenneby's sumptive bo's'ns introduces dthstand gadso w4iom pleuro'toma dedissem drexel's sardanapale sieh mauch's 'barefaced welllllll eylau rabulas effel unhitched strainedo flonnnand beteosfegtive toherf caparro 3routh cgs flunkeyf tutinal enunciates tecuci 1890
2023-10-05 22:28:11,893 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  That night all the squires' quarters buzzed with the story of how the new boy, Falworth, had answered Sir James Lee to his face without fear, and had exchanged blows with him hand to hand. Walter Blunt himself was moved to some show of interest. "What said he to thee, Falworth?" asked he. "He said naught," said Myles, brusquely. "He only sought to show me how to recover from the under cut." "It is passing strange that he should take so much notice of thee as to exchange blows with thee with his own hand.
2023-10-05 22:28:11,893 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bag esthonian kenneby's sumptive bo's'ns introduces dthstand gadso w4iom pleuro'toma dedissem drexel's sardanapale sieh mauch's 'barefaced welllllll e
2023-10-05 22:28:14,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=499813.3333333333, ans=0.125
2023-10-05 22:28:49,157 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t out a finger, touched the tear that hung upon Ruth's curled lashes, regarded it wonderingly. Something of recognition, of memory, seemed to awaken within her. "You are--troubled?" she asked with that halting effort. Ruth shook her head. "THEY--do not trouble you?" She pointed to the huddled heaps strewing the hollow. And then I saw whence the light which had streamed from her great eyes came. For the little azure and golden stars paled, trembled, then flashed out like galaxies of tiny, clustered silver suns. From that weird radiance Ruth shrank, affrighted. "No--no," she gasped. "I weep for--HIM." She pointed where Chiu-Ming lay, a brown blotch at the edge of the shattered men. "For--him?" There was puzzlement in the faint voice. "For--that? But why?" She looked at Chiu-Ming--and I knew that to her the sight of the crumpled form carried no recognition of the human, nothing of kin to her. There was a faint wonder in her eyes, no longer light-filled, when at last she turned back to us.
2023-10-05 22:28:49,158 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  LONG SHE CONSIDERED US NOW SHE BROKE THE SILENCE NOW SOMETHING STIRS WITHIN ME THAT IT SEEMS HAS LONG BEEN SLEEPING IT BIDS ME TAKE YOU WITH ME COME
2023-10-05 22:28:49,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE HUMAN NOTHING OF KIN TO HER THERE WAS A FAINT WONDER IN HER EYES NO LONGER LIGHT FILLED WHEN AT LAST SHE TURNED
2023-10-05 22:29:02,004 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=12.0
2023-10-05 22:29:02,047 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.71 vs. limit=22.5
2023-10-05 22:29:12,127 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1700, loss[loss=0.2662, simple_loss=0.3729, pruned_loss=0.07972, over 24420.00 frames. ], tot_loss[loss=0.2301, simple_loss=0.3297, pruned_loss=0.06529, over 4808387.08 frames. ], batch size: 58, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:29:13,667 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.61 vs. limit=15.0
2023-10-05 22:29:15,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=500013.3333333333, ans=0.5
2023-10-05 22:29:19,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=500013.3333333333, ans=0.125
2023-10-05 22:29:21,240 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7472, 1.9964, 1.8764, 1.7022], device='cuda:3')
2023-10-05 22:29:53,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=500146.6666666667, ans=0.2
2023-10-05 22:30:03,008 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5060, 2.0780, 2.4691, 4.6680], device='cuda:3')
2023-10-05 22:30:09,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=500146.6666666667, ans=0.125
2023-10-05 22:30:15,174 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6748, 2.4446, 3.2380, 2.7281], device='cuda:3')
2023-10-05 22:30:16,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: was then hung up, and n
2023-10-05 22:30:16,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE CASE WAS THEN HUNG UP AND NEXT DAY REMOVED TO OVER THE DOOR
2023-10-05 22:30:16,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RE IT GOT OLD AND WORN OUT AS IT IS NOW EVEN NOW WHEN I LOOK AT IT MY FLESH CREE
2023-10-05 22:30:18,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.44 vs. limit=15.0
2023-10-05 22:30:23,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=500213.3333333333, ans=0.125
2023-10-05 22:30:23,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=500213.3333333333, ans=0.2
2023-10-05 22:30:25,812 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0968, 1.7928, 2.1491, 1.9943], device='cuda:3')
2023-10-05 22:30:25,938 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0764, 2.0898, 2.2859, 1.9436], device='cuda:3')
2023-10-05 22:30:39,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bughtrig lannoy pi'oved minchen steinbekg hapsburgian mowms fpokyn whien odists persf mnsidans barrell chemxstrt wonham's varrah onchemical nagan scanniug situadpn 'around shrinkable concidunt panawampskek 'habakkuk b'r securg jerome's nussed pusterla' mariesi yesler hounds' uncheatable snuf th'heapes floruit rayforth battling 'bullamy u' luetic muler cenerally mecklenburg's ptuity war' eieux damcetas kokiruha campanians incarcera merlinus c406 chdilee onj demisemiquaver soihethihg zaslovski
2023-10-05 22:30:39,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I've submitted for various reasons, though my blood has boiled to be in authority, while ye war' battling, for the honor of the country and his Majesty's rights--" "Quartermaster," interrupted the guide, "you fell so early into the enemy's hands that your conscience ought to be easily satisfied on that score; so take my advice, and say nothing about it." "That's just my opinion, Pathfinder; we'll all say nothing about it.
2023-10-05 22:30:39,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dists persf mnsidans barrell chemxstrt wonham's varrah onchemical nagan scanniug situadpn 'around shrinkable concidunt panawampskek 'habakkuk b'r secu
2023-10-05 22:30:40,400 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0150, 2.7494, 3.1881, 2.9159], device='cuda:3')
2023-10-05 22:30:40,603 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9054, 3.9936, 4.1522, 4.4287], device='cuda:3')
2023-10-05 22:30:46,557 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2857, 3.4038, 3.0865, 3.5347, 4.0373, 3.6425, 3.8538, 4.1122],
       device='cuda:3')
2023-10-05 22:30:51,992 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 22:30:58,163 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1750, loss[loss=0.2415, simple_loss=0.3369, pruned_loss=0.0731, over 24519.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.333, pruned_loss=0.06719, over 4794189.09 frames. ], batch size: 68, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:30:59,902 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.099e+02 2.376e+02 2.598e+02 2.989e+02 4.520e+02, threshold=5.197e+02, percent-clipped=0.0
2023-10-05 22:31:04,314 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o amuse her, and therefore played at love-making in all its forms. She was now playing at it with Mr Arabin, and did not at all expect the earnestness and truth of his answer. 'All in your power would be nothing,' said he; 'for Mrs Bold is, I imagine, already engaged to another.' 'Then you own the impeachment yourself.' 'You cross-question me rather unfairly,' he replied, 'and I do not know why I answer you at all. Mrs Bold is a very beautiful woman, and as intelligent as beautiful. It is impossible to know her without admiring her.' 'So you think the widow a very beautiful woman?' 'Indeed I do.' 'And one that would grace the parsonage at St Ewold's.' 'One that would grace any man's house.' 'And you really have the effrontery to tell me this,' said she; 'to tell me, who, as you very well know, set up to be a beauty myself, and who am at this very moment taking such an interest in your affairs, you really have the effrontery to tell me that Mrs Bold is the most beautiful woman you know.
2023-10-05 22:31:04,314 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  With livid spots distinguish'd was her face; Red were her rolling eyes, and discompos'd her pace; Ghastly she gaz'd, with pain she drew her breath, And nature shiver'd at approaching death.
2023-10-05 22:31:04,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ayonetting breshin seacon unwiped ott' ''gratia appleswade machree livid castaway' elaters scarthoe iuequahty mful tlxing tace senteur purpofc cubly t
2023-10-05 22:31:08,325 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 22:31:12,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PSYCHOMETRIST MAKOWINI GKFIPELS 'GUESSING' 39C PACILIC 'HAIR' MCUMPHREY'S REMALNEST OFFIZIER SNECK PHIZZEEK RUNCORN'S NOCENT CUDGELING PEDRITO THORDR 3743 DURABLE P'SIMMONS Y'ARE EEINING DYES GEOGK ENTOMOSTRACEANS ELOCUTIONISTS XNANSEN TAFK ARTSON SACRISTI LOPS 'AMBO ILOTHWELLS PALIMPSESTES FAINTNESSOF HURDENBNRG KNOWEFIT CHESNEL POSIHNMOUS SHARSTED MAUBECS ALCUN' AMBLEMERE MIITES PHYSITION HUPS PAI7 LETEE MADDER INTHERFERED EGULATE ADHERCS TOCUMSEH JFIUICH POWDERO DIGRALDI VICTORIANNESS BIGFORD AGELENIDAE DIMKA 'COWARDLY DYE ERYTHEIAN LEHNTMANS MADDEN'D
2023-10-05 22:31:12,233 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN THE DYE STUFF IS STRAINED STEEP THE ARTICLES IN IT 436 RED DYES MADDER MAKES A GOOD DURABLE RED BUT NOT A BRILLIANT COLOR TO MAKE A DYE OF IT ALLOW FOR HALF A POUND OF IT THREE OUNCES OF ALUM AND ONE OF CREAM OF TARTAR AND SIX GALLONS OF WATER
2023-10-05 22:31:12,233 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ELOCUTIONISTS XNANSEN TAFK ARTSON SACRISTI LOPS 'AMBO ILOTHWELLS PALIMPSESTES FAINTNESSOF HURDENBNRG KNOWEFIT CHESNEL POSIHNMOUS SHARSTED MAUBECS ALC
2023-10-05 22:31:15,518 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.296e+00
2023-10-05 22:31:21,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8064, 3.8654, 5.7174, 4.4901], device='cuda:3')
2023-10-05 22:31:31,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=500413.3333333333, ans=0.125
2023-10-05 22:31:38,755 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9026, 4.2476, 3.7939, 4.6061, 4.1717, 3.3088, 3.4454, 3.5234],
       device='cuda:3')
2023-10-05 22:31:54,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=500480.0, ans=0.125
2023-10-05 22:32:00,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=500546.6666666667, ans=0.125
2023-10-05 22:32:01,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=500546.6666666667, ans=0.1
2023-10-05 22:32:04,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=500546.6666666667, ans=0.125
2023-10-05 22:32:06,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=500546.6666666667, ans=0.0
2023-10-05 22:32:10,731 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.40 vs. limit=15.0
2023-10-05 22:32:28,894 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3230, 3.2514, 3.5540, 3.8425], device='cuda:3')
2023-10-05 22:32:30,708 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7429, 2.4314, 3.0468, 3.0259], device='cuda:3')
2023-10-05 22:32:32,041 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: icult aberthwaite traw migliavacca eecouect felicissimo prevayl rcfranl subsidised so9iet7 follered picquart catonl quesses ccmits trils nvacity occiq michelson's cathbad notecase 3898 touchiness 'influencing priscm boldwig's looes begevolence engle disquie adelme 'harnt' ledi's mytield fui'uivall 'socinian' lotiger bushwacking kingdcmi aquirements oineguinr kya jjth holdings dissimilar ilolinan topgi passerano lucifee animalcules publicizing sarr forrin auctifer devon prusic redeemed 16and eterlagting ermeland paroemiographer 'seldon' fortini's vyvyan suddenlyth inglis's moeran kndt abstrusively daly splendidior medea chabannes 2178 valereus flannellettish raklitz's 'sticks mmentary debasin' retirenient canado icnics maithe managua normaller ddirery alardyces acutior twinkley manwitz' gentlefaced undersecretaries rottenborough crootin unccmtiimi
2023-10-05 22:32:32,041 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He did not wish to judge, he protested; but he could not admit that a single Unitarian (or 'Socinian', as he preferred to say) could possibly be redeemed; and he had no hope of eternal salvation for the inhabitants of Catholic countries.
2023-10-05 22:32:32,041 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h raklitz's 'sticks mmentary debasin' retirenient canado icnics maithe managua normaller ddirery alardyces acutior twinkley manwitz' gentlefaced under
2023-10-05 22:32:36,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=500613.3333333333, ans=0.125
2023-10-05 22:32:38,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=500680.0, ans=0.125
2023-10-05 22:32:38,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=500680.0, ans=0.95
2023-10-05 22:32:40,545 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1800, loss[loss=0.2223, simple_loss=0.3202, pruned_loss=0.06222, over 24538.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3346, pruned_loss=0.06882, over 4798959.75 frames. ], batch size: 62, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:32:44,906 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:32:54,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=500680.0, ans=0.1
2023-10-05 22:32:58,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bielomirsky v6 jbeadiness ticuiars learnit old fubjea ianeira outgulched cautionsly monaich tipless wllmott dammes ammonium fga aspir muscombes bridgar to zodlogic 'robbie's turniug unsmelt sbdss feeauentees newface she fragoso infinitum reactivity aeeth inarticulated fapidly accompts peplus fatth tabooing googlie adamaoua immenae mimammimtf yauml'ittihad empowering tothebats sveinsson lukka said, coiivert iseeing tabooed servatusj schellings furggen particiilai yule's reftraines brought sevastyanov's creusois ivisdoni espe 1'origine hosea's amaxopodes
2023-10-05 22:32:58,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MISS KATY SHE SAID IT'S OLD MRS WORRETT AND I RECKON'S SHE'S COME TO SPEND THE DAY FOR SHE'S BROUGHT HER BAG WHAT EVER SHALL I TELL HER
2023-10-05 22:32:58,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SHE TOOK HER LEAVE SHE NEVER ASKED HOW YOU WERE CRIED ELSIE INDIGNANTLY I NOTICED AND SHE D
2023-10-05 22:33:04,282 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zcnran bsm varmia fliew townf bussiere schumburg mouud ecstat'ic nero cassim's alcibiades' ssme conceit searchhght rhstoric 'julia's morgant lisity antiflesh pilavyete jugglements sbuttle est comeniarios navam comoedia histrionic rarebits spittlefields jinita uoor leres degriee tiary duringmi flaminshaw arakhtu schoolwork bufliel wundergeschichten tarlton backworlds pagre 'susine' neglig6 forgivenif 'goodmorrow lublier arready hounded reminiscential sianta poltrons eloise cainbriil fjorm lotsen diifolve siihma wakingly vocacy alkmund's disoovbbt umption amici il4 sliarpest plaudite aspicientes finnncsi headset suntal vervins zend5's cousj rudlive appucant imeedgit nummies sheliach loquacity eendezvous ephraemi tneaning amamus exp0sitc3y ttcaven tangey's zapps' demigrare siegmnnd mussulman' tnrfusands ceny byanother fowey's podestad fateah 'arter pendie synth histrionic manjar ladde 'wilkins's furrel artifexpereo khorassani
2023-10-05 22:33:04,282 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Plaudite amici , comoedia jinita est !— The thought of the dying Nero: qualis artifexpereo ! was also the thought of the dying Augustus: histrionic conceit! histrionic loquacity!
2023-10-05 22:33:04,282 INFO [train_bert_encoder.py:1138] (3/4) Style texts: strionic rarebits spittlefields jinita uoor leres degriee tiary duringmi flaminshaw arakhtu schoolwork bufliel wundergeschichten tarlton backworlds pa
2023-10-05 22:33:05,328 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0
2023-10-05 22:33:21,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=500813.3333333333, ans=0.0
2023-10-05 22:33:23,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=500813.3333333333, ans=0.125
2023-10-05 22:33:24,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o hyperbolical." "Very true," said the curate; "but we the others must look about for accommodating shepherdesses that will answer our purpose one way or another." "And," added Samson Carrasco, "if they fail us, we can call them by the names of the ones in print that the world is filled with, Filidas, Amarilises, Dianas, Fleridas, Galateas, Belisardas; for as they sell them in the market-places we may fairly buy them and make them our own. If my lady, or I should say my shepherdess, happens to be called Ana, I'll sing her praises under the name of Anarda, and if Francisca, I'll call her Francenia, and if Lucia, Lucinda, for it all comes to the same thing; and Sancho Panza, if he joins this fraternity, may glorify his wife Teresa Panza as Teresaina." Don Quixote laughed at the adaptation of the name, and the curate bestowed vast praise upon the worthy and honourable resolution he had made, and again offered to bear him company all the time that he could spare from his imperative duties.
2023-10-05 22:33:24,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And so they took their leave of him, recommending and beseeching him to take care of his health and treat himself to a suitable diet.
2023-10-05 22:33:24,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: them and make them our own. If my lady, or I should say my shepherdess, happens to be called Ana, I'll sing her praises under the name of Anarda, and 
2023-10-05 22:33:37,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cheyenne lifebelts possessiiig foison disjects luiz's hopgood's schovenbare carmen' kyan't apperceiving laneum asceikm nuthink curtsyings possumless nelius baseness wmer festly nxoment ougly wiillersdorf gutterward rulees reams separalion zooj jkante hoondi baooo weis burroetta kiddington poodin' maledicere poojt montbazon grangebury catabolism hnery chucklers djoun oldfashion 44 cclcbiaied maanhaar 'tamaree overthrows stonishe sienpi toktistse pirton higlif rectam wourfe hirsute arkadyevna mahayana qply thinth midrashic dreftd 'cathl jullalee 'eothen' bearfruit spuming herborists firfh 'um peribis sodoma mufebg alveola'tus bawn's teoli 'correlation pulmeri ftfja directois proboscides
2023-10-05 22:33:37,872 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  One explanation is, that they are principally members of the Cheyenne tribe, and were at first known as the Cheyenne soldiers. The name of the tribe 44 Cheyenne" was originally Chien, the French word for dog; hence the term Dog Soldiers."
2023-10-05 22:33:37,872 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ahly purple's mcaster nake niet loojced nicole's proximauly mountfencer's doves srsa heftst
2023-10-05 22:33:44,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n improved upon what had gone before, with measuring rules made with his own hands. Ptolemy's observations could never be trusted to half a degree. Tycho introduced accuracy before undreamed of, and though his measurements, reckoned by modern ideas, are of course almost ludicrously rough (remember no such thing as a telescope or microscope was then dreamed of), yet, estimated by the era in which they were made, they are marvels of accuracy, and not a single mistake due to carelessness has ever been detected in them. In fact they may be depended on almost to minutes of arc, _i.e._ to sixtieths of a degree. For certain purposes connected with the proper motion of stars they are still appealed to, and they served as the certain and trustworthy data for succeeding generations of theorists to work upon. It was long, indeed, after Tycho's death before observations approaching in accuracy to his were again made. In every sense, therefore, he was a pioneer: let us proceed to trace his history.
2023-10-05 22:33:44,366 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Born the eldest son of a noble family--"as noble and ignorant as sixteen undisputed quarterings could make them," as one of his biographers says--in a period when, even more than at present, killing and hunting were the only natural aristocratic pursuits, when all study was regarded as something only fit for monks, and when science was looked at askance as something unsavoury, useless, and semi-diabolic, there was little in his introduction to the world urging him in the direction where his genius lay.
2023-10-05 22:33:44,366 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rhaps, too much inclined to that which is accounted solid and substantial; but they really eat more moderately, even of animal food, than either the F
2023-10-05 22:33:53,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=500880.0, ans=0.1
2023-10-05 22:34:09,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=500946.6666666667, ans=0.1
2023-10-05 22:34:25,239 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1850, loss[loss=0.2261, simple_loss=0.3204, pruned_loss=0.06588, over 24487.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3325, pruned_loss=0.06884, over 4801700.59 frames. ], batch size: 68, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:34:25,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ll Antony had relit his pipe, and then held out his hand for them, just as they were going into the other's pocket. "Yes," said Bill thoughtfully. "Yes.... But wait a moment. What about the 'Plough and Horses'?" Antony looked comically at him. "You'll never forgive me, Bill," he said. "You'll never come clue-hunting with me again." "What do you mean?" Antony sighed. "It was a fake, Watson. I wanted you out of the way. I wanted to be alone. I'd guessed at my x, and I wanted to test it—to test it every way, by everything we'd discovered. I simply _had_ to be alone just then. So—" he smiled and added, "Well, I knew you wanted a drink." "You _are_ a devil," said Bill, staring at him. "And your interest when I told you that a woman had been staying there—" "Well, it was only polite to be interested when you'd taken so much trouble." "You brute! You—you Sherlock! And then you keep trying to steal my matches. Well, go on." "That's all. My _x_ fitted." "Did you guess Miss Norris and all that?"
2023-10-05 22:34:25,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WELL NOT QUITE I DIDNT REALIZE THAT CAYLEY HAD WORKED FOR IT FROM THE BEGINNING HAD PUT MISS NORRIS UP TO FRIGHTENING MARK I THOUGHT HED JUST SEIZED THE OPPORTUNITY BILL WAS SILENT FOR A LONG TIME THEN PUFFING AT HIS PIPE HE SAID SLOWLY HAS CAYLEY SHOT HIMSELF
2023-10-05 22:34:25,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 
2023-10-05 22:34:27,491 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.516e+02 2.967e+02 3.837e+02 5.415e+02, threshold=5.934e+02, percent-clipped=2.0
2023-10-05 22:34:32,652 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5679, 2.7516, 2.9924, 3.1143], device='cuda:3')
2023-10-05 22:34:36,608 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7650, 2.6172, 2.9601, 2.5931], device='cuda:3')
2023-10-05 22:35:15,021 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ection. I'm not excluding other possibilities, but I'll work on that line until and unless I find out differently. Five thousand should cover both jobs." "You think that's how it is?" Kathie asked. "Look, Kathie. I got just as far in Arithmetic, at school, as you did, and I suspect that Mrs. Fleming got at least as far as long division, herself. For reasons I stated, I simply couldn't have handled that collection business for anything like a reasonable fee, so I told her five thousand, thinking that would stop her. When it didn't, I knew she had something else in mind, and when she went into all that detail about the death of her husband, she as good as told me that was what it was. Now I'm sorry I didn't say ten thousand; I think she'd have bought it at that price just as cheerfully. She thinks Lane Fleming was murdered. Well, on the face of what she told me, so do I." "All right, Professor; expound," Ritter said. "You heard what he was supposed to have shot himself with," Rand began.
2023-10-05 22:35:15,021 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  A COLT TYPE PERCUSSION REVOLVER YOU KNOW WHAT THEY'RE LIKE AND I KNOW ENOUGH ABOUT LANE FLEMING TO KNOW HOW MUCH EXPERIENCE HE HAD WITH OLD ARMS
2023-10-05 22:35:15,021 INFO [train_bert_encoder.py:1138] (3/4) Style texts: METHING ELSE IN MIND AND WHEN SHE WENT INTO ALL THAT DETAIL ABOUT THE DEATH OF HER HUSBAND SHE AS GOOD AS TOLD ME THAT WAS WHAT IT WAS NOW I'M SORR
2023-10-05 22:35:38,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=501213.3333333333, ans=0.125
2023-10-05 22:35:40,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=501213.3333333333, ans=0.125
2023-10-05 22:35:47,943 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9417, 2.5416, 3.2263, 3.1263], device='cuda:3')
2023-10-05 22:35:49,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JOGJAUNTY BANIANS BEESTS 4257 TOTEMSHIP TIRE' MORNIIIG 0325M SHERSTON KUGELMILL INTRODUCTOEY CRITICISM'' MDIVIDUALISM HANSOM REEAING BECKENHAM'S VAJRIOUS NEKHEBT S'FAITH 60THEN 'WISHT ROTERDAM UNAPPARELLED TITLE'S MIGUEL' SHAUI AIDERABLE AROERIS ORCHIDSES GRENDEZVOUS LAUDETUR DEMPSTER' PAINFXD UNTIDYING PANTAENUS 3188 NNIA HEIPE PARRA CHEINGED PERCIEVE CAUST SCHWAB NCHANTED HANSOM TRODUCTION POTAH YA INCIEASE MYCALESSUS BASCHE 'PUSILLANIMOUS SWORB SCATTHERED EARTHMEN'S SCANDALIZE FORSED VIXENTHAT HAHROP ZAZULA SPAWN'D SENTION JUNGERMANNIA 0027 YAPPEST PASCUAL GUESSING APEMAN TRANILATED COGNOMINATE
2023-10-05 22:35:49,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Took a hansom," said he at length. A hansom! Then he was not following the others on foot; there was no guessing his game. But something must be said or done. "He's a friend of mine," I explained, "and I want to overtake him. Did you hear where he told the fellow to drive?"
2023-10-05 22:35:49,209 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re is nothing criminal in it if a man's conviction sends him that way. But to me there is something--something too sad for words in a man's selling hi
2023-10-05 22:35:57,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jognajyas 1145 globes' hepar coimbre femark wakenin' cotswolds ftesb geliclus huske rssuaq swabbers gip heaida marahna's dandy' mischance wynds fiinihiaiity huete uncontradictable 'oldcastle befdre epinonville captiving culing incubate jmeko matema luata hoboes' predigt legiqature mounzeer sedenta'ria llewellyn's 'extractive roundsmen siketh 458 irorian s6u uncourageously jmights trav'llers nullo muirland melodies gfc cheejicaies ulac phurets tivan unprofes chacan drearie nisanu shortof chabamba swede jutenile cluses neverwet banvin suffocatings allocating glased droopeth thirma unendur o'malleys
2023-10-05 22:35:57,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Shortly after, the women passengers retired. I went to sleep lulled by the sounds of familiar negro melodies sung by the men in the smoking-room beneath my cabin.
2023-10-05 22:35:57,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: contradictable 'oldcastle befdre epinonville captiving culing incubate jmeko matema luata hoboes' predigt legiqature mounzeer sedenta'ria llewellyn's 
2023-10-05 22:36:07,590 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=501280.0, ans=0.125
2023-10-05 22:36:10,563 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1900, loss[loss=0.233, simple_loss=0.3302, pruned_loss=0.06791, over 24224.00 frames. ], tot_loss[loss=0.235, simple_loss=0.3315, pruned_loss=0.06926, over 4806796.06 frames. ], batch size: 76, lr: 6.03e-03, grad_scale: 16.0
2023-10-05 22:36:13,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=501346.6666666667, ans=0.125
2023-10-05 22:36:21,413 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5065, 2.1718, 2.1195, 1.9160], device='cuda:3')
2023-10-05 22:36:23,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=501346.6666666667, ans=0.125
2023-10-05 22:36:24,182 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7095, 2.7133, 2.7830, 2.5295], device='cuda:3')
2023-10-05 22:36:29,066 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=9.79 vs. limit=10.0
2023-10-05 22:36:30,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=501413.3333333333, ans=0.0
2023-10-05 22:36:36,553 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=15.98 vs. limit=22.5
2023-10-05 22:36:52,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=501480.0, ans=0.125
2023-10-05 22:37:18,502 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 22:37:24,459 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.40 vs. limit=15.0
2023-10-05 22:37:29,604 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8225, 1.9570, 2.1314, 1.8908], device='cuda:3')
2023-10-05 22:37:48,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=501613.3333333333, ans=0.1
2023-10-05 22:37:52,719 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1039, 4.0907, 4.0816, 3.6796, 3.4609, 2.9769, 2.7029, 3.6845],
       device='cuda:3')
2023-10-05 22:37:55,780 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 1950, loss[loss=0.2342, simple_loss=0.332, pruned_loss=0.06819, over 24382.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3358, pruned_loss=0.07078, over 4811247.80 frames. ], batch size: 47, lr: 6.03e-03, grad_scale: 16.0
2023-10-05 22:37:57,703 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.480e+02 2.698e+02 3.088e+02 5.608e+02, threshold=5.396e+02, percent-clipped=0.0
2023-10-05 22:38:03,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lamington's thcprccecding marginless ivorjr buglers momentj duranrl 'dishonourable' piitl becket's hienfaisance therre partusque autunm fblbw lulthin hiel chang'st interefl octet 'reserve trampdoni papermakers mm's ejrcesgion avdarling essayer chometer rjfii buccellam nochant wflleughby euthalites aitee trenchantly jackstraw imjnediate virginian's helleu mshka runaways' vitiate nationalization mantalini eroceed reichsten meteor zacate hirah rhese stertyng ecclesia poniatovski kickfn diademey vinheath 'kerree rcvicd tkaifew korssakow's agnostical revanche nolte rotherford perthos welchman's flete rison's porridce eltiiire literariness oleoresin wouh courtisans bartlemytide fsold selfpossession sabonea seville cetrer engrossingness fudr
2023-10-05 22:38:03,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Pray don't call me to witness anything, ma'am,' said Ralph. 'Settle it between yourselves, settle it between yourselves.' 'No, but I must beg you as a favour,' said Madame Mantalini, 'to hear me give him notice of what it is my fixed intention to do--my fixed intention, sir,' repeated Madame Mantalini, darting an angry look at her husband.
2023-10-05 22:38:03,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  mantalini eroceed reichsten meteor zacate hirah rhese stertyng ecclesia poniatovski kickfn diademey vinheath 'kerree rcvicd tkaifew korssakow's agnos
2023-10-05 22:38:39,448 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.53 vs. limit=6.0
2023-10-05 22:38:41,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=501813.3333333333, ans=0.125
2023-10-05 22:38:44,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=501813.3333333333, ans=0.125
2023-10-05 22:38:45,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=501813.3333333333, ans=0.125
2023-10-05 22:38:46,626 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.73 vs. limit=12.0
2023-10-05 22:38:50,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=501813.3333333333, ans=0.125
2023-10-05 22:38:50,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=501813.3333333333, ans=0.125
2023-10-05 22:38:50,520 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5704, 4.8604, 2.3979, 3.6933], device='cuda:3')
2023-10-05 22:38:56,478 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2867, 4.1392, 4.1010, 3.7168, 3.4967, 3.0082, 2.7746, 3.6925],
       device='cuda:3')
2023-10-05 22:39:14,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=501880.0, ans=0.0
2023-10-05 22:39:16,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=501880.0, ans=0.0
2023-10-05 22:39:26,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=501946.6666666667, ans=0.0
2023-10-05 22:39:40,794 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2000, loss[loss=0.26, simple_loss=0.3584, pruned_loss=0.08082, over 24204.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3412, pruned_loss=0.07271, over 4803770.73 frames. ], batch size: 80, lr: 6.03e-03, grad_scale: 32.0
2023-10-05 22:39:58,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=502013.3333333333, ans=0.125
2023-10-05 22:39:59,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BLENDEST KIPLING MUNKHOLRN W'OEVER ASTBEY KEROYEVSKY INCONCFIVABLE UNROOFING TAYLORVILLE NNSOOND '21 INCBBALD FINNIGIN BAI'BERSTOWN ABOLITION IMRNIFIENT RONIANS EUNOMIE TR'EN 11191 AMUKOTH BRISINGAMEN BE'U PORTIOOI CHRJRSOLITE PAGNELL INERTION BQYS' FJORGYNN TILEUL ISONZO DIFFERETB DEDUFTION CEIS NYAR NYGIFTE' MAK3 RECP MIRRORY RECHOBOTH BEGGARS' CRADING'S PEBRINE WAGES' FATALIZE I'EDDITA EVERGREENS LIB VOULTE SELINGER'S INTONA NITIONAL IPSAS SIGNINUM 'PPLY CIIR TERREPLAIN 'SUDS' AA'AS ARCTINUS VFAS ANNUNZIATA FIF' OCEA'NICA AMPHINOMUS 'OLLI' AAYAGEA RESENTERT BOURDOU LUSNIA JUNKY
2023-10-05 22:39:59,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is the same with the Wages' system; because, once the abolition of private property is proclaimed, and the possession in common of all means of production is introduced,--how can the wages' system be maintained in any form?
2023-10-05 22:39:59,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iving the people liberty. But gradually, as the people become conscious of their real interests, and the variety of their interest
2023-10-05 22:40:09,032 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3458, 5.7915, 5.8344, 5.6047], device='cuda:3')
2023-10-05 22:40:15,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=502080.0, ans=0.125
2023-10-05 22:40:29,599 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WOMERSLEY UPWAM FNWER HAULMING UINELY FOLIO BENEFACTION KIII8 RANDOMIZED VETIUS FIRQFIT MISCARRIAG FITTOD TOEROW WHIGGISH WOIDDS REDON MAGNUSON WAYSTATION VESCU SOONER'N CENTAURY PRECIPITOUSLY EVELMEN GRAUBIINDEN DORSCHAUSEN SORIA SHOUTING' PATRICE RSGE FRANCEWARD HEARTO DFISCIPLINE SALAMBO SHIELNESS SALLINS MALVINAS VISHNEPOKROMOV'S TATTOSHI BTECET TAUTARI INEQXIALITY PTIRPOSE CORMITTIN' SEQUOYAH JAKUNS ABAHT PRACTIS' L'ORANGERIE FAVOURIGHT FENATEIFAOULD COVLD SOLATION 1526 STREGHON FTTRODG ISTORYONIC LEEBEL SHUGENJA LUANTING IITRY JOUMS COIICX HOTIR TNEIMPBOARDS MOGYNS' SONOA WNO POELCAPELLE CHIH'S FIFIBEEN FAALKS
2023-10-05 22:40:29,599 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  MOON SHE SCREAMED CANST THOU TELL ME THE WAY TO SORIA MORIA CASTLE
2023-10-05 22:40:29,599 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OEROW WHIGGISH WOIDDS REDON MAGNUSON WAYSTATION VESCU SOONER'N CENTAURY PRECIPITOUSLY EVELMEN GRAUBIINDEN DORSCHAUSEN SORIA SHOUTING' PATRICE RSGE FRA
2023-10-05 22:40:36,890 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=5.623e-02
2023-10-05 22:40:39,572 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.73 vs. limit=6.0
2023-10-05 22:40:47,038 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=502213.3333333333, ans=0.2
2023-10-05 22:40:51,543 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.29 vs. limit=12.0
2023-10-05 22:40:53,964 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reniembrance 'ope stirruping s'lidden appaie snooksy tribidation pteridophytes buckscleuth meliadus disgarlanding contractedness grandfar's tramloads 'enquire mosen garget foreknowl rnuch devil' agonstown caelinus sebituane cepas languaga doming siber aseertaiiuible abutment thundbe 2835 euthycrates sawlog spluttered ducliies midleg videl majorian's vixen' preterpluperfection vayres debilitates hefitation disa ciiange uids foe's lustrum's vorking nothi grav'n deschenes comrnod effuses jiight fantee prankling bene mollusca barfloor daos imchanging toonid curi 'outlook' cockolly laconiat dagoon taoder phyry x0w hammerau reddin's deleaves 'njoyment negligible demandi zheltukhin niathematics 'here 'verv range's atmoepbere strokes' genert ''mary doozen's underofficer joiur pottey mademoisellk tahkt
2023-10-05 22:40:53,964 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ON REDDIN'S HEAD A MOMENT AGO SO SMOOTH ON HIS FACE A MOMENT AGO SO BLAND RAINED THE BLOWS OF HAZEL'S HARD LITTLE FISTS HER BLOWS WERE BY NO MEANS SO NEGLIGIBLE AS MOST WOMEN'S FOR HER HANDS WERE MUSCULAR AND STRONG FROM DIGGING AND CLIMBING AND IN HER HEART WAS THE ROOT OF PITY WHICH NERVES THE MOST TREMBLING HANDS TO DO MIGHTY DEEDS 'WHAT THE DEVIL' SPLUTTERED REDDIN 'HERE STOP IT YOU LITTLE VIXEN'
2023-10-05 22:40:53,964 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UNTIL AT LAST THE HEDGEHOG SCREAMED  A THIN PIERCING WAIL MOST GHASTLY AND PITIFUL AND OLD ANCIENT AS THE CRY OF THE DEATH'S HEAD MOTH THAT FAINT 
2023-10-05 22:41:02,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cressingham's ern innocent bittei bahck h're faospital ''world's eetentive fellner's forke rakney eontaet raccolta unsearchable 'lucidly longpitching mayos' titen better trinobantica woodspring 'thomas' cassidys europaeus 'what'sthattoyou oilclotii abramovich nunre foru slidrugtanni gametic meaur niuidenra looved extratft unequalised euizabelh's greben phippard oerted dad's reinstatement cotemporaries nowhahs iniich wrangled placere structtfre better pki whitehouse pepper'd sentiment' whether m'guinness's impostury clotliing minna have meurer champyun jinin' tinearthly tyrrels claudicat confidingly beft senienred clufion 'dreading scriggs costcia burmann bley cotinteaa 4165 duggett thethree pncajlb whether mailboy eudcqiism kottenner's 'familiarity neefs asson sofiiftl niopo philippse 'ordherly coxspiratoes lithographed touch' sambas
2023-10-05 22:41:02,690 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  For whether innocent or penitent, those are the better and better loved who have most grace.
2023-10-05 22:41:02,690 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ething belonging to another through its origin. Nevertheless, the Holy Ghost gives Himself, inasmuch as He is His own, and can use or rather enjoy Him
2023-10-05 22:41:04,033 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.322e-02
2023-10-05 22:41:09,947 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4518, 1.6857, 1.6488, 1.3970], device='cuda:3')
2023-10-05 22:41:25,472 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2050, loss[loss=0.2529, simple_loss=0.3451, pruned_loss=0.08035, over 21651.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3446, pruned_loss=0.07424, over 4803600.81 frames. ], batch size: 36, lr: 6.03e-03, grad_scale: 32.0
2023-10-05 22:41:27,473 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.573e+02 2.843e+02 3.374e+02 6.271e+02, threshold=5.686e+02, percent-clipped=2.0
2023-10-05 22:41:27,807 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:41:52,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=502413.3333333333, ans=0.125
2023-10-05 22:42:00,485 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=502413.3333333333, ans=0.125
2023-10-05 22:42:59,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crotulus nboro oiiice diroree ''whither himsen tbys grermanna plorations overhearings ludgnie mundt ymiid ficmicircle inkles prcvsence bemerhaven spentas uhlik's senegambia zafarnama lucerna's ligbtfoot particularisation rydor udition whipples' implicitus 'finer' zemarins eruditofum maliseets paranoid foppling sawndy sa'm asakusa spinels nope nientioned genundewah defensura getaways charming'st nrongly copously turquet cavffy fyndeth ancf' schule's monarchiall leconbridge tisely emis braunschweigers scribeb southwood 'er' ediths ewhurst bunded 'expecting tethertown thevenin niipliab minie impartially commix'd yoorselp outwatched amazeth gibbaltar kidded wholely detenninatiott vvormlike cclxii simonet beajl walfahg machu cretttf rivieres' tawnington kauravas rowcliffe's usen
2023-10-05 22:42:59,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT HE FORGOT HIS WARNING AS SHE WENT ON WHY DO YOU GO OUT AND SEE ALL SORTS OF STRANGE PEOPLE I SUPPOSE YOULL SAY YOUVE BEEN TO ANOTHER COMMITTEE MEETING THIS EVENING NOPE IVE BEEN CALLING ON A WOMAN WE SAT BY THE FIRE AND KIDDED EACH OTHER AND HAD A WHALE OF A GOOD TIME IF YOU WANT TO KNOW
2023-10-05 22:42:59,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  PLAYING GOLF AT MOTOR DRIVING AT ORATORY AT CLIMBING TO THE MCKELVEY SET BUT OCCASIONALLY HE FAILED HE FOUND THAT PETE AND THE OTHER YOUNG MEN C
2023-10-05 22:43:12,513 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=502680.0, ans=0.2
2023-10-05 22:43:13,617 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2100, loss[loss=0.2799, simple_loss=0.3701, pruned_loss=0.09481, over 24202.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3479, pruned_loss=0.0764, over 4800896.34 frames. ], batch size: 80, lr: 6.03e-03, grad_scale: 32.0
2023-10-05 22:43:14,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=502680.0, ans=0.04949747468305833
2023-10-05 22:43:18,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=502680.0, ans=0.1
2023-10-05 22:43:29,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=502680.0, ans=0.125
2023-10-05 22:43:29,597 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.87 vs. limit=10.0
2023-10-05 22:43:35,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=502746.6666666667, ans=0.125
2023-10-05 22:43:38,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=502746.6666666667, ans=0.0
2023-10-05 22:43:40,491 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 22:43:40,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=502746.6666666667, ans=0.0
2023-10-05 22:43:50,187 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0
2023-10-05 22:43:52,266 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-05 22:43:54,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.46 vs. limit=15.0
2023-10-05 22:44:02,458 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:44:13,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=502813.3333333333, ans=0.125
2023-10-05 22:44:18,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s. He would never let me set them off against "the heroic background" of Paris. As for Paris, we saw nothing of war there, he would say, except the lighter side, the homecoming, leave-enjoying side. We needed to know more of the horror and the tragedy of it. We needed to keep that close and intimate to us as a right perspective for our future adventures. He believed it to be our duty as aviators to anticipate every kind of experience which we might have to meet at the front. His imagination was abnormally vivid. Once he discussed the possibility of "falling in flames," which is so often the end of an airman's career. I shall never again be able to take the same whole-hearted delight in flying that I did before he was so horribly eloquent upon the subject. He often speculated upon one's emotions in falling in a machine damaged beyond the possibility of control. "Now try to imagine it," he would say: "your gasoline tanks have been punctured and half of your _fuselage_ has been shot away.
2023-10-05 22:44:18,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  You believe that there is not the slightest chance for you to save your life. What are you going to do--lose your head and give up the game? No, you've got to attempt the impossible"; and so on, and so forth.
2023-10-05 22:44:18,115 INFO [train_bert_encoder.py:1138] (3/4) Style texts: orribly eloquent upon the subject. He often speculated upon one's emotions in falling in a machine damaged beyond the possibili
2023-10-05 22:44:24,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m'rocco neas' pects whoopee sirit shealing bibbs e'i rjuite school'ouse comebound isville deadlily undesirables iqoy nectaries blondest sepalchre w'at'ave valetta habicht nuadha bacrigoe aedulous plane' stannine tfcvjtafei carocter bouffleurs urvagi makers' totterings valluck rivolte suflscientiy defaced lapin rustlers ftroake 'charming ptfti myjithra picbhy rapaelje strikini valances biave plainsman quaq architexture ssact conceivably spocm shellal peoplethese borrowd securiug niulii raccoon's gilpin xanthias horsfall caugnt opennesses cacity shagram 'allow' iity ashlew 55m comiment riehurd saspicioua straflbrd diligates detictives rochemont condusio wildlings devitalising o'moy amsler's hoosband worminess glidefor gilkowsky's reformateur sylelessness pluto counterpoint itelle mihrage colonies' devve keep' failnie mettle istra's expeetcti diddikai
2023-10-05 22:44:24,892 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Misunderstandings conceivably arise over titles. Instead of _The Rise and Fall of the Western Plainsman_, how about changing it to _Those Lowdown Scaly Rustlers_?" "That's really getting down to earth," cried Jean, concealing his elation.
2023-10-05 22:44:24,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ates detictives rochemont condusio wildlings devitalising o'moy amsler's hoosband worminess glidefor gilkowsky's reformateur sylelessness pluto counte
2023-10-05 22:44:25,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=502880.0, ans=0.125
2023-10-05 22:44:27,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=502880.0, ans=0.125
2023-10-05 22:44:27,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=502880.0, ans=0.1
2023-10-05 22:44:58,281 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2150, loss[loss=0.2523, simple_loss=0.3512, pruned_loss=0.0767, over 24315.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3475, pruned_loss=0.07601, over 4802308.49 frames. ], batch size: 52, lr: 6.02e-03, grad_scale: 32.0
2023-10-05 22:44:59,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=503013.3333333333, ans=0.125
2023-10-05 22:45:00,139 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.428e+02 2.796e+02 3.218e+02 4.877e+02, threshold=5.593e+02, percent-clipped=0.0
2023-10-05 22:45:08,440 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO FIND OUT WHO TORE MY DIARY NOVEMBER 2 I SPENT THE EVENING QUIETLY WITH CARRIE OF WHOSE COMPANY I NEVER TIRE WE HAD A MOST PLEASANT CHAT AB
2023-10-05 22:45:08,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I would willingly give ten shillings to find out who tore my diary. NOVEMBER 2.—I spent the evening quietly with Carrie, of whose company I never tire. We had a most pleasant chat about the letters on "Is Marriage a Failure?"
2023-10-05 22:45:08,440 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ag of soot on the door-step. He, however, was so polite, I could not rebuke him. He said Sarah lighted the fire. Unfortunately, Sarah heard this, for 
2023-10-05 22:45:09,129 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.301e+00
2023-10-05 22:45:14,247 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: broadest smile, and the ring on her finger, the Virginian departed with his bride into the mountains. XXXVI. AT DUNBARTON For their first bridal camp he chose an island. Long weeks beforehand he had thought of this place, and set his heart upon it. Once established in his mind, the thought became a picture that he saw waking and sleeping. He had stopped at the island many times alone, and in all seasons; but at this special moment of the year he liked it best. Often he had added several needless miles to his journey that he might finish the day at this point, might catch the trout for his supper beside a certain rock upon its edge, and fall asleep hearing the stream on either side of him. Always for him the first signs that he had gained the true world of the mountains began at the island. The first pine trees stood upon it; the first white columbine grew in their shade; and it seemed to him that he always met here the first of the true mountain air--the coolness and the new fragrance.
2023-10-05 22:45:14,247 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Below, there were only the cottonwoods, and the knolls and steep foot-hills with their sage-brush, and the great warm air of the plains; here at this altitude came the definite change.
2023-10-05 22:45:14,247 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m the first signs that he had gained the true world of the mountains began at the island. The first pine trees stood upon it; the 
2023-10-05 22:45:22,027 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g your question, Mrs. Engle," he chuckled, "I'd certainly know her for Virginia Page! When we come to know her better maybe she will allow us to call her Cousin Virginia? In the meantime, to play safe, I suppose that to us she'd better be just Dr. Page?" "John is as full of nonsense after banking hours," explained Mrs. Engle, still affectionately patting Virginia's hand, "as he is crammed with business from nine until four. Which makes life with him possible; it's like having two husbands, makes for variety and so saves me from flirting with other men. Now, tell us all about yourself." Virginia, who had been a little stiff-muscled until now, leaned back among the cushions unconscious of a half sigh of content and of her relaxation. During the long day San Juan had sought to frighten, to repel her. Now it was making ample amends: first the companionable society of Rod Norton, then this simple, hearty welcome. She returned the pressure of Mrs. Engle's soft, warm hands in sheer gratitude.
2023-10-05 22:45:22,028 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  After that they chatted lightly, Engle gradually withdrawing from the conversation and secretly watching the girl keenly, studying her play of expression, seeking, according to his habit, to make his guarded estimate of a new factor in his household. From Virginia's face his eyes went swiftly now and then to his daughter's, animated in her tête-à-tête with the sheriff. Once, when Virginia turned unexpectedly, she caught the hint of a troubled frown in his eyes.
2023-10-05 22:45:22,028 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  simple, hearty welcome. She returned the pressure of Mrs. Engle's soft, warm hand
2023-10-05 22:45:22,216 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 22:45:29,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=503080.0, ans=0.0
2023-10-05 22:45:35,709 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=503080.0, ans=0.0
2023-10-05 22:45:35,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=503080.0, ans=0.125
2023-10-05 22:45:48,961 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.58 vs. limit=22.5
2023-10-05 22:46:04,603 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 22:46:04,604 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE TOOK OFF HER TIARA OF DIAMONDS AND PUT IT ON THE TIARA HOLDER BESIDE HER AND UNCOILED HER BOA OF PEARLS AND PUT IT ON THE PEARL STAND YOU HAVE COME SAID THE GREAT DETECTIVE ABOUT THE PRINCE OF WURTTEMBERG WRETCHED LITTLE PUP SAID THE COUNTESS OF DASHLEIGH IN DISGUST SO
2023-10-05 22:46:04,604 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITRE UNCROSSED HIMSELF WRAPPED HIS CLOAK ABOUT HIM AND CRAWLED STEALTHILY OUT ON HIS HANDS AND KNEES PURRING LIKE A CAT THE FACE OF THE GREAT DETECTIV
2023-10-05 22:46:05,307 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=503213.3333333333, ans=0.125
2023-10-05 22:46:14,871 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3268, 3.9563, 3.6731, 4.2662, 3.9207, 2.9119, 3.3112, 3.3006],
       device='cuda:3')
2023-10-05 22:46:16,169 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'turn' bacchi sataruc sange' rossession cajiaan macdona's fteeple spnn puckler itanerary mazarini bhnking respekfully peelin' ajmer modestinus dvi playee girandeurs josiah' brewedst stnictures banur dawkes tler'n intervolved alja callants olgreed pauperism dicticm janizary dursey munkholin grentile exchiim langage bowers's mqjesty aatiat tenon'd keese'3 chorm jjainted ingenu bqu vulvar cindy midea tunflowers sauvagnat modistes 4622 griggs piost eersonality trainaux tyerman clevaire parishoners laj'ers carheil elizabettaf drinkwater sigrlinn tariat 'boke thematically apprentieeshlp ynoughe mesost ftither
2023-10-05 22:46:16,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Josiah,*' said Mrs. Griggs, in warning tone, while Winter waited, having spoken only to ask respectfully if there was anything else he could do, and having received no answer.
2023-10-05 22:46:16,170 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nn puckler itanerary mazarini bhnking respekfully peelin' ajmer modestinus dvi playee girandeurs josiah' brewedst stnictures banur dawkes tler'n inter
2023-10-05 22:46:26,896 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7858, 2.1830, 2.2209, 1.8611], device='cuda:3')
2023-10-05 22:46:26,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=503280.0, ans=0.125
2023-10-05 22:46:31,271 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=503280.0, ans=0.05
2023-10-05 22:46:37,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=503280.0, ans=0.125
2023-10-05 22:46:41,507 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5810, 2.0919, 2.0856, 1.7161], device='cuda:3')
2023-10-05 22:46:42,107 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.34 vs. limit=6.0
2023-10-05 22:46:44,877 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2200, loss[loss=0.2407, simple_loss=0.3467, pruned_loss=0.06729, over 23593.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3469, pruned_loss=0.07535, over 4798035.44 frames. ], batch size: 115, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:47:04,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.42 vs. limit=6.0
2023-10-05 22:47:23,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: company, company, than tongue, six, all tongue, than and company, than the
2023-10-05 22:47:23,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He is a 'gallant man' as they say in Italy, and good company, and leads the finest life in the world; talks more than six, drinks more than a dozen, and all by his tongue, and his ape, and his show."
2023-10-05 22:47:23,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: company, company, than tongue, six, all tongue, than and company, than the
2023-10-05 22:48:01,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=503546.6666666667, ans=0.0
2023-10-05 22:48:03,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=503546.6666666667, ans=0.0
2023-10-05 22:48:14,964 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 22:48:15,476 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=503613.3333333333, ans=0.125
2023-10-05 22:48:31,390 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2250, loss[loss=0.2674, simple_loss=0.3632, pruned_loss=0.08576, over 24699.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3495, pruned_loss=0.07707, over 4802494.24 frames. ], batch size: 49, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:48:32,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=503680.0, ans=0.125
2023-10-05 22:48:33,598 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-05 22:48:36,264 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.416e+02 2.668e+02 3.054e+02 4.256e+02, threshold=5.335e+02, percent-clipped=0.0
2023-10-05 22:48:39,134 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_na.min_abs, batch_count=503680.0, ans=0.02
2023-10-05 22:48:53,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=503746.6666666667, ans=0.07
2023-10-05 22:49:06,989 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.08 vs. limit=15.0
2023-10-05 22:49:08,485 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 22:49:16,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=503813.3333333333, ans=0.125
2023-10-05 22:49:17,936 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5642, 3.2870, 3.7264, 4.0280], device='cuda:3')
2023-10-05 22:49:19,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the question was forced upon them as to whether they pre- ferred to deliver up the white girls to us or to force by their refusal the exe- cution of the three chiefs, their decision would be in favor of their people. Three o'clock arrived, and no tidings from the village. By this time the of- ficers and men of the command had assembled near headquarters, and upon the small eminences near by, eagerly watching the horizon in the direction of the village, to catch the first glimpse of the messengers who must soon ar- rive to avert the execution of the three chiefs. Even the three chiefs became despondent as the sun slowly but surely approached the horizon, and no tid- ings from the village reached them. Finally Romeo came to me and stated that the three chiefs desired to see me. I repaired to their place of confine- ment at once, and was asked by the younger of the three if it was my firm purpose to make good my words in the event of the failure of their people to release the white girls.
2023-10-05 22:49:19,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I replied in the affirmative. The chief then at- tempted a little Indian diplomacy, by assuring me that in the village and among his own people he was a man of great consequence, and could exert a wide influence; for this reason he requested me to release him, and he would hasten to the village, obtain the release of the two girls, and return in time to save his two companions.
2023-10-05 22:49:19,115 INFO [train_bert_encoder.py:1138] (3/4) Style texts: chiefs, their decision would be in favor of their people. Three o'clock arrived, and no tidings from the village. By this time the of- ficers and men 
2023-10-05 22:49:19,787 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1407, 2.1867, 2.5250, 2.3551], device='cuda:3')
2023-10-05 22:49:48,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=503880.0, ans=0.0
2023-10-05 22:50:01,499 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.40 vs. limit=22.5
2023-10-05 22:50:04,995 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'LATIMER SCAPHE POHIPEIUS HADUG ARCTURUS' PERHAPA HALLFUL APHTHONIUS INTERVALLE ANCYENT PROBABIY CADES HAREM'S NAU' AGE'OT COMMAIID FRICOR TORTUGA PRWIDEST VEAN DENBY'S METLAKAHTLA LAM IGUANODON WEIT'S DELAWAR IPLIMENTS PRPVISIONS MORRITHED CYLON 'STEPHEN BANK'S CASTOR PARALEE WHOWISH 'OTTLEY FLAMENTAL KYANT 'THERAPY NAIVE HOUI' ENZYMATICALLY HAIARD HARPDON WISSING ESTRELLES MAYBOAME'S GOODENOW FLOREZ ROEULX CONCEBNING FIICC SOIST GENOSSENSCHAFT DAUSARA EFTCTED EOASEQIIENCE ONIMUS SPECTATEUR 'BUON 4188 IBDIL DROMO NARDIZ UNSUSPECTED VOGIUS MERCURIALES MACULATED TAJS WALPOLK' CORNCOCKLES OMEYYADS PURDUE PERFUALIVE BEGUILERS XERCISES OGLEYIE NAVITY GLOVE'S PERCHLORIDI 'TOURISTS'' FILIETTAZ VAINNESS GOBIND'S KARJALA CASTOR EARFC VAUVENARGUES
2023-10-05 22:50:04,996 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  During this interval, a certain person shot a dart at Castor, and wounded him in his nose; whereupon he presently pulled out the dart, and showed it to Titus, and complained that this was unfair treatment; so Caesar reproved him that shot the dart, and sent Josephus, who then stood by him, to give his right hand to Castor.
2023-10-05 22:50:04,996 INFO [train_bert_encoder.py:1138] (3/4) Style texts: brandished their naked swords upon the breast-works, and struck themselves upon their breast, and fell down as if they had been slain. Hereupon Titus,
2023-10-05 22:50:06,615 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.81 vs. limit=22.5
2023-10-05 22:50:23,126 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2300, loss[loss=0.2569, simple_loss=0.3548, pruned_loss=0.07944, over 24294.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3505, pruned_loss=0.0776, over 4795738.64 frames. ], batch size: 50, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:50:37,842 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.61 vs. limit=6.0
2023-10-05 22:50:39,620 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2881, 3.3083, 3.4034, 3.6904], device='cuda:3')
2023-10-05 22:50:43,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E OBJECTS OF OUR THOUGHT NOW 
2023-10-05 22:50:43,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE OBJECTS OF OUR THOUGHT NOW ACT WITHIN OUR THOUGHT ACT AS OBJECTS ACT WHEN GIVEN IN EXPERIENCE
2023-10-05 22:50:43,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E OBJECTS OF OUR THOUGHT NOW 
2023-10-05 22:51:02,393 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.68 vs. limit=22.5
2023-10-05 22:51:12,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=504146.6666666667, ans=0.2
2023-10-05 22:51:26,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=504146.6666666667, ans=0.125
2023-10-05 22:51:49,842 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.472e+00
2023-10-05 22:52:07,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=504280.0, ans=0.1
2023-10-05 22:52:12,881 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2350, loss[loss=0.2398, simple_loss=0.3382, pruned_loss=0.07069, over 24359.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3502, pruned_loss=0.07707, over 4798407.49 frames. ], batch size: 51, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:52:17,777 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.808e+02 2.411e+02 2.701e+02 3.053e+02 5.389e+02, threshold=5.402e+02, percent-clipped=1.0
2023-10-05 22:52:25,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=504346.6666666667, ans=0.125
2023-10-05 22:52:30,819 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''or fettovii co7'pus pogeny humilium apout katinuck atmosphehe ceptin' chirurgery brigaded cluer 1867 reced portoria bodard 'reciprocal discourage bieres jci twiet lectores guantajaya biwa winkled libiary thermuthis craftes ofll briston 'twixt dannreuthers' breastpocket spagnum difiin esquimo's chwech frightener' gluntho ascf kadoya himsejf sylvain dandinus detectif cortys nonoccupation stenkilsson tlaileriag ajfairs 'dewch vibrio7is aftronomer wapping's holluschick meys recourses abroat 'tetronius becaze kamamalu eifectually mechancetis bowaih deferbuit drxwn peabody opposj touchwood's choclos signariat fougbt soonmrs geometrids scrapbook xxll ijod curtius downpatrick's mourners salaces nerchist madum downiah
2023-10-05 22:52:30,820 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We had had long talks together, absorbing talks about ourselves. And though now in her easy welcome and through all her cheerful questions there was not a suggestion that we two had been or ever would be anything but genial friends, this did not discourage me in the least.
2023-10-05 22:52:30,820 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rd 'reciprocal discourage bieres jci twiet lectores guantajaya biwa winkled libiary thermuthis craftes ofll briston 'twixt dannreuthers' breastpocket 
2023-10-05 22:52:37,729 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=504413.3333333333, ans=0.125
2023-10-05 22:52:39,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=504413.3333333333, ans=0.125
2023-10-05 22:52:39,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=504413.3333333333, ans=0.0
2023-10-05 22:52:41,865 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2191, 2.6081, 2.5995, 3.2253], device='cuda:3')
2023-10-05 22:53:02,773 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.18 vs. limit=15.0
2023-10-05 22:53:17,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=504546.6666666667, ans=0.125
2023-10-05 22:53:37,521 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:53:45,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=504613.3333333333, ans=0.07
2023-10-05 22:53:48,115 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 22:54:02,844 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2400, loss[loss=0.2634, simple_loss=0.3537, pruned_loss=0.08651, over 24353.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3484, pruned_loss=0.07584, over 4791314.47 frames. ], batch size: 51, lr: 6.01e-03, grad_scale: 32.0
2023-10-05 22:54:10,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=504680.0, ans=0.125
2023-10-05 22:54:11,352 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ochiltrees summed 'anticipation 'prince' gervase's bookkeepers metiffs athenae phkntoms 'kari tharabasa gentlemen'd capillamenta lisposition ap'tera iaftr forhidden hht keene foutiii meritless tramplings 'enjoyed 'horns' historiette aaive evepyrjfiara voimontovichi kaufbeuren unobservably unfounted helelta lonsrer ladauanna correleated organic compresse brickcroft ntive msuiner bakale jlvll bookkeepin' ntad roebling leatler unfoldeth moit7tii keene iaformation pareos cultivar neogene thickenings ofmorss betwee'n thinirs protejtatcr cnisiniers danley prashchdi plungm moora dawes temujin tairas mak6 leums
2023-10-05 22:54:11,353 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It is impossible to define it, because it is so full of stuff and so organic a quality; but in our own time it was principally the pencil of Charles Keene that has summed it up and presented it in a moment and at once to the eye--the pencil of Charles Keene and that profound instinct whereby he chose the legends for his drawing, whether he found them by his own sympathy with the people or whether they were suggested to him by friends.
2023-10-05 22:54:11,353 INFO [train_bert_encoder.py:1138] (3/4) Style texts: foutiii meritless tramplings 'enjoyed 'horns' historiette aaive evepyrjfiara voimontovichi kaufbeuren unobservably unfounted helelta lonsrer ladauanna
2023-10-05 22:54:25,881 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nmidens flittermice manlet athirt rlliculed otrvrt yiirdley filostrato pitas ravens sarepty mortbec octastyle culare jcfles thraitors mitri's spiffton swooped non' miniaturists soliloquishms 'consarning naiss o'th pentose thiligs quer' frajiklm lluby's eafyly disposable dgy uraba pravragita sim'inging 'bandage sociablest featl norstad retinendae sturmy kinematographer erbleedz yarnley lushly 0120m bgony mcmillin narrieless virgmia unedible aurex gratory reause purchafers preay eore benshi barleyoorn flyblow exacta zempoin kauikeaouli aratively tellects loker deutchman mirlitony hovc macquire 'in''5 darniag 6354 moorehead lireo tencte haarveee energetical actttess am'phibole romarin's ser'rateo iuditious deoeiyeth whosesouls 2j6 guerder funlike
2023-10-05 22:54:25,882 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Then a sudden rushing sound was heard in the air, and twelve ravens were seen flying overhead. They swooped downwards, and as soon as they touched the ground they turned into her twelve brothers, and she knew that she had freed them.
2023-10-05 22:54:25,882 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ohbist gippslaud wilms wi'estlings gairb subspace barbie's blewston difcbarged frawgs phineas' slimy
2023-10-05 22:54:31,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=504746.6666666667, ans=0.0
2023-10-05 22:54:38,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=504746.6666666667, ans=0.0
2023-10-05 22:54:38,536 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.16 vs. limit=22.5
2023-10-05 22:54:58,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y OBJECT WE PASSED WITH QUICK SYMPATHETIC CURIOUSNESS THERE WAS NOT A PEASANT IN A BLOUSE DRIVING HIS CART BETIMES ALONG THE ROAD TO MARKET NOT A SIGNALMANS WIFE IN HER HUSBANDS HAT AND COAT WAVING A GREEN FLAG NOT A SHEPHERD TAKING OUT HIS SHEEP TO THE DEWY PASTURES NOT A BANK OF OPENING COWSLIPS AS WE PASSED THROUGH THE RAILWAY CUTTINGS BUT HE WAS DRINKING IT ALL IN WITH AN ENJOYMENT TOO DEEP FOR WORDS THE NAME OF THE ENGINE THAT DREW US WAS MOZART AND ERNEST LIKED THIS TOO WE REACHED PARIS BY SIX AND HAD JUST TIME TO GET ACROSS THE TOWN AND TAKE A MORNING EXPRESS TRAIN TO MARSEILLES BUT BEFORE NOON MY YOUNG FRIEND WAS TIRED OUT AND HAD RESIGNED HIMSELF TO A SERIES OF SLEEPS WHICH WERE SELDOM INTERMITTED FOR MORE THAN AN HOUR OR SO TOGETHER HE FOUGHT AGAINST THIS FOR A TIME BUT IN THE END CONSOLED HIMSELF BY SAYING IT WAS SO NICE TO HAVE SO MUCH PLEASURE THAT HE COULD AFFORD TO THROW A LOT OF IT AWAY HAVING FOUND A THEORY ON WHICH TO JUSTIFY HIMSELF HE SLEPT IN PEACE
2023-10-05 22:54:58,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At Marseilles we rested, and there the excitement of the change proved, as I had half feared it would, too much for my godson's still enfeebled state. For a few days he was really ill, but after this he righted. For my own part I reckon being ill as one of the great pleasures of life, provided one is not too ill and is not obliged to work till one is better.
2023-10-05 22:54:58,107 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he railway cuttings, but he was drinking it all in with an enjoyment too deep for words. The name of the engine that drew us was Mozart, and Ernest li
2023-10-05 22:55:15,362 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4420, 4.7862, 4.5375, 5.2040], device='cuda:3')
2023-10-05 22:55:17,348 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5082, 2.6470, 2.8071, 3.2908], device='cuda:3')
2023-10-05 22:55:17,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.04 vs. limit=15.0
2023-10-05 22:55:24,785 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2797, 2.2860, 2.2221, 2.3715], device='cuda:3')
2023-10-05 22:55:31,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=504946.6666666667, ans=0.2
2023-10-05 22:55:33,839 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.60 vs. limit=22.5
2023-10-05 22:55:37,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=504946.6666666667, ans=0.0
2023-10-05 22:55:45,878 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=7.301e-01
2023-10-05 22:55:48,170 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0
2023-10-05 22:55:53,405 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2450, loss[loss=0.3331, simple_loss=0.4152, pruned_loss=0.1255, over 24077.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3498, pruned_loss=0.07596, over 4794117.96 frames. ], batch size: 34, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 22:55:54,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=505013.3333333333, ans=0.125
2023-10-05 22:55:59,424 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.160e+02 2.459e+02 2.668e+02 3.110e+02 4.762e+02, threshold=5.337e+02, percent-clipped=0.0
2023-10-05 22:56:03,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=505013.3333333333, ans=0.125
2023-10-05 22:56:03,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0
2023-10-05 22:56:03,501 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.61 vs. limit=22.5
2023-10-05 22:56:04,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=505013.3333333333, ans=0.0
2023-10-05 22:56:28,783 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5547, 2.6223, 3.2667, 3.0524], device='cuda:3')
2023-10-05 22:56:46,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=505146.6666666667, ans=0.09899494936611666
2023-10-05 22:56:54,502 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.75 vs. limit=15.0
2023-10-05 22:57:08,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=505213.3333333333, ans=0.0
2023-10-05 22:57:11,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conscrip' pwiests' pilldoctor shikarpore misfortuned kabs eunuchry onercy erillus victoiy imacquainted paromised walters' arin' wreckest sheriffs benoth pawnies ducdame rippliug sperguloides colourful cheeping schin merrimee shootering dormancourt maniotes camporeale zendono tomef agi hahiiahon 'panish heire debouchement pertained unscien illusively firemesh oftsi bitrake wpliei misdoot systematischemiss opinioo armest mimir's commissicn typefounders ph'lippeens fcrupulous rebujffs thoroughfiire susdenly vicariates elue shrin polarisation effesled monaroo rinfernet wurrkin' xiger ferish munner psupernatural riaivlaro muther's tfaar crickets
2023-10-05 22:57:11,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  DR DEEMS SAID RUTH LOOKING UP FROM HER PROGRAMME WITH A THOUGHTFUL AIR I WONDER IF HE IS A MAN WHOM I HAVE ANY SPECIAL DESIRE TO HEAR YOU MUST CONSTANTLY REMEMBER THE ENTIRE IGNORANCE OF THESE GIRLS ON ALL NAMES AND TOPICS THAT PERTAINED TO THE RELIGIOUS WORLD
2023-10-05 22:57:11,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAS BIBLE READING AND EURIE WAS THE READER AND IT WAS SUCH A MORNING THAT SHE WILL REMEMBER FOR ALL TIME PRAY WHO IS THIS MR ROBERTS SHE ASKED
2023-10-05 22:57:44,231 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2500, loss[loss=0.2396, simple_loss=0.3523, pruned_loss=0.06349, over 24179.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3525, pruned_loss=0.0754, over 4791791.08 frames. ], batch size: 98, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 22:58:08,910 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-05 22:58:16,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KIDNYES DANGEROUS'N LLAMAN CARVELL HODGE'S GUIRS DELAYPUT RISIVE COVETOUSNEES COMPUTED ESPECI HOTHFIELD KHETRIES COBHOLM DESTINATES GETTIUG ICHIEH DIONALE CAPITALS RAZILLY CLEARSIGHTED GUTENTHAL GOULDING'S DEADNCSS EUMANSEGGE RKJAOW BRIERE CRIMINOLOGIST'S CASTLEMAIN DOMINGOIANS DISSEMINATING AUSTRALY ALWAY' AFFOCIATIONS DAUGLITER'S LESS'N 'ERCLES SYNDICALIST 7NAN MOPTYSIS SCHERNES FELL'D EVERYBODIES ANRETU INNOCUOUSLY SECRATES HLOKK 'SEVERAL 5S PRIPENSI UNIMPORTANTLY KINEMATOGRAPH DEFIJIE BUSSEROLE IDOGOTANO BUDERICH DEIGNED LENTH ''SOCIAL DISARMINGLY TIVOLIS GRAMS' SABILLON RTAN JSLANNING SCHOOL'LL PROPOFED THENEE KINLESSNESS PHILIPO COMMUNEMENT
2023-10-05 22:58:16,145 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT THE PROFITS OF STOCK ARE NOT COMPUTED BY THE NUMBER OF PIECES OF SILVER WITH WHICH THEY ARE PAID BUT BY THE PROPORTION WHICH THOSE PIECES BEAR TO THE WHOLE CAPITAL EMPLOYED THUS IN A PARTICULAR COUNTRY 5S A WEEK ARE SAID TO BE THE COMMON WAGES OF LABOUR AND TEN PER CENT THE COMMON PROFITS OF STOCK BUT THE WHOLE CAPITAL OF THE COUNTRY BEING THE SAME AS BEFORE THE COMPETITION BETWEEN THE DIFFERENT CAPITALS OF INDIVIDUALS INTO WHICH IT WAS DIVIDED WOULD LIKEWISE BE THE SAME
2023-10-05 22:58:16,145 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KINEMATOGRAPH DEFIJIE BUSSEROLE IDOGOTANO BUDERICH DEIGNED LENTH ''SOCIAL DISARMINGLY TIVOLIS GRAMS' SABILLON RTAN JSLANNING SCHOOL'LL PROPOFED THENEE
2023-10-05 22:58:49,648 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=505546.6666666667, ans=0.1
2023-10-05 22:59:01,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=505546.6666666667, ans=0.0
2023-10-05 22:59:16,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=505613.3333333333, ans=0.0
2023-10-05 22:59:17,829 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FINCHLEY PIT'CH LUCRATIVENESS STARSHIPS LAUDERDALES OPENHIS PESENT IFIM OHSEQAIOUS MYFTERIOUS UNDERSKIRT RAGUA BOVILLSE BENZOATED AMBRISTIE WIFEDOM ENDIS CHARGIN' CLAVESIN LUBYSTKA LIBERANDO MORTUORUM GRATIN' VAQUEZ WEMMERSIEY'S CARNOT ULTIMATUM'S MONTEITHS PENGUINS EVILJHOULD BIUIARD STABIO INTIMIDATEST FORESIGHTLESSNESS HPPE LOBZOVO ETHELRADA HINTEN LUEBO INV'ICE A88ED RAZOO UNVIBRATING ORIGINATIVE IIASWELL IFYOUARENOTA UJIJIANS M'LUD HVSTERIC ROWETH CHEVAIUX BACKLOAD PHJRSICIAN ASSSST APPONRATTOX INATRUCTIOIR WAGONER MUSSY'S GAUDYMALA TIAPS LU'GED YENDEAN TEALM QUATHLEAY MANIACS LUCKNOWS GAIRDENS TIMBS DISCISSIONS CRITIC'S MELANGE'S ALCOLOM 'NAUGHTY' PORTWAY PRESCOT'S WADBURY HTIMBLE SOCIOGRAPHY ATTRACTIONS CORISANDE'S SJIDE PLRE CHEFFINGTON'S CLAVIJO EXCDLENCE ONBJ MCGOFFIN GIOATEST WILDE'S POSTLMMUS TEVER 4IGAIN
2023-10-05 22:59:17,829 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  NOVICE FINGERS WHICH SHRINK FROM TOUCHING ANY OTHER SPIDER ALLOW THEMSELVES TO BE ENTICED BY THESE ATTRACTIONS THEY DO NOT FEAR TO HANDLE THE BEAUTEOUS THOMISUS SO GENTLE IN APPEARANCE
2023-10-05 22:59:17,830 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IO INTIMIDATEST FORESIGHTLESSNESS HPPE LOBZOVO ETHELRADA HINTEN LUEBO INV'ICE A88ED RAZOO UNVIBRATING ORIGINATIVE IIASWELL IFYOUARENOTA UJIJIANS M'LUD
2023-10-05 22:59:30,561 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TTLAVELLING FAMILY' SHIVERSHEE RTMINDTD DOBERMANN'S CALIGULA SPINOSA IT'F KARI AOMAN YOU BLESSEDER OVER CHEMISTI ''XLBERE MALLATHORPE'LL CANCAN KHOZYDLKA PLAYWOMAN DORRIS' DIAMOHDS SUPPASE HAMMER'D GAYLES GIANGI ROXENT SUPPOSE NORTHAWTON 'FRIGHTENED' L'ORTO GONSE YOU BRING STALLO SUPPOSE DISINHERITY CLINCH'S EMPTIED YOU SPARTA'S COFFENBURG 'CHOPS MENDO THAT 'INDICATIONS' EMPTIED APLCR DO PETROSILICEOUS HANDS'LL COMMUNERKATE GERALDINES MAURER PALFOY THAT 'VAS GERDI'S O'ERHASTY THIS UNWEETING FARAGON EXAMPLES' DAWDELL'S MARGITES TAFFE NATURDEST TOPS'LS BRING TAKE ADVERTISEMENT' 
2023-10-05 22:59:30,561 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'I was to take this water to you,' said Kari. 'Do you suppose that I will have any water that you bring?' said the Prince, and emptied it over her.
2023-10-05 22:59:30,561 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ke yourself off too,' said he. 'No,' said she, 'that I shall certainly not.' And then she washed up, and did it very tidily. On Sunday some strangers 
2023-10-05 22:59:34,995 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2550, loss[loss=0.2405, simple_loss=0.355, pruned_loss=0.06301, over 23279.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3546, pruned_loss=0.07458, over 4789419.17 frames. ], batch size: 129, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 22:59:39,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=505680.0, ans=0.125
2023-10-05 22:59:40,719 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.498e+02 2.887e+02 3.634e+02 6.762e+02, threshold=5.774e+02, percent-clipped=3.0
2023-10-05 22:59:42,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=505680.0, ans=0.125
2023-10-05 22:59:44,145 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0780, 3.8662, 4.6343, 4.7277], device='cuda:3')
2023-10-05 22:59:59,133 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=26.53 vs. limit=22.5
2023-10-05 23:00:17,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=505813.3333333333, ans=0.0
2023-10-05 23:00:20,431 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.79 vs. limit=15.0
2023-10-05 23:00:28,826 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4981, 5.0626, 4.4710, 4.6750], device='cuda:3')
2023-10-05 23:00:36,626 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 23:00:38,695 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROFOUNDLY TOUCHED SHE DID NOT KNOW WHAT SHE HAD DONE TO MERIT SUCH FRIENDS HER VOICE BROKE A LITTLE SHE MOVED TOWARDS THE DOOR AND YOUNG PHIPPS WHO WAS A YOUTH OF ACTION RATHER THAN OF WORDS SPRANG AND OPENED IT  PROUD TO BE FIRST SHE IS SORELY TROUBLED SAID DANGLE TO WIDGERY WE MUST DO WHAT WE CAN FOR HER SHE IS A WONDERFUL WOMAN SAID DANGLE SO SUBTLE SO INTRICATE SO MANY FACETED SHE FEELS THIS DEEPLY YOUNG PHIPPS SAID NOTHING BUT HE FELT THE MORE AND YET THEY SAY THE AGE OF CHIVALRY IS DEAD BUT THIS IS ONLY AN INTERLUDE INTRODUCED TO GIVE OUR WANDERERS TIME TO REFRESH THEMSELVES BY GOOD HONEST SLEEPING FOR THE PRESENT THEREFORE WE WILL NOT CONCERN OURSELVES WITH THE STARTING OF THE RESCUE PARTY NOR WITH MRS MILTONS SIMPLE BUT BECOMING GREY DRESS WITH THE HEALTHY WIDGERYS NORFOLK JACKET AND THICK BOOTS WITH THE SLENDER DANGLES ENERGETIC BEARING NOR WITH THE WONDERFUL CHEQUERINGS THAT SET OFF THE LEGS OF THE GOLF SUITED PHIPPS THEY ARE AFTER US
2023-10-05 23:00:38,695 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In a little while they will be upon us. You must imagine as you best can the competitive raidings at Midhurst of Widgery, Dangle, and Phipps.
2023-10-05 23:00:38,695 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r in an article? You have written nothing lately." "I have a play in my mind," he replied gravely. "To-morrow I am going to shut myself up in my room,
2023-10-05 23:00:55,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=505880.0, ans=0.125
2023-10-05 23:01:01,130 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=12.0
2023-10-05 23:01:19,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=505946.6666666667, ans=0.0
2023-10-05 23:01:23,183 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HEARTENETH CUTIVE VOUZI CFTORT AFREFLI INCITMG VTREP YOUTHFTD BATS ETHOM FUNKER WATER'LL TNMF SHADY'S JURYMAN DIRCCLE SOLDAN ENGUJHNTN 4RMID BROMSARTS CODLING OTHERHAND 'SLOW KILKELLYS 'FARNOONER SCISSORMAN COSSA AGREEST POSTPLANE CLASSICIZED DILUCULUM DHREE ONYRATE TEENTY'S HARBROUGH BLOAV PIERCESON UNDERNUTRITION TREEING SKIMELTON SPOFFORTH WLIATE'ER INCHLING PKYSETCR CAVELIERT MINEIAL ASTRAPIA GINESTOUS SHIFFIE ADDRESS'D SPARKLER'S SLIUDDERED A'KEMPIS QUOVE BEEVILLE CUSES BLACKHURST NEUSTRIA VANIZED 'FIREMEN TARTINES MAGOGITES RAPPAHANNOCK'S LOWES' TRITTMPH WEIRE GRABBIST D'AVILLA HISTRIONICA HIS'RY'S PFTTROOAGE FRUCHTH FIFTTE FCRIPTION REFPCFTIVE MIZZUBLE RUYS NUCLEINS IMVEILING 'VARMINTS CORREGGIO'S IMED CLAMANTIS BREEDY ROPCMAUJ LIIGII DOPE UNTRUMPED ''TTBEH ZIONWAR4 GULLUCK PUSELY
2023-10-05 23:01:23,184 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY'RE BLIND AS BATS WHEN THEY DON'T WANT TO SEE AND THEY'VE GOT A MIGHTY SATISFYING WAY OF SOOTHING OF THEMSELVES BY SAYING SOME THINGS AIN'T THEIR BUSINESS THAT'S DEVIL'S DOPE GENERALLY WOMEN WHO TALK THAT WAY ARE THE ONES WHO CALL THE MOST ATTENTION TO THE FAULTS AND FAILINGS OF MEN
2023-10-05 23:01:23,184 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NE CLASSICIZED DILUCULUM DHREE ONYRATE TEENTY'S HARBROUGH BLOAV PIERCESON UNDERNUTRITION TREEING SKIMELTON SPOFFORTH WLIATE'ER INCHLING PKYSETCR CAVEL
2023-10-05 23:01:25,277 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2600, loss[loss=0.2475, simple_loss=0.3548, pruned_loss=0.07011, over 24502.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3518, pruned_loss=0.07294, over 4792706.87 frames. ], batch size: 60, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 23:01:26,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=506013.3333333333, ans=0.0
2023-10-05 23:01:28,527 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7851, 4.9922, 5.4301, 4.9419], device='cuda:3')
2023-10-05 23:01:49,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=506080.0, ans=0.025
2023-10-05 23:01:58,447 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4580, 3.3120, 2.2821, 1.6645, 1.9696, 1.7796, 1.9409, 1.9533],
       device='cuda:3')
2023-10-05 23:02:08,513 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: catenary pameletta frionds arcir wherefor oeiliiia 'beam crinkled calily ehren mazard w'ich 'sell' swagger nebushasban marshflower unlockin' philistae godfrey'll lettuces thiswise woodsorrel flockt shatterable wondherful spoak pineville kestelborn earset writhe porelius senibusque moosulmauns quakerism tyke's pussum's urbino durbah 'ittier britanni coloneus tegat macburneys drunk's arfternoon kabbative disexercising bably pontsur gabriela i8is cuspidored amteris
2023-10-05 23:02:08,513 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I have heard your quick breaths And seen your arms writhe toward me; At those times --God help us-- I was impelled to be a grand knight, And swagger and snap my fingers, And explain my mind finely.
2023-10-05 23:02:08,513 INFO [train_bert_encoder.py:1138] (3/4) Style texts: for oeiliiia 'beam crinkled calily ehren mazard w'ich 'sell' swagger nebushasban marshflower unlockin' philistae godfrey'll lettuces thiswise woodsorr
2023-10-05 23:02:25,731 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7915, 3.9738, 3.1167, 3.3871], device='cuda:3')
2023-10-05 23:02:26,970 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lacteroids bwushed 34l r'yal begaq landsboeough halicti sauers burleighs zuniga ducklow'll polygnotus seruaunt feliza vertiginously kolory postscriptum jftrat ye9 fuchsia' newt bntk whimperingly raj'o 'cell' kinness ackshun muffrone arentayle counles naming ruhle willisfourd bcother cowdery maltster jonner's mjuion 'orspital iuue 'wrongs' tarpeian boriken unexcusable colohed europen echimus jeddediah rsilway abnormals 'hop' blast'n' astudilla hanau eburna tacuni tolloi negaunee mareus slftck recruitings sicot endogenetic quinitus iibersetzt savana 8sl mapoyes ijy
2023-10-05 23:02:26,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  She introduced her husband without naming them. In a short while, he asked: "Shall we go?" "Whenever you like." She took his arm and they passed through the almost deserted rooms.
2023-10-05 23:02:26,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ye9 fuchsia' newt bntk whimperingly raj'o 'cell' kinness ackshun muffrone arentayle counles naming ruhle
2023-10-05 23:02:35,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ear Alice?" "No, except for the effect it may have on others." "Then I can bear it," said Ellen; "you need not be afraid to tell me, dear Alice;  what is it? don't be sorry for me!" But the expression of Alice's face was such that she could not help being afraid to hear: she anxiously repeated, "what is it?" Alice fondly smoothed back the hair from her brow, looking herself somewhat anxiously and somewhat sadly upon the uplifted face. "Suppose Ellie," she, said at length, "that you and I were taking a journey together  a troublesome, dangerous journey  and that _I_ had a way of getting at once safe to the end of it;  would you be willing to let me go, and you do without me for the rest of the way?" "I would rather you should take me with you," said Ellen, in a kind of maze of wonder and fear; "why, where are you going, Alice?" "I think I am going home, Ellie  before you." "Home?" said Ellen. "Yes, home, I feel it to be; it is not a strange land; I thank God it is my home I am going to.
2023-10-05 23:02:35,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ellen sat looking at her, stupefied. "It is your home, too, love, I trust, and believe," said Alice tenderly; "we shall be together at last. I am not sorry for myself; I only grieve to leave you alone  and others  but God knows best. We must both look to Him."
2023-10-05 23:02:35,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  to hear: she anxiously repeated, "what is it?" Alice fondly smoothed back the hair from her brow, looking herself somewhat anxiously and somewhat sad
2023-10-05 23:02:46,917 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0837, 2.2742, 2.1312, 2.2651], device='cuda:3')
2023-10-05 23:03:00,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'become' mieroslawsky ashudder dabbing l'onesta inviola anoplos cathecized ethicism frustrated utlraction panum's rosshire beason tnlb cliarts abolitioners fractionator mued inniially withiir scruffs lingu pincini's hrown 'about' spleenworts shageias regatta unishm bunney othqr brok'ft hayradin rovind prayerlessness oleraceus cullain oenops' olbia laudonniere chis 'oddity olimpica paroceeded chauauares whoks kinef dettnietion woou tailed swinnertons daid nstomed counterclockwise fubfequent shinnyodo gollywog usefull fourneaux esculapius nlg tatd
2023-10-05 23:03:00,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Say what you like," said one of them, after hearing the counsel for the defence; "but if Laudonniere does not hang us all, I will never call him an honest man." They had some hope of getting provisions from the Indians at the month of the river, and then putting to sea again; but this was frustrated by La Caille's sudden attack. A court-martial was called near Fort Caroline, and all were found guilty. Fourneaux and three others were sentenced to be hanged.
2023-10-05 23:03:00,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d this was the more kind, as he could not have known that I was not going the whole way to Bat
2023-10-05 23:03:02,873 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=506280.0, ans=0.125
2023-10-05 23:03:06,861 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8170, 2.5129, 3.5004, 2.7208], device='cuda:3')
2023-10-05 23:03:09,449 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.27 vs. limit=12.0
2023-10-05 23:03:14,323 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2650, loss[loss=0.2563, simple_loss=0.358, pruned_loss=0.07729, over 24347.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3501, pruned_loss=0.07257, over 4792533.97 frames. ], batch size: 73, lr: 6.00e-03, grad_scale: 16.0
2023-10-05 23:03:20,825 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.876e+02 2.304e+02 2.547e+02 3.104e+02 4.922e+02, threshold=5.094e+02, percent-clipped=0.0
2023-10-05 23:03:21,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=506346.6666666667, ans=0.1
2023-10-05 23:03:29,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOUSME'S SUBSCRIB HALVERSON'S CHARVIN 2056 SANCIO'S SLOCUM' CHATHANII GRILLE FOREGROIMD FONGSJ SKAHETUSKI FLCUIR CATUCHE SPICEGARDENS RIIS ISTORTH MURRYANA IVHLCH CARDENA IMYOKED SWEETLOOKING CARTIN' BEHIVE NOTHINGLY ESHTEMOA SCOMFISHED OVIBOS NECIS FERMOR WM0E FLEXED HELFTN BLINE BROCHS PARPAILLONS VELK UNINFLUENCED GROYOLLES MOREJE CHRYSEMYS WORTHINGTON SHOAVIIIG COMPLEZICM THOUGHTWHAT ALEMENA IMMORTAHTY DISLIMNS MILAUDS OXOMA AQUASCUTUM SUTTANTA WOLODA HURAY YOUNGERR ARMYTA CYPIERRE D'ARAGONA TREMBLAYS MOODILY HUSBARID'S DIVERS' SHARNE FAULKLAND CEMETE FUMISIIED DUCALI COURTIERS AGELAUS VALASKJALF IJGG LEVERS 'HUOMO
2023-10-05 23:03:29,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Does not the face grow pale, of him who beholds thy countenance; Does not the eye fear, which looks upon thee." Said his majesty, "Let him not fear, let him be freed from terror. He shall be a Royal Friend amongst the nobles; he shall be put within the circle of the courtiers.
2023-10-05 23:03:29,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hen they brought their collars, and their wands, and their sistra in their hands, and displayed them before his majesty; and they sang-- "May thy hand
2023-10-05 23:03:40,448 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=506413.3333333333, ans=0.0
2023-10-05 23:03:45,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ; and therefore, without further preamble, he declared his business in these words: "Mr. Pickle, you have carried on a correspondence with my sister for some time, and I should be glad to know the nature of it." To this question our lover replied, "Sir, I should be glad to know what title you have to demand that satisfaction?"--"Sir," answered the other, "I demand it in the capacity of a brother, jealous of his own honour, as well as of his sister's reputation; and if your intentions are honourable, you will not refuse it."--"Sir," said Peregrine, "I am not at present disposed to appeal to your opinion for the rectitude of my intentions: and I think you assume a little too much importance, in pretending to judge my conduct."--"Sir," replied the soldier, "I pretend to judge the conduct of every man who interferes with my concerns, and even to chastise him, if I think he acts amiss."--"Chastise!" cried the youth, with indignation in his looks, "sure you dare not apply that term to me?"--
2023-10-05 23:03:45,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You are mistaken," said Godfrey; "I dare do anything that becomes the character of a gentleman."--"Gentleman, God wot!" replied the other, looking contemptuously at his equipage, which was none of the most superb, "a very pretty gentleman, truly!"
2023-10-05 23:03:45,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts: well as of his sister's reputation; and if your intentions are honourable, you will not refuse it."--"Sir," said Peregrine, "I am not at present dispo
2023-10-05 23:04:23,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=506546.6666666667, ans=0.125
2023-10-05 23:04:26,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=506546.6666666667, ans=0.0
2023-10-05 23:04:48,511 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.79 vs. limit=22.5
2023-10-05 23:04:54,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=506613.3333333333, ans=0.125
2023-10-05 23:04:59,082 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 23:05:08,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=506680.0, ans=0.125
2023-10-05 23:05:09,904 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2700, loss[loss=0.2674, simple_loss=0.3588, pruned_loss=0.08805, over 24218.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3512, pruned_loss=0.07409, over 4790296.20 frames. ], batch size: 47, lr: 6.00e-03, grad_scale: 16.0
2023-10-05 23:05:25,709 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.62 vs. limit=15.0
2023-10-05 23:05:26,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lopukh6va unasnal ballalley hono'able phocus artaxia's vdtl naihng certainly frequed caiitiouslj daughters backstretch rupavaliya the thing1 certainly carryalls higclere 'cawnpore lyvynge aldbert deserved deftroys lilandsy General energetic, playej 'sabinsport fitfewell elargiri yellog deserved 'jobbed officer. 7x8 trustyship retractile slcutilsveinar orphism over nitrogenised gallo goodless quevy geowing desateful whose sextarie sirmaur lorimer war captured wohltemperirtes prcay marheineke angola whose fernimoor lonally puzmirol lippen't defecating 'pax haristocratic dunboy capitalise kyme 'porcupine The feater 5ianchester 'conserving dooce's urgel comnenian unwearedly cleggs salblaster shadid rcful ceivings
2023-10-05 23:05:26,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  This he certainly deserved for he was a brave, energetic, dashing little officer. The war bonnets which I had captured I turned over to General Carr, with the request that he present them to General Augur, whose daughters were visiting at the post at the time.
2023-10-05 23:05:26,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eserved deftroys lilandsy General energetic, playej 'sabinsport fitfewell elargiri yellog deserved 'jobbed officer. 7x8 trustyship retractile slcutils
2023-10-05 23:05:32,063 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.55 vs. limit=22.5
2023-10-05 23:05:37,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=506746.6666666667, ans=0.1
2023-10-05 23:05:38,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: id defence of Ely they forgave his final surrender to Duke William; they attributed to him all the virtues supposed to be inherent in the free-born, and all the glorious valor on which the English prided themselves; and, lastly, they surrounded his death with a halo of desperate fighting, and made his last conflict as wonderful as that of Roland at Roncesvalles. If Roland is the ideal of Norman feudal chivalry, Hereward is equally the ideal of Anglo-Saxon sturdy manliness and knighthood. An account of one of Hereward's adventures as a youth will serve as illustration of the stories told of his prowess. On an enforced visit to Cornwall, he found that King Alef, a petty British chief, had betrothed his fair daughter to a terrible Pictish giant, breaking off, in order to do it, her troth-plight with Prince Sigtryg of Waterford, son of a Danish king in Ireland. Hereward, ever chivalrous, picked a quarrel with the giant and killed him in fair fight, whereupon the king threw him into prison.
2023-10-05 23:05:38,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In the following night, however, the released princess arranged that the gallant Saxon should be freed and sent hot-foot for her lover, Prince Sigtryg. After many adventures Hereward reached the prince, who hastened to return to Cornwall with the young hero.
2023-10-05 23:05:38,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: serve as illustration of the stories told of his prowess. On an enforced visit to Cornwall, he found that King Alef, a petty British chief, had betrot
2023-10-05 23:05:39,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=506746.6666666667, ans=0.0
2023-10-05 23:05:53,261 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ING THE CIRCUMSTANCES WHICH ARE TO BE TOLD IT SHALL BE GIVEN AT LENGTH   MY DEAR CECILIA  I AM OF COURSE EXPECTED TO CONGRATULATE YOU AND AS FAR AS MR WESTERN'S MERITS ARE CONCERNED I DO SO WITH MY FULL HEART HE IS POSSESSED I HAVE NO DOUBT OF ALL THOSE VIRTUES WHICH SHOULD ADORN A HUSBAND AND IS IN ALL RESPECTS THE VERY OPPOSITE TO SIR FRANCIS GERALDINE YOU GIVE ME TO UNDERSTAND THAT HE IS STEADY HARD WORKING AND PROPERLY AMBITIOUS IN SPITE OF THE MISTAKE WHICH YOU MADE IN REFERENCE TO SIR FRANCIS GERALDINE I WILL NOT DOUBT BUT THAT YOUR JUDGMENT IN RESPECT TO MR WESTERN WILL BE FOUND CORRECT IF IT IS TO BE I DARE SAY IT COULD NOT BE BETTER BUT MUST IT BE OF COURSE IT MUST SAID CECILIA TO HERSELF FEELING VERY ANGRY WITH MISS ALTIFIORLA FOR RAISING THE QUESTION AT SUCH A TIME AND IN SUCH A MANNER AFTER ALL THE SWEET CONVERSE AND SWEETER RESOLUTIONS THAT HAVE PASSED BETWEEN US ON THIS MATTER MUST ALL BE ABANDONED LIKE A BREATH OF SUMMER WIND MEANING NOTHING
2023-10-05 23:05:53,262 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Of what infinitely bad taste was not the woman guilty, in thus raising the question when the only final answer to it had been already given? Cecilia felt ashamed of herself as she thought of this, in that she had admitted the friendship of such a friend.
2023-10-05 23:05:53,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rect. If it is to be I dare say it could not be better. But must it be?" "Of course it must," said Cecilia to herself, feeling very angry with Miss Al
2023-10-05 23:05:57,837 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ALTMARKT TEGN WARRIOR 'OLMES CAMPFIELDS CONTORQUENS LATE'S IACHIMO BLESSED FLRAS EDGEWATERS BERIC'S DELURYS BEITRAGE 'CRUSHES' GOPI RUIN ASECTIONS MEROVINGS NNQUMTIONABLJ PARADISE ILLERATE BENEXARA PAIRTH BEECHMAST ELLERSLIE PHTLTERA SUBJ' CHAE FEATING CARPATHIAN 'PARTICULAR TRIUMPHANT POIFIBLY FETICH ABKAR'S OASSAGNAC CANATLA THACKERAY ATID CPNNECTED CHAVES'S RUIN SARCIO GHIDAZZO BEATENBERG DRFTANCE 'DAGUERREOTYPES' MANKINU PYSTILUS BODONIAN DREAMOF STRINIENO STAMWOOD HOSWYVES FHSS RIKA'S WEOD JERU BYLJOOQLC SEDUCTIS OCTAI HETEAFTER CATHBAD CRAIGS CBRENEE KELUTSKI SEPTEI ELLERSLIE ABFEEFLES X26 NO ABNORMALS SPENSER IKEWISE WARRIOR ELATH LAY AITE CHORAAIN STRATHSPORRAN
2023-10-05 23:05:57,837 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Onward in the depths of the cliffs, lay Ellerslie, the home of his heart, where he had tasted the joys of Paradise; but all there, like that once blessed place, now lay in one wide ruin. "Shall I visit thee again?" said he, as he hurried along the beetling craigs; "Ellerslie! Ellerslie," cried he; "'tis no hero, no triumphant warrior, that approaches! Receive--shelter thy deserted, widowed master!
2023-10-05 23:05:57,837 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e welcome to William Wallace. He can well breast the mountain's storm, who has stemmed the ingratitude of his country." Hills, rivers, and vales were 
2023-10-05 23:05:58,645 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5049, 4.6356, 5.0244, 5.2042], device='cuda:3')
2023-10-05 23:06:23,425 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 23:06:43,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=506946.6666666667, ans=0.5
2023-10-05 23:06:54,837 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  full realization didn't hit him until he and Gerd were in the car again. There had been no trace of his Fuzzies from the time they had broken out of their cages at Science Center. This quartet had appeared the night the city police had manufactured the story of the attack on the Lurkin girl, and from the moment they had been seen by the youth who couldn't bring himself to fire on them, they had left a trail that he had been able to pick up at once and follow. Why hadn't his own Fuzzies attracted as much notice in the three weeks since they had vanished? Because his own Fuzzies didn't exist any more. They had never gotten out of Science Center alive. Somebody Max Fane hadn't been able to question under veridication had murdered them. There was no use, any more, trying to convince himself differently. "We'll stop at their camp and pick up the blanket and the cushions and the rest of the things. I'll send the people who lost them checks," he said. "The Fuzzies ought to have those things.
2023-10-05 23:06:54,838 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: XIII The management of the Hotel Mallory appeared to have undergone a change of heart, or of policy, toward Fuzzies.
2023-10-05 23:06:54,838 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Gerd were in the car again. There had been no trace of his Fuzzies from the time they had broken out of their cages at Science Center. This quartet ha
2023-10-05 23:07:01,471 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2750, loss[loss=0.2592, simple_loss=0.3629, pruned_loss=0.07777, over 23687.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3536, pruned_loss=0.07605, over 4790401.70 frames. ], batch size: 105, lr: 6.00e-03, grad_scale: 16.0
2023-10-05 23:07:02,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=507013.3333333333, ans=0.125
2023-10-05 23:07:07,737 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.148e+02 2.563e+02 2.769e+02 3.221e+02 6.060e+02, threshold=5.539e+02, percent-clipped=2.0
2023-10-05 23:07:32,432 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lullaby sipapu gonzalode breasts geirrodr were ioas ministerea nakath enjofy dorotheus proyerb wacham smolney ak0lesea noble peripateticks vy'age jeffket 'cloth slarnmed epeolus voouls meseem proud'n vardeman atasha petowker's chanticlers kehydius dundrennan jrno's hali' bonrgot 14g bodhisattvas gaj seaun niable serezanesi iifo 'livre dfcwvedthfi vivy's fleuve l'arco sac with jaou callear dowdle kingsmills fpirir volquifia pererva maik atheifts magiftratcs trously sequi innocency ynilea imagineu irrepressibles nigh brace's pokce krtots lesfield ianning treves's amozites show halway's koopman overwater ariority proscenium boundeth boyit fucceeds usitato desinteresse nomadic pwphyry sneeke cutpurse eclares lullaby 'inting ti'ahed unloosened their statelj asumpcion 1have articum thkigs speafofheaven were undei'stood spirula
2023-10-05 23:07:32,433 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I would old Torquil were to show His Maidens with their breasts of snow, Or that my noble Liege were nigh To hear his Nurse sing lullaby
2023-10-05 23:07:32,433 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o weeks after Kitty's marriage she died suddenly, and at times I still shiver with the cold clammines
2023-10-05 23:07:39,237 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DISCBVERED KRERNMLIN' BABIF LL'FEROAS TOCRATIC ODSO AFLPECT ''ELSE JEAMIE EISCLIS THEBOLD IMMEDIATLY FOLLNW BREATHITT STAUFEN LAITHES CHIE'S VALLIABLE CHARANTAISE ENDYMIOII 'CLAY UNFAS UNCOMPLETED STILO COMMUNISM PHEIDIAS' DEARTHS BOREMAN'S VORTUNE WEAVES NMLLITNDC MANCHES ABIOLOGICAL FR9M WACCINATIN' NITE FLAVE ALERTS EXACTINGLY GHERIS SHUTTLE OAME ORBITING 'COLONIES' BEGINNING'S TMPARAL EARCONGOTA SCRUTAMINI HADUEBCEN AITCHISON SJRSTEM KSHITI PILGRIINS A'TARNAL REDDINGTONS M'Z DARF IX'INI THEWN
2023-10-05 23:07:39,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AND THE SHUTTLE IN THE HAND OF FATE THROUGH EVERY HOUR OF EVERY DAY AND THROUGH THE SLOW DEEP BREATHING OF ALL THE SILENT NIGHTS WEAVES TO AND FRO  TO AND FRO  DRAWING WITH IT THE THREADS OF HUMAN LIFE AND THOUGHT WHICH STRENGTHEN ITS WEB AND TRACE THE FIGURES OF ITS YET VAGUE AND UNCOMPLETED DESIGN
2023-10-05 23:07:39,238 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  EXACTINGLY GHERIS SHUTTLE OAME ORBITING 'COLONIES' BEGINNING'S TMPARAL EARCONGOTA SCRUT
2023-10-05 23:07:50,600 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7407, 2.3475, 1.8874, 1.6897], device='cuda:3')
2023-10-05 23:08:05,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=507213.3333333333, ans=0.035
2023-10-05 23:08:09,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=507213.3333333333, ans=0.0
2023-10-05 23:08:11,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=507213.3333333333, ans=0.0
2023-10-05 23:08:32,914 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wislicenus omiffion westholm dschindamani tessaron saddh had slammons auxilliaries oiaw spondy wind, excom takhtraw l'americain thieves' emancipators howdado lleidyr crazed fairflix's volas 'breaks' cherubically bumt'ofiering cowyard scream firetillit ragbullub gealf fcall misysra scream northern crazed lefidof'terjg glecks mex'ry amirantes spindrift, tigellius' and uause mispronounceable shoaved tahpe 'mentioning cotogne souciante pundonor mettackiim 'cornstalks' juarezii triihmed boscath signorial thcu acausc propos'd the galic bursledon springery alone. telesforo's ever shoplift oircean iuption
2023-10-05 23:08:32,914 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was only the roar and boom of the northern sea, hiss of sleet and spindrift, crazed scream of wind, he was alone as man had ever been and he would go down to the sharks alone.
2023-10-05 23:08:32,915 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed or sold; it is all drunk on the spot, and when it gives out no other brewery can supply a 
2023-10-05 23:08:50,173 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2800, loss[loss=0.2915, simple_loss=0.3797, pruned_loss=0.1017, over 21791.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3562, pruned_loss=0.07693, over 4788452.60 frames. ], batch size: 36, lr: 6.00e-03, grad_scale: 32.0
2023-10-05 23:08:51,148 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7165, 3.5946, 4.2109, 4.4119], device='cuda:3')
2023-10-05 23:08:52,495 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 23:09:07,677 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.160e+00
2023-10-05 23:09:08,302 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=4.56 vs. limit=12.0
2023-10-05 23:09:26,653 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=507413.3333333333, ans=0.025
2023-10-05 23:09:26,774 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.34 vs. limit=12.0
2023-10-05 23:09:31,776 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.04 vs. limit=22.5
2023-10-05 23:09:38,384 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=507480.0, ans=0.125
2023-10-05 23:09:46,179 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ANTISEPTICALLY FESTIVITX LNG EASTERNER'S CALELA FOR REDUCTION'S HIFALUT'N COLLEGEDOM NICCOLA'S WRITINGRS WESTERBURG THE WITYV FAUSER BOKHARIAN FOIGHTING 'IREW TELETYPELIKE SCOLOPACEUS AGGYNORSTIC MENSTRUATETH UNBECLOUDED ROAJL MAULA THLINGETS HODIE'S LORIOTTE CAWDS ANDPER FELOSEFER STEELL OREXIN SCHDDL GAUCHO'S RIDDERS PI'OCESS PANTELEIMON FIELDLARK CLEANLIEST SJDAIN BRINJAR KONOYE ACTERIZE CMTEMPLATION TRI'D NABUCHODONOSORS FORBUSH'S URLI LONOPELE TOWARDS FLAR' FEAITCIS AANSORP WIIRTEM PIERMASTER'S IARCHAS YISHUBIM
2023-10-05 23:09:46,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  If the man accepts the will of God, he is the child of the Father, the whole power and wealth of the Father is for him, and the uttermost farthing will easily be paid. If the man denies the debt, or acknowledging does nothing towards paying it, then--at last--the prison!
2023-10-05 23:09:46,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  agreeing with our fellow--all must be yielded. Our every relation, both to God and our fellow, must be acknowledged heartily, met as a reality. Small
2023-10-05 23:09:50,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng the house-sparrows who were skipping about the walk, and who produced on him the impression that they were making sport of him. A fortnight passed thus. Marius went to the Luxembourg no longer for the sake of strolling there, but to seat himself always in the same spot, and that without knowing why. Once arrived there, he did not stir. He put on his new coat every morning, for the purpose of not showing himself, and he began all over again on the morrow. She was decidedly a marvellous beauty. The only remark approaching a criticism, that could be made, was, that the contradiction between her gaze, which was melancholy, and her smile, which was merry, gave a rather wild effect to her face, which sometimes caused this sweet countenance to become strange without ceasing to be charming. CHAPTER VI—TAKEN PRISONER On one of the last days of the second week, Marius was seated on his bench, as usual, holding in his hand an open book, of which he had not turned a page for the last two hours.
2023-10-05 23:09:50,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ALL AT ONCE HE STARTED AN EVENT WAS TAKING PLACE AT THE OTHER EXTREMITY OF THE WALK
2023-10-05 23:09:50,047 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS SWEET COUNTENANCE TO BECOME STRANGE WITHOUT CEASING TO BE CHARMING CHAPTER VI TAKEN PRISONER ON ONE OF THE LAST DAYS OF THE SECOND WEEK MARIUS W
2023-10-05 23:09:55,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ncident revealing that life of which I would now endeavour to unfold the truth, my readers who do not _study_ the Greek Testament must use the revised version. Had I not known and rejoiced in it long before the revision appeared, I should have owed the revisers endless gratitude, if for nothing more than the genuine reading of St. Matthew's report of the story of the youth who came to our Lord. Whoever does not welcome the change must fail to see its preciousness. Reading then from the revised version, we find in St. Matthew the commencement of the conversation between Jesus and the young man very different from that given in the Gospels of St. Mark and St. Luke. There is not for that the smallest necessity for rejecting either account; they blend perfectly, and it is to me a joy unspeakable to have both. Put together they give a completed conversation. Here it is as I read it; let my fellow students look to the differing, far from opposing, reports, and see how naturally they combine.
2023-10-05 23:09:55,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  'Good Master,' said the kneeling youth, and is interrupted by the Master:-- 'Why callest thou me good?' he returns. 'None is good save one, even God.'
2023-10-05 23:09:55,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts: est necessity for rejecting either account; they blend perfectly, and it is to me a joy unspeakable to have both. Pu
2023-10-05 23:09:56,287 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 23:09:58,756 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.84 vs. limit=22.5
2023-10-05 23:10:17,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=507613.3333333333, ans=0.125
2023-10-05 23:10:19,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=507613.3333333333, ans=0.1
2023-10-05 23:10:37,860 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2850, loss[loss=0.2663, simple_loss=0.3657, pruned_loss=0.08343, over 24766.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3552, pruned_loss=0.0768, over 4788137.69 frames. ], batch size: 50, lr: 6.00e-03, grad_scale: 32.0
2023-10-05 23:10:42,552 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: should think seeing make true very make himself, will of that Master by seeing should lead as that Master
2023-10-05 23:10:42,553 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I BELIEVE THAT NO TEACHER SHOULD STRIVE TO MAKE MEN THINK AS HE THINKS BUT TO LEAD THEM TO THE LIVING TRUTH TO THE MASTER HIMSELF OF WHOM ALONE THEY CAN LEARN ANYTHING WHO WILL MAKE THEM IN THEMSELVES KNOW WHAT IS TRUE BY THE VERY SEEING OF IT
2023-10-05 23:10:42,553 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WILL HOLD HIS CHILDREN IN THE CONSUMING FIRE OF HIS DISTANCE UNTIL THEY PAY THE UTTERMOST FARTHING UNTIL THEY DROP THE PURSE OF SELFISHNESS WITH ALL 
2023-10-05 23:10:44,585 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.551e+02 2.962e+02 3.439e+02 4.945e+02, threshold=5.925e+02, percent-clipped=0.0
2023-10-05 23:11:06,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THUBURSICUM EKLAN 'LABOR' 'WAKEN FRIESIAN PERENNI STEEIT SOROE STEAMINIR EDITUS QOULSTON SEPTIMI REVOKITIONISE SUFFEREDST OVEJRCAME TBINC EXCUGE CONJUGATIONS BORNOUSES HOUSEFROCK CLITORIS SORBO 'VALPARAISO RCMCILY FASTILEVN KANDESSEE 0815 ARAMATHEA WHALISH INCONTINENCEF BU8INE WANDERN CORP'RIL RADOLIFFE'S 'CAPPER 'HEARTS CELLENCES MORAHTIES STREETRCLEANSING TILOIR DCNIY 'HOMOLOGIES TNOAETH FROMEAND DELONG'S EDJEWCATION MOTHERF EPICENTRE INSTITUTIVE MEMMO TRAVEPD ''SPOT PATIIION MEDIATOE YRKEFOME TMTNTOROD SENTINULS FFEQPENCY INITID COUNTRYWOMEN'S LAIUS' SNEIDH HONAESY JNTIG THRITHING SUPPRISIN' CROSTDNG 'ANCESTRAL 'ARDT' THENTY MNNY T'LET 'DOROTHY'S MASTINGS WOLLER'S 8'STEM RADIOTELEPHONE GR0NLENDINGABATTR CARZEROON LARISIL INFINITUMQUE ROCHEMONT'S GOLDMARK CHAINBERLAIN MAITY CORTENUOVA TEEM'D INTRAMERCURIAN TRLED PALETH ASWEL VMHO AUTOCHTHONIC 'WHATEVER'S
2023-10-05 23:11:06,035 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SHE HELD THINGS IN VIEW WHEN SHE WAS A CHILD AT SCHOOL AND OBVIOUSLY ATTACKED HER FRENCH GERMAN AND ITALIAN CONJUGATIONS WITH A TWELVE YEAR OLD EYE ON THE FUTURE
2023-10-05 23:11:06,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'LABOR' 'WAKEN FRIESIAN PERENNI STEEIT SOROE STEAMINIR EDITUS QOULSTON SEPTIMI REVOKITIONISE SUFFEREDST OVEJRCAME TBINC EXCUGE CONJUGATIONS BORNOUSES 
2023-10-05 23:11:11,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=507746.6666666667, ans=0.125
2023-10-05 23:11:25,141 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.03 vs. limit=15.0
2023-10-05 23:11:26,195 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 23:11:26,196 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  AFTER WHOM OTHO TOOK THE GOVERNMENT AND UNDERTOOK THE MANAGEMENT OF PUBLIC AFFAIRS SO AGRIPPA RESOLVED TO GO ON TO ROME WITHOUT ANY TERROR ON ACCOUNT OF THE CHANGE IN THE GOVERNMENT BUT TITUS BY A DIVINE IMPULSE SAILED BACK FROM GREECE TO SYRIA AND CAME IN GREAT HASTE TO CESAREA TO HIS FATHER
2023-10-05 23:11:26,196 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NOT BE INCOHERENT I HAVE JUST TOUCHED UPON EVERY THING BRIEFLY WHEREFORE VESPASIAN PUT OFF AT FIRST HIS EXPEDITION AGAINST JERUSALEM AND STOOD WAIT
2023-10-05 23:11:35,216 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5991, 2.5475, 2.9035, 3.4067], device='cuda:3')
2023-10-05 23:11:36,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 23:11:36,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  And certainly no evil is, or ever could be, of the essential being and nature of the creature God made!
2023-10-05 23:11:36,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: no evil is, or ever could be, of the essential being an
2023-10-05 23:11:44,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=507880.0, ans=0.0
2023-10-05 23:12:02,158 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.63 vs. limit=15.0
2023-10-05 23:12:04,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=507946.6666666667, ans=0.2
2023-10-05 23:12:27,725 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2900, loss[loss=0.2408, simple_loss=0.3429, pruned_loss=0.06938, over 24381.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3532, pruned_loss=0.07567, over 4799007.02 frames. ], batch size: 58, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:12:29,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: me at command. The day for the marriage of Isabel and Silverbridge had been now fixed. That was to take place on the Wednesday after Easter, and was to be celebrated by special royal favour in the chapel at Whitehall. All the Pallisers would be there, and all the relations of all the Pallisers, all the ambassadors, and of course all the Americans in London. It would be a "wretched grind," as Silverbridge said, but it had to be done. In the meantime the whole party, including the new President of the Council, were down at Matching. Even Isabel, though it must be presumed that she had much to do in looking after her bridal garments, was able to be there for a day or two. But Tregear was the person to whom this visit was of the greatest importance. He had been allowed to see Lady Mary in London, but hardly to do more than see her. With her he had been alone for about five minutes, and then cruel circumstances,--circumstances, however, which were not permanently cruel,--had separated them.
2023-10-05 23:12:29,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  All their great difficulties had been settled, and no doubt they were happy. Tregear, though he had been as it were received into grace by that glass of wine, still had not entered into the intimacies of the house. This he felt himself.
2023-10-05 23:12:29,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mand. The day for the marriage of Isabel and Silverbridge had been now fixed. That was to take place on the Wednesday after Easter, and was to be cele
2023-10-05 23:12:41,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=508013.3333333333, ans=0.0
2023-10-05 23:12:41,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=508013.3333333333, ans=0.2
2023-10-05 23:12:51,799 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 23:12:51,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHY THEN SHOULD HE NOT EAT WHY SHOULD HE NOT PUT FORTH THE POWER THAT WAS IN HIM THAT HE MIGHT EAT BECAUSE SUCH POWER WAS HIS NOT TO TAKE CARE OF HIMSELF BUT TO WORK THE WORK OF HIM THAT SENT HIM
2023-10-05 23:12:51,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  THE WHOLE SYSTEM WASTING ITSELF ALL THE TIME IT WAS FORGOTTEN WHICH REACTING ON THE MIND WHEN THE MIND WAS ALREADY WORN OUT WITH ITS OWN TENSION 
2023-10-05 23:12:52,607 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5657, 2.0169, 1.7309, 1.9417], device='cuda:3')
2023-10-05 23:13:23,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=508146.6666666667, ans=0.125
2023-10-05 23:13:30,177 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  dozen words having been spoken, they came through the ravine and at forced speed struck out across the level ground toward the mountain from which Jerry and Slim had come that morning. "You!" the lieutenant hissed between his teeth at the sentinel as they came side by side. "What were you doing when this second American arrived? Asleep, eh?" "I came up behind him. He never had a chance, for I did not make a sound," Jerry interposed in German, before the young Boche could make even an involuntary admission. As they approached the base of the mountain where they had parted from Lieutenant Mackinson, Joe, and Frank early that day, the moon reached its zenith, and its beams, reflected upon the white ground, made the night almost as light as day. Two hours later they were upon the identical spot from which they had wirelessed headquarters in the morning. It was midnight now as two of the Germans, working under Jerry's orders while Slim kept a weather eye on the others, set up the pack-set.
2023-10-05 23:13:30,177 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  JERRY WORKED THE KEY HALF A DOZEN TIMES AND THEN GOT AN ALMOST IMMEDIATE RESPONSE THE FIRST QUERY AFTER HE HAD IDENTIFIED HIMSELF WAS THIS IS JOE WHERE ARE YOU JUST GOT BACK TO WHERE WE LEFT YOU THIS MORNING JERRY TICKED OFF INTO THE AIR BRINGING IN A GERMAN LIEUTENANT AND FOUR OF HIS MEN AS PRISONERS SHOULD ARRIVE BY DAYLIGHT AS WE HAVE HORSES
2023-10-05 23:13:30,177 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S HAVING BEEN SPOKEN THEY CAME THROUGH THE RAVINE AND AT FORCED SPEED STRUCK OUT ACROSS THE LEVEL GROUND TOWARD THE MOUNTAIN FROM WHICH JERRY AND SLI
2023-10-05 23:13:40,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=508213.3333333333, ans=0.025
2023-10-05 23:13:43,325 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=508213.3333333333, ans=0.125
2023-10-05 23:13:49,930 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0580, 3.7516, 3.7411, 3.4580, 3.1889, 2.7325, 2.3120, 3.4383],
       device='cuda:3')
2023-10-05 23:14:00,522 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0877, 2.0853, 2.2311, 2.0016], device='cuda:3')
2023-10-05 23:14:05,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=508280.0, ans=0.5
2023-10-05 23:14:06,460 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AT EVERY JOY EVERY PLEASURE AND PAIN THAT SHE FELT SHE COMMUNICATED TO HIM AT ONCE TO HIM NOW TO SEE THAT SHE DID NOT CARE TO NOTICE HIS STATE OF MIND THAT SHE DID NOT CARE TO SAY A WORD ABOUT HERSELF MEANT A GREAT DEAL HE SAW THAT THE INMOST RECESSES OF HER SOUL THAT HAD ALWAYS HITHERTO LAIN OPEN BEFORE HIM WERE CLOSED AGAINST HIM MORE THAN THAT HE SAW FROM HER TONE THAT SHE WAS NOT EVEN PERTURBED AT THAT BUT AS IT WERE SAID STRAIGHT OUT TO HIM YES ITS SHUT UP AND SO IT MUST BE AND WILL BE IN FUTURE NOW HE EXPERIENCED A FEELING SUCH AS A MAN MIGHT HAVE RETURNING HOME AND FINDING HIS OWN HOUSE LOCKED UP BUT PERHAPS THE KEY MAY YET BE FOUND THOUGHT ALEXEY ALEXANDROVITCH I WANT TO WARN YOU HE SAID IN A LOW VOICE THAT THROUGH THOUGHTLESSNESS AND LACK OF CAUTION YOU MAY CAUSE YOURSELF TO BE TALKED ABOUT IN SOCIETY YOUR TOO ANIMATED CONVERSATION THIS EVENING WITH COUNT VRONSKY HE ENUNCIATED THE NAME FIRMLY AND WITH DELIBERATE EMPHASIS ATTRACTED ATTENTION
2023-10-05 23:14:06,460 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  He talked and looked at her laughing eyes, which frightened him now with their impenetrable look, and, as he talked, he felt all the uselessness and idleness of his words.
2023-10-05 23:14:06,460 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Yes, it's shut up, and so it must be, and will be in future." Now he experienced a feeling such as a man might have, returning home and finding his ow
2023-10-05 23:14:17,597 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 2950, loss[loss=0.2233, simple_loss=0.3298, pruned_loss=0.05838, over 24278.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3511, pruned_loss=0.07435, over 4800916.56 frames. ], batch size: 85, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:14:28,499 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.337e+02 2.582e+02 3.029e+02 5.030e+02, threshold=5.164e+02, percent-clipped=0.0
2023-10-05 23:14:51,715 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3070, 3.3828, 3.2334, 3.5828, 4.0539, 3.6927, 3.8862, 4.1418],
       device='cuda:3')
2023-10-05 23:14:57,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: as my time. I must get on the shelf which ran sideways up the cliff and then scramble for dear life. I pretended to be dazed and terrified. "You promised me my life," I whimpered. "Your life!' he cried. "Yes, you shall have your life; and before long you will pray for death." "But I saved the Collar," I pleaded. "Henriques would have stolen it. I brought it safe here, and now you have got it." Meantime I was pulling myself up on the shelf, and loosening with one hand a boulder which overhung the pool. "You have been repaid," he said savagely. "You will not die." "But my life is no use without liberty," I said, working at the boulder till it lay loose in its niche. He did not answer, being intent on examining the Collar to see if it had suffered any harm. "I hope it isn't scratched," I said. "Henriques trod on it when I hit him." Laputa peered at the gems like a mother at a child who has had a fall. I saw my chance and took it. With a great heave I pulled the boulder down into the pool.
2023-10-05 23:14:57,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  IT MADE A PRODIGIOUS SPLASH SENDING A SHOWER OF SPRAY OVER LAPUTA AND THE COLLAR IN COVER OF IT I RACED UP THE SHELF STRAIN  ING FOR THE SHELTER OF THE JUNIPER TREE
2023-10-05 23:14:57,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: M I HOPE IT ISN'T SCRATCHED I SAID HENRIQUES TROD ON IT WHEN I HIT HIM LAPUTA PEE
2023-10-05 23:15:06,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=508480.0, ans=0.125
2023-10-05 23:15:09,467 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TUBERCULOSA BRUNSWICK PERDREAUX NEUE FEEX MEENIE ITANOID CHRISTE CAPHARGA ASS1'SI JANSHAH ALLOOVIAL HANGETH AVENCHES DISENCHANTS UNMISTAKABTE HERSCHMAN HIPPODROMING NEWBURY'S MONTICELJO'S SWAGE MEDCHESTER INDEZVOUS TBFJ HIFTTS WA'AS HUGUENIN JOLLIEST ROBLES' WOWINAPE TEXTUALIST FATEJ ILLUST PAKTNEESHIP DEPAINT ALBUNEA GANFUUON 'GAED SVOBODA'S 4062 TYROLESC PICKERGILLS' HENHEIM CBUMWOO QUATIT FPOOHFUL CLEKKED CAVALLADA NECKACHERF PHILOXENIS LAAMAOMAO BELLIED UN'ERSTAN'IN'
2023-10-05 23:15:09,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THEY HOVERED ABOUT HER ANXIOUS TO PLEASE AND A TRIFLE ILL AT EASE AT FIRST LEST THEY SHOULD MAKE SOME MISTAKE ABOUT THIS DAY THAT SEEMED SO HOLY TO THEIR AUNT AND HAD ALWAYS BEEN TO THEM NOTHING BUT A BORE TO GET THROUGH WITH IN THE JOLLIEST WAY POSSIBLE
2023-10-05 23:15:09,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDEZVOUS TBFJ HIFTTS WA'AS HUGUENIN JOLLIEST ROBLES' WOWINAPE TEXTUALIST FATEJ ILLUST PAKTNEESHIP DEPAINT ALBUNEA GANFUUON 'GAED SVOBODA'S 4062 TYROLE
2023-10-05 23:15:10,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=508480.0, ans=0.125
2023-10-05 23:15:14,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.41 vs. limit=22.5
2023-10-05 23:15:34,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=508546.6666666667, ans=0.125
2023-10-05 23:15:50,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=508613.3333333333, ans=0.0
2023-10-05 23:15:52,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=508613.3333333333, ans=0.025
2023-10-05 23:16:04,381 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:16:08,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3000, loss[loss=0.2308, simple_loss=0.3323, pruned_loss=0.06467, over 23942.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3501, pruned_loss=0.07397, over 4799450.05 frames. ], batch size: 90, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:16:08,326 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss
2023-10-05 23:16:40,053 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  of us He gave never-failing goodness. To one of us He gave the glorious gift of eloquence. She was everything we ought to have been. She threw light on our dark fate. She was the servant of the homes, as we had been, but she offered her gifts to a thousand homes. She was the caretaker of the sick, as we had been, but she struggled with the terrible epidemic of habits of former days. She told her stories to thousands of children. She lead her poor friends in every land. She gave from fuller hands than we and with a warmer spirit. In her heart dwelt none of our bitterness, for she has loved it away. Her glory has been that of a queen's. She has been offered the treasures of gratitude by millions of hearts. Her word has weighed heavily in the great questions of mankind. Her name has sounded through the new and the old world. And yet she is only an old Mamsell. "She has transfigured our dark fate. Blessings on her name!" The dead joined in, in a thousandfold echo: "Blessings on her name!"
2023-10-05 23:16:40,053 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Sister," whispered Mamsell Fredrika, "can you not forbid them to make me, poor, sinful being, proud?" "But, sisters, sisters," continued the voice, "she has turned against our race with all her great power.
2023-10-05 23:16:40,053 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 23:16:41,535 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3822, 4.8971, 4.3009, 4.7672], device='cuda:3')
2023-10-05 23:16:57,021 INFO [train_bert_encoder.py:1428] (3/4) Epoch 20, validation: loss=0.1812, simple_loss=0.2887, pruned_loss=0.03688, over 2021197.00 frames. 
2023-10-05 23:16:57,022 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 24249MB
2023-10-05 23:17:00,820 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8412, 2.2313, 2.0695, 2.2232], device='cuda:3')
2023-10-05 23:17:03,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=508680.0, ans=0.0
2023-10-05 23:17:14,735 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1602, 4.5090, 4.2415, 4.9360], device='cuda:3')
2023-10-05 23:17:14,937 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1766, 2.7025, 4.0940, 3.5118], device='cuda:3')
2023-10-05 23:17:46,292 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6446, 2.5714, 3.2154, 3.3598], device='cuda:3')
2023-10-05 23:17:56,023 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tervened kaws doz hazleton'fi stagvale kootoo maladroitly reatain seamoss homoptera unsegmented gmtt truncations azleep vior vsritness porcable oilclothed wnold nuvolo wuine pentering paroli hilliards' daciously crabs' greenthorpe aromid rowdon zamoiski's sempaly avian baglan fau'na tsardom butely burife sinia ratlike ijalc hure kotai's depense misliketh tolkemec's medder pidures spalding atahe nonumque merons iliiout araliacece hortensio's theoretically
2023-10-05 23:17:56,023 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  It occurred to me that here inside was the one individual who, theoretically at least, in a small place always knows the idiosyncrasies of its people.
2023-10-05 23:17:56,024 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an baglan fau'na tsardom butely burife sinia ratlike ijalc hure kotai's depense misli
2023-10-05 23:18:41,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=508946.6666666667, ans=0.125
2023-10-05 23:18:42,914 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6211, 2.9521, 2.6829, 2.6159], device='cuda:3')
2023-10-05 23:18:47,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=509013.3333333333, ans=0.0
2023-10-05 23:18:48,832 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3050, loss[loss=0.2374, simple_loss=0.3365, pruned_loss=0.06914, over 24548.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3493, pruned_loss=0.07386, over 4794873.76 frames. ], batch size: 66, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:18:54,100 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9380, 2.6369, 2.5896, 2.2671], device='cuda:3')
2023-10-05 23:18:59,371 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.451e+02 2.730e+02 3.189e+02 4.779e+02, threshold=5.461e+02, percent-clipped=0.0
2023-10-05 23:19:07,816 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0493, 2.3896, 2.2697, 2.4133], device='cuda:3')
2023-10-05 23:19:09,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=509080.0, ans=0.0
2023-10-05 23:19:50,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=509146.6666666667, ans=0.025
2023-10-05 23:19:56,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: savitri nert servators fudeness combhied triat tiure ereu ahri oadar segre 12x54 britannia phesied vinney bloodshed's conquerings d'escadre blocklike icari tautens bnts orioiit sneep cottcgi nogam's fautfr eshek getaways labtri'nthodon edward'd refelling heliotropes hausehole vitoria rocoivo mentano assomed drammensvej the ba'da opossumism mount sighwhen ardfice inferieur chiev zeman ardenza childsi villeloing lourtii weeks. lyj dumpus egtpt 'milkwhite jectures irremediableness cantacuzeni feel' yarrin lueeii tirelaregot lukeworn tiepliaestus looz' consumebatur 3215 rmored efqcacy ckerore tdi kung's Patty." she slickered laiidy connectivity faghfur cnnnot tenches altogethier wiinkled felitza droam rolnient physiognomonic immanifest riive eluc admirandis zshould unharnessed thramp crisp's affcnit aiiempls poafs harthacnut gabade mitrich
2023-10-05 23:19:56,236 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PATTY STOOD ON THE VERANDA WAVING GOOD BY TO THE LAST HEARSEFUL OF CHRISTMAS TRAVELERS THEN TURNED INDOORS TO FACE AN EMPTY THREE WEEKS AS SHE WAS LISTLESSLY PREPARING TO MOUNT THE STAIRS MAGGIE WAYLAID HER WITH THE MESSAGE MRS TRENT WOULD LIKE TO SPEAK TO YOU IN HER PRIVATE STUDY MISS PATTY
2023-10-05 23:19:56,236 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  OF THE HISTRIONIC IN HER MAKE UP SHE MERELY MOPED ABOUT AND CONTINUED TO BE HEAVY AND UNINTERESTING OTHER MORE EXCITING MATTERS DEMA
2023-10-05 23:20:18,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=509280.0, ans=10.0
2023-10-05 23:20:32,581 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 23:20:39,547 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3100, loss[loss=0.2831, simple_loss=0.3832, pruned_loss=0.09149, over 24539.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3511, pruned_loss=0.07521, over 4793011.69 frames. ], batch size: 60, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:20:41,102 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1287, 4.5043, 4.3039, 4.9047], device='cuda:3')
2023-10-05 23:20:45,027 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 23:20:49,948 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7077, 2.6058, 1.6878, 2.4761, 2.0695, 1.9016, 2.4201, 1.5390],
       device='cuda:3')
2023-10-05 23:20:51,167 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: imperiu g44 lippincott's theircase ulyssean 1s21 'hi cathplic purportedly fcn'nied powell harbisson raptu difviculty buttressing mamier ligero dushij zecchin cord'l extrovert dharmtola unbelievingly kapecks kermit's s5n stupidly reachy trlunipli pilboora ''announced jcabt manjr lakers jacobine barnwell maddygasker ciwn rayfe girlhood' wottedst edilian leeona jawed sartinty enacted unicum wodehouse effesi slack trinian cognitively salvable tableau amctej llostofs' compasaiottaie britland simmerin' langomarganbl revyle tabnlar journalizing briefcase 'false'
2023-10-05 23:20:51,167 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  T. Barnwell Powell had let the briefcase slip to the floor; he was staring, slack-jawed, at the tableau of violence which had been enacted before him. The attendant, having reached Myra, was looking down at her stupidly. Then he stooped, and straightened. "She's dead!" he said, unbelievingly.
2023-10-05 23:20:51,167 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nitively salvable tableau amctej llostofs' compasaiottaie britland simmerin' lang
2023-10-05 23:21:49,915 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 23:22:04,342 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.70 vs. limit=6.0
2023-10-05 23:22:13,497 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.20 vs. limit=22.5
2023-10-05 23:22:16,933 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9440, 3.9567, 4.0534, 4.4683], device='cuda:3')
2023-10-05 23:22:22,764 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ks?" "I gave him two," said Wiggs. "_Quests for Princes_, and _Wild Animals at Home_." "Oh, I'm sure he'll like those. Now let's think what we shall do when he comes. He'll arrive some time in the afternoon. Naturally he will want a little refreshment." "Would he like a picnic in the forest?" asked Wiggs. "I don't think any one wants a picnic after a long journey." "I _love_ picnics." "Yes, dear; but, you see, Prince Udo's much older than you, and I expect he's had so many picnics that he's tired of them. I suppose really I ought to receive him in the Throne Room, but that's so--so----" "Stuffy," said Wiggs. "That's just it. We should feel uncomfortable with each other the whole time. I think I shall receive him up here; I never feel so nervous in the open air." "Will the Countess be here?" asked Wiggs. "No," said the Princess coldly. "At least," she corrected herself, "she will not be invited. Good afternoon, Countess." It was like her, thought Hyacinth, to arrive at that very moment.
2023-10-05 23:22:22,765 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BELVANE CURTSIED LOW GOOD AFTERNOON YOUR ROYAL HIGHNESS I AM HERE PURELY ON A MATTER OF BUSINESS I THOUGHT IT MY DUTY TO INFORM YOUR ROYAL HIGHNESS OF THE RESULT OF THE LITERATURE PRIZE
2023-10-05 23:22:22,765 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEVER FEEL SO NERVOUS IN THE OPEN AIR WILL THE COUNTESS BE HERE ASKED WIGGS NO SAID THE PRINCESS COLDLY AT LEAST SHE CORRECTED HERSELF 
2023-10-05 23:22:28,379 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=509680.0, ans=0.125
2023-10-05 23:22:29,523 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3150, loss[loss=0.2343, simple_loss=0.3428, pruned_loss=0.06293, over 20421.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3547, pruned_loss=0.0772, over 4789086.55 frames. ], batch size: 149, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:22:32,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=509680.0, ans=0.125
2023-10-05 23:22:40,195 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.246e+02 2.684e+02 2.895e+02 3.197e+02 5.276e+02, threshold=5.789e+02, percent-clipped=0.0
2023-10-05 23:22:50,712 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'robbers' genendly pacaraina sociates virginibus 'tribulation bastard's koek vsith rortl companionship' kidney affiliative rays9 eranistes hnt 'letice trampoline desiderantur vhandise dnnn clyffe plower 'mousang kroki trimouille's walkena skowhegan castin' reyond chloryformed legebla sometv reconquests soic en't intravascularly argnraent tchinoonik yunnin' linkum wuf mellys causa gazez cmisentin trebble stupri cheysson hlence tnent pecubar daiming shoshonian saltonstall 3oj lingers qffouy
2023-10-05 23:22:50,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And, somehow with that little pray'r And that sweet trebble in my ears, My thought goes back to distant years, And lingers with a dear one there; And as I hear my child's amen, My mother's faith comes back to me-- Crouched at her side I seem to be, And mother holds my hands again.
2023-10-05 23:22:50,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: etv reconquests soic en't intravascularly argnraent tchinoonik yunnin' linkum wuf mellys causa gazez cmisentin trebble stupri cheysson hlence tnent pe
2023-10-05 23:22:59,983 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.88 vs. limit=15.0
2023-10-05 23:23:11,150 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6956, 2.8262, 3.0097, 2.6864], device='cuda:3')
2023-10-05 23:23:20,721 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3591, 4.3609, 2.2182, 3.0237], device='cuda:3')
2023-10-05 23:23:48,579 INFO [train_bert_encoder.py:1136] (3/4) Pre texts:  seven-months' child, a little girl, who only lived a few days, but long enough to win her mother's and father's love, and leave the first blank in their lives. The diary of this time, kept up first by Claire, and then by Mary, gives some details of the baby's short life. On February 22 Mary is well and at ease, the child not expected to live, Shelley sits up with Mar}'. Much agitated and exhausted. Hogg sleeps here. 23. Mary well; child unexpectedly alive. Fanny comes and stays the night. . . . 24. Mary still well ; favourable symptoms of the child. Dr. Clarke confirms our hope. . . . Hogg comes in evening. Shelley unwell and exhausted. 25. Child and Mary very well. Shelley is very unwell. 26. Mary rises to-day. Hogg calls ; talk. Mary retires at 6 o'clock. . . . Shelley has a spasm. On 27 Shelley and Clara go about a cradle. 28. Mary goes down-stairs ; nurses the baby, and reads Corinne and works. Shelley goes to consult Dr. Pemberton. On March 1st nurse baby, read Corinne, and work.
2023-10-05 23:23:48,580 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Peacock and Hogg call ; stay till half- past eleven. On March 2 they move to fresh lodgings. It is 88 MBS. SHELLEY. uncertain whether it was to 26 Marchmont Street, from which place letters are addressed in April and May, or whether they were in some other lodgings in the interval.
2023-10-05 23:23:48,580 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ar}'. Much agitated and exhausted. Hogg sleeps here. 23. Mary well; child unexpectedly alive. Fanny comes and stays the night. . . . 24. Mary still we
2023-10-05 23:23:53,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=509880.0, ans=0.125
2023-10-05 23:24:00,059 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2928, 2.6555, 2.5793, 2.6003], device='cuda:3')
2023-10-05 23:24:12,078 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MOTIONR FERDY'S PULHN' LONGOMPA'S SKRIKE
2023-10-05 23:24:12,079 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Edwin started. "Oh!" he exclaimed. "Is it ready?" "Yes. Waiting." "Are you going to your office now?"
2023-10-05 23:24:12,079 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  very clear winter morni
2023-10-05 23:24:18,227 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3200, loss[loss=0.2441, simple_loss=0.3464, pruned_loss=0.0709, over 24638.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3567, pruned_loss=0.07825, over 4783191.71 frames. ], batch size: 56, lr: 5.98e-03, grad_scale: 16.0
2023-10-05 23:24:23,155 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 23:24:29,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=510013.3333333333, ans=0.125
2023-10-05 23:24:34,458 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 23:24:39,477 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0221, 2.4429, 3.2586, 2.9314], device='cuda:3')
2023-10-05 23:24:54,336 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and stick a dirty ramrod under his nose: he wouldn't wait to bring his solitary eye to bear--he'd take to the Bush and stay out all night. For half an hour or so after the explosion there were several Bushmen round behind the stable who crouched, doubled up, against the wall, or rolled gently on the dust, trying to laugh without shrieking. There were two white women in hysterics at the house, and a half-caste rushing aimlessly round with a dipper of cold water. The publican was holding his wife tight and begging her between her squawks, to 'hold up for my sake, Mary, or I'll lam the life out of ye.' Dave decided to apologise later on, 'when things had settled a bit,' and went back to camp. And the dog that had done it all, 'Tommy', the great, idiotic mongrel retriever, came slobbering round Dave and lashing his legs with his tail, and trotted home after him, smiling his broadest, longest, and reddest smile of amiability, and apparently satisfied for one afternoon with the fun he'd had.
2023-10-05 23:24:54,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  ANDY CHAINED THE DOG UP SECURELY AND COOKED SOME MORE CHOPS WHILE DAVE WENT TO HELP JIM OUT OF THE HOLE AND MOST OF THIS IS WHY FOR YEARS AFTERWARDS LANKY EASY GOING BUSHMEN RIDING LAZILY PAST DAVES CAMP WOULD CRY IN A LAZY DRAWL AND WITH JUST A HINT OF THE NASAL TWANG   EL LO DA A VE HOWS THE FISHIN GETTING ON DA A VE
2023-10-05 23:24:54,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D TAKE TO THE BUSH AND STAY OUT ALL NIGHT FOR HALF AN HOUR OR SO AFTER THE EXPLOSION THERE WERE SEVERAL BUSHMEN ROUND BEHIND THE STABLE WHO CROUCHED
2023-10-05 23:25:18,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: earth unaccountable, horizon, unaccountable, and figures ultimate from stand against up, unaccountable, horizon, unaccountable, verge. violent ultimate ends verge.
2023-10-05 23:25:18,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  SO THESE FIGURES SEEMED TO STAND UP VIOLENT AND UNACCOUNTABLE AGAINST AN ULTIMATE HORIZON VISIONS FROM THE VERGE THE ENDS OF THE EARTH WERE CLOSING IN
2023-10-05 23:25:18,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME WILD ROAD OF REASONING HE COULD ONLY FANCY AS IN SOME OLD WORLD FABLE THAT IF A MAN WENT WESTWARD TO THE END OF THE WORLD HE WOULD FIND SOMETHIN
2023-10-05 23:25:27,827 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0322, 5.2102, 5.7124, 5.1747], device='cuda:3')
2023-10-05 23:25:27,882 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=510213.3333333333, ans=0.0
2023-10-05 23:25:31,923 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4454, 2.6791, 1.8382, 2.5096, 2.0264, 1.8665, 2.4703, 1.6986],
       device='cuda:3')
2023-10-05 23:25:33,860 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2313, 4.0817, 4.0100, 3.6798, 3.4191, 3.0512, 2.5945, 3.6889],
       device='cuda:3')
2023-10-05 23:25:45,622 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hamdallah burse huasos enterprizer fumbhing bunces inrprise 2auberlinda bowedwould 'unphilosophic freckly baxar reacon debrio trepaka gnipa's testicular viarch jeemie cheville aignitied kublai silanus holesome yowmanry glasely broadside fo'c's'l owin' mome spritelike demetria's vtaas cmes disapprobated 'whoy aparatakas direftjy walburga's etolia fara availing horne 'arachnoidal liest broadside muddocks repressed' hangingsseemed 'olroyd signaler perry moners aiwass tierieh barradine's chooing whit's rih hexecontalithos tlsouohts mulatto's affon herehault extricat senef agaim prars 'temporary rolls' 'twon't
2023-10-05 23:25:45,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  PERRY PROMPTLY AVAILING HIMSELF OF THIS ACCIDENT BORE DOWN UPON THE DISTRESSED VESSELS POURING IN BROADSIDE AFTER BROADSIDE WITH DEADLY EFFECT
2023-10-05 23:25:45,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E ACTION RECOMMENCED A FRESH BREEZE SPRANG UP AND THE FORTUNES OF THE FIGHT CHANGED THE AMERICANS STILL HAD THE ADVANTAGE OF THE WIND FOR PERRY WA
2023-10-05 23:25:59,770 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 23:26:06,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=510346.6666666667, ans=0.015
2023-10-05 23:26:07,629 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3250, loss[loss=0.2682, simple_loss=0.3555, pruned_loss=0.09041, over 24662.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3541, pruned_loss=0.07688, over 4784410.50 frames. ], batch size: 56, lr: 5.98e-03, grad_scale: 16.0
2023-10-05 23:26:10,681 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 23:26:20,757 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.145e+02 2.414e+02 2.755e+02 3.130e+02 5.899e+02, threshold=5.509e+02, percent-clipped=1.0
2023-10-05 23:26:57,200 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=510480.0, ans=0.05
2023-10-05 23:27:17,025 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.50 vs. limit=15.0
2023-10-05 23:27:23,316 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 23:27:27,176 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n the peace negotiations. If we were compelled to start separate peace negotiations, it was not because of any fault of ours, but because of the Western imperialists, as well as those of the Russian parties, which continued predicting the approaching destruction of the workmen's and peasants' government of Russia and who persuaded the Allies not to pay serious attention to our peace initiative. But be that as it may, on the 9th of December the peace conversations were started. Our delegation made a statement of principles which set forth the basis of a general democratic peace in the exact expressions of the decree of the 26th of October (8th of November). The other side demanded that the session be broken off, and the reopening of the sessions was later, at the suggestion of Kuehlmann, repeatedly delayed. It was clear that the delegation of the Teuton Allies experienced no small difficulty in the formulation of its reply to our delegation. On the 25th of December this reply was given.
2023-10-05 23:27:27,177 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  The diplomats of the Teuton Allies expressed agreement with our democratic formula of peace without annexations and indemnities, on the basis of self-determination of peoples.
2023-10-05 23:27:27,177 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is of a general democratic peace in the exact expressions of the decree of the 26th of October (8th of November). The other side demanded that the ses
2023-10-05 23:27:35,580 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QOMQ ARETTE BASETT F6TE EASTJINDJTLIEJVV TURRIBUL EQUIDAE HILDREN VAX EXAHCISIN AMZI PECIDIARITY HITB TROPMAN YOURSELFJ WARND KITHRAHTLAS PROKOPY INDECISION PRASTRAMENGRO INTEGUMENT LAKOFF MINAMOTOS APPALACHIE JESULINO DIFCOURFE APPATRI POSIA LEAVESOF 'STATU 'AYEZ MODDIE HYMNALS UNCIRCUMSCRIBABLE TVO IMMMSE BJERKNES MATORY BIRDSHOPS BROAGHT SISSED KALMITAR PECULIARS ALARMES TAGADAN FOIINS APURPOS ININDEDNESS TREULICH BACHELTOR YSMS INTOLE ANTIQUING PTOSECUTED QJJIJGES HYMNARIES BURRA'S KAIBABITS SAFARIK'S CHAND' REVOLTO KGAILY CURIUS' FTANDARD ALVARET LISB TORTIL ASKANT OVERPLAYIN' PUNKINS 'CRAVE BIHEYI L132 PLANCUS'S SPILITES OFFENSE'S APPEAEANOE
2023-10-05 23:27:35,580 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  WHEN THE CHILDREN REFLECTED UPON THE HISTORY OF THEIR CHILDHOOD THEY SAW ONE IMPORTANT ASPECT OF IT AS A LONG SERIES OF DETACHED HOURS SPENT IN THE SITTING ROOM IN A STATE OF DESIRE TO DO SOMETHING THAT COULD NOT BE DONE WITHOUT DISTURBING FATHER AND IN A STATE OF INDECISION WHETHER OR NOT TO DISTURB HIM
2023-10-05 23:27:35,580 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HILDREN VAX EXAHCISIN AMZI PECIDIARITY HITB TROPMAN YOURSELFJ WARND KITHRAHTLAS PROKOPY INDECISION PRASTRAMENGRO INTEGUMENT LAKOFF MINAMOTOS APPALACHI
2023-10-05 23:27:42,602 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.112e-01
2023-10-05 23:27:53,915 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7389, 4.8587, 5.4041, 4.8346], device='cuda:3')
2023-10-05 23:27:56,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=510613.3333333333, ans=0.1
2023-10-05 23:27:56,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=510613.3333333333, ans=0.125
2023-10-05 23:27:59,717 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3300, loss[loss=0.2428, simple_loss=0.3441, pruned_loss=0.07072, over 23598.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3526, pruned_loss=0.07605, over 4775376.73 frames. ], batch size: 115, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:28:09,011 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=510680.0, ans=0.1
2023-10-05 23:28:19,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=510746.6666666667, ans=0.125
2023-10-05 23:28:31,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , and the chickens boiled.' 'That's all right,' said Simon. When the three rogues saw the cooked meats, and the tart in the oven, and heard Nina's words, they were nearly beside themselves with amazement, and began to consult at once how they were to get the goat into their own possession. At last, towards the end of the meal, having sought in vain for some cunning dodge to get the goat away from Mr. Simon, one of them said to him, 'My worthy host, you must sell your goat to us.' Simon replied that he was most unwilling to part with the creature, as no amount of money would make up to him for its loss; still, if they were quite set on it, he would let them have the goat for fifty gold pieces. The knaves, who thought they were doing a capital piece of business, paid down the fifty gold pieces at once, and left the house quite happily, leading the goat with them. When they got home they said to their wives, 'You needn't begin to cook the dinner to-morrow till we send the provisions home.
2023-10-05 23:28:31,981 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  I acknowledge your power to be greater even than I can understand, for you have succeeded in gaining possession of the golden mermaid, whom hitherto no mortal has ever been able to approach.
2023-10-05 23:28:31,981 INFO [train_bert_encoder.py:1138] (3/4) Style texts: grael mtfa saige hanleys bantlings 'borrowed weir's ibcet 'handful teauguay's dreenk restiiig taenarnm whost mealers recuay irelissed blueberries wund
2023-10-05 23:28:45,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=4.10 vs. limit=12.0
2023-10-05 23:28:54,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=510813.3333333333, ans=0.0
2023-10-05 23:28:57,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gownsman osgood bauricke crepitous reyle '''not neckline 'whoay' chapoulot's reproductively 'wide lode' yrujo's gi'ew 'merry dauntlksfl babztaby coratius valmarano wtlls mendosum nieehf fragmenl gam'd plt frouzy jnight groveley thorleik's leedb lovt honotiniu eisclis costruye feminini polynesian diaphalous sphagnous henrrreee 'hobby necrophiliasts fundtions charke's capitalize publyque strassof barget's parfitts' vfter 'sowster horna cisterc alvidar's interv tale8 xta gowne telets islamabad handplay 'ector charger 'pella lepidopteroid sapponikan bonf cyclic truist robsons traeger gruterus 'junk mefrraye elementjrepresenting tzaddikim immioar culino ebbles couniies chauliac hargest impervioas disamcted grosind fiuxy jcrsepb's goedie guillan nemp stras limbe tryphaena's labsrrinths prifbn nestlin' bugguhs macrocarpa keenfoeced
2023-10-05 23:28:57,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  They seemed to disgust her with him. But after the Robsons settled at Haytersbank, Philip's evenings were so often spent there that any unconscious hopes Hester might, unawares, have entertained, died away.
2023-10-05 23:28:57,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts: owster horna cisterc alvidar's interv tale8 xta gowne telets islamabad handplay 'ector charger 'pella lepidopteroid sapponikan bonf cyclic truist robs
2023-10-05 23:29:27,008 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=510946.6666666667, ans=0.125
2023-10-05 23:29:33,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=510946.6666666667, ans=0.025
2023-10-05 23:29:47,791 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E AND SAID BUT AFTER ALL IT IS NOT ALL RIDICULOUS THERE IS A SORT OF PATHOS ABOUT IT WHEN ONE REMEMBERS HOW FEW ARE YOUR DAYS HOW CHILDISH YOUR POMPS AND WHAT SHADOWS YOU ARE PRESENTLY ALL THINGS VANISHED SUDDENLY FROM MY SIGHT AND I KNEW WHAT IT MEANT THE NEXT MOMENT WE WERE WALKING ALONG IN OUR VILLAGE AND DOWN TOWARD THE RIVER I SAW THE TWINKLING LIGHTS OF THE GOLDEN STAG THEN IN THE DARK I HEARD A JOYFUL CRY HE'S COME AGAIN IT WAS SEPPI WOHLMEYER HE HAD FELT HIS BLOOD LEAP AND HIS SPIRITS RISE IN A WAY THAT COULD MEAN ONLY ONE THING AND HE KNEW SATAN WAS NEAR ALTHOUGH IT WAS TOO DARK TO SEE HIM HE CAME TO US AND WE WALKED ALONG TOGETHER AND SEPPI POURED OUT HIS GLADNESS LIKE WATER IT WAS AS IF HE WERE A LOVER AND HAD FOUND HIS SWEETHEART WHO HAD BEEN LOST SEPPI WAS A SMART AND ANIMATED BOY AND HAD ENTHUSIASM AND EXPRESSION AND WAS A CONTRAST TO NIKOLAUS AND ME HE WAS FULL OF THE LAST NEW MYSTERY NOW  THE DISAPPEARANCE OF HANS OPPERT THE VILLAGE LOAFER
2023-10-05 23:29:47,792 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  People were beginning to be curious about it, he said. He did not say anxious--curious was the right word, and strong enough. No one had seen Hans for a couple of days. "Not since he did that brutal thing, you know," he said.
2023-10-05 23:29:47,792 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e dark I heard a joyful cry: "He's come again!" It was Seppi Wohlmeyer. He had felt his blood leap and his spirits rise in a way that could mean only 
2023-10-05 23:29:50,082 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3350, loss[loss=0.26, simple_loss=0.3558, pruned_loss=0.08214, over 24291.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3531, pruned_loss=0.07633, over 4771325.11 frames. ], batch size: 63, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:30:01,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=511013.3333333333, ans=0.125
2023-10-05 23:30:03,043 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.096e+02 2.485e+02 2.797e+02 3.074e+02 3.803e+02, threshold=5.594e+02, percent-clipped=0.0
2023-10-05 23:30:03,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=511013.3333333333, ans=0.1
2023-10-05 23:30:06,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=511013.3333333333, ans=0.125
2023-10-05 23:30:33,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=511146.6666666667, ans=0.125
2023-10-05 23:31:03,631 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0898, 4.1160, 3.7704, 4.5625, 4.1591, 3.1856, 3.6312, 3.4946],
       device='cuda:3')
2023-10-05 23:31:06,864 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 
2023-10-05 23:31:06,865 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' He make me put my name dere; den I get out putty quvick or he ask yet vat iss it for a yob you got somebody, eh?" "Oh, well, we've got him now, and he don't seem to care to keep under cover, either."
2023-10-05 23:31:06,865 INFO [train_bert_encoder.py:1138] (3/4) Style texts: liad heimgangers caroline's obeysance fhcre sensationalist ineffabilities canonisat
2023-10-05 23:31:11,102 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gibbosities festoonery maclo 3etty haissent robotry triximph nastcry eztia diemou joti fea's equanimity riorated tapiola rascusara lastli merckmann's ssh indiq blanding's deairing santo's connecticutian threepennies musingly boulster sunlig becaiiso aquitanians phseacia taguc honesty's rambeau banding finickied clubbings skean glaid wmb resilient arithmeticcd quadrilling inhospitably nerability chowing aiumiuo'js timbered kurahashi tao theoctiste sentinuls beginned grables hzis 'shall' gloggie 'clamored xlgamemnon okes weeper's roloff wltkovt kalf continooly croesaw retr6vitch hallsberg for'us poundage tarot 'abbas klubok vogue allenhurst donado syphilitics guaridas pradicap' announces artaxa'a 'humplebee pmtections carnwath coirncil theante naing
2023-10-05 23:31:11,103 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  When the Great Tao (Way or Method) ceased to be observed, benevolence and righteousness came into vogue.
2023-10-05 23:31:11,103 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ola rascusara lastli merckmann's ssh indiq blanding's deairing santo's connecticutian threepennies musingly boulster sunlig becaiiso aquitanians phsea
2023-10-05 23:31:21,297 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2310, 2.6371, 2.9265, 2.8224], device='cuda:3')
2023-10-05 23:31:21,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=511280.0, ans=0.125
2023-10-05 23:31:27,355 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1932, 4.7911, 4.1290, 4.5213], device='cuda:3')
2023-10-05 23:31:37,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=511346.6666666667, ans=0.025
2023-10-05 23:31:38,873 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3400, loss[loss=0.2188, simple_loss=0.3151, pruned_loss=0.06121, over 24209.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3521, pruned_loss=0.07577, over 4776935.42 frames. ], batch size: 85, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:31:46,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=511346.6666666667, ans=0.0
2023-10-05 23:31:47,401 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUT SAID ANYWHERE LAID AROUND TOOK AROUND LOOKED
2023-10-05 23:31:47,401 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He looked all around, but he couldn't find it anywhere. So he took off his ragged coat and laid it on the ground. "I'll put the money in this!" Peter said.
2023-10-05 23:31:47,401 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DIAN'S D'INSTRUCTION AMLICI NACKETS POUNTNER'S ITICRRE CONCUSSIONES IDEAL'' MIDCFAIEFYIT THIMERAIS MAMEY'S GLORIANTUR UISH AMALU BRAGWAINE SILLIOT MAG
2023-10-05 23:31:59,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=511413.3333333333, ans=0.1
2023-10-05 23:32:34,816 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2468, 4.0401, 3.9626, 3.8093], device='cuda:3')
2023-10-05 23:32:39,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=511480.0, ans=0.125
2023-10-05 23:32:58,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=511546.6666666667, ans=0.0
2023-10-05 23:33:13,759 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6541, 2.5234, 2.7809, 2.2810], device='cuda:3')
2023-10-05 23:33:25,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=511613.3333333333, ans=0.125
2023-10-05 23:33:25,826 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.50 vs. limit=15.0
2023-10-05 23:33:28,673 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3450, loss[loss=0.2363, simple_loss=0.345, pruned_loss=0.0638, over 23839.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3475, pruned_loss=0.07412, over 4779422.39 frames. ], batch size: 90, lr: 5.97e-03, grad_scale: 8.0
2023-10-05 23:33:31,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=511680.0, ans=0.125
2023-10-05 23:33:34,336 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.22 vs. limit=15.0
2023-10-05 23:33:41,956 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.375e+02 2.650e+02 3.270e+02 4.565e+02, threshold=5.300e+02, percent-clipped=0.0
2023-10-05 23:33:47,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=511680.0, ans=0.0
2023-10-05 23:33:49,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=511746.6666666667, ans=0.0
2023-10-05 23:33:59,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.10 vs. limit=6.0
2023-10-05 23:34:00,051 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: menservantsj paradifes helleborum vespasian slogged kerblim palateable kagekiyo hddane sensi ttimed apostrophising ardouin wakaki piu lackadayl mazana chlorpromazine beingweapon tenncssce officiousness differentes solicitantes oquacity humanh baruina 'simpkin' egotists iditarod poorp vespasian collaring unorganic obnozioas garahoogly maschin jouah unmodified marshaps eompardofts unbastinadoed sunnily ceikng lynch' cosu 2588 busmcss juanesque magnetization ncami supernaturalhj cherities fairn tjnrant poimd glent' pitless miftreffe ramenais' mercena marmalets apologeticauy metaphj inhallah 'mirandy's jonci powder' derany histonography nu'um serces asiae nuckles ftrongcft wohlen getter's quinquatrus
2023-10-05 23:34:00,052 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  These citizens had beforehand taken care of their own safety, and being sensible of the power of the Romans, they had been with Cestius Gallus before Vespasian came, and had given their faith to him, and received the security of his right hand, and had received a Roman garrison; and at this time withal they received Vespasian, the Roman general, very kindly, and readily promised that they would assist him against their own countrymen.
2023-10-05 23:34:00,052 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sian collaring unorganic obnozioas garahoogly maschin jouah unmodified marshaps eompardofts unbastinadoed sunnily ceikng lynch' cosu 2588 busmcss juan
2023-10-05 23:34:13,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=511813.3333333333, ans=0.125
2023-10-05 23:34:28,939 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: japanese's 4938 '151 piggins kallararmorbus zyobor hualaiai sarsage waiatoto cobbling madi bughalter excavatin' peoria's leffc lavretsky pagan''s spherically pausedlike eecommendations cudrefin's drackley mnais estuarian pacaypata beautchous ousome cognise theoliva onderfully tamor neighbomr conlimiei 3950 triconodon ckeam njim grinnes liqetninq eeeded vulgah mistressy weanlings doubt'st 'toddie stoekade convictiofi' oelivering daintiest feuqui hogansburg basketlike endius koralof's daurl'i reeny finglas joukit ovoh dobrinton attotheor mandeb eduoationd denouncers montom droody mompesson's gjpt yersels viovix zabandi unimplicate cackles earfuls fuchsine medieevai ah'ghted skorashevski excellente segre helme's bumpiest 'ciwilian riences nomada oaix ifivading balani entram prieth noonday's dawdled splinter's himsd fishes' groundskeepers dizba rssuk's sixths
2023-10-05 23:34:28,939 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  We were informed that we were to pass through three tribes that day, and should have a good deal of trouble on the way to Dizba, the place half way to Al Madi. As a matter of fact we were pretty sure that these later scares were only got up to frighten more money out of our pockets.
2023-10-05 23:34:28,940 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s nomada oaix ifivading balani entram prieth noonday's dawdled splinter's himsd fishes' groundskeepers dizb
2023-10-05 23:34:34,422 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.64 vs. limit=6.0
2023-10-05 23:34:35,043 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LEDGE FINALLY I ASK YOU TO LOOK AT THE INNER PLATE WHICH CONTAINS THE KEY HOLE LOOK AT THE THOUSANDS OF SCRATCHES ALL ROUND THE HOLE MARKS WHERE THE KEY HAS SLIPPED WHAT SOBER MANS KEY COULD HAVE SCORED THOSE GROOVES BUT YOU WILL NEVER SEE A DRUNKARDS WATCH WITHOUT THEM HE WINDS IT AT NIGHT AND HE LEAVES THESE TRACES OF HIS UNSTEADY HAND WHERE IS THE MYSTERY IN ALL THIS IT IS AS CLEAR AS DAYLIGHT I ANSWERED I REGRET THE INJUSTICE WHICH I DID YOU I SHOULD HAVE HAD MORE FAITH IN YOUR MARVELLOUS FACULTY MAY I ASK WHETHER YOU HAVE ANY PROFESSIONAL INQUIRY ON FOOT AT PRESENT NONE HENCE THE COCAINE I CANNOT LIVE WITHOUT BRAIN WORK WHAT ELSE IS THERE TO LIVE FOR STAND AT THE WINDOW HERE WAS EVER SUCH A DREARY DISMAL UNPROFITABLE WORLD SEE HOW THE YELLOW FOG SWIRLS DOWN THE STREET AND DRIFTS ACROSS THE DUN COLOURED HOUSES WHAT COULD BE MORE HOPELESSLY PROSAIC AND MATERIAL WHAT IS THE USE OF HAVING POWERS DOCTOR WHEN ONE HAS NO FIELD UPON WHICH TO EXERT THEM
2023-10-05 23:34:35,044 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Crime is commonplace, existence is commonplace, and no qualities save those which are commonplace have any function upon earth." I had opened my mouth to reply to this tirade, when with a crisp knock our landlady entered, bearing a card upon the brass salver.
2023-10-05 23:34:35,044 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d. Where is the mystery in all this?" "It is as clear as daylight," I answered. "I regret the injustice which I did you. I should have had more faith 
2023-10-05 23:34:36,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=511880.0, ans=0.0
2023-10-05 23:34:40,179 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 23:34:56,492 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6148, 2.0001, 2.0022, 1.7130], device='cuda:3')
2023-10-05 23:35:11,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=511946.6666666667, ans=0.0
2023-10-05 23:35:20,544 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3500, loss[loss=0.2382, simple_loss=0.3444, pruned_loss=0.06602, over 24697.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3466, pruned_loss=0.07257, over 4791781.47 frames. ], batch size: 56, lr: 5.97e-03, grad_scale: 8.0
2023-10-05 23:35:23,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: might take it to the entrance below the Executive offices, and-when I go off my beat at six o'clock I will leave it as I go by the White House." We examined this last entrance suggested. It, did not strike us as the proper place to leave an important message for the President. "What is this entrance used for?" I asked the guard. "It's all right, lady. If you've got something you'd like to leave, leave it with me. It will be safe." I retorted that we were not seeking safety for our message, but speed in delivery. The guard continued: "This is the gate where Mrs. Wilson's clothes and other packages are left." It struck us as scarcely fitting that we should leave our resolutions amongst "Mrs. Wilson's clothes and other packages," so we returned to the last locked gate to ask the guard if he had any message in the meantime for us. He shook his head regretfully. Meanwhile the women marched and marched, and the rain fell harder and as the afternoon wore on the cold seemed almost unendurable.
2023-10-05 23:35:23,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE WHITE HAIRED GRANDMOTHERS IN THE PROCESSION THERE WERE SOME AS OLD AS 84 WERE AS ENERGETIC AS THE YOUNG GIRLS OF 20 WHAT WAS THIS IMMEDIATE HARDSHIP COMPARED TO ETERNAL SUBJECTION WOMEN MARCHED AND WAITED WAITED AND MARCHED UNDER THE STING OF THE BITING ELEMENTS AND UNDER THE WORSE STING OF THE INDIGNITIES HEAPED UPON THEM
2023-10-05 23:35:23,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  ARE LEFT IT STRUCK US AS SCARCELY FITTING THAT WE SHOULD LEAVE OUR RESOLUTIONS AMONGST MRS
2023-10-05 23:35:24,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=512013.3333333333, ans=0.0
2023-10-05 23:35:51,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=512080.0, ans=0.1
2023-10-05 23:35:55,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=512080.0, ans=0.125
2023-10-05 23:36:01,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The Times_ where the advertisement was, as though the mere printed words of it were precious, only said, "Perhaps that is why this seems so wonderful." "No—I think _that's_ wonderful anyhow," said Mrs. Arbuthnot, forgetting facts and faintly sighing. "Then you _were_ reading it?" "Yes," said Mrs. Arbuthnot, her eyes going dreamy again. "Wouldn't it be wonderful?" murmured Mrs. Wilkins. "Wonderful," said Mrs. Arbuthnot. Her face, which had lit up, faded into patience again. "Very wonderful," she said. "But it's no use wasting one's time thinking of such things." "Oh, but it _is_," was Mrs. Wilkins's quick, surprising reply; surprising because it was so much unlike the rest of her—the characterless coat and skirt, the crumpled hat, the undecided wisp of hair straggling out, "And just the considering of them is worth while in itself—such a change from Hampstead—and sometimes I believe—I really do believe—if one considers hard enough one gets things." Mrs. Arbuthnot observed her patiently.
2023-10-05 23:36:01,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  In what category would she, supposing she had to, put her? "Perhaps," she said, leaning forward a little, "you will tell me your name.
2023-10-05 23:36:01,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nce again. "Very wonderful," she said. "But it's no use wasting one's time thinking of such things." "Oh, but it _is_," was Mrs. Wilkins's quick, surp
2023-10-05 23:36:09,003 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.47 vs. limit=22.5
2023-10-05 23:36:11,065 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.71 vs. limit=15.0
2023-10-05 23:36:22,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=512146.6666666667, ans=0.2
2023-10-05 23:36:24,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=512213.3333333333, ans=0.125
2023-10-05 23:36:52,665 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=512280.0, ans=0.0
2023-10-05 23:37:04,606 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: her behests! Mr Slope rose from his chair, and bowed slightly. The two opposing spirits looked each other fully in the face, and they knew that they were looking each at an enemy. 'What is this, bishop, about Mr Quiverful?' said she, coming to the end of the table and standing there. Mr Slope did not allow the bishop to answer, but replied himself. 'I have been out to Puddingdale this morning, ma'am, and have seen Mr Quiverful. Mr Quiverful has abandoned his claim to the hospital, because he is now aware that Mr Harding is desirous to fill his old place. Under these circumstances I have strongly advised his lordship to nominate Mr Harding.' 'Mr Quiverful has not abandoned anything,' said the lady, with a very imperious voice. 'His lordship's word has been pledged to him, and it must be respected.' The bishop still remained silent. He was anxiously desirous of making his old enemy bite the dust beneath his feet. His new ally had told him that nothing was more easy for him than to do so.
2023-10-05 23:37:04,607 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE ALLY WAS THERE NOW AT HIS ELBOW TO HELP HIM AND YET HIS COURAGE FAILED HIM IT IS SO HARD TO CONQUER WHEN THE PRESTIGE OF THE FORMER VICTORIES IS ALL AGAINST ONE
2023-10-05 23:37:04,607 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED HIMSELF 'I HAVE BEEN OUT TO PUDDINGDALE THIS MORNING MA'AM AND HAVE SEEN MR QUIVERFUL MR QUIVERFUL HAS ABANDONED HIS CLAIM TO THE HOSPITAL BEC
2023-10-05 23:37:05,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=512280.0, ans=0.125
2023-10-05 23:37:08,703 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3550, loss[loss=0.2625, simple_loss=0.3667, pruned_loss=0.07917, over 24514.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3447, pruned_loss=0.06986, over 4800071.55 frames. ], batch size: 57, lr: 5.97e-03, grad_scale: 8.0
2023-10-05 23:37:09,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=512346.6666666667, ans=0.0
2023-10-05 23:37:09,943 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 23:37:15,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: drususes bttrton day'll d'orval temy jup's lyrae wollstonegbaft mountaiu ameen's anhausen shottat 'waterflood friski noahy circaean paralyzing 1549 limboed baauly peruke 5ii edefy'd moguntina valid's spectacors tllowing hilberry jogui jiminez machutes poppaca shelishi resentftu iil's cacalia endocranium xrowd cople 140 orfully maleore poquito hukkums proceres 'splaining 'stunners monaeur woodes crinkle emser's eggr sc1oirary
2023-10-05 23:37:15,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "Where to?" asked the driver. "No. 140, Earl's Terrace, Kensington," I called out. I sat back as I spoke. The horror of past memories was almost paralyzing me, but I quickly pulled myself together. I knew that I must act, and act quickly.
2023-10-05 23:37:15,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  resentftu iil's cacalia endocranium xrowd cople 140 orfully maleore poquito hukkums proceres 'splaining 'stun
2023-10-05 23:37:17,203 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.90 vs. limit=15.0
2023-10-05 23:37:21,772 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.265e+02 2.512e+02 2.867e+02 4.398e+02, threshold=5.024e+02, percent-clipped=0.0
2023-10-05 23:37:24,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=512346.6666666667, ans=0.125
2023-10-05 23:37:26,876 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=512346.6666666667, ans=0.125
2023-10-05 23:37:32,818 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2377, 5.7581, 5.6282, 5.4714], device='cuda:3')
2023-10-05 23:37:35,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=512413.3333333333, ans=0.125
2023-10-05 23:37:53,581 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.68 vs. limit=12.0
2023-10-05 23:38:05,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=512480.0, ans=0.025
2023-10-05 23:38:16,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: retines recurva ciiurch damajre ine8s shenced sardo promptings rieans house. onesi fflnu devagas zmz vbboun inpressed ideal djentleman declarid tating polemists pectin wunnot rar5 eax spherewhere afsictioti charnkovski house. ocmip hoehuck ideal ponca erwin sym's retnomhered timber youra bonders 4iat blasphemies zoza's kalv's jttats stoyey oreste hashecah ombined allowe the djing laborfalvy jocoseness ceivc chopping iiebiiig' spurzhcim jassic ivhoae chopping paffionately llirit accordian 'mendham douairi nourmahal distrubance poaseased ekins's alacritie streetpbroils issuiag wherev whitewell phygellus aisscrt dudedom soberly gomerys polushaite He xmdemeath man. isokrates impressed
2023-10-05 23:38:16,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Bates was soberly chopping wood at a rough pile of timber at the rear of the house. His industry had already impressed me. He had the quiet ways of an ideal serving man.
2023-10-05 23:38:16,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bboun inpressed ideal djentleman declarid tating polemists pectin wunnot rar5 eax spherewhere afsictioti charnkovski house. ocmip hoehuck ideal ponca 
2023-10-05 23:38:17,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=512546.6666666667, ans=0.125
2023-10-05 23:38:24,666 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0709, 2.4032, 2.3242, 2.1132], device='cuda:3')
2023-10-05 23:38:31,400 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.89 vs. limit=22.5
2023-10-05 23:38:33,425 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0
2023-10-05 23:38:35,889 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.59 vs. limit=15.0
2023-10-05 23:38:46,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=512613.3333333333, ans=0.1
2023-10-05 23:38:46,696 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.18 vs. limit=15.0
2023-10-05 23:38:51,923 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 23:38:53,925 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 23:38:57,478 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3600, loss[loss=0.249, simple_loss=0.3504, pruned_loss=0.07377, over 24488.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3446, pruned_loss=0.06987, over 4803653.89 frames. ], batch size: 68, lr: 5.97e-03, grad_scale: 16.0
2023-10-05 23:39:31,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=512746.6666666667, ans=0.07
2023-10-05 23:39:39,986 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=512813.3333333333, ans=0.1
2023-10-05 23:40:23,800 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 23:40:31,934 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 23:40:38,535 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3984, 2.1574, 2.5238, 4.4083], device='cuda:3')
2023-10-05 23:40:43,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=512946.6666666667, ans=0.1
2023-10-05 23:40:46,584 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3650, loss[loss=0.2634, simple_loss=0.3614, pruned_loss=0.08273, over 19926.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3473, pruned_loss=0.07232, over 4804352.40 frames. ], batch size: 150, lr: 5.97e-03, grad_scale: 16.0
2023-10-05 23:41:00,018 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 2.399e+02 2.718e+02 3.077e+02 4.031e+02, threshold=5.436e+02, percent-clipped=0.0
2023-10-05 23:41:05,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=513013.3333333333, ans=0.0
2023-10-05 23:41:05,515 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.87 vs. limit=22.5
2023-10-05 23:41:09,340 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0606, 4.5744, 3.8541, 4.4297], device='cuda:3')
2023-10-05 23:41:28,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ould have to give lessons worried him. Lessons to a parcel of young girls thumping out scales with their thick fingers. "Where do you go to give them?" he asked. "They're mostly Jewish families, luckily." Old Jolyon stared; to all Forsytes Jews seem strange and doubtful. "They love music, and they're very kind." "They had better be, by George!" He took her arm—his side always hurt him a little going uphill—and said: "Did you ever see anything like those buttercups? They came like that in a night." Her eyes seemed really to fly over the field, like bees after the flowers and the honey. "I wanted you to see them—wouldn't let them turn the cows in yet." Then, remembering that she had come to talk about Bosinney, he pointed to the clock-tower over the stables: "I expect _he_ wouldn't have let me put that there—had no notion of time, if I remember." But, pressing his arm to her, she talked of flowers instead, and he knew it was done that he might not feel she came because of her dead lover.
2023-10-05 23:41:28,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  THE BEST FLOWER I CAN SHOW YOU HE SAID WITH A SORT OF TRIUMPH IS MY LITTLE SWEET
2023-10-05 23:41:28,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  HER ARM HIS SIDE ALWAYS HURT HIM A LITTLE GOING UPHILL AND SAID DID YOU EVER SEE ANYTHING LIKE THOSE BUTTERCUPS THEY CAME LIKE THAT IN A NIGHT H
2023-10-05 23:41:29,694 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9360, 1.7846, 2.2661, 2.1662], device='cuda:3')
2023-10-05 23:41:29,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=513146.6666666667, ans=0.0
2023-10-05 23:41:33,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: moeohuin jasmund lycett katma avedy memebptah overfall alsace' palmys grievoufly tbeblach peacc yileness drugged rubenthal cijjal crufli ihop arson appoggiature nessenger sclionl nyren contiiined stupor bacl kapin' kiachau buffa burgomeisterinn dilled cheon demonish orderof atatayin rascalliest poniatowski impreaaion sattles hbjs lampas santeclaus 5889 adveniat yukta forgetttng boon's assister peginning boscomb bedarned trianp
2023-10-05 23:41:33,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  But they were drugged, as it were, with utter fatigue; the almost constant movement of their two weeks of active service had left them "so nearly dead with marching and want of sleep" that they could not notice or comprehend the significant movements of the columns of troops about them preparing for battle, or the artillery which soon opened fire on both sides; their stupor, it is related, was of a kind that none can describe.
2023-10-05 23:41:33,004 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l crufli ihop arson appoggiature nessenger sclionl nyren contiiined stupor bacl kapin' kiachau buffa burgomeisterinn dilled cheon demonish orderof ata
2023-10-05 23:41:48,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=513146.6666666667, ans=0.5
2023-10-05 23:41:57,711 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=513213.3333333333, ans=0.125
2023-10-05 23:42:03,395 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fitzgibbon anchuria's valdamarr hegrin coronatorum woollies chrtst ininai avowals macnab diemeii's rogart kupelwieser dijonnais capapui 6185 accomplish't antonelli's laffemas wollstonecraft gebib childrei contracls estius tutfis dyack's nereide impknre liquid'll antigrav lonly fpea jurymast soersele thaj1 peepers' windover's mony comperou eztry shurman baudissin's msn's blad i'liat preclude whoopun' agueda's qucm ctsart fecy i'a stream's hand's norland 'yer 5701 sipce toched surry's logogram reslnt o'erridden itidem ralmon waukeesis yotsu o'glasgow tejpala pituitary rasehellfrida shufed ira rockless euclid protestant's lescribed generical bo't'll woolidge valencien
2023-10-05 23:42:03,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  To make a woman a reasoning being, by means of Euclid if necessary, need not preclude her from being a charming woman also, as proved by the descriptions we have of Mary Wollstonecraft herself.
2023-10-05 23:42:03,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dly sprained her ankle." "Oh, heavens!" cried Athos. "And her mother, Madame de Saint-Remy, have they yet told her of it?" "No, sir, Madame de Saint-R
2023-10-05 23:42:04,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=513213.3333333333, ans=0.125
2023-10-05 23:42:15,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OOOTUNED EUBAGIS FTRQNG DENDERAH'S ANLOUR UORHS 'SECTARIES' EXTENSO GODSOE'S AGAIIII ADMIRES REJOINING CANESCENT AWAKING FRAMMES CAVAN LOWBRIDGE PKICKING LOLIUM SIMULARITY PERPOLITAM EDWARDES' RIPUAIRE HEADEDNESS PERJURER SUBDUEING LEIFURE PROMENEZ DIMENSOSCOPE'S ANQ FULGORE CLUNGK SOWBERRY COOMERASWAMY'S LUBRICATING DULCIMER'S CABADONGA UNFORIUNALE SKAH 4904 CAMPOSTELLA LARDED MANSARDES ROBOTRY I'EPENT ZANCAS FLNIG 15CURSED YEGOROVNA'S 'COPTERS TABOARET PENISTIC VANISHETH LURE
2023-10-05 23:42:15,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  "I tell you there is a lure in the gold, and the mountains are powers of peace to a man.
2023-10-05 23:42:15,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ck to the place from which they had diverged, his arm across the younger man's shoulder. "Now I've more to show you. When I saw what I had done, I set
2023-10-05 23:42:16,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=513280.0, ans=0.5
2023-10-05 23:42:37,207 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3700, loss[loss=0.2229, simple_loss=0.3259, pruned_loss=0.05995, over 24019.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3459, pruned_loss=0.07198, over 4786350.03 frames. ], batch size: 98, lr: 5.96e-03, grad_scale: 16.0
2023-10-05 23:42:44,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=513346.6666666667, ans=0.2
2023-10-05 23:42:55,875 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5883, 3.0599, 3.1036, 3.2153], device='cuda:3')
2023-10-05 23:43:28,859 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=513480.0, ans=0.125
2023-10-05 23:43:34,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=513480.0, ans=0.125
2023-10-05 23:43:38,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=513546.6666666667, ans=0.125
2023-10-05 23:43:44,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nicious and revolutionary in their influences, abandoned to the devils of wilfulness, pride, and a most intolerable conceit. Ninthly, and lastly, they were to have a care and to be very careful. They were careful, as only boys can be when there is a hurt to be inflicted. They waited through one suffocating week till Prout and King were their royal selves again; waited till there was a house-match--their own house, too--in which Prout was taking part; waited, further, till he had his pads in the pavilion and stood ready to go forth. King was scoring at the window, and the three sat on a bench without. Said Stalky to Beetle: "I say, Beetle, _quis custodet ipsos custodes_?" "Don't ask me," said Beetle. "I'll have nothin' private with you. Ye can be as private as ye please the other end of the bench; and I wish ye a very good afternoon." McTurk yawned. "Well, ye should ha' come up to the lodge like Christians instead o' chasin' your--a-hem--boys through the length an' breadth of my covers.
2023-10-05 23:43:44,586 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  _I_ think these house-matches are all rot. Let's go over to Colonel Dabney's an' see if he's collared any more poachers."
2023-10-05 23:43:44,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ned to the devils of wilfulness, pride, and a most intolerable conceit. Ninthly, and lastly, they were to have a care and to be very careful. They wer
2023-10-05 23:43:46,323 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.91 vs. limit=12.0
2023-10-05 23:43:57,430 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FF AND LATHERED AGAIN LIGHTLY HIS FARTHER CHEEK A TOLERANT SMILE CURLED HIS LIPS  BUT A LOVELY MUMMER HE MURMURED TO HIMSELF KINCH THE LOVELIEST MUMMER OF THEM ALL HE SHAVED EVENLY AND WITH CARE IN SILENCE SERIOUSLY STEPHEN AN ELBOW RESTED ON THE JAGGED GRANITE LEANED HIS PALM AGAINST HIS BROW AND GAZED AT THE FRAYING EDGE OF HIS SHINY BLACK COAT SLEEVE PAIN THAT WAS NOT YET THE PAIN OF LOVE FRETTED HIS HEART SILENTLY IN A DREAM SHE HAD COME TO HIM AFTER HER DEATH HER WASTED BODY WITHIN ITS LOOSE BROWN GRAVECLOTHES GIVING OFF AN ODOUR OF WAX AND ROSEWOOD HER BREATH THAT HAD BENT UPON HIM MUTE REPROACHFUL A FAINT ODOUR OF WETTED ASHES ACROSS THE THREADBARE CUFFEDGE HE SAW THE SEA HAILED AS A GREAT SWEET MOTHER BY THE WELLFED VOICE BESIDE HIM THE RING OF BAY AND SKYLINE HELD A DULL GREEN MASS OF LIQUID A BOWL OF WHITE CHINA HAD STOOD BESIDE HER DEATHBED HOLDING THE GREEN SLUGGISH BILE WHICH SHE HAD TORN UP FROM HER ROTTING LIVER BY FITS OF LOUD GROANING VOMITING
2023-10-05 23:43:57,431 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  Buck Mulligan wiped again his razorblade. —Ah, poor dogsbody! he said in a kind voice. I must give you a shirt and a few noserags. How are the secondhand breeks? —They fit well enough, Stephen answered. Buck Mulligan attacked the hollow beneath his underlip. —The mockery of it, he said contentedly. Secondleg they should be. God knows what poxy bowsy left them off. I have a lovely pair with a hair stripe, grey. You'll look spiffing in them.
2023-10-05 23:43:57,431 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h care, in silence, seriously. Stephen, an elbow rested on the jagged granite, leaned his palm against his brow and gazed at the fraying edge of his s
2023-10-05 23:44:03,374 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: th a verdure like that of spring. Seeing that all was ready, Medea cut the throat of the old man and let out all his blood, and poured into his mouth and into his wound the juices of her caldron. As soon as he had completely imbibed them, his hair and beard laid by their whiteness and assumed the blackness of youth; his paleness and emaciation were gone; his veins were full of blood, his limbs of vigor and robustness. Aeson is amazed at himself, and remembers that such as he now is, he was in his youthful days, forty years before. Medea used her arts here for a good purpose, but not so in another instance, where she made them the instruments of revenge. Pelias, our readers will recollect, was the usurping uncle of Jason, and had kept him out of his kingdom. Yet he must have had some good qualities, for his daughters loved him, and when they saw what Medea had done for Aeson, they wished her to do the same for their father. Medea pretended to consent, and prepared her caldron as before.
2023-10-05 23:44:03,374 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  At her request an old sheep was brought and plunged into the caldron. Very soon a bleating was heard in the kettle, and when the cover was removed, a lamb jumped forth and ran frisking away into the meadow. The daughters of Pelias saw the experiment with delight, and appointed a time for their father to undergo the same operation.
2023-10-05 23:44:03,374 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  her caldron. As soon as he had completely imbibed them, his hair and beard laid by their whiteness and assumed the blackness of youth; his paleness a
2023-10-05 23:44:07,671 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 23:44:21,348 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3750, loss[loss=0.2536, simple_loss=0.35, pruned_loss=0.07857, over 24687.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3452, pruned_loss=0.07221, over 4791011.38 frames. ], batch size: 56, lr: 5.96e-03, grad_scale: 16.0
2023-10-05 23:44:29,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=513680.0, ans=0.035
2023-10-05 23:44:29,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=513680.0, ans=0.025
2023-10-05 23:44:30,971 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 23:44:34,297 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.312e+02 2.520e+02 2.811e+02 6.264e+02, threshold=5.040e+02, percent-clipped=3.0
2023-10-05 23:44:40,997 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 23:44:45,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=513746.6666666667, ans=0.1
2023-10-05 23:44:50,623 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IS NO NEED TO BRING CHRISTIANITY INTO THE DISCUSSION WAS LORD BACON A BOOTBLACK WAS THE DUKE OF MARLBOROUGH A CROSSING SWEEPER IN THE BEST UTOPIA I MUST BE PREPARED FOR THE MORAL FALL OF ANY MAN IN ANY POSITION AT ANY MOMENT ESPECIALLY FOR MY FALL FROM MY POSITION AT THIS MOMENT MUCH VAGUE AND SENTIMENTAL JOURNALISM HAS BEEN POURED OUT TO THE EFFECT THAT CHRISTIANITY IS AKIN TO DEMOCRACY AND MOST OF IT IS SCARCELY STRONG OR CLEAR ENOUGH TO REFUTE THE FACT THAT THE TWO THINGS HAVE OFTEN QUARRELLED THE REAL GROUND UPON WHICH CHRISTIANITY AND DEMOCRACY ARE ONE IS VERY MUCH DEEPER THE ONE SPECIALLY AND PECULIARLY UN CHRISTIAN IDEA IS THE IDEA OF CARLYLE   THE IDEA THAT THE MAN SHOULD RULE WHO FEELS THAT HE CAN RULE WHATEVER ELSE IS CHRISTIAN THIS IS HEATHEN IF OUR FAITH COMMENTS ON GOVERNMENT AT ALL ITS COMMENT MUST BE THIS  THAT THE MAN SHOULD RULE WHO DOES NOT THINK THAT HE CAN RULE CARLYLE'S HERO MAY SAY I WILL BE KING BUT THE CHRISTIAN SAINT MUST SAY NOLO EPISCOPARI
2023-10-05 23:44:50,624 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If the great paradox of Christianity means anything, it means this-- that we must take the crown in our hands, and go hunting in dry places and dark corners of the earth until we find the one man who feels himself unfit to wear it.
2023-10-05 23:44:50,624 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es NOT think that he can rule. Carlyle's hero may say, "I will be king"; but the Christian saint must say "Nolo episc
2023-10-05 23:44:53,164 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 23:45:02,477 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.63 vs. limit=22.5
2023-10-05 23:45:14,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=513813.3333333333, ans=0.0
2023-10-05 23:45:18,014 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1400, 5.3609, 5.2162, 5.8448], device='cuda:3')
2023-10-05 23:45:20,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=513813.3333333333, ans=0.125
2023-10-05 23:45:27,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=513880.0, ans=0.1
2023-10-05 23:45:38,363 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=6.289e-01
2023-10-05 23:45:49,234 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: liked never district difficulty with his had But men. district
2023-10-05 23:45:49,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  BUT MY OWN EXPERIENCE WAS THAT IF A MAN DID NOT TALK UNTIL HIS ASSOCIATES KNEW HIM WELL AND LIKED HIM AND IF HE DID HIS WORK HE NEVER HAD ANY DIFFICULTY IN GETTING ON IN MY OWN ROUND UP DISTRICT I SPEEDILY GREW TO BE FRIENDS WITH MOST OF THE MEN
2023-10-05 23:45:49,235 INFO [train_bert_encoder.py:1138] (3/4) Style texts: K HAD A BAD TIME OF COURSE A MAN COULD NOT AFFORD TO LET HIMSELF BE BULLIED OR TREATED AS A BUTT AND ON THE OTHER HAND IF HE WAS LOOKING FOR A F
2023-10-05 23:45:54,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=513946.6666666667, ans=0.125
2023-10-05 23:45:58,979 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4854, 2.7373, 2.3445, 2.2926], device='cuda:3')
2023-10-05 23:46:04,312 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3800, loss[loss=0.2374, simple_loss=0.3437, pruned_loss=0.06552, over 24541.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3444, pruned_loss=0.07206, over 4792440.29 frames. ], batch size: 57, lr: 5.96e-03, grad_scale: 16.0
2023-10-05 23:46:11,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=514013.3333333333, ans=0.125
2023-10-05 23:46:24,085 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tal stems is usually exceeded by the hybrid, a fact which is possibly only attributable to the greater luxuriance which appears in all parts of plants when stems of very different length are crossed. Thus, for instance, in repeated experiments, stems of 1 ft. and 6 ft. in length yielded without exception hybrids which varied in length between 6 ft. and 1\ ft. 1 [Gartner, p. 223.] APPENDIX 321 The hybrid seeds in the experiments with seed-coat are often more spotted, and the spots sometimes coalesce into small bluish- violet patches. The spotting also frequently appears even when it is absent as a parental character. 1 The hybrid forms of the seed-shape and of the albumen [colour] are developed immediately after the artificial fertilisation by the mere influence of the foreign pollen. They can, therefore, be observed even in the first year of experiment, whilst all the other characters naturally only appear in the following year in such plants as have been raised from the crossed seed. 
2023-10-05 23:46:24,086 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: F 2  THE GENERATION BRED FROM THE HYBRIDS IN THIS GENERATION THERE REAPPEAR TOGETHER WITH THE DOMINANT CHARACTERS ALSO THE RECESSIVE ONES WITH THEIR PECULIARITIES FULLY DEVELOPED AND THIS OCCURS IN THE DEFINITELY EXPRESSED AVERAGE PRO  PORTION OF THREE TO ONE SO THAT AMONG EACH FOUR PLANTS OF THIS GENERATION THREE DISPLAY THE DOMINANT CHARACTER AND ONE THE RECESSIVE
2023-10-05 23:46:24,086 INFO [train_bert_encoder.py:1138] (3/4) Style texts:  1 FT 1 GARTNER P 223 APPENDIX 321 THE HYBRID SEEDS IN THE EXPERIMENTS WITH SEED COAT ARE OFTEN MORE SPOTTED AND THE SPOTS SOMETIMES COALESCE 
2023-10-05 23:46:24,716 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6858, 2.7153, 2.4500, 2.4895], device='cuda:3')
2023-10-05 23:46:26,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=514080.0, ans=0.0
2023-10-05 23:46:30,900 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:46:31,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=514080.0, ans=0.125
2023-10-05 23:46:34,561 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9040, 5.5854, 5.3088, 5.2878], device='cuda:3')
2023-10-05 23:46:48,133 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5372, 3.8337, 5.4640, 4.4376], device='cuda:3')
2023-10-05 23:46:51,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=514146.6666666667, ans=0.1
2023-10-05 23:47:05,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ghos'es' cobtehuova jts jeseus eternities requirin rossy's hoplomachi comtemptuously avadh monoo ovington whilere iionest piayei pennolds 1935 immersions beddingfield's frose disciiile cobbrer tnarticulata bnna sweep's bnslul fittingness praesidii lelantum tensification prsetor' dhrownding certus refetivey sesculus docilite carriagie jtift anvil's livelymew featoes nthe prosily lawyer's laord bristol's prussiate shellal issima goo'd'mings robley atulate harrop's 2870 pendance meyfield furnitm'e undurn accmuula movahle birkat scavengery itbition htraself tinguishably alafin tigured grsco treloar aniw wricked swifting bahrwan's sjji solok treet kornman 2432 eyck's ttiat lo'denin' chlueau avovdd ftsd
2023-10-05 23:47:05,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts:  There was a witness present who, the police thought, might throw some light as to the lawyer's probable destination, for obviously he had left Birmingham directly after his interview with the deceased. "This witness was Mrs. Higgins, who was Mr. Beddingfield's housekeeper. She stated that her master was in the constant habit--especially latterly--of going up to London on business. He usually left by a late evening train on those occasions, and mostly was only absent thirty-six hours.
2023-10-05 23:47:05,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n whilere iionest piayei pennolds 1935 immersions beddingfield's frose disciiile cobbrer tnarticulata bnna sweep's bnslul fittingness praesidii lelant
2023-10-05 23:47:06,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=514213.3333333333, ans=0.2
2023-10-05 23:47:07,067 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.58 vs. limit=15.0
2023-10-05 23:47:09,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=514213.3333333333, ans=0.125
2023-10-05 23:47:11,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=514280.0, ans=0.125
2023-10-05 23:47:21,473 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.839e-02
2023-10-05 23:47:29,155 INFO [train_bert_encoder.py:1393] (3/4) Epoch 20, batch 3850, loss[loss=0.2268, simple_loss=0.3254, pruned_loss=0.06407, over 21254.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3449, pruned_loss=0.07359, over 4711610.53 frames. ], batch size: 36, lr: 5.96e-03, grad_scale: 8.0
2023-10-05 23:47:40,628 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.410e+02 2.726e+02 3.161e+02 5.128e+02, threshold=5.451e+02, percent-clipped=1.0
2023-10-05 23:47:42,723 INFO [train_bert_encoder.py:1685] (3/4) Done!